I have a lot of data in several hundred .mat-files where I want to extract specific data from. All the names of my .mat-files have specific numbers to identify the content like Number1_Number2_Number3_Number4.mat:
01_33_06_121.mat
01_24_12_124.mat
02_45_15_118.mat
02_33_11_190.mat
01_33_34_142.mat
Now I want to extract for example all the data from files with Number1=01 or Number1=02 and Number2=33.
Before I start to write a program from scratch, I would like to know, if there is a simple way to do this with Matlab. Does anybody know how I can solve this problem in a fast way?
Thanks a lot!
There are multiple ways you can do this; on top of my head following can work:
Obtain all the file names into an array
allFiles = dir( 'folder' );
allNames = { allFiles.name };
Loop through your file names and compare against the condition using the regex
for i=1:size(allNames)
if regexp(allNames, pattern, 'match')
disp(allNames)
end
end
Related
I am currently writing an (simple) analytisis code to sum time connected powerreadings. With the data being assumingly raw (e.g. disturbances from the measuring device have not been calculated out) I have to account for disturbances by calculation the mean of the first one thousand samples. The calculation of the mean itself is not a problem. I only am unsure of how to generate the appropriate DataSet.
For now it looks about like this:
DataSet<Tupel2<long,double>>Gyrotron_1=ECRH.includeFields('11000000000'); // obviously the line to declare the first gyrotron, continues for the next ten lines, assuming separattion of not occupied space
DataSet<Tupel2<long,double>>Gyrotron_2=ECRH.includeFields('10100000000');
DataSet<Tupel2<long,double>>Gyrotron_3=ECRH.includeFields('10010000000');
DataSet<Tupel2<long,double>>Gyrotron_4=ECRH.includeFields('10001000000');
DataSet<Tupel2<long,double>>Gyrotron_5=ECRH.includeFields('10000100000');
DataSet<Tupel2<long,double>>Gyrotron_6=ECRH.includeFields('10000010000');
DataSet<Tupel2<long,double>>Gyrotron_7=ECRH.includeFields('10000001000');
DataSet<Tupel2<long,double>>Gyrotron_8=ECRH.includeFields('10000000100');
DataSet<Tupel2<long,double>>Gyrotron_9=ECRH.includeFields('10000000010');
DataSet<Tupel2<long,double>>Gyrotron_10=ECRH.includeFields('10000000001');
for (int=1,i<=10;i++) {
DataSet<double> offset=Gyroton_'+i+'.groupBy(1).first(1000).sum()/1000;
}
It's the part in the for-loop I'm unsure of. Does anybody know if it is possible to append values to DataSets and if so how?
In case of doubt, I could always put the values into an array but I do not know if that is the wise thing to do.
This code will not work for many reasons. I'd recommend looking into the fundamentals of Java and the basic data structures and also in Flink.
It's really hard to understand what you actually try to achieve but this is the closest that I came up with
String[] codes = { "11000000000", ..., "10000000001" };
DataSet<Tuple2<Long, Double>> result = env.fromElements();
for (final String code : codes) {
DataSet<Tuple2<Long, Double>> codeResult = ECRH.includeFields(code)
.groupBy(1)
.first(1000)
.sum(0)
.map(sum -> new Tuple2<>(sum.f0, sum.f1 / 1000d));
result = codeResult.union(result);
}
result.print();
But please take the time and understand the basics before delving deeper. I also recommend to use an IDE like IntelliJ that would point to at least 6 issues in your code.
I'm using this json content (I'm open to suggestions on better formatting here):
{"forwardingZones": [{"name": "corp.mycompany.com","dnsServers": ["192.168.0.1","192.168.0.2"]}]}
Note: we may add more items to this list as we scale out, both more IPs and more names, hence the "join(',')" in the end of the code below.
And I'm trying to loop through it to get this result:
corp.mycompany.com=192.168.0.1;192.168.0.2
Using this code:
forward_zones = node['DNS']['forward_zones'].each do |forwarded_zone|
forwarded_zone_name = forwarded_zone['name']
forwarded_zone_dns_servers = forwarded_zone['dns_servers'].join(';')
"#{forwarded_zone_name}=#{forwarded_zone_dns_servers}"
end.join(',')
This is the result that I get:
{"dnsServers"=>["192.168.0.1", "192.168.0.2"], "name"=>"corp.mycompany.com"}
What am i doing wrong...?
x.each returns x. You want x.map.
So I am trying to read a rather large XML file into a String. Currently joining a list of .readLines() like this:
def is = zipFile.getInputStream(entry)
def content = is.getText('UTF-8')
def xmlBodyList = content.readLines()
return xmlBodyList[1..xmlBodyList.size].join("")
However I am getting this output in console:
java.lang.IndexOutOfBoundsException: toIndex = 21859
I don't need any explanation on IndexOutOfBoundsExceptions, but I am having a hard time figuring out how to program around this issue.
How can I implement this differently, so it allows for a large enough file size?
About Good way to avoid java.lang.IndexOutOfBoundsException
error is here:
return xmlBodyList[1..xmlBodyList.size].join("")
A good way to check variables before accessing and you can use relative range accessor:
assert xmlBodyList.size>1 //check value
return xmlBodyList[1..-1].join("") //use relative indexes -1 = the last one
About large files processing
If you need to iterate through all the lines and execute some operation here is an example:
def stream = zipFile.getInputStream(entry)
stream.eachLine("UTF-8"){line, index->
if(index>1){ //skip first line
//do something here with each line from file
println "$line $index"
}
}
there are a lot of additional groovy methods over java.io.InputStream that could help you to process large file without loading it into memory:
http://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/InputStream.html
Sorry I'am new on Ruby (just a Java programmer), I have two string arrays:
Array with file paths.
Array with patterns (can be a path or a file)
I need to check each patter over each "file path". I do with this way:
#flag = false
["aa/bb/cc/file1.txt","aa/bb/cc/file2.txt","aa/bb/dd/file3.txt"].each do |source|
["bb/cc/","zz/xx/ee"].each do |to_check|
if source.include?(to_check)
#flag = true
end
end
end
puts #flag
This code is ok, prints "true" because "bb/cc" is in source.
I have seen several posts but can not find a better way. I'm sure there should be functions that allow me to do this in fewer lines.
Is this is possible?
As mentioned by #dodecaphonic use Enumerable#any?. Something like this:
paths.any? { |s| patterns.any? { |p| s[p] } }
where paths and patterns are arrays as defined by the OP.
While that will work, that's going to have geometric scaling problems, that is it has to do N*M tests for a list of N files versus M patterns. You can optimize this a little:
files = ["aa/bb/cc/file1.txt","aa/bb/cc/file2.txt","aa/bb/dd/file3.txt"]
# Create a pattern that matches all desired substrings
pattern = Regexp.union(["bb/cc/","zz/xx/ee"])
# Test until one of them hits, returns true if any matches, false otherwise
files.any? do |file|
file.match(pattern)
end
You can wrap that up in a method if you want. Keep in mind that if the pattern list doesn't change you might want to create that once and keep it around instead of constantly re-generating it.
Trying to put together a file diff route... could someone help? here is what I have ->
CsvDataFormat csv = new CsvDataFormat();
csv.setDelimiter(",");
from("file:inputdir?delete=true&sortBy=ignoreCase:file:name")
.unmarshal(csv)
.pollEnrich("file:backup?fileName=test.csv&sendEmptyMessageWhenIdle=true")
.unmarshal(csv)
// Need to aggregate here!!!!
.log("test");
A csv file gets dropped in the /input directory and then a backup file is consumed from the /backup directory. I would like to compare these two files and output the difference.
This is not a specific Camel problem. In order to solve this problem you may implement a diff functionality on your own, or you may use an existing library such as java-diff-utils.
Pseudocode:
// read file 1 into a list "list1"
// read file 2 into a list "list2"
// use java-diff-utils to calculate the difference
Patch patch = DiffUtils.diff(list1, list2);