I am using an API to get real data times of trains and am trying to get the closest train time to a user entered time and then display that train time, and the next 4 granted the trains are running. I am reading in the information and the code goes through what its supposed to do but when I look at the array its a bunch of [] brackets in 7 cells instead of the calculated numbers. Any suggestions? Code is below with the API
TEST VALUES:
requestStationSelected = 'University%20City' and requestEndStation = 'Roslyn'
%this is the API link for the live data from Septa this will get 30
%results and see which time is closer to the user entered time
requestInfoSeptaLive = ['http://www3.septa.org/hackathon/NextToArrive/' requestStationSelected '/' requestEndStation '/30'];
%Again tries to get the information and if there is a failure it will give
%a probable cause and terminate the program
try
getInfoSeptaLive = urlread(requestInfoSeptaLive);
catch
if getInfoSeptaLive ~= '[]'
disp...
('Either the arrival/depart stations dont quite match up or theres a server error. Try again.');
return;
else
disp('Unable to fetch the information from Septa, please try again')
return;
end
end
%parses the information returned from the Live API
dataReturnedFromLiveAPI = parse_json(getInfoSeptaLive);
dataReturnedFromLiveAPI = dataReturnedFromLiveAPI{1};
%gets the size of the API in case there are no trains running
sizeOfDataNoTrains = size(dataReturnedFromLiveAPI, 1);
sizeOfData = size(dataReturnedFromLiveAPI, 2);
counter = 0;
for i = 1:sizeOfData
scanForClosestTime = dataReturnedFromLiveAPI{1,i}.orig_departure_time;
trainTimeGivenH = sscanf(scanForClosestTime, '%i');
findColonTrain = strfind(scanForClosestTime, ':');
trainTimeGivenMStr = scanForClosestTime(findColonTrain+1:4);
trainTimeGivenM = int32(str2num(trainTimeGivenMStr));
trainDepartTimeM = (trainTimeGivenH(1,1) * 60) + (trainTimeGivenM);
differenceBetweenTimes = trainDepartTimeM - userEnteredMins;
if trainDepartTimeM < userEnteredMins
differenceBetweenTimes = userEnteredMins - trainDepartTimeM;
end
stopAtEndOfData = sizeOfData;
goodTimeFrame = 60;
closestTime = cell(1, stopAtEndOfData);
storeTheDifference = cell(1, stopAtEndOfData);
if(differenceBetweenTimes < 60)
if (counter < 5)
closestTime{i} = scanForClosestTime;
storeTheDifference{i} = differenceBetweenTimes;
counter = counter + 1;
end
end
end
You assign your cell arrays inside the for loop:
for i = 1:sizeOfData
...
closestTime = cell(1, stopAtEndOfData);
storeTheDifference = cell(1, stopAtEndOfData);
...
end
This means that you turn both of them into an array of {[],[],[],[],[]...} on every iteration of the loop - so unless the last iteration has a valid "closest Time" in it, your cell array will be all empty arrays - and if it does, all but the last element will still be [].
To fix this, move the two lines to before the start of the for loop.
The second problem seems to be the indexing of the arrays where you store the results. If you only want five results, I am assuming you want to store them in elements 1 - 5 of your array, and not in "just any" locations. I would change the code to
if (counter < 5)
counter = counter + 1;
closestTime{counter} = scanForClosestTime;
storeTheDifference{counter} = differenceBetweenTimes;
end
But maybe I misinterpreted how you want to handle that?
Unrelated to your question, you might want to take a look at the line
trainTimeGivenMStr = scanForClosestTime(findColonTrain+1:4);
It is quite possible that this is not what you intended to do - looking at an example of the response, I found the string "orig_departure_time":"11:57PM". I expect that findColonTrain == 3, so that the above line becomes
trainTimeGivenMStr = scanForClosestTime(4:4);
just a single character. Perhaps you meant
trainTimeGivenMStr = scanForClosestTime(findColonTrain+(1:4));
which would turn into
trainTimeGivenMStr = scanForClosestTime(4:7);
so that
trainTimeGivenMStr = '57PM';
I hope these three things help you get it all working!
EDIT: had a chance to run your code this morning - discovered a number of other problems. I include below an annotated "working" code: the biggest problem was most likely that you were not handling AM/PM in your code. Note that I used a different json parser - this changed a couple of lines very slightly. I'm sure you can put it back together to work the way you want. This returned valid data in all cells.
dataReturnedFromLiveAPI = loadjson(getInfoSeptaLive);
% next line not needed - loadjson returns struct array, not cell array
%dataReturnedFromLiveAPI = dataReturnedFromLiveAPI{1};
%gets the size of the API in case there are no trains running
sizeOfDataNoTrains = size(dataReturnedFromLiveAPI, 1);
sizeOfData = size(dataReturnedFromLiveAPI, 2);
counter = 0;
stopAtEndOfData = sizeOfData;
closestTime = cell(1, stopAtEndOfData);
storeTheDifference = cell(1, stopAtEndOfData);
userEnteredMins = 12*60+30; % looking for a train around 12:30 pm
for ii = 1:sizeOfData
scanForClosestTime = dataReturnedFromLiveAPI(ii).orig_departure_time;
trainTimeGivenH = sscanf(scanForClosestTime, '%i');
% since we'll be considering AM/PM, have to set 12 = 0:
if (trainTimeGivenH == 12), trainTimeGivenH = 0; end
findColonTrain = strfind(scanForClosestTime, ':');
% change next line to get minutes plus AM/PM:
trainTimeGivenMStr = scanForClosestTime(findColonTrain+(1:4));
% look at just minutes:
trainTimeGivenM = int32(str2num(trainTimeGivenMStr(1:2)));
% adjust for AM/PM:
if(trainTimeGivenMStr(3:4)=='PM'), trainTimeGivenH = trainTimeGivenH+12; end;
% compute time in minutes:
trainDepartTimeM = (trainTimeGivenH * 60) + (trainTimeGivenM);
differenceBetweenTimes = trainDepartTimeM - userEnteredMins;
if trainDepartTimeM < userEnteredMins
differenceBetweenTimes = userEnteredMins - trainDepartTimeM;
end
% added a couple of lines to see what is happening:
fprintf(1, 'train %d: depart %s - in minutes this is %d vs user entered %d\n', ...
ii, scanForClosestTime, trainDepartTimeM, userEnteredMins);
goodTimeFrame = 60;
if(differenceBetweenTimes < 600)
if (counter < 10)
counter = counter + 1;
closestTime{counter} = scanForClosestTime;
storeTheDifference{counter} = differenceBetweenTimes;
end
end
end
It's supposed to save each line in the text file into a list and split it based on commas and categorize them into multiple different lists. The error occurs at the while loop stating that the index is out of range.
lineCounter = 0
j = 0
file = open("savefile.txt","r")
with open('savefile.txt', 'r') as f:
string = [line.strip() for line in f]
for line in file:
lineCounter += 1
while(j<lineCounter):
tempList = string[j].split(',')
firstName[j] = tempList[0]
lastName[j] = tempList[1]
postition[j] = tempList[2]
department[j] = tempList[3]
seniority[j] = tempList[4]
vacationWeeks[j] = tempList[5]
sickDays[j] = tempList[6]
iD[j] = tempList[7]
status[j] = tempList[8]
j += 1
print firstName
file.close() # close the text file
NVM the problem was that the list needed to be appended rather than replaced.
I am working in MATLAB.
I have a function that loops through all the files in a directory, runs them and concatenates their dataset outputs into a single dataset.
Is there a way that I can alter my function so that it outputs all the individual datasets as well as the unified one?
Below, the array named "FileInfo" has 3 columns. The first has the file name, and the second and third columns are the inputs
function [AllFunOutputs] = RunAllFuns(FileInfo)
fileDir = dir('C:\MATLAB\Funs'); % get all file names in directory 'Funs'
files = {fileDir.name};
funNames = strrep(files, '.m', ''); % strip the '.m' suffix from all files
funNames(:,1:2) = [];
funNames = transpose(funNames);
k = 1; % below, match the function name with its argument
for i=1:length(FileInfo)
if strcmp(FileInfo(i,1),funNames(k,1))
funNames(k,3) = FileInfo(i,2);
k = k+1;
end
end
% create function handles
fh_array = cellfun(#str2func,{funNames{:,1}},'UniformOutput', false);
X = []; % below, concatenate all output datasets into a single dataset
for i=1:size((funNames),1)
X=[fh_array{i}(funNames(i,2),(funNames(i,3)))];
X = X+1;
end
so..... why doesn't this work to give me the output dataset of all the functions?
nFcns = numel(fh_array); % number of functions to evaluate
for i=1:size(nFcns)
[allresults] = feval(#(i)funNames(i,2),funNames(i,3));
end
Thank you so much for your help and time!
EDIT 3
Hi! I had problems with the matrix dimensions but I've solved it. Now my problem is that I want to do the same operation on a large series of files on the same folder and I want write the output values on a separate line on text.txt. With the first one it works but it doesn't 'write' to the 'text', the rest. Is there something wrong?
myPath = 'C:\EX\';
a= dir (fullfile(myPath,'*.DIM'));
fileNames = { a.name };
% Rename files
for k = 1:length(fileNames)
newFileName = [fileNames{k}(1:2) fileNames{k}(4:6) '.txt'];
movefile([myPath fileNames{k}], [myPath newFileName]);
end
filePattern=fullfile( myPath,'*.txt');
txtFiles= dir(filePattern);
for k = 1:length(txtFiles)
baseFileName=txtFiles(k).name;
fullFileName= fullfile(myPath,baseFileName);
fid=fopen(fullFileName, 'r');
for i = 1:18
m{i} = fgetl(fid);
end
result2 = m{18};
result2b= result2([12:19]);
fid=fopen(fullFileName, 'r');
for i = 1:30
m{i} = fgetl(fid);
end
result3 = m{30};
result3b= result3([12:19]);
fid=fopen(fullFileName, 'r');
for i = 1:31
m{i} = fgetl(fid);
end
result4 = m{31};
result4b= result4([12:20]);
fid=fopen(fullFileName, 'r');
for i = 1:19
m{i} = fgetl(fid);
end
result5 = m{19};
result5b= result5([12:20]);
text= {baseFileName, result2b, result3b, result4b, result5b};
final= [Fields'; text];
end
Really thanks in advance!
Index exceeds dimensions is exactly what it means.
Try to put a breakpoint at the line where it occurs and check the dimension of result2. Assuming it is a vector, you will find that its length is less than 19.
My customer is sending TDM/TDX files captured in National Instruments Diadem, which I haven't got. I'm looking for a way to convert the files into .CSV, XLS or .MAT files for analysis in Matlab (without using Diadem or Diadem DLLs!)
The format consists of a well structured XML file (.TDM) and a binary (.TDX), with the .TDM defining how fields are packed as bits in the binary TDX. I'd like to read the files (for use in Matlab and other environments). Does anyone have a general purpose tool or conversion script in for instance Python or Perl (not using the NI DLL's) or directly in Matlab?
I've looked into buying the tool, but didn't like it for anything other than one-time conversion to a compatible file format.
Thanks!
I know this is a little late, but I have a simple library to read TDM/TDX files in Python. It works by parsing the TDM file to figure out the data type, then using NumPy.memmap to open the TDX file. It can then be used like a standard NumPy array. The code is pretty simple, so you could probably implement something similar in Matlab.
Here's the link: https://bitbucket.org/joshayers/tdm_loader
Hope that helps.
Maybe a little too late, but I think there is a simple way to get the data from TDM files: NI provides plug-ins for reading TDM files into Excel and OpenOffice Calc. Having the data in one of these programs you could use the CSV export. Search google for "tdm excel" or "tdm openoffice".
Hope this helps...
Gemue
The following script can convert all variables into 'variable' struct.
CurrDirectory = '...//'; % Path to current directory
fileNametdx = '.../utility/'; % Path to TDX file
%%
% Data type conversion
Dtype.eInt8Usi='int8';
Dtype.eInt16Usi='int16';
Dtype.eInt32Usi='int32';
Dtype.eInt64Usi='int64';
Dtype.eUInt8Usi='uint8';
Dtype.eUInt16Usi='uint16';
Dtype.eUInt32Usi='uint32';
Dtype.eUInt64Usi='uint64';
Dtype.eFloat32Usi='single';
Dtype.eFloat64Usi='double';
%% Read .tdx file Name
wb=waitbar(0,'Reading *.tdx Files');
fileNameTDM = strrep(fileNametdx,'.tdx','.TDM');
%% Read .TDM
tdm=xml2struct(fileNameTDM);
for i=1:numel(tdm.usi_colon_tdm.usi_colon_data.tdm_channel)
waitbar((1/numel(tdm.usi_colon_tdm.usi_colon_data.tdm_channel))*i,wb,['File ' fileNametdx ' conversion started']);
s1=strsplit(string(tdm.usi_colon_tdm.usi_colon_data.tdm_channel{1, i}.local_columns.Text),'"');
usi1=s1(2);
% if condition match untill we get usi2
for j=1:numel(tdm.usi_colon_tdm.usi_colon_data.localcolumn)
usi2=string(tdm.usi_colon_tdm.usi_colon_data.localcolumn{1, j}.Attributes.id);
if usi1==usi2
%take new usi
s2=strsplit(string(tdm.usi_colon_tdm.usi_colon_data.localcolumn{1, j}.values.Text),'"');
new_usi1=s2(2);
w1=strsplit(string(tdm.usi_colon_tdm.usi_colon_data.tdm_channel{1, i}.datatype.Text),'_');
str_1=char(strcat('tdm.usi_colon_tdm.usi_colon_data.',lower(w1(2)),'_sequence'));
str_2=char(strcat('tdm.usi_colon_tdm.usi_colon_data.',lower(w1(2)),'_sequence{1, k}.Attributes.id'));
str_3=char(strcat('tdm.usi_colon_tdm.usi_colon_data.',lower(w1(2)),'_sequence{1, k}.values.Attributes.external'));
str_4=char(strcat('tdm.usi_colon_tdm.usi_colon_data.',lower(w1(2)),'_sequence{1, k}.values'));
for k=1:numel(eval(str_1))
new_usi2=string(eval(str_2));
if new_usi1==new_usi2
if isfield(eval(str_4), 'Attributes')
inc_value1=string(eval(str_3));
for m=1:numel(tdm.usi_colon_tdm.usi_colon_include.file.block)
inc_value2=string(tdm.usi_colon_tdm.usi_colon_include.file.block{1, m}.Attributes.id);
if inc_value1==inc_value2
% offset=round(str2num(tdm.usi_colon_tdm.usi_colon_include.file.block{1, m}.Attributes.byteOffset)/8);
length = round(str2num(tdm.usi_colon_tdm.usi_colon_include.file.block{1, m}.Attributes.length));
offset1=round(str2num(tdm.usi_colon_tdm.usi_colon_include.file.block{1, m}.Attributes.byteOffset));
value_type = tdm.usi_colon_tdm.usi_colon_include.file.block{1, m}.Attributes.valueType;
m = memmapfile(fullfile(CurrDirectory,fileNametdx),'Offset',offset1,'Format',{Dtype.(value_type) [length 1] 'dat'},'Writable',true,'Repeat',1);
dat=m.Data.dat ;
end
end
else
str_5=char(strcat('tdm.usi_colon_tdm.usi_colon_data.',lower(w1(2)),'_sequence{1, k}.values.',char(fieldnames(tdm.usi_colon_tdm.usi_colon_data.string_sequence{1, k}.values))));
dat=eval(str_5)';
end
name_variable = string(tdm.usi_colon_tdm.usi_colon_data.tdm_channel{1, i}.name.Text);
varname = genvarname(char(name_variable));
variable.(varname) = dat;
end
end
end
end
end
waitbar(1,wb,[fileNametdx ' conversion completed']);
pause(1)
close(wb)
delete(fullfile(CurrDirectory,fileNametdx),fullfile(CurrDirectory,fileNameTDM));
%Output Variable is Struct
clearvars -except variable
This script requires following XML parser
function [ s ] = xml2struct( file )
%Convert xml file into a MATLAB structure
% [ s ] = xml2struct( file )
%
% A file containing:
% <XMLname attrib1="Some value">
% <Element>Some text</Element>
% <DifferentElement attrib2="2">Some more text</Element>
% <DifferentElement attrib3="2" attrib4="1">Even more text</DifferentElement>
% </XMLname>
%
% Will produce:
% s.XMLname.Attributes.attrib1 = "Some value";
% s.XMLname.Element.Text = "Some text";
% s.XMLname.DifferentElement{1}.Attributes.attrib2 = "2";
% s.XMLname.DifferentElement{1}.Text = "Some more text";
% s.XMLname.DifferentElement{2}.Attributes.attrib3 = "2";
% s.XMLname.DifferentElement{2}.Attributes.attrib4 = "1";
% s.XMLname.DifferentElement{2}.Text = "Even more text";
%
% Please note that the following characters are substituted
% '-' by '_dash_', ':' by '_colon_' and '.' by '_dot_'
%
% Written by W. Falkena, ASTI, TUDelft, 21-08-2010
% Attribute parsing speed increased by 40% by A. Wanner, 14-6-2011
% Added CDATA support by I. Smirnov, 20-3-2012
%
% Modified by X. Mo, University of Wisconsin, 12-5-2012
if (nargin < 1)
clc;
help xml2struct
return
end
if isa(file, 'org.apache.xerces.dom.DeferredDocumentImpl') || isa(file, 'org.apache.xerces.dom.DeferredElementImpl')
% input is a java xml object
xDoc = file;
else
%check for existance
if (exist(file,'file') == 0)
%Perhaps the xml extension was omitted from the file name. Add the
%extension and try again.
if (isempty(strfind(file,'.xml')))
file = [file '.xml'];
end
if (exist(file,'file') == 0)
error(['The file ' file ' could not be found']);
end
end
%read the xml file
xDoc = xmlread(file);
end
%parse xDoc into a MATLAB structure
s = parseChildNodes(xDoc);
end
% ----- Subfunction parseChildNodes -----
function [children,ptext,textflag] = parseChildNodes(theNode)
% Recurse over node children.
children = struct;
ptext = struct; textflag = 'Text';
if hasChildNodes(theNode)
childNodes = getChildNodes(theNode);
numChildNodes = getLength(childNodes);
for count = 1:numChildNodes
theChild = item(childNodes,count-1);
[text,name,attr,childs,textflag] = getNodeData(theChild);
if (~strcmp(name,'#text') && ~strcmp(name,'#comment') && ~strcmp(name,'#cdata_dash_section'))
%XML allows the same elements to be defined multiple times,
%put each in a different cell
if (isfield(children,name))
if (~iscell(children.(name)))
%put existsing element into cell format
children.(name) = {children.(name)};
end
index = length(children.(name))+1;
%add new element
children.(name){index} = childs;
if(~isempty(fieldnames(text)))
children.(name){index} = text;
end
if(~isempty(attr))
children.(name){index}.('Attributes') = attr;
end
else
%add previously unknown (new) element to the structure
children.(name) = childs;
if(~isempty(text) && ~isempty(fieldnames(text)))
children.(name) = text;
end
if(~isempty(attr))
children.(name).('Attributes') = attr;
end
end
else
ptextflag = 'Text';
if (strcmp(name, '#cdata_dash_section'))
ptextflag = 'CDATA';
elseif (strcmp(name, '#comment'))
ptextflag = 'Comment';
end
%this is the text in an element (i.e., the parentNode)
if (~isempty(regexprep(text.(textflag),'[\s]*','')))
if (~isfield(ptext,ptextflag) || isempty(ptext.(ptextflag)))
ptext.(ptextflag) = text.(textflag);
else
%what to do when element data is as follows:
%<element>Text <!--Comment--> More text</element>
%put the text in different cells:
% if (~iscell(ptext)) ptext = {ptext}; end
% ptext{length(ptext)+1} = text;
%just append the text
ptext.(ptextflag) = [ptext.(ptextflag) text.(textflag)];
end
end
end
end
end
end
% ----- Subfunction getNodeData -----
function [text,name,attr,childs,textflag] = getNodeData(theNode)
% Create structure of node info.
%make sure name is allowed as structure name
name = toCharArray(getNodeName(theNode))';
name = strrep(name, '-', '_dash_');
name = strrep(name, ':', '_colon_');
name = strrep(name, '.', '_dot_');
attr = parseAttributes(theNode);
if (isempty(fieldnames(attr)))
attr = [];
end
%parse child nodes
[childs,text,textflag] = parseChildNodes(theNode);
if (isempty(fieldnames(childs)) && isempty(fieldnames(text)))
%get the data of any childless nodes
% faster than if any(strcmp(methods(theNode), 'getData'))
% no need to try-catch (?)
% faster than text = char(getData(theNode));
text.(textflag) = toCharArray(getTextContent(theNode))';
end
end
% ----- Subfunction parseAttributes -----
function attributes = parseAttributes(theNode)
% Create attributes structure.
attributes = struct;
if hasAttributes(theNode)
theAttributes = getAttributes(theNode);
numAttributes = getLength(theAttributes);
for count = 1:numAttributes
%attrib = item(theAttributes,count-1);
%attr_name = regexprep(char(getName(attrib)),'[-:.]','_');
%attributes.(attr_name) = char(getValue(attrib));
%Suggestion of Adrian Wanner
str = toCharArray(toString(item(theAttributes,count-1)))';
k = strfind(str,'=');
attr_name = str(1:(k(1)-1));
attr_name = strrep(attr_name, '-', '_dash_');
attr_name = strrep(attr_name, ':', '_colon_');
attr_name = strrep(attr_name, '.', '_dot_');
attributes.(attr_name) = str((k(1)+2):(end-1));
end
end
end