Suppose I have a data set arranged in the following way
19 10 1 1
12 15 1 1
13 12 4 5
10 5 2 3
...
and so on, at a particular iteration in a for loop I have to read only the 1st and the 4th row and in the next iteration I have to access some other set of rows,for example
1st iteration:
1st row: 19 10 1 1
4th row: 10 5 2 3
i will access my data using the fscanf() function. But how will i ensure that I choose only the 1st and 4th rows or any two rows for that matter at a given iteration?
(I have not considered reading it into a 2D array since the size of data set is 10^8 )
Thank you.
As you read through your data (say, stored in a standard file), get byte offsets for rows by looking for row delimiters (a newline character). You can then read out rows based on the start and end byte offset with C pointer arithmetic on a FILE * and fseek(). Storing a few byte offsets (an eight byte long or equivalent, often) is cheap.
Related
Consider an example:
5
1 0 5
1 1 7
1 0 3
2 1 0
2 1 1
Here, in the first line, 5 denotes the size of the array.
I'm entering five sequences one by one.
I want the first sequence ie. 1 0 5 to be stored in arr[0].
Note: 1, 0 and 5 are seperated by spaces.
However, arr[0] should contain 105 without any space.
I want to accept the next sequence into arr[1] only after pressing 'Enter'.
So that arr[1] should contain 117, arr[2] should contain 103 and so on up to arr[4].
Is there any operator that I can use for this?
There are no operators that do I/O in C at all, so no.
I also don't think there's any standard function with those semantics, they tend to view all whitespace as equal.
You should write your own, probably using fgets() to read in whole lines and then extracting the digits to convert to integers.
i am doing very simple Matrix indexing examples . where code is as give below
>> A=[ 1 2 3 4 ; 5 6 7 8 ; 9 10 11 12 ]
A =
1 2 3 4
5 6 7 8
9 10 11 12
>> A(end, end-2)
ans =
10
>> A(2:end, end:-2:1)
ans =
8 6
12 10
here i am bit confused . when i use A(end, end-2) it takes difference of two till first column and when there is just one column left there is no further processing , but when i use A(2:end, end:-2:1) it takes 6 10 but how does it print 8 12 while there is just one column left and we have to take difference of two from right to left , Pleas someone explain this simple point
The selection A(end, end-2) reads: take elements in last row of A that appear in column 4(end)-2=2.
The selection A(2:end, end:-2:1) similarly reads: take elements in rows 2 to 4(end) and starting from last column going backwards in jumps of two, i.e. 4 then 2.
To check the indexing, simply substitute the end with size(A,1) or size(A,2) if respectively found in the row and col position.
First the general stuff: end is just a placeholder for an index, namely the last position in a given array dimension. For instance, for an arbitrary array A(end,1) will pick the last element in column 1, and A(1,end) will pick the last element in the first row.
In your example, A(end, end-2) picks an element in the last row two columns before the last one.
To interpret a statement such as
A(2:end, end:-2:1)
it might help to substitute end with the actual index of the last row/column elements, so this is equivalent to
A(2:3, 4:-2:1)
Furthermore 4:-2:1 is equivalent to the list 4,2 since we are instructing to make the list starting at 4, decreasing in steps of 2, up to (minimum) 1. So this is equivalent to
A([2 3],[4 2])
Finally, the following combination of indices is implied by A([2 3],[4 2]):
A(2,4) A(2,2)
A(3,4) A(3,2)
I need to read a file and store each number (int) in a variable, when it sees \n or a "-" (the minus sign means that it should store the numbers from 1 to 5 (1-5)) it needs to store it into the next variable. How should I proceed?
I was thinking of using fgets() but I can't find a way to do what I want.
The input looks like this:
0
0
5 10
4
2 4
5-10 2 3 4 6 7-9
4 3
These are x y positions.
I'd use fscanf to read one int at a time, and when it's negative, it is obviously the second part of a range. Or is -4--2 a valid input?
I have a 4-dimensional dictionary I made with a Python script for a data mining project I'm working on, and I want to read the data into Matlab to do some statistical tests on the data.
To read a 2-dimensional matrix is trivial. I figured that since my first dimension is only 4-deep, I could just write each slice of it out to a separate file (4 files total) with each file having many 2-dimensional slices, looking something like this:
2 3 6
4 5 8
6 7 3
1 4 3
6 6 7
8 9 0
This however does not work, and matlab reads it as a single continuous 6 x 3 matrix. I even took a look a dlmread but could not figure out how to get it do what I wanted. How do I format this so I can put 3 (or preferably more) dimensions in a single file?
A simple solution is to create a file with two lines only: the first line contains the target array size, the second line contains all your data. Then, all you need to do is reshape the data.
Say your file is
3 2 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
You do the following to read the array into the variable data
fid = fopen('myFile'); %# open the file (don't forget the extension)
arraySize = str2num(fgetl(fid)); %# read the first line, convert to numbers
data = str2num(fgetl(fid)); %# read the second line
data = reshape(data,arraySize); %# reshape the data
fclose(fid); %# close the file
Have a look at data to see how Matlab orders elements in multidimensional arrays.
Matlab stores data column wise. So from your example (assuming its a 3x2x3 matrix), matlab will store it as first, second and third column from the first "slice", followed by the first, second third columns from the second slice and so on like this
2
4
3
5
6
8
6
1
7
4
3
3
6
8
6
9
7
0
So you can write the data out like this from python (I don't know how) and then read it into matlab. Then you can reshape it back into a 3x2x3 matrix and you'll retain your correct ordering.
Hey guys, I have a question.
If given a four dimensional array in FORTRAN, and told to find a location of a certain part of it (with a starting location of 200 and 4 bytes per integer). Is there a formula to find the location if is stored in row-major and column-major order.
Basiically given array A(x:X, y:Y, z:Z, q:q) and told to find the location at A(a,b,c,d) what is the formula for finding the location
This comes up all the time when using C libraries with Fortran -- eg, calling MPI routines trying to send particular subsets of Fortran arrays.
Fortran is row-major, or more usefully, the first index moves fastest. That is, the item after A(1,2,3,4) in linear order in memory is A(2,2,3,4). So in your example above, an increase in a by one is a jump of 1 index in the array; a jump in b by one corresponds to a jump of (X-x+1); a jump in c by one corresponds to a jump of (X-x+1)x(Y-y+1), and a jump in d by one is a jump of (X-x+1)x(Y-y+1)x(Z-z+1). In C-based languages, it would be just the opposite; a jump of 1 in the d index would move you 1 index in memory; a jump in c would be a jump of (Q-q+1), etc.
If you have m indicies, and ni is the (zero-based) index in the ith index from the left, and that index has a range of Ni, then the (zero-based) index from the starting position is something like this:
where the product is 1 if the upper index is less than the lower index. To find the number of bytes from the start of the array, you'd multiply that by the size of the object, eg 4 bytes for 32-bit integers.
Been over 25 years since I did any FORTRAN.
I believe FORTRAN, unlike many other languages, lays arrays out in
column major order. That means the leftmost index is the
one that changes most frequently when processing a multi
dimensional array in linear order. Once
the maximum dimension of the leftmost index is reached, set it back to 1, assuming 1 based
indexing, and increment the next level index by 1 and start the process over again.
To calculate the index configuration for any given address offset
you need to know the value of each of the 4 array dimensions. Without this
you can't do it.
Example:
Suppose your array has dimensions 2 by 3 by 4 by 5. This implies a
total of 2 * 3 * 4 * 5 = 120 cells in the matrix. You want the index corresponding
to the 200th byte.
This would be the (200 / 4) - 1 = 49th cell (this assumes 4 bytes per cell and offset zero
is the first cell).
First observe how specific indices translate into offsets...
What cell number does the element X(1,1,1,1) occur at? Simple answer: 1
What cell number does element X(1, 2, 1, 1) occur at? Since we cycled through
the leftmost dimension it must be that dimension plus 1. In other words,
2 + 1 = 3. How about X(1, 1, 2, 1)? We cycled trough the first two dimensions
which is 2 * 3 = 6 plus 1 to give us 7. Finally X(1, 1, 1, 2) must be:
2 * 3 * 4 = 24 plus 1 gives the 25th cell.
Notice that the next righmost index does not increment until the cell number
exceeds the product of the indices to its left. Using this observation you can
calculate the indices for any given cell number by working from the rightmost
index to the left most as follows:
Right most index increments every (2 * 3 * 4 = 24) cells. 24 goes into 49 (the cell number
we want to find the indexing for) twice
leaving 1 left over. Add 1 (for 1 based indexing) that gives us a rightmost
index value of 2 + 1 = 3. Next index (moving left) changes every (2 * 3 = 12) cells. One goes into 12
zero times, this gives us index 0 + 1 = 1. Next index changes every 2 cells. One goes into 2 zero
times giving an incex value of 1. For the last (leftmost index) just add 1 to whatever is
left over, 1 + 1 = 2. This gives us the following reference X(2, 1, 1, 2).
Double check by working it back to an offset:
((2 - 1) + ((1 - 1) * 2) + ((1 - 1) * 2 * 3) + ((3 - 1) * 2 * 3 * 4) = 49.
Just change the numbers and use the same process for any number of dimensions
and/or offsets.
Fortran has column-major order for arrays. This is described at http://en.wikipedia.org/wiki/Row-major_order#Column-major_order. Further down in that article there is the equation for the memory offset of a higher dimensional array.