Haproxy header differs from what is expected - arrays

I am working with Haproxy and need to read the v2 header from TCP stream, according to spec it is supposed to have first 13 bytes following:
\x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A\x02
in byte
{byte[13]}
[0]: 13
[1]: 10
[2]: 13
[3]: 10
[4]: 0
[5]: 13
[6]: 10
[7]: 81
[8]: 85
[9]: 73
[10]: 84
[11]: 10
[12]: 2
however I get
[0]: 13
[1]: 10
[2]: 0
[3]: 13
[4]: 10
[5]: 81
[6]: 85
[7]: 73
[8]: 84
[9]: 10
[10]: 33
[11]: 17
[12]: 0
My read does seem to be missing first two bytes 13 10 from the spec, however ending seems to differ too, I can't seem to figure out why...
What is even more confusing I can't see from IP address (54.219.177.232) nor to (45.32.136.69) nowhere in the header they are supposed to be 17th byte onward
Count = 91
[0]: 12
[1]: 180
[2]: 183
[3]: 211
[4]: 173
[5]: 172
[6]: 31
[7]: 23
[8]: 237
[9]: 228
[10]: 70
[11]: 1
[12]: 187
[13]: 22
[14]: 3
[15]: 1
[16]: 0
[17]: 178
[18]: 1
[19]: 0
[20]: 0
[21]: 174
[22]: 3
[23]: 3
[24]: 65
[25]: 71
[26]: 172
[27]: 83
[28]: 43
[29]: 146
[30]: 158
[31]: 30
[32]: 150
[33]: 85
[34]: 84
[35]: 34
[36]: 158
[37]: 111
[38]: 242
[39]: 247
[40]: 173
[41]: 198
[42]: 13
[43]: 133
[44]: 9
[45]: 39
[46]: 111
[47]: 9
[48]: 44
[49]: 64
[50]: 232
[51]: 25
[52]: 14
[53]: 51
[54]: 110
[55]: 76
[56]: 0
[57]: 0
[58]: 32
[59]: 74
[60]: 74
[61]: 192
[62]: 43
[63]: 192
[64]: 47
[65]: 192
[66]: 44
[67]: 192
[68]: 48
[69]: 204
[70]: 169
[71]: 204
[72]: 168
[73]: 204
[74]: 20
[75]: 204
[76]: 19
[77]: 192
[78]: 19
[79]: 192
[80]: 20
[81]: 0
[82]: 156
[83]: 0
[84]: 157
[85]: 0
[86]: 47
[87]: 0
[88]: 53
[89]: 0
[90]: 10`
Code that reads header looks following:
var childSocketThread = new Thread(() =>
{
var getOrPostHttp1Acc = new List<byte>();
var getOrPostHttp1 = new byte[1];
while (getOrPostHttp1[0] != 10)
{
socket.Receive(getOrPostHttp1);
getOrPostHttp1Acc.Add(getOrPostHttp1[0]);
}
bool isGet = Encoding.ASCII.GetString(getOrPostHttp1Acc.ToArray()).ToUpper().Contains("GET");
byte[] proxyv2headerIdentifier = new byte[13];
socket.Receive(proxyv2headerIdentifier);
var proxyv2header = new List<byte>();
var proxyv2HeaderBuffer = new byte[1];
if(proxyv2headerIdentifier.SequenceEqual(proxyv2HeaderStartRequence))//TODO:uncomment this, hardcoded (true) is only for testing
{
while (proxyv2HeaderBuffer[0] != 10)
{
socket.Receive(proxyv2HeaderBuffer);
proxyv2header.Add(proxyv2HeaderBuffer[0]);
}
string headerString = $"proxyv2:{Encoding.ASCII.GetString(proxyv2header.ToArray())}";
byte[] bodyBuff = new byte[0];
byte[] buffer = new byte[1];
int contentLength = 0;
while (!headerString.Contains("\r\n\r\n"))
//at this point header is read into proxyv2header ....
Where am I misreading specification, or doing something wrong?

Ok I have figured this one myself...
mistake was to read the http method first (if it's GET or POST) haproxy does not start with that... so removing method read
var getOrPostHttp1Acc = new List<byte>();
var getOrPostHttp1 = new byte[1];
while (getOrPostHttp1[0] != 10)
{
socket.Receive(getOrPostHttp1);
getOrPostHttp1Acc.Add(getOrPostHttp1[0]);
}
bool isGet = Encoding.ASCII.GetString(getOrPostHttp1Acc.ToArray()).ToUpper().Contains("GET");
and reducing start sequence to 12 bytes did it...
In regards to IP, it was hiding in the plain sight all this time...
(check bytes 4-7)
Hope this saves you some time.

Related

Splitting a large matrix

I'm using Julia to ingest a large two dimensional data array (data) of size 1000 x 32768; I need to break up the array into smaller square arrays along both dimensions. For instance, I would like to break data into a grid of smaller, square arrays similar to the following image:
Note that no pixels get left out -- when another square cannot be fit in along either axis, the last possible array of square pixels is returned as another array (hence the shifted pink squares on the right hand side).
Currently, I'm doing this through a function I built to decimate the raw dataset:
function decimate_square(data,fraction=4)
# Read size of input data / calculate length of square side
sy,sx = size(data)
square_side = Int(round(sy/fraction))
# Number of achievable full squares
itersx,itersy = [Int(floor(s/square_side)) for s in [sx,sy]]
# Find left/right X values
for ix in 1:itersx
if ix!=itersx
# Full sliding square can be calculated
left = square_side*(ix-1) + 1
right = square_side*(ix)
else
# Capture last square of data
left = sx-square_side + 1
right = sx
end
# Find top/bottom Y values for each X
for iy in 1:itersy
if iy!=itersy
# Full sliding square can be calculated
top = square_side*(iy-1) + 1
bottom = square_side*(iy)
else
# Capture last square of data
top = sy-square_side + 1
bottom = sy
end
# Record data in 3d stack
cursquare = data[top:bottom,left:right]
if (ix==1)&&(iy==1); global dstack=cursquare
else; dstack=cat(dstack,cursquare,dims=3)
end
end
end
return dstack
end
Which currently takes ~20 seconds to run:
rand_arr = rand(1000,32768)
t1 = Dates.now()
dec_arr = decimate_square(rand_arr)
t2 = Dates.now()
#info(t2-t1)
[ Info: 19666 milliseconds
This is the biggest bottleneck of my analysis. Is there a pre-built function that I can use, or is there a more efficient way to decimate my array?
You can take views as Przemyslaw Szufel suggests, and the CartesianIndex type comes in handy for selecting blocks of the matrix.
julia> function squareviews(data, fraction = 4)
squareside = floor(Int, size(data, 1) / fraction)
[#view(M[CartesianIndex(ix-squareside+1, iy-squareside+1):CartesianIndex(ix, iy)])
for ix in squareside:squareside:size(data, 1),
iy in squareside:squareside:size(data, 2)]
end
squareviews (generic function with 2 methods)
julia> result = squareviews(M)
4×40 Matrix{SubArray{Int64, 2, Matrix{Int64}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}}:
[346 392 … 746 429; 380 193 … 476 757; … ; 424 329 … 285 427; 591 792 … 710 891] … [758 916 … 7 185; 26 846 … 631 808; … ; 945 713 … 875 137; 793 655 … 400 322]
[55 919 … 402 728; 292 238 … 266 636; … ; 62 490 … 913 126; 293 475 … 492 20] [53 8 … 146 365; 216 673 … 157 909; … ; 955 635 … 332 945; 354 913 … 922 272]
[278 966 … 128 334; 700 560 … 226 701; … ; 529 398 … 17 674; 237 830 … 4 788] [239 274 … 983 911; 591 669 … 762 675; … ; 213 949 … 917 903; 336 890 … 633 578]
[723 483 … 135 283; 729 579 … 1000 942; … ; 987 383 … 764 544; 682 942 … 376 179] [370 859 … 444 566; 34 106 … 320 161; … ; 310 41 … 868 349; 719 341 … 718 800]
This divides the data matrix into blocks such that result[2, 3] gives the square that is 2nd from the top and 3rd from the left. (My matrix M was 100x1000 in size, so there are 100/25 = 4 blocks vertically and 1000/25 = 40 blocks horizontally.)
If you want the results linearly like in your original function, you can instead have the second line of the function be:
julia> function squareviews(data, fraction = 4)
squareside = floor(Int, size(data, 1) / fraction)
[#view(M[CartesianIndex(ix-squareside+1, iy-squareside+1):CartesianIndex(ix, iy)])
for iy in squareside:squareside:size(data, 2)
for ix in squareside:squareside:size(data, 1)]
end
squareviews (generic function with 2 methods)
julia> squareviews(M)
160-element Vector{SubArray{Int64, 2, Matrix{Int64}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}}:
(Note the subtle changes in the for syntax - the iy comes before ix here, there's no comma, and there's an extra for.)
This returns a vector of square matrices (views).
Your original function returned a three-dimensional matrix, in which you'd access values as originalresult[i, j, k]. Here, the equivalent would be result[k][i, j].
There's a lot of stuff going on in your code with is not recommended and making things slow. Here's a somewhat idiomatic solution, with the additional bonus of generalizing to arbitrary ranks:
julia> function square_indices(data; fraction=4)
splits = cld.(size(data), fraction)
return Iterators.map(CartesianIndices, Iterators.product(Iterators.partition.(axes(data), splits)...))
end
square_indices (generic function with 1 method)
The result of this is an iterator over CartesianIndices, which are objects that you can use to index your squares. Either the regular data[ix], or view(data, ix), which does not create a copy. (Different fractions per dimension are possible, try test_square_indices(println, 4, 4, 4; fraction=(2, 1, 1)).)
And to see whether it works as expected:
julia> function test_square_indices(f, s...; fraction=4)
arr = reshape(1:prod(s), s...)
for ix in square_indices(arr; fraction)
f(view(arr, ix))
end
end
test_square_indices (generic function with 1 method)
julia> # just try this on some moderatly costly function
#btime test_square_indices(v -> inv.(v), 1000, 32768)
81.980 ms (139 allocations: 250.01 MiB)
julia> test_square_indices(println, 9)
[1, 2, 3]
[4, 5, 6]
[7, 8, 9]
julia> test_square_indices(println, 9, 5)
[1 10; 2 11; 3 12]
[4 13; 5 14; 6 15]
[7 16; 8 17; 9 18]
[19 28; 20 29; 21 30]
[22 31; 23 32; 24 33]
[25 34; 26 35; 27 36]
[37; 38; 39;;]
[40; 41; 42;;]
[43; 44; 45;;]
julia> reshape(1:9*5, 9, 5)
9×5 reshape(::UnitRange{Int64}, 9, 5) with eltype Int64:
1 10 19 28 37
2 11 20 29 38
3 12 21 30 39
4 13 22 31 40
5 14 23 32 41
6 15 24 33 42
7 16 25 34 43
8 17 26 35 44
9 18 27 36 45
julia> test_square_indices(println, 4, 4, 4; fraction=2)
[1 5; 2 6;;; 17 21; 18 22]
[3 7; 4 8;;; 19 23; 20 24]
[9 13; 10 14;;; 25 29; 26 30]
[11 15; 12 16;;; 27 31; 28 32]
[33 37; 34 38;;; 49 53; 50 54]
[35 39; 36 40;;; 51 55; 52 56]
[41 45; 42 46;;; 57 61; 58 62]
[43 47; 44 48;;; 59 63; 60 64]
julia> reshape(1:4*4*4, 4, 4, 4)
4×4×4 reshape(::UnitRange{Int64}, 4, 4, 4) with eltype Int64:
[:, :, 1] =
1 5 9 13
2 6 10 14
3 7 11 15
4 8 12 16
[:, :, 2] =
17 21 25 29
18 22 26 30
19 23 27 31
20 24 28 32
[:, :, 3] =
33 37 41 45
34 38 42 46
35 39 43 47
36 40 44 48
[:, :, 4] =
49 53 57 61
50 54 58 62
51 55 59 63
52 56 60 64
Here's a bit of an illustration of how this works:
julia> data = reshape(1:9*5, 9, 5); fraction = 3;
julia> size(data)
(9, 5)
julia> # chunk sizes
splits = cld.(size(data), fraction)
(3, 2)
julia> # every dimension chunked
Iterators.partition.(axes(data), splits) .|> collect
(UnitRange{Int64}[1:3, 4:6, 7:9], UnitRange{Int64}[1:2, 3:4, 5:5])
julia> # cross product of all chunks
Iterators.product(Iterators.partition.(axes(data), splits)...) .|> collect
3×3 Matrix{Vector{UnitRange{Int64}}}:
[1:3, 1:2] [1:3, 3:4] [1:3, 5:5]
[4:6, 1:2] [4:6, 3:4] [4:6, 5:5]
[7:9, 1:2] [7:9, 3:4] [7:9, 5:5]
You could just go with views. Suppose you want to slice your data into 64 matrices, each having size 1000 x 512. In that case you could do:
dats = view.(Ref(rand_arr),Ref(1:1000), [range(1+(i-1)*512,i*512) for i in 1:64])
The time for this on my machine is 600 nanoseconds:
julia> #btime view.(Ref($rand_arr),Ref(1:1000), [range(1+(i-1)*512,i*512) for i in 1:64]);
595.604 ns (3 allocations: 4.70 KiB)

Adding Character String with Numbers / Concatenate Number with String in Matlab Array

How to add character ',' or '+' in a matlab 2d array.
I've tried the following way.
clc
clear all
close all
min=0;
max=1052;
random_int = randi([min max],5,10)
% random_int=[515,586,942,742;353,588,916,436]
% load('Random_Int_x.mat')
% random_int
[m,n]=size(random_int);
for i=1:1:m
allOneString = sprintf('%d,' , random_int(i,:));
allOneString= allOneString(1:end-1)% strip final comma
Str_1(i,:)=allOneString
% allOneString= strjoin(arrayfun(#(x) num2str(x),random_int(i,:),'UniformOutput',false),',');
end
Str_1
Example of Input / Matrix
random_int =
2 9 7 7 9 8 2 5 7 5
6 1 9 9 6 1 9 4 1 0
5 0 8 8 5 6 9 0 4 6
0 9 9 8 7 5 6 3 7 8
8 4 2 0 5 5 1 8 2 6
Output:
Str_1 =
5×19 char array
'2,9,7,7,9,8,2,5,7,5'
'6,1,9,9,6,1,9,4,1,0'
'5,0,8,8,5,6,9,0,4,6'
'0,9,9,8,7,5,6,3,7,8'
'8,4,2,0,5,5,1,8,2,6'
This works properly with random number between 0-9.. However if I put input above 9 --> 10 .. then matlab throws matrix dimension error.
Subscripted assignment dimension mismatch.
Error in Number_with_String (line 14)
Str_1(i,:)=allOneString;
For Input above 9:
random_int =
76 96 88 23 26 25 92 5 61 86
87 69 32 36 86 39 46 21 55 69
42 26 56 69 55 97 91 78 76 41
74 74 24 3 46 52 29 70 88 4
7 48 13 69 15 12 79 91 90 24
Expecting output:
'76,96,88,23,26,25,92,5,61,86'
'87,69,32,36,86,39,46,21,55,69' ... etc
Any suggestion to resolve this ..
Here's a way:
random_int = randi([0 500],5,10); % example data
y = mat2cell(random_int, ones(1,size(random_int,1)), size(random_int,2)); % split into rows
y = cellfun(#(x) sprintf('%i,', x), y, 'UniformOutput', false); % strings with commas
y = cellfun(#(s) s(1:end-1), y, 'UniformOutput', false); % remove last comma from each
Example result:
>> y
y =
5×1 cell array
'74,281,294,376,124,203,211,170,242,334'
'488,268,31,84,404,74,205,178,215,20'
'120,242,390,37,113,199,140,375,395,469'
'455,94,115,476,28,20,365,213,181,31'
'130,62,138,421,261,105,114,226,398,90'
I would recommend you use string which shipped in 16b. You can convert the result to char or cellstr if you need.
>> min=0; max=1052;
>> random_int = randi([min max],5,10)
random_int =
532 145 857 264 616 793 558 494 327 688
736 157 256 648 578 400 820 12 556 725
938 271 978 498 965 597 983 354 174 787
1010 885 368 370 300 79 136 170 633 474
576 267 207 874 797 56 598 836 276 88
>> str = join(string(random_int),',')
str =
5×1 string array
"532,145,857,264,616,793,558,494,327,688"
"736,157,256,648,578,400,820,12,556,725"
"938,271,978,498,965,597,983,354,174,787"
"1010,885,368,370,300,79,136,170,633,474"
"576,267,207,874,797,56,598,836,276,88"
>> char(str)
ans =
5×39 char array
'532,145,857,264,616,793,558,494,327,688'
'736,157,256,648,578,400,820,12,556,725 '
'938,271,978,498,965,597,983,354,174,787'
'1010,885,368,370,300,79,136,170,633,474'
'576,267,207,874,797,56,598,836,276,88 '
clc
clear all
close all
min=0;
max=1052;
random_int = randi([min max],200,10);
[m,n]=size(random_int);
for i=1:1:m
allOneString = sprintf('%d,' , random_int(i,:));
allOneString= allOneString(1:end-1); % strip final comma
Str_1{i}=allOneString;
end
Str_1=Str_1'

Why am I encountering this error pertaining to non-matching object lengths when the individual parts of this script work fine on their own?

I have a script to process weather polar radar data into cartesian coordinates and then plot it. I have tested each individual component and the individual components do what they're supposed to every time. Recently I had the need to streamline everything and so I put it in a script, but when I try to run my data through it, it kicks out an error message that I don't fully understand. Any help would be greatly appreciated. Thanks in advance. I am executing the script with the command radProcess(test, 1, ref).
My data looks like this (although this example has been scaled down from the 3600x800 data frame that it's in)-
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10
1 -96 -75 -69 -62 51 40 47 50 52 47
2 -94 -80 -67 -57 53 37 44 51 54 50
3 -100 -81 -72 -61 54 42 50 48 56 50
4 -101 -82 -72 -63 55 43 40 47 50 48
5 -999 -78 -73 -59 55 40 46 49 54 54
6 -102 -81 -71 -59 51 37 44 52 55 57
7 -101 -79 -74 -59 54 43 42 47 55 47
8 -95 -80 -73 -59 52 40 48 54 58 54
9 -96 -78 -75 -58 57 44 39 50 47 55
10 -99 -79 -73 -59 57 45 46 56 55 53
I'm encountering an error message when I try to run my data that looks like this-
Error in radProcess(test, 10, ref) :
dims [product 864000] do not match the length of object [2880000]
In addition: Warning message:
In final.levl[cbind(z.rad, t.rad, r.rad)] * conversion.factor :
longer object length is not a multiple of shorter object length
The script can be seen below-
radProcess <- function(file, level, product){
## Convert file to 3 dimensional array format organized ##
## by scan level (10-top, 1-bottom of array) ##
print("Converting to 3D Array")
x.arr.vert <- array(unlist(file), dim = c(10,360,800))
x.arr.horz <- aperm(array(x.arr.vert, dim = c(360,10,800)), c(2,1,3))
final.levl <- x.arr.horz[ c(10:1),,]
## Create matrix of coordinates and values and then ##
## converts from polar to cartesian coordinates ##
print("Creating Matrix of Polar Coordinates")
mat <- which( (final.levl > -1000), arr.ind = TRUE)
z.rad <- mat[, 1]
t.rad <- mat[, 2]
r.rad <- mat[, 3]
print("Converting to Cartesian Coordinates")
theta.polar <- t.rad * pi / 180
r.polar <- r.rad * 0.075
x.cart <- r.polar * cos(theta.polar)
y.cart <- r.polar * sin(theta.polar)
## Reflectivity adjustment constant = .514245 ##
## Velocity adjustment constant = .1275628 ##
print("Determining Conversion Factor")
conversion.factor <- ifelse( (product == "ref"), yes = .514245, no = .1275628)
print("Copying Values from Array to Matrix")
value <- (final.levl[cbind(z.rad, t.rad, r.rad)] * conversion.factor)
Cart.Coord.Matrix <- matrix( NA, nrow = 2880000, ncol = 4)
Cart.Coord.Matrix <- cbind(z.rad, y.cart, x.cart, value)
## Create new matrix level wanted from transposed value matrix ##i
print("Reducing down to Level Wanted")
specified.level <- Cart.Coord.Matrix[z.rad == level,]
## Plot level values in Radar plot ##
print("Plotting Data Points")
x1<-specified.level[,3]
y2<-specified.level[,2]
z3<-specified.level[,4]
d1 <- data.frame(x1,y2,z3)
dg1 <-qplot(y2,x1,colour=z3,data=d1)
dg1 + scale_colour_gradientn(limits = c(0, 60), colours = rev(rainbow(10)))
}
Example of finished product

CipherSaber bug

So I implemented ciphersaber-1. It almost works, I can decrypt the cstest1.cs1. But i have trouble getting cstest2.cs1 to work.
The output is:
The Fourth Amendment to the Constitution of the Unite ▀Stat→s of America
"The right o☻ the people to be secure in their persons, houses, papers, and
effects, against unreasonab→e searches an╚A)┤Xx¹▼☻dcðþÈ_#­0Uc.?n~J¿|,lómsó£k░7╠▄
íuVRÊ ╣├xð"↕(Gû┤.>!{³♫╚Tƒ}Àõ+»~C;ÔÙ²÷g.qÏø←1ß█yÎßsÈ÷g┐ÅJÔÞ┘Îö║AÝf╔ìêâß╗È;okn│CÚê
õ&æÄ[5&Þ½╔s╦Nå1En♂☻♫ôzÓ9»Á╝ÐÅ├ðzÝÎòeØ%W¶]¤▲´Oá╗e_Ú)╣ó0↑ï^☻P>ù♂­¥¯▄‗♦£mUzMצվ~8å
ì½³░Ùã♠,H-tßJ!³*²RóÅ
So I must have a bug in initializing the state. The odd thing is that I can encrypt and decrypt long texts without problems, so the bug is symmetric.
I implemented the rc4 cipher as a reentrent single byte algorithm as you can see in rc4.c.
The state is stored in the rc4_state struct:
typedef unsigned char rc4_byte;
struct rc4_state_
{
rc4_byte i;
rc4_byte j;
rc4_byte state[256];
};
typedef struct rc4_state_ rc4_state;
The state is initialized with rc4_init:
void rc4_init(rc4_state* state, rc4_byte* key, size_t keylen)
{
rc4_byte i, j, n;
i = 0;
do
{
state->state[i] = i;
i++;
}
while (i != 255);
j = 0;
i = 0;
do
{
n = i % keylen;
j += state->state[i] + key[n];
swap(&state->state[i], &state->state[j]);
i++;
}
while (i != 255);
state->i = 0;
state->j = 0;
}
The actual encryption / decryption is done in rc4:
rc4_byte rc4(rc4_state* state, rc4_byte in)
{
rc4_byte n;
state->i++;
state->j += state->state[state->i];
swap(&state->state[state->i], &state->state[state->j]);
n = state->state[state->i] + state->state[state->j];
return in ^ state->state[n];
}
For completeness, swap:
void swap(rc4_byte* a, rc4_byte* b)
{
rc4_byte t = *a;
*a = *b;
*b = t;
}
I have been breaking my head on this for more than two days... The state, at least for the "asdfg" key is correct. Any help would be nice.
The whole thing can be found in my github reopsitory: https://github.com/rioki/ciphersaber/
I stumbled across your question while searching online, but since you haven't updated your code at GitHub yet, I figured you might still like to know what the problem was.
It's in this bit of code:
i = 0;
do
{
state->state[i] = i;
i++;
}
while (i != 255);
After this loop has iterated 255 times, i will have a value of 255 and the loop will terminate. As a result, the last byte of your state buffer is being left uninitialised.
This is easily fixed. Just change while (i != 255); to while (i);.
Sorry you haven't gotten feedback, I finally pulled this off in Python 3 today, but don't know enough about C to debug your code.
Some of the links on the main ciphersaber page are broken (pointing to ".com" instead of ".org"), so you might not have found the FAQ:
http://ciphersaber.gurus.org/faq.html
It includes the following debugging tips:
Make sure you are not reading or writing encrypted files as text files. You must use binary mode for file I/O.
If you are writing in the C language, be sure to store bytes as unsigned char.
Watch out for classic indexing problems. Do arrays in you chosen programming language start with 0 or 1?
Make sure you are writing out a random 10 byte IV when you encrypt and are reading the IV from the start of the file when you decrypt.
If your program still does not work, put in some statements to print out the S array after the key setup step. Then run your program to
decrypt the file cstest1.cs1 using asdfg as the key. Here is how the S
array should look:
file: cstest1.cs1
key: asdfg
176 32 49 160 15 112 58 8 186 19 50 161 60 17 82 153 37 141 131 127 59
2 165 103 98 53 9 57 41 150 174 64 36 62 191 154 44 136 149 158 226
113 230 227 247 155 221 34 125 20 163 95 128 219 1 181 201 146 88 204
213 80 143 164 145 234 134 248 100 77 188 235 76 217 194 35 75 99 126
92 243 177 52 180 83 140 198 42 151 18 91 33 16 192 101 48 97 220 114
110 124 72 139 218 142 118 81 84 31 29 195 68 209 172 200 214 93 240
61 22 206 123 152 7 203 10 119 171 79 250 109 137 199 167 11 104 211
129 208 216 178 207 242 162 30 120 65 115 87 170 47 69 244 212 45 85
73 222 225 185 63 0 179 210 108 245 202 46 96 148 51 173 24 182 89 116
3 67 205 94 231 23 21 13 169 215 190 241 228 132 252 4 233 56 105 26
12 135 223 166 238 229 246 138 239 54 5 130 159 236 66 175 189 147 193
237 43 40 117 157 86 249 74 27 156 14 133 251 196 187 197 102 106 39
232 255 121 122 253 111 90 38 55 70 184 78 224 25 6 107 168 254 144 28
183 71
I also found the "memorable test cases" helpful here:
http://www.cypherspace.org/adam/csvec/
Including:
key="Al"+ct="Al Dakota buys"(iv="Al Dakota "):
pt = "mead"
Even though the memorable test cases require cs2, upgrading to cs2 from cs1 is fairly trivial, you may be able to confidently convert your program to cs2 from cs1 even without fully debugging the rest of it.
Also note that the FAQ claims there used to be a file on the site that wouldn't decode, make sure your target file doesn't begin with "0e e3 f9 b2 40 11 fc 3e ..."
(Though I think that was a smaller test file, not the certificate.)
Oh, and also know that the site's not really up to date on the latest research into RC4 and derivatives. Just reserve this as a toy program unless all else fails.
Python
Here's one I wrote in Python for a question that later got deleted. It processes the file as a stream so memory usage is modest.
Usage
python encrypt.py <key> <rounds> < <infile> > <outfile>
python decrypt.py <key> <rounds> < <infile> > <outfile>
rc4.py
#!/usr/bin/env python
# coding: utf-8
import psyco
from sys import stdin,stdout,argv
def rc4(K):
R=range(256)
S=R[:]
T=bytearray(K*256)[:256]
j=0
for i in R*int(argv[2]):
j=j+S[i]+T[i]&255
S[i],S[j]=S[j],S[i]
i=j=0
while True:
B=stdin.read(4096)
if not B: break
for c in B:
i+=1&255
j=j+S[i]&255
S[i],S[j]=S[j],S[i]
stdout.write(chr(ord(c)^S[S[i]+S[j]&255]))
psyco.bind(rc4)
encrypt.py
from rc4 import *
import os
V=os.urandom(10)
stdout.write(V)
rc4(argv[1]+V)
decrypt.py
from rc4 import *
V=stdin.read(10)
rc4(argv[1]+V)

Read file with strings and integers in matlab

I have to read a file in Matlab that looks like this:
D:\Classified\positive-videos\vid.avi 163 3 14 32 54 79 105 130 155 202 216 224 238 250 262 288 288 322 357 369 381 438 457 478 499 525 551
D:\Classified\positive-videos\vid2.avi 163 3 14 32 54 79 105 130 155 202 216 224 238 250 262 288 288 322 357 369 381 438 457 478 499 525 551
There are many such lines separated by newline. I need to read it such that: I discard path of video name and first integer(eg 163 in first line) and read rest all the numbers in an array till new line occurs. How can this be done?
You could do the following:
fid = fopen('test1.txt','r');
my_line = fgetl(fid);
while(my_line ~= -1)
my_array = regexp(my_line,' ','split');
my_line = fgetl(fid);
disp(my_array(3:end));
end
fclose(fid);
This would give you:
ans =
Columns 1 through 11
'3' '14' '32' '54' '79' '105' '130' '155' '202' '216' '224'
Columns 12 through 22
'238' '250' '262' '288' '288' '322' '357' '369' '381' '438' '457'
Columns 23 through 26
'478' '499' '525' '551'
ans =
Columns 1 through 11
'3' '14' '32' '54' '79' '105' '130' '155' '202' '216' '224'
Columns 12 through 22
'238' '250' '262' '288' '288' '322' '357' '369' '381' '438' '457'
Columns 23 through 26
'478' '499' '525' '551'
EDIT
For a numeric matrix result you can change it as:
clear;
close;
clc;
fid = fopen('test1.txt','r');
my_line = fgetl(fid);
my_array = regexp(my_line,' ','split');
my_matrix = zeros(0, numel(my_array(3:end)));
ii = 1;
while(my_line ~= -1)
my_array = regexp(my_line,' ','split');
my_line = fgetl(fid);
my_matrix = [my_matrix;zeros(1,size(my_matrix,2))];
for jj=1:numel(my_array(3:end))
my_matrix(ii,jj) = str2num(cell2mat(my_array(jj+2)));
end
ii = ii + 1;
end
fclose(fid);
This would yeild:
my_matrix =
3 14 32 54 79 105 130 155 202 216 224 238 250 262 288 288 322 357 369 381 438 457 478 499 525 551
3 14 32 54 79 105 130 155 202 216 224 238 250 262 288 288 322 357 369 381 438 457 478 499 525 551
A way easier method follows up:
fid = importdata(filename)
results = fid.data;
Ad maiora.
EDIT
Since you wanna discard the first value after the string, you will have to call
res = fid.data(:,2:end);
instead of results.

Resources