Find SIFT features matching between two video - arrays

Question

I extracted SIFT features from two videos to make a matching.
I need to compare each of the second video features with those stored in an array of features of the first video. I'm having trouble set the code so that when I have the correspondence I can get the frame in which the feature is present. How can I do? Can anyone show me a code example?
This is my code:
obj = VideoReader('video2.avi');
lastFrame = read(obj, inf);
numFrames = obj.NumberOfFrames;
%estrazione frame e sift
for k = 1 : 3 % numFrames / 5
disp(['Processing frame #', num2str(k)]);
this_frame = read(obj, k * 5); % leggi solo un fotogramma ogni 5 per velocizzarle la cosa
this_frame = imresize(this_frame, 0.5); % rimpiccioliamolo per questioni di efficienza!
I = single(rgb2gray(this_frame)) ;
[f,d] = vl_sift(I); % estrazione feature
features{k} = f; % salviamo le feautre e i relativi descrittori in delle celle
descriptors{k} = d;
end
save('feature_input', 'features');
save('descrittori_input', 'descriptors');
%%% un esempio di come ripescare i dati...
pippo = load('feature_input');
newfeat = pippo.features;
pippo = load('descrittori_input');
newdesc = pippo.descriptors;
for k = 1 : 3
disp(['Le feature del fotogramma', num2str(k), ' sono: ']);
f = cell2mat( newfeat(k) );
f(:, 1:10) % ne mostriamo solo un pezzetto... le posizioni delle prime 10 features
end
obj2 = VideoReader('video2u.avi');
lastFrame = read(obj2, inf);
numFrames = obj2.NumberOfFrames;
%estrazione frame e sift video2
for k2 = 1 : 3 % numFrames / 5
disp(['Processing frame #', num2str(k2)]);
this_frame2 = read(obj2, k2 * 5); % leggi solo un fotogramma ogni 5 per velocizzarle la cosa
this_frame2 = imresize(this_frame2, 0.5); % rimpiccioliamolo per questioni di efficienza!
K = single(rgb2gray(this_frame2)) ;
[f2,d2] = vl_sift(K); % estrazione feature
features2{k2} = f2; % salviamo le feautre e i relativi descrittori in delle celle
descriptors2{k2} = d2;
end
save('feature2_input', 'features2');
save('descrittori2_input', 'descriptors2');
%%% un esempio di come ripescare i dati...
pippo2 = load('feature2_input');
newfeat2 = pippo2.features2;
pippo2 = load('descrittori2_input');
newdesc2 = pippo2.descriptors2;
for k2 = 1 : 3
disp(['Le feature del fotogramma', num2str(k2), ' sono: ']);
f2 = cell2mat( newfeat2(k2) );
f2(:, 1:10) % ne mostriamo solo un pezzetto... le posizioni delle prime 10 features
end
[matches, scores] = vl_ubcmatch(d, d2, 1.5) ;
% sift points plot
subplot(1,2,1);
imshow(uint8(I));
hold on;
plot(f(1,matches(1,:)),f(2,matches(1,:)),'b*');
subplot(1,2,2);
imshow(uint8(K));
hold on;
plot(f2(1,matches(2,:)),f2(2,matches(2,:)),'r*');
figure;
%-------------
% RANSAC
X1 = f(1:2,matches(1,:)) ; X1(3,:) = 1 ;
X2 = f2(1:2,matches(2,:)) ; X2(3,:) = 1 ;
numMatches = size(matches,2) ;
for t = 1:100
% estimate homograpyh
subset = vl_colsubset(1:numMatches, 4) ;
A = [] ;
for i = subset
A = cat(1, A, kron(X1(:,i)', vl_hat(X2(:,i)))) ;
end
[U,S,V] = svd(A) ;
H{t} = reshape(V(:,9),3,3) ;
% score homography
X2_ = H{t} * X1 ;
du = X2_(1,:)./X2_(3,:) - X2(1,:)./X2(3,:) ;
dv = X2_(2,:)./X2_(3,:) - X2(2,:)./X2(3,:) ;
ok{t} = (du.*du + dv.*dv) < 6*6 ;
score(t) = sum(ok{t}) ;
end
[score, best] = max(score) ;
H = H{best};
ok = ok{best};
% sift feature matching
dh1 = max(size(K,1)-size(I,1),0) ;
dh2 = max(size(I,1)-size(K,1),0) ;
subplot(2,1,1) ;
imagesc([padarray(I,dh1,'post') padarray(K,dh2,'post')]) ;
colormap (gray);
o = size(I,2) ;
line([f(1,matches(1,:));f2(1,matches(2,:))+o], ...
[f(2,matches(1,:));f2(2,matches(2,:))]) ;
axis image off ;
subplot(2,1,2) ;
imagesc([padarray(I,dh1,'post') padarray(K,dh2,'post')]) ;
colormap (gray);
o = size(I,2) ;
line([f(1,matches(1,ok));f2(1,matches(2,ok))+o], ...
[f(2,matches(1,ok));f2(2,matches(2,ok))]) ;
title(sprintf('%d (%.2f%%) inliner matches out of %d', ...
sum(ok), ...
100*sum(ok)/numMatches, ...
numMatches)) ;
axis image off ;
drawnow ;
end

Answer 1

You are matching using [matches, scores] = vl_ubcmatch(d, d2, 1.5) ;. d contains only the descriptors of the latest frame. You should do something like:
for nFrames=1:3
[matches{nFrames}, scores{nFrames}] = vl_ubcmatch(descriptors{nFrames}, descriptors2{nFrames}, 1.5);
end
From here, you should be able to get the matches between the frames.