C: Find and analyze clusters in a binary data stream - c

Question

Structure of the file and description of the system
The stream I want to analyze (a large binary file) is composed as follows:
40-bytes header
A stream of 10-bytes signals:
The first 8 bytes represent the time when the signal was registered
The last 2 bytes describe the channel where the signal was registered
The signal is emitted by a source which sends an impulse every SIGNAL_INTERVAL, and it may or may not be retrieved by the detector. If a detector counts, it sends the result to a counter's channel, which prints the count as shown above. The counter has 8 channels in total.
Multiplexing
In order to increase the number of detectors, a multiplexing approach is used. Two detectors send their counts to the same channel (say, detectors 1 and 9 are coupled on channel 1 of the counter). One of the signals (for example, 9) is delayed by DELAY, so that the delayed counts are shifted with respect to the non delayed ones.
Demultiplexing
The idea would be to divide the delayed data from the non delayed ones, then subtract the delay (adding 8 to the channel value so that a delayed count on channel 1 will be shown as a count on channel 9) and then rejoin the two arrays.
If SIGNAL_INTERVAL is constant, this is relatively easy: I define a "mask" [0, DELAY, SIGNAL_INTERVAL] and, taking a reference timestamp value, see where every count falls in the mask.
Trying different masks and counting which one gives the most counts, one can identify the delayed counts from the non delayed ones. This last part is done because we allow an error to the time count, so the stream will not be perfectly clustered. Moreover, it's impossible to know a priori if the first count is a delayed one, a non delayed one or even a casual count.
This is done channel by channel, as the channels may have a different response time from each other.
With this approach, the code is quite simple:
uint64_t maskCheck(struct count *data, int ch_num, int elements){
const int MAX_NUM = 27; // Maximum number of masks checked
int ref = 0; // The reference variable used as starting point at every cycle
uint64_t sing_count[2][MAX_NUM]; // The array containing the singles counts
uint64_t max_count; // Variable used to find the maximum in the array
int t = 0; // Time index for the following loop
uint64_t result = 0; // The final result, i.e. the index with the most singles counts
// Initializing max_count (it has MAX_NUM as length, so it must be initialized after being declared)
for(int i = 0; i < MAX_NUM; i++)sing_count[0][i] = 0;
for(int i = 0; i < MAX_NUM; i++)sing_count[1][i] = 0;
while(getChannel(data[t])!=ch_num){
t++;
if(t == elements - 1){
printf("%s\n", "Nothing found");
return 0;
}
}
if(getChannel(data[t]) == ch_num) ref = getTimestamp(data[t]);
uint64_t ref_indexed = ref;
for(int index = 0; index < MAX_NUM; index++){
sing_count[1][index] = ref + nsToBins(index) - nsToBins(MAX_NUM/2);
ref_indexed = sing_count[1][index];
for(t = 0; t < elements; t++){
// Skips the counts not occurring at ch_num
if(getChannel(data[t]) != ch_num) {
continue;
}
if(longAbs(getTimestamp(data[t]), ref_indexed) % nsToBins(SIGNAL_INTERVAL) <= nsToBins(MASK) + nsToBins(COUNT_ERROR) &&
longAbs(getTimestamp(data[t]), ref_indexed) % nsToBins(SIGNAL_INTERVAL) >= nsToBins(MASK) - nsToBins(COUNT_ERROR)){
sing_count[0][index]++;
}
else if(longAbs(getTimestamp(data[t]), ref_indexed) % nsToBins(SIGNAL_INTERVAL) <= nsToBins(COUNT_ERROR) ||
longAbs(getTimestamp(data[t]), ref_indexed) % nsToBins(SIGNAL_INTERVAL) >= nsToBins(SIGNAL_INTERVAL) - nsToBins(COUNT_ERROR)){
sing_count[0][index]++;
}
}
}
// This last part maximizes the array.
max_count = sing_count[0][0];
result = sing_count[1][0];
for(int i = 1; i < MAX_NUM; i++){
if(sing_count[0][i] > max_count)
{
max_count = sing_count[0][i];
result = sing_count[1][i];
}
}
where struct count is defined as a 10-byte array read by the functions getTimestamp() and getChannel(), and nsToBins() simply converts the time units.
Having the "best mask", I can divide the array through it and then perform all the other needed operations.
The problem
Now, here comes the problem. SIGNAL_INTERVAL is not constant, and it's not even well determined (to give you an idea, the frequency oscillates between 75.6 Mhz and 76.3 MHz).
The above approach turns out to be very unsuccessful now:
SIGNAL_INTERVAL has an error of about 0.3 ns
The measurement is performed over 30 seconds
Keeping in mind that the order of magnitude of SIGNAL_INTERVAL is of 10 ns, after just one second the error would be too big
This results in the timestamps being incorrectly divided, affecting all the subsequent operations.
What I had in mind was something to analyze the clusters in the data (SIGNAL_INTERVAL is not constant, but the oscillation is much smaller than DELAY so some clustering could in principle be observed) and find another way to divide the two arrays.
But so far I have nothing. Any help would be appreciated.