Find two worst values and delete in sum - c

Question

A microcontroller has the job to sample ADC Values (Analog to Digital Conversion). Since these parts are affected by tolerance and noise, the accuracy can be significantly increased by deleting the 4 worst values. The find and delete does take time, which is not ideal, since it will increase the cycle time.
Imagine a frequency of 100MHz, so each command of software does take 10ns to process, the more commands, the longer the controller is blocked from doing the next set of samples
So my goal is to do the sorting process as fast as possible for this i currently use this code, but this does only delete the two worst!
uint16_t getValue(void){
adcval[8] = {};
uint16_t min = 16383 //14bit full
uint16_t max = 1; //zero is physically almost impossible!
uint32_t sum = 0; //variable for the summing
for(uint8_t i=0; i<8;i++){
if(adc[i] > max) max = adc[i];
if(adc[i] < min) min = adc[i];
sum=sum+adcval[i];
}
uint16_t result = (sum-max-min)/6; //remove two worst and divide by 6
return result;
}
Now I would like to extend this function to delete the 4 worst values out of the 8 samples to get more precision. Any advice on how to do this?
Additionally, it would be wonderful to build an efficient function that finds the most deviating values, instead of the highest and lowest. For example, imagine the this two arrays
uint16_t adc1[8] {5,6,10,11,11,12,20,22};
uint16_t adc2[8] {5,6,7,7,10,11,15,16};
First case would gain precision by the described mechanism (delete the 4 worst). But the second case would have deleted the values 5 and 6 as well as 15 and 16. But this would theoretically make the calculation worse, since deleting 10,11,15,16 would be better. Is there any fast solution of deleting the 4 most deviating?

Answer 1

If your ADC is returning values from 5 to 16 14 bits and the voltage reference 3.3V, the voltage varies from 1mV to 3mV. It is very likely that it is the correct reading. It is very difficult to design good input circuit for 14 bits ADC.
It is better to run the running average. What is the running average? It is software low pass filter.
Blue are readings from the ADC, red -running average
Second signal is the very low amplitude sine wave (9-27mV - assuming 14 bits and 3.3Vref)
The algorithm:
static int average;
int running_average(int val, int level)
{
average -= average / level;
average += val * level;
return average / level;
}
void init_average(int val, int level)
{
average = val * level;
}
if the level is the power of 2. This version needs only 6 instructions (no branches) to calculate the average.
static int average;
int running_average(int val, int level)
{
average -= average >> level;
average += val << level;
return average >> level;
}
void init_average(int val, int level)
{
average = val << level;
}
I assume that average will no overflow. If yes you need to chose larger type

Answer 2

This answer is kinda of topic as it recommends a hardware solution but if performance is required and the MCU can't implement P__J__'s solution than this is your next best thing.
It seems you want to remove noise from your input signal. This can be done in software using DSP (digital signal processing) but it can also be done by configuring your hardware differently.
By adding the proper filter at the proper space before your ADC, it will be possible to remove much (outside) noise from your ADC output. (you can't of course go below a certain amount that is innate in the ADC but alas.)
There are several q&a on electronics.stackexchange.com.
One solution is adding a capacitor to filter some high frequency noise. As noted by DerStorm8
The Photon has another great solution here by suggesting RC, Sallen-Key and a cascade of Sallen-Key filters for a continuous signal filter.
Here (ADN007) is a Analog Design Note from Microchip on "Techniques that Reduce System Noise in ADC Circuits"
It may seem that designing a low noise, 12-bit Analog-to-Digital
Converter (ADC) board or even a 10-bit board is easy. This is
true, unless one ignores the basics of low noise design. For
instance, one would think that most amplifiers and resistors work
effectively in 12-bit or 10-bit environments. However, poor device
selection becomes a major factor in the success or failure of the
circuit. Another, often ignored, area that contributes a great deal
of noise, is conducted noise. Conducted noise is already in the
circuit board by the time the signal arrives at the input of the
ADC. The most effective way to remove this noise is by using a
low-pass (anti-aliasing) filter prior to the ADC. Including by-pass
capacitors and using a ground plane will also eliminate this type
of noise. A third source of noise is radiated noise. The major
sources of this type of noise are Electromagnetic Interference
(EMI) or capacitive coupling of signals from trace-to-trace.
If all three of these issues are addressed, then it is true that
designing a low noise 12-bit ADC board is easy.
And their recommended solution path:
It is easy to design a true 12-bit ADC system by using a few
key low noise guidelines. First, examine your devices (resistors
and amplifiers) to make sure they are low noise. Second, use a
ground plane whenever possible. Third, include a low-pass filter
in the signal path if you are changing the signal from analog to
digital. Finally, and always, include by-pass capacitors. These
capacitors not only remove noise but also foster circuit stability.
Here is a good paper by Analog Devices on input noise. They note in here that "there are some instances where input noise can actually be helpful in achieving higher resolution."
All analog-to-digital converters (ADCs) have a certain amount of input-referred noise—modeled as a noise source connected in series with the input of a noise-free ADC. Input-referred noise is not to be confused with quantization noise, which is only of interest when an ADC is processing time-varying signals. In most cases, less input noise is better; however, there are some instances where input noise can actually be helpful in achieving higher resolution. If this doesn’t seem to make sense right now, read on to find out how some noise can be good noise.

Answer 3

Given that you have a fixed size array, a hard-coded sorting network should be able to correctly sort the entire array with only 19 comparisons. Currently you have 8+2*8=24 comparisons already, although it is possible that the compiler unrolls the loop, leaving you with 16 comparisons. It is conceivable that, depending on the microcontroller hardware, a sorting network can be implemented with some degree of parallelism -- perhaps you also have to query the adc values sequentially which would give you opportunity to pre-sort them, while waiting for the comparison.
An optimal sorting network should be searchable online. Wikipedia has some pointers.
So, you would end up with some code like this:
sort_data(adcval);
return (adcval[2]+adcval[3]+adcval[4]+adcval[5])/4;
Update:
As you can take from this picture (source) of optimal sorting networks, a complete sort takes 19 comparisons. However 3 of those are not strictly needed if you only want to extract the middle 4 values. So you get down to 16 comparisons.

Answer 4

to delete the 4 worst values out of the 8 samples
The methods are described on geeksforgeeks k largest(or smallest) elements in an array and you can implement the best method that suits you.
I decided to use this good site to generate best sorting algorithm with SWAP() macros needed to sort the array of 8 elements. Then I created a small C program that will test any combination of 8 element array on my sorting function. Then, because we only care of groups of 4 elements, I did something bruteforce - for each of the SWAP() macros I tried to comment the macro and see if the program still succeeds. I could comment 5 SWAP macros, leaving 14 comparisons needed to identify the smallest 4 elements in the array of 8 samples.
/**
* Sorts the array, but only so that groups of 4 matter.
* So group of 4 smallest elements and 4 biggest elements
* will be sorted ok.
* s[0]...s[3] will have lowest 4 elements
* so they have to be "deleted"
* s[4]...s[7] will have the highest 4 values
*/
void sort_but_4_matter(int s[8]) {
#define SWAP(x, y) do { \
if (s[x] > s[y]) { \
const int t = s[x]; \
s[x] = s[y]; \
s[y] = t; \
} \
} while(0)
SWAP(0, 1);
//SWAP(2, 3);
SWAP(0, 2);
//SWAP(1, 3);
//SWAP(1, 2);
SWAP(4, 5);
SWAP(6, 7);
SWAP(4, 6);
SWAP(5, 7);
//SWAP(5, 6);
SWAP(0, 4);
SWAP(1, 5);
SWAP(1, 4);
SWAP(2, 6);
SWAP(3, 7);
//SWAP(3, 6);
SWAP(2, 4);
SWAP(3, 5);
SWAP(3, 4);
#undef SWAP
}
/* -------- testing code */
#include <assert.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
int cmp_int(const void *a, const void *b) {
return *(const int*)a - *(const int*)b;
}
void printit_arr(const int *arr, size_t n) {
printf("{");
for (size_t i = 0; i < n; ++i) {
printf("%d", arr[i]);
if (i != n - 1) {
printf(" ");
}
}
printf("}");
}
void printit(const char *pre, const int arr[8],
const int in[8], const int res[4]) {
printf("%s: ", pre);
printit_arr(arr, 8);
printf(" ");
printit_arr(in, 8);
printf(" ");
printit_arr(res, 4);
printf("\n");
}
int err = 0;
void test(const int arr[8], const int res[4]) {
int in[8];
memcpy(in, arr, sizeof(int) * 8);
sort_but_4_matter(in);
// sort for memcmp below
qsort(in, 4, sizeof(int), cmp_int);
if (memcmp(in, res, sizeof(int) * 4) != 0) {
printit("T", arr, in, res);
err = 1;
}
}
void test_all_combinations() {
const int result[4] = { 0, 1, 2, 3 }; // sorted
const size_t n = 8;
int num[8] = { 0, 1, 2, 3, 4, 5, 6, 7 };
for (size_t j = 0; j < n; j++) {
for (size_t i = 0; i < n-1; i++) {
int temp = num[i];
num[i] = num[i+1];
num[i+1] = temp;
test(num, result);
}
}
}
int main() {
test_all_combinations();
return err;
}
Tested on godbolt. The sort_but_4_matter with gcc -O2 on x86_64 compiles to less then 100 instruction.