C Code Wavelet Transform and Explanation - c

I am trying to implement a wavelet transform in C and I have never done it before. I have read some about Wavelets, and understand the 'growing subspaces' idea, and how Mallat's one sided filter bank is essentially the same idea.
However, I am stuck on how to actually implement Mallat's fast wavelet transform. This is what I understand so far:
The high pass filter, h(t), gives you the detail coefficients. For a given scale j, it is a reflected, dilated, and normed version of the mother wavelet W(t).
g(t) is then the low pass filter that makes up the difference. It is supposed to be the quadrature mirror of h(t)
To get the detail coefficients, or the approximation coefficients for the jth level, you need to convolve your signal block with h(t) or g(t) respectively, and downsample the signal by 2^{j} (ie take every 2^{j} value)
However these are my questions:
How can I find g(t) when I know h(t)?
How can I compute the inverse of this transform?
Do you have any C code that I can reference? (Yes I found the one on wiki but it doesn't help)
What I would like some code to say is:
A. Here is the filter
B. Here is the transform (very explicitly)
C.) Here is the inverse transform (again for dummies)
Thanks for your patience, but there doesn't seem to be a Step1 - Step2 - Step3 -- etc guide out there with explicit examples (that aren't HAAR because all the coefficients are 1s and that makes things confusing).

the Mallat recipe for the fwt is really simple. If you look at the matlab code, eg the script by Jeffrey Kantor, all the steps are obvious.
In C it is a bit more work but that is mainly because you need to take care of your own declarations and allocations.
Firstly, about your summary:
usually the filter h is a lowpass filter, representing the scaling function (father)
likewise, g is usually the highpass filter representing the wavelet (mother)
you cannot perform a J-level decomposition in 1 filtering+downsampling step. At each level, you create an approximation signal c by filtering with h and downsampling, and a detail signal d by filtering with g and downsampling, and repeat this at the next level (using the current c)
About your questions:
for a filter h of an an orthogonal wavelet basis, [h_1 h_2 .. h_m h_n], the QMF is [h_n -h_m .. h_2 -h_1], where n is an even number and m==n-1
the inverse transform does the opposite of the fwt: at each level it upsamples detail d and approximation c, convolves d with g and c with h, and adds the signals together -- see the corresponding matlab script.
Using this information, and given a signal x of len points of type double, scaling h and wavelet g filters of f coefficients (also of type double), and a decomposition level lev, this piece of code implements the Mallat fwt:
double *t=calloc(len+f-1, sizeof(double));
memcpy(t, x, len*sizeof(double));
for (int i=0; i<lev; i++) {
memset(y, 0, len*sizeof(double));
int len2=len/2;
for (int j=0; j<len2; j++)
for (int k=0; k<f; k++) {
y[j] +=t[2*j+k]*h[k];
y[j+len2]+=t[2*j+k]*g[k];
}
len=len2;
memcpy(t, y, len*sizeof(double));
}
free(t);
It uses one extra array: a 'workspace' t to copy the approximation c (the input signal x to start with) for the next iteration.
See this example C program, which you can compile with gcc -std=c99 -fpermissive main.cpp and run with ./a.out.
The inverse should also be something along these lines. Good luck!

The only thing that is missing is some padding for the filter operation.
The lines
y[j] +=t[2*j+k]*h[k];
y[j+len2]+=t[2*j+k]*g[k];
exceed the boundaries of the t-array during first iteration and exceed the approximation part of the array during the following iterations. One must add (f-1) elements at the beginning of the t-array.
double *t=calloc(len+f-1, sizeof(double));
memcpy(&t[f], x, len*sizeof(double));
for (int i=0; i<lev; i++) {
memset(t, 0, (f-1)*sizeof(double));
memset(y, 0, len*sizeof(double));
int len2=len/2;
for (int j=0; j<len2; j++)
for (int k=0; k<f; k++) {
y[j] +=t[2*j+k]*h[k];
y[j+len2]+=t[2*j+k]*g[k];
}
len=len2;
memcpy(&t[f], y, len*sizeof(double));
}

Related

Elementwise product between a vector and a matrix using GNU Blas subroutines

I am working on C, using GNU library for scientific computing. Essentially, I need to do the equivalent of the following MATLAB code:
x=x.*(A*x);
where x is a gsl_vector, and A is a gsl_matrix.
I managed to do (A*x) with the following command:
gsl_blas_dgemv(CblasNoTrans, 1.0, A, x, 1.0, res);
where res is an another gsl_vector, which stores the result. If the matrix A has size m * m, and vector x has size m * 1, then vector res will have size m * 1.
Now, what remains to be done is the elementwise product of vectors x and res (the results should be a vector). Unfortunately, I am stuck on this and cannot find the function which does that.
If anyone can help me on that, I would be very grateful. In addition, does anyone know if there is some better documentation of GNU rather than https://www.gnu.org/software/gsl/manual/html_node/GSL-BLAS-Interface.html#GSL-BLAS-Interface which so far is confusing me.
Finally, would I lose in time performance if I do this step by simply using a for loop (the size of the vector is around 11000 and this step will be repeated 500-5000 times)?
for (i = 0; i < m; i++)
gsl_vector_set(res, i, gsl_vector_get(x, i) * gsl_vector_get(res, i));
Thanks!
The function you want is:
gsl_vector_mul(res, x)
I have used Intel's MKL, and I like the documentation on their website for these BLAS routines.
The for-loop is ok if GSL is well designed. For example gsl_vector_set() and gsl_vector_get() can be inlined. You could compare the running time with gsl_blas_daxpy. The for-loop is well optimized if the timing result is similar.
On the other hand, you may want to try a much better matrix library Eigen, with which you can implement your operation with the code similar to this
x = x.array() * (A * x).array();

GSL eigenvalues order

I am using the functions gsl_eigen_nonsymm and/or gsl_eigen_symm from the GSL library to find the eigenvalues of an L x L matrix M[i][j] which is also a function of time t = 1,....,N so i have M[i][j][t] to get the eigenvalues for every t i allocate an L x L matrix E[i][j] = M[i][j][t] and diagonalize it for every t.
The problem is that the program gives the eigenvalues at different order after some iteration. For example (L = 3) if at t = 0 i get eigen[t = 0] = {l1,l2,l3}(0) at t = 1 i may get eigen[t = 1] = {l3,l2,l1}(1) while i need to always have {l1,l2,l3}(t)
To be more concrete: consider the matrix M (t) ) = {{0,t,t},{t,0,2t},{t,t,0}} the eigenvalues will always be (approximatevly) l1 = -1.3 t , l2 = -t , l3 = 2.3 t When i tried to diagonalize it (with the code below) i got several times a swap in the result of the eigenvalues. Is there a way to prevent it? I can't just sort them by magnitude i need them to be always in the same order (whatever it is) a priori. (the code below is just an example to elucidate my problem)
EDIT: I can't just sort them because a priori I don't know their value nor if they reliably have a structure like l1<l2<l3 at every time due to statistical fluctuations, that's why i wanted to know if there is a way to make the algorithm behave always in the same way so that the order of the eigenvalues is always the same or if there is some trick to make it happen.
Just to be clearer I'll try to re-describe the toy problem I presented here. We have a matrix that depends on time, I, maybe naively, expected to just get lambda_1(t).....lambda_N(t), instead what I see is that the algorithm often swaps the eigenvalues at different times, so if at t = 1 I've got ( lambda_1,lambda_2,lambda_3 )(1) at time t = 2 (lambda_2,lambda_1,lambda_3)(2) so if for instance I wanted to see how lambda_1 evolves in time I can't because the algorithm mixes the eigenvalues at different times. The program below is just an analytical-toy example of my problem: The eigenvalues of the matrix below are l1 = -1.3 t , l2 = -t , l3 = 2.3 t but the program may give me as an output(-1.3,-1,2.3)(1), (-2,-2.6,4.6)(2), etc As previously stated, I was wondering then if there is a way to make the program order the eigenvalues always in the same way despite of their actual numerical value, so that i always get the (l1,l2,l3) combination. I hope it is more clear now, please tell me if it is not.
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <gsl/gsl_linalg.h>
#include <gsl/gsl_eigen.h>
#include <gsl/gsl_sort_vector.h>
main() {
int L = 3, i, j, t;
int N = 10;
double M[L][L][N];
gsl_matrix *E = gsl_matrix_alloc(L, L);
gsl_vector_complex *eigen = gsl_vector_complex_alloc(L);
gsl_eigen_nonsymm_workspace * w = gsl_eigen_nonsymm_alloc(L);
for(t = 1; t <= N; t++) {
M[0][0][t-1] = 0;
M[0][1][t-1] = t;
M[0][2][t-1] = t;
M[1][0][t-1] = t;
M[1][1][t-1] = 0;
M[1][2][t-1] = 2.0 * t;
M[2][1][t-1] = t;
M[2][0][t-1] = t;
M[2][2][t-1] = 0;
for(i = 0; i < L; i++) {
for(j = 0; j < L; j++) {
gsl_matrix_set(E, i, j, M[i][j][t - 1]);
}
}
gsl_eigen_nonsymm(E, eigen, w); /*diagonalize E which is M at t fixed*/
printf("#%d\n\n", t);
for(i = 0; i < L; i++) {
printf("%d\t%lf\n", i, GSL_REAL(gsl_vector_complex_get(eigen, i)))
}
printf("\n");
}
}
Your question makes zero sense. Eigenvalues do not have any inherent order to them. It sounds to me like you want to define eigenvalues of M_t something akin to L_1(M_t),..., L_n(M_t) and then track how they change in time. Assuming your process driving M_t is continuous, then so will your eigenvalues be. In other words they will not significantly change when you make small changes to M_t. So if you define an ordering by enforcing L_1 < L_2... < L_n, then this ordering will not change for small changes in t. When you have two eigenvalues cross, you'll need to make a decision about how to assign changes. If you have "random fluctuations" which are larger than the typical distance between your eigenvalues, then this becomes essentially impossible.
Here's another way of tracking eigenvectors, which might prove better. To do this, suppose that your eigenvectors are v_i, with components v_ij . What you do is first "normalize" your eigenvectors such that v_i1 is nonnegative, i.e. just flip the sign of each eigenvector appropriately. This will define an ordering on your eigenvalues through an ordering on v_i1, the first component of each eigenvector. This way you can still keep track of eigenvalues that cross over each other. However if your eigenvectors cross on the first component, you're in trouble.
I don't think you can do what you want. As t changes the output changes.
My original answer mentioned ordering on the pointers but looking at the data structure it won't help. When the eigenvalues have been computed the values are stored in E. You can see them as follows.
gsl_eigen_nonsymm(E, eigen, w);
double *mdata = (double*)E->data;
printf("mdata[%i] \t%lf\n", 0, mdata[0]);
printf("mdata[%i] \t%lf\n", 4, mdata[4]);
printf("mdata[%i] \t%lf\n", 8, mdata[8]);
The following code is how the the data in the eigenvector is layed out...
double *data = (double*)eigen->data;
for(i = 0; i < L; i++) {
printf("%d n \t%zu\n", i, eigen->size);
printf("%d \t%lf\n", i, GSL_REAL(gsl_vector_complex_get(eigen, i)));
printf("%d r \t%lf\n", i, data[0]);
printf("%d i \t%lf\n", i, data[1]);
printf("%d r \t%lf\n", i, data[2]);
printf("%d i \t%lf\n", i, data[3]);
printf("%d r \t%lf\n", i, data[4]);
printf("%d i \t%lf\n", i, data[5]);
}
If, and you can check this when you see the order change, the order of the data in mdata changes AND the order in data changes then the algorithm does not have a fixed order ie you cannot do what you're asking it to do. If the order does not change in mdata and it changes in data then you have a solution but I really doubt that will be the case.
According to the docs, those functions return unordered:
https://www.gnu.org/software/gsl/manual/html_node/Real-Symmetric-Matrices.html
This function computes the eigenvalues of the real symmetric matrix A. Additional workspace of the appropriate size must be provided in w. The diagonal and lower triangular part of A are destroyed during the computation, but the strict upper triangular part is not referenced. The eigenvalues are stored in the vector eval and are unordered.
Even the functions that return ordered results, do so by simple ascending/descending magnitude:
https://www.gnu.org/software/gsl/manual/html_node/Sorting-Eigenvalues-and-Eigenvectors.html
This function simultaneously sorts the eigenvalues stored in the vector eval and the corresponding real eigenvectors stored in the columns of the matrix evec into ascending or descending order according to the value of the parameter sort_type as shown above.
If you're looking for the time evolution of the eigenvalues, just do like you have been doing and solve for the time-dependent representations, e.g.:
lambda_1(t).....lambda_N(t)
For your simple time-as-scalar example,
l1 = -1.3 t , l2 = -t , l3 = 2.3 t
You literally have a parameterization of all possible solutions and because you've assigned them identifiers ln you don't run into the issue of degeneracy. Even if any M[i][j] are nonlinear functions of t, it shouldn't matter because the system itself is linear and solutions are computed purely by the characteristic equation (which will hold t constant while solving for lambda).

Roberts-Slaney-Bouras integer FFT implementation woes

I've implemented a signal processing FFT algorithm in Python using np.fft (too easy). Now I'm working on doing this in C using an integer algorithm. After some research, I found that one of the most popular integer FFT libraries in C on the internet is the one by Roberts, Slaney, and Bouras which can be found in many places including the second entry here.fft,
There are a few concepts I don't understand and am hoping for guidance.
Specifically, the example script included in the library linked above separates the input signal into two bins, real and imaginary, by putting all of the even indexes of the signal in the first half and the odd indexes in the second half of the signal.
for (i=0; i<N; i++){
x[i] = AMPLITUDE*cos(i*FREQUENCY*(2*3.1415926535)/N);
if (i & 0x01) // only odd index
fx[(N+i)>>1] = x[i]; // N+i >> 1 is len(input)+i/2
else // only even index
fx[i>>1] = x[i];
}
fix_fftr(fx, log2N, 0);
The signal array has not changed length but now contains two of almost the same signal. Then FFT driver function (fix_fftr) takes the entire input signal as an argument and does the exact same thing
if (inverse)
scale = fix_fft(fr, fi, m-1, inverse);
for (int i=1; i<n; i+=2) {
tt = f[n+i-1]; // even index
f[n+i-1] = f[i]; // odd index into the second half
f[i] = tt; // even index into the first half
}
if (!inverse)
scale = fix_fft(fr, fi, m-1, inverse);
return scale;
whats the reason for this?
The first part is computing the twiddle factors, which are constants for a given length FFT and independent of the data.
The second part appears to be part of the data shuffling based on recursive bit-reversed addressing, which is a component within an in-place FFT.

Anyone having an idea how to implement a polyphase filter using c programming

Does anyone knows how to implement a polyphase filter using C programming. Polyphase filter is a combination of filtering plus down sampling. It uses an Fir filter for filtering and then down sample it with the given decimation factor.
I understand with filtering it's the convolution of the input signal with the impulse response, so you just have to call the convolution function.
In simple terms the code would be like this.
for ( i = 0; i < sampleCount; i++ )
{
y[i] = 0; // set to zero before sum
for ( j = 0; j < kernelCount; j++ )
{
y[i] += x[i - j] * h[j]; // convolve: multiply and accumulate
}
}
The question I have is how should I convert this basic algorithm to a polyphase filter, because it has one more step as down sampling. I do not understanding what should I do 1st, Filtering and then down sampling? or is there a way to combine both and do it?
You have code to calculate each point. For any point i, that code uses data in x[i-j] and h[j] (looping over values of j), and it writes the result to y[i].
For downsampling, you do not want all the points y[i]; you only want some of them. So write code that, given an i, uses data in the same x[i-j] and h[j] to calculate the output value. But, instead of doing that for every value of i, do it only for multiples of the decimation factor (say it is named d), and write the result to y[i/d] instead of y[i].

Solve a banded matrix system of equations

I need to solve a 2D Poisson equation, that is, a system of equations in the for AX=B where A is an n-by-n matrix and B is a n-by-1 vector. Being A a discretization matrix for the 2D Poisson problem, I know that only 5 diagonals will be not null. Lapack doesn't provide functions to solve this particular problem, but it has functions for solving banded matrix system of equations, namely DGBTRF (for LU factorization) and DGBTRS. Now, the 5 diagonals are: the main diagonal, the first diagonals above and below the main and two diagonals above and below by m diagonals wrt the main diagonal. After reading the lapack documentation about band storage, I learned that I have to create a (3*m+1)-by-n matrix to store A in band storage format, let's call this matrix AB. Now the questions:
1) what is the difference between dgbtrs and dgbtrs_? Intel MKL provides both but I can't understand why
2) dgbtrf requires the band storage matrix to be an array. Should I linearize AB by rows or by columns?
3) is this the correct way to call the two functions?
int n, m;
double *AB;
/*... fill n, m, AB, with appropriate numbers */
int *pivots;
int nrows = 3 * m + 1, info, rhs = 1;
dgbtrf_(&n, &n, &m, &m, AB, &nrows, pivots, &info);
char trans = 'N';
dgbtrs_(&trans, &n, &m, &m, &rhs, AB, &nrows, pivots, B, &n, &info);
It also provides DGBTRS and DGBTRS_. Those are fortran administrativa that you should not care about. Just call dgbtrs (reason is that on some architectures, fortran routine names have underscore appended, on other not, and names may be either upper or lower case -- Intel MKL #defines the right one to dgbtrs).
LAPACK routines expects column major matrices (ie. Fortran style): store columns one after the others. The banded storage you must use is not hard : http://www.netlib.org/lapack/lug/node124.html.
It seems good to me, but please try it on small problems beforehand (always a good idea by the way). Also make sure you handle non-zero info (this is the way errors are reported).
Better style is to use MKL_INT instead of plain int, this is a typedef to the right type (may be different on some architectures).
Also make sure to allocate memory for pivots before calling dgbtrf.
This might be off topic. But for Poisson equation, FFT based solution is much faster. Just do 2D FFT of your potential field, divided by -(k^2+lambda^2) then do IFFT. lambda is a small number to avoid divergence for k=0. The 5-diagonal equation is a band-limited approximation of the Poisson equation, which approximate the differential operator by finite difference.
http://en.wikipedia.org/wiki/Screened_Poisson_equation

Resources