Calling Convention error - C - c

Question

In the following code there is a calling convention error(possibly leading to an eternal loop), and i cannot detect it. I try to verify the code using 'Satabs'. What kind of model can bring the error to the surface. With the following model i get a segfault.
By changing the VLEN and TMAX you can play a bit.
Q1. What is the calling convention error?
Q2. What kind of model would be most appropriate to use for finding the error?
#include <stdio.h>
#if MODEL==1
#define VLEN 3
#define TMAX 4
int trans(int T,int*src,int*dst){
if (T < VLEN && T < TMAX && src[T] < 4){
dst[T]=src[T]+1;
return 1;
} else {
return 0;
}
}
#endif
struct next_state {
int next;
int src[VLEN];
};
typedef struct next_state *iterator_t;
void init(iterator_t iter,int *src){
for(int i=0;i<VLEN;i++){
iter->src[i]=src[i];
}
iter->next=0;
}
int next(iterator_t iter,int *dst){
#ifdef FIX_ARRAY
for(int i=0;i<VLEN;i++){
#else
for(int i=0;i<TMAX;i++){
#endif
dst[i]=iter->src[i];
}
int res=0;
while(!res&&iter->next<TMAX){
res=trans(iter->next,iter->src,dst);
iter->next++;
}
return res;
}
int find_depth(iterator_t iter,int *src){
int table[VLEN*TMAX];
int N=0;
init(iter,src);
for(int i=0;i<TMAX;i++){
if(next(iter,&(table[N*VLEN]))){
N++;
}
}
int depth=0;
for(int i=0; i<N;i++){
printf("Eimai stin for \n");
int tmp=find_depth(iter,&(table[i*VLEN]));
printf("tmp= %d\n",tmp);
if(tmp>=depth){
depth=tmp+1;
//assert(depth);
}
}
printf("\n\n");
return depth;
}
int main(int argc,char*argv[]){
int state[VLEN];
struct next_state ns;
for(int i=0;i<VLEN;i++){
state[i]=0;
}
int depth=find_depth(&ns,state);
printf("depth is %d\n",depth);
}

Answer 1

int depth=find_depth(&ns,state);
You are passing &ns, but taking arg in function as iterator_t iter, is this correct ?
void init(iterator_t iter,int *src){
for(int i=0;i<VLEN;i++){
iter->src[i]=src[i];
iter->src[i] is this expression fine?

Answer 2

I dont know 'Satabs' but the most promising candidate for an endless loop for me seems to be
while(!res&&iter->next<TMAX){
res=trans(iter->next,iter->src,dst);
iter->next++;
}
All other loops rather look like fix count.
This loop might be dangerous for itself even without the so called calling convention error, which doest jump to my eye yet.
Anyhow you should take a look not only to the call of the funtion trans but the whole call tree to it.
You could also try to paste your code there
http://gimpel-online.com//cgi-bin/genPage.py?srcFile=intro.txt&cgiScript=analyseCode.py&title=Introduction+and+Welcome&intro=Introducing+the+testing+facility&compilerOption=online32.lnt&in
Maybe you get a few more hints.
Just a guess:
Maybe 'Satabs' doesn't like undefined preprocessor conditions
like
#if MODEL==1