TF Lite C API crashes on second iteration - c

Question

I am trying to use the C API of TF Lite to execute the inference cyclically, e.g. I set up the interpreter and feed him inputs every second or so to get predictions.
For this purpose, I have built libtensorflowlite_c.so via bazel. As documented here, I try to do the inference like this, but inside a for loop to simulate the cyclic execution:
#include "tensorflow/lite/c/c_api.h"
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char* argv[]) {
for(int i = 0; i < 3; i++)
{
printf("Iteration: %d\n", i);
float input[49] = { 0.0 };
TfLiteModel* model = TfLiteModelCreateFromFile("model.tflite");
TfLiteInterpreterOptions* options = TfLiteInterpreterOptionsCreate();
TfLiteInterpreterOptionsSetNumThreads(options, 2);
TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model, options);
TfLiteInterpreterAllocateTensors(interpreter);
TfLiteTensor* input_tensor = TfLiteInterpreterGetInputTensor(interpreter, 0);
TfLiteTensorCopyFromBuffer(input_tensor, input, 49 * sizeof(float));
TfLiteInterpreterInvoke(interpreter);
const TfLiteTensor* output_tensor = TfLiteInterpreterGetOutputTensor(interpreter, 14);
float output[49];
TfLiteTensorCopyToBuffer(output_tensor, output, 49 * sizeof(float));
printf("Output: \n\n");
for (int j = 0; j < 49; j++) {
printf("%d: %f\n", j, output[j]);
}
TfLiteInterpreterDelete(interpreter);
TfLiteInterpreterOptionsDelete(options);
TfLiteModelDelete(model);
}
return 0;
}
The first iteration runs fine and returns something. But on the second iteration, I get a SegFault when calling TfLiteTensorCopyToBuffer(output_tensor, output, 49 * sizeof(float));. Reason for this is that the previous function TfLiteInterpreterGetOutputTensor returns a nullpointer.
I expected to run this multiple times without any problems, as I destroy all old instances of variables at the end of the for-loop and thus start a fresh interpreter everytime. Obviously, this is not the case.
Can somebody provide any guidance on this? Also, I know that I probably do not have to create an interpreter on every iteration, but I wanted to make sure that everything is created new when I start over again.
EDIT:
I tried rewriting the code to exclude unnecessary parts from the actual loop:
#include "tensorflow/lite/c/c_api.h"
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char* argv[]) {
float input[49] = {0.0};
float output[49] = {[0 ... 48] = 2.5};
TfLiteModel* model = TfLiteModelCreateFromFile("VariationalAutoencoder_440.tflite");
TfLiteInterpreterOptions* options = TfLiteInterpreterOptionsCreate();
TfLiteInterpreterOptionsSetNumThreads(options, 2);
TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model, options);
TfLiteInterpreterAllocateTensors(interpreter);
TfLiteTensor* input_tensor = TfLiteInterpreterGetInputTensor(interpreter, 0);
const TfLiteTensor* output_tensor = TfLiteInterpreterGetOutputTensor(interpreter, 14);
for(int i = 0; i < 3; i++)
{
printf("\nIteration: %d\n", i);
TfLiteTensorCopyFromBuffer(input_tensor, input, 49 * sizeof(float));
TfLiteInterpreterInvoke(interpreter);
TfLiteTensorCopyToBuffer(output_tensor, output, 49 * sizeof(float));
printf("Output: \n");
for (int j = 0; j < 49; j++)
{
printf("%02d: %f\n", j, output[j]);
}
}
TfLiteInterpreterDelete(interpreter);
TfLiteInterpreterOptionsDelete(options);
TfLiteModelDelete(model);
return 0;
}

Answer 1

Remove all variable declarations outside and prior to the for loop, eg:
int main (int argc, char* argv[]) {
float input[49] = { 0.0 };
float output[49] = {0.0};//also needs to be initialized
//and others...
for(int i = 0; i < 3; i++)
{
printf("Iteration: %d\n", i);
....
Do the same for any calls that are creating re-usable objects, or allocating memory. Re-declaring re-usable objects in a loop (without freeing them before re-declare) can have similar results to calling malloc in a loop rather than using realloc for subsequent calls.
Your code snippet shows that you have created and deleted the following inside the loop:
TfLiteInterpreterDelete(interpreter);
TfLiteInterpreterOptionsDelete(options);
TfLiteModelDelete(model);
Calling this in a loop may also be problematic.
TfLiteTensor* input_tensor = TfLiteInterpreterGetInputTensor(interpreter, 0);
input_tensor, I believe should be created once, then, in the loop, resized as needed.
From the link you provide:
// NOTE: After a resize, the client *must* explicitly allocate tensors before
// attempting to access the resized tensor data or invoke the interpreter.
// REQUIRES: 0 <= input_index < TfLiteInterpreterGetInputTensorCount(tensor)
TFL_CAPI_EXPORT extern TfLiteStatus TfLiteInterpreterResizeInputTensor(
TfLiteInterpreter* interpreter, int32_t input_index, const int* input_dims,
int32_t input_dims_size);
Edit: One other item that seems odd:
const TfLiteTensor* output_tensor = TfLiteInterpreterGetOutputTensor(interpreter, 14);
The modifier const seems an odd bedfellow to output_tensor. It would seem if this variable will change inside the loop, then it should not be modified to const.

Answer 2

Your code is running well if TfLiteInterpreterGetOutputTensor use index below TfLiteInterpreterGetOutputTensorCount.
Maybe the tensor index 14 should be 13, but this depends on your model.
Adding some check like :
int count = TfLiteInterpreterGetOutputTensorCount(interpreter);
printf("output tensor count:%d\n", count);
if (count > 14) {
const TfLiteTensor* output_tensor = TfLiteInterpreterGetOutputTensor(interpreter, 14);
float output[49];
TfLiteTensorCopyToBuffer(output_tensor, output, 49 * sizeof(float));
printf("Output: \n\n");
for (int j = 0; j < 49; j++) {
printf("%d: %f\n", j, output[j]);
}
}