Store array in SQLite that is referenced in another table - arrays

Question

I'm trying to store arrays (or vectors) of data in a SQLite database but I'm having problems trying to find a decent way to do so. I found some other post on StackOverflow, that I can't seem to find anymore, which mentioned storing the data in a table like the following:
CREATE TABLE array_of_points
(
id integer NOT NULL,
position integer NOT NULL,
x integer NOT NULL,
y integer NOT NULL,
PRIMARY KEY (id, position)
);
So to store all the data for a single array you would insert each item under the same ID and just increment the position. So for example to insert an array with three values it would be something like:
INSERT INTO array_of_points VALUES (0, 0, 1, 1);
INSERT INTO array_of_points VALUES (0, 1, 2, 2);
INSERT INTO array_of_points VALUES (0, 2, 3, 3);
And then to retrieve the values you would select everything with the same ID and order by the position:
SELECT x,y FROM array_of_points WHERE id = 0 ORDER BY position;
This is all great and works wonderfully, but I'm now running into a problem where I don't know how to reference an array in a different table. For example I want to do something like the following:
CREATE TABLE foo
(
id integer NOT NULL,
array_id integer NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY (array_id) REFERENCES array_of_points (id)
);
This will create the table just fine but once you try to execute a query on it the foreign key constraint throws an error since it must reference both the id and position of the array_of_points table since they are part of a composite primary key.
The only solution I currently have is to just remove the foreign key from the foo table, but that is not a good solution since it means it can now hold any value even if it doesn't actually map to an array in the array_of_points table.
Is there any way to work around this problem? Or maybe there's some other way to store the data so that this is possible?
Just as an FYI, please do not suggest I store the data in some sort of comma/semi-colon/whatever delimited list because that is an even worse option that I am not going to consider. It is also not possible to do with some of the more complex objects that are going to be stored in the database.

Answer 1

There is one special case that this schema cannot handle: it is not possible to store an array of size zero.
This might not be a concern in practice, but it shows that the database is not fully normalized.
A foreign key always references a single parent record.
Therefore, what is missing is a table that has a single record for each array.
Implementing this would result in a schema like this:
CREATE TABLE array
(
id integer PRIMARY KEY
-- no other properties
);
CREATE TABLE array_points
(
array_id integer REFERENCES array(id),
position integer,
x, y, [...],
PRIMARY KEY (array_id, position)
) WITHOUT ROWID; -- see http://www.sqlite.org/withoutrowid.html
CREATE TABLE foo
(
[...],
array_id integer REFERENCES array(id)
);
The additional table requires more effort to manage, but now you have the ability to generate array IDs through autoincrementing.