I have a NoSQL setup as follows:
UserId : ContentId : OrderId
User1 : Content1 : 0
User1 : Content2 : 1
User2 : Content3 : 0
User2 : Content4 : 1
User2 : Content5 : 2
User2 : Content6 : 3
User2 : Content7 : 4
I get the list of User2 items sorted by order which
SELECT * FROM table WHERE UserId = 'User2' SORT BY OrderId DESC
which results in
UserId : ContentId : OrderId
User2 : Content3 : 0
User2 : Content4 : 1
User2 : Content5 : 2
User2 : Content6 : 3
User2 : Content7 : 4
Great! Now I want to swap so that the table looks like this:
UserId : ContentId : OrderId
User2 : Content3 : 0
User2 : Content6 : 3
User2 : Content4 : 1
User2 : Content5 : 2
User2 : Content7 : 4
So I move Content6 to after Content3 and before Content4. The drawback now is that to update the OrderId I have to update every row after Content3 resulting in multiple writes to the datastore.
What is a better way of doing this in a NoSQL database?
You can solve this with a more sophisticated algorithm, you can create a big gap between the keys and then you could move item from one place to inbetween other keys.
After a while some spaces may run out so the algorithm in this case will have to normalize the table and even the gaps between the keys, resulting in a one time procedure which will be a bit heavier on the database. this can be done periodically or on demand when you detect that you are running/ran out of space for example.
So the original table would look like:
Before
UserId : ContentId : OrderId
User2 : Content3 : 0
User2 : Content4 : 1000
User2 : Content5 : 2000
User2 : Content6 : 3000
User2 : Content7 : 4000
After
UserId : ContentId : OrderId
User2 : Content3 : 0
User2 : Content6 : 500
User2 : Content4 : 1000
User2 : Content5 : 2000
User2 : Content7 : 4000
There is no problem in massive updating in a good NoSQL solution, as under the hood an update looks like an append to a file which is commonly called Write Ahead Log. For example, if you issue 1000000 updates and each of them is, say, 32 bytes in size, then it will result in just writing 32MB to a file which can be done even on magnetic disks in less than 1 second. Moreover if all of those updates are in one transaction then this should be exactly one write/writev syscall with a large buffer.
Related
How to modify my object to order to show first a specific id my object looks like this
{ 1 : [{ id : 1 , name : 'toto'}],
9: [{ id : 9 , name : 'toto'}]
15 : [{ id : 15 , name : 'toto'}]
}
I want to show first 9 how can I do this
I'm working on a web application for a farmer, a permacultor to be more precise. It implies that he's handling various seeds and plants. The purpose of the application is to store in database data regarding his garden. Once enough data are gathered, the application is to provide a frame to analyze data.
For now I'm developping the first fonctionality : store data in database and display them on webpages.
Let's focus on the main topic of this question.
The garden has several fields. A field can contain several plants. Plants can have several state throught time (seed, plant, flower, harvestable).
When a plant reach a state, we need to store specific information :
the state
the date when state was observed (and other time-related data)
the quantity (from seed state to flower state, we can have loss for many reasons)
So far so good, nothing fancy.
NOW a plant can grow on a field until a specitifc state then be planted into another field until its harvesting.
For instance, 12 carrots that are growing in tray n°3 from the seed to germination state.
At germination state, 2 carrots didn't make it. The farmer now intend to resume the growing of his carrots not in tray n°3 but in field n°1
In model, let's say "state_plant_table" you have 2 entries :
carrots - 12 - seeds - tray n°3
carrots - 10 - germ - field n°1
You might see it coming.
Let's say now that... there isn't enough room in field n°1 for the 10 carrots, only 8 can fit. So he just put the 2 left in the field aside - field n°2.
We now have
carrots - 12 - seeds - tray n°3
carrots - 8 - germ - field n°1
carrots - 2 - germ - field n°2
NOW, on display we would show HTML table for each fields, trays or w/e. When you click on a field you have the detail of every plants rooted in it.
For field n°1 we would have :
carrots - 8
For field n°2 we would have :
carrots - 2
And, unfortunately, for tray n°3, we would have :
carrots - 12
But we should have 0 (if 0 => exclude from display of course).
I'm struggling with the theorical design of my process right now... any tips, hints, suggestions are welcome !
I have thought about a "parent" quantity and a "child" quantity where the initial quantity would be store in "plant_table" as "parent" quantity and "children" quantity would be stored in "state_plant_table" - the quantity is more linked to a state in which it's being observed than the plant itself.
I feel like this is the right way, but I don't manage to push the reasoning to its end either.
Reasoning with "parent" and "children" was one of the correct approach.
There are actually 3 natures of quantity to store :
quantity of plants observed at a certain state (quantite_etat)
quantity of plants actually in the field (quantite)
quantity of plants from the same parent (quantite_lot)
models.py
class EtatLot(models.Model):
id = models.AutoField(primary_key = True)
id_lot = models.ForeignKey('Lot', on_delete = models.PROTECT, verbose_name = 'Lot', related_name = 'lot_parent', null = True, blank = True)
etat = models.CharField(verbose_name = 'État', max_length = 50, null = True, blank = True)
quantite = models.PositiveSmallIntegerField(verbose_name = 'Quantité', null = True, blank = True)
quantite_etat = models.PositiveSmallIntegerField(verbose_name = 'Quantité relatée', null = True, blank = True)
class Lot(models.Model):
id = models.AutoField(primary_key = True)
quantite_lot = models.PositiveSmallIntegerField(verbose_name = 'Quantité', null = True, blank = True)
The quantity displayed is the one from quantite. The quantity used to analyze data is the one from quantite_etat.
Example
Let's say we have 12 cauliflower on field n°1 and the farmer want to plant 10 of them on field n°2.
Within database we have :
Lot table
id : 1 quantite_lot : 12
EtatLot table
id : 1 id_lot : 1 etat : Seed quantite : 12 quantite_etat : 12
id : 2 id_lot : 1 etat : Germ quantite : 12 quantite_etat : 12
At the end of operations, we should have this :
Lot table
id : 1 quantite_lot : 2
EtatLot table
id : 1 id_lot : 1 etat : Seed quantite : 12 quantite_etat : 12 field : fieldn°1
id : 2 id_lot : 1 etat : Germ quantite : 2 quantite_etat : 12 field : fieldn°1
id : 3 id_lot : 1 etat : Plan quantite : 10 quantite_etat : 10 field : fieldn°2
For this operation, quantite_lot is irrelevant. However I store it in order to do some stock check : you cannot plant more plants that you have.
This is how I achieved to the table above :
get quantity from the last child of lot_parent (quantite)
update this value with the difference between it and the value added by the farmer in the form, update the value from quantite_lot in the parent as well
store the value of the form in quantite and quantite_etat of the entry that is about to be added in the database
I have solr document which is like below.
agentId : 100
emailDeliveredDate : 2018-02-08,
emailSentDate : 2018-02-07
agentId : 100
emailSentDate : 2018-02-06
agentId : 101
emailDeliveredDate : 2018-02-08,
emailSentDate : 2018-02-07
I need a result like below.
agentId : 100
emailDeliveredDate : 1,
emailSentDate : 2
agentId : 101
emailDeliveredDate : 1,
emailSentDate : 1
In mysql it will be :
select count(emailDeliveredDate),count(emailSentDate) group by agentId;
I need help in solr for this.
I did not get any way in Solr which can help me. So I used facet with pivot which gave me half results. Rest half calculation I did in Java.
I have a problem with a filter condition in BO.
Imagine that I have this database
ID | DESC
0 | None
1 | Company
2 | All
In BO I have a filter that ask where do you want to find the objects and 2 options:
"Company" or "All".
If I choose "All" then I should have all the datas with the "ID" 0,1,2 and if I choose "Company" only the data with the "ID" 1.
So I did something like this:
TABLE_NAME.ID <= (CASE WHEN #Prompt('where do you want to find the objects','A',{'Company', 'All'},mono,constrained,not_persistent,{'Company'}) = 'Company' THEN 1 ELSE 2 END)
This filter is OK when I choose "All" because I have all the "ID" smaller than 2, i.e, 0,1,2.
But It does not work when my option is company, because it also shows the data with the "ID" 0.
I should have some with "=" combined with "<="
If it's really only that simple, the following will work:
TABLE_NAME.ID =
(CASE #Prompt('where do you want to find the objects',
'A',
{'Company', 'All'},
mono,
constrained,
not_persistent,{'Company'}
)
WHEN 'Company'
THEN 1
WHEN 'All'
THEN TABLE_NAME.ID
END)
I want to design like os file system,
with specific display order (sequence) can be update.
I want file and folder can be same layer,
file doesn't have to inside a folder.
But in below design, if the file not in any folder I don't know how to save the sequence, save in where??
Any suggestion will be apperciate
data example
folder(id:1) top layer: sequence: 0
file(id:1) sequence_in_folder: 0
file(id:2) sequence_in_folder: 1
folder(id:2) top layer: sequence: 1
file(id:3) sequence_in_folder: 0
file(id:4) top layer: sequence: 2 << **sequence save in which table ??**
file(id:5) top layer: sequence: 3 << **sequence save in which table ??**
folder
id sequence parent_folder_id
1 0
2 1
file
id sequence_in_folder folder_id
1 0 1
2 1 1
3 0 2
4 ?????
5 ????
schema
CREATE TABLE IF NOT EXISTS "folder"(
"id" SERIAL NOT NULL,
"sequence" integer NOT NULL,
"parent_folder_id" integer Default NULL,
PRIMARY KEY ("id")
);
CREATE TABLE IF NOT EXISTS "file"(
"id" SERIAL NOT NULL,
"sequence_in_folder" integer Default NULL,
"parent_folder_id" integer NOT NULL,
PRIMARY KEY ("id")
);
UPDATE
base on #Laurenz Albe answer, no need change table design,
just create a root folder.
but how to sorting data order by a field cross/exist in two table?
the sequence exist in folder table and file table, how to sort them together
query
SELECT * FROM folder fo
LEFT JOIN file fi ON fi.parent_folder_id = fo.id
WHERE fo.parent_folder_id = $1 AND fi.parent_folder_id = $1
ORDER BY fo.sequence fi.sequence ?? ;
[1]
data example
folder
id | sequence | parent_folder_id | name
1 | 0 | | root
2 | 0 | 1 |
3 | 2 | 1 |
file
id | sequence | parent_folder_id |
1 | 1 | 1 |
output
folder(id:1, sequence:0 name:root)
folder(id:2, sequence:0)
file(id:1, sequence:1)
folder(id:3 sequence:2)
Two suggestions:
Introduce an “anonymous” top folder that contains all the top level elements.
Rename the sequence column of bookmerk_folder to max_sequence or so to avoid confusion with bookmark.sequence.
Supplemental to Laurenz's answer:
unify your bookmark and folder columns, maybe bookmark_node and require that everything have a parent which is not a bookmark. Something like
CREATE TABLE IF NOT EXISTS fsnode(
"id" SERIAL NOT NULL,
"name" text,
"is_folder" bool,
"parent_is_folder" bool not null,
"sequence" integer NOT NULL,
"parent_folder_id" integer Default NULL,
CHECK (parent_is_folder),
PRIMARY KEY ("id"),
UNIQUE(id, is_folder), # needed for fkey below
FOREIGN KEY (parent_folder_id, parent_is_folder) REFERENCES fsnode (id, is_folder)
);