I have a deeply nested structure for a Grades App I for which I want to use Firestore as my db. The nested structure is like this:
--Grades
|
--Semesters
|
--Subjects
|
--Grade
Each of the types consists of metadata like name and id, and a list of the child type. For Example Semester has the fields: String name, String id, List<Subjects>
The idea is that the user can create subject and grades.
Each datatype has its own screen which shows the metadata and the list of its children. Therefore it should be possible to efficiently get the datatype and its children.
What would be the most nosql/firestore way to store this nested structure?
Related
Can hive array contain custom java objects? i.e. Lets say that there is a custom POJO class called Car.java, can the hive table be defined as follows?
Create table X (arr: Array < Car >)
I know there is a bit of serialization and de-serialization to be done to make this happen, but just wanted to check.
The link just says data_type but not clear on the possible data type.
I have a specific situation to which I haven't found a solution yet.
I have several databases with the same structure where I have a table, (lets say Users), which has known columns such as: UserID, UserName, UserMail, etc...
In the same table, I have dynamic custom columns which I can know only on runtime, such as: customField54, customField75, customField82, etc...
I have a screen where I must show a list of users, and there are thousands of records (Must show ALL the users - no question about it).
The Users table columns in database A look like this:
| UserID | UserName | UserMail | customField54 | customField55 |
and for the example, lets say I have another database B, and the table Users there looks like this:
| UserID | UserName | UserMail | customField109 | customField211 | customField235 | customField302 |
I have a single code which each time connects to another database. So I have a single code - > multiple databases, while the difference in each database is the custom fields of the Users table.
If I work with a DataTable, I can query:
SELECT * FROM Users
And then, dynamically I can retrieve the custom fields values, like this:
Dim customFieldsIDs() As Integer = GetCustomFieldsIDs()
Dim dt As DataTable = GetUserListData() // All users data in a DataTable
For Each dr In dt.Rows
Response.Write(dr.Item("UserID"))
Response.Write(dr.Item("UserName"))
Response.Write(dr.Item("UserMail"))
For Each cfID in customFieldsIDs
Response.Write(dr.Item("customField" & cfID))
Next
Next
My intention is not to work with DataTables. I want to work with strong typed objects. I cannot create a POCO of Users with the customFields as is inside, because for each database the Users table has different customFields columns, so I can't create an object with strongly typed variables.
Then, I decided to create a class Users with the known columns inside, and also a dictionary holding the customFields.
In VB.NET, I created a class Users, which looks as follows:
Public Class User
Public Property UserID As Integer
Public Property UserName As Integer
Public Property UserMail As Integer
Public Property customFieldsDictionary As Dictionary(Of Integer, String)
End Class
The class has the static values: UserID, UserName, etc...
Also, it has a dictionary of the customFieldIDs and their values, so I can retrieve the values in a single action (in O(1) complexity)
I use MicroORM PetaPoco\NPoco to populate the values.
The ORM allows me to fetch the Users data without me having to iterate the data by myself, by calling:
Dim userList As List(Of User) = db.Fetch(Of User)("SELECT * FROM Users")
But then the customFields dictionary is not populated.
In order to populate I have to iterate the userList and for each user retrieve the customFields data.
This is a very expensive way to fetch the data and results in a very bad performance.
I'd like to know if there is a way to fetch the data into the User class using PetaPoco\NPoco with a single command and manage to populate the known values and the custom fields dictionary for every user without having to iterate through the whole collection.
I hope it is understood. It is really difficult for me to explain and a very difficult issue to find a solution to.
You could try fetching everything into a dictionary and then you could map specific keys/values to your User object properties.
EDIT:
I'm not using VB.NET anymore, but I'll try to explain.
Create the indexer similar to this one:
http://www.java2s.com/Tutorial/VB/0120__Class-Module/DefineIndexerforyourownclass.htm
In the indexer you would do something like:
if (index == "FirsName") then
me.FirstName = value
end if
....
if (index.startWith("customField") then
var indexValue = int.Parse(index.Replace("customField",""))
me.customFieldsDictionary[indexValue] = value
end if
NPoco supports materializing the data into dictionary:
var users = db.Fetch<Dictionary<string, object>>("select * from users");
You should be able to pass your class to the NPoco and force it to use the Fetch overload for dictionary.
I've used this approach before, but I can't find the source at the moment.
I think nested data loading in ExtJS 4 is impossible in real world. For example, I have some models for trainings, for users, and some model which describes how trainings assigned to users, which user assigned training, etc. I think it's wrong to get join in PHP Model (I use ZF 1.x) to get users first names and last names instead their identifiers, and it's wrong to merge data from two selects (select join with traings and trainings-to-users table and select from users where identifier in (list of identifiers from join above)) in Controller. Good practise to do it in View, isn't it?
But ExtJS 4 nested data loading tells me: make hierarchial structure (with a lot of duplicates of nested data!) and pass it me. Why I need to do this work? I'll better merge data in Controller!
My final question is: if I have JSON like this:
{
trainings: [{"id":"1", "user_id":278,"assigner_id":30, ...}, {...}],
users: ["278": {"firstname":"Guy", "lastname":"Fawkes"}, "300":{....}, ...]
}
Can I to create another nested data loading from this data?
Thanks.
Yes you can ... sort of. One way to do it is to define separate stores for users and trainings and then use loadData(JSON.users) to load data that was prefetched.
My structure
cat:id:name -> name of category
cat:id:subcats -> set of subcategories
cat:list -> list of category ids
The following gives me a list of cat ids:
lrange cat:list 0, -1
Do I have to iterate each id from the above command to get the name field in my script? Because that seems inefficient. How can I get a list of category names from redis?
There are a couple different approaches. You may want to have the values in the list be delimited/encoded strings that contain both the id, the name, and any other value you need quick access to. I recommend JSON for interoperability and efficient string length, but there are other formats which are more performant.
Another option is to, like you said, iterate. You can make this more efficient by getting all your keys in a single request and then using MGET, pipelining, or MULTI/EXEC to fetch all the names in a single, efficient, operation.
How would you model these relationships in a db?
You have a Page entity that can contain PageElements.
A PageElement can for instance be an Article, or a Picture. An Article table obviously has other members / columns than a Picture. An article could have ie. "Title", "Lead", "Body" columns that are all of type nvarchar, while a Picture might have something like "AltText", "Path", "Width", "Height". I like this to be extensible, who knows what PageElements I might need in 3 months? So I guess I'd need a PageElementTypes table.
For the relationships, what about tables like these:
Pages with an Id, and other mumbo jumbo. (Create Date, Visible, what not)
Pages_PageElements with PageId and PageElementId.
PageElements with an Id and a PageElementTypeId and more mumbojumbo (SortOrder, Visibility etc.).
PageElementTypes with an Id and a Name (for instance "Article", "Picture", "AddressBlock")
Now, should I create a PageElementId column in every Articles, Pictures, AddressBlocks table to finish things up? That's where I'm a bit stuck, it's a simple 1:1 relationship so this should work, but somehow I might miss something.
Follow up:
The recommended solutions below with separate attributes would force me to store all attributes as the same type, or not? What If one PageElement has attributes that are nvarchar(255) and some are nvarchar(1000), what if some are integers?
If I got the EAV way I would have to create tons of tables for holding the attribute values for all the different data types out there.
The two common choices are Single Table Inheritance and Multi Table Inheritance. Other approaches include having tables for each concrete class which I've never used, and what I'd call a meta-table implementation, where the attribute definitions are moved into data rather than any sort of schema.
I've had generally good experiences with STI, and provided you don't expect a plethora of classes and attributes it's the simplest solution. Simple is very good in my book.
Unless new page element types need to be created by users at runtime, I'd avoid the meta-tables approach and anything that begins to look like it. In my experience such code quickly becomes a quagmire and rarely delivers much value compared to a more concrete implementation updated at regular intervals by developers.
Just as you have configured Page Elements, you need to configure the Attributes associated with the Page Elements.
So we have two items that are extensible Page Elements & their Attributes.
I sugges the following tables:
Page : Page ID | ...
Page Elements : Page Element ID | Element Type ID | Page ID | ...
Page Element Type : Element Type ID | Page Element Type Label
Page Element Attribute Type : Attribute Type ID | Element Type ID | Attribute Label
Page Element Attributes : Page Element ID | Attribute Type ID | Attribute Value
The Page Element Attribute Type table will contain the list of attributes associated with an element. Example :
Atttibute Type ID 1 | Article | "Title"
Atttibute Type ID 2 | Article | "Lead"
Atttibute Type ID 3 | Picture | "AltText"
The Page Element Attributes table will store the actual value for the attributes assciated with a page element. Example :
Page Element ID 1 | Attribute Type ID 1 | "Everybody Loves Raymond"
Page Element ID 2 | Attribute Type ID 3 | "World Map"
The universal solution would be:
PageElementType: ID, Name, [Mumbo Jumbo]
PageElementTypeParameter: ID, PageElementTypeID, [Mumbo Jumbo]
Page: ID, [Mumbo Jumbo]
PageElement: ID, PageElementTypeID, [Mumbo Jumbo]
PageElementParameters: ID, PageElementID, PageElementTypeParameterID, Value, [Mumbo Jumbo]
In human words: There is a table for page element types, and an associated table, which lists possible parameters for each page element (like SRC and ALT for an image; TEXT for an article, etc).
Then there is a table with all the pages; an associated table which lists elements in each page; and a table which lists parameter values for each element.
I use a different naming convention then you but this is essentially what I would do:
PageElementType(PageElementTypeID, PageElementTypeName)
PageElement(PageElementID, PageElementTypeID)
Article(ArticleID, PageElementID, ...)
Picture(PictureID, PageElementID, ...)
Page(PageID, ...)
PageHasPageElement(PageHasPageElementID, PageID, PageElementID) => {PageID, PageElementID} are unique
This what I do and seems to be fairly well normalized and performs fine.
I guess I'll just go with what I got, EAV is no option for me. What I got now is a somewhat hybrid approach.