I am working out the structure for a JSON database for an app like onlyFans. Basically, someone can create a club, then inside of that club, there are sections where the creator's posts are shown and another where the club members posts are shown. There is however a filter option where both can be seen.
In order to make option 1 below work, I need to be able to filter based on if isFromCreator=true and at the same time based on timstamp. How can I do this?
Here are the 2 I have written down:
ClubContent
CreatorID
clubID
postID: {isFromCreator: Bool}
OR
creatorPosts
postID: {}
MemeberPosts
postID: {}
Something like the below would be what I want:
ref.child("Content").child("jhTFin5npXeOv2fdwHBrTxTdWIi2").child("1622325513718")
.queryOrdered(byChild: "timestamp")
.queryLimited(toLast: 10)
.queryEqual(toValue: true, childKey: "isFromCreator")
I triedqueryEqual yet it did not return any of the values I know exist with the configuration I specified.
You can use additional resource locations within rules by referencing the parent/child directories specifically and comparing the val() of the respective node structure.
for example:
".write": "data.parent().child('postID').child('isFromCreator').val()"
Just be aware that Security Rules do not filter or process the data in the request, only allow or deny the requested operation.
You can read more about this from the relevant documentation:
https://firebase.google.com/docs/database/security/rules-conditions#referencing_data_in_other_paths
https://firebase.google.com/docs/database/security/core-syntax#rules-not-filters
Related
For ref, using "#apollo/client": "^3.5.5",
I've defined my typePolicies like so as suggested in docs:
HistoricalData: {
keyFields: ["variable", "workspace"],
fields:{...}
}
and when my cache is built, I am expecting my cacheId to be like
<__typename>:<id>:<id>
HistoricalData:${props.variable}:${props.workspace}`;
but instead, when I look in the Apollo cache, it's been created using the keyField names and the values in an object, such as
HistoricalData:{"variable":"GAS.TOTAL","workspace":"ABC"}
instead of
HistoricalData:GAS.TOTAL:ABC
so when I try to readFragment it returns null
client.readFragment({
id: `HistoricalData:${props.variable}:${props.workspace}`,
fragment: apolloGQL`fragment MyHistorical on Historical {
variable
workspace
}`})
It does actually return a value from the cache if I create my id in the structure that exists in the cache and readFragment using this.
Has anyone else noticed that Apollo client is not creating the cache id's in the structure that they describe in the docs?
After some research I came upon the correct way to handle this case. I know that you have already moved on, but just in case anyone else has the same problem in the future, here goes:
As described in the documentation for customizing the cache ID, the cache ID will be an stringified object, as you pointed out. It's not quite explicit in the documentation, but at this point in time it provides this nested example for a cache ID:
Book:{"title":"Fahrenheit 451","author":{"name":"Ray Bradbury"}}
But as users we don't have to preoccupy us with the format of this ID, because there's a helper for that, called cache.identify.
For your specific case, you could use something like this:
const identifiedId = cache.identify({
__typename: 'HistoricalData',
variable: 'GAS.TOTAL',
workspace: 'ABC'
});
cache.readFragment({
id: identifiedId,
fragment: apolloGQL`fragment MyHistorical on Historical {
variable
workspace
}`
});
I working on an application that has its own database and gets user information from another serivce (an LDAP is this case, through an API package).
Say I have a tables called Articles, with a column user_id. There is no Users table, instead a user or set of users is retrieved through the external API:
$user = LDAPConnector::getUser($user_id);
$users = LDAPConnector::getUsers([1, 2, 5, 6]);
Of course I want retrieving data from inside a controller to be as simple as possible, ideally still with something like:
$articles = $this->Articles->find()->contain('Users');
foreach ($articles as $article) {
echo $article->user->getFullname();
}
I'm not sure how to approach this.
Where should I place the code in the table object to allow integration with the external API?
And as a bonus question: How to minimise the number of LDAP queries when filling the Entities?
i.e. it seems to be a lot faster by first retrieving the relevant users with a single ->getUsers() and placing them later, even though iterating over the articles and using multiple ->getUser() might be simpler.
The most simple solution would be to use a result formatter to fetch and inject the external data.
The more sophisticated solution would a custom association, and a custom association loader, but given how database-centric associations are, you'd probably also have to come up with a table and possibly a query implementation that handles your LDAP datasource. While it would be rather simple to move this into a custom association, containing the association will look up a matching table, cause the schema to be inspected, etc.
So I'll stick with providing an example for the first option. A result formatter would be pretty simple, something like this:
$this->Articles
->find()
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
$userIds = array_unique($results->extract('user_id')->toArray());
$users = LDAPConnector::getUsers($userIds);
$usersMap = collection($users)->indexBy('id')->toArray();
return $results
->map(function ($article) use ($usersMap) {
if (isset($usersMap[$article['user_id']])) {
$article['user'] = $usersMap[$article['user_id']];
}
return $article;
});
});
The example makes the assumption that the data returned from LDAPConnector::getUsers() is a collection of associative arrays, with an id key that matches the user id. You'd have to adapt this accordingly, depending on what exactly LDAPConnector::getUsers() returns.
That aside, the example should be rather self-explanatory, first obtain a unique list of users IDs found in the queried articles, obtain the LDAP users using those IDs, then inject the users into the articles.
If you wanted to have entities in your results, then create entities from the user data, for example like this:
$userData = $usersMap[$article['user_id']];
$article['user'] = new \App\Model\Entity\User($userData);
For better reusability, put the formatter in a custom finder. In your ArticlesTable class:
public function findWithUsers(\Cake\ORM\Query $query, array $options)
{
return $query->formatResults(/* ... */);
}
Then you can just do $this->Articles->find('withUsers'), just as simple as containing.
See also
Cookbook > Database Access & ORM > Query Builder > Adding Calculated Fields
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Custom Finder Methods
I am trying to pull all the documents in the collection 'users', but it only pulls 'fred' and 'lisa', and ignores all the italicized documents:
For this data:
Trying to get all documents:
Will yield:
info: length 2
info: fred => { gender: 'male', contacts: [ '' ] }
lisa => { contacts: [ '' ] }
According to the Firebase documentation (Firebase: Add and Manage Data):
Warning: Even though non-existent ancestor documents appear in the console, they do not appear in queries and snapshots. You must create the document to include it in query results.
Note: The non-existent ancestor users seem to be auto-created when the user hits the sign-up button that triggers a firebase.auth() function (fred and lisa were created manually).
How would I print the contacts of each user if some of the users do not show up in my queries? Would I need to periodically run a script that manually re-adds all the users or is there a more elegant solution?
As you have mentioned, these "documents" are displayed with an italic font in the Firebase console: this is because these documents are only present (in the console) as "container" of one or more sub-collection but they are not "genuine" documents.
As matter of fact, if you create a document directly under a col1 collection with the full path doc1/subCol1/subDoc1, no intermediate documents will be created (i.e. no doc1 document).
The Firebase console shows this kind of "container" (or "placeholder") in italics in order to "materialize" the hierarchy and allow you to navigate to the subDoc1 document but doc1 document doesn't exist in the Firestore database.
Let's take an example: Imagine a doc1 document under the col1 collection
col1/doc1/
and another one subDoc1 under the subCol1 (sub-)collection
col1/doc1/subCol1/subDoc1
Actually, from a technical perspective, they are not at all relating to each other. They just share a part of their path but nothing else. One side effect of this is that if you delete a document, its sub-collection(s) still exist.
So, if you want to be able to query for these parent documents, you will have to create them yourself, as jackz314 mentioned in the comments.
If you're trying to list all your registered users from Firebase auth, you can use the Firebase SDK function:
function listAllUsers(nextPageToken) {
admin.auth().listUsers(1000, nextPageToken)
.then(function(listUsersResult){
listUsersResult.users.forEach(function(userRecord) {
console.log('user', userRecord.toJSON());
})
if (listUsersResult.pageToken) {
// list next batch of users
}
})
.catch(function(err) {
console.log('Error listing users: ', error)
});
}
listAllUsers();
via http://firebase.google.com/docs/auth/admin/manage-users#list_all_users
Ok Im starting out fresh with Firebase. I've read this: https://www.firebase.com/docs/data-structure.html and I've read this: https://www.firebase.com/blog/2013-04-12-denormalizing-is-normal.html
So I'm suitably confused as one seems to contradict the other. You can structure your data hierarchically, but if you want it to be scalable then don't. However that's not the actual problem.
I have the following structure (please correct me if this is wrong) for a blog engine:
"authors" : {
"-JHvwkE8jHuhevZYrj3O" : {
"userUid" : "simplelogin:7",
"email" : "myemail#domain.com"
}
},
"posts" : {
"-JHvwkJ3ZOZAnTenIQFy" : {
"state" : "draft",
"body" : "This is my first post",
"title" : "My first blog",
"authorId" : "-JHvwkE8jHuhevZYrj3O"
}
}
A list of authors and a list of posts. First of all I want to get the Author where the userUid equals my current user's uid. Then I want to get the posts where the authorId is the one provided to the query.
But I have no idea how to do this. Any help would be appreciated! I'm using AngularFire if that makes a difference.
Firebase is a NoSQL data store. It's a JSON hierarchy and does not have SQL queries in the traditional sense (these aren't really compatible with lightning-fast real-time ops; they tend to be slow and expensive). There are plans for some map reduce style functionality (merged views and tools to assist with this) but your primary weapon at present is proper data structure.
First of all, let's tackle the tree hierarchy vs denormalized data. Here's a few things you should denormalize:
lists you want to be able to iterate quickly (a list of user names without having to download every message that user ever wrote or all the other meta info about a user)
large data sets that you view portions of, such as a list of rooms/groups a user belongs to (you should be able to fetch the list of rooms for a given user without downloading all groups/rooms in the system, so put the index one place, the master room data somewhere else)
anything with more than 1,000 records (keep it lean for speed)
children under a path that contain 1..n (i.e. possibly infinite) records (example chat messages from the chat room meta data, that way you can fetch info about the chat room without grabbing all messages)
Here's a few things it may not make sense to denormalize:
data you always fetch en toto and never iterate (if you always use .child(...).on('value', ...) to fetch some record and you display everything in that record, never referring to the parent list, there's no reason to optimize for iterability)
lists shorter than a hundred or so records that you always as a whole (e.g. the list of groups a user belongs to might always be fetched with that user and would average 5-10 items; probably no reason to keep it split apart)
Fetching the author is as simple as just adding the id to the URL:
var userId = 123;
new Firebase('https://INSTANCE.firebaseio.com/users/'+userId);
To fetch a list of posts belonging to a certain user, either maintain an index of that users' posts:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
fb.child('/my_posts/'+userId).on('child_added', function(indexSnap) {
fb.child('posts/'+indexSnap.name()).once('value', function(dataSnap) {
console.log('fetched post', indexSnap.name(), dataSnap.val());
});
});
A tool like Firebase.util can assist with normalizing data that has been split for storage until Firebase's views and advanced querying utils are released:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
var ref = Firebase.util.intersection( fb.child('my_posts/'+userId), fb.child('posts') );
ref.on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val();
});
Or simply store the posts by user id (depending on your use case for how that data is fetched later):
/posts/$user_id/$post_id/...
new Firebase('https://INSTANCE.firebaseio.com/posts/'+userId).on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val());
});
I'm using app engine's bulkloader to import a CSV file into my datastore. I've got a number of columns that I want to merge into one, for example they're all URLs, but not all of them are supplied and there is a superseding order, eg:
url_main
url_temp
url_test
I want to say: "Ok, if url_main exists, use that, otherwise user url_test and then use url_temp"
Is it, therefore, possible to create a custom import transform that references columns and merges them into one based on conditions?
Ok, so after reading https://developers.google.com/appengine/docs/python/tools/uploadingdata#Configuring_the_Bulk_Loader I learnt about import_transform and that this can use custom functions.
With that in mind, this pointed me the right way:
... a two-argument function with the keyword argument bulkload_state,
which on return contains useful information about the entity:
bulkload_state.current_entity, which is the current entity being
processed; bulkload_state.current_dictionary, the current export
dictionary ...
So, I created a function that handled two variables, one would be the value of the current entity and the second would be the bulkload_state that allowed me to fetch the current row, like so:
def check_url(value, bulkload_state):
row = bulkload_state.current_dictionary
fields = [ 'Final URL', 'URL', 'Temporary URL' ]
for field in fields:
if field in row:
return row[ field ]
return None
All this does is grab the current row (bulkload_state.current_dictionary) and then checks which URL fields exist, otherwise it just returns None.
In my bulkloader.yaml I call this function simply by setting:
- property: business_url
external_name: URL
import_transform: bulkloader_helper.check_url
Note: the external_name doesn't matter, as long as it exists as I'm not actually using it, I'm making use of multiple columns.
Simples!