I've been looking at how to create multiple Firestore instances in Firebase, I need different Db's for prod, staging and development. I read the documentation and seems that I only need to modify the "Google-services.json" file in the applications. what I don't get is what should I modify and how that would look in the console.
Am I going to see different instances in the same project?
I need to create different projects for every environment and modify those values in the file?
If I need to test something that requires testing in all environments and all of them require Blaze to run that test do I have
to pay Triple?
Thanks in advance
Firebase doesn't support the use of multiple Firestore instances in a single project.
The Firebase team strongly recommends creating different projects for each of your environments to keep them separate. This probably means you will have to build different apps that each point to different instances, or you will have to somehow configure your app at runtime to select which project you want to work with.
There is no obligation to add billing to any of the projects you don't want to pay for, as long as you accept the limitations of the Spark plan.
Yes Firebase doesn't support multiple instance in a single project. However my case, i created 3 different document in Firestore root and setup each environment to refer these documents accordingly. Each document will have the same collections, sub collections and documents.
For example,
dev -> users -> details
-> others_collection -> details
stag -> users
-> others_collection -> details
prod -> users
-> others_collection -> details
On the client side, each environment will get the collection like this :
db.collection(`${env}/users`)
I am doing this on small project and seem pretty happy with what Firestore provided. In single project. i can create many ios/android apps according to the environment and each environment have it own document and collections in Firestore.
The safest way is to create a new google cloud project. You can export your current firestore database to your new project if needed.
Step by step:
Backup your current database by exporting all your documents into a
bucket:
https://firebase.google.com/docs/firestore/manage-data/export-import
gcloud beta firestore export gs://[BUCKET_NAME]
Create a new project -> https://cloud.google.com/resource-manager/docs/creating-managing-projects
In the dashboard, select Firestore into your new project and create an empty database
In the dashboard, go to your BACKUP_BUCKET and add your new project service account in the permission tab
Switch project to your new project
gcloud config set project 'new-project'
Then export your data
gcloud beta firestore import gs://[BUCKET_NAME]/[EXPORT_PREFIX]/ The EXPORT_PREFIX being the name of the folder created in your bucket.
I use this method to have clearly separated 'preprod' and 'production' environments (the team can access and synchronize with production without having access to the production environment codebase and configuration).
Related
I'm building a web app using React with a Firebase backend.
I want to be able to deploy my app on different environments (staging, prod). To do this, I have added my app to 2 different Firebase projects, one for each environment.
What I want to be able to do is to deploy slightly different cloud functions depending on which environment I am deploying to. I want to automate all this for CI/CD.
Is there a way to specify different functions/ folders for each Firebase project, or is my only option to have a script that moves the correct cloud functions code to the functions/ folder on deploy? Similarly, how can I have different Firestore rules for each environment?
Thanks for the help.
Your option, as clarified in the official documentation here, is to add the following lines into your firebase.json file to specify different paths.
"functions": {
"source": "another-folder"
}
Since the firebase.json is individual for each project, with this lines, you should be able to set a different folder for each one of them, so they reference different files for your functions.
Regarding Firestore rules, due to the fact that they are in separate projects, you can access the Firestore Console and configure the security rules there. Each project will have their only set of rules for you to configure. You can get more information on how to structure them in this article here.
Let me know if the information helped you!
I deploy my Docker image to default GAE by gcloud app deploy --image-url=us.gcr.io
I have successfully mapped my custom domain to this application with custom runtime and flex env.
My dispatch.yaml sends requests to its sub-domain:
dispatch:
- url: "dev.domain.com/*"
service: default
Now I want to use different images from Container Registry for test.domain.com and domain.com
While having all these images sharing same Cloud Storage and Firebase credentials.
Being new to GCP I wanted to learn any simple approach to organize such basic structure without going into services and versions (just by assigning proper images to serve relevant domains).
Is it even possible to do within one GAE project or I should create separate projects for it?
Mapping custom domains can only be done at the service level, so if you don't want to go into services separate projects is your only choice.
Actually using separate projects instead of services (or service versions) for implementing different environments has some advantages, I'd choose separate projects, too. See Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
I'm not sure if sharing the storage and credentials between production and other environments is a good idea (what if something goes wrong?). I'd keep them separate, too (maybe with some jobs to populate non-production projects with production data, if you need to). But if you do want to share them across projects you'll probably need to make some extra steps.
I want to access datastore (and storage) data of an AppEngine project via google-cloud-datastore and google-cloud-storage with an Python program on my own server.
This works with my AppEngine staging server, by creating a service account and giving it owner access (to the project).
Doing the same thing with the production AppEngine instance fails with
google.api_core.exceptions.PermissionDenied: 403 Missing or insufficient permissions.
Part of the problem might be, that I might be using the wrong project to create the service account with. There are more than one project with the same name in my cloud console. How do I identify the correct one?
How do I get more details about the problem?
First, note that the Datastore and the Cloud Storage are 2 different products with 2 different accessing methods.
The Datastore is closely tied to the GAE project - each project has its own datastore. The external access procedure in general is captured in How do I use Google datastore for my web app which is NOT hosted in google app engine?.
When switching the project (staging to production in your case) there are 2 things to keep in mind:
as you observed, you need to change the project you're accessing.
you also need to change the credentials you load and use for access to match the project you select, as each project has it own service account key configured in the above-mentioned procedure
For the google-cloud-datastore library both of these are simultaneously configured via the datastore.Client() call parameters (emphasis mine):
class google.cloud.datastore.client.Client(project=None,
namespace=None, credentials=None, _http=None, _use_grpc=None)
project (str) – (Optional) The project to pass to proxied API methods.
credentials (Credentials) – (Optional) The OAuth2 Credentials to use for this client. If not passed (and if no _http object is passed),
falls back to the default inferred from the environment.
The Cloud Storage is completely independent from GAE, the GAE project/credentials you use (if any) have no bearing on bucket/object access restrictions whatsoever. There's nothing you need to do from the google-cloud-storage library perspective when switching from one GAE project to another
To eliminate the confusion created by multiple projects having the same name just go to the IAM & admin Settings page, select the respective projects from the drop-down list on the top blue bar and rename them using meaningful names (click in the Project name box to edit the name, then click SAVE). Then re-check if you're using the right keys for the desired project.
I am very, very new to Google App Engine, but I need to administer a WebApp2 (so, Python) -based website that uses it. Specifically I need to filter a queryset and delete certain model instances.
Building the site locally is fine. Locally I can get at the models by hitting
http://localhost:8000/console
in my browser. This pulls up the "interactive console", and in there I can put some code like
from application.models.user import User
and it will work. From there I can create / delete User objects in the Datastore, so this local interactive console seems to have ORM-like functionality. (I'm coming from a Django background, so that's what it reminds me of).
However I don't seem to have this option on the live website. I was hoping to find it at
https://console.cloud.google.com/home/dashboard?project=<my project>
And I thought I had found it when I found the control to "activate google cloud shell". This brings up a command line on the web server, but the website's codebase doesn't seem to exist here, so launching the Python cli and trying to import the applications models like I can do locally doesn't work. And even if I could, I doubt it would be a Django-like ORM the way it is on my local build.
Am I thinking about this in the wrong way? How do you create / filter / delete / etc. Datastore objects via the backend in a GAE / WebApp2 website? Is the "Google cloud shell" even part of the answer?
To get access to your app code in the cloud shell one option would be to create in the shell a local copy of your app repository. For an example (addressing a different question, true) you can peek at Google Cloud: How to deploy mirrored Repository. But I'm not 100% certain if that will automatically give you access to your app's datastore. Worth a try IMHO.
Another option is to hook the desired operations as handler actions inside your app itself and execute them from there. Eventually hidden/protected/restricted via authentication, for example for admin users only. This is what I use for one-time datastore migrations that I need from time to time when I make changes to my entity models.
Finally, but not really a programatic access - you could use the Datastore page in the developer console to manually find, read and modify your entities, see Managing Datastore from the Console.
Use the remote_api_shell. You run this locally where your application code lives. Then you can import your models, and perform the same queries etc, and add modify/delete entities. I use this frequently for a range of tasks. Updating more than 100K records can become slow using this method.
https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi
I'm starting a new project using Firebase as my database to store the data of my application. Let's say, I finish my app and it's deployed in the app stores, so real users are using my app and they are entering new data to my Firebase database, what happens If I want to add a new functionality to my app, obviusly I cannot use the database that are using my real users, so I create a new Firebase project called myapp-dev, I import the data from my production database and I work in there, I added new data and new inserts/updates that will modify the structure of my production database, is there a way that I can combine the 2 databases dev into prod to not generate conflicts???
The Firebase team recommends using two different projects to isolate different working environments. It's not advised to combine them into a single project.