I want to create application with consists of Desktop application and google cloud storage. So, each my client should have separate cloud storage. Does google provide such thing?
More info.
Because I do not know what can offer google app engine i wrote this question.
I need some database hosting for my desktop application. In future I think I will switch to GWT and app engine. I want to sell my application so each my client can't access my other client databases. I was thinking that would be safer if each client will have data in a separate database so I can't do some mistakes in code.
You can separate data in the datastore using namespaces on google app engine:
https://developers.google.com/appengine/docs/java/multitenancy/multitenancy
It's up to you to decide how to implement the namespaces. You can separate them out by your user authentication system.
You can create a folder per client and restrict the folder access to the user (works only with Google Accounts) or your can do the same with buckets, create a bucket per user (which might be an overhead if you have a lot of users).
For database AppEngine datastore has the ability to separate the data by namespaces. this doen't require any user account and its your responsibility to select with which namespace to work with per request.
You can use GAE namespace capability as pointed above by #dragonx without Google authentication.
Use a client name as a namespace identifier (needs to be unique) . How you fetch this client name is upto you. It can be stored in GAE itself if you wish or can be deciphered from the url used specific to a client.
Do have a look at the GAE multitenancy link https://developers.google.com/appengine/docs/java/multitenancy/multitenancy
The example here can be easily adapted to use any string identifier per client.
Related
I have spent 3 days researching this problem and cannot find a solution or similar use case that shows how to solve the problem, so any pointers would be greatly appreciated.
I am creating a web-app that uses Google Cloud Storage and Bigquery. A user registers on the web app and then can upload data to Cloud Storage and Big Query. Two users could be from the same company and therefore should be able to view the same data - i.e. Jack and Jill work for company A and if Jack uploads a massive dataset via this app, Jill should also be able to view it later.
Another scenario will be I have two completely separate clients with users using this web-app. If users from Company A upload data, users from Company B should not be able to view Company A's data, and vice versa. But users from the same company should be able to view the data within their company.
Currently, I have an app that works for a single company. This has a React front-end that uses Firebase for authentication. Once the user is logged in, they can use the app which sends off API calls to a Flask back-end that does some error checking and authentication checking and then fires off an API call to GCP. This uses a service account and the key is loaded as an environment variable in the environment in which the Flask app is running.
However, if Company B want to use the app now, both Company A and Company B will be able to see each other's data and visualize it through the app. In addition, they will be sharing a project (I would like to change this to allocate billing more easily to have each client have their own project).
I ultimately want to get this app onto Kubernetes and ensure that each company is independent of each other, however, do not want to have to have separate URL's for every company using the app. Also, I want to abstract GCP away from the client. I would prefer to authenticate a user based on their login credentials and then they will be given access to their GCP project (via my front-end) accordingly.
I thought about perhaps having separate service keys for each client and then storing the service key info in Firebase, while using the respective keys for API calls but not sure this is best practice. It is however the only strategy I can think of.
If anyone could provide some help or guidance it would be very much appreciated. This is my first GCP project and have not been able to find any answers on GCP, SO, Google Groups, Slack or Medium.
Thanks,
TJ
First if all, welcome on GCP! It's an awesome platform, very powerful and flexible. But not magic.
Indeed, the use case that you describe is specific to your business logic. GCP provides told for securing access for user and VM(through service account) but not for customer. Here you have to implement your own custom and authorisation logic, with a database (I don't recommend bigquery for website, the latency is too high) to list three users, the companies where they work, the blobs of each company...
Nothing is magic and your use case specific.
If you want to discuss more about which component to use and to start, no problem. Let a comment.
I want to access datastore (and storage) data of an AppEngine project via google-cloud-datastore and google-cloud-storage with an Python program on my own server.
This works with my AppEngine staging server, by creating a service account and giving it owner access (to the project).
Doing the same thing with the production AppEngine instance fails with
google.api_core.exceptions.PermissionDenied: 403 Missing or insufficient permissions.
Part of the problem might be, that I might be using the wrong project to create the service account with. There are more than one project with the same name in my cloud console. How do I identify the correct one?
How do I get more details about the problem?
First, note that the Datastore and the Cloud Storage are 2 different products with 2 different accessing methods.
The Datastore is closely tied to the GAE project - each project has its own datastore. The external access procedure in general is captured in How do I use Google datastore for my web app which is NOT hosted in google app engine?.
When switching the project (staging to production in your case) there are 2 things to keep in mind:
as you observed, you need to change the project you're accessing.
you also need to change the credentials you load and use for access to match the project you select, as each project has it own service account key configured in the above-mentioned procedure
For the google-cloud-datastore library both of these are simultaneously configured via the datastore.Client() call parameters (emphasis mine):
class google.cloud.datastore.client.Client(project=None,
namespace=None, credentials=None, _http=None, _use_grpc=None)
project (str) – (Optional) The project to pass to proxied API methods.
credentials (Credentials) – (Optional) The OAuth2 Credentials to use for this client. If not passed (and if no _http object is passed),
falls back to the default inferred from the environment.
The Cloud Storage is completely independent from GAE, the GAE project/credentials you use (if any) have no bearing on bucket/object access restrictions whatsoever. There's nothing you need to do from the google-cloud-storage library perspective when switching from one GAE project to another
To eliminate the confusion created by multiple projects having the same name just go to the IAM & admin Settings page, select the respective projects from the drop-down list on the top blue bar and rename them using meaningful names (click in the Project name box to edit the name, then click SAVE). Then re-check if you're using the right keys for the desired project.
I am building an iPhone app that stores user logon credentials in an AWS DynamoDB. In another DynamoDB I am storing locations of files (stored in S3) for that user. What I don't understand is how to make this secure. If I use a Token Vending Machine that gives that application an ID with access to the user DynamoDB, isn't it possible that any user could access the entire DB and just add or delete any information that they desire? They would also be able to access the entire S3 bucket using this setup. Any recommendations on how I could set this up securely and properly?
I am new to user DB management, and any links to helpful resources would be much appreciated.
Regarding S3 and permissions, you may find the answer on the following question useful:
Temporary Credentials Using AWS IAM
IAM permissions are more finegrained than you think. You can allow/disallow specific API calls, so for example you might only allow read operations. You can also allow access to a specific resource only. On S3 this means that you can limit access to a specific file or folder , but dynamodb policies can only be set at the table level.
Personally I wouldn't allow users direct access to dynamodb - I'd have a webservice mediating access to that, although users being able to upload directly to s3 or download straight from s3 is a good thing (Your web service can in general give out pre signed urls for that though)
I am interested in developing a site similar to youtube. I want to have a site that users upload videos.
I imagine technically the website would upload the video to the azure cloud. Where the file will automatically be encoded to silverlight and hosted.
Can azure host my site, take care of encoding and host the videos all programmatically?
And can azure host the rest of the website pages that are not part of the app like a (homepage or about us page) and have a domain name or do i need a web host?
thanks
Azure can do the lot.
You'll probably want to use Azure Blob Storage for the initial upload, then use queues and the worker role functionality to do the encoding and other processing. Then you can store the resulting file back in Blob storage, and have an index either in Azure Tables or SQL Azure, depending on the architecture of the rest of the application.
And yes, an Azure Web role can quite happily host static content, standard dynamic ASPX pages, and a whole lot more (and can do it all on your own domain).
I suggest you grab the Windows Azure SDK (from http://www.microsoft.com/windowsazure/) and take a look through the documentation. Your example scenario is pretty simple actually, and working through the samples should give you all the information you need.
Good luck!
Azure can host your site indeed. However don't forget that the costs will probably be a minimum ~ $80-90 per month even without any load. If your website gets traffic this amount will increase
However you will have to implement video encoding yourself (or better yet find libraries to do it), Azure is purely a host.
Is that somehow possible to access one datastore? Or access one app from different domains.
App Engine recently add support for a feature called modules (Docs: go, python, java)
App Engine Modules (or just "Modules" hereafter) is a feature that lets developers factor large applications into logical components that can share stateful services and communicate in a secure fashion. An app that handles customer requests might include separate modules to handle other tasks:
API requests from mobile devices
Internal, admin-like requests
Backend processing such as billing pipelines and data analysis
When you create a new module, App Engine will create a url that corresponds to the module name. If you only have one module then the name will be default. e.g.
http://default.myapp.appspot.com
http://mobile‑frontend.myapp.appspot.com
http://my-module.myapp.appspot.com
Using Domain masking, you could then direct from:
www.myapp.com => http://default.myapp.appspot.com
www.myapp-mobile.com => http://mobile‑frontend.myapp.appspot.com
www.example.com => http://my-module.myapp.appspot.com
Every app has its own datastore and memcache (shared among all versions of that app).
It seems not possible to share datastores between applications right now (unless you provide some web service for that), but that would be a nice feature to have, so maybe you should file a feature request with Google vote for it.
As for domains, you can associate your app with domains managed by Google Apps. Multiple domains for the same application should be no problem (except for SSL certificates).
Every version of an app is backed by the same datastore. If you want to limit access for individual requests, you'll need to add a field to your model to enforce that restriction. There are low level hooks in the datastore API for this sort of thing, if you want to go that far.
And yes, you can add a single App Engine app to multiple domains - even in multiple Apps accounts.
Kyle's solution would work, but App Engine was never designed to be used in this way. So if you architect your app(s) to rely on this kind of setup and Google clamps down for whatever reason then you'd be screwed.
You can have multitenancy using the Namespace Java API