I am work on a web application as an interface with Google Cloud Storage(GCS).
I am using a backend service to retrieve the list of files I stored on GCS and their URL with the JSON API and return that to my web application. However, I was not really able to load the files through those URL, which always came back with 403 forbidden.
I am not sure how GCS authentication work behind the scene and whether it is possible to directly grant access to web application. I am not sure how could I attach application authentication information via http request. What I know is I can do that via the backend service but for the reason of simplicity I wonder if it is possible to get around with that. One of the thing I tried is adding the web application domain(which will be sent via referrer in http request) into ACL to that bucket, which doesn't work at all.
And thanks to what #Brandon pointed out below. I am ok to grant anyone whoever have access to the application to view the content of the GCS since it is an internal app and I have already checked their authentication when I first serve the web application.
====
Solution
I ended up using the signedUrl that expire in 5 minutes and I highly recommend interact with gcs using gcloud (Their python document is really good). Thanks again for the thorough answer!
You have a user on a web browser who wants to download an object that only your application's service account has read access for. You have a few options:
Expand access: make these object publicly readable. Probably not the best choice if this info is sensitive, but if it's not, this is the easiest solution.
Give your app's credentials to the user so that they can authenticate as your app. This is a REALLY bad idea, and I probably shouldn't even list it here.
When a user wants to download a file, have them ask your app for it, and then have your app fetch the file and stream its contents to the user. This is the easiest solution for the client-side code, but it makes your app responsible for streaming file contents, which isn't really great.
When a user wants to download a file, have them ask your app for permission, and reply to them with some sort of token they can use to fetch the data directly from GCS.
#4 is what you want. Your users will ask your app for a file, your app will decide whether they are allowed to access that file via whatever you're doing (passwords? IP checks? Cookies? Whatever.) Then, your app will respond with a URL the user can use to fetch the file directly from GCS.
This URL is called a "signed URL." Your app uses its own private key to add a signature to a URL that indicates which object may be downloaded by the bearer and for long the URL is valid. The procedure for signing URLs is somewhat tricky, but fortunately the gcloud storage libraries have helper functions that can generate them.
Related
I'm currently building a react project, which is embeds small, static applications that are uploaded to an S3 bucket.
Those applications all are built using HTML/CSS/Vanilla JS, meaning they all have the same structure with an index.html file as an entry point.
Embedding those applications on the site through an iframe with the source link to the index.html works well, but I now want to make sure only users, who are registered and have the correct access rights have access to a given application.
Originally I wanted to handle this using pre-signed urls, but this doesn't seem to work since I couldn't find a way to use a pre-signed url to get access to all the files in a folder in S3.
I then thought about handling everything in React/Express, by making sure the user is authenticated, has the correct role and only then sending the src link back to the Frontend, where it gets embedded in the iframe. Additional I would add a bucket policy that only allows my specific domain to fetch the resources.
Apparently (from other threads) I saw that it's easy to spoof the HTTP referrer, meaning that if somebody gets the access link to the application on S3 they could simply send an HTTP request with a spoofed referrer and get their hands on the content.
I'm in over my head here and trying to figure out what the best architecture is. If it's something completely removed from the setup I currently have I'm happy to change it all around.
Generally though I would hope for something that just gives me an added layer of security, that makes it impossible to access the content in the S3 bucket unless it's coming directly from one specific host after authenticating there.
I've been googling the entire afternoon and I'm still not able to figure out what's the best solution to implement the following:
We have build a webapp in AngularJS that consumes interacts with REST API build using Symfony. The app allows users to register, login and do stuff. Now, these users need to upload very big files (>60GB) into their personal folders. A separate VM have been setup for this purpose (data server), located in the same VLAN as the frontend, backend and the MySQL db serving the data. The data upload will be done using either HTTP (using JQuery File Upload plugin) or an FTP client.
I'd like the users to authenticate into the data server (both via FTP or HTTP) using the credentials they already have for the app. For the FTP case, I'll use PureFTP as FTP server, which validates user/pass directly from the MySQL. As far as I know, this is the most convenient solution, but criticism is accepted.
For the HTTP upload, we could proceed in a similar way: POST user/pass, validate against DB and return true/false. Since all the communication will happen within the VLAN, security issues are less problematic. Nonetheless, I believe much more sophisticated solutions have already been developed.
My first thought was to build an OAuth server on Symfony and then authenticate the uploader (and future services) with their respective clients. Is this a right approach or is this a too complicated solution?
Alternatively, a service in the dataserver could validate user's credentials sent by the client against the REST API, receive a JWT and generate a new session for that particular client to list and update files on a particular folder. I'm not sure how to build this middleware though, do I need another Symfony instance or a simple PHP script will do the trick?
Please do not hesitate to share any thought you have on this. Any point of view will be much appreciated.
Thanks a lot
I studied and could successfully replicate the quickstart.py example on https://developers.google.com/drive/web/quickstart/quickstart-python to upload a file to my google drive using command line.
However, I wish to write an app that does the same, but through a cron job i.e. uploads a file everyday at 8am say, without the need to authenticate each time. Is there sample code/examples that I can look at to implement the oauth steps without the command line intervention?
Thanks!
You can use your App Engine app's built-in Service Account to authorize requests to the Google Drive API.
https://cloud.google.com/appengine/docs/python/appidentity/
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
Your app will need to have an embedded Refresh Token, or some way of fetching it from a secure server. The Refresh Token acts a bit like a stored username/password, albeit with constrained access. Therefore you need to consider the security implications. For example, since it's uploading, it will only need drive.file scope, so your corpus of Drive files remain inaccessible.
If you're happy with the security implications, then the steps you need are described How do I authorise an app (web or installed) without user intervention? (canonical ?)
I'm writing an application that takes a user's Evernote notes and displays them in a website inline. By its very nature, people accessing the resources attached to a note will not be logged in. I'm looking at the bottom of this page and saw how to pass authentication credentials via POST and get the resource. This is exactly what I need.
My question is how does this work in the real world? If I pass authentication tokens to the Javascript client (not secure in the first place), I can't get the resource because of Access-Control-Allow-Origin restrictions. The only other way I can think of doing this is saving all of the resources to my server and serving them from there, but that's not ideal (Google App Engine).
Ideas?
Yea, Evernote does not support CORS yet. You can do it in a chrome extension or get it on the server side.
I am eager to build an application with Gwt and App Engine. I'm more familiar with App Engine: creating dynamic html pages with servlets and jsp's. I'm wondering however, if this type of application technology belongs with GWT?
The two examples I can foresee being a problem are login and database retrieval.
For user log-in, my current approach is simple: at the beginning of a servlet, check if the user object exists. If it does, show the page. If it doesn't, redirect to Google's login service. How does this model fit in with GWT? It seems to me that GWT compiles into static html/javascript/css files you place on your server. With this approach, it doesn't seem possible to do any server processing to check for a vlid user before serving the static page (because any user could just bypass the servlet and type the static page url directly).
The other example would be show data from the app engine datastore. If i wanted to create a table which each row being an entry from the data store, I would dynamiclly create the html in a servlet, and do my datastore access there etc, etc. It seems with GWT I would have to serve a container html page, then use ajax to load the database content after the fact?
Maybe GWT isn't right for my type of application, or maybe I'm just thing about web application development the wrong way. Some clarification would be appreciated.
In a over simplified sense, a typical GWT app will work like this:
User navigates to your web page. Their browser uses a static url(s) to download all the javascript, css, and images necessary to run your app.
From now on, every time a user presses a button or otherwise interacts with the page, data is retrieved via an AJAX call. So yes, they might download your app before logging in, but all your security sensitive data would only be returned via an ajax call, giving you the chance to validate their identity however you wish (cookie, user/pass, etc)
This is basically what you describe in your second example about loading data from the datastore. It sounds like you think this is bad for some reason, but you don't say why.