I have simple MERN stack application that allows to upload gallery images. I host all static assets including all gallery images on public s3 bucket. I did it on purpose because I want every image to be public and load time is much faster because we don't need to retrieve and encode images to display them on the website. However, I'm worried about some unexpected cost in case that someone will try to generate unwanted traffic to my bucket. As we know AWS charge for every request and data transfer so my question is what are my options to prevent it. Although, generating images using server like mentioned before is probably the best option, I don't want to do it. I read somewhere that I could use budget alerts and trigger lambda function to make s3 bucket private but I can't find information how to implement it.
Maybe there is some other way to do it?
Related
Sorry about the "stupid" title, but I don't really know how to explain this.
I want to have a webpage on my site (built in React), that will show the release notes for each version of my site/product. I can hardcode the content of the release notes in the page, but I want to do something that allows me not to have to recompile my site just to change content.
My site is hosted in AWS, so I was thinking if there are any patterns to store the content of the page in an S3 bucket as a text file, or as an entry in DynamoDB.
Does this make sense?
These are things I remember, but I would like to ask how "you" have done this in the past.
Thank you in advance!
You could really use either S3 or DynamoDB, though S3 ends up being more favorable for a few reasons.
For S3, the general pattern would be to store your formatted release notes as an HTML file (or multiple files) as S3 objects and have your site make AJAX requests to the S3 object(s) and load the HTML stored there as your formatted release notes.
Benefits:
You can make the request client-side and asynchronous via AJAX, so the rest of the page load time isn't negatively impacted.
If you want to change the formatting of the release notes, you can do so by just changing the S3 object. No site recompilation required.
S3 is cheap.
If you were to use DynamoDB, you would have to request the contents server-side and format them server-side (the format would not be changeable without site recompilation). You get 25 read capacity units for free, but if your site sees a lot of traffic, you could end up paying much more than you would with S3.
I would like to know how do CDNs serve private data - images / videos. I came across this stackoverflow answer but this seems to be Amazon CloudFront specific answer.
As a popular example case lets say the problem in question is serving contents inside of facebook. So there is access controlled stuff at an individual user level and also at a group of users level. Besides, there is some publicly accessible data.
All logic of what can be served to whom resides on the server!
The first request to CDN will go to application server and gets validated for access rights. But there is a catch - keep this in mind:
Assume that first request is successful and after that, anyone will be able to access the image with that CDN URL. I tested this with Facebook user uploaded restricted image and it was accessible with the CDN URL by others too even after me logging out. So, the image will be accessible till the CDN cache expiry time.
I believe this should work - all requests first come to the main application server. After determining whether access is allowed or not, a redirect to the CDN server or access-denied error can be shown.
Each CDN working differently, so unless you specify which CDN you are looking for its hard to tell.
I have been reading all over stackoverflow concerning datastore vs blobstore for storing and retrieving image files. Everything is pointing towards blobstore except one: privacy and security.
In the datastore, the photos of my users are private: I have full control on who gets a blob. In the blobstore, however, anyone who knows the url can conceivable access my users photos? Is that true?
Here is a quote that is supposed to give me peace of mind, but it's still not clear. So anyone with the blob key can still access the photos? (from Store Photos in Blobstore or as Blobs in Datastore - Which is better/more efficient /cheaper?)
the way you serve a value out of the Blobstore is to accept a request
to the app, then respond with the X-AppEngine-BlobKey header with the
key. App Engine intercepts the outgoing response and replaces the body
with the Blobstore value streamed directly from the service. Because
app logic sets the header in the first place, the app can implement
any access control it wants. There is no default URL that serves
values directly out of the Blobstore without app intervention.
All of this is to ask: Which is more private and more secure for trafficking images, and why: datastore or blobstore? Or, hey, google-cloud-storage (which I know nothing about presently)
If you use google.appengine.api.images.get_serving_url then yes, the url returned is public. However the url returned is not guessable from a blob's key, nor does the url even exist before calling get_serving_url. (Or after calling delete_serving_url).
If you need access control on top of the data in the blobstore you can write your own handlers and add the access control there.
BlobProperty is just as private and secure as BlobStore, all depends on your application which serves the requests. your application can implement any permission checking before sending the contents to the user, so I don't see any difference as long as you serve all the images yourself and don't intentionally create publicly available URLs.
Actually, I would not even thinlk about storing photos in the BlobProperty, because this way the data ends up in the database instead of the BlobStore and it costs significantly more to store data in the database. BlobStore, on the other hand, is cheap and convenient.
I am using Azure Mobile Services to store images for a web application.
I have managed to successfully upload images to a private container. I've followed the logic in this introductory guide (http://code.msdn.microsoft.com/windowsapps/Upload-File-to-Windows-c9169190), i.e. when uploading the file to the database an SAS is generated by a node script called when inserting a record into a table.
One of the reasons to use this approach from mobile apps is so that the storage key is not stored within the application source itself.
Conforming with that idea I am now struggling to find an example of how to download the images.
Perhaps I should update the read function for the same table and have that return an SAS which can be used to accessed the image.
Does this sound reasonable or are they better approaches?
Any assistance is greatly appreciated.
It sounds to me like you are on the right track. If you are storing the image in a private container and want the mobile device to read it back then yes, you will want to produce a SAS that allows reading and get that back to the device. The device code can then make a call directly against BLOB storage using that SAS URL to retrieve the image.
This applies only if you want the container private. If the container is public then just returning the URL (like they have in the article you link to) should be fine.
It also depends on how private you care the image to be. For example, let's say you have a container created per user. If the container has a Shared Access Signature Policy on it with a really far off expiration date then technically someone still needs the URL with the SAS to view it, but you can create that SAS and store it like the sample. The mobile app can then be given the URL when it reads data from your service and get to the BLOB directly without having it create an additional SAS. In my opinion this option only really works if the images aren't going to be around very long, or you don't really care that if someone sniffs the URL from the network traffic that they can access it.
If you want it fairly secure and do not know how long the images will be around, then you should go with your stated approach of getting a SAS for read when the app reads from the related table data. The SAS can have a fairly short expiry on it and the mobile device can cache the result.
Is it possible to copy images into a static directory under my app engine project domain?
For example, when a user signs up for my app, I want them to supply an image for themselves, and I would copy it to a static directory but rename the image using their username, like:
www.mysite.com/imgs/username.jpg
www.mysite.com/imgs/john.jpg
www.mysite.com/imgs/jane.jpg
but I don't know where to start with this, since the JDO api doesn't really deal with this sort of thing (I think using JDO, they'd want me to store the image data as a blob associated with my User objects). Can I just upload the images to a static directory like this?
Thanks
No. App Engine has a provision for static files, but only static files you upload along with your code. If users can upload the data, it is not really "static" in the app engine context. Depending on how large a picture you want users to be able to upload, you will want to use either the regular datastore (for storing up to 1MB) or the Blobstore for bigger files (up to 2 gig)
I'm almost certain you need to use the blobstore for dynamic upload. Even if you need not, for reasons of session independence you probably want to. As blobstore operations are expensive relative to a static file, you could have a task queue move the (now static) images into static store.