Where does AWS Sagemaker online featurestore store the features. Does it use DynamoDB?
SageMaker Feature store has an online and offline store.
The online store is a low latency cache and the offline store is S3.
https://docs.aws.amazon.com/sagemaker/latest/dg/feature-store-getting-started.html#feature-store-concepts
Related
I've enabled the accelerated transfer feature of my s3 bucket and I want to use its endpoint to read and write in that bucket. I am using AWS amplify to access the s3 bucket but cannot find enough resources on this issue.
The Amplify team is tracking this recent feature request here: https://github.com/aws-amplify/amplify-js/issues/4468
I have a Google App Engine project and I use Datastore.
I see that under the firebase console, I also have the option to create a Firestore database.
Does this Firestore database use the underlying Datastore in the same GAE project? I'm guessing maybe it lives side by side with Datastore?
That being said, can I use Firestore and Datastore in the same project?
Let me know if I am missing something...
They are related.
But in addition to not being able to use both in the same project you should be careful if/when you decide to switch as the switch isn't reversible (or at least not yet). Take the time to dive into the documentation to make an informed decision.
From Choosing between Cloud Firestore and Cloud Datastore:
Cloud Firestore is the next major version of Cloud Datastore and a re-
branding of the product. Taking the best of Cloud Datastore and the
Firebase Realtime Database, Cloud Firestore is a NoSQL document
database built for automatic scaling, high performance, and ease of
application development.
and
Warning: Once your database option is set, you cannot switch. You can't use both Cloud Firestore and Cloud Datastore in the same
project.
You can not. From the documentation:
Cloud Firestore and App Engine: You can't use both Cloud Firestore and
Cloud Datastore in the same project, which might affect apps using App
Engine. Try using Cloud Firestore with a different project.
In the docs, they claims that Cloud Firestore duplicates data in multiple servers located in different regions across the world.
My question is : By using Cloud Firestore, do i need to cache my data even though ?
Firestore is a document-based storage system. It's typically used as a database for your app, so for content that changes more regularly.
A CDN is a caching layer for delivering (mostly) static assets to regionally diverse users. I'd recommend looking at Cloud Storage for Firebase for that.
Cloud Firestore synchronously replicates data for durability and availability, rather than for serving speed. In this sense, it is very different from a CDN or a latency cache.
The Mobile and Web SDKs do store data locally though and provide a concept called latency compensation, so on that side it's got you covered out of the box.
I am building a mobile app with Google Cloud as back-end. It stores information about restaurants including their menu. I would need to search based on the menu items and geo location.
I am fairly confused about which Google Cloud solutions to use. Initially I have planned to use Google App Engine with Cloud Datastore for storing data, but later found out, it doesn't support geolocation search, and even if I use the App Engine search api, I won't get 'like' text search. About using cloud SQL, I am worried about its price. I am also thinking about using Google Compute Engine and installing required MongoDB or SQL in it and use some custom search libraries.
So my question is which of the below should be an feasible and economical solution for storing large amount of data and searching using text and geo-location
GAE with Cloud Datastore
GAE with Cloud SQL
GCE with MongoDB or SQL installed
Any other feasible solutions are also welcome
Google has a page to give high-level guidance on what storage option to choose on GCP, called Choosing a storage option.
Specifically for Cloud Datastore:
Description: A scalable, fully-managed NoSQL document database for your web and mobile applications.
Good for:
Semi-structured application data
Hierarchical data
Durable key-value
Common Workloads:
User profiles
Product catalogs
Game state
Specifically for Cloud SQL:
Description: A fully-managed MySQL database service that is built on the strength and reliability of Google’s infrastructure.
Good for:
Web-frameworks
Structured data
OLTP workloads
Common Workloads:
Websites, blogs, and content management systems (CMS)
Business Intelligence (BI) applications
ERP, CRM, and eCommerce applications
As you can see, your use case could border either option. Restaurants + Menus is very similar to product catalogs.
Cloud Datastore would require less ops work/thoughts on your side. For example, you don't need to think about what type of VM instance and memory size it should use. However, to do analytics on data rather than just serving the menus, you'd probably want to do daily dumps into BigQuery.
Cloud SQL will require a little more thought at the beginning, and at small sizes will be a little more expensive than Cloud Datastore. It does give you more flexibility on the analytics side in that it doesn't really need you to dump into BigQuery.
Running a different database on GCE is certainly an option if you want more control. The trade-off here will be in requiring much more active ops work on your behalf, such as installing, patching, tuning, etc.
So what makes google cloud storage preferred choice for saving files. As I know google cloud storage is saving the file as blob so it is immutable and cannot be edited.
if the main aspect of your website/application is a database (as is often the case), check out Google Cloud SQL. Again it allows you to host your MySQL database on Google’s infrastructure, increasing the speed, reliability and security.
Whereas Google Cloud Storage enables application developers to store their data on Google’s infrastructure with very high reliability, performance and availability, and can be used to distribute large data objects to users via direct download.