I have a flask server running on a GAE (flexible env). The application generates certain files during runtime based on the API requests received. But after the instance starts, after idling, the files are lost. How do i prevent this ?
Your application seems to write those files on the filesystem. However, whatever is written in the filesystem will not be persisted and will be lost on instance death. Also, what you write in one instance won't be available to other instances serving your app.
The solution to this is to write your files to Google Cloud Storage. Any instance can write and retrieve its files there and they'll be available for any instance of your service.
Related
To verify the ownership of a domain to a mail service, I need to put a file with a specific name for verification. Is there a better way than pushing it into my app source repository?
For security reasons you would have to put the file in your source and do a deployment to App Engine. If you’ve worked with a traditional web server in the past where you basically dump files into a folder and serve them this will be a bit of a change. The App Engine files are going to execute only. If you want to get in to adding other files on the fly you would need a Cloud Storage Bucket, but I don’t think that will do it for your domain verification.
I have been struggling in getting solution for below scenario:
I have 2 instances of solr cloud on composite engine.
I have 2 instances of application rest api which calls above solr cluster for data.
From my application I wanna take backup of solr and copy zipped backup file to google storage automatically and restore it automatically with the an url endpoint.
For that I am trying to make a api endpoint in my application that will call below solr api to take back up
admin/collections?action=BACKUP
And making another endpoint that call below url to restore
/admin/collections?action=RESTORE
However after taking backup my application doesnt have access to the back up files as they are getting saved on solr instances. So I am not able to save them to google bucket.
Please guide me a simpler way to achieve this i.e automatically backup and restore solr from other GCP instance.
Have you considered something like gcs-fuse? It'll allow you to mount a GCS bucket directly on the file system.
You can then point the BACKUP command directly to the mount point for gcs-fuse on your Solr compute engine VMs, and the whole thing is abstracted away through how the VM is configured (instead of having to be manually uploaded afterwards with a separate tool when a local copy has been made).
I found GCSFuse a bit unreliable, and decided to write a wrapper script which first detects the master for the given collection and then just executed the backup directly on that node.
Q1: Where do you think is the right place to put a SQLite database file (database.sqlite) in Azure Web App file system? For example:
D:\home\data\database.sqlite
D:\home\site\database.sqlite
D:\home\site\wwwroot\database.sqlite
other?
Q2: What else should be taken into consideration in order to make sure that the database file won't be accessible to public users as well as not being accidentally overwritten during deployments or when the app is scaled up/down? (The Web App is configured for deployments from a Local Git Repository)
Q3: Where to learn more about the file system used in Azure App Service, the official source URL? E.g. how it's shared between multiple VMs within a single Web App, how does it work when the App is scaled up/down, what's the difference between D:\home (persistent) vs D:\local (non-persistent)...
Note that SQLite does not work in Azure Blob Storage, so that one is not an option. Please, don't suggest alternative storage solutions, this question is specifically about SQLite.
References
Appropriate Uses For SQLite
In a Web App, your app is deployed to d:\home\site\wwwroot. This is the area where you may write files. As an example, the ghost deployment writes its SQLite database to d:\home\site\wwwroot\content\data\ghost.db. (easy to see this, if you open up the kudu console via yourapp.scm.azurewebsites.net):
This file area is shared amongst your web app instances. Similar to an SMB file share, but specific to web apps (and different than Azure's File Service).
The content under wwwroot is durable, unless you delete your app service. Scaling up/down impacts the amount of space available. (I have no idea what happens if you scale down and the smaller size has less disk space than what you're consuming already).
I would say the best location would be app_data folder in the site/wwwroot folder. Create the folder if it doesn't exist.
Web Apps can connect to storage accounts so you can in fact use blob storage and connect that to your web app. So in terms of learning more about it then you need to be looking at the appropriate page of documentation.
In your Web App settings you can then select which storage account to use. You can find this under Settings > Data Connections where you can select Storage from the drop down box.
I was handed an assignment but I don't know where to start.
The aim is to have 2 piece of code running. One will run in Open stack private cloud and perform the task of indexing two sets of text, with another running in EC2 with the task of matching the two indexed tests.
I want to access them via google app engine.
Ideally, I would like to click a button or perform an action on Google app engine, which then sends a request to Openstack to run the code and retrieve the output of a txt file.
That outputted text files will then be forwarded onto EC2 where the matching will occur and the results sent back to Google App Engine.
My question is, how can I send the files between the systems using REST requests?
FrankN --
EC2, GAE and OpenStack are disparate cloud computing platforms. To integrate them might include, say, using one platform while saving backups to another.
CloudU.Rackspace.com is a vendor-neutral education site about cloud computing (note: It'll take six or so hours to finish it all). This might help.
Disclaimer: I work for Rackspace.
Firstly, not really sure what your requirements are, why your code does or why are you trying to mix cloud providers in that way.
That said, I would suggest taking the upload from GAE and push it to AWS S3 where you can then retrieve and use as you please from EC2.
Not sure what functionality you are trying to get out of OpenStack that is not present in AWS; however, I would suggest building whatever you are building in EC2 first, then duplicate in on OpenStack services to avoid future vendor lock in.
I and my friend are working on a GWT-Google App Engine project, using Tortoise SVN and Google Code to synchronize the code.
We also synchronize the local_db.bin file in appengine-generated folder. But we cant get it work. After synchronize the db file, our local datastore is not updated as we expected.
That is a pain. Im worrying about our future, when our database get bigger and more complicated #A#.
Anyone please give me an advice. What should i do to synchronize our local datastore?
I have to suggestions:
1) Use remote api : https://developers.google.com/appengine/articles/remote_api to share a GAE hosted db locally.
2) Maybe you can use Gdrive to sync folders.
This is a really bad idea. Even if you weren't having trouble making both ends read from the same datastore file, the local datastore is in a binary format, and thus you won't both be able to work on the app at the same time, or you'll get merge conflicts you will be unable to resolve.
Instead, both for collaboration purposes and for testing and deployment, you should provide a set of test data you can easily load into the datastore. Store the test data in version control, and load it in using bulkloader or your own code.