I want to import a CSV file into Memgraph Cloud. I'm using 2GB free instance.
I tried to use LOAD CSV for a file located on AWS S3, but Memgraph Lab returns "Query failed: Load CSV not allowed on this instance because it was disabled by a config."
Is this limitation of the free version? Which options are available for loading data into Memgraph Cloud?
This is not related to the fact that you are using a free version of Memgraph Cloud.
Memgraph Cloud doesn’t support importing data from CSV files directly. Both the LOAD CSV clause and the CSV Import Tool require access to the local file system, and this is not possible when it comes to Memgraph Cloud.
There are two options you could try out:
Import the data programmatically by connecting to the instance via driver. This way, you could use each entry in your CSV file to execute a Cypher CREATE query.
Using the .cypherl bulk import 2 in Memgraph Lab. You can transform your CSV entries into Cypher CREATE queries and save them to a file with the extension .cypherl. Such a file can be imported by going to Memgraph Lab → Datasets tab → Select or drag and drop a file.
You can look up documentation on the Memgraph website.
Related
I have zero experience with ETL.
Whenever a file(a .csv) is moved into a specific folder, it should be uploaded to SalesForce I don't know how to get this automated flow.
I hope I was clear enough.
I gotta use the opensource version, any helpful links or resources will be appreciated.
Thank you in advance
You could definitely use Talend Open Studio for ESB : this studio contains 'Routes' functionalities : you'll be able to use a cFile component, which will check your folder for new files, and raise an event that will propagate throughout the route to a designed endpoint (for example a salesForce API). Talend ESB maps Apache Camel components , which are well documented.
Check about Routes with Talend for ESB, it should do the trick.
We have tFileExists component, you can use that and configure to check the file.
Also you have tFileWait component, where you can defile the frame of the arrival of he files and the number of iterations it has to check the file.
But i would suggest id you have any scheduling tool, use file watcher concept and then use talend job to upload the file to a specific location.
Using talend itself to check the file arrival is not a feasible way as the jobs has be in running state continuously which consumes more java resource
I'm trying to convert a bunch of blob files back into PDFs.
Currently they look like this:
I haven't really found much on the internet, and the few guides or post i've found, haven't yielded any results.
Has anyone ever tried doing this?
I'm not aware of any existing bulk export utility, however you should be able to use the Enterprise Manager tool in your image to save each individual BLOB as PDF. A bulk export utility could be created on request.
Until now I was using the BigQuery web tool to load from a backup of my data automatically saved on the Cloud Storage. I'm storing these backups three times a week, on three different buckets depending the week day (monday, wednesday, friday).
The GAE backup tool saves the .backup_info files with such a long name (for example: ahNzfmVnb2xpa2Vwcm9kdWN0aW9uckELEhxfQUVfRGF0YXN0b3JlQWRtaW5fT3BlcmF0aW9uGIrD6wMMCxIWX0FFX0JhY2t1cF9JbmZvcm1hdGlvbhgBDA.entityName.backup_info) and don't know how it is determined or if I can determine easier one. I can only give name to the "output-X-retry-Y" files. Is there any way to change this?
On the other hand, I'm trying the command-line tool, I want to move from the web tool to this one.
I've tried the load command but don't know how to automatically generate the schema from the backup, the same way I'm doing it from the web tool on the 'specify schema' section.
I'm always taking an error because of not specifying the schema trying this format:
bq load dataset.table gs://path
Is it possible to not determine the schema, the same way I'm not doing it on the web tool?
If you're running bq load to import a GAE datastore backup, you should add the --source_format=DATASTORE_BACKUP flag. Note you need to add that flag after load but before the table name:
bq load --source_format=DATASTORE_BACKUP dataset.table gs://path
That will tell BigQuery that you're loading from a datastore backup, which has a self-describing schema.
As far as I know, there isn't a way to control the generated name of the datastore backup.
On my local machine (i.e. http://localhost:8080/), I have entered data into my GAE datastore for some entity called Article. After turning off my computer and then restarting next day, I find the datastore empty: no entity. Is there a way to prevent this in the future?
How do I make a copy of the data in my local datastore? Also, will I be able to upload said data later into both localhost and production?
My model is ndb.
I am using Max OS X and Python 2.7, if theses matter.
I have experienced the same problem. Declaring the datastore path when running dev_appserver.py should fix it. These are the arguments I use when starting the dev_appserver
python dev_appserver.py --high_replication --use_sqlite --datastore_path=myapp.datastore --blobstore_path=myapp_blobs
This will use sqlite and save the data in the file myapp.datastore. If you want to save it in a different directory, use --datastore_path=/path/to/myapp/myapp.datastore
I also use --blobstore_path to save my blobs in a specific directory. I have found that it is more reliable to declare which directory to save my blobs. Again, that is --blobstore_path=/path/to/myapp/blobs or whatever you would like.
Since declaring blob and datastore paths, I haven't lost any data locally. More info can be found in the App Engine documentation here:
https://developers.google.com/appengine/docs/python/tools/devserver#Using_the_Datastore
Data in the local datastore is preserved unless you start it with the -c flag to clear it, at least on the PC. You therefore probably have a different issue with temp files or permissions or something.
The local data is stored using a different method to the actual production servers, so not sure if you can make a direct backup as such. If you want to upload data to both the local and deployed servers you can use the Upload tool suite: uploading data
The bulk loader tool can upload and download data to and from your application's datastore. With just a little bit of setup, you can upload new datastore entities from CSV and XML files, and download entity data into CSV, XML, and text files. Most spreadsheet applications can export CSV files, making it easy for non-developers and other applications to produce data that can be imported into your app. You can customize the upload and download logic to use different kinds of files, or do other data processing.
So you can 'backup' by downloading the data to a file.
To load/pull data into the local development server just give it the local URL.
The datastore typically saves to disk when you shut down. If you turned off your computer without shutting down the server, I could see this happening.
Is there a way to setup initial data in a database when using RhoMobile framework?
It is using sqlite database as I see. Is there a way to add a preset sqlite file to the project and make it built in the application bundle?
You can seed the database with a flat file containing the initial data, through the use of the Rho::RhoUtils.load_offline_data method. Full documentation here.