I am new to Neo4j and have been trying to load a CSV from my local disk, but without a success.
LOAD CSV WITH HEADERS FROM "file:C:/Neo4j/Persons.csv" AS csvLine
CREATE (p:Person { id: toInt(csvLine.id), name: csvLine.name })
I am getting the following response and error
Couldn't load the external resource at: file:C:/Neo4j/Persons.csv
Neo.TransientError.Statement.ExternalResourceFailure
Can you verify that the file is in the C:/Neo4j/ directory?
Could not be the perfect solutiuon but might be work for you.
You should try this one:
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS
FROM "file:C:/Neo4j/Persons.csv" AS csvLine
CREATE (:PERSONS {id:csvLine.id, name:csvLine.name})
But you should take notice the headers from your Persons.csv file.
Imagine that your file got this header
id | name |
you must use this Cypher code, before the CREATE statment:
FIELDTERMINATOR "|"
Related
Im working with GQLAlchemy and was wondering how to properly use WITH clause and import a CSV file with headers using Memgraph’s GQLAlchemy.
For example,
match().node(variable=“n”).with_({“n”: “node”})….
it is with underscore method in the query builder and an argument is a dictionary that in the above example means:
WITH n AS node
Importing CSV files with (or without) headers using GQLAlchemy can be done by executing a Cypher query. Check out how the CSV files with a header were imported in our workshop.
For example:
memgraph.execute(
"""LOAD CSV FROM "/movies.csv"
WITH HEADER AS row
MERGE (m:Movie {id: toInteger(row.movieId), title: row.title})
WITH m, row
UNWIND split(row.genres, '|') as genre
MERGE (m)-[:OF_GENRE]->(:Genre {name: genre});
""" )
I am trying to create the external table(xyz) in snowflake by using pattern to load historical file from stage, there are multiple files and using following pattern to load the file name started with below
201802242300_5d80272d1abcd32cc7a981da083ed498.gz. ( Feb 24th 2018 file)
Create external table xyz
(
samplecol1 varchar as (value:samplecol1::varchar),
samplecol2 varchar as (value:samplecol2::varchar),
date as to_date(substr(metadata$filename,1,8),yyyymmdd)
)
partition by (date)
location = #snowflakestage.largetable
pattern='.*/20180224.*[_].*.gz'
file_format = (type = 'JSON');
it's executing successfully but not loading any data. Is my pattern right to pick the file name listed above?
A good way to test patterns is via the LIST command as it takes the same PATTERN option.
thus for you:
LIST #snowflakestage.largetable pattern='.*/20180224.*[_].*.gz'
for example using the CitiBike example data, there are not parque files
so if you try load all files, you get errors.
create stage citibike.public.citibike_trips
url = 's3://snowflake-workshop-lab/citibike-trips';
list #citibike_trips;
name
s3://snowflake-workshop-lab/citibike-trips-parquet/2022/01/08/data_01a19496-0601-8b21-003d-9b03003c624a_3106_4_0.snappy.parquet
s3://snowflake-workshop-lab/citibike-trips-parquet/2022/01/09/data_01a19496-0601-8b21-003d-9b03003c624a_1906_6_0.snappy.parquet
s3://snowflake-workshop-lab/citibike-trips-parquet/2022/01/10/data_01a19496-0601-8b21-003d-9b03003c624a_2206_6_0.snappy.parquet
s3://snowflake-workshop-lab/citibike-trips/json/2013-06-01/data_01a304b5-0601-4bbe-0045-e8030021523e_005_7_0.json.gz
s3://snowflake-workshop-lab/citibike-trips/json/2013-06-01/data_01a304b5-0601-4bbe-0045-e8030021523e_005_7_1.json.gz
s3://snowflake-workshop-lab/citibike-trips/json/2013-06-01/data_01a304b5-0601-4bbe-0045-e8030021523e_005_7_2.json.gz
So I played around till I found this pattern worked for the files I wanted.
list #citibike_trips pattern = '.*trips_.*csv.gz';
I have this JSON on a Bucket that has been crawled with a classifier that splits arrays into record with this JSON classifier $[*].
I noticed that the JSON is on a single line - nothing wrong with syntax - but this results in the table being created having a single column of type array containing a struct which contains the actual fields I need.
In Athena I wasn't able to access the data and Glue was not able to read the columns as in array.field; so I manually changed the structure of the table making a single struct type with the other fields inside. This I'm able to query on Athena and get the Glue wizard to recognise the single columns as part of the struct.
When I create the job and map the fields accordingly (this is what is automatically generated, note the array.field notation applymapping1 = ApplyMapping.apply(frame = datasource0, mappings = [("array.col1", "long", "col1", "long"), ("array.col2", "string", "col2", "string"), ("array.col3", "string", "col3", "string")], transformation_ctx = "applymapping1")) I test the output on a table in an S3 Bucket. The Job does not fail at all, BUT creates files in the Bucket that are empty!
Another thing I've tried is to modify the Source JSON and add return lines:
this is before:
[{"col1":322,"col2":299,"col3":1613552400000,"col4":"TEST","col5":"TEST"},{"col1":2,"col2":0,"col3":1613552400000,"col4":"TEST","col5":"TEST"}]
this is after:
[
{"col1":322,"col2":299,"col3":1613552400000,"col4":"TEST","col5":"TEST"},
{"col1":2,"col2":0,"col3":1613552400000,"col4":"TEST","col5":"TEST"}
]
Having the file modified as stated before lets me correctly read and write data; this led me to believe that the problem is having a bad JSON at the beginning. Before asking to change the JSON is there something I can implement in my Glue Job (Spark 2.4, Python 3) to handle a JSON on a single line? I've searched everywhere but found nothing.
The end goal is to load data into Redshift, we're working S3 to S3 to check on why data isn't being read.
Thanks in advance for your time and consideration.
Currently I'm writing a React App and struggling with a simple reading from a SQlite database.
Edit because of unclear question:
***The goal is to read from the database without any backend, because it needs to read from the database even when it is offline.
***I'm aiming for a ONE TIME file conversion, then just pouchdb queries offline. But I don't want to do it manually because there are around 6k+ registries.
***Or SQL queries from the browser without any APIs, but I need to support Internet Explorer, so WebSQL is not an option. I've tried sqlite3 library, but I can't make it work with Create React App.
The solution I tried was to use PouchDB for reading the file, but I'm coming to a conclusion that it is NOT possible to PRELOAD a SQlite file with PouchDB without using cordova (I'm not comfortable with it, I don't want any servers running), or even with some kind of adapter.
So is this the right way of doing things?
Is there any way that I would not loose my .db data, and have to convert it all of it manually?
Should I forget about supporting this features on IE?
Thanks :)
Try this:
sqlite3 example "DROP TABLE IF EXISTS some_table;";
sqlite3 example "CREATE TABLE IF NOT EXISTS some_table (id INTEGER PRIMARY KEY AUTOINCREMENT, anattr VARCHAR, anotherattr VARCHAR);";
sqlite3 example "INSERT INTO some_table VALUES (NULL, '1stAttr', 'AttrA');";
sqlite3 example "INSERT INTO some_table VALUES (NULL, '2ndAttr', 'AttrB');";
## Create three JSON fragment files
sqlite3 example ".output result_prefix.json" "SELECT '{ \"docs\": ['";
sqlite3 example ".output rslt.json" "SELECT '{ \"_id\": \"someTable_' || SUBSTR(\"000000000\" || id, LENGTH(\"000000000\" || id) - 8, 9) || '\", \"anattr\": \"' || anattr || '\", \"anotherattr\": \"' || anotherattr || '\" },' FROM some_table;";
sqlite3 example ".output result_suffix.json" "SELECT '] }'";
## strip trailing comma of last record
sed -i '$ s/.$//' rslt.json;
## concatenate to a single file
cat result_prefix.json rslt.json result_suffix.json > result.json;
cat result.json;
You should be able simply to paste the above lines onto the (unix) command line, seeing output:
{ "docs": [
{ "_id": "someTable_000000001", "anattr": "1stAttr", "anotherattr": "AttrA" },
{ "_id": "someTable_000000002", "anattr": "2ndAttr", "anotherattr": "AttrB" }
] }
If you have jq installed you can do instead ...
cat result.json | jq .
... obtaining:
{
"docs": [
{
"_id": "someTable_000000001",
"anattr": "1stAttr",
"anotherattr": "AttrA"
},
{
"_id": "someTable_000000002",
"anattr": "2ndAttr",
"anotherattr": "AttrB"
}
]
}
You'll find an example of how quickly to initialize PouchDB from JSON files in part 2 of the blog post Prebuilt databases with PouchDB.
So, if you have a CouchDB server available you can do the following;
export COUCH_DB=example;
export COUCH_URL= *** specify yours here ***;
export FILE=result.json;
## Drop database
curl -X DELETE ${COUCH_URL}/${COUCH_DB};
## Create database
curl -X PUT ${COUCH_URL}/${COUCH_DB};
## Load database from JSON file
curl -H "Content-type: application/json" -X POST "${COUCH_URL}/${COUCH_DB}/_bulk_docs" -d #${FILE};
## Extract database with meta data to PouchDB initialization file
pouchdb-dump ${COUCH_URL}/${COUCH_DB} > example.json
## Inspect PouchDB initialization file
cat example.json | jq .
Obviously you'll need some adaptations, but the above should give you no problems.
Since Couch/Pouch-DB are document-oriented DBs all records aka docs there are just JSON aka JS-objects. In my RN app when I met similar task I just put all docs I wanted to be "prepopulated" in PouchDB in an array of JS-objects, import it as module in my app and then write them during app init to PDB as necessarry docs. That's all prepopulation. How to export your SQL DB records to JSON - you decide, surely it depends on source DB structure and data logic you want to be in PDB.
I'm trying to create a database where I'm to store data from a form into an sqlite database. I can't seem to be able to link the two, I keep getting errors. Here's my code for the query in the php file:
<?php
$dbhandle = new SQLiteDatabase('sqlitedb');
$result = $dbhandle->arrayQuery('SELECT name, email FROM users LIMIT 25', SQLITE_ASSOC);
foreach ($result as $entry) {
echo 'Name: ' . $entry['name'] . ' E-mail: ' . $entry['email'];
}
?>
And this is the error message that shows:
Fatal error: Call to undefined function sqlite_open() in C:\Program Files\Apache Software Foundation\Apache2.2\vhosts\scotiaguestpro\extract.php on line 2
I manually created a database and tables with data in command prompt to query through. The database file is in the same directory as the php file.
PHP 5.2.x does not support sqlite3, only sqlite2.
If you really need sqlite3 support, update to PHP 5.3.x.
Probably the best option anyway since PHP 5.2.x is not supported anymore by the PHP crew.