I have MongoDB as Data source. I want to query in redash which will be the equivalent following:-
db.collection.find({“templateId”:/XYZ$/})
This query return all data from the collection where key templateId end with string XYZ. How can I use the same in redash JSON?
Also please help in using $exist in Redash.
I use this example with Redash.
{
"collection": "Users",
"query": {
"Phone": {
"$regex": "1234$",
"$options": "i"
}
}
}
Options i to make it case insensitive
Got here from googling the same question.
It seems to me that next pipeline step works:
{"$match": {"templateId": {"$regex": "XYZ$"}}}
Related
All the examples I read online seem to refer to using the compass or the command line, not MongoDB cloud. I try emulate the same query code in the search filter but not having any luck.
Search function
For example with the sample collections..
_id:ObjectId("5bd761dcae323e45a93ccff1")
saleDate:2014-08-18T04:37:26.849+00:00
storeLocation:"Denver"
couponUsed:false
purchaseMethod:"In store"
_id:ObjectId("5bd761dcae323e45a93ccff7")
saleDate:2017-12-08T21:40:34.527+00:00
storeLocation:"Denver"
couponUsed:false
purchaseMethod:"Phone"
_id:ObjectId("5bd761dcae323e45a93ccfef")
saleDate:2014-03-31T16:02:06.624+00:00
storeLocation:"Austin"
couponUsed:false
purchaseMethod:"Online"
How could I filter saleDate to show objects from 2014?
Botti you want to use the range operator in $search pipeline:
{
"$search": {
"index": "<index Name>",
"range": {
"path": "saleDate",
"gte":"2014-03-31T16:02:06.624+00:00",
"lte": "2017-12-08T21:40:34.527+00:00"
}
}
}
Im trying to load a JOSN file where some of the arrays are empty.
{"house_account_payable":"0.00","house_account_receivable":"0.00","gift_sales_payable":"0.00","gift_sales_receivable":"0.00","store_credit_sales_payable":"0.00","percentage_row":null,"sales_per_period":[["02:00AM - 02:59AM",{"amount":0,"qty":0}],["03:00AM - 03:59AM",{"amount":0,"qty":0}]],"revenue_centers":[],"tax_breakdowns":[]}
This is giving the error:
rror while reading table: test2, error message: Failed to parse JSON: No object found when new array is started.; BeginArray returned false; Parser terminated before end of string
Could somebody help me on this?
Are you trying to load data from your local machine or GCS? Please, remember about exporting in JSONL(Newline delimited JSON):
{"open_orders_ids": []}
{"unpaid_orders_ids": []}
The output:
Take a look for documentation about nested and repeated columns.
EDIT:
Your JSON schema should look like this:
{
"items": [
{
"house_account_payable": "0.00",
"house_account_receivable": "0.00",
"gift_sales_payable": "0.00",
"gift_sales_receivable": "0.00",
"store_credit_sales_payable": "0.00",
"percentage_row": "",
"sales_per_period": [
{
"AM02_00_AM02_59": {
"amount": "0",
"qty": "0"
}
},
{
"AM03_00_AM03_59": {
"amount": "0",
"qty": "0"
}
}
]
}
]
}
Regarding to Felipe Hoffa's post, run following commands:
jq -c .items[] <FILENAME>.json > <FILENAME>.jq.json
bq load --source_format NEWLINE_DELIMITED_JSON --autodetect <DATASET_ID>.<TABLENAME> <FILENAME>.jq.json
The schema:
Let me know if this is what you are looking for.
There's no problem with the null arrays.
The problem lies in this shorter json:
{"sales_per_period":[["02:00AM - 02:59AM",{"amount":0,"qty":0}],["03:00AM - 03:59AM",{"amount":0,"qty":0}]]}
The arrays there hold elements of different types, and to bring it into a structured table, a different schema is needed.
For example:
{"sales_per_period":[{"a":"02:00AM - 02:59AM","b":{"amount":0,"qty":0}},{"a":"03:00AM - 03:59AM","b":{"amount":0,"qty":0}}]}
Now this loads easily into BigQuery:
bq load --source_format=NEWLINE_DELIMITED_JSON --autodetect temp.short delete.short.json
Can you change this source JSON easily outside BigQuery? Otherwise load it raw into BigQuery, and parse it with a JS UDF inside BigQuery.
I'm trying to query the following object from mongodb
[
{
"id": "6b3a9814c1990a0578988d9e",
"details": {
"buyerId": "5bd450ed0307fa0a3a904376",
"offerId": "1",
"productId": "5b3a9814c1880a0578988d6a",
"productTitle": "Watch",
"amount": 50,
"status": "Open",
}
}
]
I'm using spring-boot-starter-data-mongodb so first, I tried it the standard way.
Here is what's in my repository,
public interface OfferRepository extends MongoRepository<Offer, String> {
List<Offer> findOffersByDetailsBuyerId(String buyerId);
}
I've also tried a custom query,
#Query(value = "{'details.buyerId' : ?0 }")
List<Offer> findOfferByDetails_BuyerId(#Param("buyerId") String buyerId);
Both are coming back with an empty array. But if I hard code the buyerId in the query I get the results I want.
Also, when I debug it, I see the param but with double quotes around it?
screenshot from mongo compass
In MongoUI you should check the datatype of field ID. It is ObjectId not String So you need to pass org.bson.types.ObjectId instead of String in the repository method.
List<Offer> findByDetailsBuyerId(ObjectId buyerId);
ANSWER
#Query("{ 'details.buyerId' : ?0 }")
List<OfferOverview> findOffersByDetailsBuyerId(String buyerId);
but my main issue was passing the buyerId with double quotes.
I'm using couchdb to store large documents, which is causing some trouble when fetching them to memory. I do realize the database is not meant to be used this way. As a fallback solution, is it possible to fetch partial documents from the database, without creating a view?
In example, if a document has the fields id, content and extra_content, I would like to retrieve only the first two.
Thank you in advance.
If you are using CouchDB 2.x, you can use /db/_find endpoint as a mechanism to retrieve part of the doc.
POST /db/_find
{
"selector": {
"_id": "a-doc-id"
},
"fields": [
"_id",
"content"
]
}
You'll get only the set of fields you have specified in the query
This is not possible prior to CouchDB 2.x. For CouchDB 2.x or greater, see JuanjoRodriguez's answer.
But one possible work around for any version of CouchDB would be to take advantage of file attachments, which by default are excluded from a fetch. If some of your data isn't always needed, and doesn't need to be included in indexes, you could potentially store it as (JSON) attachments, rather than as part of the document directly:
{
"id": "foo",
"content": "stuff",
"extra_content": "other stuff"
}
becomes:
{
"id": "foo",
"content": "stuff",
"_attachments": {
"extra_content": {
"content_type": "application/json",
"data": "ZXh0cmEgc3R1ZmYK"
}
}
}
I am using Cloudant database and I would like to retrieve all the documents within the db that match specific fields.
I mean - I'd want to get only those documents which have some fields with specific values that I put.
Could you please help me with an example of code I can test?
Thanks in advance
A good, but quite general question.
You can achieve this in several ways. The most canonical CouchDB way would be to create a map-reduce view (secondary index) keyed on the field you wish to be able to query on, for example:
function (doc) {
if (doc && doc.surname) {
emit(doc.surname, 1);
}
}
You can create such views using the Cloudant dashboard:
You can now query this for all documents with a particular surname:
curl 'https://ACCOUNT.cloudant.com/examples/_design/example/_view/by_surname?limit=100&reduce=false&include_docs=true&startkey="kruger"&endkey="kruger0"'
If you want to be able to query on a combination of fields, you can create a vector-valued key:
function (doc) {
if (doc && doc.surname && doc.firstname) {
emit([doc.surname, doc.firstname], 1);
}
}
which is queried as this:
curl 'https://ACCOUNT.cloudant.com/examples/_design/example/_view/by_name?limit=100&reduce=false&include_docs=true&startkey=\["kruger", "stefan"\]&endkey=\["kruger","stefan0"\]'
If you're new to Cloudant, another way to query is by using the aptly named Cloudant Query (a.k.a. Mango), which is a json-based declarative query language.
It is well documented (https://console.bluemix.net/docs/services/Cloudant/api/cloudant_query.html), but the gist of it is queries of the type:
{
"selector": {
"year": {
"$gt": 2010
}
},
"fields": ["_id", "_rev", "year", "title"],
"sort": [{"year": "asc"}],
"limit": 10,
"skip": 0
}