I have a snowpipe. I do not have control over the incoming files. The files each contain a single, relatively small json record.
The stage had about 38000 files when I restarted the pipe due to an error.
I am currently only processing 6-8 files per minute.
EDIT: Correction, I have processed about 750 files in 55 min. Still very slow.
How can I speed this up without changing the one record per file constraint?
{
"id": "55555555-5555-5555-5555-555555555555",
"val": {
"bank": "bank name",
"browser_version": "1.7.4044.138",
"cpu": "Intel(R) Core(TM) i5-8265U CPU # 1.60GHz",
"cpu_architecture": "x86_64",
"created_at": "2020-07-31T10:21:12.992345225Z",
"current_address": "address",
"current_city": null,
"current_country": "my country",
"date_of_birth": "1990-03-21",
"document_number": "55-55-55-55555",
"document_type": "this type of document",
"email": "some.email#email.com",
"full_name": "some name",
"gender": "Male",
"internal_storage": "237.87 GB",
"joined_date": "2020-07-31",
"last_logged_in": null,
"last_worked_on": "2020-12-02",
"latitude": "Not Found",
"longitude": "Not Found",
"memory": "7.74 GB",
"number_of_processors": "8",
"app_client_uid": "{55555555-5LLL-5555-55L5-555B5555DB5A}",
"operating_system": "win 10.0",
"other_display": null,
"id_number": "555555555",
"permanent_address": null,
"permanent_city": null,
"permanent_country": null,
"personal_email": "anotheremail#gmail.com",
"primary_display": "1920x1080",
"role_id": "55555555-5555-5555-5555-555555555555",
"source_system": [
{
"id": "astringidthiny",
"system": "system_name"
}
],
"status": "Active",
"type": "worker",
"updated_at": "2020-12-02T08:32:56Z",
"webcam": "true",
"workstream_ids": [
"55555555-5555-5555-5555-555555555555"
]
}
}
Related
How can one split the below json example by different email in a for each loop? So the end result is two completely separate entities keeping there own objects and arrays.
This result comes from a sql dump but the sql connector alters the output, so I've had to parse that output in a for each loop of its own even though there is no loop it's all in one line.
[
{
"Email": "Steve#gmail.com",
"Name": "Steve Larson",
"ID": "1111",
"Date": "2022-09-12",
"Address": "10 Chicken Place",
"Town": "Gatebody",
"Postcode": "xxx 1xx",
"Data": [
{
"Ref": "34546",
"Transaction_Date": "2018-11-29",
"Amount": 27.76,
"Balance": 27.76,
"Notes": "Beep1"
},
{
"Ref": "34546v2",
"Transaction_Date": "2018-12-24",
"Amount": 27.76,
"Balance": 27.76,
"Notes": "Beep2"
}
]
},
{
"Email": "innis#gmail.com",
"Name": "Innis Blitz",
"ID": "3456",
"Date": "2022-10-12",
"Address": "33 Snake Road",
"Town": "March",
"Postcode": "cxc 3cd",
"Data": [
{
"Ref": "078776701",
"Transaction_Date": "2021-08-27",
"Amount": 984.68,
"Balance": 984.68,
"Notes": "Yes please"
},
{
"Ref": "078776701v2",
"Transaction_Date": "2021-08-27",
"Amount": 98422.6,
"Balance": 98432.6,
"Notes": "Please not now"
}
]
}
]
Would like to do a for each so it splits the above like so. The idea then for my to load the result of the for each into a template that takes arrays.
{
"Email": "Steve#gmail.com",
"Name": "Steve Larson",
"ID": "1111",
"Date": "2022-09-12",
"Address": "10 Chicken Place",
"Town": "Gatebody",
"Postcode": "xxx 1xx",
"Data": [
{
"Ref": "34546",
"Transaction_Date": "2018-11-29",
"Amount": 27.76,
"Balance": 27.76,
"Notes": "Beep1"
},
{
"Ref": "34546v2",
"Transaction_Date": "2018-12-24",
"Amount": 27.76,
"Balance": 27.76,
"Notes": "Beep2"
}
]
}
{
"Email": "innis#gmail.com",
"Name": "Innis Blitz",
"ID": "3456",
"Date": "2022-10-12",
"Address": "33 Snake Road",
"Town": "March",
"Postcode": "cxc 3cd",
"Data": [
{
"Ref": "078776701",
"Transaction_Date": "2021-08-27",
"Amount": 984.68,
"Balance": 984.68,
"Notes": "Yes please"
},
{
"Ref": "078776701v2",
"Transaction_Date": "2021-08-27",
"Amount": 98422.6,
"Balance": 98432.6,
"Notes": "Please not now"
}
]
}
I have reproduced in my environment and got expected results and I followed below process:
I have initialized your input as below:
Then i added for each control and send the output of previous step as input as inside of foreach loop, i have added send email action.
Outputs:
I have got tow jsons in two seperate emails as above.
I wrote this code to get the product information with it's images and category:
->where('category_id', 5)
->with('category', 'Files')->get();
my result is:
{
"id": 2,
"name": "test",
"price": 13000,
"description": "some text ...",
"shop_id": 1,
"rate": 0,
"category_id": 5,
"discount_percent": 20,
"category": {
"id": 5,
"name": "cat1",
"shop_id": 1
},
"files": [
{
"id": 99,
"disk_name": "5ef1af07d6d98778754621.jpg",
"file_name": "13330983_xl.jpg",
"file_size": 69813,
"content_type": "image/jpeg",
"title": null,
"description": null,
"field": "product_gallery",
"sort_order": 99,
"created_at": "2020-06-23 07:28:07",
"updated_at": "2020-06-23 07:28:10",
"path":...... storage/app/uploads/public/5ef/1af/07d/5ef1af07d6d98778754621.jpg",
"extension": "jpg"
}
]
}
now i want to access the path field, how can i do it?
i use this way for access but i don't get result:
products[0].files[0].path
You should use toArray() function to convert data likes this
->where('category_id', 5)
->with('category', 'Files')->get()->toArray();
And then access
products[0]['files'][0]['path']
I'm using Elasticsearch and Grafana to show stock growth, ex,
(price[i]-price[first])/price[first] just like growth rate,
but i don't know how to query
here is the sample data
PUT stock/_doc/1
{
"Code": "0000666",
"Name": "test",
"Price": 0.665,
"Date": "2020-01-01"
}
PUT stock/_doc/2
{
"Code": "0000666",
"Name": "test",
"Price": 0.670,
"Date": "2020-01-02"
}
PUT stock/_doc/3
{
"Code": "0000666",
"Name": "test",
"Price": 0.685,
"Date": "2020-01-03"
}
PUT stock/_doc/4
{
"Code": "0000666",
"Name": "test",
"Price": 0.690,
"Date": "2020-01-04"
}
PUT stock/_doc/5
{
"Code": "0000666",
"Name": "test",
"Price": 0.692,
"Date": "2020-01-05"
}
I used painless ,but it only use on single doc
I used derivative,but it only use on two doc,not the first doc
I am currently not specifying any routing order for signers and non-signers(cc recipients). In this case non-signers get the email only when the signers have signed and completed the envelope. I need for non-signers to get email same time as signers (before signers sign) as well as after signers sign and complete. How to achieve this. I alresy looked at this solution Docusign - Adding a non signing CC role with same routing order
But when I tried different roleNames and same routing order I get "errorCode": "ENVELOPE_HAS_DUPLICATE_RECIPIENTS",
How do we differtiate between signers and non-signers? I tried to use different role names, but did not work.
My request json:
{
"status": "created",
"recipients": {
"signers": [{
"tabs": {
"signHereTabs": [],
"initialHereTabs": [],
"dateSignedTabs": []
},
"routingOrder": '1',
"roleName": "Signer",
"recipientId": "1",
"name": "Signer Name",
"email": "someone#somewhere1.com",
"clientUserId": null
}],
"carbonCopies": [{
"routingOrder": "1",
"roleName": "CCRole",
"recipientId": "1",
"email": "someone1#somewhere.com"
}, {
"routingOrder": "2",
"roleName": "CCRole",
"recipientId": "2",
"email": "someone2#somewhere.com"
}]
},
"messagelock": "false",
"emailsubject": "DocuSign:1xxxxx78-03 - Testcase",
"emailblurb": "Please sign attached document(s)",
"documents": [{
"name": "11xxxxxx8-01- Documents",
"documentid": "1",
"documentBase64": null
}],
"customFields": {
"textCustomFields": [{
"value": "a54c00000000ogaAAA",
"show": "false",
"required": "true",
"name": "DSFSSourceObjectId",
"fieldId": null
}]
}
}
Recipients at the same routing order will receive emails simultaneously when the envelope reaches the particular routing order. In addition a CarbonCopy recipient will also receive an email that an envelope is completed.
Carbon Copy Recipients
Documentation here
Carbon copy recipients get a copy of the envelope but don't need to sign, initial, date or add information to any of the documents. This type of recipient can be used in any routing order. Carbon copy recipients receive their copy of the envelope when the envelope reaches the recipient's order in the process flow and when the envelope is completed.
Sending Envelope with notifications to CC recipients before and after Signer
Documentation here
You are getting the ENVELOPE_HAS_DUPLICATE_RECIPIENTS error message since you have specified recipientId = 1 for multiple recipients. Also you can remove the roleName as there is no template involved.
Here is an example that should work for your use case. I am using unique routing order and recipient ID for each of the recipients.
POST /v2/accounts/{accountId}/envelopes
{
"emailSubject": "Envelope with 1 cc - 1 signer - 1 cc",
"status": "sent",
"recipients": {
"signers": [
{
"email": "AndySigner#acme.com",
"name": "Andy Signer",
"recipientId": "2",
"routingOrder" : "2",
"tabs": {
"signHereTabs": [
{
"documentId": "1","pageNumber": "1", "xPosition": "80", "yPosition": "80",
}
]
}
}
],
"carbonCopies": [
{
"name": "Jane CC",
"email": "Janecc#acme.com",
"recipientId": "1",
"routingOrder": "1"
},
{
"name": "Bob CC",
"email": "BobCC#acme.com",
"recipientId": "3",
"routingOrder": "3"
}
],
},
"documents": [
{
"documentId": "1",
"name": "Contract",
"fileExtension": "txt",
"documentBase64": "RG9jIFRXTyBUV08gVFdP"
}
]
}
I have an array of job objects that gets pulled from the server, and an array of people objects also pulled from the server. I want the persons job to be represented by a select element, populated from the jobs array. I have populated the select using ng-options, and when changing the selection, the person model's job object is updated. The only issue I have is that to start with the select shows an empty value. I can't work out how to have it show the person objects current job?
<select
ng-model='person.job'
ng-options='job.title for job in jobs'>
</select>
the person object looks like this
{
"id": "1",
"firstName": "Marianne",
"lastName": "Jenkins",
"middleNames": null,
"ext": "4680",
"phoneCell": "1-174-668-3846",
"phoneHome": "+10(2)5744088105",
"takerNumber": "180",
"hidden": "0",
"created_at": "2014-01-09 12:55:12",
"updated_at": "2014-01-09 12:55:12",
"job": {
"id": "25",
"title": "Office Manager",
"created_at": "2014-01-09 12:55:11",
"updated_at": "2014-01-09 19:25:03"
},
"office": {
"id": "4",
"name": "Salt Lake City",
"prefix": "702",
"order": "2",
"takerNumber": "103",
"address_id": "1"
}
}
and the jobs array looks like this
[
{
"id": "1",
"title": "Field Service Tech",
"created_at": "2014-01-09 12:55:11",
"updated_at": "2014-01-09 19:25:03"
},
{
"id": "2",
"title": "Inside Sales Manager",
"created_at": "2014-01-09 12:55:11",
"updated_at": "2014-01-09 19:25:03"
},
{
"id": "3",
"title": "Office Assistant",
"created_at": "2014-01-09 12:55:11",
"updated_at": "2014-01-09 19:25:03"
},
...
{
"id": "25",
"title": "Office Manager",
"created_at": "2014-01-09 12:55:11",
"updated_at": "2014-01-09 19:25:03"
}
...
]
Hope I've provided enough information and sorry if my description is a bit unclear, I struggled a bit to accurately describe my problem here.
Thanks in advance!
Note: The person shown here was generated and is not real.
please try this:
<select
ng-model='person.job'
ng-options='job.title for job in jobs track by job.id'>
</select>
e.g. extends the expression with: track by job.id