I have an Azure Data Factory pipeline with an array as parameter with potentially duplicate (simple string) values.
How can I remove the duplicates from the array without using a custom activity?
I defined a variable of type array and applied a union on the parameter itself. This removes the duplicate values.
Like #union(pipeline().parameters.FilePaths,pipeline().parameters.FilePaths).
Related
i have planner buckets (Tasks, in progress, Backlog) and I want to create a new Bucket depending on elements in a SharePoint list.
But I can't save the bucket names to an array and then add the missing value e.g. like "on hold" and the go trough the array again. It always set my array to blank again.
Maybe you can help me. Here is my PowerAutomate Flow so far:
You could use a couple of Select actions to create two arrays which you can use for the comparison. With a Filter Array you can find the ones which are missing. The output of that Filter Array can be used to loop through and create the missing buckets.
Below is an example of that approach.
1. In both Select actions the Map field is switched to text mode to only get Arrays with values (without keys)
2. In the Filter Array this is used in the From
body('Select_-_SharePoint_Items')
3. And this is the expression used in the advanced mode
#not(contains(body('Select_-_Existing_Buckets'), item()))
I have an Azure Data Factory Pipeline with a Copy Activity. Source is a JSON file (some generated pipeline variables too) and Sink is a Cosmos SQL Collection. Before Mapping I imported the schema and a particular field from the sink collection has been identified to be of type array [] with string elements.
The problem however is that this field mapping now only accepts a STRING and not an Array of strings. I am not sure if this a bug with ADF or something to do with how I genrated the source variable for this mapping.
If the source variable accessGroups is of string the copy works fine(it shouldn't though right???). However if I substitute this with a variable containing an array of strings I get the following error:
Failed to convert the value in 'value' property to 'System.String' type. Please make sure the payload structure and value are correct. Object must implement IConvertible.
Just to be clear, I am mapping the source variable accessGroups to ['accessGroups'] and not ['accessGroups'][0]. Besides ADF does not allow mapping source variables to individual array elements anyway.
Any one faced this issue before or aware of any workaround?
I am trying to pass an array as query parameter in Postman.
I am calling DELETE method to delete a user from list of databases. It is expecting list of database names as array in the query parameter. When I pass as given below, I am getting error.
{"code":2,"message":"data_services must be an array and not empty"}
Please let me know, how can I pass an array as query parameter in Postman.
You need to create 2 keys with the same name and add postfix []
You can specify the parameter key multiple times with different values.
Don't use square brackets or numbers like an array in code.
This will result in a query string like so:
?data_services=somename&data_services=anothername
I mis-read the DELETE method. It was expecting array of databases in the body of the Http body section, instead of on query parameter section. I provided the array as given below and it got succeeded.
The below is the way, I passed array to Http request in the body section.
can you check below link it is useful for you
https://medium.com/#darilldrems/how-to-send-arrays-with-get-or-post-request-in-postman-f87ca70b154e
I would like to know which field are array in my index in ES 6.8.3.
When I retrieve the mapping of the index via Kibana API :
GET structured_data/_mapping
I only get the kind of the value within the array.
It is a problem because when I want to load the data from spark, if there is an array field, I need to specify it otherwise it will raise an error.
Thanks in advance,
Louis
there is no dedicated data type array in Elasticsearch. Look at this link you don't have to specify anything specific in the mapping to store an array of values. Look in your source data if that contains an array, ES should take that array, given all the data inside that array is same data type
I need to search an array function for specific values in hive.
I have a table that creates a row for each event. I used collect_set(event_id) to roll the table up to the person level. I now need to search the array to put users into separate categories. I tried using a case when statement, find_in_set, and in_file but none of those functions work on arrays.
Any ideas? Can I convert an array into a string? Ideally I would have used a group_concat function in SQL - but hive does not support it
ARRAY_CONTAINS(events, search_param) will return a Boolean if search_param is found in the events array.
It's available in Hive 0.7.1, and possibly earlier versions.