Ansible loop with array - arrays

Could someone let me know how we can create a code as below?
- name: TEST1
set_fact:
list_a: "{{ list_a + [item.json.SearchResult.resources] }}"
with_items:
- "{{ source_list.results[0] }}"
- "{{ source_list.results[1] }}"
- "{{ source_list.results[x] }}"
... (unknown how many items in result from API)
vars:
list_a: []
source_list.results[x] comes from an API result. The reason why I need to create an array is that the number of API result is maximum 100. But there are over 500 items.

Note: since we have no idea what you initial data structure looks like exactly, the below might not be 100% fitting your use case. For your next questions, please read How to ask and pay attention to the Minimal, complete and reproducible example section. Thanks
You are taking this the wrong way. Simply extract the attribute you need from each result using the map(attribute=x) Jinja2 filter.
For the below I inferred (see above note) that:
you called your API with ansible.builtin.uri in a loop to get batches of 100 results which are returned as a list in the SearchResult.ressources field
you want in the end a flattened list where all resources are at top level
- name: Show my list of single attributes
ansible.builtin.debug:
var: "source_list.results
| map(attribute='json.SearchResult.resources') | flatten"
You actually don't need to set_fact:
For a single use, just use the above expression directly in the relevant parameter (e.g. loop or a module param....) or eventually declare this in a var at task level.
If you want to reuse this in different parts of your playbook, just declare a var at play level and expand it anywhere once you have called your API and populated the source_list var. In that case, just add a default value to prevent an error if API was not yet called.
Example for the second case above in this pseudo playbook
---
- hosts: localhost
gather_facts: false
vars:
list_a: "{{ source_list.results | d([])
| map(attribute='json.SearchResult.resources') | flatten }}"
tasks:
- name: "This will return an empty list (i.e. [])
as we did not populate source_list yet"
ansible.builtin.debug:
var: list_a
- name: Call our API and register source_list
ansible.builtin.uri:
uri: https://my.api.com/api/v1/some/endpoint
# [... more parameters here ... ]
loop: "{{ my_list_of_ressources }}"
register: source_list
- name: "This will now return a populated list
after calling the API and registering source_list"
ansible.builtin.debug:
var: list_a
Now, to still give a direct answer to your initial question: you can construct that list iteratively inside a set_fact task. This is definitely not efficient as it involves a task running inside a loop (both unneeded as demonstrated above) and possibly on multiple hosts in your play. But for learning purpose, here it is:
- name: very inefficient way to get the same result as above
set_fact:
list_a: "{{ list_a | d([]) + item.SearchResult.resources }}"
loop: "{{ source_list.results }}"

After reading #Zeitounator's answer, I realized I was taking this the wrong way. I used the idea in the answer (json query and flatten) and I changed my task to the below which works as expected.
- name: Create ID list from API result data
set_fact:
list_a: "{{ source_list | json_query('results[*].json.SearchResult.resources[*].id') | flatten }}"
- name: API get again with id
uri:
url: "{{ request_url }}/{{ item }}"
...
register: result_data
with_items:
- "{{ list_a }}"

Related

Is There a Way to Have a Value Stored in an Array Every Time the Loop Occurs in Ansible?

I have an ansible-playbook which aims to display an A Record of a particular host from a DNS Server within Domain Controller. Here’s what I did on Ansible-Playbook:
Use powershell to obtain information related to A Record on the DNS Server.
Save it as a variable named test_var.
Divide the contents of the variable test_var into line by line.
Retrieves the important line containing the string host I'm looking for.
Take the important attributes of those important lines and show it as msg.
Here's the code:
# hostname and domain are necessary
---
- hosts: all
gather_facts: no
vars:
search_name: "{{hostname}}"
tasks:
- name: powershell query
win_shell: "Get-DnsServerResourceRecord -Name '{{hostname}}' -ZoneName '{{domain}}' -RRType A"
register: result1
when: (hostname is defined) and (domain is defined)
- set_fact:
test_var: "{{ result1.stdout_lines }}"
- name: pickup lines
set_fact:
important_lines: "{{ important_lines |default([]) + [item] }}"
with_items:
- "{{ test_var }}"
- name: find the line
set_fact:
target_line: "{{item}}"
when: item|trim is search(search_name)
loop: "{{ important_lines | flatten(1) }}"
- name: get all attributes
set_fact:
attribute_record: "{{ target_line.split()[1]|trim}}"
attribute_type: "{{ target_line.split()[2]|trim}}"
attribute_timestamp: "{{ target_line.split()[3]|trim}}"
attribute_timetolive: "{{ target_line.split()[4]|trim}}"
attribute_ipaddress: "{{ target_line.split()[5]|trim}}"
- name: print results
debug:
msg: "name: {{search_name}}, Ip Address: {{attribute_ipaddress}}"
And here's my DNS Server configuration:
And the results are as follows (host=test1):
However, I have a problem. In the Find the line task which runs the loop, the target_line variable stores only the last line at the end of the task. So, when the print results task is executed, only the last host and IP address are displayed. The question is, is there some way to have each line stored in an array every time the loop occurs? Thus, I can call the contents of the array to display it one by one. Thank you.
Here's the solution that I got:
# hostname and domain are necessary
---
- hosts: all
gather_facts: no
vars:
correct_line: []
search_name: "{{hostname}}"
tasks:
- name: powershell query
win_shell: "Get-DnsServerResourceRecord -Name '{{hostname}}' -ZoneName '{{domain}}' -RRType A"
register: result1
when: (hostname is defined) and (domain is defined)
- set_fact:
test_var: "{{ result1.stdout_lines }}"
- name: pickup lines
set_fact:
important_lines: "{{ important_lines |default([]) + [item] }}"
with_items:
- "{{ test_var }}"
- name: find the line
set_fact:
correct_line: "{{correct_line + [item]}}"
when: item|trim is search(search_name)
loop: "{{ important_lines | flatten(1) }}"
- name: print results
debug:
msg: "name: {{item.split()[0]|trim}}, Ip Address: {{item.split()[5]|trim}}"
loop: "{{ correct_line | flatten(1) }}"
And here's the result:

Can I create dynamic lists within Ansible vars_files?

I have a variables file that includes important info about our databases; the server they are on, the db version, the DB_HOME directory, etc. In the variables file, I would like to dynamically create lists that capture the unique values of those properties, so they can be easily iterated through in a task.
I have equivalent functionality by creating the list on the fly in a task's loop option, but that means repeating that loop syntax (violates DRY principle) and I would like less sophisticated Ansible colleagues to be able to use a pre-defined list.
example of the variables file databases.yml:
databases:
- name: test_db1
server: ora_901
listener: LISTENER_XYZ
version: '11.2.0.4'
oracle_home: '/app/oracle/product/11.2.0.4/db_home'
- name: test_db2
server: ora_902
listener: LISTENER_ABC
version: '11.2.0.4'
oracle_home: '/app/oracle/product/11.2.0.4/db_home'
## This didn't work... was hoping I could build this list dynamically
listeners:
- name: "{{ item }}"
loop: "{{ databases | map(attribute = 'listener') | list | unique }}"
servers:
- name: "{{ item }}"
loop: "{{ databases | map(attribute = 'server') | list | unique }}"
I would then use this loop through either the 'listeners' or 'servers' lists directly with some tasks.
When I tried a task that referenced the listeners variable, it failed. Referencing databases works and all items are returned, so I know it's getting some data from the vars_file...
- vars_files:
- vars/databases.yml
tasks:
- debug:
msg: "{{ databases }}"
- debug:
msg: "{{ listeners }}"
TASK [debug] **************************************************************************
ok: [FQDN] => {
"msg": [
{
"listener": "LISTENER_XYZ",
"name": "test_db1",
"oracle_home": "/app/oracle/product/11.2.0.4/db_home",
"server": "ora_901",
"version": "11.2.0.4"
},
{
"listener": "LISTENER_ABC",
"name": "test_db2",
"oracle_home": "/app/oracle/product/11.2.0.4/dbhome_1",
"server": "ora_902",
"version": "11.2.0.4"
},
fatal: [FQDN]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'item' is undefined\n\nThe error appears to have been in '/home/xxx/test_vars.yml': line 21, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - debug:\n ^ here\n"}
I would really like to keep these dynamic definitions in the same place as the server definitions, and don't see why it wouldn't be possible, I'm just sure I'm using the wrong mechanism.
Check this out from Ansible doc// Just to give you an idea, about "loop_control".
You can nest two looping tasks using include_tasks. However, by default Ansible sets the loop variable item for each loop. This means the inner, nested loop will overwrite the value of item from the outer loop. You can specify the name of the variable for each loop using loop_var with loop_control:
# main.yml
- include_tasks: inner.yml
loop:
- 1
- 2
- 3
loop_control:
loop_var: outer_item
# inner.yml
- debug:
msg: "outer item={{ outer_item }} inner item={{ item }}"
loop:
- a
- b
- c

Ansible - How do you reference the item currently being looped in another loop

I have a variable that looks like (the reality is a lot longer):
deploy_dirs:
conf:
base: '/etc/projects/'
subs:
apache: '/httpd'
modsec: '/modsec'
php: '/php'
haproxy: '/haproxy'
varnish: '/varnish'
logs:
base: '/var/log/projects/'
subs:
apache: '/httpd'
modsec: '/modsec'
php: '/php'
haproxy: '/haproxy'
varnish: '/varnish'
Then I have a task that looks like:
- debug:
msg: '{{ item.0.value.base }}SOME-OTHER-VAR{{ item.1 }}'
loop: '{{ deploy_dirs | dict2items | subelements("value.subs") }}'
Which I would love to output:
/etc/projects/SOME-OTHER-VAR/httpd
/etc/projects/SOME-OTHER-VAR/modsec
/etc/projects/SOME-OTHER-VAR/php
/etc/projects/SOME-OTHER-VAR/haproxy
/etc/projects/SOME-OTHER-VAR/varnish
/var/log/projects/SOME-OTHER-VAR/httpd
/var/log/projects/SOME-OTHER-VAR/modsec
/var/log/projects/SOME-OTHER-VAR/php
/var/log/projects/SOME-OTHER-VAR/haproxy
/var/log/projects/SOME-OTHER-VAR/varnish
These variables get re-used in various places all over my playbooks, and the structure makes sense to me and seems to be straightforward enough.
I've tried changing the subelements() filter in to the product() filter (with_nested equivalent), but it doesn't seem able to reference the variable from the earlier loop in the same way that the subelements() filter can.
Obviously the problem with the above is that subelements() needs a list, not a dictionary. I'm unable to convert the "value.subs" argument in to a list, as I can't find a way to nest filters like that..?
Does anyone have any ideas on how to get this working?
Thanks!
UPDATE: If I also have:
www:
base: '/srv/www/projects/'
subs: {}
And I just want it to output:
/srv/www/projects/SOME-OTHER-VAR
Do you know how I can swing the subs bit so that it runs and generates the expected result? At the moment it just skips it. i.e., no loop occurs for www.
You can transform your subs dict in a list with json_query before using the subelements filter.
- debug:
msg: "{{ item.0.base }}SOME-OTHER-VAR{{ item.1 }}"
loop: >-
{{
deploy_dirs
| json_query('*[].{base: base, subs: subs.*}')
| subelements('subs')
}}
Have a look at jmespath documentation for the syntax. json_query is really handy when your data structure gets complex.
This solution will not loop over the elements with an empty subs element like the 'www' map entry in your example and I have no "clean and easy" way to do it in a single task. You can easily have a second task looping over the elements with empty subs => deploy_dirs | json_query("* | [?!subs]")
I guess your final goal is not to have debug msg written to the screen. If you really need a single structure to make your final task (e.g. create directories) in a single step, you can populate a var with set_fact and the above techniques and then use it to acheive your real task
- name: Get all elements with subs to loop over
set_fact:
my_var: >-
{{
deploy_dirs
| json_query('*[].{base: base, subs: subs.*}')
| subelements('subs')
}}
- name: Add entries with empty item1 for elements with empty subs
set_fact:
my_var: >-
{{
my_var
+
[[item, '']]
}}
loop: >-
{{ deploy_dirs | json_query("* | [?!subs]") }}
- name: This will be replaced with a real task
debug:
msg: "{{ item.0.base }}SOME-OTHER-VAR{{ item.1 }}"
loop: "{{ my_var }}"

Loop vars with loop_control and loop_var Ansible

I would need to loop some vars inside an include_task.
What is currently working:
- include_tasks: example.yml
loop: "{{ flowers|flatten(levels=1) }}"
loop_control:
loop_var: flower
I would like to achieve something like this:
- include_tasks: example.yml
loop: - "{{ flowers|flatten(levels=1) }}"
- "{{ cars|flatten(levels=1) }}"
loop_control:
loop_var:
- flower
- car
In the example.yml, I will have 2 sections: "flowers" and "cars". In "flowers" I should trigger just "flower" var and in "cars" I should trigger just "car" var.
Is any way to achieve this using loop and loop_var?
Thank you in advance.
You can have one loop_var only. But this loop_var may be any data-structure that fits your problem. You can for example loop combined flowers and cars, and select the elements in the included tasks.

Ansible looping over nested variables

I have a set of variables which define FQDNs.
domains:
- erp: erp.mycompany.com
- crm: crm.mycompany.com
- git: git.mycompany.com
Indeed, I both need to loop over them and access them namely (in a template file). So accessing them like domains.erpworks like a charm. But I can't get ansible to loop over these.
Obviously, if I do:
- name: Print domains
debug:
msg: test {{ item }}
with_items:
- "{{ domains }}"
It prints both the key and the value… And if I do:
- name: Print domains
debug:
msg: test {{ domains[{{ item }}] }}
with_items:
- "{{ domain }}"
But that doesn't work. I also tried the hashes form as mentionned in the docs, but didn't get any luck either…
Finally, I had to use a dict.
It didn't work the first time because unlike with_items, which has the items going each on their own line, with_dict is a one liner without - before the element to loop through.
domains:
erp:
address: erp.mycompany.com
crm:
address: crm.mycompany.com
git:
address: git.mycompany.com
# used by letsencrypt
webserverType: apache2
withCerts: true
tasks:
- name: Print phone records
debug:
msg: "{{ item.value.address }}"
with_dict: "{{ domains }}"
# I can still access a given domain by its name when needed like so:
{{ domains.erp.address }}
Looks like you figured out your issue. Your original attempt uses a list of dictionaries that do not contain the same keys, making it difficult to access the values uniformly across each list item.
Your second solution creates a dictionary where the keys refer to other dictionaries.
Another solution than what you posted if you still wanted to use a list:
- hosts: localhost
vars:
domains:
- name: erp
address: erp.mycompany.com
- name: crm
address: crm.mycompany.com
- name: git
address: git.mycompany.com
tasks:
- name: Print phone records
debug:
msg: "{{ item.address }}"
with_items: "{{ domains }}"
To me this approach is simpler but your second approach works as well.

Resources