Loop isn't iterating over last line of list - loops

I have created a template within a playbook which I want to iterate through with a list of hashes. The output of this I want to add to another var to use in a following module.
The template works and the loop looks like it works, but it never adds the last item in the list. I have recreated it in a test play.
---
- hosts: localhost
tasks:
- name: Init
set_fact:
foo: []
fqdn: "test.com"
template: []
- name: portlist
set_fact:
portlist:
- { port: 9091, index: 1 }
- { port: 9092, index: 2 }
- { port: 9093, index: 3 }
- { port: 9094, index: 4 }
- name: generate policy
set_fact:
template:
- name: "traffic to {{ item.port }}"
index: "{{ item.index }}"
match:
desc: "[{{ item.port }}]" # The field needs to be passed as a list
name: "{{ fqdn }}_{{ item.port }}"
port: "{{ item.port }}"
foo: "{{ foo + template }}"
loop: "{{ portlist }}"
- debug:
var: foo
I understand I can make this play smaller with defaults rather than initializing vars but this felt easier to read for troubleshooting.
The play results in a list of hashes which then I can input into a policy module. However it only ever gives me 3 items in the list and misses off the last item in the portlist.
TASK [debug] ***************************************************
ok: [localhost] => {
"foo": [
{
"action": {
"ref": "test.com_9091",
"this": true
},
"enable": true,
"index": "1",
"match": {
"port": {
"criteria": "IS_IN",
"port": [
9091
]
}
},
"name": "traffic to 9091"
},
{
"action": {
"ref": "test.com_9092",
"this": true
},
"enable": true,
"index": "2",
"match": {
"port": {
"criteria": "IS_IN",
"port": [
9092
]
}
},
"name": "traffic to 9092"
},
{
"action": {
"ref": "test.com_9093",
"this": true
},
"enable": true,
"index": "3",
"match": {
"port": {
"criteria": "IS_IN",
"port": [
9093
]
}
},
"name": "traffic to 9093"
}
]
}

Your problem is caused by the fact that variables defined via set_fact aren't available until after the set_fact task has finished. This means that when you set:
foo: "{{ foo + template }}"
You see the value of template from the previous loop iteration.
One way of dealing with this is to rewrite your set_fact task to set foo directly:
---
- hosts: localhost
gather_facts: false
vars:
fqdn: "test.com"
portlist:
- {port: 9091, index: 1}
- {port: 9092, index: 2}
- {port: 9093, index: 3}
- {port: 9094, index: 4}
tasks:
- name: generate policy
set_fact:
foo: >-
{{
foo + [{
'name': 'traffic to {}'.format(item.port),
'index': item.index,
'match': {
'desc': "[{}]".format(item.port),
'name': '{}_{}'.format(fqdn, item.port),
'port': item.port
}
}]
}}
vars:
foo: []
loop: "{{ portlist }}"
- debug:
var: foo
This will output:
TASK [debug] *********************************************************************************************************************************************************************************
ok: [localhost] => {
"foo": [
{
"index": 1,
"match": {
"desc": "[9091]",
"name": "test.com_9091",
"port": 9091
},
"name": "traffic to 9091"
},
{
"index": 2,
"match": {
"desc": "[9092]",
"name": "test.com_9092",
"port": 9092
},
"name": "traffic to 9092"
},
{
"index": 3,
"match": {
"desc": "[9093]",
"name": "test.com_9093",
"port": 9093
},
"name": "traffic to 9093"
},
{
"index": 4,
"match": {
"desc": "[9094]",
"name": "test.com_9094",
"port": 9094
},
"name": "traffic to 9094"
}
]
}
If you find your template-based solution more readable, you could rewrite it using two set_fact tasks like this:
---
- hosts: localhost
gather_facts: false
vars:
fqdn: "test.com"
portlist:
- {port: 9091, index: 1}
- {port: 9092, index: 2}
- {port: 9093, index: 3}
- {port: 9094, index: 4}
tasks:
- name: generate policy
set_fact:
template:
name: "traffic to {{ item.port }}"
index: "{{ item.index }}"
match:
desc: "[{{ item.port }}]" # The field needs to be passed as a list
name: "{{ fqdn }}_{{ item.port }}"
port: "{{ item.port }}"
loop: "{{ portlist }}"
register: foo
- set_fact:
foo: "{{ foo.results | map(attribute='ansible_facts.template') | list }}"
- debug:
var: foo

Related

Registering output to complex variable and placing conditionals on sub-components

I'm getting a bit lost with next variable types and am hoping for some direction in a specific task please:
The Goal:
Based on a list of username:publickey values. I'd like to:
ensure the user exists on the target system
if the user does exist then:
- ensure the "/home/$user/.ssh/authorized_keys" file exists with the correct permissions through the path.
The Scene:
I have a variable:
ssh_vars:
auth_keys:
bob: "bobs_public_key_string"
anne: "annes_public_key_string"
anon: "anons_public_key_string
I need to iterate over this variable and for each auth_keys item call a tasklist:
- name: loop through the auth_keys and call ssh_dirs.yml for each
ansible.builtin.include_tasks: "ssh_dirs.yaml"
loop: "{{ ssh_vars.auth_keys }}"
However, I only really want to do this when the auth_key(key) is a user which already exists on the host.
I have been playing with getent, within "ssh_dirs.yaml":
- name: "Ensure the user exists on the target system"
ansible.builtin.getent:
database: passwd
key: "{{ item.name }}"
fail_key: false
register: userlookup
which creates what i think is a list of dictionaries:
ok: [ans-client.local] => {
"userlookup": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_facts": {
"getent_passwd": {
"bob": [
"x",
"1003",
"1003",
"",
"/home/bob",
"/usr/bin/bash"
]
}
},
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"database": "passwd",
"fail_key": false,
"key": "bob",
"service": null,
"split": null
}
},
"item": {
"key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDvIZuaBhAIGShw21rkvgqyvNePunbVs6OtOBhYJOY2P anne#ans-server",
"name": "bob"
}
},
{
"ansible_facts": {
"getent_passwd": {
"anne": [
"x",
"1000",
"1000",
"anne",
"/home/anne",
"/bin/bash"
]
}
},
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"database": "passwd",
"fail_key": false,
"key": "anne",
"service": null,
"split": null
}
},
"item": {
"key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKr/76O3hLJlcyZuy7EJxf7sC1z9BSHMuxGsFGBibJY3 anne#ans-server",
"name": "anne"
}
},
{
"ansible_facts": {
"getent_passwd": {
"anon": null
}
},
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"database": "passwd",
"fail_key": false,
"key": "anon",
"service": null,
"split": null
}
},
"item": {
"key": "SOMEKEY",
"name": "anon"
},
"msg": "One or more supplied key could not be found in the database."
}
],
"skipped": false
}
}
But I can't figure out how to isolate this list to ensure the include_tasks: is not called if the user doesn't exist.
- name: loop through the auth_keys and call ssh_dirs.yml for each
ansible.builtin.include_tasks: "ssh_dirs.yaml"
loop: "{{ ssh_vars.auth_keys }}"
when: userlookup.results.???????
How can I figure out how to reference this nested variable, and how best to isolate a non-missing user?
Something like userlookup.results.msg is not defined might work but it's very loose - is there something better I'm missing?
See registering variables with loop
The global idea is
loop over you var to get the existing/unavailable users
loop over the results of that previous task for you next one. The original loop variable is available in the item key of each result and you can filter as you like.
For your particular case, in a nutshell (untested):
- name: Ensure the user exists on the target system
ansible.builtin.getent:
database: passwd
key: "{{ item.name }}"
register: userlookup
ignore_errors: true
loop: "{{ ssh_vars.auth_keys }}"
- name: Call ssh_dirs.yml for each existing users
ansible.builtin.include_tasks: ssh_dirs.yaml
loop: "{{ userlookup.results | select('success') }}"
loop_control:
loop_var: user_checked
vars:
userloop: "{{ user_checked.item }}"
I think I've solved it, although maybe there's a better thing to look for in the getent response than just the msg?
The logic and variable reference which works:
- name: "Ensure the user exists on the target system"
ansible.builtin.getent:
database: passwd
key: "{{ item.name }}"
fail_key: false
register: userlookup
loop: "{{ ssh_vars.auth_keys }}"
- name: Build a list of usernames which don't exist on the remote host (missing_users)
ansible.builtin.set_fact:
missing_users: "{{ missing_users | default([]) + [usercheck.item.name | string] }}"
loop: "{{ userlookup.results }}"
loop_control:
loop_var: usercheck
when: usercheck.msg is defined
- name: loop through the users and ensure the necessary user folders and files are there
ansible.builtin.include_tasks: "ssh_dirs.yaml"
loop: "{{ ssh_vars.auth_keys }}"
loop_control:
loop_var: userloop
when: userloop.name not in missing_users
Although this is still checking the msg: output mindlessly so only a partial solution

Bad file descriptor in Ansible when read JSON content is numeric

Below is my JSON file:
[
{
"?xml": {
"attributes": {
"encoding": "UTF-8",
"version": "1.0"
}
}
},
{
"domain": [
{
"name": "mydom"
},
{
"domain-version": "12.2.1.3.0"
},
{
"server": [
{
"name": "AdminServer"
},
{
"ssl": {
"name": "AdminServer"
}
},
{
"listen_port": "12400"
},
{
"listen_address": "mydom.host1.bank.com"
}
]
},
{
"server": [
{
"name": "myserv1"
},
{
"ssl": [
{
"name": "myserv1"
},
{
"login-timeout-millis": "25000"
}
]
},
{
"listen_port": "22421"
}
]
}
]
}
]
Here is the code to get the listen port number form the json
---
- name: ReadJsonfile
hosts: localhost
tasks:
- name: Display the JSON file content
shell: "cat this.json"
register: result
- name: save the Json data to a Variable as a Fact
set_fact:
jsondata: "{{ result.stdout | from_json }}"
- name: create YML for server name with Listen port
shell: "echo {{ server.0.name }}_httpport: {{ httpport[0].listen_port }}>>{{ playbook_dir }}/wlsdatadump.yml"
loop: "{{ jsondata[1].domain }}"
vars:
server: "{{ item.server | selectattr('name', 'defined') }}"
httpport: "{{ item.server | selectattr('listen_port', 'defined') | list }}"
when: item.server is defined and (item.server | selectattr('listen_port', 'defined')) != []
I get the below error when executing the play
TASK [create YML for server name with Listen port] ************************************************************
skipping: [localhost] => (item={'name': 'mydom'})
skipping: [localhost] => (item={'domain-version': '12.2.1.3.0'})
failed: [localhost] (item={'server': [{'name': 'AdminServer'}, {'ssl': {'name': 'AdminServer'}}, {'listen_port': '12400'}, {'listen_address': 'mydom.host1.bank.com'}]}) => {"ansible_loop_var": "item", "changed": true, "cmd": "echo AdminServer_httpport: 12400>>/web/aes/admin/playbooks/dump.yml", "delta": "0:00:00.007706", "end": "2022-03-17 04:43:24.665832", "item": {"server": [{"name": "AdminServer"}, {"ssl": {"name": "AdminServer"}}, {"listen_port": "12400"}, {"listen_address": "mydom.host1.bank.com"}]}, "msg": "non-zero return code", "rc": 1, "start": "2022-03-17 04:43:24.658126", "stderr": "/bin/sh: 12400: Bad file descriptor", "stderr_lines": ["/bin/sh: 12400: Bad file descriptor"], "stdout": "", "stdout_lines": []}
If i change the port number from numeric to non-numeric say change "12400" to "portfirst" the playbook works fine.
This issue may have to do with the data type.
Can you please suggest how can i overcome this error?
you create a file listenport.j2 in folder templates:
{% for item in jsondata[1].domain if item.server is defined and (item.server | selectattr('listen_port', 'defined')) != [] %}
{{ item.server.0.name }}_httpport: {{ (item.server | selectattr('listen_port', 'defined')| list).0.listen_port }}
{% endfor %}
playbook:
- name: ReadJsonfile
hosts: localhost
tasks:
- name: Display the JSON file content
shell: "cat ./file2.json"
register: result
- name: save the Json data to a Variable as a Fact
set_fact:
jsondata: "{{ result.stdout | from_json }}"
- name: template
template:
src: listenport.j2
dest: "{{ playbook_dir }}/wlsdatadump.yml"
result in file wlsdatadump.yml:
AdminServer_httpport: 12400
myserv1_httpport: 22421
if you are working on lot of hosts (not only on localhost) and just want to create the file on localhost, use delegate_to: localhost inside the task

how to loop over hostvars of register output

I'm struggling to loop over hostvars of the registered output of all the hosts in dynamic inventory.
Here is the code.
$cat collect.yaml
---
- hosts: "{{ env }}"
become: True
tasks:
- name: Get dockerinfo
docker_host_info:
containers: yes
register: result
- name: Debug dockerInfo
debug:
var: result.containers
- name: dynamic grouping
add_host:
name: "{{ item[0] }}"
groups: "{{ item[1].Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
loops:
- "{{ ansible_play_batch }}"
- "{{ hostvars[item].result.containers }}"
Error i get is item not defined. I would want the hosts refer to their respective result.containers. Not sure on how to use hostvars for the host to refer their respective result.containers.
Here is result.container output.
TASK [Debug dockerInfo]
ok: [vm1.nodekite.com] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
},
{
"Image": "docker.local/egacustomer:1.0.1",
},
]}
ok: [vm2.nodekite.com ] => {
"result.containers": [
{
"Image": "ca.docker/webproxyui:1.0.0",
},
{
"Image": "cna-docker-local/lega-customer:1.0.1",
},
]}
Here is the what i'm trying to achieve
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"webproxy"
],
"host_name": "vm1.nodekite.com",
},
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer"
],
"host_name": "vm1.nodekite.com",
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"webproxy" >> this should be webproxyui
],
"host_name": "vm2.nodekite.com",
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer" >> this should be lega-customer
],
"host_name": "vm2.nodekite.com",
},
Any help would be greatly appreciated.
I would run this task using Images Names
- hosts: "{{ group }}"
gather_facts: false
become: true
become_method: sudo
tasks:
- name: stop or restart docker containers
command: "docker {{ state }} {{ container_name }}"
How about just group_by paired with the loop?
- hosts: "{{ env }}"
become: True
tasks:
- name: Get dockerinfo
docker_host_info:
containers: yes
register: result
- debug:
var: result.containers
- group_by:
key: "container_{{ item.image | regex_replace('.*?/(.*?):.*', '\\1') }}"
loop: "{{ result.containers }}"
- debug:
var: group_names
You don't need to add the prefix. But this would add each host to groups with their container image prefixes. You should be able to use the group later in the play or playbook.

Ansible: How to loop over registered output of dynamic hosts in the next task

I am struggling to loop over registered results.containers for all the hosts in the dynamic inventory.
Here is the code.
$cat collect.yaml
---
- hosts: "{{ env }}"
become: True
tasks:
- name: Get dockerinfo
docker_host_info:
containers: yes
register: result
- name: Debug dockerInfo
debug:
var: result.containers
- name: dynamic grouping
add_host:
name: "{{ item[0] }}"
groups: "{{ item[1].Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
container_name: '{{ item[1].Names[0] | regex_replace("^/", "") }}'
with_nested:
- "{{ ansible_play_batch }}"
- "{{ result.containers }}"
Here is result.containers output.
TASK [Debug dockerInfo]
ok: [vm1.nodekite.com] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
"Names": [
"/customer1"
],
},
{
"Image": "docker.local/egacustomer:1.0.1",
"Names": [
"/webproxy"
],
},
]}
ok: [vm2.nodekite.com ] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
"Names": [
"/webproxyui"
],
},
{
"Image": "cna-docker-local/lega-customer:1.0.1",
"Names": [
"/webproxy"
],
},
]}
ok: [vm3.nodekite.com ] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
"Names": [
"/webproxy"
],
},
{
"Image": "local.docker/saga-customer:1.0.1",
"Names": [
"/customerr
],
},
]}
Right now item[1].Image and item[1].Names[0] is only taken from first host's(vm1.nodekite.com) results.containers output. I would like to loop over for every hosts. So that, I could create dynamic group for all the hosts with their respective containers. With my code, hosts vm1,vm2,vm3 all are referring to vm1.nodekite.com's result.containers but i want the hosts to refer to their respective containers. Any help would be greatly appreciated.
I have update dynamic grouping task ouput for clarification.
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"webproxy"
],
"host_name": "vm1.nodekite.com",
"host_vars": {
"container_name": "customer1" }
},
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer"
],
"host_name": "vm1.nodekite.com",
"host_vars": {
"container_name": "webproxy" }
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"webproxy" >> this should be webproxy
],
"host_name": "vm2.nodekite.com",
"host_vars": {
"container_name": "customer1" } >> this should be webproxyui
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer" >> this should be lega-customer
],
"host_name": "vm2.nodekite.com",
"host_vars": {
"container_name": "webproxy" } >> this should be webproxy
},
if you see vm2 is still referring to vm1's result.containers output.
when i try this...i get item not defined error.
- name: adding it to groups using images
add_host:
name: "{{ item[0] }}"
groups: "{{ item[1].Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
container_name: '{{ item[1].Names[0] | regex_replace("^/", "") }}'
loop:
- "{{ ansible_play_batch }}"
- "{{ myresult.containers }}"
vars:
myresult: "{{ hostvars[item].result }}"
run_once: true
Q: "Hosts shall refer to their respective containers."
A: Use hostvars. For example
- name: dynamic grouping
debug:
msg:
- "name: {{ item }}"
- "groups: {{ my_result.containers|
map(attribute='Image')|
map('regex_replace', '.*?/(.*?):.*', '\\1')|
list }}"
- "container_names: {{ my_result.containers|
map(attribute='Names')|
map('regex_replace', '\/', '')|
list }}"
loop: "{{ ansible_play_batch }}"
vars:
my_result: "{{ hostvars[item].result }}"
run_once: true
gives
ok: [vm1.nodekite.com] => (item=vm1.nodekite.com) =>
msg:
- 'name: vm1.nodekite.com'
- 'groups: [''webproxy'', ''egacustomer'']'
- 'container_names: ["[''customer1'']", "[''webproxy'']"]'
ok: [vm1.nodekite.com] => (item=vm2.nodekite.com) =>
msg:
- 'name: vm2.nodekite.com'
- 'groups: [''webproxy'', ''lega-customer'']'
- 'container_names: ["[''webproxyui'']", "[''webproxy'']"]'
ok: [vm1.nodekite.com] => (item=vm3.nodekite.com) =>
msg:
- 'name: vm3.nodekite.com'
- 'groups: [''webproxy'', ''saga-customer'']'
- 'container_names: ["[''webproxy'']", "[''customer'']"]'
(Feel free to fit the code to your needs.)
I was having an issue of getting the item passed into the name below to be a plain string of: item='nginx' and not item='[u'/nginx]'
To get around this, I did the following:
- name: Get docker containers
become: docker
community.docker.docker_container
containers: yes
register: docker_info
- name: Stop running containers
become: docker
community.docker.docker_container
name: "{{ item }}"
state: stopped
loop: "{{ docker_info.containers | sum(attribute='Names', start=[]) | map('regex_replace','\\/','') | list }}"
when: item in apps.split(,)
In this case the apps is a comma deliminated string variable I passed into the ansible playbook to limit which apps to stop.
The sum piece, flattens the Names of all the apps running into a single list.
The regex piece removes the / in the Names parameter

How to access register variable in task

This is my output of EC2 instance. I am trying to access "instance_type".
And here is my task.
ec2:
key_name: redhat
group: MY_EC2
instance_type: t2.micro
image: ami-cfe4b2b0
region: us-east-1
zone: us-east-1a
wait: true
exact_count: 1
count_tag:
name: MyProjectInstances
instance_tags:
name: Ansible
register: ec2
- set_fact:
inst: "{{ ec2 }}"
- debug:
msg: "{{ inst }}"
I can reach Instances block through trying this.
debug:
msg: "{{ inst.instances }}" but cannot go further, getting error of undefined variable.
#
Output:
ok: [localhost] => {
"msg": {
"changed": true,
"failed": false,
"instance_ids": [
"i-0be089202b191769e"
],
"instances": [
{
"ami_launch_index": "0",
"architecture": "x86_64",
"block_device_mapping": {
"/dev/xvda": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-02b129004f1a5fb89"
}
},
"dns_name": "ec2-34-204-84-170.compute-1.amazonaws.com",
"ebs_optimized": false,
"groups": {
"sg-06c09a2c83d7b1a96": "MY_EC2"
},
"hypervisor": "xen",
"id": "i-0be089202b191769e",
"image_id": "ami-cfe4b2b0",
"instance_type": "t2.micro",
"kernel": null,
"key_name": "redhat",
"launch_time": "2018-07-15T14:34:43.000Z",
"placement": "us-east-1a",
"private_dns_name": "ip-172-31-35-24.ec2.internal",
"private_ip": "172.31.35.24",
"public_dns_name": "ec2-34-204-84-170.compute-1.amazonaws.com",
"public_ip": "34.204.84.170",
"ramdisk": null,
"region": "us-east-1",
"root_device_name": "/dev/xvda",
"root_device_type": "ebs",
"state": "running",
"state_code": 16,
"tags": {
"name": "Ansible"
},
"tenancy": "default",
"virtualization_type": "hvm"
}
Please try as below::
- name: Get instance Type
debug: msg={{ inst | json_query('instances[].instance_type') }}

Resources