Variable file
I have a variable file
fruits:
- apple
- banana
- orange
- strawberry
- grapes
- papaya
users:
- user_name: 'john'
user_age: 45
- user_name: 'yash'
user_age: 95
- user_name: 'srk'
user_age: 52
- user_name: 'alia'
user_age: 26
Playbook Tasks
and my ansible tasks, just trying to make a text file and adding variables in file in vertical order.
- hosts: localhost
gather_facts: true
vars_files:
- variables.var # this is my variable file in the same dir that playbook have.
tasks:
- name: add fruits to the list
lineinfile:
create: yes
line: "{{ item }}"
path: /home/ansible/ansible-demo2/fruits.txt
loop:
- "{{ fruits|flatten }}"
- name: add uses to the list
lineinfile:
create: yes
line: "{{ item.user_name }} ==> {{ item.user_age }}"
path: /home/ansible/ansible-demo2/users.txt
loop:
- "{{ users|flatten(levels=1) }}"
Errors
But I am getting weird behavior. Below is the output of fruits task and error of the users task.
TASK [add fruits to the list] ***************************************************************************************************************************
ok: [localhost] => (item=[u'apple', u'banana', u'orange', u'strawberry', u'grapes', u'papaya'])
[WARNING]: The value ['apple', 'banana', 'orange', 'strawberry', 'grapes', 'papaya'] (type list) in a string field was converted to u"['apple',
'banana', 'orange', 'strawberry', 'grapes', 'papaya']" (type string). If this does not look like what you expect, quote the entire value to ensure it
does not change.
TASK [add uses to the list] *****************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'list object' has no attribute 'user_name'
\n\nThe error appears to be in '/home/ansible/ansible-demo2/myansible.yml': line 18, column 7, but may\nbe elsewhere in the file depending on the exact s
yntax problem.\n\nThe offending line appears to be:\n\n\n - name: add uses to the list\n ^ here\n"}
Expected Output
List in vertical order in text file.
Comment on question:
I am amazed every thing works fine with with_items and not with loop, even user_name variable is defined but still it is saying undefined. Now I unable to find out what's wrong going on.
Reference:
Here is the doc that I am referencing : Migrating from with_X to loop
Edit: Variables debug output
I debug the variables. output is below
TASK [debug] ********************************************************************************************************************************************
ok: [localhost] => {
"fruits": [
"apple",
"banana",
"orange",
"strawberry", "grapes", "papaya"
]
}
TASK [debug] ********************************************************************************************************************************************
ok: [localhost] => {
"users": [
{
"user_age": 45,
"user_name": "john"
},
{
"user_age": 95,
"user_name": "yash"
},
{
"user_age": 52,
"user_name": "srk"
},
{
"user_age": 26,
"user_name": "alia"
}
]
}
Q: *"[WARNING]: The value ['apple', ... ] (type list) in a string field was converted to u"['apple', ... ]" (type string).
A: From the code, it's not clear what is the reason for this conversion. The data and the playbook below work as expected.
shell> cat data.yml
fruits:
- apple
- banana
- orange
- strawberry
- grapes
- papaya
users:
- user_name: 'john'
user_age: 45
- user_name: 'yash'
user_age: 95
- user_name: 'srk'
user_age: 52
- user_name: 'alia'
user_age: 26
shell> cat playbook.yml
- hosts: localhost
vars_files:
- data.yml
tasks:
- name: add fruits to the list
lineinfile:
create: yes
line: "{{ item }}"
path: fruits.txt
loop: "{{ fruits }}"
- name: add uses to the list
lineinfile:
create: yes
line: "{{ item.user_name }} ==> {{ item.user_age }}"
path: users.txt
loop: "{{ users }}"
Give
shell> cat fruits.txt
apple
banana
orange
strawberry
grapes
papaya
shell> cat users.txt
john ==> 45
yash ==> 95
srk ==> 52
alia ==> 26
Q: "2 variables fruits1 and fruits2 ... append their data into single file ... in single task with 2 vars"
A: With the modified data
shell> cat data.yml
fruits1:
- apple
- banana
- orange
fruits2:
- strawberry
- grapes
- papaya
this task gives the same result
- name: add fruits to the list
lineinfile:
create: yes
line: "{{ item }}"
path: fruits.txt
loop: "{{ fruits1 + fruits2 }}"
See Append list variable to another list in Ansible.
Comment on the question: "I am amazed everything works fine with with_items and not with loop"
A: See Comparing loop and with_*.
Related
I have made an ansible script to collect IP in the nginx log and then the IP that has been collected will be analyzed, if a malicious IP is detected it will be blocked.
The idea is that in the ansible script each IP analyzed is not created individually but using a loop function
But I'm having a hard time making a looping script in ansible, can anyone help?
Here is the script that i have created
- name: Execute a command using the shell module
shell: awk '{print $1}' /var/log/nginx/*.log | sort | uniq -c | sort -nr| head -5
register: results
- debug: var=results.stdout_lines
- name: Write Output
local_action: shell echo "{{ results.stdout_lines }}" > /tmp/output
#Collect IP
- name: Check 1st IP
local_action: shell cat /tmp/output | awk '{ print $3 }'| sed 's/.$//'|sed 's/.$//'
register: ip1
- debug: var=ip1.stdout
- name: Check 2nd IP
local_action: shell cat /tmp/output | awk '{ print $6 }'| sed 's/.$//'|sed 's/.$//'
register: ip2
- debug: var=ip2.stdout
- name: Check 3rd IP
local_action: shell cat /tmp/output | awk '{ print $9 }'| sed 's/.$//'|sed 's/.$//'
register: ip3
- debug: var=ip3.stdout
- name: Check 4th IP
local_action: shell cat /tmp/output | awk '{ print $12 }'| sed 's/.$//'|sed 's/.$//'
register: ip4
- debug: var=ip4.stdout
- name: Check 5th IP
local_action: shell cat /tmp/output | awk '{ print $15 }'| sed 's/.$//'|sed 's/.$//'
register: ip5
- debug: var=ip5.stdout
#Anlyze IP
- name: Analyze 1st IP
local_action: shell /usr/bin/abuseipdb -C {{ ip1.stdout }} -o plaintext
register: vb1
- debug: var=vb1.stdout
- name: Analyze 2nd IP
local_action: shell /usr/bin/abuseipdb -C {{ ip2.stdout }} -o plaintext
register: vb2
- debug: var=vb2.stdout
- name: Analyze 3rd IP
local_action: shell /usr/bin/abuseipdb -C {{ ip3.stdout }} -o plaintext
register: vb3
- debug: var=vb3.stdout
- name: Analyze 4th IP
local_action: shell /usr/bin/abuseipdb -C {{ ip4.stdout }} -o plaintext
register: vb4
- debug: var=vb4.stdout
- name: Analyze 5th IP
local_action: shell /usr/bin/abuseipdb -C {{ ip5.stdout }} -o plaintext
register: vb5
- debug: var=vb5.stdout
##Block IP if score more than 25
- name: Check 1rst IP and Block IP if score more than 25
shell: /sbin/route add "{{ ip1.stdout }}" gw 127.0.0.1 lo
when: "{{ vb1.stdout }} >= 25"
- name: Check 2nd IP and Block IP if score more than 25
shell: /sbin/route add "{{ ip2.stdout }}" gw 127.0.0.1 lo
when: "{{ vb2.stdout }} >= 25"
- name: Check 3rd IP and Block IP if score more than 25
shell: /sbin/route add "{{ ip3.stdout }}" gw 127.0.0.1 lo
when: "{{ vb3.stdout }} >= 25"
- name: Check 4th IP and Block IP if score more than 25
shell: /sbin/route add "{{ ip4.stdout }}" gw 127.0.0.1 lo
when: "{{ vb4.stdout }} >= 25"
- name: Check 5th IP and Block IP if score more than 25
shell: /sbin/route add "{{ ip5.stdout }}" gw 127.0.0.1 lo
when: "{{ vb5.stdout }} >= 25"
#Check Route
- name: Make Sure the IP has been blocked
shell: /sbin/route -n
register: route
- debug: var=route.stdout_lines
Here is the /tmp/output
└─$ cat /tmp/output
[' 40545 87.250.224.147', ' 20873 87.250.224.126', ' 16665 213.180.203.67', ' 15420 87.250.224.142', ' 14503 13.81.52.25']
Here is the output when running
└─$ cat /tmp/output | awk '{ print $3 }'| sed 's/.$//'|sed 's/.$//'
87.250.224.147
└─$ cat /tmp/output | awk '{ print $6 }'| sed 's/.$//'|sed 's/.$//'
87.250.224.126
└─$ cat /tmp/output | awk '{ print $9 }'| sed 's/.$//'|sed 's/.$//'
213.180.203.67
└─$ cat /tmp/output | awk '{ print $12 }'| sed 's/.$//'|sed 's/.$//'
87.250.224.142
└─$ cat /tmp/output | awk '{ print $15 }'| sed 's/.$//'|sed 's/.$//'
13.81.52.25
It is probably better to generate a JAML file from your Nginx log in order to use include_vars after that.
Btw: The idea of Ansible is to be platform independent. Because of that, it is better to have most of the platform specific stuff in an external script and not in the playbook. When you change the platform, you just have to migrate the external script, instead of rewriting the playbook.
Another alternative is to use scripted inventories. You can create a dynamic inventory by reading the IP addresses to block from your Nginx log. If you put the hosts to block into a group, you can define a single task for the group and delegate it to the firewall, where you block the hosts. You do not need to care about iterations. It is the job of Ansible to iterate.
This is much easier to maintain, than the playbook in your question. And it is extremely inefficient to put every line of a shell script into one local action. Don't do it.
PS: Yes you can program in a playbook, somehow. But try to avoid it. A playbook is not a useful programming language. You can not abstract with procedures and it has almost no scope. When you work with Ansible you have to change the way you think. Do not think about procedures. Instead thing about data. Take your data and arrange the data in a way, that is suitable for Ansible. If the data does not fit, use Jinja filters to rearrange the data until it fits. But avoid "programming" in a playbook by all means. And if you need to program, better do it in Jinja statements or Python.
so you could use range to loop over your command shell, i show you an example: when you loop with register, it records all output in list.
- name: "tips2"
hosts: localhost
tasks:
- name: Check 1st IP
local_action: shell echo {{ item }}
register: result
loop: "{{ range(0, 4 + 1, 2)|list }}" #loop from 0 to 4 with a step 2
- debug: var=result
#then you build your final list
- set_fact:
ips: "{{ ips | d([]) + item.stdout_lines }}"
loop: "{{ result.results }}"
- debug: var=ips
display variable result:
ok: [localhost] => {
"result": {
"changed": true,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
:
:
}
},
"item": 0,
"rc": 0,
"start": "2022-01-21 09:38:12.092426",
"stderr": "",
"stderr_lines": [],
"stdout": "0",
"stdout_lines": [
"0"
]
},
{
"ansible_loop_var": "item",
:
:
}
},
"item": 2,
"rc": 0,
"start": "2022-01-21 09:38:12.348007",
"stderr": "",
"stderr_lines": [],
"stdout": "2",
"stdout_lines": [
"2"
]
},
{
"ansible_loop_var": "item",
:
:
}
},
"item": 4,
"rc": 0,
"start": "2022-01-21 09:38:12.607555",
"stderr": "",
"stderr_lines": [],
"stdout": "4",
"stdout_lines": [
"4"
]
}
]
}
}
and result final ips after set_fact:
ok: [localhost] => {
"ips": [
"0",
"2",
"4"
]
}
just adapt the solution to your case: as i dont know your output, check the value of register...
- name: Check All IPs
local_action: shell cat /tmp/output | awk '{ print ${{item}} }'| sed 's/.$//'|sed 's/.$//'
register: result
loop: "{{ range(3, 15 + 1, 3)|list }}" #loop from 3 to 15 with a step 3
- set_fact:
ips: "{{ ips | d([]) + item.stdout_lines }}"
loop: "{{ result.results }}"
- name: Analyze All IPs
local_action: shell /usr/bin/abuseipdb -C {{ item }} -o plaintext
register: vbresult
loop: "{{ ips }}"
- debug: var=vbresult
another solution will be to trap alls your ips from your file in one task..
but you have to know your output, but the solution i show is easily adaptable to your case quickly
EDITED:
following your output, you could work directly from your result.stdout_lines:
- name: simulate your output result.stdout_lines
set_fact:
values:
- ' 40545 87.250.224.147'
- ' 20873 87.250.224.126'
- ' 16665 213.180.203.67'
- ' 15420 87.250.224.142'
- ' 14503 13.81.52.25'
- name: trap ips values
set_fact:
ips: "{{ ips | d([]) + [_ip] }}"
loop: "{{ values }}"
vars:
_ip: "{{ (item | trim).split(' ') | last }}"
- debug:
var: ips
result:
ok: [localhost] => {
"ips": [
"87.250.224.147",
"87.250.224.126",
"213.180.203.67",
"87.250.224.142",
"13.81.52.25"
]
}
I need to skip the value null from a list in ansible loop. I am using the when condition, still the null value gets printed.
Below is my playbook:
- hosts: localhost
vars:
show:
- read
- write
- null
- test
val: []
tasks:
- name: Fact
set_fact:
val: "{{val+[item]}}"
loop: "{{show}}"
when: item != "null"
- name: Print
debug:
msg: "{{val}}"
Output:
TASK [Print] ***
ok: [localhost] => {
"msg": [
"read",
"write",
null,
"test"
]
}
Please advise.
Quoting from YAML 10.2.1.1. Null
"Represents the lack of a value. This is typically bound to a native null-like value (e.g., undef in Perl, None in Python). ..."
There are more options on how to test null.
Compare to Python None
when: item != None
Use Jinja test none
when: item is not none
If you for whatever reason have to compare to a string the Jinja filter string converts null to string 'None'
when: item|string != 'None'
The most efficient way is to remove null values from the list before the iteration
loop: "{{ show|reject('none')|list }}"
I don't know if you want to check if the variable is "null" as a text or if the variable doesn't have a value and it's nullified so I will write both examples :), make null value a string and compare as a string:
- hosts: localhost
vars:
show:
- read
- write
- "null"
- test
val: []
tasks:
- name: Fact
set_fact:
val: "{{val+[item]}}"
loop: "{{show}}"
when: "'{{item}}' != 'null'"
- name: Print
debug:
msg: "{{val}}"
Test if the value is nullified and it really doesn't have any value we will use, is none for that example:
- hosts: localhost
vars:
show:
- read
- write
- null
- test
val: []
tasks:
- name: Fact
set_fact:
val: "{{val+[item]}}"
loop: "{{show}}"
when: item is not none
- name: Print
debug:
msg: "{{val}}"
I have a file in host_vars folder which looks like this:
tags:
tag1: value1
tag2: value2
tag3: '' <<-- This is an empty value which I would like to fill with some string
Now I wanted to go through each of these items and check if the value is empty and if it is add "NA" as a string to it, I used the following code for this:
- name: Clean any Tag inconsistencies
set_fact:
tags: "{{ item }} + NA"
when: (item | length == 0)
with_items: tags
- debug: var=tags
However, this does not do anything, it just prints the same list when checking the debug. What is wrong with this approach?
Edit:
I changed my code to respect that I am using a dictionary. This is my current approach. It prints out the values with key and name but does not change any item. I also tried using the jinja default filter which did not do anything:
- name: "Clean Tags"
set_fact:
tags: "{{ (tags | default('NA', true)) }}"
loop: "{{ tags | dict2items }}"
What I would like to achieve in the end is that it checks each dict value and if it is empty, it should add the value "NA" to it, so in the end above tags dictionary will look like this:
tags:
tag1: value1
tag2: value2
tag3: 'NA'
The play below
- hosts: localhost
vars:
my_tags:
tag1: value1
tag2: ''
tag3: ''
tasks:
- set_fact:
my_tags: "{{ my_tags|combine(dict(my_empty_tags|product(['NA']))) }}"
vars:
my_empty_tags: "{{ my_tags|dict2items|json_query(query) }}"
query: "[?value == ''].key"
- debug:
var: my_tags
gives (abridged)
my_tags:
tag1: value1
tag2: NA
tag3: NA
The same result can be achieved without json_query. Replace the task vars
vars:
my_empty_tags: "{{ my_tags|dict2items|
selectattr('value', 'eq', '')|
map(attribute='key')|
list }}"
tags is a playbook's keyword. You should have seen:
[WARNING]: Found variable using reserved name: tags
Q: "What happens in the set_fact section?"
A: Decompose the filter. First, create the product of the empty tags and the string 'NA'
- debug:
msg: "{{ my_empty_tags|product(['NA'])|list }}"
gives
msg:
- - tag2
- NA
- - tag3
- NA
Then create dictionaries from the items in the list
- debug:
msg: "{{ dict(my_empty_tags|product(['NA'])) }}"
gives
msg:
tag2: NA
tag3: NA
The last step in the filter is to combine the dictionaries.
I have the following array of record with the value:
myRecord.record.0.number = "Number 0"
myRecord.record.1.number = "Number 1"
myRecord.record.2.number = "Number 2"
myRecord.record.3.number = "Number 3"
How to create a playbook to debug the above value using loop dynamically/for every array?
for other common languange it can be done as follows:
for(int i = 0; i < myRecord.length(); i++)
{
echo myRecord.record.[i].number
}
for the repeating task of the playbook, it will looks like this:
---
- hosts: localhost
name: Array of Object
gather_facts: false
tasks:
- name: using debugMsg
debug:
msg:
- "{{ myRecord.record.0.number }}"
- "{{ myRecord.record.1.number }}"
- "{{ myRecord.record.2.number }}"
- "{{ myRecord.record.3.number }}"
I have figured out how to do this. Basically I just need to use loop_control to filter which specific value I need. Here is the playbook:
---
- hosts: localhost
name: Array of Object
gather_facts: false
tasks:
- name: using loop_control
debug:
msg: "{{ item.number }}"
with_items:
- "{{ myRecord.record }}" #this will become 'item'
loop_control:
label: "{{ item.number }}" #filter to display the value of number only
There are two methods. The first method uses the with_* keyword, and depends on the Lookup plugin. The second method uses loop keyword, which is equivalent to 'with_' + 'list lookup plugin' (so you get 'with_list').
Now, assuming your data structure looks like this:
---
# vars file for print_variable_from_list
myRecord:
record:
- number: "Number 0"
- number: "Number 1"
- number: "Number 2"
- number: "Number 3"
Every number is indexable with the "number" key.
- name: loop through myRecord
debug:
msg: "{{ item.number }}"
loop: "{{ myRecord.record }}"
Kindly refer to other posts for more complex queries.
I am newbie to ansible and trying to write my first playbook.
- name: create volume
volume:
state: present
username: "{{ username }}"
password: "{{ password }}"
hostname: "{{ inventory_hostname }}"
vserver: "{{item[0]}}"
name: "{{item[1]}}"
aggregate_name: "{{output}}"
with_nested:
- [ 'vs10' , 'vs11' ]
- [ 'vol1' , 'vol2', 'vol3' , 'vol4' ,'vol5', ''vol6']
connection: local
Actual output:
vs10-vol1 vol2 vol3 vol4
vs11- vol1 vol2 vol3 vol4
Expected output:
vs10-vol1, vol3 vol5
vs11-vol2, vol4 vol6
This would probably work. I'm basically looping the task against volumes and calculating the vserver in the task.
- name: create volume
volume:
state: present
username: "{{ username }}"
password: "{{ password }}"
hostname: "{{ inventory_hostname }}"
# Calculate which vserver to use based on 'current volume index in the volumes list' and 'length of vservers list'.
# The logic uses modulus to try and distribute volumes across given vcenters
vserver: "{{vservers[(current_index % (vservers|length))]}}"
# Name is item itself because you are looping volumes
name: "{{item}}"
aggregate_name: "{{output}}"
# Loop the volumes
loop: [ 'vol1' , 'vol2', 'vol3' , 'vol4' ,'vol5', 'vol6']
# This is make a loop_control variable available. This will give us 'current_index'
loop_control:
index_var: current_index
# Vservers are defined here
vars:
vservers: [ 'vs10' , 'vs11' ]