I need to use ansible to do the following job:
1. send a Http Post reuqest to a WebApi
2. check wheter the response meet requirement, otherwise extract some data from the response
3. send another Http Post request with body filled with the data got in step 2
4. loop step 1 to step 3 until the response meet requirement
My code is as the following, but it did not work. The finished_res seems to be undefined forever inside uri module. what I mean is that the variable finished_res inside the uri module seems to never change because I keep receiving the same request body in server. However, the finished_res outside the loop works, I could see the debug message.
- hosts: all
remote_user: ubuntu
gather_facts: no
vars:
default_job_list: "xxx,yyy,zzz,aaa,bbb"
tasks:
- name: test is jobs finish
uri:
url: "{{ API_URL }}"
method: POST
body: "{% if finished_res is defined %}{{ finished_res.json.remainingJobs }}{% else %}{{ default_job_list }}{% endif %}"
return_content: yes
register: finished_res
changed_when: True
until: finished_res is defined and finished_res.json is defined and finished_res.json.status is defined and finished_res.json.status != "FALSE"
- set_fact:
RES: "{{ finished_res }}"
- debug: msg="{{ RES }}"
Could anybody help me ? Thanks in advance !
At last, I solved the problem in another way : create a new ansible module
Related
Could someone let me know how we can create a code as below?
- name: TEST1
set_fact:
list_a: "{{ list_a + [item.json.SearchResult.resources] }}"
with_items:
- "{{ source_list.results[0] }}"
- "{{ source_list.results[1] }}"
- "{{ source_list.results[x] }}"
... (unknown how many items in result from API)
vars:
list_a: []
source_list.results[x] comes from an API result. The reason why I need to create an array is that the number of API result is maximum 100. But there are over 500 items.
Note: since we have no idea what you initial data structure looks like exactly, the below might not be 100% fitting your use case. For your next questions, please read How to ask and pay attention to the Minimal, complete and reproducible example section. Thanks
You are taking this the wrong way. Simply extract the attribute you need from each result using the map(attribute=x) Jinja2 filter.
For the below I inferred (see above note) that:
you called your API with ansible.builtin.uri in a loop to get batches of 100 results which are returned as a list in the SearchResult.ressources field
you want in the end a flattened list where all resources are at top level
- name: Show my list of single attributes
ansible.builtin.debug:
var: "source_list.results
| map(attribute='json.SearchResult.resources') | flatten"
You actually don't need to set_fact:
For a single use, just use the above expression directly in the relevant parameter (e.g. loop or a module param....) or eventually declare this in a var at task level.
If you want to reuse this in different parts of your playbook, just declare a var at play level and expand it anywhere once you have called your API and populated the source_list var. In that case, just add a default value to prevent an error if API was not yet called.
Example for the second case above in this pseudo playbook
---
- hosts: localhost
gather_facts: false
vars:
list_a: "{{ source_list.results | d([])
| map(attribute='json.SearchResult.resources') | flatten }}"
tasks:
- name: "This will return an empty list (i.e. [])
as we did not populate source_list yet"
ansible.builtin.debug:
var: list_a
- name: Call our API and register source_list
ansible.builtin.uri:
uri: https://my.api.com/api/v1/some/endpoint
# [... more parameters here ... ]
loop: "{{ my_list_of_ressources }}"
register: source_list
- name: "This will now return a populated list
after calling the API and registering source_list"
ansible.builtin.debug:
var: list_a
Now, to still give a direct answer to your initial question: you can construct that list iteratively inside a set_fact task. This is definitely not efficient as it involves a task running inside a loop (both unneeded as demonstrated above) and possibly on multiple hosts in your play. But for learning purpose, here it is:
- name: very inefficient way to get the same result as above
set_fact:
list_a: "{{ list_a | d([]) + item.SearchResult.resources }}"
loop: "{{ source_list.results }}"
After reading #Zeitounator's answer, I realized I was taking this the wrong way. I used the idea in the answer (json query and flatten) and I changed my task to the below which works as expected.
- name: Create ID list from API result data
set_fact:
list_a: "{{ source_list | json_query('results[*].json.SearchResult.resources[*].id') | flatten }}"
- name: API get again with id
uri:
url: "{{ request_url }}/{{ item }}"
...
register: result_data
with_items:
- "{{ list_a }}"
I'm rather new to ansible and would like to deploy prometheus-grok-exporter (via ansible-grok-exporter role) with a specific configuration for all my nodes that run the cacti application.
My inventory is like this:
cacti_first ansible_host=192.168.50.50
cacti_second ansible_host=192.168.50.51
[group__cacti]
cacti_first
cacti_second
Inside group_vars/group__cacti I want to add something like this:
---
prometheus_grok_services_template:
- name: cacti_metrics
config_version: 3
input:
type: file
paths:
{% for cacti_dir in cacti_path %}
- "{{cacti_dir}}/log/cacti.log"
{% endfor %}
readall: false
extaConfigContinuesFromHere: true
And I have host config like this:
host_vars/cacti_first:
---
cacti_path:
- /usr/share/cacti
prometheus_grok_services:
- prometheus_grok_services_template
host_vars/cacti_second:
---
cacti_path:
- /usr/share/cacti
- /usr/share/cacti2
prometheus_grok_services:
- prometheus_grok_services_template
Inside the playbook I do a loop for prometheus_grok_services and use the yaml data to provision the service.
Now - this works as long as I don't try to use a loop inside group_vars/group__cacti. ansible-inventory reports that:
$ ansible-inventory -i hosts --list cacti_second
ERROR! Syntax Error while loading YAML.
found character that cannot start any token
The error appears to be in '/home/bastion/ansible-playbooks/group_vars/group__cacti': line 8, column 10, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
paths:
{% for cacti_dir in cacti_path %}
^ here
So, I'd like to ask - is it allowed to do jinja loops to build yaml for group vars? Is it a syntax error on my end? How am I supposed to template it?
I'd like to avoid moving the block to host vars (which I know works), mostly because it's a large piece of code (about 2KB of yaml config) and it's not as elegant as using group vars.
Thanks!
Fix the group_vars. For example
shell> cat group_vars/group__cacti
---
prometheus_grok_services_template:
- name: cacti_metrics
config_version: 3
input:
type: file
paths: "{{ paths_str|from_yaml }}"
readall: false
extaConfigContinuesFromHere: true
paths_str: |
{% for cacti_dir in cacti_path %}
- {{ cacti_dir }}/log/cacti.log
{% endfor %}
Then, the playbook
- hosts: group__cacti
gather_facts: false
tasks:
- debug:
msg: "{{ lookup('vars', item) }}"
loop: "{{ prometheus_grok_services }}"
gives
ok: [cacti_first] => (item=prometheus_grok_services_template) =>
msg:
- config_version: 3
input:
extaConfigContinuesFromHere: true
paths:
- /usr/share/cacti/log/cacti.log
readall: false
type: file
name: cacti_metrics
ok: [cacti_second] => (item=prometheus_grok_services_template) =>
msg:
- config_version: 3
input:
extaConfigContinuesFromHere: true
paths:
- /usr/share/cacti/log/cacti.log
- /usr/share/cacti2/log/cacti.log
readall: false
type: file
name: cacti_metrics
You can't use this kind of for loop in a variables file or a playbook - it only works in template files. To acheive what you're after, you can use product filters, as described https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#products
In your example, you would have:
---
prometheus_grok_services_template:
- name: cacti_metrics
config_version: 3
input:
type: file
paths: "{{ cacti_path | product(['/log/cacti.log']) | map('join') | list }}"
readall: false
extaConfigContinuesFromHere: true
I'm running Ansible and I try to make this task work. I have defined the default of the variable "docker_registries" to be an empty list: docker_registries: [] I noticed that I will get error when running the ansible playbook if the list is empty. this is the error I get:
fatal: [***.cloudapp.azure.com]: FAILED! => {"msg": "Invalid data passed to 'loop', it requires a list, got this instead: None. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}
I'm trying to add the condition that if the "docker_registries" is empty the task continues without raising an error.
this is the code for this task:
- name: Log into additional docker registries, when required
command: docker login -u {{item.username}} -p {{item.password}} {{item.server}}
become: true
loop: "{{docker_registries}}"
I tried to change the loop to loop: "{{ lookup(docker_registries, {'skip_missing': True})}}"
but I get the error
An exception occurred during task execution. To see the full
traceback, use -vvv. The error was: AttributeError: 'NoneType' object
has no attribute 'lower' fatal:
[***.cloudapp.azure.com]: FAILED! =>
{"msg": "Unexpected failure during module execution.", "stdout": ""}
I'm pretty new with this. Can anyone help?
You could use the when statement in combinaison with the iterable Jinja test.
Because, well, if you want to loop over your variable, then it should be iterable.
- name: Log into additional docker registries, when required
command: docker login -u {{ item.username }} -p {{ item.password }} {{ item.server }}
become: true
loop: "{{ docker_registries }}"
when: docker_registries is iterable
Based on the content of the variable, the when condition could still not do the trick, because the validity of the data passed to the loop will be considered first.
Now this said, you can use a conditional expression in the loop declaration itself to come around this issue:
- name: Log into additional docker registries, when required
command: docker login -u {{ item.username }} -p {{ item.password }} {{ item.server }}
become: true
loop: "{{ docker_registries if docker_registries is iterable else [] }}"
Also mind that a string is an iterable in Python, as they are, simply put, a list of characters.
So you might want to go:
- name: Log into additional docker registries, when required
command: docker login -u {{ item.username }} -p {{ item.password }} {{ item.server }}
become: true
loop: "{{ docker_registries if docker_registries is iterable and docker_registries is not string else [] }}"
Thank you all guys. I had a line in my group_vars file which I had commented out the all docker_registries username and password and all, but I had not comment the docker_registries: line itself. That's why I was given the None instead of the empty list.
I want to create user and gather their info in local file but with loop register is now working as expected.
I thought it was an indentation problem but no luck.
My playbook
---
- hosts: localhost
tasks:
- name: Clearing Local file
local_action: shell echo "Zone,docode,doname,testuser Output" > user.csv
- hosts: app
tasks:
- name: Creating user Testuser
become: yes
user:
name: "{{ item }}"
state: present
shell: "/bin/bash"
password: "$6$mysecretsalt$qyctTVhRMS1ZSnCuzQNAM8Y7V/yqSEnyRbal0IYXSqSEVKkXF8ZmXBZoRIaN/PvzE/msq8iOJO830OOCG89va/"
update_password: always
groups: santosh
shell: id "{{item}}"
ragister: userout
loop:
- newuser1
- newuser2
- newuser3
- debug:
var=userout
which gives the following error when executed
ERROR! conflicting action statements: shell, user
The error appears to have been in '/home/santosh/ans-home/playbooks/Create_User_and_Gather_output.yml': line 12, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Creating user Testuser
^ here
You cannot call several modules in one task, you need to separate each call in its own task, as reported by the error message.
I understand why you tried it: to take advantage of the same loop for several task. Unfortunately this is not possible. You could move your set of tasks to a separate file and include it in a loop if you really have to loop over a significant amount of tasks. This is not really required in your situation because:
you only have two tasks
you can compact writing of your loop by using a declared var for reuse
and most essentially because you don't need your second task
In fact, the user module will return the uid of the user it created or that is existing in its result. You just have to register the result of calling the user module.
Just try the following for your second play:
- name: Resgister application users
hosts: app
vars:
user_list:
- newuser1
- newuser2
- newuser3
tasks:
- name: Create the users if they don't exist
user:
name: "{{ item }}"
state: present
shell: "/bin/bash"
password: "$6$mysecretsalt$qyctTVhRMS1ZSnCuzQNAM8Y7V/yqSEnyRbal0IYXSqSEVKkXF8ZmXBZoRIaN/PvzE/msq8iOJO830OOCG89va/"
update_password: always
groups: santosh
register: create_users
loop: "{{ user_list }}"
- name: Show ids of users
debug:
msg: "The uid of user {{ item.name }} is: {{ item.uid }}"
loop: "{{ create_users.results }}"
And as a side note: for your first play, do yourself a favor and stop using the old local_action syntax in favor of delegate_to: localhost for a task. It is not even required in your case as your are already targeting your play to localhost only.
I have the following tasks in Ansible:
- name: Getting Job Info
uri:
url: "https://{{ hostname }}/job/JobService"
method: POST
user: "{{ username }}"
password: "{{ password }}"
body: "{{ lookup( 'template' , 'jobInfo.xml.j2' ) }}"
status_code: 200
validate_certs: false
headers:
soapaction: "getJobInfoVO"
return_content: true
register: job_status_soap
- name: Converting Job Info response to JSON
set_fact:
job_status_json: "{{ job_status_soap.content | xml2json }}"
- name: Setting Job Status
set_fact:
job_status: "{{ job_status_json['soapenv:Envelope']['soapenv:Body']['multiRef'][0]['status']['#text'] }}"
- debug: msg="{{ job_status }}"
As you can see, it makes a web service call, converts the SOAP response to JSON and then extracts the relevant value (an integer) and sets it as a fact. I need to repeat this block of code until the aforementioned integer value is equal to certain integer.
Any ideas on how I might approach this task?
This would have been easy to implement using until to execute ansible do-until loop if the response was in JSON format.
But here's a working example with xml response.
Pre-requisites: (Make sure you have the python dependencies installed on your ansible host)
1) Using custom xml_to_json filter.
2) Using json_query filter
P.S. Executing following play will not trigger retry as the until contion is already met.
If you want to test if retry is working then change the until condition or the value of tag in the request header.
---
- name: Retry url until xml content equals certain value
hosts: 127.0.0.1
connection: local
become_user: root
become: yes
tasks:
- name: Check job status
uri:
url: "https://httpbin.org/anything"
method: GET
timeout: 10
validate_certs: no
headers:
content: "<envelope><body><request><status>2</status></request></body></envelope>"
register: get_search_job_status_response
until: get_search_job_status_response.json.headers.Content|from_xml|json_query('envelope.body.request.status')|int == 2
retries: 2
delay: 2
...