Create missing file in all subdirs with ansible - loops

I want to check if there is a specific file in all subdirs of a folder. If the file doesn't exists, I want to create it.
This is my playbook:
---
- hosts: localhost
become: 'yes'
vars:
dir_path: "/tmp/test"
tasks:
- name: Find /tmp/test/ all directories
find:
paths: /tmp/test/
recurse: no
file_type: directory
register: dir_list
- debug: var=dir_list
- name: check if test file exists
stat:
path: "{{ dir_list.files | map(attribute='path') }}/test.txt"
register: file_exists
loop: "{{ dir_list.files | map(attribute='path') | list }}"
- name: create file
shell: touch {{ files_exists.item }}/test.txt
register: create_file
when: not file_exists.stat.exists
loop: "{{ file_exists.item }} | list }}"
But the "create file" fails, any idea how to solve this?

Your use of loop in the last tasks is wrong: you are not using the item created during iteration when checking for file existence.
(Note for your next question: avoid using sentences like "it fails" or "it doesn't work" on their own as they do not accurately describe your problem)
In your example, you are trying to create the file with shell when there is a file module made just for that purpose. I used it my below example:
The playbook:
---
- name: Create missing file in subdirs
hosts: localhost
gather_facts: false
vars:
dir_path: "/tmp/test"
file_name: "test.txt"
tasks:
- name: "Find subdirs in {{ dir_path }}"
find:
paths: "{{ dir_path }}"
recurse: no
file_type: directory
register: dir_list
- name: "Show result of dir listing "
debug:
var: dir_list
verbosity: 1
- name: "Check if {{ file_name }} exists in subdirs"
stat:
path: "{{ item }}/{{ file_name }}"
register: file_check
loop: "{{ dir_list.files | map(attribute='path') | list }}"
- name: "Show the file check result"
debug:
var: file_check
verbosity: 1
- name: "Create file where it does not exist"
file:
path: "{{ item }}/{{ file_name }}"
state: touch
loop: "{{ file_check.results | rejectattr('stat.exists') | map(attribute='item') | list }}"
Which gives (use ansible-playbook -v to see verbosity: 1 debug tasks)
# Create test dir and subdirs
$ for d in {a..d}; do mkdir -p /tmp/test/$d; done
# Add the test file in one dir for demo
$ touch /tmp/test/c/test.txt
# Show the situation before running the playbook
$ tree /tmp/test/
/tmp/test/
├── a
├── b
├── c
│   └── test.txt
└── d
4 directories, 1 file
# Run the playbook (run with debug for yourself if you wish)
$ ansible-playbook play.yml
PLAY [Create missing file in subdirs] *****************************************************************************************************************************************************************************
TASK [Find subdirs in /tmp/test] **********************************************************************************************************************************************************************************
ok: [localhost]
TASK [Show result of dir listing] *********************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Check if test.txt exists in subdirs] ************************************************************************************************************************************************************************
ok: [localhost] => (item=/tmp/test/a)
ok: [localhost] => (item=/tmp/test/b)
ok: [localhost] => (item=/tmp/test/c)
ok: [localhost] => (item=/tmp/test/d)
TASK [Show the file check result] *********************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Create file where it does not exist] ************************************************************************************************************************************************************************
changed: [localhost] => (item=/tmp/test/a)
changed: [localhost] => (item=/tmp/test/b)
changed: [localhost] => (item=/tmp/test/d)
PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost : ok=3 changed=1 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
# Check the result
$ tree /tmp/test/
/tmp/test/
├── a
│   └── test.txt
├── b
│   └── test.txt
├── c
│   └── test.txt
└── d
└── test.txt
4 directories, 4 files
# Run the playbook again to demonstrate idempotency
$ ansible-playbook play.yml
PLAY [Create missing file in subdirs] *****************************************************************************************************************************************************************************
TASK [Find subdirs in /tmp/test] **********************************************************************************************************************************************************************************
ok: [localhost]
TASK [Show result of dir listing] *********************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Check if test.txt exists in subdirs] ************************************************************************************************************************************************************************
ok: [localhost] => (item=/tmp/test/a)
ok: [localhost] => (item=/tmp/test/b)
ok: [localhost] => (item=/tmp/test/c)
ok: [localhost] => (item=/tmp/test/d)
TASK [Show the file check result] *********************************************************************************************************************************************************************************
skipping: [localhost]
TASK [Create file where it does not exist] ************************************************************************************************************************************************************************
PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0

Related

Use host specific file if it exists instead of the one from role?

Is there a way to replace files that are copied, when such file exists on host similarly as you can do with variables (you can override variable on host, to use custom value if needed)?
Currently I'm doing like this:
- name: Check if path exists
set_fact:
file_nginx_logo: "files/{{ inventory_hostname }}/maintenance_logo.png"
- name: Create Nginx Maintenance Image file (custom).
ansible.builtin.copy:
src: "files/{{ inventory_hostname }}/maintenance_logo.png"
dest: "{{ path_nginx_logo }}"
when: file_nginx_logo is file
- name: Create Nginx Maintenance Image file (default).
ansible.builtin.copy:
src: maintenance_logo.png
dest: "{{ path_nginx_logo }}"
when: file_nginx_logo is not file
But this is quite ugly as I need 3 (or 2) tasks to copy one file.. Maybe there is a place in inventory path where if you put it, it would overwrite file defined on role?
I mean, something like:
files/my-host/my-file.txt
roles/my-role/files/my-file.txt
So if you call my-file.txt on my-role it would use the one from my-host if such file exists there?..
Here's a full blown example. Adapt to you needs.
Global test project stucture:
.
├── files
│   └── host2
│   └── maintenance_logo.txt
├── inventories
│   └── default
│   └── hosts.yml
├── roles
│   └── test
│   ├── files
│   │   └── maintenance_logo.txt
│   └── tasks
│   └── main.yml
└── test_playbook.yml
The test inventory in inventories/default/hosts.yml points to localhost in all cases:
all:
vars:
ansible_connection: local
hosts:
host1:
host2:
As you can see, the role holds the default logo where we will fallback whereas there is specific logo for host2 in the files dir adjacent to the playbook. (I used txt source files for this example for convenience but this will work with any type of file)
In roles/test/tasks/main.yml we have a single task:
- name: Copy maintenance logo to target
vars:
candidate_files:
- "{{ inventory_hostname }}/maintenance_logo.txt"
- maintenance_logo.txt
ansible.builtin.copy:
src: "{{ item }}"
dest: /tmp/{{ inventory_hostname }}.png
loop: "{{ q('ansible.builtin.first_found', candidate_files) }}"
The key here is using the ansible.builtin.first_found lookup which will exit on first found file (as its name obviously suggest...) and return its name.
This lookup obeys the ansible search path so we can use the file name relative to any correctly placed files folder anywhere in playbook directory, roles, etc...
Note that I used inventory_hostname in the destination file names as I faked all my test hosts to local in my inventory. You obviously do not have to do that on your separate targets (i.e. use whatever target name you wish)
Now the test_playbook.yml to wrap it all up
---
- hosts: all
gather_facts: false
tasks:
- ansible.builtin.import_role:
name: test
Which gives:
$ ansible-playbook -i inventories/default/ test_playbook.yml
PLAY [all] *****************************************************************************************************************************************************************************************************************************
TASK [test : Copy maintenance logo to target] ******************************************************************************************************************************************************************************************
changed: [host2] => (item=/tmp/test/files/host2/maintenance_logo.txt)
changed: [host1] => (item=/tmp/test/roles/test/files/maintenance_logo.txt)
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
host1 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host2 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Ansible find only files which have changed

I'm trying to look for a pattern in Postgres configuration files, and
only wants to find changed files that match the search pattern.
---
- name: Query postgres pattern matched files
hosts: pghosts
gather_facts: false
tasks:
- name: "Find postgres conf files"
find:
paths: "/srv/postgresql/config/conf.d"
patterns: "*.conf"
file_type: "file"
register: search_files
- name: "grep postgres conf files with searchpattern"
shell: "grep -i 'pg_show_plans' {{ item.path | basename }}"
args:
chdir: "/srv/postgresql/config/conf.d"
loop: "{{ search_files.files }}"
loop_control:
label: "{{ item.path | basename }}"
ignore_errors: true
- name: "find changed files"
debug:
msg: "{{ search_files.files |map(attribute='path') }}"
These are the changed files:
changed: [pgsql14.techlab.local] => (item=00_global_default.conf)
...ignoring
changed: [pgsql13.techlab.local] => (item=00_global_default.conf)
...ignoring
How do I get only those filenames, which realy have changed, and passed the pattern test.
Thanks a lot for your help
I do now have simplified the code which works for me:
- name: read the postgres conf file
shell: cat /srv/postgresql/config/conf.d/00_global_default.conf
register: user_accts1
- name: a task that only happens if the string exists
when: user_accts1.stdout.find('pg_show_plans') != -1
ansible.builtin.copy:
content: "{{ user_accts1.stdout }}"
dest: 'fileresults/{{ inventory_hostname }}.00_global_default.out'
delegate_to: localhost
- name: read the postgres conf file
shell: cat /srv/postgresql/config/conf.d/01_sizing_specific.conf
register: user_accts2
- name: a task that only happens if the string exists
when: user_accts2.stdout.find('pg_show_plans') != -1
ansible.builtin.copy:
content: "{{ user_accts2.stdout }}"
dest: 'fileresults/{{ inventory_hostname }}.01_sizing_specific.out'
delegate_to: localhost
- name: read the postgres conf file
shell: cat /srv/postgresql/config/conf.d/02_local_overrides.conf
register: user_accts3
- name: a task that only happens if the string exists
when: user_accts3.stdout.find('pg_show_plans') != -1
ansible.builtin.copy:
content: "{{ user_accts3.stdout }}"
dest: 'fileresults/{{ inventory_hostname }}.02_local_overrides.out'
delegate_to: localhost

Preserve format of the file while printing in Ansible

Below is my playbook to print the file.
I used couple of approaches but the file is not printed as is i.e. the new line formatting is gone when ansible prints the file contents.
- name: List startup files
shell: cat /tmp/test.txt
register: readws
- debug:
msg: "/tmp/test.txt on {{ inventory_hostname }} is: {{ readws.stdout_lines }}"
- debug:
msg: "/tmp/test.txt on {{ inventory_hostname }} is: {{ lookup('file', '/tmp/test.txt') }}"
cat /tmp/test.txt
i
m
good
Expected Ansible output:
TASK [debug] *****************************************************************************************
ok: [localhost] => {
"msg": "/tmp/test.txt on localhost is:
i
m
good
"
}
Ansible output:
TASK [List startup files] ******************************************************************
changed: [localhost]
TASK [debug] *****************************************************************************************
ok: [localhost] => {
"msg": "/tmp/test.txt on localhost is: [u'i', u'm ', u'good']"
}
TASK [debug] *****************************************************************************************
ok: [localhost] => {
"msg": "/tmp/test.txt on localhost is: i\nm \ngood"
}
Can you please suggest ?
You cannot really get what you require (unless maybe if you change the output callback plugin...).
The closest you can get is by displaying a list of lines like in the following example:
- name: Show file content
vars:
my_file: /tmp/test.txt
msg_content: |-
{{ my-file }} on localhost is:
{{ lookup('file', my_file) }}
debug:
msg: "{{ msg_content.split('\n') }}"
This might help who might come looking for simpler way to display a file in ansible.
stdout_lines prints the file with reasonable formatting.
- name: display the file needed
shell: cat /tmp/test.txt
register: test_file
- debug:
var: test_file.stdout_lines

Check if a file is 20 hours old in ansible

I'm able to get the timestamp of a file using Ansible stat module.
- stat:
path: "/var/test.log"
register: filedets
- debug:
msg: "{{ filedets.stat.mtime }}"
The above prints mtime as 1594477594.631616 which is difficult to understand.
I wish to know how can I put a when condition check to see if the file is less than 20 hours old?
You can also achieve this kind of tasks without going in the burden to do any computation via find and its age parameter:
In your case, you will need a negative value for the age:
Select files whose age is equal to or greater than the specified time.
Use a negative age to find files equal to or less than the specified time.
You can choose seconds, minutes, hours, days, or weeks by specifying the first letter of any of those words (e.g., "1w").
Source: https://docs.ansible.com/ansible/latest/modules/find_module.html#parameter-age
Given the playbook:
- hosts: all
gather_facts: no
tasks:
- file:
path: /var/test.log
state: touch
- find:
paths: /var
pattern: 'test.log'
age: -20h
register: test_log
- debug:
msg: "The file is exactly 20 hours old or less"
when: test_log.files | length > 0
- file:
path: /var/test.log
state: touch
modification_time: '202007102230.00'
- find:
paths: /var
pattern: 'test.log'
age: -20h
register: test_log
- debug:
msg: "The file is exactly 20 hours old or less"
when: test_log.files | length > 0
This gives the recap:
PLAY [all] **********************************************************************************************************
TASK [file] *********************************************************************************************************
changed: [localhost]
TASK [find] *********************************************************************************************************
ok: [localhost]
TASK [debug] ********************************************************************************************************
ok: [localhost] => {
"msg": "The file is exactly 20 hours old or less"
}
TASK [file] *********************************************************************************************************
changed: [localhost]
TASK [find] *********************************************************************************************************
ok: [localhost]
TASK [debug] ********************************************************************************************************
skipping: [localhost]
PLAY RECAP **********************************************************************************************************
localhost : ok=5 changed=2 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
- stat:
path: "/var/test.log"
register: filedets
- debug:
msg: "{{ (ansible_date_time.epoch|float - filedets.stat.mtime ) > (20 * 3600) }}"

Ansible: Adding values to variable in a loop

I'm working on a playbook that does the following:
Goes into a specified path on each Windows server
Slurps text from a file and adds it to a variable
Performs a check on the variable to see if a string of text exists
Writes the results to a file based on the outcome.
Here is the code I have for this:
---
- name: Slurps text from file on Windows server
hosts: win
gather_facts: false
tasks:
- name: Get text
slurp:
src: D:\testsearch.ini
register: norequest
- name: Check for norequest=false in variable
lineinfile:
dest: ./norequest.csv
line: "{{ inventory_hostname }} There is a false value"
state: present
create: true
insertafter: EOF
when: '"''NoRequest = False'' in norequest.content|b64decode"|lower'
delegate_to: localhost
- name: Check for norequest=true in variable
lineinfile:
dest: ./norequest.csv
line: "{{ inventory_hostname }} There is a true value."
state: present
create: true
insertafter: EOF
when: '"''NoRequest = True'' in norequest.content|b64decode"|lower'
delegate_to: localhost
Based on my results, it looks like the playbook slurps the text from the files on both test servers and adds it all to the variable, then performs the conditional check against one of the servers (since the task itself is being delegated to localhost) and outputs the results to the file as though they all came from SERVER1 (the last part seems to be due to the delegation).
PLAY [Slurps text from file on Windows server] *******************************
TASK [Delete previous norequest file] *******************************
changed: [SERVER1 -> localhost]
TASK [Get text] ***************************************
ok: [SERVER2]
ok: [SERVER1]
TASK [Check for norequest=false in variable] ********************************
changed: [SERVER1 -> localhost]
TASK [Check for norequest=true in variable] *******************************
changed: [SERVER1 -> localhost]
PLAY RECAP *******************************
SERVER1 : ok=4 changed=3 unreachable=0 failed=0
SERVER2 : ok=1 changed=0 unreachable=0 failed=0
Here are the contents of the file after the playbook is run:
SERVER1 There is a false value
SERVER1 There is a true value.
This is what the outcome should be if the playbook worked as I want it to:
SERVER1 There is a false value
SERVER2 There is a true value.
I feel like part (or all) of my issue might be that I'm looking at this through a PowerShell lens; as in, "FOR EACH server, get the text from the file, perform a conditional check, write the output to the outfile, then move on to the next server." Is something like that possible in an Ansible playbook? I've looked into dictionaries as a way to solve this, but the only good examples I could find used pre-existing dictionaries or dictionaries populated at runtime with basic server info.
Seems to me, that the when: condition was wrong. Matching is now done via a regexp. Tested it with this playbook:
---
- name: Slurps text from file on Windows server
hosts:
- SERVER1
- SERVER2
gather_facts: false
tasks:
- name: Get text
slurp:
src: D:\testsearch.ini
register: norequest
- name: Check for norequest=false in variable
lineinfile:
dest: ./norequest.csv
line: "{{ inventory_hostname }} There is a false value"
state: present
create: true
insertafter: EOF
when: 'norequest["content"] | b64decode | lower | regex_search("norequest *= *false")'
delegate_to: localhost
- name: Check for norequest=true in variable
lineinfile:
dest: ./norequest.csv
line: "{{ inventory_hostname }} There is a true value"
state: present
create: true
insertafter: EOF
when: 'norequest["content"] | b64decode | lower | regex_search("norequest *= *true")'
delegate_to: localhost
The file testsearch.ini has the following contents on the systems:
SERVER1
NoRequest = False
SERVER2
NoRequest = True
Executing the playbook with ansible-playbook -i hosts play.yml gives the following output:
PLAY [SERVER1,SERVER2] *********************************************************
TASK [Get text] ****************************************************************
ok: [SERVER1]
ok: [SERVER2]
TASK [Check for norequest=false in variable] ***********************************
skipping: [SERVER2]
ok: [SERVER1 -> localhost]
TASK [Check for norequest=true in variable] ************************************
skipping: [SERVER1]
ok: [SERVER2 -> localhost]
PLAY RECAP *********************************************************************
SERVER1 : ok=2 changed=0 unreachable=0 failed=0
SERVER2 : ok=2 changed=0 unreachable=0 failed=0
The contents of norequest.csv after the run is
SERVER1 There is a false value
SERVER2 There is a true value

Resources