Prior to Ansible 2.5, the syntax for loops used to be with_x. Starting at 2.5, loop is favored and with_x basically disappeared from the docs.
Still, the docs mention exemples of how to replace with_x with loop. But I'm clueless as to how we're now supposed to loop through a directory of files.
Let's say I need to upload all the files within a given dir, I used to use with_fileglob.
- name: Install local checks
copy:
src: "{{ item }}"
dest: /etc/sensu/plugins/
owner: sensu
group: sensu
mode: 0744
with_fileglob:
- plugins/*
So what's the modern equivalent? Is it even possible? I know I still can use with_fileglob but as I'm writing new roles, I'd better have them future-proof.
The equivalent is
loop: "{{ lookup('fileglob', 'plugins/*', wantlist=True) }}"
Here is the doc.
From the current Ansible loops doc:
Any with_* statement that requires using lookup within a loop should
not be converted to use the loop keyword. For example, instead of
doing:
loop: "{{ lookup('fileglob', '*.txt', wantlist=True) }}"
it’s cleaner to keep:
with_fileglob: '*.txt'
Related
Not sure how to have this logic implemented, I know how to do it a single file :
- name: Obtain information about a file
win_stat:
path: "C:\myfile.txt"
register: fileinfo
- [...]
when: fileinfo.exists == False
how should I go with a list of files?
If you just want to reduce the steps for doing this, you should be able to do your download step (not shown in your example) with ignore_errors: yes on your download commands. If you use a combination of ignore_errors: yes and register, you can even tell whether the command failed.
If you're looking to make it a bit more efficient, you can do the stat in a single task and then examine the results of that. When you execute a task with a list, you get a hash of answers.
Assuming you have a list of file names/paths in ssh_key_config, you use the stat and then you can loop over the items (which conveniently have the file name in them).
- name: Check to see if file exists
stat:
path: "{{ remote_dir }}/{{ item }}"
register: stat_results
with_items: "{{ target_files }}"
ignore_errors: True
- name: perform operation
fetch:
src: "{{ remote_dir }}/{{ item.item }}"
dest: "{{ your_dest_dir }}"
flat: yes
with_items: "{{ stat_results.results }}"
when: item.stat.exists == False
In this case, the assumptions are that remote_dir contains the remote directory on the host, target_files contains the actual file names, and your_dest_dir contains the location you want the files placed locally.
I don't do much with Windows and Ansible, but win_stat is documented pretty much the same as stat, so you can likely just replace that.
Also note that this expects the list of files, not a glob. If you use a glob (for example, you want to retrieve all files with a certain extension from the remote), then you would not use the with_items clause, and you'd need to use the item.stat.filename and/or item.stat.path to retrieve the file remotely (since the item.item would contain the request item, which would be the glob.
I would like to create a playbook that reads a local file to a var, Then be able to loop through this var line by line and use the lines in a task.
To get the file content i used:
file_contents: "{{lookup('file', './myfile.txt')}}"
I tried using:
with_file
with_item
with_lines
But I did not get the result i wanted.
any help would be appreciated.
You can use Python built-ins for some types, like strings, for example.
So this will do the trick for you:
file_contents_lines: "{{ lookup('file', './aaa.txt').splitlines() }}"
and
with_items: "{{ file_contents_lines }}"
Essentially, I want to be able to handle "wildcard filenames" in Linux using ansible. In essence, this means using the ls command with part of a filename followed by an "*" so that it will list ONLY certain files.
However, I cannot store the output properly in a variable as there will likely be more than one filename returned. Thus, I want to be able to store these results no matter how many there might be in an array during one task. I then want to be able to retrieve all of the results from the array in a later task. Furthermore, since I don't know how many files might be returned, I cannot do a task for each filename, and an array makes more sense.
The reason behind this is that there are files in a random storage location that are changed often, but they always have the same first half. It's their second half of their names that are random, and I don't want to have to hard code that into ansible at all.
I'm not certain at all how to properly implement/manipulate an array in ansible, so the following code is an example of what I'm "trying" to accomplish. Obviously it won't function as intended if more than one filename is returned, which is why I was asking for assistance on this topic:
- hosts: <randomservername>
remote_user: remoteguy
become: yes
become_method: sudo
vars:
aaaa: b
tasks:
- name: Copy over all random file contents from directory on control node to target clients. This is to show how to manipulate wildcard filenames.
copy:
src: /opt/home/remoteguy/copyable-files/testdir/
dest: /tmp/
owner: remoteguy
mode: u=rwx,g=r,o=r
ignore_errors: yes
- name: Determine the current filenames and store in variable for later use, obviously for this exercise we know part of the filenames.
shell: "ls {{item}}"
changed_when: false
register: annoying
with_items: [/tmp/this-name-is-annoying*, /tmp/this-name-is-also*]
- name: Run command to cat each file and then capture that output.
shell: cat {{ annoying }}
register: annoying_words
- debug: msg=Here is the output of the two files. {{annoying_words.stdout_lines }}
- name: Now, remove the wildcard files from each server to clean up.
file:
path: '{{ item }}'
state: absent
with_items:
- "{{ annoying.stdout }}"
I understand the YAML format got a little mussed up, but if it's fixed, this "would" run normally, it just won't give me the output I'm looking for. Thus if there were 50 files, I'd want ansible to be able to manipulate them all, and/or be able to delete them all.. etc etc etc.
If anyone here could let me know how to properly utilize an array in the above test code fragment that would be fantastic!
Ansible stores the output of shell and command action modules in stdout and stdout_lines variables. The latter contains separate lines of the standard output in a form of a list.
To iterate over the elements, use:
with_items:
- "{{ annoying.stdout_lines }}"
You should remember that parsing ls output might cause problems in some cases.
Can you try as below.
- name: Run command to cat each file and then capture that output.
shell: cat {{ item.stdout_lines }}
register: annoying_words
with_items:
- "{{ annoying.results }}"
annoying.stdout_lines is already a list.
From doc of stdout_lines
When stdout is returned, Ansible always provides a list of strings, each containing one item per line from the original output.
To assign the list to another variable do:
..
register: annoying
- set_fact:
varName: "{{annoying.stdout_lines}}"
# print first element on the list
- debug: msg="{{varName | first}}"
I have several files on a server that I need to download from an ansible playbook, but because the connection has good chances of interruption I would like to check their integrity after download.
I'm considering two approaches:
Store the md5 of those files in ansible as vars
Store the md5 of those files on the server as files with the extension .md5. Such a pair would look like: file.extension and file.extension.md5.
The first approach introduces overhead in maintaining the md5s in ansible. So everytime someone adds a new file, he needs to make sure he adds the md5 in the right place.
But as an advantage, there is a solution for this, using the built in check from get_url action in conjunction with checksum=md5. E.g.:
action: get_url: url=http://example.com/path/file.conf dest=/etc/foo.conf checksum=md5:66dffb5228a211e61d6d7ef4a86f5758
The second approach is more elegant and the narrows the responsibility. When someone adds a new file on the server, he will make sure to add the .md5 as well and won't even need to use the ansible playbooks.
Is there a way to use the checksum approach to match the md5 from a file?
If you wish to go with your method of storing the checksum in files on the server, you can definitely use the get_url checksum arg to validate it.
Download the .md5 file and read it into a var:
- set_fact:
md5_value: "{{ lookup('file', '/etc/myfile.md5') }}"
And then when you download the file, pass the contents of md5_value to get_url:
- get_url:
url: http://example.com
dest: /my/dest/file
checksum: "md5:{{ md5_value }}"
force: true
Note that it is vital to specify a path to a file in dest; if you set this to a directory (and have a filename in url), the behavior changes significantly.
Note also that you probably need the force: true. This will cause a new file to download every time you run it. The checksum is only triggered when files are downloaded. If the file already exists on your host it won't bother to validate the sum of the existing file, which might not be desirable.
To avoid the download every time you could stat to see if the file already exists, see what its sum is, and set the force param conditionally.
- stat:
path: /my/dest/file
register: existing_file
- set_fact:
force_new_download: "{{ existing_file.stat.md5 != md5_value }}"
when: existing_file.stat.exists
- get_url:
url: http://example.com
dest: /my/dest/file
checksum: "md5:{{ md5_value }}"
force: "{{ force_new_download | default ('false') }}"
Also, if you are pulling the sums/artifacts from some sort of web server you can actually get the value of the sum right from the url without having to actually download the file to the host. Here is an example using a Nexus server that would host the artifacts and their sums:
- set_fact:
md5_value: "{{ item }}"
with_url: http://my_nexus_server.com:8081/nexus/service/local/artifact/maven/content?g=log4j&a=log4j&v=1.2.9&r=central&e=jar.md5
This could be used in place of using get_url to download the md5 file and then using lookup to read from it.
With the stat module:
- stat:
path: "path/to/your/file"
register: your_file_info
- debug:
var: your_file_info.stat.md5
The elegant solution will be using the below 3 modules provided by ansible itself
http://docs.ansible.com/ansible/stat_module.html
use the stat module to extract the md5 value and register it in a variable
http://docs.ansible.com/ansible/copy_module.html
while using the copy module to copy the file from the server, register the return value of md5 in another variable
http://docs.ansible.com/ansible/playbooks_conditionals.html
use this conditional module to compare the above 2 variables and print the results whether the file is copied properly or not
Another solution is to use url lookup (tested on ansible-2.3.1.0):
- name: Download
get_url:
url: "http://localhost/file"
dest: "/tmp/file"
checksum: "md5:{{ lookup('url', 'http://localhost/file.md5') }}"
Wrote an ansible module with the help of https://pypi.org/project/checksumdir
The module can be found here
Example:
- get_checksum:
path: path/to/directory
checksum_type: sha1/md5/sha256/sha512
register: checksum
ansible version 1.9.1
cat files.yml
tasks:
- name: stat files
stat: path=~/{{ item }}
register: {{ item }}.stat
with_items:
- foo.zip
- bar.zip
- name: copy files
copy: src=~/{{ item }} dest=/tmp/{{ item }}
register: {{ item }}.result
when: {{ item }}.stat.stat.exists == False
with_items:
- foo.zip
- bar.zip
- name: unzip files
shell: cd /tmp/ && unzip -o {{ item }}
when: {{ item }}.result|changed == True
with_items:
- foo.zip
- bar.zip
ERROR: Syntax Error while loading YAML script
If so, how?
The preferred way would be to use synchronize module.
From ansible documentation ( http://docs.ansible.com/ansible/synchronize_module.html#synopsis )
This is a wrapper around rsync. Of course you could just use the command action to call rsync yourself, but you also have to add a fair number of boilerplate options and host facts. You still may need to call rsync directly via command or shell depending on your use case. The synchronize action is meant to do common things with rsync easily. It does not provide access to the full power of rsync, but does make most invocations easier to follow.
Here is an example:
- name: Sync files
synchronize: src=some/relative/path dest=/some/absolute/path