I've created an Ansible playbook that creates a cloud instance and then installs some programs on the instance. I want to run this playbook multiple times (without using a bash script). Is it possible to use a loop to loop over those two tasks together (I.E. One loop for two tasks?). All I've been able to find so far is one loop for each individual task
An update:
In 2.0 you are able to use with_ loops and task includes (but not playbook includes), this adds the ability to loop over the set of tasks in one shot. There are a couple of things that you need to keep in mind, a included task that has it’s own with_ loop will overwrite the value of the special item variable. So if you want access to both the include’s item and the current task’s item you should use set_fact to create a alias to the outer one.:
- include_tasks: test.yml
with_items:
- 1
- 2
- 3
in test.yml:
- set_fact: outer_loop="{{item}}"
- debug: msg="outer item={{outer_loop}} inner item={{item}}"
with_items:
- a
- b
- c
Source: Ansible Docs
No that's currently not possible. with_items used to work with the include statement in previous versions of Ansible but was unfortunately dropped.
Though it will be brought back in Ansible 2.0, see slide 14/15 of What's New in v2 - AnsibleFest London 2015
You could try to work with the v2 branch from github, the feature should be available in there.
What you can do with 1.9.1 is to move your tasks into a role and reference this role multiple times in your playbook.
I managed to do this by recursively including the same yaml file based on a condition. Here is the gist: https://gist.github.com/ParagDoke/5ddfc3d5647ce9b0110d1b9790090092. Effectively, in your playbook, include a file with some vars:
- name: Invoke poller
vars:
some_condition: '"failed" not in response.content and response.json.status=="running"'
include_tasks: status-poller.yml
Then in status-poller.yml, include itself:
- include_tasks: includes/status-poller.yml
when: some_condition
Related
I am using getdbt on redshift for data analytics operation. Can anyone please suggest, how to use --selector & --defer with "dbt run" commands.
What is the syntax ? What is the use of selectors.yml file?
Please share some examples.
Thanks
My interpretation of defer is a way to utilize the dbt cli to work with unbuilt or differential versions of the current & futures state defined versions of a model.
Example of why you may want to interact with that here: #2740 - Automating Non Regression Test
selectors being a relatively new feature, I also haven't seen much documentation to back this up but it is effectively a naming convention for a set of logical criteria (more than 1 tag, multiple directories, etc.)
I'd recommend this article in general for understanding the build path generation of a typical dbt run: How we made dbt runs 30% faster
From there, you can imagine that within a large project, there are huge interconnecting chains for each raw -> analytics ready transformation pipeline that you have.
We'll use Gitlab's open dbt project as an example.
Gitlab doesn't currently use selectors but they do make use of tags.
So they could build up a selectors.yml file using logical definitions like:
selectors.yml
selectors:
- name: sales_funnel
definition:
tag: salesforce
tag: sales_funnel
- name: arr
description: builds all arr models to current state + all upstream dependencies (zoho, zuora subscriptions, etc.)
default: true
definition:
tag: zuora_revenue
tag: arr
- name: month_end_process
description: builds reporting models about customer segments based on subscription activity for latest closed month
definition:
- union:
- method: fqn
value: rpt_available_to_renew_month_end
greedy: eager # default: will include all tests that touch selected model
- method: fqn
value: rpt_possible_to_churn_month_end
greedy: eager
Full list of valid selector definitions here: https://docs.getdbt.com/reference/node-selection/yaml-selectors#default
What that gives them the ability to do is on a cron job, via airflow, or some other orchestrator simply execute:
dbt run --selector month_end_process --full-refresh
And have confidence that the logical selection of models to run for that process is 100% accurately reproduced instead of another more fallible approach like assuming that all the models needed are in a single directory:
dbt run --models marts.finance.restricted_safe.reports --full-refresh
Architecturally, you likely won't need selectors until you get to the level of having multiple layers of tags and / or multiple layers of use-case directories to be mindful of within a single run.
Example: tags for the models' function, tags for the sources, tags for the bi/analyst consumers, tags for the materialization schedule, etc.
I have created a react environment variable in .env file and is able to update it through command prompt or power shell now my aim is to update it through azure pipeline so add a powershell script.But I am not able to figure out how to read or write data in .env file through a azure pipeline powershell script. I wish to change base url for differnet envirnoments dev stage prod (base_url_dev = https://projectName.dev.azurewebsites.net/). Please let me know If there is any other way
You would commit your file like this:
base_url = '#{base_url}#' (never commit enivornment specific values into Source Code)
And then set a base_url variable in each AzDo Stage to the actual value. (You can also use stage scoped variable groups)
in each stage, use a replacetokens step and target your .env file to replace the #{base_url}# to the actual value you declared for each stage.
update source code of react through azure pipeline
We could remove the base url from the .env file, then use a Inline powershell task to set the base url via Logging Command：
Write-Host "##vso[task.setvariable variable=REACT_BASE_URL;]$(REACT_BASE_URL)"
Then define the variable REACT_BASE_URL with different value for different stages in the Variables tab:
Alternatively, just like michiel said, we could use the task Replace Tokens to update the value in the .env file.
Change the base_urls values to #{REACT_BASE_URL}#`：
REACT_BASE_URL = #{REACT_BASE_URL}#
Then, also defined the variable REACT_BASE_URL with different value for different stages in the Variables tab.
And add the task Replace Tokens in the pipeline:
I'm currently making the transition from Puppet to Ansible and so far so good. Yet I want to automatize as much as possible.
I'm trying to use a with_items loop inside vars_files to load variable files based on a given list of items. Ansible complains about the syntax and I can't seem to find an example of a similar solution, only examples that use with_items inside tasks and roles.
For example:
vars_files:
- ["vars/{{ item }}-{{ ansible_fqdn }}.yml", "vars/{{ item }}-{{ system_environment }}.yml", "vars/{{ item }}.yml"]
with_items:
- php
- nginx
The goal here is to loop the second line for as long as there are items in with_items using an array to fallback on the next item if it can't find the given file (which works).
Not sure if this is at all possible, but I wanted to ask before taking another direction.
with_items, or in general all loops, are a feature of tasks. vars_files though is no task. So it won't work the way you tried it and the short answer would be: It is not possible.
I don't know of a clean way to solve your exact problem. A custom vars plugin might be an option. But vars plugin work on a global level while your vars seem to be used in a role.
A custom lookup plugin might be a solution if solving this on task level is an option for you. The lookup plugin takes your input, checks for presence of the files and returns an array of the files which need to be include. This then can be used with the include_vars module.
- include_vars: "{{ item }}"
with_my_custom_plugin:
- php
- nginx
An ugly solution would be to combine the with_items loop with a with_first_found loop. Though, since you cannot directly nest loops, you need to work with an include.
- include: include_vars.yml
with_items:
- php
- nginx
And inside include_vars.yml you then can use with_first_found with the include_vars module.
- include_vars: "{{ item }}"
with_first_found:
- vars/{{ item }}-{{ ansible_fqdn }}.yml
- vars/{{ item }}-{{ system_environment }}.yml
- vars/{{ item }}.yml
Putting this in a separate answer to expand on the group and host variables solution I eventually came up with (cc #udondan).
Basically I group all my hosts in my inventory file under several sub and parent groups no matter what. Then I create files for group vars whenever applicable so it follows a certain order of precedence (first is highest and overrides all others, last applies to all hosts and can be overridden down the chain):
task vars > playbook vars > host_vars > web/database-local > local > web/database > all
That way I can define variables for all hosts to use (all), just web/database (mostly production values), all local servers (local group), all local web/database servers, et cetera, or per-host (the standard host_vars). Of course playbook and task vars override these further. All of this following the Ansible guidelines.
An example of a local inventory (replace default with your hostname or IP, add as many as you like per group, x-local can be omitted if this would be a production inventory):
[web-local]
default
[database-local]
default
[local:children]
web-local
database-local
[web:children]
web-local
[database:children]
database-local
Then my group_vars folder with directories for each inventory group and variables split into files to keep it structured (could just have one database-local.yaml file for the database-local group for instance instead of folders and split YAML-files):
group_vars/
all/
always_applied_variables.yaml
swap.yaml
web/
database/
database_only_variables.yaml
database-production/
production_database_variables.yaml
production/
random_production_only_variables.yaml
local/
users.yaml
web-local/
database-local/
local_database_variables.yaml
host_vars/
default/
php.yaml
mysql.yaml
other_specific_host_variables.yaml
Hope this is somewhat clear. I'd be happy to answer any questions.
In my playbooks, I SSH in as a non-root user, then use become to become root.
What is the Ansible variable name that stores the user that originally SSH'd into the box?
In Ansible < 2.0, I could use {{ ansible_ssh_user }} to access the username SSH'ing into the box.
Just tried with Ansible 2.0.2 and that returns null. I tried ansible_user as suggested by the FAQ, but that also returns null. I also tried ansible_user_id, but that returns the result of become, not the original user.
You can accesss this via ansible_env.SUDO_USER.
I tried a number of other variables, and almost all of them changed their values as soon as I used become on the remote node.
{{ ansible_user }} does actually work fine for me with Ansible 2.5:
with_items:
- root
- "{{ ansible_user }}"
I am writing a playbook that creates a network with a name I assign. Later on in the playbook I need to access the IP that is assigned by this network task, so I get it from the hostvars. So for example it if I was calling the network 'my_website' the value I would be targeting in hostvars would be
server: "{{hostvars.localhost.rax_nfs.results[0].success[0].rax_addresses.my_website[0].addr}}"
This is fine, but I want to name the network based on the contents of a variable passed in by a var file to make it reusable across multiple setups, and then still be able to get that IP back, so
network_label: "{{ my_website }}"
server: "{{hostvars.localhost.rax_nfs.results[0].success[0].rax_addresses.network_label[0].addr}}"
Obviously this doesn't work as it just assigns a string. How do I use network_label as the key inside another variable? Like in php something like
$array[$variable], or $object->$variable
Is this possible?
Not sure what the playbook looks like but does accessing the IP based on the gathered facts not work in your context? Example: {{ ansible_eth0.ipv4.address }}
Alternatively on the task that creates the network you could register that as a variable for later access using that variable name.
You can also access elements of an array in a variable: {{ var_name[0] }}