Ansible custom nested looping - loops

I have to process output from CloudFormation Outputs that is:
Ansible code that produces this output:
- debug:
var: stack.stack_outputs
Output:
ok: [localhost] => {
"stack.stack_outputs": {
"Roles": "webserver balancer dbserver",
"dbserver": "54.0.1.1 54.0.1.2",
"balancer": "54.0.2.3",
"webserver": "54.0.2.5 54.0.2.7 54.0.3.1"
}}
With that, I want to create 3 (dynamic number!) groups named accordingly filled with appropriate IPs.
Ansible code that I want to HELP WITH:
- name: fill roles with proper hosts
local_action: add_host hostname={{item}} groupname={{role}}
with_whatever: ?...?
In pseudo ansible python it would look like this:
for role in stack.stack_outputs.Roles.split(): # Python
for ip in stack.stack_outputs[role].split(): # Python
local_action: add_host hostname={{ip}} groupname={{role}} # Ansible
Note:
The way to do it for these three roles statically is obviously:
- name: fill role WEBSERVER
local_action: add_host hostname={{item}} groupname=webserver
with_items: stack.stack_outputs.webserver.split()
- name: fill role DBSERVER
local_action: add_host hostname={{item}} groupname=dbserver
with_items: stack.stack_outputs.dbserver.split()
- name: fill role BALANCER
local_action: add_host hostname={{item}} groupname=balancer
with_items: stack.stack_outputs.balancer.split()
I want to do it dynamically, is it even possible in Ansible?
Yes, I can use shell module to hack it putting everything in temporary file and then looping over that; but is there a better solution?
Thanks for any suggestions.

I understand you want the answer to fit a very specific framework. Within that, a custom lookup_plugin is your best bet. Otherwise it'll be an ugly sequence of set_fact and add_host. Sophisticated control structures are the antithesis of Ansible.
You don't explicitly rule out the following, so even if it's too out of the box for you, consider it because I've been reconciling cfn and ansible for a good long while:
Don't use stack outputs to fill your inventory. Use a dynamic inventory script for that (e.g. one that goes over stack outputs or tags set in the templates).
I'm aware of the implications like, you can't have this in a single playbook, but if that's paramount, use group_by.
Hope this helps.

For anyone that comes asking without reading the documentation first. Ansible has been updated to support nested indexing:
http://docs.ansible.com/ansible/playbooks_loops.html#nested-loops
Cheers!

Related

Access list sub-elements with wildcard in Ansible?

I want to loop through the sub-elements of a list in Ansible.
I have a var file like this:
config:
machine1:
services:
sub-element11:
- value
ports:
sub-element21:
- value
- value
sub-element22:
- value
- value
machine2:
.
.
.
And I want to loop through the sub-elements while not giving the name of the element that comes before it:
- debug
var: item
with_items: "{{ config.machine1.* | list }}"
where item would have the value: sub-element11, sub-element21, sub-element22
I can use 2 loops as I know I'll always have "services" and "ports" as a key-word. But I don't want to clone everything that comes after.
Maybe I can use nested loops with set facts but is seems tedious.
Thank you for your help :)

Run a command with different args everytime in Ansible playbook

I am trying to run a command in Ansible so as to find the neighbors in my network:
- name: Get neighbors
junos_rpc:
rpc: "get-lldp-interface-neighbors"
output: 'xml'
args:
interface_device: A
register: net_topology
So my problem comes when in this task I need to loop over a list and give another arg for the interface_device and register the result also in another variable 'net_topology' every time.
- name: Get neighbors
junos_rpc:
rpc: "get-lldp-interface-neighbors"
output: 'xml'
args:
interface_device: "{{ item }}"
loop:
- A
- B
- C
register: net_topology
Once you modify your task like this, it will play three times: once for each element in my example loop. The variable item will get the value of the current element in the list.
You do not need to change your register variable: it will automatically be modified as explained in the ansible documentation:
When you use register with a loop, the data structure placed in the variable will contain a results attribute that is a list of all responses from the module. This differs from the data structure returned when using register without a loop
So you can inspect all your results in a subsequent task by looping over net_topology.results which contains the list of individual results.
Actually i did something similar with the above, but i just passed my list with a different way:
- name: building network topology
junos_rpc:
rpc: "get-lldp-interface-neighbors"
output: 'xml'
args:
interface_device: "{{item}}"
loop:
"{{my_list}}"
register: net_topology
And this is actually the same as doing this also:
- name: building network topology
junos_rpc:
rpc: "get-lldp-interface-neighbors"
output: 'xml'
args:
interface_device: "{{item}}"
with_items:
"{{my_list}}"
register: net_topology
I must say that my initial mistake was the identation of the loop, because it was placed inside the junos_rpc and by doing this I could not get any result !!!

Concourse CI - array variable

I'm trying to figure out how to create an array with some CIDR ip address in order to have the same array in my pipeline. So here is an example var file:
whitelist-ip-ranges: |-
- 10.0.0.0/24
- 11.0.0.0/24
- 12.0.0.0/24
My pipeline is like:
....
....
....
params:
variables:
loadBalancerSourceRanges:
{{whitelist-ip-ranges}}
And I want it to be:
....
....
....
params:
variables:
loadBalancerSourceRanges:
- 10.0.0.0/24
- 11.0.0.0/24
- 12.0.0.0/24
or
....
....
....
params:
variables:
loadBalancerSourceRanges: [10.0.0.0/24,11.0.0.0/24,12.0.0.0/24]
Inside my helm template I have my values.yaml file I have of course:
loadBalancerSourceRanges: null
and it will be override by the pipeline.
And finaly, in my service file I'm making a loop:
{{if .Values.loadBalancerSourceRanges}}
loadBalancerSourceRanges:
{{range $rangeList := .Values.loadBalancerSourceRanges}}
- {{ $rangeList }}
{{end}}
{{end}}
Does any of you guys was able to do something like that?
I'm sorry, I cannot speak to anything helm based. I can speak for a concourse pipeline, though.
Concourse does not support providing params to tasks that are an array. params are passed in as environment variables to a running task, so they are transformed from YAML to a simple string key-value pair.
If you want to pass more complex information. There are two options:
encode the param as JSON/YAML so that it can be parsed as string from your tasks environment
provide the task an input from a resource, where a file can be provided -- for example an s3 resource with the contents of the loadBalanceSourceRanges
These programmatic ways are examples I've used before to accomplish passing more complex data (ie arrays) to a task.

How to use template parameters in page content in hugo

Is it possible to use template parameters in the content of a post with Hugo? E.g. if I have the following parameters:
[params.company]
name = "My Company"
Can I then do something like this in the content of a post?
This site, {{ .Site.BaseURL }} is operated by {{ params.company.name }}
I've tried, but it's literally printing the above instead of interpolating the variables.
1. Front matter way
As far as I'm aware, it's not possible* to put variables within the markdown file's content because MD parser would strip them, but it's possible to do it using custom variables on the front matter of each .md content file. The Hugo engine can target any fields you set in front matter. Front matter fields can be unique as well.
In your case, the template which is called to show the rendered .MD file has access to front matter parameters and you can change template's behaviour (like add classes of extra <div>'s) or even pull the content right from the parameter.
For example, on my .md files I have:
---
title: "Post about the front matter"
<more key pairs>
nointro: false
---
The key nointro: true would make my first paragraph to be normal size. Otherwise, if key is absent or false, first paragraph would be shown at larger font size. Technically, it's adding a custom class on a <div>.
In the template, I tap into the custom parameter nointro this way:
parent template which shows your markdown file, which has front matter parameters:
<div class="article-body {{ if .Params.nointro }} no_intro {{ end }}">
{{ .Content }}
</div><!-- .article-body -->
Notice I can't put variables within {{ .Content }}, but I can outside of it.
For posterity, that's piece of the content from a file hugo/themes/highlighter/layouts/partials/blog-single-content.html, it's a partial for single post content. You can arrange your partials any way you like.
Obviously, that's Boolean parameter flag, but equally it could be content which you could use directly:
MD file's top:
---
title: "One of our clients"
<more key pairs>
companyname: "Code and Send Ltd"
---
Text content is put here.
Then, reference it like this (extra insurance against blank value using IF):
Anywhere in Hugo template files:
{{ if .Params.companyname }}{{ .Params.companyname }}{{ end }}
2. Using config.(toml/yaml/json)
Now, looking at your example, "This site is operated by " would almost warrant a custom field in more global location, for example, hugo/config.toml. If I wanted to add a companyname into my config, I'd do it this way:
hugo/config.toml:
BaseURL = "_%%WWWPATH%%_"
languageCode = "en-uk"
title = "Code and Send"
pygmentsUseClasses = true
author = "Roy Reveltas"
theme = "Highlighter"
[params]
companyname = ""
Then I'd use it anywhere via {{ .Site.Params.headercommentblock }}.
I guess if you want your client pages to be static pages then single location might not be the best and you might want to tap into front-matter. Otherwise, if it's a site's footer, this way is better. Alternatively, you could even put this data even on data files.
3. Using custom placeholders and replacing via Gulp/npm scripts
I said not possible*, but it's possible, although unconventional and more risky.
I had such setup when I needed two builds for my website: 1) Prod and 2) Dev. Prod URL's were coming from two places: CDN and my server. Dev had to come from a single place in a static folder because I wanted to see images and was often working offline on a train.
To solve that, I used two custom variables in all my templates (including markdown content): _%%WWWPATH%%_ and _%%CDNPATH%%_. I came up with this unique pattern myself by the way, feel free to adapt it. Then, I put it also on hugo/config.toml as:
hugo/config.toml:
BaseURL = "_%%WWWPATH%%_"
After Hugo happily generated the website with those placeholders, I finished off the HTML's using a Grunt task:
grunt file:
replace: {
dev: {
options: {
patterns: [{
match: /_%%CDNPATH%%_+/g,
replacement: function () {
return 'http://127.0.0.1:1313/'
}
}, {
match: /_%%WWWPATH%%_+/g,
replacement: function () {
return 'http://127.0.0.1:1313/'
}
}...
For posterity, I recommend Gulp and/or npm scripts, I'd avoid Grunt. This is my older code example above from the days when Grunt was the best.
If you go this route, it's riskier than Hugo params because Hugo won't error-out when your placeholder values are missing or anything else wrong happens and placeholders might spill into the production code.
Going this route you should add multiple layers of catch-nets, ranging from simple following Gulp/Grunt/npm scripts step which searches for your placeholder pattern to pre-commit hooks ran via Husky on npm scripts that prevent from committing any code that has certain patterns (for example, %%_). For example, at a very basic level, you would instruct Husky to search for anything before allowing committing this way:
package.json of your repo:
"scripts": {
"no-spilled-placeholders": "echo \"\n\\033[0;93m* Checking for spilled placeholders:\\033[0m\"; grep -q -r \"%%_\" dist/ && echo \"Placeholders found! Stopping.\n\" && exit 1 || echo \"All OK.\n\"",
"precommit": "npm run no-spilled-placeholders"
},
Basically, grep for pattern %%_ and exit with error code if found. Don't forget to escape the code because it's JSON. I use similar (more advanced) setup in production and nothing slips through. In proper setup you should creatively look for anything mis-typed, including: %_, _%, %__, __% and so on.
Normal Go template is not supported in the markdown file, but shortcodes are:
{{< param "company.name" >}}
To access arbitrary other Go template values, create a custom shortcode for it and call that from your markdown file.
For your example, you need the site's baseUrl, so save this as layouts/shortcodes/base_url.html:
{{ .Site.BaseURL }}
And write this in your markdown file:
+++
[company]
name = "My Company"
+++
This site, {{< base_url >}} is operated by {{< param "company.name" >}}
There is also the shortcode param : {{< param "companyName" >}} : https://gohugo.io/content-management/shortcodes/#param

Build an array from yaml in rails

I'm working on a simple rails app that does SMS. I am leveraging Twilio for this via the twilio_ruby gem. I have 10 different phone numbers that I want to be able to send SMS from randomly.
I know if I do something like this:
numbers = ["281-555-1212", "821-442-2222", "810-440-2293"]
numbers.sample
281-555-1212
It will randomly pull one of the values from the array, which is exactly what I want. The problem is I don't want to hardcode all 10 of these numbers into the app or commit them to version control.
So I'm listing them in yaml (secrets.yml) along with my Twilio SID/Token. How can I build an array out of the 10 yaml fields i.e. twilio_num_1, twilio_num_2, etc, etc so that I can call numbers.sample?
Or is there a better way to do this?
You can also use
twilio_numbers:
- 281-555-1122
- 817-444-2222
- 802-333-2222
thus you don't have to write the numbers in one line.
Figured this out through trial and error.
In secrets.yml
twilio_numbers: ["281-555-1122","817-444-2222","802-333-2222"]
In my code:
Rails.application.secrets.twilio_numbers.sample
Works like a charm.
create a file: config/twilio_numbers.yml
---
- 281-555-1122
- 817-444-2222
- 802-333-2222
and load it in your config/application.rb like this:
config.twilio_numbers = YAML.load_file 'config/twilio_numbers.yml'
you can then access the array from inside any file like this:
Rails.application.config.twilio_numbers
=> ["281-555-1122", "817-444-2222", "802-333-2222"]

Resources