Ansible: Iterate and include multiple variable files - loops

Question

I have hundreds of files (generated through some application) which I am trying to iterate and include them as variable files.
See below files as example. There are many more variables in each of the files, I have toned down to make the example clear.
# cat /tmp/abc/dev1.yml
---
app_name: dev1
instance: dev
port: 1234
server: test1
#
# cat /tmp/abc/dev2.yml
---
app_name: dev2
instance: dev
port: 4567
server: test2
#
# cat /tmp/abc/dev3.yml
---
app_name: dev3
instance: dev
port: 2223
server: test3
#
Now, when I use these tasks in my playbook, I can see the variables (app_name, instance, port, etc) defined in the files (/tmp/abc/*.yml) in the output as ansible_facts.
- action: shell echo "{{ item }}"
with_fileglob: /tmp/abc/*
register: res
- include_vars: "{{ item.item }}"
with_items: res.results
when: item.changed == True
register: task1
This is my output, when I run the playbook.
root#vikas027:~# ansible-playbook -v configs.yml
PLAY [dev] **************************************************************
GATHERING FACTS ***************************************************************
ok: [vikas027.test.com]
TASK: [shell echo "{{ item }}"] ***********************************************
changed: [vikas027.test.com] => (item=/tmp/abc/dev3.yml) => {"changed": true, "cmd": "echo \"/tmp/abc/dev3.yml\"", "delta": "0:00:00.004915", "end": "2015-08-31 20:11:57.702623", "item": "/tmp/abc/dev3.yml", "rc": 0, "start": "2015-08-31 20:11:57.697708", "stderr": "", "stdout": "/tmp/abc/dev3.yml", "warnings": []}
changed: [vikas027.test.com] => (item=/tmp/abc/dev2.yml) => {"changed": true, "cmd": "echo \"/tmp/abc/dev2.yml\"", "delta": "0:00:00.004945", "end": "2015-08-31 20:11:58.130295", "item": "/tmp/abc/dev2.yml", "rc": 0, "start": "2015-08-31 20:11:58.125350", "stderr": "", "stdout": "/tmp/abc/dev2.yml", "warnings": []}
changed: [vikas027.test.com] => (item=/tmp/abc/dev1.yml) => {"changed": true, "cmd": "echo \"/tmp/abc/dev1.yml\"", "delta": "0:00:00.004864", "end": "2015-08-31 20:11:58.440205", "item": "/tmp/abc/dev1.yml", "rc": 0, "start": "2015-08-31 20:11:58.435341", "stderr": "", "stdout": "/tmp/abc/dev1.yml", "warnings": []}
TASK: [include_vars {{ item.item }}] ******************************************
ok: [vikas027.test.com] => (item={u'cmd': u'echo "/tmp/abc/dev3.yml"', u'end': u'2015-08-31 20:11:57.702623', u'stderr': u'', u'stdout': u'/tmp/abc/dev3.yml', u'changed': True, u'rc': 0, 'item': '/tmp/abc/dev3.yml', u'warnings': [], u'delta': u'0:00:00.004915', 'invocation': {'module_name': u'shell', 'module_args': u'echo "/tmp/abc/dev3.yml"'}, 'stdout_lines': [u'/tmp/abc/dev3.yml'], u'start': u'2015-08-31 20:11:57.697708'}) => {"ansible_facts": {"app_name": "dev3", "instance": "dev", "port": 2223, "server": "test3"}, "item": {"changed": true, "cmd": "echo \"/tmp/abc/dev3.yml\"", "delta": "0:00:00.004915", "end": "2015-08-31 20:11:57.702623", "invocation": {"module_args": "echo \"/tmp/abc/dev3.yml\"", "module_name": "shell"}, "item": "/tmp/abc/dev3.yml", "rc": 0, "start": "2015-08-31 20:11:57.697708", "stderr": "", "stdout": "/tmp/abc/dev3.yml", "stdout_lines": ["/tmp/abc/dev3.yml"], "warnings": []}}
ok: [vikas027.test.com] => (item={u'cmd': u'echo "/tmp/abc/dev2.yml"', u'end': u'2015-08-31 20:11:58.130295', u'stderr': u'', u'stdout': u'/tmp/abc/dev2.yml', u'changed': True, u'rc': 0, 'item': '/tmp/abc/dev2.yml', u'warnings': [], u'delta': u'0:00:00.004945', 'invocation': {'module_name': u'shell', 'module_args': u'echo "/tmp/abc/dev2.yml"'}, 'stdout_lines': [u'/tmp/abc/dev2.yml'], u'start': u'2015-08-31 20:11:58.125350'}) => {"ansible_facts": {"app_name": "dev2", "instance": "dev", "port": 4567, "server": "test2"}, "item": {"changed": true, "cmd": "echo \"/tmp/abc/dev2.yml\"", "delta": "0:00:00.004945", "end": "2015-08-31 20:11:58.130295", "invocation": {"module_args": "echo \"/tmp/abc/dev2.yml\"", "module_name": "shell"}, "item": "/tmp/abc/dev2.yml", "rc": 0, "start": "2015-08-31 20:11:58.125350", "stderr": "", "stdout": "/tmp/abc/dev2.yml", "stdout_lines": ["/tmp/abc/dev2.yml"], "warnings": []}}
ok: [vikas027.test.com] => (item={u'cmd': u'echo "/tmp/abc/dev1.yml"', u'end': u'2015-08-31 20:11:58.440205', u'stderr': u'', u'stdout': u'/tmp/abc/dev1.yml', u'changed': True, u'rc': 0, 'item': '/tmp/abc/dev1.yml', u'warnings': [], u'delta': u'0:00:00.004864', 'invocation': {'module_name': u'shell', 'module_args': u'echo "/tmp/abc/dev1.yml"'}, 'stdout_lines': [u'/tmp/abc/dev1.yml'], u'start': u'2015-08-31 20:11:58.435341'}) => {"ansible_facts": {"app_name": "dev1", "instance": "dev", "port": 1234, "server": "test1"}, "item": {"changed": true, "cmd": "echo \"/tmp/abc/dev1.yml\"", "delta": "0:00:00.004864", "end": "2015-08-31 20:11:58.440205", "invocation": {"module_args": "echo \"/tmp/abc/dev1.yml\"", "module_name": "shell"}, "item": "/tmp/abc/dev1.yml", "rc": 0, "start": "2015-08-31 20:11:58.435341", "stderr": "", "stdout": "/tmp/abc/dev1.yml", "stdout_lines": ["/tmp/abc/dev1.yml"], "warnings": []}}
PLAY RECAP ********************************************************************
vikas027.test.com : ok=3 changed=1 unreachable=0 failed=0
root#vikas027:~#
How can I reference variables like app_name, instance, port, etc in other tasks ? I tried using below code and few other combinations in vain.
- debug: msg="{{ task1.app_name }}"
with_items: task1.results

Answer 1

Your variable files, dev1.yml, dev2.yml, etc. all reference the same variable names. Is this on purpose, or just part of your example? I ask because your example as it's currently shown, would result in just the last set of variables being defined, so as far as ansible is concerned it appears that the variables would ultimately just be defined as if you did this:
vars:
app_name: dev3
instance: dev
port: 2223
server: test3
You would just reference the variables by their given names:
- debug: var=app_name
- debug: var=instance
etc.
What I'm guessing you actually want to be doing is having those variable files look something like this:
---
app:
dev1:
instance: "dev"
port: "1234"
server: "host1"
and
---
app:
dev2:
instance: "dev"
port: "4321"
server: "host2"
You would then reference your objects something like this:
# should list "dev1", "dev2", "dev3"...
- debug: msg={{ item }}
with_dict: app
# should list the server names for each device
- debug: var = app[item][server]
with_dict: app

Answer 2

I was working on this whole day today, tried umpteen configuration changes in vain. Finally, it is working the way I wanted it to work.
This is what one needs to do if in a similar situation. Hope this helps someone.
First, register your facts locally. I chose the default /etc/ansible/facts.d/ directory for the same. Here are more details.
Key things to remember:-
Extension should be .fact
File should be executable (I gave 0755)
Format is JSON (I've used yaml-to-json to convert my yaml files to json. You can use ruby or perl one-liners too.)
Then, to iterate the facts registered in the previous step, we need to load/reload the facts in the playbook in order to use them in tasks/playbooks.
- local_action: setup filter=ansible_local
- template: src=nginx_lb.conf.j2 dest=/etc/nginx/conf.d/{{ item.key }}.conf
with_dict: "{{ ansible_local }}"
All variables can now be used in the jinja2 template. For example, port can be referenced as item.value.port.