Terraform for_each, Count Index - loops

I am trying to access all the values from a for_each statement in google_compute_instance resource
I want to get all the values [dev-1, dev-2] in thename attribute and parse it to vm_name in my metadata_startup_script
resource "google_compute_instance" "compute_instance" {
project = var.project_id
for_each = toset(["1", "2"])
name = "dev-${each.key}"
machine_type = "e2-micro"
zone = "${var.region}-b"
boot_disk {
initialize_params {
image = "ubuntu-os-cloud/ubuntu-1804-lts"
}
}
network_interface {
network = "default"
access_config {
}
}
lifecycle {
ignore_changes = [attached_disk]
}
metadata_startup_script = templatefile("copy.tftpl", {
vm_name = "${google_compute_instance.compute_instance.0.name}"
nfs_ip = "${google_filestore_instance.instance.networks[0].ip_addresses[0]}"
file_share_name = "${google_filestore_instance.instance.file_shares[0].name}"
zone = "${var.region}-b"
})
}
I am unable to get all compute instance from the name argument
I get this error message
╷
│ Error: Cycle: google_compute_instance.compute_instance["2"], google_compute_instance.compute_instance["1"]
│
│
How do I resolve this issue so I can get all the virtual machine name and parse it to vm_name variable?

I would change the hardcoded elements in your for_each to a variable,
and pass that to your vm_name, something like this:
locals {
compute_names = ["dev-1", "dev-2"]
}
resource "google_compute_instance" "compute_instance" {
project = var.project_id
for_each = toset(local.compute_names)
name = each.key
machine_type = "e2-micro"
zone = "${var.region}-b"
...
metadata_startup_script = templatefile("copy.tftpl", {
vm_name = local.compute_names
...
zone = "${var.region}-b"
})
}

Related

Terraform - Adding specific users to a specific group

I have a 2 csv files named "_member.csv" and "_groups.csv".
Content from _member.csv :
first_name,last_name,department,job_title
Albator,Albator,Exploitation,Integrateur d'exploitation
Bingo,Bingo,Exploitation,Ingenieur de production
Cresus,Cresus,Etudes et developpements, Analyste developpeur
Content from _groups.csv :
members_groups
adm-xxx
exp-xxx
cod-xxx
tec-xxx
met-xxx
svc-xxx
My resource to create users :
resource "azuread_user" "Terra-Aad-User-Member" {
for_each = { for user in local.azure_members : user.first_name => user }
user_principal_name = format(
"%s%s#%s",
substr(lower(each.value.first_name), 0, 1),
lower(each.value.last_name),
local.domain_name
)
password = format(
"%s%s%s!",
lower(each.value.last_name),
substr(lower(each.value.first_name), 0, 1),
length(each.value.first_name)
)
force_password_change = true
display_name = "${each.value.first_name} ${each.value.last_name}"
department = each.value.department
job_title = each.value.job_title
}
My resource to create groups :
resource "azuread_group" "Terra-Aad-Group" {
for_each = { for group in local.azure_groups : group.members_groups => group }
display_name = format("%s", lower(each.value.members_groups))
security_enabled = true
}
My resource to add users to groups :
resource "azuread_group_member" "Terra-Aad-Member-In-Group" {
for_each = { for u in azuread_user.Terra-Aad-User-Member : u.department => u... if u.department == "Exploitation" }
group_object_id = values(azuread_group.Terra-Aad-Group).0.object_id
member_object_id = each.value.id
}
The provider :
terraform {
required_providers {
azuread = {
source = "hashicorp/azuread"
version = "=2.28.1"
}
}
}
provider "azuread" {
tenant_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
data "azuread_domains" "Terra-Aad" {
only_initial = true
}
My variables.tf
locals {
domain_name = data.azuread_domains.Terra-Aad.domains.0.domain_name
azure_members = csvdecode(file("${path.module}/_member.csv"))
azure_groups = csvdecode(file("${path.module}/_groups.csv"))
}
When I launch 'terraform plan', I have this error :
│ Error: Unsupported attribute
│
│ on groupmember.tf line 5, in resource "azuread_group_member" "Terra-Aad-Member-In-Group":
│ 5: member_object_id = each.value.id
│ ├────────────────
│ │ each.value is tuple with 2 elements
│
│ This value does not have any attributes.
Is it possible to automatically add all users who have the department 'Exploitation' to the group 'adm-xxx'? The problem is in the line 'member_object_id = each.value.id' I may need a conditional expression but I don't know exactly what that line is.
Does anyone have an idea?
Ok. So I added a null_resource to get all the ids of my users who are on the "Exploitation" department like this :
resource "null_resource" "Id-Department" {
provisioner "local-exec" {
command = "az ad user list --filter \"department eq 'Exploitation'\" --query \"[].id\" -o tsv"
interpreter = ["PowerShell", "-Command"]
}
depends_on = [azuread_user.Terra-Aad-User-Member]
}
I modified my resource azuread_group_member like this :
resource "azuread_group_member" "Terra-Aad-Member-In-Group" {
for_each = { for u in azuread_user.Terra-Aad-User-Member : u.department => u... if u.department == "Exploitation" }
group_object_id = values(azuread_group.Terra-Aad-Group).0.object_id
member_object_id = null_resource.Id-Department
depends_on = [null_resource.Id-Department]
}
Now I get this error :
│ Error: Incorrect attribute value type
│
│ on groupmember.tf line 13, in resource "azuread_group_member" "Terra-Aad-Member-In-Group":
│ 13: member_object_id = null_resource.Id-Department
│ ├────────────────
│ │ null_resource.Id-Department is object with 2 attributes
│
│ Inappropriate value for attribute "member_object_id": string required.
What I need is to take all the results of the command in null_resource to this: member_object_id = null_resource.Id-Department
Is there a way to do this ?
For starters, it seems most simple just to extract the groups to a list. Then for azuread_group, you can just use toset. Once that is done, you can reference this group by the group name in the object. So within azuread_group_member you use the for expression to get only members that are have "Exploitation" - you can reference their member_object_id by each.value.object_id. And you can get the group_object_id from the set-built azuread_group objects.
locals {
azure_groups = [for v in csvdecode(file("${path.module}/_groups.csv")) : v.members_groups]
azure_members = csvdecode(file("${path.module}/_member.csv"))
}
resource "azuread_user" "Terra-Aad-User-Member" {
for_each = { for user in local.azure_members : "${user.first_name}-${user.last_name}" => user }
user_principal_name = format(
"%s%s#%s",
substr(lower(each.value.first_name), 0, 1),
lower(each.value.last_name),
local.domain_name
)
password = format(
"%s%s%s!",
lower(each.value.last_name),
substr(lower(each.value.first_name), 0, 1),
length(each.value.first_name)
)
force_password_change = true
display_name = "${each.value.first_name} ${each.value.last_name}"
department = each.value.department
job_title = each.value.job_title
}
resource "azuread_group" "Terra-Aad-Group" {
for_each = toset(local.azure_groups)
display_name = lower(each.value)
security_enabled = true
}
resource "azuread_group_member" "Terra-Aad-Member-In-Group" {
for_each = {
for k, v in azuread_user.Terra-Aad-User-Member :
k => v if v.department == "Exploitation"
}
group_object_id = azuread_group.Terra-Aad-Group["adm-xxx"].object_id
member_object_id = each.value.object_id
}
As an aside, I have no idea how a null_resource is relevant here. You should have all the information you need without it.

Better handling of multiple loops within 1 resource with Terraform

I'm trying to create multiple CNAME resources based on multiple A record resources and am struggling to find a way to achieve this.
I currently have the following in my variable.tfvars file:
hostnames = [
"webserver1",
"webserver2"
]
domain = domain.com
subdomain_addresses = [
"www-1",
"www-2",
"www-3"
]
I have the following output from my ec2 instance module which is based off the length of var.hostnames: (in this instance there will be 2 private IPs)
output "server_ip_addresses" {
value = aws_instance.this.*.private_ip
}
I currently have the following in my resource.tf file:
resource "aws_route53_record" "instance_record" {
for_each = toset(var.hostnames)
zone_id = data.aws_route53_zone.route53_zone.zone_id
name = "${each.value}.${var.domain}"
type = "A"
ttl = "300"
records = module.ec2_instance.server_ip_addresses
}
resource "aws_route53_record" "webservice_records" {
for_each = toset(var.subdomain_addresses)
zone_id = data.aws_route53_zone.route53_zone.zone_id
name = "${each.value}.${var.domain}"
type = "CNAME"
ttl = "300"
records = [values(aws_route53_record.instance_record)[*].name]
}
What I need is a way to have the webservice_records resource create a subdomain for each entry in var.subdomain_addresses but associate each subdomain with each aws_route53_record.instance_record.name output.
So for example instance_record creates the following A records:
webserver1.domain.com = 10.1.1.18
webserver2.domain.com = 10.1.1.19
and webservice_records creates the following CNAMEs:
www-1.domain.com = webserver1.domain.com, webserver2.domain.com
www-2.domain.com = webserver1.domain.com, webserver2.domain.com
www-3.domain.com = webserver1.domain.com, webserver2.domain.com
I am currently getting the following error:
╷
│ Error: Incorrect attribute value type
│
│ on resource.tf line 62, in resource "aws_route53_record" "webservice_records":
│ 62: records = [values(aws_route53_record.instance_record)[*].name]
│ ├────────────────
│ │ aws_route53_record.instance_record is object with 2 attributes
│
│ Inappropriate value for attribute "records": element 0: string required.
╵
Does someone have a better idea of how this can be achieved?
you can use the setproduct function to achieve that output like below:
so it would be like something like this to create a map:
locals {
hostnames = [
"webserver1",
"webserver2"
]
domain = "domain.com"
subdomain_addresses = [
"www-1",
"www-2",
"www-3"
]
webserver_list = [ for web_list in setproduct(local.hostnames, [local.domain]) : join(".",web_list) ]
cname_map = {for sub_domain in local.subdomain_addresses: sub_domain => local.webserver_list}
}
resource "aws_route53_record" "cnames" {
for_each = local.cname_map
zone_id = data.aws_route53_zone.route53_zone.zone_id
name = each.key
type = "CNAME"
ttl = "300"
records = each.value
}
cname_map looks like this:
cname_map= {
www-1 = [
"webserver1.domain.com",
"webserver2.domain.com",
]
www-2 = [
"webserver1.domain.com",
"webserver2.domain.com",
]
www-3 = [
"webserver1.domain.com",
"webserver2.domain.com",
]
}

Typolink to index Pages on solr

I setup a TYPO3 v.9 website with solr.
And for some reason, the Pages indexer not working. So I used the custom TS configuration bellow, to work around and to force Pages indexing. With this custom TS, pages was indexed, BUT the links to a Page from the "Search results" is not working.
This is my custom TS :
#TS added to force page indexing
plugin.tx_solr.index.queue {
snar_pages = 1
snar_pages {
table = pages
fields {
title = title
content = SOLR_CONTENT
content {
cObject = COA
cObject {
10 = TEXT
10 {
field = bodytext
noTrimWrap = || |
}
}
}
url = CASE
url {
key.field = type
# Internal
1 = TEXT
1 {
if.isTrue.field = internalurl
typolink.parameter.field = internalurl
typolink.useCacheHash = 1
typolink.returnLast = url
}
# External
2 = TEXT
2 {
if.isTrue.field = externalurl
field = externalurl
}
default = TEXT
default {
typolink.parameter = {link}
typolink.additionalParams >
typolink.useCacheHash = 1
typolink.returnLast = url
}
}
}
}
is there an error in my TS that prevents the links to the pages from working ?
hmm i think you missed something to setup too for indexing pages. The DataFrontendHelper
see the docs
edited:
plugin.tx_solr {
index {
queue {
pages {
indexer {
frontendDataHelper {
host = {$plugin.tx_solr.indexer.frontendDataHandler.host}
scheme = {$plugin.tx_solr.indexer.frontendDataHandler.scheme}
}
}
}
}
}
}
Changing my TS as follows, and this fixed my problem :
url = CASE
url {
key.field = doktype
default = TEXT
default {
field = title
typolink.parameter.field = uid
typolink.returnLast = url
}
PS : pages need to be re-indexed for good results.

terraform database creation in athena

I am trying to create a database using terraform and this seems very complicated for a poor query...
Could you help me, please?
I have tried null_resource with local-exec and data "external" Python...
I think I am looking the wrong way
ex which doesn't works in terraform 0.12
resource "null_resource" "create-endpoint" {
provisioner "local-exec" {
query = <<EOF
{
CREATE EXTERNAL TABLE `dashboard_loading_time`(
`timestamp_iso` string,
`app_identification` struct<service:string,app_name:string,app_type:string,stage:string>,
`user` struct<api_gateway_key:struct<id:string,name:string>,mashery_key:struct<id:string,name:string>,employee:struct<id:string,name:string>>,
`action` struct<action_type:string,path:string>,
`result` struct<status:string,http_status:string,response:struct<response:string>>)
PARTITIONED BY (
`year` int)
ROW FORMAT SERDE
'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/dev'
}
EOF
command = "aws athena start-query-execution --query-string "query""
}
}
I would like to find the simplest way to do this using terraform.
If you wanna make it for athena, need to make glue resources.
try below code with terraform.
variable "service_name" {
default = "demo-service"
}
variable "workspace" {
default = "dev"
}
variable "columns" {
default = {
id = "int"
type = "string"
status = "int"
created_at = "timestamp"
}
}
resource "aws_glue_catalog_database" "athena" {
name = "${var.service_name}_db"
}
resource "aws_glue_catalog_table" "athena" {
name = "${var.service_name}_logs"
database_name = "${aws_glue_catalog_database.athena.name}"
table_type = "EXTERNAL_TABLE"
parameters = {
EXTERNAL = "TRUE"
}
storage_descriptor {
location = "s3://${var.service_name}-${var.workspace}-data-pipeline/log/"
input_format = "org.apache.hadoop.mapred.TextInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat"
ser_de_info {
name = "jsonserde"
serialization_library = "org.openx.data.jsonserde.JsonSerDe"
parameters = {
"serialization.format" = "1"
}
}
dynamic "columns" {
for_each = "${var.columns}"
content {
name = "${columns.key}"
type = "${columns.value}"
}
}
}
partition_keys {
name = "year"
type = "string"
}
partition_keys {
name = "month"
type = "string"
}
partition_keys {
name = "day"
type = "string"
}
partition_keys {
name = "hour"
type = "string"
}
}
refer to this repository : aws-serverless-data-pipeline-by-terraform
resource "aws_glue_catalog_table" "aws_glue_catalog_table" {
name = "mytable"
database_name = aws_glue_catalog_database.aws_glue_catalog_database.name
table_type = "EXTERNAL_TABLE"
parameters = {
"classification" = "json"
}
storage_descriptor {
location = "s3://mybucket/myprefix"
input_format = "org.apache.hadoop.mapred.TextInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
ser_de_info {
name = "myserdeinfo"
serialization_library = "org.openx.data.jsonserde.JsonSerDe"
parameters = {
"paths" = "jsonrootname"
}
}
columns {
name = "column1"
type = "array<struct<resourcearn:string,tags:array<struct<key:string,value:string>>>>"
}
}
partition_keys {
name = "part1"
type = "string"
}
partition_keys {
name = "part2"
type = "string"
}
}

TYPO3 Formhandler finisher does not write to database

After the TYPO3 (6.1.7) website of a customer has gone online, the Formhandler forms do not work correctly anymore. They do send an email, but it seems that they do not execute the Finisher_DB for writing into the database anymore.
The TypoScript settings look like this:
plugin.Tx_Formhandler.settings {
debug = 0
# GENERAL CONFIGURATION
name = Default
addErrorAnchors = 1
formValuesPrefix = formhandler
fillValueMarkersBeforeLangMarkers = 1
# ERRORS LAYOUT
singleErrorTemplate {
totalWrap = <div>|</div>
singleWrap = <span class="error">|</span><br />
}
errorListTemplate {
totalWrap = <ul>|</ul>
singleWrap = <li class="error">|</li>
}
validators {
1.class = Tx_Formhandler_Validator_Default
1.config {
fieldConf {
wish.errorCheck.1 = required
alternative.errorCheck.1 = required
firstname.errorCheck.1 = required
surname.errorCheck.1 = required
nationality.errorCheck.1 = required
dateofbirth.errorCheck.1 = required
phone.errorCheck.1 = required
email.errorCheck.1 = required
street.errorCheck.1 = required
zip.errorCheck.1 = required
city.errorCheck.1 = required
country.errorCheck.1 = required
}
}
}
# Finishers configuration
finishers {
1.class = Tx_Formhandler_Finisher_Mail
1.config {
checkBinaryCrLf = registrationMessagePlain, registrationMessageHtml
limitMailsToUser = 10
admin {
}
user {
}
}
2.class = Tx_Formhandler_Finisher_DB
2.config{
table = tx_chilifhregistration
key = uid
fields {
timeslot = Sommerplatz
timeslot_july.mapping = timeslotSummerJuly
timeslot_august.mapping = timeslotSummerAugust
timeslot_september.mapping = timeslotSummerSeptember
wish.mapping = wish
wishcategory11.mapping = wishCategory11
wishcategory19.mapping = wishCategory19
wishcategory22.mapping = wishCategory22
wishcategorydb.mapping = wishCategoryDb
alternative.mapping = alternative
alternativecategory11.mapping = alternativeCategory11
alternativecategory19.mapping = alternativeCategory19
alternativecategory22.mapping = alternativeCategory22
alternativecategorydb.mapping = alternativeCategoryDb
salutation.mapping = salutation
firstname.mapping = firstname
surname.mapping = surname
nationality.mapping = nationality
dateofbirth.mapping = dateofbirth
phone.mapping = phone
email.mapping = email
street.mapping = street
zip.mapping = zip
city.mapping = city
country.mapping = country
salutation2.mapping = salutation2
firstname2.mapping = firstname2
surname2.mapping = surname2
nationality2.mapping = nationality2
dateofbirth2.mapping = dateofbirth2
phone2.mapping = phone2
email2.mapping = email2
street2.mapping = street2
zip2.mapping = zip2
city2.mapping = city2
country2.mapping = country2
}
}
}
}
What could be the problem?
You should better use the predef-definition-style for forms. This will save you trouble with multiple forms and is a cleaner implementation.
plugin.Tx_Formhandler.settings.predef.yourformularname { ...config... }
You can find a bunch of examples on the offical site/examples
I assume that your admin and user mail-config is only empty because you won´t post any customerinformation?
Did the form work if you fill in every single field?
In my own usecases all field i map with the finisher are required, maybe you should set a ...IfEmpty-option for non-require fields.
Here are the available ifEmpty-options.

Resources