How can i establish rpc properties with the datasource type DB in Corda community edition? - database

To establish an RPC connection in the community edition we need to specify the rpc username, password and permissions but when we are integrating external database like MySQL and change the datasource type from INMEMORY to "DB" it does not allows to give user properties.
these are the settings I am using in my node.conf
security = {
authService = {
dataSource = {
type = "DB"
passwordEncryption = "SHIRO_1_CRYPT"
connection = {
jdbcUrl = "jdbc:mysql://localhost:3306"
username = "root"
password = "password"
driverClassName = "com.mysql.jdbc.Driver"
}
}
options = {
cache = {
expireAfterSecs = 120
maxEntries = 10000
}
}
}

Maybe I didn't understand your question, but database setup in node.conf is separate from RPC user setup in node.conf:
Database (PostGres in my case)
extraConfig = [
'dataSourceProperties.dataSourceClassName' : 'org.postgresql.ds.PGSimpleDataSource',
'dataSourceProperties.dataSource.url' : 'jdbc:postgresql://localhost:5432/postgres',
'dataSourceProperties.dataSource.user' : 'db_user',
'dataSourceProperties.dataSource.password' : 'db_user_password',
'database.transactionIsolationLevel' : 'READ_COMMITTED',
'database.initialiseSchema' : 'true'
]
RPC User
rpcUsers = [[ user: "rpc_user", "password": "rpc_user_password", "permissions": ["ALL"]]]
Ok, I'm adding my node's node.config (it's part of Corda TestNet, and it's deployed on Google Cloud):
baseDirectory = "."
compatibilityZoneURL = "https://netmap.testnet.r3.com"
emailAddress = "xxx"
jarDirs = [ "plugins", "cordapps" ]
sshd { port = 2222 }
myLegalName = "OU=xxx, O=TESTNET_xxx, L=London, C=GB"
keyStorePassword = "xxx"
trustStorePassword = "xxx"
crlCheckSoftFail = true
database = {
transactionIsolationLevel = "READ_COMMITTED"
initialiseSchema = "true"
}
dataSourceProperties {
dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource"
dataSource.url = "jdbc:postgresql://xxx:xxx/postgres"
dataSource.user = xxx
dataSource.password = xxx
}
p2pAddress = "xxx:xxx"
rpcSettings {
useSsl = false
standAloneBroker = false
address = "0.0.0.0:xxx"
adminAddress = "0.0.0.0:xxx"
}
rpcUsers = [
{ username=cordazoneservice, password=xxx, permissions=[ ALL ] }
]
devMode = false
cordappSignerKeyFingerprintBlacklist = []
useTestClock = false

Related

Assign random passwords while creating multiple Azure Active Directory users with Terraform reading a CSV file

I am trying to assign random passwords to multiple AAD users -in a csv file- with Terraform and resources "azuread_user"
First of all, I have this CSV file with some users:
user_name
User1
User2
User3
User4
Following, I read this CSV file using:
locals {
users = csvdecode(file("${path.module}/users.csv"))
}
Then, using "random_password" resource, I am generating a new password:
resource "random_password" "password" {
length = 16
special = true
override_special = "!#$%&*()-_=+[]{}<>:?"
}
Next, with "azuread_user" I am trying to create the user with the password generated:
resource "azuread_user" "users" {
for_each = { for user in local.users : user.first_name => user }
user_principal_name = format(
"%s#%s",
each.value.user_name,
"mydomain.com"
)
password = each.value.password
display_name = "${each.value.first_name} ${each.value.last_name}"
}
but the problem is that every user has the same password from resource "random_password" "password".
How can I assign a randomly password for each user?
I tried to create users with random passwords as below:
locals {
users = {
"divv#xxxxxxx.onmicrosoft.com" = { first_name = "John", last_name = "Doe" , department = "Marketing Department" },
"shrav#xxxxxxxxxx.onmicrosoft.com" = { first_name = "Jane", last_name = "Doe" , department = "IT Department"}
}
}
resource "random_password" "passwords" {
for_each = local.users
length = 16
special = true
}
resource "azuread_user" "users" {
for_each = local.users
display_name = "${each.value.first_name} ${each.value.last_name}"
mail_nickname = each.value.first_name
user_principal_name = each.key
password = random_password.passwords[each.key].result
department = each.value.department
}
In order to check if random passwords are generated I stored them in keyvault and checked .
They seem to be different for different user.
resource "azurerm_key_vault" "example" {
name = "kavyaexmplekeyvault"
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
enabled_for_disk_encryption = true
tenant_id = data.azurerm_client_config.current.tenant_id
soft_delete_retention_days = 7
purge_protection_enabled = false
sku_name = "standard"
access_policy {
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
key_permissions = [
"Create",
"Get",
]
secret_permissions = [
"Set",
"Get",
"Delete",
"Purge",
"Recover",
"List"
]
storage_permissions = [
"Get","Set"
]
}
}
resource "azurerm_key_vault_secret" "password_one" {
for_each = local.users
name = "passwrdone${each.value.first_name}"
value = random_password.passwords[each.key].result
key_vault_id = azurerm_key_vault.example.id
}
Password for jane:
Password for john:

How to use terraform to enable Managed private endpoint on datafactory azure sql database linked service

I am trying to use terraform to create adf linked services however the terraform resource doesn't give the option to select an already existing managed private endpoint for the linked service to communicate over but when creating from the portal, this is possible. bellow is my code
resource "azurerm_data_factory" "process-adf" {
resource_group_name = module.resourcegroup.resource_group.name
location = module.resourcegroup.resource_group.location
name = "adf"
managed_virtual_network_enabled = true
public_network_enabled = false
tags = var.tags
identity {
type = "SystemAssigned"
}
}
resource "azurerm_data_factory_linked_service_azure_sql_database" "process-mssql-adf" {
name = "mssql-adf"
data_factory_id = azurerm_data_factory.process-adf.id
integration_runtime_name = azurerm_data_factory_integration_runtime_azure.adf.id
connection_string = "data source=servername;initial catalog=databasename;user id=admin;Password=password;integrated security=True;encrypt=True;connection timeout=30"
}
resource "azurerm_data_factory_managed_private_endpoint" "adf-msssql-pe" {
name = "adf"
data_factory_id = azurerm_data_factory.process-adf.id
target_resource_id = azurerm_mssql_server.process-control.id
subresource_name = "sqlServer"
}
resource "azurerm_data_factory_integration_runtime_azure" "adf" {
name = "adf"
data_factory_id = azurerm_data_factory.process-adf.id
location = module.resourcegroup.resource_group.location
virtual_network_enabled = true
}
how do i point the resource azurerm_data_factory_linked_service_azure_sql_database to the resource azurerm_data_factory_managed_private_endpoint ?

Dynamics 365 Business Central terraform scopes

I am trying to create a terraform script that will register an application in Azure AD.
I have been successful when generating a script that only reads from Microsoft Graph scopes. But I am having trouble figuring out what the equivalent of those scopes are in Business Central (Cloud version).
For Microsoft Graph I have these permissions:
email
offline_access
openid
profile
Financials.ReadWrite.All
User.Read
And I read them like this in terraform:
provider "azuread" {
# Whilst version is optional, we /strongly recommend/ using it to pin the version of the Provider being used
version = "~> 0.10"
subscription_id = var.subscription_id
}
data "azuread_service_principal" "graph-api" {
display_name = "Microsoft Graph"
}
locals {
MAIL_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("email"))[0]}"
USER_READ_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("User.Read"))[0]}"
FINANCIALS_READ_WRITE_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("Financials.ReadWrite.All"))[0]}"
OFFLINE_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("offline_access"))[0]}"
OPENID_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("openid"))[0]}"
PROFILE_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("profile"))[0]}"
}
Which seems to be working fine. I am just struggling to find the similar way of doing this for the Dynamics 365 Business Central
I am interested in these:
app_access
Financials.ReadWrite.All
user_impersonation
Does anybody know what that endpoint might look like? The documentation is very limited.
EDIT:
This is the final script for anybody interested in setting up an Business Central application registration
variable "subscription_id" {
type = string
}
variable "app_name" {
type = string
}
variable "app_homepage" {
type = string
}
variable "app_reply_urls" {
type = list(string)
}
provider "azuread" {
# Whilst version is optional, we /strongly recommend/ using it to pin the version of the Provider being used
version = "~> 0.10"
subscription_id = var.subscription_id
}
data "azuread_service_principal" "graph-api" {
display_name = "Microsoft Graph"
}
data "azuread_service_principal" "d365bc" {
display_name = "Dynamics 365 Business Central"
}
locals {
APP_ACCESS_PERMISSION = "${matchkeys(data.azuread_service_principal.d365bc.app_roles.*.id, data.azuread_service_principal.d365bc.app_roles.*.value, list("app_access"))[0]}"
USER_IMPERSONATION_PERMISSION = "${matchkeys(data.azuread_service_principal.d365bc.oauth2_permissions.*.id, data.azuread_service_principal.d365bc.oauth2_permissions.*.value, list("user_impersonation"))[0]}"
BC_FINANCIALS_READ_WRITE_PERMISSION = "${matchkeys(data.azuread_service_principal.d365bc.oauth2_permissions.*.id, data.azuread_service_principal.d365bc.oauth2_permissions.*.value, list("Financials.ReadWrite.All"))[0]}"
GRAPH_FINANCIAL_READ_WRITE_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("Financials.ReadWrite.All"))[0]}"
MAIL_READ_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("User.Read"))[0]}"
MAIL_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("email"))[0]}"
OFFLINE_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("offline_access"))[0]}"
OPENID_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("openid"))[0]}"
PROFILE_PERMISSION = "${matchkeys(data.azuread_service_principal.graph-api.oauth2_permissions.*.id, data.azuread_service_principal.graph-api.oauth2_permissions.*.value, list("profile"))[0]}"
}
resource "azuread_application" "businessCentral" {
name = var.app_name
homepage = var.app_homepage
identifier_uris = []
reply_urls = var.app_reply_urls
available_to_other_tenants = true
type = "webapp/api"
required_resource_access {
resource_app_id = data.azuread_service_principal.graph-api.application_id
resource_access {
id = local.GRAPH_FINANCIAL_READ_WRITE_PERMISSION
type = "Scope"
}
resource_access {
id = local.MAIL_PERMISSION
type = "Scope"
}
resource_access {
id = local.MAIL_READ_PERMISSION
type = "Scope"
}
resource_access {
id = local.OFFLINE_PERMISSION
type = "Scope"
}
resource_access {
id = local.OPENID_PERMISSION
type = "Scope"
}
resource_access {
id = local.PROFILE_PERMISSION
type = "Scope"
}
}
required_resource_access {
resource_app_id = data.azuread_service_principal.d365bc.application_id
resource_access {
id = local.APP_ACCESS_PERMISSION
type = "Role"
}
resource_access {
id = local.USER_IMPERSONATION_PERMISSION
type = "Scope"
}
resource_access {
id = local.BC_FINANCIALS_READ_WRITE_PERMISSION
type = "Scope"
}
}
app_role {
allowed_member_types = [
"Application"
]
description = "Admins can manage roles and perform all task actions"
display_name = "Admin"
is_enabled = true
value = "Admin"
}
}
One thing to note is that the app_access is Role and the rest of the API permissions are Scope.
You can call the above terraform with:
terraform plan -var="subscription_id={your_scription_id}" -var='app_reply_urls={your_urls_array}' -var="app_name={your_app_name}" -var="app_homepage={your_app_homepage}"
Try this:
provider "azuread" {
# Whilst version is optional, we /strongly recommend/ using it to pin the version of the Provider being used
version = "=0.10.0"
}
data "azuread_service_principal" "d365bc" {
application_id = "996def3d-b36c-4153-8607-a6fd3c01b89f"
}
locals {
APP_ACCESS_PERMISSION = "${matchkeys(data.azuread_service_principal.d365bc.app_roles.*.id, data.azuread_service_principal.d365bc.app_roles.*.value, list("app_access"))[0]}"
USER_IMPERSONATION_PERMISSION = "${matchkeys(data.azuread_service_principal.d365bc.oauth2_permissions.*.id, data.azuread_service_principal.d365bc.oauth2_permissions.*.value, list("user_impersonation"))[0]}"
FINANCIALS_READ_WRITE_PERMISSION = "${matchkeys(data.azuread_service_principal.d365bc.oauth2_permissions.*.id, data.azuread_service_principal.d365bc.oauth2_permissions.*.value, list("Financials.ReadWrite.All"))[0]}"
}
996def3d-b36c-4153-8607-a6fd3c01b89f is the client id of Microsoft Dynamics 365 BC service principal.
app_access is app permission so we need to use "app_roles" rather than "oauth2_permissions" here.

Create database schema with terraform

I created RDS instance using aws_db_instance (main.tf):
resource "aws_db_instance" "default" {
identifier = "${module.config.database["db_inst_name"]}"
allocated_storage = 20
storage_type = "gp2"
engine = "mysql"
engine_version = "5.7"
instance_class = "db.t3.micro"
name = "${module.config.database["db_name_prefix"]}${terraform.workspace}"
username = "${module.config.database["db_username"]}"
password = "${module.config.database["db_password"]}"
parameter_group_name = "default.mysql5.7"
skip_final_snapshot = true
}
Can I also create database schemas from file schema.sql with terraform apply?
$ tree -L 1
.
├── main.tf
└── schema.sql
You can use a provisioner (https://www.terraform.io/docs/provisioners/index.html) for that:
resource "aws_db_instance" "default" {
identifier = module.config.database["db_inst_name"]
allocated_storage = 20
storage_type = "gp2"
engine = "mysql"
engine_version = "5.7"
instance_class = "db.t3.micro"
name = "${module.config.database["db_name_prefix"]}${terraform.workspace}"
username = module.config.database["db_username"]
password = module.config.database["db_password"]
parameter_group_name = "default.mysql5.7"
skip_final_snapshot = true
provisioner "local-exec" {
command = "mysql --host=${self.address} --port=${self.port} --user=${self.username} --password=${self.password} < ./schema.sql"
}
}
#Apply scheme by using bastion host
resource "aws_db_instance" "default_bastion" {
identifier = module.config.database["db_inst_name"]
allocated_storage = 20
storage_type = "gp2"
engine = "mysql"
engine_version = "5.7"
instance_class = "db.t3.micro"
name = "${module.config.database["db_name_prefix"]}${terraform.workspace}"
username = module.config.database["db_username"]
password = module.config.database["db_password"]
parameter_group_name = "default.mysql5.7"
skip_final_snapshot = true
provisioner "file" {
connection {
user = "ec2-user"
host = "bastion.example.com"
private_key = file("~/.ssh/ec2_cert.pem")
}
source = "./schema.sql"
destination = "~"
}
provisioner "remote-exec" {
connection {
user = "ec2-user"
host = "bastion.example.com"
private_key = file("~/.ssh/ec2_cert.pem")
}
command = "mysql --host=${self.address} --port=${self.port} --user=${self.username} --password=${self.password} < ~/schema.sql"
}
}
mysql client needs to be installed on your device.
If you don't have direct access to your DB, there is also a remote-exec provisioner, where you can use a bastion host (transfer file to remote place with file provisioner first).
If your schema is not to complex, you could also use the MySQL provider of terraform:
https://www.terraform.io/docs/providers/mysql/index.html

terraform database creation in athena

I am trying to create a database using terraform and this seems very complicated for a poor query...
Could you help me, please?
I have tried null_resource with local-exec and data "external" Python...
I think I am looking the wrong way
ex which doesn't works in terraform 0.12
resource "null_resource" "create-endpoint" {
provisioner "local-exec" {
query = <<EOF
{
CREATE EXTERNAL TABLE `dashboard_loading_time`(
`timestamp_iso` string,
`app_identification` struct<service:string,app_name:string,app_type:string,stage:string>,
`user` struct<api_gateway_key:struct<id:string,name:string>,mashery_key:struct<id:string,name:string>,employee:struct<id:string,name:string>>,
`action` struct<action_type:string,path:string>,
`result` struct<status:string,http_status:string,response:struct<response:string>>)
PARTITIONED BY (
`year` int)
ROW FORMAT SERDE
'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/dev'
}
EOF
command = "aws athena start-query-execution --query-string "query""
}
}
I would like to find the simplest way to do this using terraform.
If you wanna make it for athena, need to make glue resources.
try below code with terraform.
variable "service_name" {
default = "demo-service"
}
variable "workspace" {
default = "dev"
}
variable "columns" {
default = {
id = "int"
type = "string"
status = "int"
created_at = "timestamp"
}
}
resource "aws_glue_catalog_database" "athena" {
name = "${var.service_name}_db"
}
resource "aws_glue_catalog_table" "athena" {
name = "${var.service_name}_logs"
database_name = "${aws_glue_catalog_database.athena.name}"
table_type = "EXTERNAL_TABLE"
parameters = {
EXTERNAL = "TRUE"
}
storage_descriptor {
location = "s3://${var.service_name}-${var.workspace}-data-pipeline/log/"
input_format = "org.apache.hadoop.mapred.TextInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat"
ser_de_info {
name = "jsonserde"
serialization_library = "org.openx.data.jsonserde.JsonSerDe"
parameters = {
"serialization.format" = "1"
}
}
dynamic "columns" {
for_each = "${var.columns}"
content {
name = "${columns.key}"
type = "${columns.value}"
}
}
}
partition_keys {
name = "year"
type = "string"
}
partition_keys {
name = "month"
type = "string"
}
partition_keys {
name = "day"
type = "string"
}
partition_keys {
name = "hour"
type = "string"
}
}
refer to this repository : aws-serverless-data-pipeline-by-terraform
resource "aws_glue_catalog_table" "aws_glue_catalog_table" {
name = "mytable"
database_name = aws_glue_catalog_database.aws_glue_catalog_database.name
table_type = "EXTERNAL_TABLE"
parameters = {
"classification" = "json"
}
storage_descriptor {
location = "s3://mybucket/myprefix"
input_format = "org.apache.hadoop.mapred.TextInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
ser_de_info {
name = "myserdeinfo"
serialization_library = "org.openx.data.jsonserde.JsonSerDe"
parameters = {
"paths" = "jsonrootname"
}
}
columns {
name = "column1"
type = "array<struct<resourcearn:string,tags:array<struct<key:string,value:string>>>>"
}
}
partition_keys {
name = "part1"
type = "string"
}
partition_keys {
name = "part2"
type = "string"
}
}

Resources