I have defined the variable in dbt_project.yml file and want to override the same when running the dbt model..But its not overriding and take the variable from yml file
getting this from log :
02:04:13 Database Error in model AUD_RECON_INPUT (models\AUD_RECON_INPUT.sql)
02:04:13 002003 (42S02): SQL compilation error:
02:04:13 Object 'ETS_REVAP02_DB_DEV.REVAP_AUDIT.ABC' does not exist or not authorized.
02:04:13
02:04:13 Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
Command used :
dbt run --select AUD_RECON_INPUT --vars '{"RECON_INPUT":"XYZ"}'
Related
I am building my Release Pipeline which contains a stage to deploy the migrations to my SQL Server Database on my own Server. The build pipeline produces the .sql script and publishes it into the migrations folder
Inside the release pipeline, I added a Task named SQL Server database deploy and set the path to the .sql file
Now if I run that pipeline, it runs into an error
[error]No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
So there must be a wrong path provided, but I can't find out, which might be the correct one.
Here is the debug output
2020-06-14T21:00:46.9140268Z ##[debug]Evaluating condition for step: 'Deploy using : sqlQuery'
2020-06-14T21:00:46.9142671Z ##[debug]Evaluating: succeeded()
2020-06-14T21:00:46.9143229Z ##[debug]Evaluating succeeded:
2020-06-14T21:00:46.9144608Z ##[debug]=> True
2020-06-14T21:00:46.9145556Z ##[debug]Result: True
2020-06-14T21:00:46.9146106Z ##[section]Starting: Deploy using : sqlQuery
2020-06-14T21:00:46.9277557Z ==============================================================================
2020-06-14T21:00:46.9277903Z Task : SQL Server database deploy
2020-06-14T21:00:46.9278185Z Description : Deploy a SQL Server database using DACPAC or SQL scripts
2020-06-14T21:00:46.9278427Z Version : 0.3.23
2020-06-14T21:00:46.9278632Z Author : Microsoft Corporation
2020-06-14T21:00:46.9278995Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/deploy/sql-dacpac-deployment-on-machine-group
2020-06-14T21:00:46.9279376Z ==============================================================================
2020-06-14T21:00:48.0530812Z ##[debug]VstsTaskSdk 0.8.2 commit
2020-06-14T21:00:48.2649638Z ##[debug]Entering D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\Main.ps1.
2020-06-14T21:00:48.2752917Z ##[debug]Loading resource strings from: D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\Task.json
2020-06-14T21:00:48.3003693Z ##[debug]Loaded 6 strings.
2020-06-14T21:00:48.3288927Z ##[debug]SYSTEM_CULTURE: 'en-US'
2020-06-14T21:00:48.3289509Z ##[debug]Loading resource strings from: D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\Strings\resources.resjson\en-US\resources.resjson
2020-06-14T21:00:48.3331827Z ##[debug]Loaded 6 strings.
2020-06-14T21:00:48.3607995Z ##[debug]INPUT_TASKTYPE: 'sqlQuery'
2020-06-14T21:00:48.3663637Z ##[debug]INPUT_DACPACFILE: 'D:\a\r1\a'
2020-06-14T21:00:48.3665646Z ##[debug]INPUT_SQLFILE: 'D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql'
2020-06-14T21:00:48.3680833Z ##[debug]INPUT_EXECUTEINTRANSACTION: 'false'
2020-06-14T21:00:48.3721277Z ##[debug] Converted to bool: False
2020-06-14T21:00:48.3755339Z ##[debug]INPUT_EXCLUSIVELOCK: 'false'
2020-06-14T21:00:48.3778148Z ##[debug] Converted to bool: False
2020-06-14T21:00:48.3809928Z ##[debug]INPUT_APPLOCKNAME (empty)
2020-06-14T21:00:48.3882461Z ##[debug]INPUT_INLINESQL (empty)
2020-06-14T21:00:48.3919370Z ##[debug]INPUT_TARGETMETHOD: 'server'
2020-06-14T21:00:48.3954451Z ##[debug]INPUT_SERVERNAME: '88.198.44.249'
2020-06-14T21:00:48.3990602Z ##[debug]INPUT_DATABASENAME: 'Stage.TimeService'
2020-06-14T21:00:48.4026165Z ##[debug]INPUT_AUTHSCHEME: 'sqlServerAuthentication'
2020-06-14T21:00:48.4050818Z ##[debug]INPUT_SQLUSERNAME: 'Zoo-Director'
2020-06-14T21:00:48.4083226Z ##[debug]INPUT_SQLPASSWORD: 'Daniel#2019!newS'
2020-06-14T21:00:48.4112120Z ##[debug]INPUT_CONNECTIONSTRING (empty)
2020-06-14T21:00:48.4138318Z ##[debug]INPUT_PUBLISHPROFILE (empty)
2020-06-14T21:00:48.4172155Z ##[debug]INPUT_ADDITIONALARGUMENTS (empty)
2020-06-14T21:00:48.4205938Z ##[debug]INPUT_ADDITIONALARGUMENTSSQL (empty)
2020-06-14T21:00:48.4240088Z ##[debug]Loading module from path 'D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\ps_modules\TaskModuleSqlUtility\TaskModuleSqlUtility.psd1'.
2020-06-14T21:00:48.4269587Z ##[debug]Loading module from path 'D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\ps_modules\TaskModuleSqlUtility\TaskModuleSqlUtility.psm1'.
2020-06-14T21:00:48.4423108Z ##[debug]NonInteractive: False
2020-06-14T21:00:48.6017509Z ##[debug]Exporting function 'Invoke-DacpacDeployment'.
2020-06-14T21:00:48.6044940Z ##[debug]Exporting function 'Invoke-SqlQueryDeployment'.
2020-06-14T21:00:48.6091408Z ##[debug]Adding exceptions types.
2020-06-14T21:00:50.8081584Z ##[debug]Importing function 'Invoke-DacpacDeployment'.
2020-06-14T21:00:50.8099297Z ##[debug]Importing function 'Invoke-SqlQueryDeployment'.
2020-06-14T21:00:50.8847884Z ##[debug]Processed: ##vso[telemetry.publish area=SqlDacpacDeploymentOnMachineGroup;feature=SqlDacpacDeploymentOnMachineGroup]{"serverName": "56531c73afc443fce9712faf6ea6473e2b28f77562757d20dc806bf6549c3b0e","databaseName": "a44f4ee78fc10bbef97ea36bc9819a0a47d0e11f6a0aa5cedd395548086fa4e5"}
2020-06-14T21:00:50.9277590Z ##[debug]Finding files with pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
2020-06-14T21:00:50.9331163Z ##[debug]Entering Find-VstsFiles.
2020-06-14T21:00:50.9388734Z ##[debug] LegacyPattern: 'D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql'
2020-06-14T21:00:50.9831065Z ##[debug]Entering Get-MatchingItems.
2020-06-14T21:00:50.9873935Z ##[debug] IncludePatterns: 'D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql'
2020-06-14T21:00:50.9931616Z ##[debug] ExcludePatterns: ''
2020-06-14T21:00:50.9981552Z ##[debug] IncludeFiles: 'True'
2020-06-14T21:00:51.0003180Z ##[debug] IncludeDirectories: 'False'
2020-06-14T21:00:51.0031742Z ##[debug] Force: 'False'
2020-06-14T21:00:51.2179973Z ##[debug]No paths.
2020-06-14T21:00:51.2213869Z ##[debug]Leaving Get-MatchingItems.
2020-06-14T21:00:51.2233073Z ##[debug]Total found: 0
2020-06-14T21:00:51.2257660Z ##[debug]Leaving Find-VstsFiles.
2020-06-14T21:00:51.2275406Z ##[debug]Matched files =
2020-06-14T21:00:51.3304392Z ##[debug]Write-Exception error:
2020-06-14T21:00:51.3348086Z ##[debug]No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
2020-06-14T21:00:51.3538650Z ##[debug]Caught exception from task script.
2020-06-14T21:00:51.3575755Z ##[debug]Error record:
2020-06-14T21:00:51.4275609Z ##[debug]No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
2020-06-14T21:00:51.4293155Z ##[debug]At D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\Main.ps1:51 char:13
2020-06-14T21:00:51.4294350Z ##[debug]+ throw (Get-VstsLocString -Key "Nofileswerefoundtodeploywi ...
2020-06-14T21:00:51.4295348Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-06-14T21:00:51.4307202Z ##[debug] + CategoryInfo : OperationStopped: (No files were f...DataContext.sql:String) [], RuntimeException
2020-06-14T21:00:51.4319906Z ##[debug] + FullyQualifiedErrorId : No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceD ataContext.sql
2020-06-14T21:00:51.4330337Z ##[debug]
2020-06-14T21:00:51.4356258Z ##[debug]Script stack trace:
2020-06-14T21:00:51.4395383Z ##[debug]at Get-SingleFile, D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\Main.ps1: line 51
2020-06-14T21:00:51.4407230Z ##[debug]at <ScriptBlock>, D:\a\_tasks\SqlDacpacDeploymentOnMachineGroup_4b506f7f-720f-47bb-bf21-029bac6a690d\0.3.23\Main.ps1: line 175
2020-06-14T21:00:51.4422508Z ##[debug]at <ScriptBlock>, <No file>: line 1
2020-06-14T21:00:51.4440151Z ##[debug]at <ScriptBlock>, <No file>: line 22
2020-06-14T21:00:51.4454203Z ##[debug]at <ScriptBlock>, <No file>: line 18
2020-06-14T21:00:51.4478858Z ##[debug]at <ScriptBlock>, <No file>: line 1
2020-06-14T21:00:51.4497567Z ##[debug]Exception:
2020-06-14T21:00:51.4549822Z ##[debug]System.Management.Automation.RuntimeException: No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
2020-06-14T21:00:51.4853912Z ##[error]No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
2020-06-14T21:00:51.4865099Z ##[debug]Processed: ##vso[task.logissue type=error]No files were found to deploy with search pattern D:\a\r1\a/drop/migrations/TimeServiceDataContext.sql
2020-06-14T21:00:51.4867288Z ##[debug]Processed: ##vso[task.complete result=Failed]
2020-06-14T21:00:51.5269963Z ##[section]Finishing: Deploy using : sqlQuery
OP updated:
Running the tasks on his self hosted machine finally solved the issue.
(Required) Location of the SQL file on the target. Provide semi-colon
separated list of SQL script files to execute multiple files. The SQL
scripts will be executed in the order given. Location can also be a
UNC path like, \BudgetIT\Web\Deploy\FabrikamDB.sql. The UNC path
should be accessible to the machine's administrator account.
Environment variables are also supported, such as $env:windir,
$env:systemroot, $env:windir\FabrikamFibre\DB. Wildcards can be used.
For example, /*.sql for sql file present in all sub folders
Source Link
You should use "\". Normally, the path will be something like below:
$(System.DefaultWorkingDirectory)\[Artifact name]\migrations\TimeServiceDataContext.sql
If you are using default drop name it will directly be
D:\a\r1\a\migrations\TimeServiceDataContext.sql
Besides, seems you are using self-host agent, you could also directly remote to the machine which host build agent and check the valid folder path.
I am running the Rundeck war file directly
java -jar rundeck-3.0.17-20190311.war
I get this error message when I trigger a build.
Failed to read SSH Private key stored at path:
keys/rundeck.pem: org.rundeck.storage.api.StorageException:
Path does not exist: keys/rundeck.pem
Failed: ConfigurationFailure: Failed to read SSH
Private key stored at path: keys/rundeck.pem
It makes sense that the reference in the Default Node Executor is invalid and that Rundeck cannot find the .pem file.
I've tried
referencing the full working directory (/home/user/rundeck/keys/rundeck.pem) It wants the location to start with keys/.
referencing it to its relative path (keys/rundeck.pem)
copied the keys directory to /home/user/
In desperation, I ran chmod 700 on the pem file.
Most of the questions and examples I found were on older versions of Rundeck.
I'd like to know where the .pem file must be configured and how it should be referenced. Any other information that could help me configure the SSH keys will be appreciated.
You must add the key using the GUI and use the path that you are defined in your resources.xml.
For add your key, you can follow this. Although the video is based on Rundeck 2.x it is valid for Rundeck 3.x:
Check that https://www.youtube.com/watch?v=qOA-kWse22g
And for generate your resources.xml file select your new project and go to Project Settings > Edit Nodes > Click on "Configure Nodes" button (up to right) > Click on "Add Sources +" Button > Select "+ File" option > in "Format" field select "resourcexml" and fill the path in "File Path" field (put the file name at the end, usually "resources.xml"), then select "Generate", "Include Server Node" and "Writeable" checkboxes and click on "Save" button.
I am using Cloudera cluster and it is kerberosed with tsl and ssl enabled. I am trying to use create an instance using the --jaas option in solrctl command but it is not working.
The solrctl command I am using is below
solrctl --jaas jaas-client.conf instancedir --create testindex3 /home/myuserid/testindex3
The jass-client.conf file is below
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="<PATH TO KEYTAB>/username.keytab"
storeKey=true
useTicketCache=false
debug=true
principal="fully.qualified.domain.name#<YOUR-REALM";
}
For the above command I am getting the error
Uploading configs from /user/myuserid/testindex3/conf to ZookeeperHost1:2181,ZookeeperHost2:2181,ZookeeperHost3:2181/solr. This may take up to a minute.
Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is /username.keytab refreshKrb5Config is false principal is fully.qualified.domain.name#
Error: can't upload configuration
I am not sure why the instance is not created and what i am missing. Is there anything wrong in my jaas.conf file. Please advise
Note: I haven't used "user/fully.qualified.domain.name#
I want to prepare my application to be compatible with many databases types. To try it i've used H2, MySql and Postgresql. So 'ive added into build.sbt :
"mysql" % "mysql-connector-java" % "5.1.35",
"org.postgresql" % "postgresql" % "9.4-1201-jdbc41"
and i've added conf/prod.conf with all configuration without database configuration, and 3 files:
conf/h2.conf
include "prod.conf"
db.h2.driver=org.h2.Driver
db.h2.url="jdbc:h2:mem:dontforget"
db.h2.jndiName=DefaultDS
ebean.h2="fr.chklang.dontforget.business.*"
conf/mysql.conf
include "prod.conf"
db.mysql.driver=com.mysql.jdbc.Driver
db.mysql.jndiName=DefaultDS
ebean.mysql="fr.chklang.dontforget.business.*"
conf/postgresql.conf
include "prod.conf"
db.postgresql.driver=org.postgresql.Driver
db.postgresql.jndiName=DefaultDS
ebean.postgresql="fr.chklang.dontforget.business.*"
Add to it i've three folders into conf/evolutions with
evolutions/h2
evolutions/mysql
evolutions/postgresql
with these things user can start my application with this command :
-Dconfig.file=dontforget-conf.conf -DapplyEvolutions.default=true -Dhttp.port=10180 &
And this conf file is
include "postgresql.conf"
db.postgresql.url="jdbc:postgresql:dontforget"
db.postgresql.user=myUserName
db.postgresql.password=myPassword
But with this configuration, when my application try to connect to DB :
The default EbeanServer has not been defined? This is normally set via the ebean.datasource.default property. Otherwise it should be registered programatically via registerServer()]]
So i've tried to add, into my configuration :
ebean.datasource.default=postgresql
but when i add it i've :
Configuration error: Configuration error[Configuration error[]]
at play.api.Configuration$.play$api$Configuration$$configError(Configuration.scala:94)
at play.api.Configuration.reportError(Configuration.scala:743)
at play.Configuration.reportError(Configuration.java:310)
at play.db.ebean.EbeanPlugin.onStart(EbeanPlugin.java:56)
at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91)
at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91)
at scala.collection.immutable.List.foreach(List.scala:383)
at play.api.Play$$anonfun$start$1.apply$mcV$sp(Play.scala:91)
at play.api.Play$$anonfun$start$1.apply(Play.scala:91)
at play.api.Play$$anonfun$start$1.apply(Play.scala:91)
at play.utils.Threads$.withContextClassLoader(Threads.scala:21)
at play.api.Play$.start(Play.scala:90)
at play.core.StaticApplication.<init>(ApplicationProvider.scala:55)
at play.core.server.NettyServer$.createServer(NettyServer.scala:253)
at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:289)
at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:284)
at scala.Option.map(Option.scala:145)
at play.core.server.NettyServer$.main(NettyServer.scala:284)
at play.core.server.NettyServer.main(NettyServer.scala)
Caused by: Configuration error: Configuration error[]
at play.api.Configuration$.play$api$Configuration$$configError(Configuration.scala:94)
at play.api.Configuration.reportError(Configuration.scala:743)
at play.api.db.BoneCPApi.play$api$db$BoneCPApi$$error(DB.scala:271)
at play.api.db.BoneCPApi$$anonfun$getDataSource$3.apply(DB.scala:438)
at play.api.db.BoneCPApi$$anonfun$getDataSource$3.apply(DB.scala:438)
at scala.Option.getOrElse(Option.scala:120)
at play.api.db.BoneCPApi.getDataSource(DB.scala:438)
at play.api.db.DB$$anonfun$getDataSource$1.apply(DB.scala:142)
at play.api.db.DB$$anonfun$getDataSource$1.apply(DB.scala:142)
at scala.Option.map(Option.scala:145)
at play.api.db.DB$.getDataSource(DB.scala:142)
at play.api.db.DB.getDataSource(DB.scala)
at play.db.DB.getDataSource(DB.java:25)
at play.db.ebean.EbeanPlugin.onStart(EbeanPlugin.java:54)
So i don't understand how i can do it.
YES!!! I've found it! After debug mode (etc...)
There was 2 problems.
First problem : I must add a key into my application.conf :
ebeanconfig.datasource
For me (for exemple), postgresql.conf is modified to :
db.postgresql.driver=org.postgresql.Driver
db.postgresql.jndiName=DefaultDS
ebean.postgresql="fr.chklang.dontforget.business.*"
ebeanconfig.datasource.default=postgresql
Second problem : include into play 2.3.x don't works because conf folder isn't added into classpath (ref Load file from '/conf' directory on Cloudbees ) so we must concat prod.conf, postgresql.conf and dontforget.conf into an only single file.
I hope i have helped any other developper...
I am unable to connect to a project on Google Cloud using the cloud SDK. I want to upload file on my instance using SDK. The error I am getting is:
You are now logged in as [ishanatech#gmail.com].
Your current project is [Trial-2015-1]. You can change this setting by running:
$ gcloud config set project PROJECT
C:\Program Files\Google\Cloud SDK>gcloud compute instances list
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.list) Some requests did not succeed:
- Invalid value 'Trial-2015-1'. Values must match the following regular expression: '(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](
?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?))'
C:\Program Files\Google\Cloud SDK>gcloud compute instances create instance-2 --image centos-6 --zone uscentral1-a
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Failed to find image for alias [centos-6] in public image project [centos-cloud
]. Try specifying a different image using [--image].
- Invalid value 'Trial-2015-1'. Values must match the following regular expression: '(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](
?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?))'
C:\Program Files\Google\Cloud SDK>gcloud compute instances create instance-2 --image opensuse-13 --zone uscentral1-a
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Failed to find image for alias [opensuse-13] in public image project [opensuse-
cloud]. Try specifying a different image using [--image].
- Invalid value 'Trial-2015-1'. Values must match the following regular expression: '(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](
?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?))'
If you look carefully at the complex RE the error message displays:
'(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](
?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:a-z?))'
you'll spot that all the letters there are lowercase -- always carefully expressed as a-z.
Trial-2015-1 starts with an uppercase letter, thus it's not deemed a valid project name.
Are you sure that your project isn't actually named trial-2015-1, with a leading lowercase initial...?
i am getting the below error while creating Bastion sever using terraform script
Error creating instance: googleapi: Error 400: Invalid value for field 'resource.name': 'Bastion-server'. Must be a match of regex '(?:a-z?)', invalid