In the Abp Framework (ABP.IO, Version: 5.2.1)
I need a way to cancel/stop a very long-running background workers/jobs when the application is stopped.
Currently, background workers/jobs continue running and prevent the application from stopping.
example log output:
10 May 2022 13:52:29.311 Parsing ended for a cdr file: 'xxx' in '122' seconds
10 May 2022 13:52:19.874 Parsing ended for a cdr file: 'xxx' in '113' seconds
10 May 2022 13:52:18.093 Parsing file 'xxx'
10 May 2022 13:52:17.969 Parsing started for a cdr file: 'xxx'
10 May 2022 13:52:07.572 Parsing ended for a cdr file: 'xxx' in '100' seconds
10 May 2022 13:52:06.959 Application is shutting down...
10 May 2022 13:51:36.174 Parsing file 'xxx'
10 May 2022 13:51:36.072 Parsing started for a cdr file: 'xxx'
10 May 2022 13:51:29.453 Parsing ended for a cdr file: 'xxx' in '62' seconds
10 May 2022 13:50:56.258 Parsing file 'xxx'
10 May 2022 13:50:56.223 Parsing started for a cdr file: 'xxx'
10 May 2022 13:50:52.202 Parsing ended for a cdr file: 'xxx' in '25' seconds
I have tried injecting an ICancellationTokenProvider, but it didn't help.
await GetFilesList().ParallelForEachAsync(async file =>
{
// Do some parsing
}, _cancellationTokenProvider.Token);
I am using custom SQL in SQL override section in Advance property of snowflake source in IICS Informatica cloud.
select (max(LAST_UPDATED_DATE)) from $$DB.test.org. $$DB parameter is defined in the parameter file $$DB=Dev. Parameter is resolving correctly at run time but getting below error.
READER_1_2_1> SNOWFLAKECLOUDDATAWAREHOUSE_10000 [2020-10-30 03:07:00.440] [INFO] The Snowflake Connector uses the following SQL query to read data: select (max(LAST_UPDATED_DATE)) from Dev.test.org
READER_1_2_1> CMN_1761 [2020-10-30 03:07:00.440] Timestamp Event: [Fri Oct 30 03:07:00 2020]
READER_1_2_1> SNOWFLAKECLOUDDATAWAREHOUSE_10000 [2020-10-30 03:07:00.441] [ERROR] The Snowflake Connector failed to read data from the source because of the following error: [SnowFlake_READER__FAILED_906] The Snowflake Connector failed to read data from the source because of the following error: null
READER_1_2_1> CMN_1761 [2020-10-30 03:07:00.441] Timestamp Event: [Fri Oct 30 03:07:00 2020]
READER_1_2_1> SNOWFLAKECLOUDDATAWAREHOUSE_10000 [2020-10-30 03:07:00.441] [ERROR] Error occured while reading data using CCI Data Adapter com.informatica.cci.runtime.internal.utils.impl.CExceptionImpl: Unable to get result data: Adapter read method has returned failure.
READER_1_2_1> CMN_1761 [2020-10-30 03:07:00.441] Timestamp Event: [Fri Oct 30 03:07:00 2020]
READER_1_2_1> CONNECTOR_10000 [2020-10-30 03:07:00.441] [ERROR] Error Occured while reading data from the source for Transformation - SRC_Organization | Connector Name - SnowflakeCloudDataWarehouse - Unable to get result data: Adapter read method has returned failure.
Thanks in Advance.
The issue was with the column list. Custom query should be like select (max(LAST_UPDATED_DATE)) as LAST_UPDATED_DATE from $$DB.test.org and the Field list should have only one column LAST_UPDATED_DATE. Mapping is working fine now.
I am a first-time user of Scriptella, and am really struggling with getting started.
I seem to be running into the same issue on both SQL Server 2012 as well as MySQL. With both, I am able to CREATE tables and INSERT records, but get the following errors when using SELECT statements.
SQL Server using driver mssql-jdbc-8.4.0.jre8.jar (and earlier versions):
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
MYSQL using driver mysql-connector-java-5.1.45.jar:
Driver exception: java.sql.SQLException: Can not issue SELECT via executeUpdate() or executeLargeUpdate().
I am unable to figure out if this is an issue with my script or if it is a server configuration issue. I tried to set nocount on and nocount off on sql server, but to no avail.
I tried Scriptella version 1.1 as well as version 1.2, and have the same issue.
I am so stuck on this problem that after years of following stackoverflow.com, I am finally posting my first question here. I would so very much appreciate any help.
Please see below for the script I am using as well as the stacktrace. Thanks.
<!DOCTYPE etl SYSTEM "http://scriptella.org/dtd/etl.dtd">
<etl>
<properties>
<include href="etl.properties"/> <!--Load from external properties file-->
</properties>
<!-- Connection declarations -->
<connection driver="$driver" url="$url" user="$user" password="$password" classpath="$classpath"/>
<!--ETL Queries and Scripts -->
<script>
<!-- CREATE TABLE WORKED!
create table TestDB.dbo.test (
id int
); -->
<!-- INSERT WORKED!
insert into TestDB.dbo.test (id) values (1);
-->
<!-- BELOW FAILS -->
select * from TestDB.dbo.GLAccounts
<script>
insert into TestDB.dbo.test (id) values (?1);
</script>
</script>
</etl>
etl-scriptsS java -jar ~/Downloads/scriptella/scriptella-1.1/scriptella.jar my-first-etl.xml
Aug 12, 2020 8:11:33 AM <WARNING> XML configuration warning in file:/home/user1/workspace/etl-scripts/my-first-etl.xml(28:14): The content of element type "script" must match "(include|dialect|onerror)".
Aug 12, 2020 8:11:33 AM <INFO> Execution Progress.Initializing properties: 1%
Aug 12, 2020 8:11:36 AM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.
Aug 12, 2020 8:11:36 AM <INFO> Execution Progress.Initialized connection JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}, Dialect{Microsoft SQL Server 11.00.2218}, properties {}: 5%
Aug 12, 2020 8:11:36 AM <INFO> Execution Progress./etl/script[1] prepared: 10%
Aug 12, 2020 8:11:36 AM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/home/user1/workspace/etl-scripts/my-first-etl.xml"
Aug 12, 2020 8:11:36 AM <SEVERE> Script /home/user1/workspace/etl-scripts/my-first-etl.xml execution failed.
Location: /etl/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
select * from TestDB.dbo.GLAccounts
Error codes: [null, 0]
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
etl-scriptsS java -jar ~/Downloads/scriptella/scriptella-1.1/scriptella.jar my-first-etl.xml
Aug 12, 2020 8:20:19 AM <WARNING> XML configuration warning in file:/home/user1/workspace/etl-scripts/my-first-etl.xml(28:14): The content of element type "script" must match "(include|dialect|onerror)".
Aug 12, 2020 8:20:19 AM <INFO> Execution Progress.Initializing properties: 1%
Aug 12, 2020 8:20:21 AM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.
Aug 12, 2020 8:20:22 AM <INFO> Execution Progress.Initialized connection JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}, Dialect{Microsoft SQL Server 11.00.2218}, properties {}: 5%
Aug 12, 2020 8:20:22 AM <INFO> Execution Progress./etl/script[1] prepared: 10%
Aug 12, 2020 8:20:22 AM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/home/user1/workspace/etl-scripts/my-first-etl.xml"
Aug 12, 2020 8:20:22 AM <SEVERE> Script /home/user1/workspace/etl-scripts/my-first-etl.xml execution failed.
Location: /etl/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
select * from TestDB.dbo.GLAccounts
Error codes: [null, 0]
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
etl-scriptsS java -jar ~/Downloads/scriptella/scriptella-1.1/scriptella.jar -debug my-first-etl.xml
Aug 12, 2020 9:30:52 AM <WARNING> XML configuration warning in file:/home/user1/workspace/etl-scripts/my-first-etl.xml(28:14): The content of element type "script" must match "(include|dialect|onerror)".
Aug 12, 2020 9:30:52 AM <INFO> Execution Progress.Initializing properties: 1%
Aug 12, 2020 9:30:53 AM <FINE> registerDriver: SQLServerDriver:1
Aug 12, 2020 9:30:53 AM <FINE> Found driver class com.microsoft.sqlserver.jdbc.SQLServerDriver
Aug 12, 2020 9:30:53 AM <FINE> DriverManager.getConnection("jdbc:sqlserver://XXX.XXX.XXX.XXX:12345")
Aug 12, 2020 9:30:53 AM <FINE> trying com.microsoft.sqlserver.jdbc.SQLServerDriver
Aug 12, 2020 9:30:54 AM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.
Aug 12, 2020 9:30:54 AM <FINE> SQLWarning: reason(TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.)
Aug 12, 2020 9:30:54 AM <FINE> getConnection returning com.microsoft.sqlserver.jdbc.SQLServerDriver
Aug 12, 2020 9:30:54 AM <FINE> jdbc:sqlserver://XXX.XXX.XXX.XXX:12345: Statement cache is enabled (cache size 64). Statement separator ';'. Autocommit: false.
Aug 12, 2020 9:30:54 AM <INFO> Execution Progress.Initialized connection JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}, Dialect{Microsoft SQL Server 11.00.2218}, properties {}: 5%
Aug 12, 2020 9:30:54 AM <INFO> Execution Progress./etl/script[1] prepared: 10%
Aug 12, 2020 9:30:54 AM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/home/user1/workspace/etl-scripts/my-first-etl.xml"
Aug 12, 2020 9:30:54 AM <FINE> Executing script /etl/script[1]
Aug 12, 2020 9:30:55 AM <FINE> SQLState(null) vendor code(0)
Aug 12, 2020 9:30:55 AM <FINE> Rolling back JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}
Aug 12, 2020 9:30:55 AM <FINE> Closing JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}
Aug 12, 2020 9:30:55 AM <SEVERE> Script /home/user1/workspace/etl-scripts/my-first-etl.xml execution failed.
Location: /etl/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
select * from TestDB.dbo.GLAccounts
Error codes: [null, 0]
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
---------------Debug Stack Trace-----------------
scriptella.core.ExceptionInterceptor$ExecutionException: /etl/script[1] failed: Unable to execute statement
at scriptella.core.ExceptionInterceptor.execute(ExceptionInterceptor.java:44)
at scriptella.core.Session.execute(Session.java:103)
at scriptella.execution.EtlExecutor.execute(EtlExecutor.java:227)
at scriptella.execution.EtlExecutor.execute(EtlExecutor.java:183)
at scriptella.tools.launcher.EtlLauncher.execute(EtlLauncher.java:276)
at scriptella.tools.launcher.EtlLauncher.launch(EtlLauncher.java:193)
at scriptella.tools.launcher.EtlLauncher.main(EtlLauncher.java:321)
Caused by: scriptella.jdbc.JdbcException: Unable to execute statement. Error statement:
select * from TestDB.dbo.GLAccounts. Error codes: [null, 0]
at scriptella.jdbc.SqlExecutor.statementParsed(SqlExecutor.java:140)
at scriptella.jdbc.SqlParserBase.handleStatement(SqlParserBase.java:129)
at scriptella.jdbc.SqlParserBase.parse(SqlParserBase.java:72)
at scriptella.jdbc.SqlExecutor.execute(SqlExecutor.java:85)
at scriptella.jdbc.SqlExecutor.execute(SqlExecutor.java:63)
at scriptella.jdbc.JdbcConnection.executeScript(JdbcConnection.java:211)
at scriptella.core.ScriptExecutor.execute(ScriptExecutor.java:49)
at scriptella.core.ContentExecutor.execute(ContentExecutor.java:73)
at scriptella.core.ElementInterceptor.executeNext(ElementInterceptor.java:56)
at scriptella.core.StatisticInterceptor.execute(StatisticInterceptor.java:41)
at scriptella.core.ElementInterceptor.executeNext(ElementInterceptor.java:56)
at scriptella.core.ExceptionInterceptor.execute(ExceptionInterceptor.java:39)
... 6 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:886)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:767)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7375)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3200)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:247)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:222)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeUpdate(SQLServerStatement.java:710)
at scriptella.jdbc.StatementWrapper$Simple.update(StatementWrapper.java:167)
at scriptella.jdbc.SqlExecutor.statementParsed(SqlExecutor.java:130)
... 17 more
The issue was with my script. The corrected script below worked fine ...
<!DOCTYPE etl SYSTEM "http://scriptella.org/dtd/etl.dtd">
<etl>
<description>Scriptella ETL File Template.</description>
<properties>
<include href="etl.properties"/> <!--Load from external properties file-->
</properties>
<!-- Connection declarations -->
<connection driver="$driver" url="$url" user="$user" password="$password" classpath="$classpath"/>
<!--ETL Queries and Scripts -->
<query>
select * from TestDB.dbo.GLAccounts;
<script>
insert into TestDB.dbo.test (id) values (?1);
</script>
</query>
</etl>
I have a cron job in GAE. This cron job will create a BigQuery Job and submit it.
When I login into BigQuery console, I don't see this Job. I see only the Job created from BigQuery console not the jobs created via this CRON job.
I want to see whether the job is completed successfully or not. How can see the jobs created via GAE in BigQuery console?
I am using Python in GAE and here is the code snippet of it
job = bigquery_client.run_async_query(job_name, query)
# We are using Standard SQL
job.use_legacy_sql = False
# Start BigQuery job
job.begin()
use the cli tool to obtain the most recent jobs:
bq ls -j -a --max_results=15
running this would display a table with job ids and the respective timing.
jobId Job Type State Start Time Duration
--------------------------------- ---------- --------- ----------------- ----------
bquijob_1864e679_15a84d8878a query SUCCESS 28 Feb 07:11:06 0:00:04
bquijob_770b512_15a84d8122c query FAILURE 28 Feb 07:10:35 0:00:00
bquijob_de0df03_15a84d6a4fa query FAILURE 28 Feb 07:09:02 0:00:00
bquijob_52c4f7d7_15a84d660e6 query FAILURE 28 Feb 07:08:44 0:00:00
bquijob_76a2c1be_15a84d5e769 query FAILURE 28 Feb 07:08:13 0:00:00
bquijob_7f51dde5_15a84d55afb query SUCCESS 28 Feb 07:07:41 0:00:08
bquijob_34f25864_15a84d50503 query SUCCESS 28 Feb 07:07:18 0:00:08
job_Ca0cuRTAjY7MEHAs7vTJMxtVYTs query SUCCESS 28 Feb 07:00:47 0:00:09
job_hHfmcdwyBsPsYF5dDvvOdR1Rmd0 load SUCCESS 28 Feb 07:00:26 0:00:20
job_mkiLf_mFHLKSplGJOtg-XDKzvv4 load SUCCESS 28 Feb 02:52:50 0:00:02
job_3RsPvttxWwv3SzVoOI9Cv_2yWtA query SUCCESS 27 Feb 21:18:40 0:00:08
job_JLsqJO0NEIlKNac6jkDWbwneGMg extract SUCCESS 27 Feb 11:35:04 0:00:17
job_KOS7vKX4aX0FNbK6dibE7cxzcQA query SUCCESS 27 Feb 11:33:44 0:00:37
bquijob_44046bec_15a802f703a query SUCCESS 27 Feb 09:27:48 0:00:07
job_2qQ6YSWeXaP2y2doONQJsIoga3c query SUCCESS 27 Feb 08:53:20 0:00:06
Second option is to enable audit logs, and locate the queries under the tables that are automatically created in one of the BigQuery datasets
https://cloud.google.com/bigquery/audit-logs
I am sending email with swiftmailer, All of them goes to the spam folder, The SPF shows the status pass. what could be the changes i need to do to unspam it
Delivered-To: josemon.zixent#gmail.com
Received: by 10.107.191.131 with SMTP id p125csp117228iof;
Tue, 18 Oct 2016 22:20:57 -0700 (PDT)
X-Received: by 10.55.18.28 with SMTP id c28mr4620542qkh.125.1476854457365;
Tue, 18 Oct 2016 22:20:57 -0700 (PDT)
Return-Path: <tips#interviewtips.xyz>
Received: from interviewtips.xyz (li1017-172.members.linode.com. [45.33.71.172])
by mx.google.com with ESMTP id w74si23142432qkw.3.2016.10.18.22.20.57
for <josemon.zixent#gmail.com>;
Tue, 18 Oct 2016 22:20:57 -0700 (PDT)
Received-SPF: pass (google.com: domain of tips#interviewtips.xyz designates 45.33.71.172 as permitted sender) client-ip=45.33.71.172;
Authentication-Results: mx.google.com;
spf=pass (google.com: domain of tips#interviewtips.xyz designates 45.33.71.172 as permitted sender) smtp.mailfrom=tips#interviewtips.xyz
Received: by interviewtips.xyz (Postfix, from userid 1001) id B36DE817B9; Wed, 19 Oct 2016 05:20:54 +0000 (UTC)
To: josemon.zixent#gmail.com
Subject: [josemon.zixent] Is your Resume not getting shortlisted?
X-PHP-Originating-Script: 0:SimpleMailInvoker.php
Message-ID: <a6d39b3f95ee3608da294edc48eee3e7#interviewtips.xyz>
Date: Wed, 19 Oct 2016 05:20:54 +0000
From: tips#interviewtips.xyz
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="_=_swift_v4_1476854454_e4fa42ac587c6d9e0a47f9b64d71b9ec_=_"
tips: tips#interviewtips.xyz
--_=_swift_v4_1476854454_e4fa42ac587c6d9e0a47f9b64d71b9ec_=_
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
--_=_swift_v4_1476851958_4161d478461e54e72c39c80d6e24bfa1_=_
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
If you want your emails not to get in spam you need a few more things than just an SPF record. Specially when you are sending mails to Windows or Google you need to have a good reputation. They got spam filters wich will add you to the spam folder quickly if you have a small problem with your reputation.
Get some more information about the following protocols if you want to get a good repu:
SPF
DKIM
DMARC
PTR
There are a lot of domain checking tools you can find by googling.
I prefer this: http://mxtoolbox.com/domain/