In the Abp Framework (ABP.IO, Version: 5.2.1)
I need a way to cancel/stop a very long-running background workers/jobs when the application is stopped.
Currently, background workers/jobs continue running and prevent the application from stopping.
example log output:
10 May 2022 13:52:29.311 Parsing ended for a cdr file: 'xxx' in '122' seconds
10 May 2022 13:52:19.874 Parsing ended for a cdr file: 'xxx' in '113' seconds
10 May 2022 13:52:18.093 Parsing file 'xxx'
10 May 2022 13:52:17.969 Parsing started for a cdr file: 'xxx'
10 May 2022 13:52:07.572 Parsing ended for a cdr file: 'xxx' in '100' seconds
10 May 2022 13:52:06.959 Application is shutting down...
10 May 2022 13:51:36.174 Parsing file 'xxx'
10 May 2022 13:51:36.072 Parsing started for a cdr file: 'xxx'
10 May 2022 13:51:29.453 Parsing ended for a cdr file: 'xxx' in '62' seconds
10 May 2022 13:50:56.258 Parsing file 'xxx'
10 May 2022 13:50:56.223 Parsing started for a cdr file: 'xxx'
10 May 2022 13:50:52.202 Parsing ended for a cdr file: 'xxx' in '25' seconds
I have tried injecting an ICancellationTokenProvider, but it didn't help.
await GetFilesList().ParallelForEachAsync(async file =>
{
// Do some parsing
}, _cancellationTokenProvider.Token);
Related
I am using custom SQL in SQL override section in Advance property of snowflake source in IICS Informatica cloud.
select (max(LAST_UPDATED_DATE)) from $$DB.test.org. $$DB parameter is defined in the parameter file $$DB=Dev. Parameter is resolving correctly at run time but getting below error.
READER_1_2_1> SNOWFLAKECLOUDDATAWAREHOUSE_10000 [2020-10-30 03:07:00.440] [INFO] The Snowflake Connector uses the following SQL query to read data: select (max(LAST_UPDATED_DATE)) from Dev.test.org
READER_1_2_1> CMN_1761 [2020-10-30 03:07:00.440] Timestamp Event: [Fri Oct 30 03:07:00 2020]
READER_1_2_1> SNOWFLAKECLOUDDATAWAREHOUSE_10000 [2020-10-30 03:07:00.441] [ERROR] The Snowflake Connector failed to read data from the source because of the following error: [SnowFlake_READER__FAILED_906] The Snowflake Connector failed to read data from the source because of the following error: null
READER_1_2_1> CMN_1761 [2020-10-30 03:07:00.441] Timestamp Event: [Fri Oct 30 03:07:00 2020]
READER_1_2_1> SNOWFLAKECLOUDDATAWAREHOUSE_10000 [2020-10-30 03:07:00.441] [ERROR] Error occured while reading data using CCI Data Adapter com.informatica.cci.runtime.internal.utils.impl.CExceptionImpl: Unable to get result data: Adapter read method has returned failure.
READER_1_2_1> CMN_1761 [2020-10-30 03:07:00.441] Timestamp Event: [Fri Oct 30 03:07:00 2020]
READER_1_2_1> CONNECTOR_10000 [2020-10-30 03:07:00.441] [ERROR] Error Occured while reading data from the source for Transformation - SRC_Organization | Connector Name - SnowflakeCloudDataWarehouse - Unable to get result data: Adapter read method has returned failure.
Thanks in Advance.
The issue was with the column list. Custom query should be like select (max(LAST_UPDATED_DATE)) as LAST_UPDATED_DATE from $$DB.test.org and the Field list should have only one column LAST_UPDATED_DATE. Mapping is working fine now.
I am a first-time user of Scriptella, and am really struggling with getting started.
I seem to be running into the same issue on both SQL Server 2012 as well as MySQL. With both, I am able to CREATE tables and INSERT records, but get the following errors when using SELECT statements.
SQL Server using driver mssql-jdbc-8.4.0.jre8.jar (and earlier versions):
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
MYSQL using driver mysql-connector-java-5.1.45.jar:
Driver exception: java.sql.SQLException: Can not issue SELECT via executeUpdate() or executeLargeUpdate().
I am unable to figure out if this is an issue with my script or if it is a server configuration issue. I tried to set nocount on and nocount off on sql server, but to no avail.
I tried Scriptella version 1.1 as well as version 1.2, and have the same issue.
I am so stuck on this problem that after years of following stackoverflow.com, I am finally posting my first question here. I would so very much appreciate any help.
Please see below for the script I am using as well as the stacktrace. Thanks.
<!DOCTYPE etl SYSTEM "http://scriptella.org/dtd/etl.dtd">
<etl>
<properties>
<include href="etl.properties"/> <!--Load from external properties file-->
</properties>
<!-- Connection declarations -->
<connection driver="$driver" url="$url" user="$user" password="$password" classpath="$classpath"/>
<!--ETL Queries and Scripts -->
<script>
<!-- CREATE TABLE WORKED!
create table TestDB.dbo.test (
id int
); -->
<!-- INSERT WORKED!
insert into TestDB.dbo.test (id) values (1);
-->
<!-- BELOW FAILS -->
select * from TestDB.dbo.GLAccounts
<script>
insert into TestDB.dbo.test (id) values (?1);
</script>
</script>
</etl>
etl-scriptsS java -jar ~/Downloads/scriptella/scriptella-1.1/scriptella.jar my-first-etl.xml
Aug 12, 2020 8:11:33 AM <WARNING> XML configuration warning in file:/home/user1/workspace/etl-scripts/my-first-etl.xml(28:14): The content of element type "script" must match "(include|dialect|onerror)".
Aug 12, 2020 8:11:33 AM <INFO> Execution Progress.Initializing properties: 1%
Aug 12, 2020 8:11:36 AM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.
Aug 12, 2020 8:11:36 AM <INFO> Execution Progress.Initialized connection JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}, Dialect{Microsoft SQL Server 11.00.2218}, properties {}: 5%
Aug 12, 2020 8:11:36 AM <INFO> Execution Progress./etl/script[1] prepared: 10%
Aug 12, 2020 8:11:36 AM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/home/user1/workspace/etl-scripts/my-first-etl.xml"
Aug 12, 2020 8:11:36 AM <SEVERE> Script /home/user1/workspace/etl-scripts/my-first-etl.xml execution failed.
Location: /etl/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
select * from TestDB.dbo.GLAccounts
Error codes: [null, 0]
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
etl-scriptsS java -jar ~/Downloads/scriptella/scriptella-1.1/scriptella.jar my-first-etl.xml
Aug 12, 2020 8:20:19 AM <WARNING> XML configuration warning in file:/home/user1/workspace/etl-scripts/my-first-etl.xml(28:14): The content of element type "script" must match "(include|dialect|onerror)".
Aug 12, 2020 8:20:19 AM <INFO> Execution Progress.Initializing properties: 1%
Aug 12, 2020 8:20:21 AM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.
Aug 12, 2020 8:20:22 AM <INFO> Execution Progress.Initialized connection JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}, Dialect{Microsoft SQL Server 11.00.2218}, properties {}: 5%
Aug 12, 2020 8:20:22 AM <INFO> Execution Progress./etl/script[1] prepared: 10%
Aug 12, 2020 8:20:22 AM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/home/user1/workspace/etl-scripts/my-first-etl.xml"
Aug 12, 2020 8:20:22 AM <SEVERE> Script /home/user1/workspace/etl-scripts/my-first-etl.xml execution failed.
Location: /etl/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
select * from TestDB.dbo.GLAccounts
Error codes: [null, 0]
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
etl-scriptsS java -jar ~/Downloads/scriptella/scriptella-1.1/scriptella.jar -debug my-first-etl.xml
Aug 12, 2020 9:30:52 AM <WARNING> XML configuration warning in file:/home/user1/workspace/etl-scripts/my-first-etl.xml(28:14): The content of element type "script" must match "(include|dialect|onerror)".
Aug 12, 2020 9:30:52 AM <INFO> Execution Progress.Initializing properties: 1%
Aug 12, 2020 9:30:53 AM <FINE> registerDriver: SQLServerDriver:1
Aug 12, 2020 9:30:53 AM <FINE> Found driver class com.microsoft.sqlserver.jdbc.SQLServerDriver
Aug 12, 2020 9:30:53 AM <FINE> DriverManager.getConnection("jdbc:sqlserver://XXX.XXX.XXX.XXX:12345")
Aug 12, 2020 9:30:53 AM <FINE> trying com.microsoft.sqlserver.jdbc.SQLServerDriver
Aug 12, 2020 9:30:54 AM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.
Aug 12, 2020 9:30:54 AM <FINE> SQLWarning: reason(TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.)
Aug 12, 2020 9:30:54 AM <FINE> getConnection returning com.microsoft.sqlserver.jdbc.SQLServerDriver
Aug 12, 2020 9:30:54 AM <FINE> jdbc:sqlserver://XXX.XXX.XXX.XXX:12345: Statement cache is enabled (cache size 64). Statement separator ';'. Autocommit: false.
Aug 12, 2020 9:30:54 AM <INFO> Execution Progress.Initialized connection JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}, Dialect{Microsoft SQL Server 11.00.2218}, properties {}: 5%
Aug 12, 2020 9:30:54 AM <INFO> Execution Progress./etl/script[1] prepared: 10%
Aug 12, 2020 9:30:54 AM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/home/user1/workspace/etl-scripts/my-first-etl.xml"
Aug 12, 2020 9:30:54 AM <FINE> Executing script /etl/script[1]
Aug 12, 2020 9:30:55 AM <FINE> SQLState(null) vendor code(0)
Aug 12, 2020 9:30:55 AM <FINE> Rolling back JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}
Aug 12, 2020 9:30:55 AM <FINE> Closing JdbcConnection{com.microsoft.sqlserver.jdbc.SQLServerConnection}
Aug 12, 2020 9:30:55 AM <SEVERE> Script /home/user1/workspace/etl-scripts/my-first-etl.xml execution failed.
Location: /etl/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
select * from TestDB.dbo.GLAccounts
Error codes: [null, 0]
Driver exception: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
---------------Debug Stack Trace-----------------
scriptella.core.ExceptionInterceptor$ExecutionException: /etl/script[1] failed: Unable to execute statement
at scriptella.core.ExceptionInterceptor.execute(ExceptionInterceptor.java:44)
at scriptella.core.Session.execute(Session.java:103)
at scriptella.execution.EtlExecutor.execute(EtlExecutor.java:227)
at scriptella.execution.EtlExecutor.execute(EtlExecutor.java:183)
at scriptella.tools.launcher.EtlLauncher.execute(EtlLauncher.java:276)
at scriptella.tools.launcher.EtlLauncher.launch(EtlLauncher.java:193)
at scriptella.tools.launcher.EtlLauncher.main(EtlLauncher.java:321)
Caused by: scriptella.jdbc.JdbcException: Unable to execute statement. Error statement:
select * from TestDB.dbo.GLAccounts. Error codes: [null, 0]
at scriptella.jdbc.SqlExecutor.statementParsed(SqlExecutor.java:140)
at scriptella.jdbc.SqlParserBase.handleStatement(SqlParserBase.java:129)
at scriptella.jdbc.SqlParserBase.parse(SqlParserBase.java:72)
at scriptella.jdbc.SqlExecutor.execute(SqlExecutor.java:85)
at scriptella.jdbc.SqlExecutor.execute(SqlExecutor.java:63)
at scriptella.jdbc.JdbcConnection.executeScript(JdbcConnection.java:211)
at scriptella.core.ScriptExecutor.execute(ScriptExecutor.java:49)
at scriptella.core.ContentExecutor.execute(ContentExecutor.java:73)
at scriptella.core.ElementInterceptor.executeNext(ElementInterceptor.java:56)
at scriptella.core.StatisticInterceptor.execute(StatisticInterceptor.java:41)
at scriptella.core.ElementInterceptor.executeNext(ElementInterceptor.java:56)
at scriptella.core.ExceptionInterceptor.execute(ExceptionInterceptor.java:39)
... 6 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:886)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:767)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7375)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3200)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:247)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:222)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeUpdate(SQLServerStatement.java:710)
at scriptella.jdbc.StatementWrapper$Simple.update(StatementWrapper.java:167)
at scriptella.jdbc.SqlExecutor.statementParsed(SqlExecutor.java:130)
... 17 more
The issue was with my script. The corrected script below worked fine ...
<!DOCTYPE etl SYSTEM "http://scriptella.org/dtd/etl.dtd">
<etl>
<description>Scriptella ETL File Template.</description>
<properties>
<include href="etl.properties"/> <!--Load from external properties file-->
</properties>
<!-- Connection declarations -->
<connection driver="$driver" url="$url" user="$user" password="$password" classpath="$classpath"/>
<!--ETL Queries and Scripts -->
<query>
select * from TestDB.dbo.GLAccounts;
<script>
insert into TestDB.dbo.test (id) values (?1);
</script>
</query>
</etl>
I have a cron job in GAE. This cron job will create a BigQuery Job and submit it.
When I login into BigQuery console, I don't see this Job. I see only the Job created from BigQuery console not the jobs created via this CRON job.
I want to see whether the job is completed successfully or not. How can see the jobs created via GAE in BigQuery console?
I am using Python in GAE and here is the code snippet of it
job = bigquery_client.run_async_query(job_name, query)
# We are using Standard SQL
job.use_legacy_sql = False
# Start BigQuery job
job.begin()
use the cli tool to obtain the most recent jobs:
bq ls -j -a --max_results=15
running this would display a table with job ids and the respective timing.
jobId Job Type State Start Time Duration
--------------------------------- ---------- --------- ----------------- ----------
bquijob_1864e679_15a84d8878a query SUCCESS 28 Feb 07:11:06 0:00:04
bquijob_770b512_15a84d8122c query FAILURE 28 Feb 07:10:35 0:00:00
bquijob_de0df03_15a84d6a4fa query FAILURE 28 Feb 07:09:02 0:00:00
bquijob_52c4f7d7_15a84d660e6 query FAILURE 28 Feb 07:08:44 0:00:00
bquijob_76a2c1be_15a84d5e769 query FAILURE 28 Feb 07:08:13 0:00:00
bquijob_7f51dde5_15a84d55afb query SUCCESS 28 Feb 07:07:41 0:00:08
bquijob_34f25864_15a84d50503 query SUCCESS 28 Feb 07:07:18 0:00:08
job_Ca0cuRTAjY7MEHAs7vTJMxtVYTs query SUCCESS 28 Feb 07:00:47 0:00:09
job_hHfmcdwyBsPsYF5dDvvOdR1Rmd0 load SUCCESS 28 Feb 07:00:26 0:00:20
job_mkiLf_mFHLKSplGJOtg-XDKzvv4 load SUCCESS 28 Feb 02:52:50 0:00:02
job_3RsPvttxWwv3SzVoOI9Cv_2yWtA query SUCCESS 27 Feb 21:18:40 0:00:08
job_JLsqJO0NEIlKNac6jkDWbwneGMg extract SUCCESS 27 Feb 11:35:04 0:00:17
job_KOS7vKX4aX0FNbK6dibE7cxzcQA query SUCCESS 27 Feb 11:33:44 0:00:37
bquijob_44046bec_15a802f703a query SUCCESS 27 Feb 09:27:48 0:00:07
job_2qQ6YSWeXaP2y2doONQJsIoga3c query SUCCESS 27 Feb 08:53:20 0:00:06
Second option is to enable audit logs, and locate the queries under the tables that are automatically created in one of the BigQuery datasets
https://cloud.google.com/bigquery/audit-logs
I am working with a software that will output a text file containing various information regarding the measurement runs. I want to edit this file so that a certain set of strings can be deleted and then save this new file. Below is sample of the text file.
featureListName All Characteristics
meMode cf
naviMode automatic
pdfExport false
presProtocol false
printer false
printPlots false
protNumbers 1
protocolDefinition nil
protocolHead true
psExport false
ptxExport false
runMode slowToFirst
selection nil
speed 300
changedate July 5, 2017 10:00:35 am
changeoper Master
changeswi 6.2.1602
changeTimestamp 3676701635
changeTimestampOld 3676701635
creationdate July 5, 2017 9:57:53 am
creationdme Prismo
creationswi 6.2.1602
date 2017-7-5
dateandtime July 5, 2017 5:05:53 pm
datedayofmonth 5
datemonthindex 7
datesecondclock 3676745153
datestring July 5, 2017
dateyear 2017
dmeid Prismo
dmesn 000000
drawingno
mccplanversion
nameOfFile c:\Users\Public\Documents\Zeiss\CALYPSO\workarea\inspections\test
operid Master
order
partnbinc 3
partnbpre 3
planid test
produceoper Master
runswv 6.2.1602
time 5:05:53 pm
u_field1
u_field2
u_field3
u_field4 Contura1
My goal in editing this file is to remove the string "Contura1" and then save the file in the same directory with the same name. Any insight as to how I can accomplish this is very much appreciated.
I am having a issue in my company. We are getting a bunch of bad email delivery failure notifications either from a SQL Job, stored procedure, SSIS package or something else. Below is the sample Delivery failure email.
Delivery has failed to these recipients or groups:
NoLonger.Exists#somewhere.com
Your message wasn't delivered because the recipient's email provider rejected it.
Diagnostic information for administrators:
Generating server: server-3.bemta-4.messagelabs.com
NoLonger.Exists#somewhere.com
Remote Server returned '< #5.7.0 smtp; 551 5.7.0 recipient domain does not exist in DNS>'
Original message headers:
Return-Path:
Received: from [85.158.143.35] by server-3.bemta-4.messagelabs.com id C4/27-29399-60505E45; Wed, 18 Feb 2015 21:32:54 +0000
Received: (qmail 4065 invoked from network); 18 Feb 2015 21:32:54 -0000
Received: from a96-076-static.datachambers.com (HELO ussmtp02.hvwan.net) (204.15.96.76)
by server-14.tower-21.messagelabs.com with SMTP; 18 Feb 2015 21:32:54 -0000
Received: from MCR-SWEB01.hcop.us.hvwan.net ([10.73.65.2])
by ussmtp02.hvwan.net (IBM Domino Release 9.0.1FP1)
with ESMTP id 2015021821325373-22365 ;
Wed, 18 Feb 2015 21:32:53 +0000
X-Env-Sender: sqladmin#elsewhere.com
X-Msg-Ref: server-14.tower-21.messagelabs.com!1424295174!12730599!1
X-Originating-IP: [204.15.96.76]
X-StarScan-Received:
X-StarScan-Version: 6.13.4; banners=elsewhere.com,-,-
X-VirusChecked: Checked
Subject: =?iso-8859-1?Q?PART_REJECT_NMN#:_26477,_PN:_585864804?=
Sender: "=?iso-8859-1?Q?sqladmin#yad=2Ecom?="
From: "=?iso-8859-1?Q?sqladmin#yad=2Ecom?="
Date: Wed, 18 Feb 2015 16:32:12 -0500
To: "=?iso-8859-1?Q?carlton=2Eevans#yad=2Ecom?="
,
"=?iso-8859-1?Q?norman=2Ebuckhouse#anonymizing=2Ecom?="
,
"=?iso-8859-1?Q?scott=2Echaney#yad=2Ecom?="
,
"=?iso-8859-1?Q?jay=2Evandevoorde#yad=2Ecom?="
,
"=?iso-8859-1?Q?elliott=2Eortiz#yad=2Ecom?="
,
"=?iso-8859-1?Q?robert=2Ewallace#anonymizing=2Ecom?="
CC: "=?iso-8859-1?Q?carlton=2Eevans#anonymizingAgain=2Ecom?="
,
"=?iso-8859-1?Q?wesley=2Ewasden#yad=2Ecom?="
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
MIME-Version: 1.0
X-Mailer: JMail 4.4 by Dimac
X-MIMETrack: Itemize by SMTP Server on ussmtp02.hvwan.net/Servers/yad(Release
9.0.1FP1|April 03, 2014) at 02/18/2015 09:32:53 PM,
Serialize by Router on ussmtp02.hvwan.net/Servers/yad(Release
9.0.1FP1|April 03, 2014) at 02/18/2015 09:32:54 PM,
Serialize complete at 02/18/2015 09:32:54 PM,
Itemize by SMTP Server on ussmtp01.hvwan.net/Servers/yad(Release
9.0.1FP1|April 03, 2014) at 02/18/2015 09:32:57 PM,
Serialize by Router on ussmtp01.hvwan.net/Servers/yad(Release
9.0.1FP1|April 03, 2014) at 02/18/2015 09:33:00 PM
X-TNEFEvaluated: 1
Message-ID:
Content-Type: text/html; charset="US-ASCII"
Content-Disposition: inline
If you see the top of the Delivery Failure notification instead of NoLonger.Exists#anonymizing.com, inside a stored procedure or a DTS script or a job, we have carlton.evan#somewhere.com.
How can I solve this problem without going through each and every stored procedure or the SSIS packages on the server?
You seem to have entered a bad domain for the recipient. Husqvarnaroup instead of husqvarnagroup
you can use something like
SELECT OBJECT_NAME(object_id), *
FROM sys.sql_modules
WHERE definition LIKE '%yourharcodeemail#dontdoit.again%'
to check if its on your procedures but if you have it hardcoded you should change it to a table so it wouldn't happen again