I am trying to update postgresSQL table with psycopg2 (python package) sometimes it is failing with below error.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Here is the code
from psycopg2 import pool
now = datetime.now()
logoff_time = datetime(now.year, now.month, now.day, 15, 0, 0)
while True:
time.sleep(1)
try:
status = 'EXECUTED'
exec_type1 = 'CANCELLED'
exec_type2 = 'COMPLETED'
try:
postgreSQL_pool = pool.SimpleConnectionPool(1, 20, host = db_host,
database = db_name,
port = db_port,
user = db_user,
password = db_pwd)
if postgreSQL_pool:
print("Connection pool created successfully")
conn = postgreSQL_pool.getconn()
except (Exception, psycopg2.DatabaseError) as error:
print(error)
sql = """ UPDATE orders SET status = %s, executed_type = %s WHERE order_id = %s"""
updated_rows = 0
try:
cur = conn.cursor()
cur.execute(sql, (status, exec_type1, order_id,))
conn.commit()
updated_rows = cur.rowcount
cur.close()
break
except (Exception, psycopg2.DatabaseError) as error:
print(error)
print(updated_rows)
except Exception as e:
print(e)
psycopg2 version: '2.8.6 (dt dec pq3 ext lo64)'
Postgres: PostgreSQL 12.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-9), 64-bit
it is pretty much simple task but facing challenges. suggestions please
The server is crashing for some reason that you might be able to read in the server's logs.
I run the following code in python:
exchange_path = f"dfs://market/{exchange.value}"
script = f'''if(existsDatabase(\"{exchange_path}\")) {{ dropDatabase(\"{exchange_path}\") }}'''
session.run(script)
There is an error:
RuntimeError: <Server Exception> in run: dropDatabase("dfs://market/BINANCE") => deleteSubChunks failed on '/market/BINANCE', chunk 17a19f25-cc16-15ae-714b-5ee30d5d6795 is not in COMPLETE state
Firstly drop the partitions by dropPartition and then use dropDatabase to delete the database.
dbName=exchange_path
fileCond=dbName + "%"
t=exec substr(file,strlen(dbName)) from rpc(getControllerAlias(),getClusterChunksStatus) where file like fileCond, state != "COMPLETE"
dropPartition(database("dfs:/"+dbName),t,,true)
I get Error Number: 22001/8152
SQL [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]String or binary data would be truncated.
that means
one of my columns in the source table is bigger than my destination columns.
unfortunately that messsage not display which column is it who , I work with much table columns structure it is about 161 column. I think checking one by one my parameter value and column size is not good deal.
is there fastest way to do that?
this is my inssert query look like
INSERT INTO "my_db"."dbo"."my_table" ("Indeks_Keterpaduan", "NA", "admin_pembiayaan", "adopt_b2", "aktivitas_id", "aktivitas_nama", "assessment", "catatan", "dba_", "dlokasi", "dukungan_kawasan", "ikk_id", "ikk_nama", "ikp_id", "ikp_nama", "isu_strategis_id", "isu_strategis_nama", "jaringan_jalan", "jenis_arahan", "jenis_kegiatan", "jenis_kontrakID", "jenis_kontrakNama", "jenis_kotrakKode", "jenis_sub_aktivitas", "kabkot", "kawasan_nama", "kecamatan_id", "kegiatan_id", "kegiatan_nama", "ket_lokasi", "keterangan", "kewenangan", "kode_ssp", "kode_sspn", "kpk", "kws_kode", "outcome_id", "outcome_nama", "output_id", "output_nama", "param_a", "param_b", "param_c", "param_d", "param_e", "param_total", "phln", "politik_anggaran", "prioritas", "program_id", "program_nama", "provinsi_id", "rc_DED", "rc_DEDalokasi", "rc_DEDsatuan", "rc_DEDvol", "rc_Dokling", "rc_Doklingalokasi", "rc_Doklingsatuan", "rc_Doklingvol", "rc_FS", "rc_FSalokasi", "rc_FSsatuan", "rc_FSvol", "rc_dokptanah", "rc_kesesuaian_rtrw", "rc_konstruksi", "rc_konstruksialokasi", "rc_konstruksisatuan", "rc_konstruksivol", "rc_lahan", "rc_lahanalokasi", "rc_lahansatuan", "rc_lahanvol", "rc_masterplan", "rc_pnp1", "rc_pnp2", "rc_proposalpnp", "rc_rdtr", "rc_rdtralokasi", "rc_rdtrsatuan", "rc_rdtrvol", "rc_rispam", "rc_rispamalokasi", "rc_rispamsatuan", "rc_rispamvol", "rc_rp2kpkp", "rc_rp2kpkpalokasi", "rc_rp2kpkpsatuan", "rc_rp2kpkpvol", "rc_rtbl", "rc_rtblalokasi", "rc_rtblsatuan", "rc_rtblvol", "rc_sddt", "rc_skbup", "rc_skbupalokasi", "rc_skbupsatuan", "rc_skbupvol", "rc_skma", "rc_skt", "rc_spb", "rc_spd", "rc_spp", "readines", "rmp", "rpm", "sa1thn_id", "sajp_id", "sastra_id", "sastra_uraian", "satuan_output", "sbsn", "score_admin_pembiayaan", "score_jaringan_jalan", "score_kpk", "score_politik_anggaran", "score_readines", "sinkronisasi", "status", "status_konreg", "status_rakor", "status_verifikasi", "sub_aktivitas", "subkawasan_nama", "suboutput_id", "suboutput_nama", "sumber_arahan", "sumber_dana", "sumber_data_lainnya", "sumber_input_data", "tahun_anggaran", "tapisan", "time_create", "time_update", "total_score", "unit_id", "unit_nama", "uraian_ssp", "uraian_sspn", "user_create", "user_update", "volume", "waa_", "wab_", "wac_", "wba_", "wbb_", "wbc_", "wca_", "wcb_", "wcc_", "wda_", "wdb_", "wdc_", "wps_kode", "wps_nama", "x", "y")values (value1,value2,...,value (n))
Unfortunately, prior to SQL Server 2019, there's no easy way to pinpoint the cause of error msg 8152 "String or Binary data would be truncated".
While not an ideal solution (but it could save your sanity!), what you could do is install a local, side-by-side preview instance of SQL Server 2019, and run your application against it just to get the exact cause of the error!
Ref: String or Binary data would be truncated: replacing the infamous error 8152
I have a perl script which prepares a statement with params 1,2,3,4 using DBI interface .The prepare statement executes a stored procedure with 4 params.Then it runs a foreach loop and does execute
If the parameters are not right the stored procedure throws error and execute fails. I am interested in printing a warning and proceeding to next set of params.
After the first execute failure for an "invalid parameter", I get "Attempt to initiate a new Adaptive Server operation with results pending" as errmsg in SQL execute for subsequent loop iteration.
foreach my $file (#filelist)
{
#.. get param1 , param2 , param3, param4 from $file
unless ( dbh->execute($param1,$param2,$param3,$param4) )
{
#print some warning
next;
}
}
How do I continue processing with this error?
These are the version I am using
>perl -MDBI -e 'DBI-> installed_versions;'
Perl : 5.010001 (x86_64-linux-thread-multi)
OS : linux (2.6.18-348.12.1.el5)
DBI : 1.609
DBD::Sybase : 1.15
DBD::Sponge : 12.010002
DBD::SQLite : 1.27
DBD::Proxy : install_driver(Proxy) failed: Can't locate RPC/PlClient.pm in #INC
DBD::Informix : 2013.0521
DBD::Gofer : 0.011565
DBD::File : 0.37
DBD::ExampleP : 12.010007
DBD::DBM : 0.03
I added { RaiseError => 0, PrintWarn => 1, PrintError => 1 } in the connect as suggested by mekazu, (The code is exact same as yours mekazu ) but it is not helping still getting the same error.
In foreach I get success,
iteration 1 no error
iteration 2 gives a stored procedure error
" Database execute failed. ERROR MESSAGE: Server message number=52001
severity=16 state=1 line=124 server=dev procedure=sp1 text=Data
Error:Invalid parameter: P1
iteration 3
Database execute failed. ERROR MESSAGE: OpenClient message: LAYER =
(0) ORIGIN = (0) SEVERITY = (78) NUMBER = (51) Server dev, database
dev Message String: Attempt to initiate a new Adaptive Server
operation with results pending
iteration 4
Database execute failed. ERROR MESSAGE: OpenClient message: LAYER =
(0) ORIGIN = (0) SEVERITY = (78) NUMBER = (51) Server dev, database
dev Message String: Attempt to initiate a new Adaptive Server
operation with results pending
Ensure you disable RaiseError (default: false) when creating the handle. You can use PrintWarn (default: false) and PrintError (default: true) to save you doing it yourself:
$dbh = DBI->connect($dsn, $user, $password,
{ RaiseError => 0, PrintWarn => 1, PrintError => 1 });
my $sth = $dbh->prepare($query);
foreach my $file (#filelist)
{
#.. get param1 , param2 , param3, param4 from $file
unless ( $sth->execute($param1,$param2,$param3,$param4) )
{
#warning already printed
next;
}
}
i have many data.frames() that i am trying to send to MySQL database via RMySQL().
# Sends data frame to database without a problem
dbWriteTable(con3, name="SPY", value=SPY , append=T)
# stock1 contains a character vector of stock names...
stock1 <- c("SPY.A")
But when I try to loop it:
i= 1
while(i <= length(stock1)){
# converts "SPY.A" into SPY
name <- print(paste0(str_sub(stock1, start = 1, end = -3))[i], quote=F)
# sends data.frame to database
dbWriteTable(con3,paste0(str_sub(stock1, start = 1, end = -3))[i], value=name, append=T)
i <- 1+i
}
The following warning is returned & nothing was sent to database
In addition: Warning message:
In file(fn, open = "r") :
cannot open file './SPY': No such file or directory
However, I believe that the problem is with pasting value onto dbWriteTable() since writing dbWriteTable(con3, "SPY", SPY, append=T) works but dbWriteTable(con3, "SPY", name, append=T) will not...
You are probably using a non-base package for str_sub and I'm guessing you get the same behavior with substr. Does this succeed?
dbWriteTable(con3, substr( stock1, 1,3) , get(stock1), append=T)