I am unable to append to a table when writing to a table using dataframe write
Here is the command I am using
df1.write.mode("append").jdbc("jdbc:jtds:sqlserver://noi-nipuna-w81/sams","sams", props)
This is the exception I get
java.sql.SQLException: There is already an object named 'sams' in the database.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:372)
Where am I making a mistake
I am using spark 1.4 version
Urug and Hafiz have already answered. Below code did the job for me.
sqlDF
.write()
.mode(SaveMode.Append)
.jdbc(jdbcDBUrl, "DBName.SchemaName.TableName", connProperties);
I had the same problem with MS SQL Server. I found that updating Spark to 1.6.0 fixed the problem (link to the other SO answer). The reason was a buggy query for checking if the table exists.
Related
I am using datatables 1.10+ in my project. I am using MSSQL as my backend. To do server side processing, i have used an old server side scripting found here:
https://www.datatables.net/development/server-side/php_mssql_odbc
Everything is working well including dynamically passing values to that script. My problem is:
How can i get that script process a filtered query string instead of having all records in the supplied table. That is initially execute a query like
SELECT * FROM A WHERE Ax='10'
instead of
SELECT * FROM sTable
Hope i am clear with my question. Thank you in advance.
I've an SSIS package which works perfectly on various server but doesn't on 2 newly added server.
It uses OLEDB (64bit, I've certified that) to get data from an ORACLE DB from a table with 3 VARCHAR2 columns and upsert the rows to a SQL 2012 table with 3 nvarchar columns, with the same size.
The package was working flawlessly in all the servers we deployed it. We recently added 2 new servers, with hopefully (I personally checked the possible major culprit: ODAC, dtexec version, tns names ) the same software with the same versions but the package is not working anymore. It gives us the famous error:
Description: Column "C_CODE" cannot convert between unicode and non-unicode s
tring data types.
End Error
Error: 2016-08-26 08:33:26.20
Code: 0xC02020F6
Source: Upsert Lines OLE DB Source [87]
Description: Column "S_DESC" cannot convert between unicode and non-unicode s
tring data types.
End Error
Warning: 2016-08-26 08:33:26.20
Code: 0x800470C8
Source: Upsert Lines OLE DB Source [87]
Description: The external columns for OLE DB Source are out of synchronizatio
n with the data source columns. The external column "C_CODE" needs to be updated
.
The external column "S_DESC" needs to be updated.
End Warning
The first warning I got was the "Cannot read the code page from Oracle server" so I tried to put the "use the default one every time" in combination with ANSI or UTF-8 code pages ( used this website to get the codepages: https://msdn.microsoft.com/en-us/library/windows/desktop/dd317756%28v=vs.85%29.aspx ).
The only way I made it work was to manually modify the OLE DB Source block into the package by changing the external columns to ansi strings, making it not compilable in the dev environment, then put a data transform that would change the exernal columns to utf-8 string again.
This way it doesn't work in the other servers but does flawlessly in the "faulty" servers.
I was thinking that the fact the package cannot read the code page from Oracle server is the culprit but I couldn't figure out how to fix it.
The problem is not like the million of questions about this issue you can find anywhere. The error being given is from the OLE DB Source block. I know about the data conversion.
Thanks in advance
The issue was the NLS_LANG registry key's value, it was set to an incorrect value.
I solved by deleting the said key, all worked fine.
Just come across this post (and many others about this issue) and believe I have found the underlying cause. On my servers at least, the 64bit version of the Windows Oracle drivers has a bug in the install process. I have checked the 32bit version and this bug is not there which explains the behaviour.
It appears that the 64bit version creates the NLS_LANG registry key in the wrong location. By default the 64bit driver create it here - "HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE"
I manually created it here - "HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\KEY_OraClient12Home1" which cured my issue. For reference my NLS_LANG key is set to "ENGLISH_UNITED KINGDOM.WE8MSWIN1252". I just created it as a copy of the string in the branch above.
I hope this helps others.
We use 32 bit SSIS with Oracle instant client, I need to add entry in this registration location
"Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\ORACLE\KEY_OraClient12Home1_32bit"
The “NLS_LANG” value is “AMERICAN_AMERICA.WE8MSWIN1252”. This solved my problem.
I'm using SolrStorageHandler to insert data into Solr from Hive tables.
I'm following this tutorial but when lunching oozie workflow i get an error
"Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hadoop.hive.solr.SolrStorageHandler' (state=42000,code=40000)".
I cannot find SolrStorageHandler.jar on the web to add it manually to hive script. I'm doing something wrong or is this approach deprecated? If it's depracted how can I insert data to Solr from Hive in efficient way?
btw. Cloudera version is 5.4.3
This question already has answers here:
How to run a SQL query on an Excel table?
(11 answers)
Closed 8 years ago.
I have one excel sheet(name as:-LTATestData.xls). It contain some column with corresponding some value. I want select sql query which read the column value from excel sheet and display the result.
I have tried to search in google, i get below query which may solve my purpose
SELECT * FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 10.0;Database=C:\work\LTATestData.xls',
'SELECT * FROM [Sheet1$]')
But when I run this query I am getting below error message:- "OLE DB provider 'Microsoft.Jet.OLEDB.4.0' cannot be used for distributed queries because the provider is configured to run in single-threaded apartment mode."
Please let me know how to do this. Thanks in advance.
The main cause for this might be because of a mismatch between a 64-bit SQL Server and a 32-bit Microsoft Office installation. Therefore rather upgrade to a 64-bit Microsoft Office than downgrade a sql server version.
Alternatively, here a couple of posts if your versions are all up to date (Please read the comments sections as well)
The following posts might be helpful solving the problem: HOW TO: FIX ERROR - "the 'microsoft.ace.oledb.12.0' provider is not registered on the local machine"
AND
SQL SERVER – Fix: Error: MS Jet OLEDB 4.0 cannot be used for distributed queries because the provider is used to run in apartment mode.
Is there any chance of connecting opencart with mssql? Have anyone tried? If so what is the procedure of doing that?
That should not be a big problem, You only need to do:
create /system/database/mssql.php class - the class should have the same methods, properties and functionality as e.g. the mysql.php one
rewrite all of the model classes method's queries to meet the MS SQL / T-SQL SQL syntax
in both config files (/config.php and /admin/config.php) set the proper DB_DRIVER - mssql
I am supposing You have the OpenCart database created already due to the /install/opencart.sql file.
I guess nothing more should be done.
Anyway, what is the reason for switching to MS SQL?
EDIT: In /system/database/ there is this mmsql.php file which actually contains the MSSQL class thus this do not have to be implemented, just renamed to mssql.php file.