Is there a SQL Server hashbyte equivalent in Databricks? I have a project that migrates data from Databricks database to SQL Server database, I want to make sure that the data transferred is complete by comparing hash. Is there a way to compute hash of a dataframe ?
It seems that sql server command:
SELECT CONVERT( VARCHAR( MAX), HASHBYTES('SHA2_256', 'This is a test.'), 2)
gives the same result as this sql spark command in Databricks
SELECT upper( sha2( 'This is a test.', 256))
just put column name instead of 'This is a test.' and add FROM table_name
Related
I want to create an SSIS package with source as azure sql and destination as DB2 table.
I have to check if Id (int) in sql matches with Id (X12 basically varchar) in DB2, if it matches, I have to update name in db2 with the name in SQL
else insert new record in DB2.
Since, datatype of id does not match, I have to do a data type conversion- that is if id in sql is 1 .. I have to convert it to varchar with leading zeros - 00000000001
Can some one please provide a step by step approach?
and where exactly can I make this conversion.. in which task?
I have a pretty long Oracle SQL query that needs to be compatible for Azure SQL. I am new to both database types. Here is query:
MERGE INTO studies
USING dual
ON (study_id = :study_id)
WHEN NOT MATCHED THEN
INSERT (study_id, study_date)
VALUES (:study_id, :study_date)
I am not sure USING dual would work. I read some solution saying that USING dual is not necessary in SQL Server.
I really appreciate if you can explain what this query means and how I can translate this for Azure SQL query.
This Oracle merge query has just a WHEN NOT MATCHED clause and no WHEN MATCHED, so basically that's insert and not exists:
insert into studies(study_id, study_date)
select x.*
from (values(#study_id, #study_date)) as x(study_id, study_date)
where not exists (select 1 from studies s1 where s1.study_id = x.study_id)
This is logically equivalent to the original Oracle query.
As for your original question: SQL Server does supports its own flavor or merge statement, whose syntax is different than Oracle. You would rewrite the Oracle merge as:
merge studies as s
using (values(#study_id, #study_date)) as x(study_id, study_date)
on (s.study_id = x.study_id)
when not matched
then insert (study_id, study_date) values(x.study_id, x.study_date)
Want to retrieve records from Access through linked server in SQL Server and need to convert/cast the column with VARCHAR for some constraint.
My attempt:
SELECT Cola, Colb
FROM MSAccess_LinkedServer...TableName
WHERE CAST([Cola] AS VARCHAR) = 'A123'
Unable to get the result from above query.
But when I remove CAST then I will get the result, but I want to know how to put the cast/convert.
No casting should be needed for text. How about simply:
WHERE [Cola] = 'A123'
I am converting Access query to SQL Server.
I want to convert below lines to SQL
1. Format (210.6, "Standard")
2. Format (210.6, "#,##0.00")
How do i convert it to SQL query.
I have tried with below, but still not able to find the solution.
For the first query, i found below solution, which is correct
1. CONVERT(varchar, CAST(tSRO.OutputF11 AS money), 1)
Now, for second query, i do not know what i have to do.
From SQL Server 2012+ you can use FORMAT:
SELECT FORMAT(210.6, '#,##0.00') -- 210.60
SELECT FORMAT(1210.6, '#,##0.00') -- 1,210.60
LiveDemo
SQL Server before 2012:
SELECT REPLACE(CONVERT(VARCHAR,CONVERT(MONEY, 1210.6),1),'.00','') -- 1,210.60
LiveDemo2
Warning:
This operation is pure for presentation layer and should be done in application.
When obtaining an image grabde with sql server linked servers from PostgreSQL, I get the following error: OLE DB provider 'MSDASQL' for linked server 'bd_acceso_ruisegip' returned data that does not match expected data length for column '[MSDASQL] . fot_imagen '. The data length (maximum) expected is 255 and the data returned is 38471.
Don't know if you were dealing with a bytea column but I was having the same problem. Found the answer in the configuring of the postrgres ODBC system dsn. Under the Options/Datasource-page 2 there is a option for bytea as LO. Clicked that and now it works like a champ.
I found a similar issue when replicating some Forum data from PostgreSQL to MSSQL, using the PostgreSQL 64-bit driver and a Linked Server (.
When I coded like this:...
select * into Post from OpenQuery(PostgreSQL_Test1, 'select * From public.post')
... the MSSQL table defaulted to a column size of nvarchar(4000).
My fix: First, run it once with a small limit on the number of rows copied:
select * into Post from OpenQuery(PostgreSQL_Test1, 'select * From public.post limit 10')
Next, right-click on the local Post table. Choose "Script table as drop and create"
In the create script, replace the size of the offending column with VARCHAR(MAX)
Next, create the table.
Then use:
Insert Post select * from OpenQuery(PostgreSQL_Test1, 'select * From public.post')
Hope that helps.
Your mileage may vary.