I am using SSIS to move excel data to a temp sql server table and from there to the target table.
So my temp table consists of only varchar columns - my target table expects money values for some columns. In my temp table the original excel columns have a formula but leave an empty cell on some rows which is represented by the temp table with an empty cell as well. But when I cast one of these columns to money these originally blank cells become 0,00 in the target column.
Of course that is not what I want, so how can I get NULL values in there? Keeping in mind that it is possible that a wanted 0,00 shows up in one of these columns.
I guess I would need to edit my temp table to turn the empty cells to NULL. Can I do this from within a SSIS package or is there a setting for the table I could use?
thank you.
For existing data you can write a simple script that updates data to NULL where empty.
UPDATE YourTable SET Column = NULL WHERE Column = ''
For inserts you can use NULLIF function to insert nulls if empty
INSERT INTO YourTable (yourColumn)
SELECT NULLIF(sourceColum, '') FROM SourceTable
Edit: for multiple column updates you need to combine the two solutions and write something like:
UPDATE YourTable SET
Column1 = NULLIF(Column1, '')
, Column2 = NULLIF(Column2, '')
WHERE Column1 = '' OR Column2 = ''
etc
That will update all
Related
I have a stored procedure that mergers Local temp table and existing table.
ALTER PROCEDURE [dbo].[SyncProductVariantsFromServices]
#Items ProductVariantsTable readonly
AS
BEGIN
CREATE TABLE #ProductVariantsTemp
(
ItemCode nvarchar(10) collate SQL_Latin1_General_CP1_CI_AS,
VariantCode nvarchar(10) collate SQL_Latin1_General_CP1_CI_AS,
VariantDescriptionBG nvarchar(100) collate SQL_Latin1_General_CP1_CI_AS,
VariantDescriptionEN nvarchar(100) collate SQL_Latin1_General_CP1_CI_AS
)
insert into #ProductVariantsTemp
select ItemCode, VariantCode, VariantDescriptionBG, VariantDescriptionEN
from #Items
MERGE ProductVariants AS TARGET
USING #ProductVariantsTemp AS SOURCE
ON (TARGET.ItemCode = SOURCE.ItemCode AND TARGET.VariantCode= SOURCE.VariantCode)
WHEN NOT MATCHED BY TARGET THEN
INSERT (ItemCode, VariantCode, VariantDescriptionBG, VariantDescriptionEN)
VALUES (SOURCE.ItemCode, SOURCE.VariantCode, SOURCE.VariantDescriptionBG, SOURCE.VariantDescriptionEN)
OUTPUT INSERTED.ItemCode, INSERTED.VariantCode, GETDATE() INTO SyncLog;
The problem is - i know in the output clause i have access to inserted or deleted records in case of Not merged by source. But in case 'not merged by source' I want to update
Update ProductVariants Set Active = 0
// when not matched by source
What is the most efficient way to do this?
Necessarily use `WHEN NOT MATCHED BY SOURCE 'when you want to delete a record that is not in the target table. if you want to 'inactivate' a record this must necessarily be done when the clause 'MATCHED' adding exceptions.
If you want to keep history of the records evaluate using "Slowly Changing Dimensions", I leave you some examples that Kimball uses for this treatment of historical data.
Slowly Changing Dimensions - Part 1
Slowly Changing Dimensions - Part 2
Use the WHEN NOT MATCHED BY SOURCE clause of the MERGE with an UPDATE statement.
MERGE
ProductVariants AS TARGET
USING
#ProductVariantsTemp AS SOURCE ON (TARGET.ItemCode = SOURCE.ItemCode AND TARGET.VariantCode= SOURCE.VariantCode)
WHEN
NOT MATCHED BY TARGET THEN
INSERT (ItemCode, VariantCode, VariantDescriptionBG, VariantDescriptionEN)
VALUES (SOURCE.ItemCode, SOURCE.VariantCode, SOURCE.VariantDescriptionBG, SOURCE.VariantDescriptionEN)
WHEN
NOT MATCHED BY SOURCE THEN
UPDATE SET Active = 0
OUTPUT
INSERTED.ItemCode, INSERTED.VariantCode, GETDATE() INTO SyncLog;
Since the OUTPUT clause for the INSERTED table might return either inserted or updated records now, you can add the special column $action that will tell you the original operation as INSERT or UPDATE. Will have to change the SyncLog table to recieve this value though.
OUTPUT
INSERTED.ItemCode, INSERTED.VariantCode, GETDATE(), $action
INTO SyncLog;
I'm copying the contents of a table into another identical table. But there are already data in the destination table.
Some data in the destination table has the same code as the source table.
Is it possible to skip the duplicates and not to block the insertion for the rest of the data without it failing?
insert into [DB2].[dbo].[MAN] values([MAN],[DES])
SELECT [MAN]
,[DES]
FROM [DB1].[dbo].[MAN]
You can use NOT EXISTS :
INSERT INTO [DB2].[dbo].[MAN] ([MAN], [DES])
SELECT M.[MAN], M.[DES]
FROM [DB1].[dbo].[MAN] AS M
WHERE NOT EXISTS (SELECT 1 FROM [DB2].[dbo].[MAN] M1 WHERE M1.COL = M.COL);
You need to change the M1.COL = M.COL with your actual column name from which you can identify the duplicate values.
If you have your unique col then you can go like this.
insert into [DB2].[dbo].[MAN] values([MAN],[DES])
SELECT [MAN]
,[DES]
FROM [DB1].[dbo].[MAN] WHERE uniqueCol NOT IN (SELECT uniqueCol FROM [DB2].[dbo].[MAN])
Otherwise append few columns to get unique one and compare like that.
I have a table with CDC enabled that's throwing the following weird behavior.
In an update where one of three nullable columns already has a value [8,23|NULL|NULL] and I update only the other two columns [AlexJ, 1], CDC tracks a change against all three columns.
2018-06-22 13:55:37.763 NULL NULL NULL
2018-06-22 13:55:37.763 8,23 AlexJ 1
I use a templated query to get these data from the cdc.dbo_Tablename_CT table.
...
SELECT sys.fn_cdc_map_lsn_to_time([__$start_lsn]) AS 'ModifiedDate',
[Tags],[ModifiedBy], [IsInactive]
FROM cdc.fn_cdc_get_all_changes_dbo_Tablename
(#from_lsn, #to_lsn, N'all update old')
WHERE Id = #Id
...
How do I get around this? It's most annoying and may direct me away from using CDC, not that deploying and maintaining a CDC'ed table is a walk in the park in the best of times.
https://learn.microsoft.com/en-us/sql/relational-databases/system-tables/cdc-capture-instance-ct-transact-sql?view=sql-server-2017
It’s always going to put null for varchar(max) … the tags column.
Large Object Data Types
Columns of data type image, text, and ntext are always assigned a NULL value when _$operation = 1 or _$operation = 3. Columns of data type varbinary(max), varchar(max), or nvarchar(max) are assigned a NULL value when __$operation = 3 unless the column changed during the update. When __$operation = 1, these columns are assigned their value at the time of the delete. Computed columns that are included in a capture instance always have a value of NULL.
I have a database in SQL Server with its data. I need change a part of some columns value in some conditions.
Imagine the value as "0010020001".
002 belongs to another value in my database and whenever I want to change it to 005, I must update the previous 10-digits code to "001005001".
Actually, I need to update just a part of columns value using UPDATE statement. How can I do it (in this example)?
While everyone else is correct that if you have control of the schema you should definitely not store your data this way, this is how I would solve the issue you as you described it if I couldn't adjust the schema.
IF OBJECT_ID('tempdb..#test') IS NOT NULL
DROP TABLE #test
create table #test
(
id int,
multivaluecolumn varchar(20)
)
insert #Test
select 1,'001002001'
UNION
select 2,'002004002'
UNION
select 3,'003006003'
GO
declare #oldmiddlevalue char(3)
set #oldmiddlevalue= '002'
declare #newmiddlevalue char(3)
set #newmiddlevalue = '005'
select * from #Test
Update #Test set multivaluecolumn =left(multivaluecolumn,3) + #newmiddlevalue + right(multivaluecolumn,3)
where substring(multivaluecolumn,4,3) = #oldmiddlevalue
select * from #Test
Why dont you use CSV(comma separated values) or use any other symbol like ~ to store tha values. Once you need to update a part of it use php explode function and then update it. After your work is done, concat all the values again to get the desired string to be stored in your column.
In that case your column will have values VARCHAR like 001~002~0001
I am trying to write a Stored Procedure in SQL Server (2005) to do something that sounds simple, but is actually proving to be more difficult that I thought.
I have a table with 30 columns and 50,000 rows.
The number of records is fixed, but users can edit the fields of existing records.
To save them having to re-key repetitive data, I want to give them the ability to select a record, and specify a range of IDs to copy those details to.
The SP I'm trying to write will take 3 parameters: The source record primary key, and the lower and upper primary keys of the range of records that the data will be copied into.
Obviously the PKs of the destination records remain unchanged.
So I figured the SP needs to do a SELECT - to get all the data to be copied, and an UPDATE - to write the data into the specified destination records.
I just don't know how to store the results of the SELECT to slot them into the UPDATE.
A temp table wouldn't help - selecting from that would be just the same as selecting from the table!
What I need is a variable that is effectively a single record, so I can go something like:
#tempRECORD = SELECT * FROM SOURCETABLE WHERE ID = #sourcePK
UPDATE SOURCETABLE
SET FIELD1 = #tempRECORD.FIELD1,
FIELD2 = #tempRECORD.FIELD2,
...
FIELD30 = #tempRECORD.FIELD30
WHERE ID >= #LOWER_id AND ID <= #UPPER_id
But I don't know how, or if you even can.
I'm also open to any other clever way I haven't even thought of!
Thanks guys!
So I figured the SP needs to do a SELECT - to get all the data to be copied, and an UPDATE - to write the data into the specified destination records.
What you need is the T-SQL-specific extension to UPDATE, UPDATE ... FROM:
UPDATE T
SET
Field1 = source.Field1
, Field2 = source.Field2
, Field3 = source.Field3
FROM
(SELECT * FROM T AS source_T WHERE source_T.ID = #sourcePK) as source
WHERE
T.ID BETWEEN #LOWER_Id AND #UPPER_Id
Note that this ability to put a FROM clause in an UPDATE statement is not standard ANSI SQL, so I don't know how this would be done in other RDBMSs.
I am pretty sure this ain't the easiest way to do it, but it should work without any problems:
DECLARE #tempField1 varchar(255)
DECLARE #tempField2 varchar(255)
...
DECLARE #tempField30 varchar(255)
SELECT #tempField1 = FIELD1, #tempField2 = FIELD2, ... ,#tempField30 = FIELD30 FROM SOURCETABLE WHERE ID = #sourcePK
UPDATE SOURCETABLE
SET FIELD1 = #tempField1,
FIELD2 = #tempField2,
...
FIELD30 = #tempField30
WHERE ID >= #LOWER_id AND ID <= #UPPER_id
You would need to edit the tempField variables so that they have the right type.