I'm useing SQL Server 2008 and I have a small script to split a row of data into several different columns. My question is how do I get the script to effect all the rows in the column? Right now the data row is just a string literal. I'm wondering if there is another way to wright this so I do not have to keep replaceing the DATA ROW EXAMPLE line with each rows data. The script is below.
DECLARE #TEXTINPUT VARCHAR(120)
SET #TEXTINPUT = 'DATA ROW EXAMPLE 12 89564 DATA ROW EXAMPLE'
SELECT #TEXTINPUT AS TEXTINPUT
SELECT SUBSTRING(#TEXTINPUT,1,4) AS UNIT,
SUBSTRING(#TEXTINPUT,6,3) AS NAME,
SUBSTRING(#TEXTINPUT,10,7) AS ACCOUNT,
SUBSTRING(#TEXTINPUT,18,2) AS NOTE
INSERT INTO ExampleTable
(UNIT, NAME, ACCOUNT, NOTE)
VALUES (SUBSTRING(#TEXTINPUT,1,4), SUBSTRING(#TEXTINPUT,6,3), SUBSTRING(#TEXTINPUT,10,7),
SUBSTRING(#TEXTINPUT,18,2))
GO
I assume below the table with source data is called SourceTable and the column you are splitting is called SourceCol
INSERT INTO ExampleTable
(UNIT, NAME, ACCOUNT, NOTE)
SELECT SUBSTRING(SourceCol,1,1) AS UNIT,
SUBSTRING(SourceCol,3,9) AS NAME,
SUBSTRING(SourceCol,13,6) AS ACCOUNT,
SUBSTRING(SourceCol,20,3) AS NOTE
FROM SourceTable
Related
I queried some data from table A(Source) based on certain condition and insert into temp table(Destination) before upsert into Crm.
If data already exist in Crm I dont want to query the data from table A and insert into temp table(I want this table to be empty) unless there is an update in that data or new data was created. So basically I want to query only new data or if there any modified data from table A which already existed in Crm. At the moment my data flow is like this.
clear temp table - delete sql statement
Query from source table A and insert into temp table.
From temp table insert into CRM using script component.
In source table A I have audit columns: createdOn and modifiedOn.
I found one way to do this. SSIS DataFlow - copy only changed and new records but no really clear on how to do so.
What is the best and simple way to achieve this.
The link you posted is basically saying to stage everything and use a MERGE to update your table (essentially an UPDATE/INSERT).
The only way I can really think of to make your process quicker (to a significant degree) by partially selecting from table A would be to add a "last updated" timestamp to table A and enforcing that it will always be up to date.
One way to do this is with a trigger; see here for an example.
You could then select based on that timestamp, perhaps keeping a record of the last timestamp used each time you run the SSIS package, and then adding a margin of safety to that.
Edit: I just saw that you already have a modifiedOn column, so you could use that as described above.
Examples:
There are a few different ways you could do it:
ONE
Include the modifiedOn column on in your final destination table.
You can then build a dynamic query for your data flow source in a SSIS string variable, something like:
"SELECT * FROM [table A] WHERE modifiedOn >= DATEADD(DAY, -1, '" + #[User::MaxModifiedOnDate] + "')"
#[User::MaxModifiedOnDate] (string variable) would come from an Execute SQL Task, where you would write the result of the following query to it:
SELECT FORMAT(CAST(MAX(modifiedOn) AS date), 'yyyy-MM-dd') MaxModifiedOnDate FROM DestinationTable
The DATEADD part, as well as the CAST to a certain degree, represent your margin of safety.
TWO
If this isn't an option, you could keep a data load history table that would tell you when you need to load from, e.g.:
CREATE TABLE DataLoadHistory
(
DataLoadID int PRIMARY KEY IDENTITY
, DataLoadStart datetime NOT NULL
, DataLoadEnd datetime
, Success bit NOT NULL
)
You would begin each data load with this (Execute SQL Task):
CREATE PROCEDURE BeginDataLoad
#DataLoadID int OUTPUT
AS
INSERT INTO DataLoadHistory
(
DataLoadStart
, Success
)
VALUES
(
GETDATE()
, 0
)
SELECT #DataLoadID = SCOPE_IDENTITY()
You would store the returned DataLoadID in a SSIS integer variable, and use it when the data load is complete as follows:
CREATE PROCEDURE DataLoadComplete
#DataLoadID int
AS
UPDATE DataLoadHistory
SET
DataLoadEnd = GETDATE()
, Success = 1
WHERE DataLoadID = #DataLoadID
When it comes to building your query for table A, you would do it the same way as before (with the dynamically generated SQL query), except MaxModifiedOnDate would come from the following query:
SELECT FORMAT(CAST(MAX(DataLoadStart) AS date), 'yyyy-MM-dd') MaxModifiedOnDate FROM DataLoadHistory WHERE Success = 1
So the DataLoadHistory table, rather than your destination table.
Note that this would fail on the first run, as there'd be no successful entries on the history table, so you'd need you insert a dummy record, or find some other way around it.
THREE
I've seen it done a lot where, say your data load is running every day, you would just stage the last 7 days, or something like that, some margin of safety that you're pretty sure will never be passed (because the process is being monitored for failures).
It's not my preferred option, but it is simple, and can work if you're confident in how well the process is being monitored.
I am currenly creating a SSIS package that selects the top 2000 entries from one table and insert those entries into another table. How can I update those entries in the original table (my select contains a condition whereby I check where the status is still empty) so in my next run of the SSIS package they won't be imported?
**EDIT
As I can't give the DLL due to security reasons, to sketch my scenario, I have TableA and TableB who share the same structure:
Index Number (Primary Key, Indexed, Auto Increment)
Name (varchar)
Surname (varchar)
Status (varchar)
Type (varchar)
Import Date (date)
Using an OLE DB Source within my Data Flow Task, the following query is used to determine which cases I should import from TableA to TableB (an OLE DB Destination):
SELECT TOP(2000) *
FROM TableA
WHERE Status = ''
AND [Type] = 'New Case'
ORDER BY IndexNumber ASC
Now those cases imported from TableA to TableB, I want to update the cases in Table A to not import them again. The data in TableB is often moved to another database, so I can't use that for comparison.
I suggest you use this pattern:
FIRST update the source table and mark 1000 records as 'ready to extract', i.e. set them to 1
Next, select all records from your source table that are ready to extract
Lastly, if the extract was succesful, update the 1's (ready to extract) to 2's (extracted OK)
Here is how I would do it:
I would add a Execute SQL Task at the end of the control flow to update these first 2000 records in your original table to some other status (status = 3 or something). That way, since you already have a null check in your query, it would not select those initial 2000 records in your next run.
I am very new to SQL and SQL server, would appreciate any help with the following problem.
I am trying to update a share price table with new prices.
The table has three columns: share code, date, price.
The share code + date = PK
As you can imagine, if you have thousands of share codes and 10 years' data for each, the table can get very big. So I have created a separate table called a share ID table, and use a share ID instead in the first table (I was reliably informed this would speed up the query, as searching by integer is faster than string).
So, to summarise, I have two tables as follows:
Table 1 = Share_code_ID (int), Date, Price
Table 2 = Share_code_ID (int), Share_name (string)
So let's say I want to update the table/s with today's price for share ZZZ. I need to:
Look for the Share_code_ID corresponding to 'ZZZ' in table 2
If it is found, update table 1 with the new price for that date, using the Share_code_ID I just found
If the Share_code_ID is not found, update both tables
Let's ignore for now how the Share_code_ID is generated for a new code, I'll worry about that later.
I'm trying to use a merge query loosely based on the following structure, but have no idea what I am doing:
MERGE INTO [Table 1]
USING (VALUES (1,23-May-2013,1000)) AS SOURCE (Share_code_ID,Date,Price)
{ SEEMS LIKE THERE SHOULD BE AN INNER JOIN HERE OR SOMETHING }
ON Table 2 = 'ZZZ'
WHEN MATCHED THEN UPDATE SET Table 1.Price = 1000
WHEN NOT MATCHED THEN INSERT { TO BOTH TABLES }
Any help would be appreciated.
http://msdn.microsoft.com/library/bb510625(v=sql.100).aspx
You use Table1 for target table and Table2 for source table
You want to do action, when given ID is not found in Table2 - in the source table
In the documentation, that you had read already, that corresponds to the clause
WHEN NOT MATCHED BY SOURCE ... THEN <merge_matched>
and the latter corresponds to
<merge_matched>::=
{ UPDATE SET <set_clause> | DELETE }
Ergo, you cannot insert into source-table there.
You could use triggers for auto-insertion, when you insert something in Table1, but that will not be able to insert proper Shared_Name - trigger just won't know it.
So you have two options i guess.
1) make T-SQL code block - look for Stored Procedures. I think there also is a construct to execute anonymous code block in MS SQ, like EXECUTE BLOCK command in Firebird SQL Server, but i don't know it for sure.
2) create updatable SQL VIEW, joining Table1 and Table2 to show last most current date, so that when you insert a row in this view the view's on-insert trigger would actually insert rows to both tables. And when you would update the data in the view, the on-update trigger would modify the data.
So I'm trying to copy some data from database table to another. The problem is though, the target database table has 2 new columns that are required. I wanted to use the export/import wizard on SQL Server Management Studio but if I use that I will need to write a query for each table and I can only execute 1 query at a time. Was wondering if there are a more efficient way of doing it.
Here's an example of 1 table:
dbase1.dbo.Appointment { id, name, description, createdate }
dbase2.dbo.Appointment { id, name, description, createdate, auditby, auditat}
I have a total of 8 tables with those 2 additional columns. and most of them are related to each other via fk, so I wanted to use the wizard as it figures out which table gets inserted first. The problem with that is, it only works if I do a "copy data from one or more tables " and not the "write a query to specify data" (I use this to populate those two new columns).
I've been doing this very slow process in copying data as I'm using MVC Code First for my application and I dont have access to the server to be able to drop and create the table at my leisure. So I have to resort to this to maintain the data that I already have.
An idea: temporarily disable the foreign key constraints in the destination database. Then it doesn't matter what order you run your inserts. In order to populate the two new and required columns, you just need to pick some stock values to put in there (since obviously these rows initially are not subject to initial auditing). For example:
INSERT dbase2.dbo.appointment
(id, name, description, createdate, auditby, auditat)
SELECT id, name, description, createdate,
auditby = 'me', auditat = GETDATE()
FROM dbo.appointment;
Since it seems the challenge is merely that the destination requires columns that aren't in the source, and that you need to determine what should be populated in these audit columns, this seems to solve multiple problems at once. You just need to figure out what to put in there instead of 'me' and GETDATE().
(To get the wizard to pull these 8 tables for you, you might be able to create a view similar to the select portion of the above query, but that's more work and it won't see the underlying FK constraints to generate them in the right order anyway.)
Write the sql query for each of the insert processes in the order you want it. That would be the simplest approach.
Set the Default values for these two columns
Like for AuditAt - Default Date i.e. GetDate()
For AuditBy - The Person ID/Name
Now, you can Insert into these tables without entering for these two columns
I need to create a new table called “customer” that include some of columns from the “user table”, and also “project table”. I built my suppliers table with specific column names and I need to fill its column by using data of the other tables. Finally I am trying to finish; when user create a new account and project, the customer table automatically fill with some of other two tables varieties with different column names.
INFO: I have three different user types such as “suppliers”, “costumers”, “managers”. I am holding their information(include user types) in one table called users.
Use the following query as an example and write a query to insert the rows to destination table from source table.
Ex:-
INSERT INTO TestTable (FirstName, LastName)
SELECT FirstName, LastName
FROM Person.Contact
WHERE EmailPromotion = 2
Note: Use Join in the select query to join two tables
The 1st step would be to couple the data from the different tables using a table join command. If you can create a search result that matched your new table, then creating the table is simple a call to the below.
Create table CUSTOMER as (Select ...)
"when user create a new account and project.." this is something you plan on doing at run time in your application and not something you need to collate using sql at this point?