This is my tables in SQL Server:
Table: Customer (CustNum is the PK and auto increments)
CustNum Firstname Lastname
123 Bob Smith
456 John Paton
789 Fred Bloggs
Table: Job (CustNum is the FK, JobNum is the PK and auto increments)
JobNum CustNum Item Desc
852 123 ABCDE Widgets
654 456 WERT More Widgets
987 789 QWE Mouse mat
I then have data coming from two csv's in the below format
Table: Customer
Firstname Lastname
Bob Smith
John Paton
Fred Bloggs
Table: Job
Item Desc
ABCDE Widgets
WERT More Widgets
QWE Mouse mat
I need to import the customers then the jobs into SQL tables but maintain referential integrity. When the Job record is created, it needs to lookup for the correct CustNum - how do I do this? I have found many examples doing this from one CS file, but not from multiple csv's. Also, there is no reliable key to tie the job rows to the customer table - I don't think I will be able to use the SSIS lookup - will I?
You can use an SSIS Package to feed your data base.
the package will look like it
Result Customer
CustNum Firstname Lastname
----------- ---------- ----------
1 Bob Smith
2 Fred Bloggs
3 John Paton
Result
JobNum CustNum Item Desc
----------- ----------- ---------- --------------------------------------------------
1 1 ABCDE Widgets
2 2 QWE Mouse mat
3 3 WERT More Widgets
How to do it:
Main task:
Add a DTS to Customer
Add a DTS to Job
Create the flow (green arrow) from Customer to Job
Customer DTS
Add a connection to Customer File
Add a connection to Customer SQL
Sort Both outputs by Firstname, Lastname
Configure the merge join task like the image
Insert and configure the conditional Split with one output called "new" and expression ISNULL(CustNum)
Insert to the Customer table
Job DTS
Insert a connection to Job file
Insert a connection to Customer table
Sort both outputs by Firstname, Lastname
Configure the join like the image:
Insert a connection to Job table
Sort both by CustNum, Item
Insert and configure the conditional Split with one output called "new" and expression ISNULL(JobNum)
Insert to the Job table
I was able to change the source data so it appears in 1 CSV file as opposed to being in 2. I was then able to use the SQL Lookup function in SSIS and it now works. Thanks for all those who posted.
Related
I am trying to create two relationships between tables in Power BI. I have a table (Observations) that contains an employee_id and a supervisor_id that I am trying to connect to my Contact table.
I need to look up the employee information in the Contact table by using employee_id in the Observation table and also look up the supervisor information in the Contact table using the supervisor_id in the Observation table.
Contact Table Observation Table
=================== =================
Employee_ID Name Position Observation_ID Employee_ID Supervisor_ID
--------------------------------------- ----------------------------------------------
123 John Doe employee 1 123 321
321 Harvey Sample supervisor 2 456 654
456 Mickey Mouse employee 3 456 654
654 Donald Duck supervisor 4 123 321
One possible way is to create two relationships between the observation and contact table:
Create a relationship between Observation Table(Employee_ID) and Contact Table(Employee_ID). The Cardinality should be many to one, and set this relationship as active.
Create a relationship between Observation Table(Supervisor_ID) and Contact Table(Employee_ID). The Cardinality should be many to one, and set this relationship as inactive.
create an inactive relationship Power BI
You will then need to use the DAX USERELATIONSHIP function when you want to use the inactive relationship in a formula. Here is a detailed explanation of how to use USERELATIONSHIP: https://www.sqlbi.com/articles/userelationship-in-calculated-columns/
Alternatively, you can duplicate the contact table and use one copy for the Employee_ID relationship and the other for the Supervisor_ID relationship. Which option is best will depend on your use case.
Hello I don't know how to do changes of my MS SQL DB to integrate it to work with new software.
The case is related with software limitations.
Our old software can write, read and work with MS SQL DB with multiple tables but the new software understand only from one table or one view table.
My question is how can I edit my MS SQL DB form 20 tables to do it to be one DB with one table with all data from 20 tables and columns without data lost?
And one last question is true about View Tables in MS SQL that they are read only for applications and software?
Why is it not a good idea to put data from all your 20 tables into one table ?
I will try to explain with an example, since I do not know your database I just think of some tables here
suppose you have a table Clients
ClientID Name Street City
1 John ChuchStreet Denver
2 Anna FlowerStreet Boston
and a table Products
ProductID Name Price
1 Mouse 10
2 Keyboard 30
3 Usb Cable 10
and table Orders
OrderID OrderNumber CLientID TotalAmount
1 123 1 10
2 345 1 20
3 678 2 30
and finally table OrderDetail
OrderDetailID OrderID ProductID Quantity
1 1 1 1
2 2 1 1
3 2 3 1
4 3 2 1
Now to put this into one table, you could do this
ID ClientName ClientStreet ClientCity OrderNumber TotalAmount ProductName ProductPrice ProductQuantity
1 John ChurchStreet Denver 123 10 Mouse 10 1
1 John ChurchStreet Denver 345 20 Mouse 10 1
2 John ChurchStreet Denver 345 20 Usb Cable 10 1
3 Anna FlowerStreet Boston 678 30 Keyboard 30 1
Now you can already see the redundancy,
you need to repeat the address of each customer, time and time again in your table
you need to repeat the ordernumber and total amount time and time again
you need to repeat the productname and price time and time again
Now suppose that John moves to another address, now you have to search for John in every row in the table, and adjust the address
Now suppose a productname changes, again you have to search all rows and update
That is lots of work, very inefficient, and guaranteed to go wrong at some point
Now I only used 4 tables in this example, can you image what will happen if you would merge 20 tables into 1 ?
And the redundancy is not your only problem, what if you want to look at a client, what row should you use ?
What if you want to look at an order, what row should you use ?
What if you want to look at a product, what row should you use ?
In this one table design, you cannot identify a single row for customer, or order anymore. That is because each row contains everyting, there is no distinct row anymore for a customer, or a product, or an order...
Merging all tables into one big table is simply not possible to maintain
Hi thank you for your answers!
I am trying to integrate old ms sql db to new IDFLOW Software.
This software understand from ms sql db but only from one table but my old db contains 18 tables....
IDFLOW understand from views, but it is not good to work with views, they are okay for card design in IDFLOW, but views are not okay to write new data in db from IDFLOW interface , because in views it is not possible to write new data in db!
And now I started to think to create one db with one table with all columns, and I completed it successfully!
Now in my sql server I am with two db, old db and new db.
Now I don't know how to export data from old db and import it to new db?
I am talking about export column2 from table1 from db1 and import it in column2 from table1 from db2 ....
and so and so ........... column2 from table2 from db1 and import it in column3 from table1 from db2 ............
and so and so .... column 5 from table18 from db1 and import it in column10 form table1 from db2.
Is it possible to do that?
Okay, but IDFLOW works only with one table...
And I asked Jolly and they say utilize your DB for one table.
IDFLOW is a software for ID CARDS .
I think it is possible to use one db with one table and all columns for employee information.
The goal is.. in IDFLOW enter all data for new employee, when you enter data from IDFLOW interface they go into MS SQL DB and all fields from DB are in card design configured, and when you select records for one employee from IDFLOW interface (from db) you can print ID CARD with all data for employee.
The question is how to migrate old records from old db in new.
It is not big database, it is only 6 GB from 2006.
And we use it only for printing id cards.
Scenarios:
Table 1
Name ID dept
John 112 Fin
Mary 113 Act
Table 2
Name email
John John#gmail.com
Mary Max#gmail.com
Table 3
Name Supervisor
John Kelly
Mary Adam
Table 4
Namevalue Salary
John 1000
Mary 1000
Expected Output:
I would like to know the occurrence of John and Mary in all these tables. There are more than 20 tables in difference database.
Firstly, specify the master table according to data quality, use this table as base and then use left joins to concat with other tables.
But, I think, there is a design problem in the database so maybe creating keys for each user can be useful.
For example, name + surname + User Something to make records unique then use joins otherwise all records will duplicate.
The short version is I'm trying to map from a flat table to a new set of tables with a stored procedure.
The long version: I want to SELECT records from an existing table, and then for each record INSERT into a new set of tables (most columns will go into one table, but some will go to others and be related back to this new table).
I'm a little new to stored procedures and T-SQL. I haven't been able to find anything particularly clear on this subject.
It would appear I want to something along the lines of
INSERT INTO [dbo].[MyNewTable] (col1, col2, col3)
SELECT
OldCol1, OldCol2, OldCol3
FROM
[dbo].[MyOldTable]
But I'm uncertain how to get that to save related records since I'm splitting it into multiple tables. I'll also need to manipulate some of the data from the old columns before it will fit into the new columns.
Thanks
Example data
MyOldTable
Id | Year | Make | Model | Customer Name
572 | 2001 | Ford | Focus | Bobby Smith
782 | 2015 | Ford | Mustang | Bobby Smith
Into (with no worries about duplicate customers or retaining old Ids):
MyNewCarTable
Id | Year | Make | Model
1 | 2001 | Ford | Focus
2 | 2015 | Ford | Mustang
MyNewCustomerTable
Id | FirstName | LastName | CarId
1 | Bobby | Smith | 1
2 | Bobby | Smith | 2
I would say you have your OldTable Id to preserve in new table till you process data.
I assume you create an Identity column Id on your MyNewCarTable
INSERT INTO MyNewCarTable (OldId, Year, Make, Model)
SELECT Id, Year, Make, Model FROM MyOldTable
Then, join the new table and above table to insert into your second table. I assume your MyNewCustomerTable also has Id column with Identity enabled.
INSERT INTO MyNewCustomerTable (CustomerName, CarId)
SELECT CustomerName, new.Id
FROM MyOldTable old
JOIN MyNewCarTable new ON old.Id = new.OldId
Note: I have not applied Split of Customer Name to First Name and
Last Name as I was unsure about existing data.
If you don't want your OldId in MyNewCarTable, you can DELETE it
ALTER TABLE MyNewCarTable DROP COLUMN OldId
You are missing a step in your normalization. You do not need to duplicate your customer information per vehicle. You need three tables for 4th Normal form. This will reduce storage size and more importantly allow an update to the customer data to take place in one location.
Customer
CustomerID
FirstName
LastName
Car
CarID
Make
Model
Year
CustomerCar
CustomerCarID
CarID
CustomerID
DatePurchaed
This way you can have multiple owners per car, multiple cars per owner and only one record needs to be updated per car and or customer...4th Normal Form.
If I am reading this correctly, you want to take each row from table 1, and create a new record into table A using some of that row data, and then data from the same original row into Table B, Table C but referencing back to Table A again?
If that's the case, you will create TableA with an Identity and make thats the PK.
Insert the required column data into that table and use the #IDENTITY to retrieve the last identity value, then you will insert the remaining data from the original table into the other tables, TableB, TableC, etc. and use the identity you retrieved from TableA as the FK in the other tables.
By Example:
Table 1 has columns col1, col2, col3, col4, col5
Table A has TabAID, col1, col2
Table B has TabBID, TabAID, col3
TableC has TabCID, TabAID, col4
When the first row is read, the values for col1 & col2 are inserted into TableA.
The Identity is captured from that row inserted, and then value for col3 AND the identity are entered into TableB, and then value for col4 AND the identity are entered into TableC.
This is a standard data migration technique for normalizing data.
Hope this assists,
I have an employee table with multiple entries for the same person.
Using their SIN number you are able to determine if they are the same person.
Example:
Id Name SIN
1 John Smith 1234
2 John Smith 1234
3 Jess Jones 4321
I want to be able to copy everyone in this table, into a new table.
I want to create a new column (UserKey [GUID]) that is unique to the user who is in the table multiple times.
I want to use this new UserKey [GUID] instead of the SIN number.
Example:
Id Name UserKey (Guid)
1 John Smith 1234DFKJ2328LJFD
2 John Smith 1234DFKJ2328LJFD
3 Jess Jones 9543SLDFJ28EKJFF
I have no idea on how to approach this query. Any help would be great.
I am using MS SQL Server 2008.
Thanks.
You can just create a temp table, mapping SIN to a new GUID. Then you can join the original table with the mapping table, and create the new table from that.
# Create a table called #temp with the mappings SID => new GUID
SELECT SIN, newid() UserKey INTO #temp FROM (SELECT DISTINCT SIN FROM Table1) a
# ...and join that and the original table to a new table.
SELECT id, name, userkey
INTO NewTable
FROM Table1 t1
JOIN #temp t2
ON t1.SIN = t2.SIN
SQLFiddle here.