export multiple table with ssis - sql-server

I have 2 tables are like this (Table1 and Table2)
ID NAME No Addrress Notes
------------ ----------------------------
1 John 111 USA Done
2 Steve 222 Brazil Done
Now I want to create a SSIS package which will create a csv file like:
Table1;ID;NAME
Table2;No;Addrress;Notes
"Detail1";"1";"John";"2";"Steve"
"Detail2";"111";"USA";"Done";"222";"Brazil";"Done"
Can we achieve the same output? I have searched on google but haven't found any solution.
Please help ....

You can create a script task to generate a CSV file for you which can handle your issue:
You can try this:
SqlConnection sqlCon = new SqlConnection("Server=localhost;Initial Catalog=LegOgSpass;Integrated Security=SSPI;Application Name=SQLNCLI11.1");
sqlCon.Open();
SqlCommand sqlCmd = new SqlCommand(#"Select ID,Name from dbo.Table1", sqlCon);
SqlDataReader reader = sqlCmd.ExecuteReader();
string fullpath = #"C:\Users\thoje\Desktop\stack\New folder\table1.csv";
StreamWriter sw = new StreamWriter(fullpath);
object[] output = new object[reader.FieldCount];
for (int i = 0; i < reader.FieldCount; i++)
output[i] = reader.GetName(i);
sw.WriteLine(#"Table1;"+string.Join(";", output));
List<object> values = new List<object>();
while (reader.Read())
{
reader.GetValues(output);
values.Add($"\"{output[0]}\"");
values.Add($"\"{output[1]}\"");
}
sw.WriteLine(#"""Detail1"";"+ string.Join(";", values));
sw.Flush();
sw.Close();
reader.Close();
sqlCon.Close();
Dts.TaskResult = (int)ScriptResults.Success;
Result:

You really should put in your question what you have tried so far, it helps out a lot and makes it more fun to help people.
The two ways I can think of in t-sql to solve this still need you to specify in your code what your column names are. You can get around this with using dynamic SQL and creating a view that spits out data in the same fashion for all the tables you need.
If SSIS is more your thing you could use the dynamic approach with BIML.
--Option 1 (SQL Server 2008 R2 and later)
with Table1 AS (
SELECT * FROM (values(1,'John'),(2,'Steve')) AS x(ID,NAME)
)
,Table2 AS (
SELECT * FROM (values(111,'USA','Done'),(222,'Brazil','Done'))AS y(No,Addrress,Notes)
)
SELECT '"Detail1"'+ CAST(foo as VARCHAR(4000))
FROM (
SELECT ';"' + CAST(ID AS VARCHAR(4))+'";"' + [NAME] +'"' FROM Table1 FOR XML PATH('')
) AS bar(foo)
UNION ALL
SELECT '"Detail2"'+ CAST(foo as VARCHAR(4000))
FROM (
SELECT ';"' + CAST([No] AS VARCHAR(4))+'";"' + [Addrress] +'";"' + [Notes] +'"' FROM Table2 FOR XML PATH('')
) AS bar(foo)
--Option 2 (SQL Server 2017 and later)
with Table1 AS (
SELECT * FROM (values(1,'John'),(2,'Steve')) AS x(ID,NAME)
)
,Table2 AS (
SELECT * FROM (values(111,'USA','Done'),(222,'Brazil','Done'))AS y(No,Addrress,Notes)
)
SELECT '"Detail1";' + STRING_AGG('"'+CAST(ID AS varchar(4))+'";"'+[NAME]+'"',';') FROM Table1
UNION ALL
SELECT '"Detail2";' + STRING_AGG('"'+CAST([No] AS varchar(4))+'";"'+[Addrress]+'";'+'"'+[Notes]+'"',';') FROM Table2
;

Related

Incorrect syntax near "Exists" … - Convert Query from MySQL to SQL Server

My application uses a query that is working fine using a MySQL/MariaDB-Database.
I did modify my application to be more flexible and work with Microsoft SQL Server too.
Unfortunately the following query does NOT work using a SQL Server database:
select
p.PrinterGUID,
(exists (select 1
from computerdefaultprinter cdp
where cdp.PrinterGUID = p.PrinterGUID and
cdp.ComputerGUID = '5bec3779-b002-46ba-97c4-19158c13001f')
) as is_computer_default,
(exists (select 1
from userdefaultprinter udp
where udp.PrinterGUID = p.PrinterGUID and
udp.UserGUID = 'd3cf699b-8d71-4dbc-92f3-402950042054')
) as is_user_default
from
((select cm.PrinterGUID
from computermapping cm
where cm.ComputerGUID = '5bec3779-b002-46ba-97c4-19158c13001f'
) union -- to remove duplicates
(select PrinterGUID
from usermapping um
where um.UserGUID = 'd3cf699b-8d71-4dbc-92f3-402950042054')) p;
Running this query throws an error
Incorrect syntax near the keyword 'exists'
Microsoft SQL Server Management Studio returns the following in German:
I have created a SQL Fiddle with some example data: SQL-Fiddle
If necessary, more background-information is available here: UNION 2 Select-queries with computed columns
Is it possible to modify this query to work in both MySQL and SQL Server?
Thank you very much!
The literal translation of your query would be the following:
select
p.PrinterGUID,
CASE WHEN exists (select 1
from computerdefaultprinter cdp
where cdp.PrinterGUID = p.PrinterGUID and
cdp.ComputerGUID = '5bec3779-b002-46ba-97c4-19158c13001f')
THEN 1 ELSE 0 END as is_computer_default,
CASE WHEN exists (select 1
from userdefaultprinter udp
where udp.PrinterGUID = p.PrinterGUID and
udp.UserGUID = 'd3cf699b-8d71-4dbc-92f3-402950042054')
THEN 1 ELSE 0 END as is_user_default
from (select cm.PrinterGUID
from computermapping cm
where cm.ComputerGUID = '5bec3779-b002-46ba-97c4-19158c13001f'
union -- to remove duplicates
select PrinterGUID
from usermapping um
where um.UserGUID = 'd3cf699b-8d71-4dbc-92f3-402950042054') p;
Notice the use of a CASE expressions to determine the value of the column for when the EXISTS evaluates to true or not.
Just try out the following query. You don't need to use exists like this in sql server. Instead filter out the rows at the end using is_computer_default or is_user_default
select p.PrinterGUID,
(select 1
from computerdefaultprinter cdp
where cdp.PrinterGUID = p.PrinterGUID and
cdp.ComputerGUID = '5bec3779-b002-46ba-97c4-19158c13001f')
as is_computer_default,
(select 1
from userdefaultprinter udp
where udp.PrinterGUID = p.PrinterGUID AND
udp.UserGUID = 'd3cf699b-8d71-4dbc-92f3-402950042054'
) as is_user_default
from ((select cm.PrinterGUID
from computermapping cm
where cm.ComputerGUID = '5bec3779-b002-46ba-97c4-19158c13001f'
) union -- to remove duplicates
(select PrinterGUID
from usermapping um
where um.UserGUID = 'd3cf699b-8d71-4dbc-92f3-402950042054'
)
) p;

Concatenating rows in a multi-column format in T-SQL [duplicate]

This question already has answers here:
Simulating group_concat MySQL function in Microsoft SQL Server 2005?
(12 answers)
Closed 6 years ago.
If there is a table called employee
EmpID EmpName
---------- -------------
1 Mary
1 John
1 Sam
2 Alaina
2 Edward
Result I need in this format:
EmpID EmpName
---------- -------------
1 Mary, John, Sam
2 Alaina, Edward
Q: this record is in same Employee table. I have almost no experience using UDFs, stored procedures, I need to be done this thing through query.Is this possible without using UDFs, SP's.
FOR XML PATH trick and article
CLR User defined aggregate
for sql server prior version 2005 - temporary tables
An example of #1
DECLARE #t TABLE (EmpId INT, EmpName VARCHAR(100))
INSERT #t VALUES
(1, 'Mary'),(1, 'John'),(1, 'Sam'),(2, 'Alaina'),(2, 'Edward')
SELECT distinct
EmpId,
(
SELECT EmpName+','
FROM #t t2
WHERE t2.EmpId = t1.EmpId
FOR XML PATH('')
) Concatenated
FROM #t t1
How to strip the final comma - is on your own
A CLR aggregate c# code for #2
using System;
using System.Collections.Generic;
using System.Data.SqlTypes;
using System.Text;
using Microsoft.SqlServer.Server;
using System.IO;
namespace DatabaseAssembly
{
[Serializable]
[SqlUserDefinedAggregate(Format.UserDefined,
IsInvariantToNulls = true,
IsInvariantToDuplicates = true,
IsInvariantToOrder = true,
MaxByteSize = -1)]
public struct StringJoin : IBinarySerialize
{
private Dictionary<string, string> AggregationList
{
get
{
if (_list == null)
_list = new Dictionary<string, string>();
return _list;
}
}
private Dictionary<string, string> _list;
public void Init()
{
}
public void Accumulate(SqlString Value)
{
if (!Value.IsNull)
AggregationList[Value.Value.ToLowerInvariant()] = Value.Value;
}
public void Merge(StringJoin Group)
{
foreach (var key in Group.AggregationList.Keys)
AggregationList[key] = Group.AggregationList[key];
}
public SqlChars Terminate()
{
var sb = new StringBuilder();
foreach (var value in AggregationList.Values)
sb.Append(value);
return new SqlChars(sb.ToString());
}
#region IBinarySerialize Members
public void Read(System.IO.BinaryReader r)
{
try
{
while (true)
AggregationList[r.ReadString()] = r.ReadString();
}
catch (EndOfStreamException)
{
}
}
public void Write(System.IO.BinaryWriter w)
{
foreach (var key in AggregationList.Keys)
{
w.Write(key);
w.Write(AggregationList[key]);
}
}
#endregion
}
}
The chosen answer from #OlegDok's may return the correct result. But the performance can be terrible. This test scenario will illustrate it.
Creation of a temp table:
CREATE table #temp (EmpId INT, EmpName VARCHAR(100))
;WITH N(N)AS
(SELECT 1 FROM(VALUES(1),(1),(1),(1),(1),(1),(1),(1),(1),(1))M(N)),
tally(N)AS(SELECT ROW_NUMBER()OVER(ORDER BY N.N)FROM N,N a,N b,N c,N d,N e,N f)
INSERT #temp
SELECT EmpId, EmpName FROM (values(1, 'Mary'),(1, 'John'),(1, 'Sam')) x(EmpId, EmpName)
CROSS APPLY
(SELECT top 2000 N FROM tally) y
UNION ALL
SELECT EmpId, EmpName FROM (values(2, 'Alaina'),(2, 'Edward')) x(EmpId, EmpName)
CROSS APPLY
(SELECT top 2000 N FROM tally) y
This is only 10.000 rows. But lots of identical EmpId.
This query in Oleg's answer took 64 seconds on my database.
SELECT distinct
EmpId,
(
SELECT EmpName+','
FROM #temp t2
WHERE t2.EmpId = t1.EmpId
FOR XML PATH('')
) Concatenated
FROM #temp t1
Distinct is not the correct way of cleaning up rows in this situation.
To avoid this cartesian join, reduce the initial number of IDs before joining like this.
This is the correct way of handling this:
;WITH CTE as
(
SELECT distinct EmpId
FROM #temp
)
SELECT
EmpId,
STUFF((
SELECT ','+EmpName
FROM #temp t2
WHERE t2.EmpId = t1.EmpId
FOR XML PATH('')
), 1,1,'') Concatenated
FROM CTE t1
This takes less than 1 second
I think there is no GROUP_CONCAT function in MSSQL. This article shows different ways of concactenating row values.
Concatenating values when the number of items is small and known upfront
SELECT CategoryId,
MAX( CASE seq WHEN 1 THEN ProductName ELSE '' END ) + ', ' +
MAX( CASE seq WHEN 2 THEN ProductName ELSE '' END ) + ', ' +
MAX( CASE seq WHEN 3 THEN ProductName ELSE '' END ) + ', ' +
MAX( CASE seq WHEN 4 THEN ProductName ELSE '' END )
FROM ( SELECT p1.CategoryId, p1.ProductName,
( SELECT COUNT(*)
FROM Northwind.dbo.Products p2
WHERE p2.CategoryId = p1.CategoryId
AND p2.ProductName <= p1.ProductName )
FROM Northwind.dbo.Products p1 ) D ( CategoryId, ProductName, seq )
GROUP BY CategoryId ;
More ways on this link.
This is the solution for the example given in the beginning:
SELECT DISTINCT emp_name,
STUFF(
(SELECT ', ' + RTRIM(proj_id)
FROM project_members AS t1
WHERE t1.emp_name = t2.emp_name
FOR XML PATH (''))
, 1, 1, '')
FROM project_members t2

How to make a Select from DB where DB is from a Table

I have a .NET project with 3 databases (Common, Akten_DE, Akten_IT). All 3 databases are on same SQL Server. In databases Akten_DE and Akten_IT I have a table Akt where I store acts from DE or IT. But in database Common I have the table AktComm where I store all acts from both database. In table AktComm in Common I also have a column DB where I store acts database name.
Table Akt in Akten_DE:
|ID | ...
|AKT140122 | ...
|AKT140131 | ...
Table Akt in Akten_IT:
|ID | ...
|AKT140127 | ...
|AKT140162 | ...
Table AktComm in Common:
|ID |DB |
----------------------
|AKT140122 |Akten_DE |
|AKT140127 |Akten_IT |
|AKT140131 |Akten_DE |
|AKT140162 |Akten_IT |
----------------------
The question is how can I write a select in the Common database but take the data from Akten_IT or from Akten_DE depending on the value of Common.dbo.AktComm.DB.
Something like this:
SELECT akt.*
FROM AktComm comAkt
INNER JOIN "comAkt.DB".dbo.Akt akt ON comAkt.ID = akt.ID
Can anybody help me?
Thanks.
I am assuming the table Akt has the same schema in each database.
The basic query to do this is a simple join across database and union together the results:
SELECT co.ID, de.*
FROM AktComm co
JOIN AKTEN_DE.dbo.Akt de
ON co.ID = de.ID collate database_default
AND co.DB = 'Akten_DE'
UNION ALL
SELECT co.ID, it.*
FROM AktComm co
JOIN AKTEN_DE.dbo.Akt it
ON co.ID = it.ID collate database_default
AND co.DB = 'Akten_IT'
Of course, this is not very flexible as when you add new databases you need to refactor your query. The dynamic sql below will query the databases if new ones are added. Do note that I don't filter on DB because this would introduce a SQL injection leak and in the grand scheme of things the performance difference will be marginal if the AktComm table is not partitioned on the DB field.
declare #sql nvarchar(max)
;WITH dbs as (select distinct DB from AktComm where db_id(DB) is not null)
select #sql = isnull(#sql + N' UNION ALL', N'') + N'
SELECT co.ID, ' + QUOTENAME(DB) + N'.*
FROM AktComm co
JOIN ' + QUOTENAME(DB) + N'.dbo.Akt de
ON co.ID = de.ID collate database_default'
from dbs
print (#sql)
exec (#sql)
You can't do that easily - you'd have to do a comparison against the Common.AktComm.DB column and then execute on or the other SELECT statement:
DECLARE #DB VARCHAR(20)
SELECT #DB = AktComm.DB
FROM Common
WHERE ID = akt.ID
IF (#DB = 'Akten_DE') THEN
SELECT akt.*
FROM AktComm comAkt
INNER JOIN Akten_DE akt ON comAkt.ID = akt.ID
END
IF (#DB = 'Akten_IT') THEN
SELECT akt.*
FROM AktComm comAkt
INNER JOIN Akten_IT akt ON comAkt.ID = akt.ID
END
You cannot "parametrize" database, table or column names with parameter values or values from a database column.
This will work (though it may be too slow for your needs, depending on the amount of data etc.) for your example, though it is probably untenable if your real world problem has more than two or three databases.
SELECT
COALESCE(AKTde.ID, aktIT.ID) AS ID
FROM
AktComm comAkt
LEFT JOIN AKTEN_DE.dbo.Akt aktDE
ON comAkt.ID = aktDE.ID
AND comAkt.DB = 'Akten_DE'
LEFT JOIN AKTEN_IT.dbo.Akt aktIT
ON comAkt.ID = aktIT.ID
AND comAkt.DB = 'Akten_IT'

Sql Select from another table (loop?)

My SQL skills aren't great hence the post.
I'm trying to get all the contact names based on a company out.
For example I have two statements:
Select Id, CompanyName, Address From Clients
Select ClientId, ContactName From Contacts
You may have many contacts to a single client
Result: (I need all the contact names in a single column)
ContactName Company Address
----------------------------------------
Johh, Steve 123 Comp 12345 Address
David,Mike, Sarah 44 Comp 111 Address
A working example would be very much appreciated.
SELECT DISTINCT (
SELECT ISNULL(ct.ContactName, '') + ', '
FROM dbo.Clients cl JOIN dbo.Contacts ct ON cl.Id = ct.ClientId
WHERE cl.ID = cl2.Id
FOR XML PATH('')) AS ContactName, CAST(cl2.Id AS nvarchar(7)) + ' ' + cl2.CompanyName AS Company, Address
FROM dbo.Clients cl2
ORDER BY 2
Demo on SQLFiddle
Firstly build all the Contact Names for a Company into a Single Column. Assuming the database to be SQL Server, I'm using a Common Table Expression to store the single column contact list. Once the CTE is built, join it with the Clients table to get the ContactNames. FOR XML is used to concatenate rows.
WITH CTEContactList(ClientID,ContactNames)
AS
(
SELECT c1.ClientID,
Names = SUBSTRING(( SELECT ', ' + c2.ContactName
FROM Contacts c2
WHERE c1.ClientID = c2.ClientID
FOR XML PATH ('')),3,8000 ))
FROM Contacts c1
GROUP BY c1.ClientID
)
SELECT
cl.ID,
cl.CompanyName,
cl.Address,
ctelist.ContactNames
FROM Clients cl
INNER JOIN CTEContactList ctelist
ON cl.ID = cteList.ClientID
Sounds like you need to do a table join.
Example: two tables here
1. Person
2. Orders
Query:
SELECT
Persons.LastName, Persons.FirstName, Orders.OrderNo
FROM Persons
INNER JOIN Orders ON Persons.P_Id = Orders.P_Id
ORDER BY Persons.LastName
You didn't specify your DBMS, so I'm assuming PostgreSQL:
select string_agg(ct.contactName, ', '), cl.companyname, cl.address
from contacts ct
join clients cl on cl.id = ct.clientId
group by cl.companyname, cl.address

SQL Server - IN clause with multiple fields

Is it possible to include in a IN clause multiple fields? Something like the following:
select * from user
where code, userType in ( select code, userType from userType )
I'm using ms sql server 2008
I know this can be achieved with joins and exists, I just wanted to know if it could just be done with the IN clause.
Not the way you have posted. You can only return a single field or type for IN to work.
From MSDN (IN):
test_expression [ NOT ] IN
( subquery | expression [ ,...n ]
)
subquery - Is a subquery that has a result set of one column.
This column must have the same data type as test_expression.
expression[ ,... n ] - Is a list of expressions to test for a match.
All expressions must be of the same type as
test_expression.
Instead of IN, you could use a JOIN using the two fields:
SELECT U.*
FROM user U
INNER JOIN userType UT
ON U.code = UT.code
AND U.userType = UT.userType
You could use a form like this:
select * from user u
where exists (select 1 from userType ut
where u.code = ut.code
and u.userType = ut.userType)
Only with something horrific, like
select * from user
where (code + userType) in ( select code + userType from userType )
Then you have to manage nulls and concatenating numbers rather than adding them, and casting, and a code of 12 and a usertype of 3 vs a code of 1 and a usertype of 23, and...
..which means you start heading into perhaps something like:
--if your SQLS supports CONCAT
select * from user
where CONCAT(code, CHAR(9), userType) in ( select CONCAT(code, CHAR(9), userType) from ... )
--if no concat
select * from user
where COALESCE(code, 'no code') + CHAR(9) + userType in (
select COALESCE(code, 'no code') + CHAR(9) + userType from ...
)
CONCAT will do a string concatenation of most things, and won't zip the whole output to NULL if one element is NULL. If you don't have CONCAT then you'll string concat using + but anything that might be null will need a COALESCE/ISNULL around it.. And in either case you'll need something like CHAR(9) (a tab) between the fields to prevent them mixing.. The thing between the fields should be southing that is not naturally present in the data..
Tis a shame SQLS doesn't support this, that Oracle does:
where (code, userType) in ( select code, userType from userType )
but it's probably not worth switching DB for; I'd use EXISTS or a JOIN to achieve a multi column filter
So there ya go: a solution that doesn't use joins or exists.. and a bunch of reasons why you shouldn't use it ;)
How about this instead:
SELECT user.* FROM user JOIN userType on user.code = userType.code AND user.userType = userType.userType
You can either use joins
SELECT * FROM user U
INNER JOIN userType UT on U.code = UT.code
AND U.userType = UT.userType
I had to do something very similar but EXISTS didn't work in my situation. Here is what worked for me:
UPDATE tempFinalTbl
SET BillStatus = 'Non-Compliant'
WHERE ENTCustomerNo IN ( SELECT DISTINCT CustNmbr
FROM tempDetailTbl dtl
WHERE dtl.[Billing Status] = 'NEEDS FURTHER REVIEW'
AND dtl.CustNmbr = ENTCustomerNo
AND dtl.[Service] = [Service])
AND [Service] IN ( SELECT DISTINCT [Service]
FROM tempDetailTbl dtl
WHERE dtl.[Billing Status] = 'NEEDS FURTHER REVIEW'
AND dtl.CustNmbr = ENTCustomerNo
AND dtl.[Service] = [Service])
EDIT: Now that I look, this is very close to #v1v3kn's answer
I don't think that query is quite portable,it would be safer to use something like
select * from user
where code in ( select code from userType ) and userType in (select userType from userType)
select * from user
where (code, userType) in ( select code, userType from userType );

Resources