I am very new to ADF, I am trying to migrate a single customers data from one database to another. We would like to read files from blob storage, loop through each file, read the contents of the file which is a sql statement and use that during/within the copy activity. The name of each file will be the name of the source and destination table, we would like to pass in the company id that will be used in the query. (maybe do a replace to populate the id) Is this possible?
File name(s) in blob:
company.sql
Appointment.sql
SQL
-- company
SELECT c.*
FROM [dbo].[CompanyInfo] c
WHERE c.ID = '0000290'
-- appointment
SELECT appt.*
FROM [dbo].[CompanyInfo] c
INNER JOIN [dbo].[Patient] p ON c.id = p.CompanyID
INNER JOIN [dbo].[Appointment] appt ON appt.PatientID = appt.PatientID
WHERE c.ID = '0000290'
Related
I want to get data from source using OLE DB.
used SQL command to get that.
I tried to set it with WITH clause. It worked. But it took more time to give the output.
WITH Temp
AS
(
SELECT C.*
FROM DimCUSTOMER C
INNER JOIN DimSHOP S
ON C.CUST_ID = S.CUST_ID
)
SELECT *
FROM TEMP
WHERE ADDRESS IS NOT NULL
Then I tried it with # temporary table in SSMS.
it gave less time comparing to with clause.
SQL Code as Below.
SELECT C.*
INTO #Temp
FROM DimCUSTOMER C
INNER JOIN DimSHOP S
ON C.CUST_ID = S.CUST_ID
SELECT *
FROM #TEMP
WHERE ADDRESS IS NOT NULL
Then I set this code inside SSIS package OLE DB Scource.
But I'm getting an error when setting the SQL Code inside it.
First, use an Execute SQL Task into in the design view and rename it Create Temp Table :
IF OBJECT_ID('tempdb..##tmp') IS NOT NULL
DROP TABLE ##tmp
CREATE TABLE ##tmp
(
//your columns
)
INSERT INTO ##tmp
SELECT C.*
FROM DimCUSTOMER C
INNER JOIN DimSHOP S
ON C.CUST_ID = S.CUST_ID
Once the table has been created into the SSIS package. Right click OLE DB Source and choose Edit. Choose your data source and choose SQL command for the Data access mode dropdown. In the SQL command text you will need to create our SQL statement:
SELECT *
FROM ##temp
WHERE ADDRESS IS NOT NULL
There are multiple records in db against single ID. I want to get only single value from duplicate data and single different value in next column against single ID. do i need to use the pivot or something else.
i am trying this on SQL server
select distinct (d.SetId), s.Description, sd.SourceValue, d.TargetValue, d.TargetEntityDetailCode,e.entitydisplayname,e.entitycode from P2P_ADRSet s
inner join P2P_ADRSourceDataMaster sd on sd.SourceCombinationId = s.SourceCombinationId
inner join P2P_ADRDataMaster d on d.SourceDataId = sd.SourceDataId and d.SetId = s.SetId
Inner join org_entitydetails e on e.Entitydetailcode = d.TargetEntityDetailCode
where sd.SourceValue in (249,66)
-- i want this source value which is 249 and 66 in different column. for now its showing in single column.
I execute request
select *
from tempdb.sys.objects
where type_desc = 'USER_TABLE'
and see tables named like #AB12CD34, #ABCDEF01, etc.
I don't use such naming convention for temp tables. Is that possible to determine real names for these tables?
Any table that starts with '#' is a temporary table that exists until the session or connection is lost. The table is visible only within the current session. Any table that starts with '##' is a similar type table except that it is global in nature and other sessions / connections can see it.
This is not the system naming convention for standard temporary tables.
Temporary tables will generally show up as a 128 character name in the format
#YourTempTableName______________ ... _________00000000000D
Where the hex at the end acts to prevent collisions between different sessions.
Tables named #AB12CD34 are either table variables/table valued parameters or they are cached temporary tables from stored procedures.
When the stored procedure finishes executing a temp table can be cached so it does not have to be re-created again on next use. The FCheckAndCleanupCachedTempTable transaction renames the temp tables to this format as part of this process.
More about temporary table caching in this blog post.
The cached temp tables belong to the execution context of a cached execution plan. You can see stored procedures with cached execution contexts with
SELECT DB_NAME(dbid) AS DatabaseName,
OBJECT_NAME(objectid, dbid) AS ObjectName
FROM sys.dm_exec_cached_plans cp
CROSS apply sys.dm_exec_sql_text(cp.plan_handle) t
JOIN sys.dm_os_memory_objects m1
ON m1.memory_object_address = cp.memory_object_address
JOIN sys.dm_os_memory_objects m2
ON m1.page_allocator_address = m2.page_allocator_address
WHERE m2.type = 'MEMOBJ_EXECUTE'
AND cp.objtype = 'Proc'
You can also see cached temp tables with
select *
from sys.dm_os_memory_cache_entries
where name='tempdb' AND entry_data LIKE '<entry database_id=''2'' entity_type=''object'' entity_id=''-%'
But I don't see any way of linking these together to see which plan caches what temp object.
You could look at the column names and see if you recognize the table structure from one of your procs.
WITH T
AS (SELECT *
FROM tempdb.sys.objects
WHERE type_desc = 'USER_TABLE'
AND name = '#' + CONVERT(VARCHAR, CAST(object_id AS BINARY(4)), 2))
SELECT T.name,
c.name,
type_name(c.user_type_id) AS Type
FROM T
JOIN tempdb.sys.columns c
ON c.object_id = T.object_id;
Is there a way to save only some records to full text catalog? For example we have a table taxablePerson with columns
Id, VatNumber, Name, Street, ....other fields ...,
RecCreatedOn, RecCreatedBy, RecModifiedOn, RecModifiedBy, RecActive
Column RecActive means if this record is valid. Records don't get deleted, they are deleted by setting RecActive = 0. When I search I run:
SELECT TOP 50 *
FROM TaxPerson.TaxablePerson AS tp
INNER JOIN CONTAINSTABLE(TaxPerson.TaxablePerson, (Name), #find) AS key_table
ON key_table.[key] = tp.id
WHERE tp.RecActive = 1
ORDER BY key_table.rank DESC;
Performance is very good by not applying inner join. Is there a way to put only records which have RecActive = 1 to full text search catalog. This would be a great feature of ms sql fts.
You can create a VIEW and then add a fulltext index to the view.
in my DB there are more than 100 tables. Some of them have a column name "date".
I want to get all tables (table names) for a specific date.
so far I have been able to retrieve the table names that contain a date attribute:
SELECT pg_class.relname
FROM pg_class
INNER JOIN pg_attribute
ON pg_attribute.attrelid = pg_class.oid
WHERE pg_attribute.attname = 'date'
but I want to do somethin like that and it doesn´t work of course:
SELECT pg_class.relname
FROM pg_class
INNER JOIN pg_attribute
ON pg_attribute.attrelid = pg_class.oid
WHERE pg_attribute.attname = 'date'
AND date = '2014-12-05'
You can create a function that loops through the tables you've found and execute dynamically composed queries in it.
(Executing dynamic queries in PL/pgSQL)