Table emp with columns:
id | name | sal | deptno | location
One user is inserting/updating 20 million records into that table.
Another user is retrieving data using select statement from the same emp table.
insert / update statement will take 2 hours and at the same time another user retrieve data in the mid of insertion/update process
How many records will come for execute the select statement in SQL?
How to check or what steps we need to follow to achieve this task in SQL Server?
Related
I am trying to make a table that shows changes in a database with the following format:
TableName
ColumnName
OldValue
NewValue
RowID
Company
CompName
ABC
XYZ
1
Company
Address
123
456
1
Company
CompName
EDF
TRQ
2
This is all in Microsoft SQL Server, and want to try and use the system tables to connect with the data using triggers as the data is updated (using Magic Tables to capture the old/new value as it is changed).
I just don't know how to connect the table and column information with the row data to get it to display in this format.
So far I have been able to find the system tables and get them to show their updates, and the regular tables with their updates, but have not had success in figuring out how to connect them together.
How to append specific data in flatfile destination using SSIS package loop iterate?
Source: emp (SQL Server) sample data. In this table I have 1 million records
I need to load specific empid information into flat file destination using loop iterate.
Sample data
empid | Ename
------+---------
1 | ab
2 | dg
3 | hg
Load the specific empid information maintain in empmaster table.empids values frequenctly changing in empmaster table
Table: Empmaster
Empid
-----
1
2
5
6
Destination: text files emp.txt
empid | Ename
-------+--------
1 | ab
2 | dg
I tried like this:
Step 1: create 2 variables empid (datatype varchar) and empidresult (datatype object)
Step 2: drag and drop the ExecuteSQL task and using oledb connection and SQL command is: select distinct empdid from empmaster and resultset is: fullresultset and resultset variable select the empidresult
Step 3: drag and drop the foreachloopcontainer and container type is foreachadoenumerator adoobjectsource variable is: empidresult and variable mapping is: empid
Step 4: inside container drag and drop the dataflow task and using oledb source connection SQL statement with parameters
select empid,ename from emp where empid =? and parameter mapping to empid
Step 5: drag and drop flatfiledestination and configure emp.txt file
here I select the columns names in the first data row unchecked and overwrite data in the file unchecked and executed the package i can append the data for required empids data but header is missing
I tried another methods empmaster records i have converts rows to columns and result store into object with commasepareate buts its supporting only up to 9000 records
if master table have more than 10,000 records then its not working
I need first row as header then remain each loop iterate should be data append in destination files.
How to achive this task in sql server?
what's wrong with a data flow sourced by:
SELECT empid,ename
FROM emp
JOIN Empmaster on emp.empid = Empmaster.empid
and then just run that into a flatfile output.
I have to find a leader of a group and update employee's leader. I am not sure how to proceed with this in DataStage.
I have an employee table as shown below
Emp_id mgr_id leader_id
1 100 400
101 201 500
3 202 600
I get a file to update employee table when an employee changes group. Change code = CHG means it is a job/group change.
I do an equi join between file and employee table and can update manager id. At the same time, I need to find a leader. I need to get all the employees who report to that top level leader and use as the leader id's for every employee.
File:
emp_id mgr_id chg_cd
1 102 CHG
101 301 CHG
File Row 1: There is change in manager for emp_id = 1; need to update mgr_id, leader_id in employee table
File Row 2: There is change in manager for emp_id = 102, need to change mgr_id and leader_id for in employee table
Can you please suggest me on how to proceed with this in DataStage?
ok this problem requires a solution with recursion. As DataStage has no way to do it (if the levels between managers and leaders are variable).
So load the data into a database table and use recursive SQL to query it - this will provide you the solution you are asking for.
Example:
Extract all leaders with their business units they manage inculding different levels) with the recursive SQL statement and use this data in da DataStage lookup to enrich the file data.
I would like to do data mining. But my data is not useful.
my table structure is something like:
date customerid age residence prosubsclassid productid
----------------------------------------------------------------------------
21.11.2001 123232323 a b 2099 23232322
amount asset sales
------------------------
4 34 56
Now I have to show the data in this way:
prosubsclassid 130207 130208 130209
------ ------ ------
1413232 1 3 1
3435545 2 1 2
3534344 3 1 sum(amount)
Column(customerid)
I want to convert to tabular form in my data.
There is no automatic way to do this. There are support for pivot in SQL server, but the columns still needs to be specified.
Depending on if you want to have a text report or a table with dynamic columns I would group the data per date and prosubsclassid and then using a cursor to build the data.
If you want a dynaic table build a dynamic sql query based on the grouped data and run Exec.
If you want text report, just concatenate the string data the way you want per line into
a temp table with one textcolumn and when you are done, select the table.
I have sqlite local database. I want to insert only fresh data from remote server to local database.Since there is no time field,it is difficult to insert only new records.How can i acheive this? I require this for my hybrid mobile app. Any helps apperciated..Thanks in advance.
Two tables:
my local db table is
tbl_orders
id name age
1 yyy 30
2 xxx 20
my remote db table is
tbl_orders
id name age
1 yyy 36
2 xxx 20
3 vvv 40
4 zzz 37
In the above the remote table contains additionally two records and also the value in first record(age column) get changed.now i want to insert and update this(i.e 1st,3rd,4th) to my local sqlite table without deleting and reinserting the whole table.
You should add a UNIQUE constrain in id.
Redefining your local table:
CREATE TABLE tbl_orders(UNIQUE id, name, age);
or without redefining table
CREATE UNIQUE INDEX tbl_orders_id ON tbl_orders(id);
With constrain, your updates become a single statement:
INSERT OR REPLACE INTO local.tbl_orders SELECT * FROM remote.tbl_orders;
Your table is replicated in two DB?
If yes, you can do an INSERT with NOT EXISTS clause in WHERE condition. Alternatively, add datetime field in your source table.
Another way:
Add a boolean field in your source DB (fl_sent), populated by a trigger when create a new row in your DB or update them. Default value is false, and when you want to syncronize your DBs your select is based on this field
SELECT * FROM myTable WHERE fl_sent = 0
For a more complete answer please post your tables.
EDIT AFTER COMMENT:
Solution 1:
Add field date in your remote table (source) (if you want, you write a trigger about toggle this field) and then execute your sendable query based on this date.
Solution 2:
Add field flag (fl_sent) set by zero (if you want, you write a trigger about toggle this field) and then execute your sendable query based on this flag.