I have to migrate data in sqlserver from one table to another. Most of the standalone table were possible with select and insert into sp's.
The issue is in one table from the old database there is a column with a primary key say unit.
Now this unit is divided into two parts in the new database for example 'industry' and 'resource'.i.e industry + resource together constitute the unit.
How can i go about with the migration of this.
I have done similar "Database Transformation" tasks with SQL Server Integration Services it does require that you have SQL Server Standard or Enterprise (not Express) but because it is part of SQL Server there is no additional licencing, if it isn't installed you may need to re-run the installer and select "Integration Services".
SSIS will allow you to create a data flow, aranging the data more or less how you want it. There are several tutorials on MSDN.
Related
We are working on a project to migrate our Oracle database to SQL Server. There are many off the shelf products that seem to do this such as Microsoft SQL Server Migration Assistant & Navicat premium for example.
One of the requirements of this migration is that we are changing the naming convention of the tables & columns as part of the transition.
In Oracle we currently have adopted the naming convention of:
Table: T_USERS, T_REPORTS, T_USERS_REPORTS etc.
Columns: USER_ID, CREATE_DT etc.
While moving to SQL Server we wish to adopt a SQL Server specific naming convention (Camel case):
Table: Users, Reports, UsersReports etc.
Columns: UserId, CreateDt etc.
Both of these tools mentioned seem to only create a SQL Server object with the exact same table & column names as part of the data migration process. Is it possible to create this sort of mapping with either of these two tools? Or is there a better tool available?
Both of these also do not feature incremental data migration. They are designed to completely clean out all records and then migrate all data. This is obviously not ideal for performance. Is there a tool that has a delta type migration where it will compare existing records on both target and source destinations?
Background
I have a production SQL Server 2005 server to which 4 different applications connect and make changes.
There are no foreign keys and in some cases no primary keys.
Unfortunately throwing the whole thing out and starting from scratch is not an option.
So my solution is to start migrating each of the applications to a service layer approach so that there is only one application directly connecting to the database.
However there are problems that need to be fixed before that service layer is written and all the applications are migrated over.
So rather than make changes and hope they don't break any one of the 4 badly written applications (with no way of quickly testing all functionality) my solution is to start auditing the database
Problem
How do I audit what stored procedures, tables, columns, views are being accessed/updated/called by each user on SQL Server 2005.
I can find out which tables are being updated but I have no idea which columns and by what users.
I also don't know if certain tables are being accessed only through stored procedures/views.
I know that SQL Server 2008 has better auditing features but if I could do this without spending money that would be great. That said if the best solution is to upgrade or buy software that's also an option.
Check out SQL Server 2008's CDC feature. You can't use this directly in 2005 but you can write a trigger for each table to log all data changes to a new audit table. i.e. you'd have an audit table for each table in your db, with all the same columns plus some additional columns saying what the operation was and when it occurred.
If the nature of your applications means you can get user information and/or application information from CURRENT_USER and APP_NAME() you could include that information in the audit table too.
And check out this answer for more goodness.
I'm developing a Java EE JBoss service which will be deployed in two different environments: one using SQL Server as the database, and a different one which uses Oracle 10g.
My database schemas are very similar, except that the primary key columns are identity-auto-increment in SQL Server and generated using sequences in Oracle.
With JPA, is it possible to use a single set of entities with both DB's?
Thanks!
Of course you can, but just specify the generated-value definition for the PK field in XML metadata rather than annotations. Use 2 different "orm.xml" files, so, for example, use "persistenceUnitOracle" for Oracle that references orm-oracle.xml, and "persistenceUnitSqlServer" for SQLServer that references orm-sqlserver.xml
I’m after a bit of advice on the best way to go about this is SQL server 2008R2 express. I have a number of applications that are in separate databases on the same server. They are all “plugins” that use a central staff/structure list that will be in a separate database. The application is in the process of being migrated from JET.
What I’m looking for is the best way of all the “plugin” databases being able to see the central database and use those tables in standard queries and views etc.
As I’m using express that rules out any replication solution and so far the only option I can think of is to use triggers or a stored procedure to “push” out all the changes to the plugins. The information needs to be populated on a near enough real time basis however the number of changes will be very small maybe up to 100 a day and the biggest table only has about 1000 rows at the moment (the staff names table).
Hopefully that will cover all everything but if anyone needs any more details then just ask
Thanks
Apologies if I've misunderstood, but from your description it sounds like all these databases are hosted on the same instance of SQL Server - it's your mention of replication that makes me uncertain.
Assuming that's the case, you should be able to replace any copies of tables from the central database which are held in the "plugin" databases with views or synonyms which reference the central tables directly, since SQL server allows you to make references between databases on the same server using three-part naming (database_name.schema_name.object_name)
For example, if each plugin db has a table StaffNames, you could replace this with a view by dropping the table, then creating a view:
drop table StaffNames
go
create view StaffNames
as
select * from <centraldbname>.<schema - probably dbo>.StaffNames
go
and your code should continue to work seamlessly, as long as permissions are set up.
Alternatively, you could replace all the references to the shared tables in the plugin databases with three-part name references to the central database, but the view method requires less work.
This involves data replication, kind of:
We have many sites with SQL Express installed, there is an 'audit' database on each site that has one table in 1st normal form (to make life simple :)
Now I need to get this table from each site, and copy the contents (say, with a Date Time Value > 1/1/200 00:00, but this will change obviously) and copy it to a big 'super table' in sql server proper, that also has the primary key as the Site Name (That needs injecting in) and the current primary key from the SQL Express table)
e.g. Many SQL Express DBs with the following table columns
ID, Definition Name, Definition Type, DateTime, Success, NvarChar1, NvarChar2 etc etc etc
And the big super table needs to have:
SiteName, ID, Definition Name, Definition Type, DateTime, Success, NvarChar1, NvarChar2 etc etc etc
Where items in bold are the primary key(s)
Is there a Microsoft (or non MS I suppose) app/tool/thing to manager copying all this data accross already, or do we need to write our own?
Many thanks.
You can use SSIS (which comes with SQL Server) to populate, it can be set up with variables to change the connection string to the various databases. I have one that loops through the whole list and does the same process using three differnt files from three differnt vendors. You could so something simliar to loop through the different site databases. Put the whole list of database you want to copy the audit data from in a table and loop through it changing the connection string each time.
However, why on earth would you want one mega audit table per site? If every table in the database populates the audit table as changes happen, then the audit table eventually becomes a huge problem for performance. Every insert, update and delete has to hit this table and then you are proposing to add an export on top of that. This seems to me to be a guaranteed structure for locking and deadlocks and all sorts of nastiness. Do yourself a favor and limit each audit table to the table it is auditing.
Things to consider:
Linked servers and sp_msforeachdb as part of a do-it-yourself solution.
SQL Server Replication (by Microsoft) (which I believe can pull data from SQL Server Express)
SQL Server Integration Services which can pull data from SQL Server Express instances.
Personally, I would investigate Integration Services first.
Good luck.
You could do this with SymmetricDS. SymmetricDS is open source, web-enabled, database independent, data synchronization/replication software. It uses web and database technologies to replicate tables between relational databases in near real time. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outage.
As of right now, however, you would need to implement a custom IDataLoaderFilter extension point (in Java) to add the extra column. The metadata would be available though because your SiteName would be the external_id.