I have a Dynamics CRM 2011 instance where the database has become corrupted. The corrupted data appears to be isolated to a few tables (e.g. PrincipalObjectAccess) and the instance still functions normally to all appearances. The data is irretrievable (all forms of DBCC CHECKDB, etc. have been run) and a backup is not available (preaching on backups will not help resolve the issue).
I've tried using schema and data synchronization tools like those offered by dbForge and Red-Gate, the schema sync works but the data sync always seems to come up inconsistent.
At this juncture I think my best route is probably to export all data from Dynamics CRM 2011 and then import it into a new instance of Dynamics CRM 2011. Any thoughts on the best way to accomplish this? Or alternative methods of rectifying the situation?
Exporting all data and importing it into new organization will likely create more errors and I wouldn't really go with that option unless everything else fails.
You said data synchronization failed: have you tried deleting all data from new instance first and then running data synchronization. It should be simpler than synchronization when data already exists there.
Have you tried synchronizing data using ApexSQL Data Diff ?
Another option you can try that doesn't require you to create new organization is reading your SQL Server transaction logs and checking if corrupted data can be found there. If you can retrieve the data then you can just re-create tables with valid data and you'll be all good. Unfortunately this is only possible using 3rd pary tools such as ApexSQL Log
I would recommend looking into the CRM 2011 Instance Adapter
Unlike Scribe, it's free.
Microsoft blog post: http://blogs.msdn.com/b/crm/archive/2012/10/24/the-microsoft-dynamics-crm-2011-instance-adapter-has-released.aspx
PowerObjects wrote an article about it as well:
http://www.powerobjects.com/blog/2012/10/26/introduction-microsoft-dynamics-crm-2011-instance-adapter/
Peter
If you can, export to Excel and import from there.
Advantage: easy, fast, graspable
If you can't, design a console application that connects to the server, queries it, fetches data and shoves it into the other instance.
Advantage: full control, repeatibility, configurability, coolness factor and you get to type some code
This really depends on the scope of your data. Are you talking about millions of records with a huge list of entities or are you talking a couple entities with a thousand or so records?
If it's something small, you could always try exporting via excel and then importing into the new org.
SSIS, CozyRoc or Scribe will do the trick. I'd opt for Scribe and go entity by entity if it is a mission critical situation.
Related
We have an application that requires our customers to have a SQL server instance on site. At their request, the application needs to synchronize the data in their database with a copy in our datacenter.
We're using .Net 3.5 SP1. We need to synchronize the data exactly, including IDENTITY columns.
We'd prefer to use something like LINQ to SQL that would let us make some simple select and insert/update calls against mapped entities. However, the IDENTITY columns seem to be a problem with LINQ and similar approaches.
We can do this all with built-up SQL statements and turn IDENTITY INSERT on / off as needed, but I'd prefer a more elegant solution.
Thanks!
** Edit - We DO need to write our own solution, and we do need to use .Net 3.5 SP1 to do it. I won't spend your time explaining all the reasons why, but please limit suggestions to options within the .Net playground.
Microsoft Sync Framework can be your solution. This is framework description from Microsoft:
Microsoft Sync Framework is a data synchronization platform from Microsoft that can be used to synchronize data across multiple data stores. Sync Framework includes a transport-agnostic architecture, into which data store-specific synchronization providers, modelled on the ADO.NET data provider API, can be plugged in.
Sync Framework is a comprehensive data synchronization solution that enables developers to build solutions that support synchronization of any database, on any data protocol over any network topology. msdn.microsoft.com
For your convinience providing link to good tutorial on the subject
If it is just a couple of tables that need to be synchronized and there is not a lot of data in the tables (now and future) you could develop some sort of bulk copy from your servers and bulk insert routine on the customer's server.
Since you said you can't use SQL Server replication services or SSIS, then perhaps a backup/restore procedure could be written. You could take a scheduled backup of your database and make it available to calling applications which could then copy the backup, restore it to another instance on the customers server, then pull all data you need via any number of methods and it would exist locally on the customers servers.
Beyond that, I think you may be asking for a maintenance and synchronization nightmare if you can't base your solution on tools that are made to do this sort of thing.
Can I store any custom tables in SharePoint's own database?
Is this supported behavior or not?
(I mean tables in MS SQL database, not SharePoint lists.)
If I can, how well does this play with backup/restore functionality?
What are possible caveats?
For anyone wondering why I'm asking: there's an app which is bound to SharePoint server and needs to store some purely relational internal information that doesn't make sense apart from that SharePoint instance. I would like to narrow down data storage to one place but I'm not sure if SharePoint likes its database being used for other purposes.
I'm using SharePoint 2007.
Is it possible? Sure. Should you? Nope.
The SharePoint content/configuration databases are subject to change with any update Microsoft releases, and any changes you make will very likely be destroyed, and if your farm depends on them, be left non-functional.
If you want to store purely relational data in a set of tables, just create another database. There's nothing stopping you from using the same SQL Server instance that houses your SharePoint content and/or configuration databases to store other relational databases as well.
Not a good idea: Support for changes to the databases used by Windows Sharepoint Services
...
Making any modification to the database schema
Adding tables to any of the databases
...
If an unsupported database modification is discovered during a support call, the customer must perform one of the following procedures at a minimum:
Perform a database restoration from the last known good backup that did not include the database modifications
Roll back all the database modifications
It is even worse than the above. It is likely that future upgrades will notice your changes to the content database schema and refuse to upgrade the database period.
I have two applications with own database.
1.) Desktop application which has vb.net winforms interface, runs in offline enterprise network and stores data in central database [SQL Server]
**All the data entry and other office operations are carried out and stored in central database
2.) Second application has been build on php. it has html pages and runs as website in online environment. It stores all data in mysql database.
**This application is accessed by registered members only and they are facilitied with different reports of the data processed by 1st application.
Now I have to synchronize data between online and offline database servers. I am planning for following:
1.) Write a small program to export all the data of SQL Server [offline server] to a file in CVS format.
2.) Login to admin Section of live server.
3.) Upload the exported cvs file to the server.
4.) Import the data from cvs file to mysql database.
Is the method i am planning good or it can be tunned to perform good. I would also appreciate for other nice ways for data synchronisation other than changing applications.. ie. network application to some other using mysql database
What you are asking for does not actually sound like bidirectional sync (or movement of data both ways from SQL Server to MySQL and from MySQL to SQL Server) which is a good thing as it really simplifies things for you. Although I suspect your method of using CSV's (which I would assume you would use something like BCP to do this) would work, one of the issues is that you are moving ALL of the data every time you run the process and you are basically overwriting the whole MySQL db everytime. This is obviously somewhat inefficient. Not to mention during that time the MySQL db would not be in a usable state.
One alternative (assuming you have SQL Server 2008 or higher) would be to look into using this technique along with Integrated Change Tracking or Integrated Change Capture. This is a capability within SQL Server that allows you to determine data that has changed since a certain point of time. What you could do is create a process that just extracts the changes since the last time you checked to a CSV file and then apply those to MySQL. If you do this, don't forget to also apply the deletes as well.
I don't think there's an off the shelf solution for what you want that you can use without customization - but the MS Sync framework (http://msdn.microsoft.com/en-us/sync/default) sounds close.
You will probably need to write a provider for MySQL to make it go - which may well be less work than writing the whole data synchronization logic from scratch. Voclare is right about the challenges you could face with writing your own synchronization mechanism...
Do look into SQL Server Integration Service as a good alternate.
A little background:
I have a remote, stand alone SQL Server database that is truncated at the end of every weekend. The data is hardly relational, not normalized at all, and pretty annoying to work with. On top of that, the schema for this database cannot be modified at all, because it is recreated by a third party application. Before the database is destroyed each week, a backup is created of that week's data. On average each database will have between 500,000 and 2,000,000 records.
My task is to create a historical version of this database that is a superset of all of these database backups. It should tie into our other databases which contain related sets of information. I have already started on an application to perform this task, and I've gotten to the point where I'm able to match data with our other databases, but I'm wondering if theres any best practice to handling this kind of import.
How do I make sure that I have unique IDs in my historical version of this database? Are there any features in SQL Server that can do some of the heavy lifting in this for me?
Thanks for your time on this.
There's definitely a feature in SQL Server that can assist you and that feature is called SSIS (SQL Server Integration Services). One of the main uses of SSIS is for ETL (Extract, Transform, Load), which means extracting data from several diverse source, transforming it into whatever you need to get into your destination database (such as a data warehouse - any linking with existing data will also happen here), and finally loading it into your destination DB.
I think the best way to get started, if that's what you want of course, is to pick up a good book on SSIS and go through it. While reading, don't forget to play around with the BIDS (Business Intelligence Development Studio - one of the SQL Server tools) to create some test packages.
Furthermore, on the internet you'll find plenty of "getting started" articles.
For your case in particular what I would do is:
create a generic package that can import the data from a source DB (one of your weekly DBs) and insert it into the destination DB - this package can be parameterized using Parent Package Configuration.
create a main package that loops over all backups in a certain folder, restores them one by one and calls the generic import package for each restore. After each successful import, the Control Flow would delete the previously-restored DB.
I think I've given you enough material to investigate on now :-)
Good luck,
Valentino.
I want to retrieve some data. How can I make a query on a Sharepoint database?
You shouldn't because of these reasons:
This is completely unsupported by the EULA you agreed to when you installed SharePoint. (I have to add a note that changing or calling triggers (except some) directly is unsupported, but not selecting)
Your queries are not guaranteed to work after applying any patches or service packs to SharePoint since Microsoft could change the database schema anytime.
Directly querying the database can place extra load on a server and hence performance issues.
Direct SELECT statements against the database take shared read locks at the default transaction level so your custom queries might cause deadlocks and hence stability issues.
Your custom queries might lead to incorrect data being retrieved.
Let me clarify, that #1 DOES NOT ALLOW you to modify sharepoint database in any way. SELECT`ing is permitted, however, as mentioned, that may lead to other problems.
However, if you are not interested in these points, then just use Visual Studio to connect to existing database, just do the regular procedure on how you connect to any other database.
But you can make your own database and store some additional information there.
Access SharePoint data the right way
Use SharePoint Object Model (Code can only be run on SharePoint server)
Use SharePoint WebServices (Run code from anywhere, from any application)
SharePoint 2013 now features REST API.
I have one thing to add. If you do decide to query sharepoint content databases directy, use the NOLOCK hint to prevent shared lock being taken out and potentially creating dead locks in the application.
If you don't mind using other proprietary Microsoft programs, Access/Excel/PowerBI all offer native connectivity to data stored in sharepoint lists/document libraries/meta data.