It seems like something like this should exist, but I have never heard of it and would find such a utility to be incredibly useful. Many times, I develop applications that have a database backed - SQL Server or Oracle. During development, end users of the app are encouraged to test the site - I can verify this by looking for entries in the database...if there are entries, they have been testing...if not, they haven't.
What I would like is a tool/utility that would do this checking for me. I would specify the Database and connection parameters and the tool would pool the database periodically (based on values that I specify) and alert me if there was any new activity in the database (perhaps it would pop up a notification in the system tray). I could also specify multiple database scenarios to monitor in the tool. If such an app existed, I wouldn't have to manually run queries against databases for new activity. I'm aware of SQL Profiler, but when I reviewed it, it seemed like overkill for what I wanted to do (and it also wouldn't do the Oracle DB monitoring). Also, to use SQL Profiler, you have to be an admin of the database. I would need to monitor databases where I only have a read-only account.
Does someone know if such a tool exists?
Sounds like something really easy to write yourself. Just query the database schema, then do a select count(*) or select max(lastUpdateTime) query on each table and save the result. If something is different send yourself an email. JDBC in Java gives you access to the schema information in a cross-database manner. Don't know about ADO.
Related
I have a vb.net project that uses a SQLite database. I do this by using dataset/table adapters. The client is happy and all works well. However I have just heard that they plan on providing this product to another customer that wishes to use their SQL Server database. So I am writing this post so I can mentally prepare for this before I begin. I am not a database pro and have really enjoyed the simplicity of setting up and managing an SQLite database.
So any ideas on the easiest way to support SQL Server as well? I am happy to run them parallel to each other. Can I just make a separate service / middleware that syncs the SQLite database to the SQL Server on a timer and does not care about what the main app is up to?
Any pointers are appreciated.
Synchronizing two databases is possible, if rather complex. You need some mechanism to find out which records have changed, and if it is possible to have new changes in both databases, you also have to resolve conflicts.
A timer-based approach doesn't sound efficient: in most cases, the timer doesn't have anything to do; and after some data change, there is some amount time where the databases are not synchronized.
Can't you just replace SQLite with MS SQL Server?
I.e., have some configuration settings that determines whether your program's data lies in SQLite or on a server?
Assuming that an SQL Server database with the required structure already exist, this would, in theory, need nothing more than a changed connection string, and supplying some user name/password (if the server isn't configured to automatically use Windows logins).
There shouldn't be any big differences in the SQL dialects used. You have, of course, to test all your queries.
Is there any way to obscure the schema of a database on SQL Server?
If I have SQL Server Express installed on a client site, is there a way to obscure the schema and data so that someone else cannot come along and learn the schema in order to extract data out of it and into another product?
The best way to obscure your database schema is to not let it leave your servers.
Even if you encrypt the schema you still will have to provide the key somewhere, and if the client is determined to get it, they'll spend time and money to do so.
So you're better off either offering your product as service or making your client loyal by doing good job.
AFAIK, "no".
The best way to "lock down" your database is:
1) Install with appropriate roles and users (ideally, SQL roles and SQL users you create)
2) Explicitly restrict object permissions in SQL Server
3) Code your application to use SQL Server stored procedures (instead of raw T-SQL) as much as possible
4) Encrypt your stored procedures
Here's a good link on "SQL Server Best Practices" that might be of interest. It discusses security issues and a (relatively) new feature, "User Schema Separation":
http://msdn.microsoft.com/en-us/library/dd283095%28v=sql.100%29.aspx
This is a tricky one and may not even be 100% possible. However, there are a few tricks to setting it up:
Install a new named instance of SQL server with a custom SA account (both name and password). There is an installation method for SQL server call "Unattended Installation" which allows you to specify all the installation parameters for SQL server in an ini file and then run the install silently. Check out the documentation here: Unattended Installation of SQL Server 2008 r2
Create your database, tables, procedures, etc. with your magic SQL install script (use encrypted stored procs if you want, but they too are crackable)
Add/Verify the schema permissions for the custom SA account and Drop all schema permissions for all Administrator roles. The goal here is that no roles have any schema permissions to your database and only your custom SA user has permission (not assigned by role, but directly to the user).
There are several commercial applications that I know of that don't even tell you they are installing an instance of MS SQL express. They too will create their own named instance with a named SA account. I can't say I like that as a customer (as SQL takes a hit on the CPU and I don't want "secret" instances running on my workstation). But so long as you disclose this to your customers upfront, they may understand.
**Keep in mind a skilled DBA may have the knowledge to mess with system tables and what not to manually grant access to your database. These techniques really are just "obfuscation" and won't be 100% bullet proof.
As a side note: With the plethora of available 3rd party datalayers and webservice technologies, I think many companies are finding their database schema alone isn't so proprietary or valuable anymore. There was a time when the database schema alone could have represented hundreds of hours of coding. But today tools like EntityFramework, NHibernate, Linq-to-SQL, XPO, etc all create your database schema for you based on your software class definitions and in code attributes. So just seeing a DB table isn't really very valuable. Plus you might write a bunch of business logic, statistical analysis or other helper methods in your software that aren't in your database schema. In my opinion, this is where today's "value add" is found, in the business logic, analysis and reporting functionality of your software - not in the raw datatables.
This is also why another poster recommended obfuscating stored procedures, because these could be many times the work of the database schema itself if you have some nice analysis and reporting procedures written up. Its also what customer's would most likely want to customize for their own reporting needs. You may be inclined to have a policy that custom reporting can only be done by your company (hey, even the big guys like SAP are sticky with who can modify what).
There is a way, it's convoluted and ugly but it works.
You have a master table that acts as a lookup table for your other tables. This master table would look sort of like this:
id, guid, entityname, parent_id
then all of your table names and column names get renamed to be guids. after that you put an entry in the lookup table for each of them. When you want to select data you have to do so by pulling the guid's out of the lookup table by their entitynames which then give you the obscured table and column names.
There is a major software vendor out there that does something very similar to this, so it has been done before.
My customer is a dental practice that has bought a piece of practice management software. This software was installed on their local server, including a patient database, a schedule and all manner of medical records. Now they want me to write some utilities for them that aren't provided with their package, and for this I need the ability to query this database.
I tried calling tech support of the software manufacturers (Patterson/EagleSoft), and it's difficult finding anyone who understands the technology enough to answer my questions. As far as I can tell, there's no API for their software, and understandably they're reluctant to tell me how to query the DB directly, programmatically. They do have an interactive query window, but obviously that's no good for writing automated queries. All that they would let on is that somewhere there's a SQL Server DB, but the ODBC drivers to connect to it are SQL Anywhere drivers (huh?).
So I searched around on the server and couldn't find any database files. Then I discovered that the installation creates some kind of proprietary virtual machine, which is only visible to the EagleSoft software. But while they've been very good at insulating their DB in layers of obfuscation, they have left open an ODBC driver, which is indeed an SQL Anywhere connection.
Now after that fascinating and lengthy preamble, here is my question: What queries can I run over this ODBC connection to interrogate the DB as to its structure? If it's a SQL Server DB underneath I could use the sysobjects table, but I don't fully grasp how you can use a SQL Anywhere ODBC connection to connect to a MSSQL DB. And If they were misinforming me and it really is a SQL Anywhere DB underneath, what are the queries to run to get at the DB structure?
And if there's anyone else out there who's ever succeeded in actually querying EagleSoft (or any similar proprietary package) - please tell me how you did it!
Turns out the simplest way to do it was to write a little app using OdbcDbConnection, and connect using the DSN installed with the software. It took one probing 'select * from sysobjects' to reveal that it is, indeed a MS-SQL database underneath all that, and I'm good to go from there!
I'd use a tool like squirel which is great at browsing any database to check if anyone was successful with "SQL Anywhere" this google result:
http://blog.gmane.org/gmane.comp.db.squirrel-sql.users/month=20091001
Shows that others have managed to get squirrel to do this. It's quite easy to use... assuming of course you manage to get the connection working!
A few tools that might help are SQLWorkbench and Django. I use SQLWorkbench to copy the data from the production system into a Postgres database so I can hack on it without damaging the production environment. Then I use Django's inspectdb to generate models of the database environment. From there it's easy to create 'views' into the database and templates to display exactly what I want.
UPDATE: As of Eaglesoft 19, it looks like Patterson has password protected the database and they have gone out of their way to prevent users from getting at the data without paying them for access.
UPDATE: Like I mentioned before, Eaglesoft 19 has a locked-down version of the database. For read-only access you can call Patterson and ask them for the password to the "Database Admin" section of their "Technical Reference" tool that is installed on your server. Once you are in there, there's an option to set a read-only password for access to the database. The username is 'dba' and whatever password you set. Some times it takes a bit of back-and-forth with them to give you access, but my solution was to say "We've been putting patient data into Eaglesoft for over a decade and we've always had access to the database. Now you're restricting it and telling us we have to pay for access. It sounds like you are trying to extort money by holding our data hostage. I should probably run this by our legal team."
EDIT: Nov 18 2022: You can still easily get read-only access to an Eaglesoft database in 21.20.08 (the latest version) by calling Patterson and getting the "day password" for Technical Reference. From there you can enable a read-only user. Based on some of the changes Patterson is making to their application architecture and the database, I suspect they will stop using direct connections to the database in the next year or two. When they make that change, you will only be able to access the database through their API Server. After playing "phone tag" with one of their salesbros for several weeks and doing some light social engineering, I managed to get their price list for going through the API server. It's atrocious. Most offices pay ~$500/mo to Patterson for free tech support and free upgrades. They want developers to pay nearly as much per office for access to the API. They've realized they can lock practices out of their own data and monetize it. We are working with a company that is actively developing a replacement for Eaglesoft to get away from this horrible vendor lock-in.
I have written my own PHP driven website to access and manipulate data in my eaglesoft database. You simply create odbc connection to local DNS entry and done. To see database structure you can use the technical reference included in eaglesoft or advanced query tool.
I'm looking for a strategy to allow automatic updates for a number of databases at customer sites through a publish-subscribe kind of mechanism. Right now there is a datacenter which has all the master data that get fed through extractions from hundreds of databases out there. The problem is that, whenever I need to do create a new view in the remote customer databases, I have to manually roll out an installation patch and ask the users to run it (their sites are behind firewalls, so I can't remotely do that from my end). Ideally, I would like to have a "DDL image" of the customer database schema at the datacenter, and whenever any change happens to it, all the subscribing customer databases would update their table view codes. The target databases are mostly SQL Server 2005 and Oracle.
I heard the MS SQL replication services could do such a thing? What about Oracle? anybody had experience with such?
Thanks!
Not sure about existing solutions, but how about writing your own auto-update mechanism that would run on a timer on the client machines and pull the latest schemas and views from some service table in your master database? Your change wouldn't get propagated straight away to all sites and some sites would update before others, but they would all eventually see the changes.
Golden gate might fit your needs.
This is probably the most classic database problem.
I have an E-commerce software solution hosted on a SQL server for data, and a web server for the frontend. Every instance/customer has its own database on SQL Server 2008.
During development of the next version, I might change or add tables, views, stored procedures etc.
How do I publish this change to all databases, without losing data? It should be done via a script or something similar. Centralized management is the key...
Perhaps it's something you've already considered, but my company uses software from Red Gate (http://www.red-gate.com/) which compares our development version of the DB and the production one, generates and executes scripts to bring production on par with development.
(I'm not a sales person from Red Gate, but I think this might be what you're looking for)
I use SQL Compare for schema changes and SQL Data Compare for data changes. Works like a charm!
This problem is essentially one of automating the manual process of logging on to a SQL Server, and running a script against one or more databases, that does the modifications you need.
It's made worse, of course, if the instances of SQL Server that you need to update are remote from you, and therefore not directly accessible.
It's also vital to ensure that the scripts are applied in sequence - it would be no point running the "add index" script before the "create table" script.
The way we've solved this is with a web service that packages script files as datasets, and delivers them in the correct sequence to the remote systems when they call home.
On the remote SQL Server, we have a .NET application which calls the web service, downloads the script files, unpacks them and applies them to the database.
When the remote system calls in, it supplies the ID of the most recent upgrade it has. When the web service completes, it knows the last one it delivered. It's therefore trivial to know what level the remote systems are at.
The only manual intervention required is to create the scripts in the first place, and upload them to the central server.
A script should be executed on sql server machine by db admin.
Main algorithm of such scipt is about to create backup, lock table each table in loop, alter it, release.
Another poster mentioned the Red Gate products, and I'll throw another commercial product out there - Quest Change Director:
http://www.quest.com/change-director-for-sql-server/
Disclaimer: I work for Quest, although I'm not in sales. Change Director does comparisons, syncing, links to a change management system, can use your dev/qa server as a source or use T-SQL scripts, has an audit trail and rollback capabilities, etc.
Like you said, central management is key, and this product focuses on that.