I'm after a DB Comparison tool for SQL Server that allows me to do the following:
Schedule a comparison to happen on a recurring schedule
Email me the results (in a nice readable format and not the generated script)
Allow me to exclude/include certain object names (for example exclude table names containing %test%. That's not a real example but there is a good reason why that would come in useful.)
As well as the obvious:
Have the usual options for ignoring things like comments, identity seeds etc
Options for selecting different types of objects
If it was free or at least didn't cost a forture that would be an extra bonus of course.
I have tried out RedGate's SQL Compare and also the built-in DB Comparison in Visual Studio but neither seem able to do the first 3 points above. I also looked at other tools recommended in various threads on here but again they don't mention in their features the 3 points above.
One option I found is RedGate's SQL Comparison SDK with which I think I could write something to do what I want.
I just wanted to investigate tools that might do all of the above out of the box.
Thank you!
SQL Compare Pro comes with a command line, which will be easier to set up than the SDK. If you call this via the Windows Scheduler or in an Agent Job you can achieve what you're looking for.
An example of how to invoke the command line from Powershell it can be found here:
http://www.simple-talk.com/sql/database-administration/auditing-ddl-changes-in-sql-server-databases/
This article also covers how to send an email in Powershell. SQL Compare can also be passed a filter using the /filter switch to exclude objects based on various rules.
http://www.red-gate.com/supportcenter/Content/SQL_Compare/help/10.0/sc_cl_Switches_in_the_cl
Do please email support#red-gate.com should you have trouble getting this working.
I don't think any tool would do all of this out of the box. Have you had a chance to look at
sp_CompareDB. I had a similar requirement and ended up writing my own routine based on the same.
http://www.sql-server-performance.com/2001/database-comparison-sp/
Related
I did see some other posts on this, but they were rather old and there does not appear to be any solutions at this point.
I'm trying to determine where a particular table(s) that SSIS is loading during a monthly job is being used in other packages. The package that loads these tables have in the past several months been taking much longer than before, and I'm trying to see if I can eliminate this load all together.
I just happened to check the Allocation packages in our database to see how the tables were being used, and discovered that I can't find anywhere when/where those tables are being used. Is there a function or query I can run in SSMS or elsewhere to determine how to find this information?
Thx in advance - please let me know if I need to clarify something.
The packages are just XML files. If you have the packages somewhere on your file system you can use any program that searches through text files.
I'm not sure about older SSIS projects but with an SSIS project in Data Tools for SQL Server 2012 you can just use the build in search function to search through your entire solution. It will also search in the XML of all the packages.
If you don't have this particular information saved anywhere already in your documentation then I think you are going to have some difficulty in finding an accurate way to retrieve this information. However, there are a few automated data collection options that might help you get most of the way there.
The first option is that because all SSIS Packages are essentially glorified XML that is being fed into an engine you can perform a patterned search on the packages like GREP to look for that particular table name. Any packages that dynamically retrieve and build the table name though would not be found through this method.
Another option would be to run a server side SQL trace with a pattern match based on the table name(s) and limited to the host or application name of SSIS. Run over the course of a month or so would make for a fairly accurate list.
I haven't used it myself, but the DOC xPress tool from PragmaticWorks might be what you're looking for.
I'm running Oracle 11g SE1 .
Just wondering if there're any tools that would allow me to test the data integrity of a ( mostly read-only ) schema. Essentially, what I want to do is to have some queries that run every night or so and see if they return the expected result. For example:
SELECT COUNT(*) FROM PATIENTS WHERE DISEASE = 'Clone-Killing Nanovirus';
Expected result : 59.
How do people normally do such testing ?
I've used SQLUnit and written about it here. I don't believe any new development is being done on it but it should accomplish your goal.
SQL Developer (free, as in beer) also has a Unit Testing framework. I have installed it and that's about it. I want to use it more, but I've been working with BI the past few years so no external pressure to learn it.
The tests that you want to create sound pretty simple, so either of those should work well for you. The next step would be to have them run on a schedule (cron, windows scheduler, etc) or you can go crazy with a continuous integration tool like Atlassian's Bamboo (haven't used it).
Of course you could skip the tools altogether and just write up scripts that are called from the command line. Fancy would have you writing the results to a database table so you can easily skin it, simple would be piping the results to a text file and reviewing that each day.
Hope this helps.
You could batch up your queries and run a simple perl script using DBI that would run the queries and check them against an accepted tolerance and email you if something didn't meet thresholds. I know I have written such db checking code before to make sure items were within thresholds. Perl is a good tool for this sort of thing as the DBI module can connect to your database and then you can run some canned queries and easily send yourself an email using the MIME package. http://www.perl.com/pub/1999/10/DBI.html
I know a little about SQL injections and URL decode, but can someone who's more of an expert than me on this matter take a look at the following string and tell me what exactly it's trying to do?
Some kid from Beijing a couple weeks ago tried a number of injections like the one below.
%27%20and%20char(124)%2Buser%2Bchar(124)=0%20and%20%27%27=%27
It's making a guess about the sort of SQL statement that the form data is being substituted into, and assuming that it will be poorly sanitised at some step along the road. Consider a program talking to an SQL server (Cish code purely for example):
fprintf(sql_connection, "SELECT foo,bar FROM users WHERE user='%s';");
However, with the above string, the SQL server sees:
SELECT foo,bar FROM users WHERE user='' and char(124)+user+char(124)=0 and ''='';
Whoops! That wasn't what you intended. What happens next depends on the database back-end and whether or not you've got verbose error reporting turned on.
It's quite common for lazy web developers to enable verbose error reporting unconditionally for all clients and to not turn it off. (Moral: only enable detailed error reporting for a very tight trusted network, if at all.) Such an error report typically contains some useful information about the structure of the database which the attacker can use to figure out where to go next.
Now consider the username '; DESCRIBE TABLE users; SELECT 1 FROM users WHERE 'a'='. And so it goes on... There are a few different strategies here depending on exactly how the data comes out. SQL injection toolkits exist which can automate this process and attempt to automatically dump out the entire contents of a database via an unsecured web interface. Rafal Los's blog post contains a little more technical insight.
You're not limited to the theft of data, either; if you can insert arbitrary SQL, well, the obligatory xkcd reference illustrates it better than I can.
You'll find detailed info here:
http://blogs.technet.com/b/neilcar/archive/2008/03/15/anatomy-of-a-sql-injection-incident-part-2-meat.aspx
These lines are double-encoded -- the
first set of encoded characters, which
would be translated by IIS, are
denoted by %XX. For example, %20 is a
space. The second set aren't meant to
be translated until they get to the
SQL Server and they use the char(xxx)
function in SQL.
' and char(124)+user+char(124)=0 and ''='
that's strange..however, make sure you escape strings so there will be no sql injections
Other people have covered what's going on, so I'm going to take a moment to get on my high-horse and strongly suggest that if you're not already (I suspect not from a comment below) that you use parameterized queries. They literally make you immune to SQL injection because they cause parameters and the query to be transmitted completely separately. There's also potential performance benefits, yadda yadda, etc.
But seriously, do it.
Are there tools that make the job easier? If command-line only tools exist, then can anyone speculate if there is a market for a GUI tool? For example, you can create a relational database by modeling visually. Should the same notion exist for LDAP?
Apache Directory Studio includes an ldif-Editor. It is still a text editor but with syntax highlighting, autocompletion and group collapsing for ldif files:
http://directory.apache.org/studio/
I don't know if there are any tools but it isn't that hard to create them by hand.
If you are using IPlanet LDAP then they had a nice interface for creating and modifying schemas though. :)
I don't know if you would consider that to be by hand otherwise that is one tool to use.
I've done some LDIF handling using Perl and the Net::LDAP::LDIF module and it made scripting custom LDAP conversions very easy.
Have you looked at the command-line tool, LDIFDE.exe? Should be on your domain controller.
Business people give me Excel spreadsheets with inconsistent formatting of user and group data and want it loaded right away (then they come back with a new version and tell me they've only added some new users, but some are missing, some data is now invalid, there's a missing column etc.) They want unique passwords assigned, group memberships set up based on department id fields, and so forth.
Then they come back two weeks later and want to know about the differences between that spreadsheet and one from six months ago. Sigh.
I generally just do it all with a few hand-crafted Python scripts.
A lot of times you may be copying objects from one tree to another. Or backing them up. In that case, most LDAP tools have some way of exporting as LDIF. Then you can easily modify the files as needed.
Or copy examples to reuse.
I have seen a number of tools that will do tasks and output the results as LDIF, which can be handy, but they are basically point usage tools.
Can anyone provide some real examples as to how best to keep script files for views, stored procedures and functions in a SVN (or other) repository.
Obviously one solution is to have the script files for all the different components in a directory or more somewhere and simply using TortoiseSVN or the like to keep them in SVN, Then whenever a change is to be made I load the script up in Management Studio etc. I don't really want this.
What I'd really prefer is some kind of batch script that I can run periodically (nightly?) that would export all the stored procedures / views etc that had changed in a given timeframe and then commit them to SVN.
Ideas?
Sounds like you're not wanting to use Revision Control properly, to me.
Obviously one solution is to have the
script files for all the different
components in a directory or more
somewhere and simply using TortoiseSVN
or the like to keep them in SVN
This is what should be done. You would have your local copy you are working on (Developing new, Tweaking old, etc) and as single components/procedures/etc get finished, you would commit them individually until you have to start the process over.
Committing half-done code just because it's been 'X' time since it was last committed is sloppy and guaranteed to cause anyone else using the repository grief.
I find it best to treat Stored Procedures just like any other compilable code: Code lives in the repository, you check it out to make changes and load it in your development tool to compile or deploy the code.
You can create a batch file and schedule it:
delete the contents of your scripts directory
using something like ExportSQLScript to export all objects to script/scripts
svn commit
Please note: That although you'll have the objects under source control, you'll not have the data or it's progression (is that a renamed field, or 1 new field and 1 deleted?).
This approach is fine for maintaining change history. But, of course, you should never be automatically committing to the "production build" (unless you like broken builds).
Although you didn't ask for it: This approach also won't produce a set of scripts that will upgrade a current DB. You'll only have initial creation scripts. Recording data progression and creation upgrade scripts is beyond basic source control systems.
I'd recommend Redgate SQL Compare for this - it allows you to compare database versions and generate change scripts - it's also fairly easily scriptable.
Based on your expanded question, you really want to use DDL triggers. Check out this article that details how to create a changelog system for your database.
Not sure on your price range, however DB Ghost could be an option for you.
I don't work for this company (or own the product) but in my researching of the same issue, this product looked quite promising.
I should've been a little more descriptive. The database in question is for an internal ERP system and thus we don't have many versions of our database, just Production/Testing/Development. When we've done a change request, some new fancy feature or something, we simply execute a script or series of scripts to update the procedures in question on the Testing database, if that is all good, then we do the same to Production.
So I'm not really after a full schema script per se, just something that can keep track of the various edits to the stored procedures over time. For example, PROCESS_INVOICE does stuff. It gets updated in some minor way in March. Some time later in say May it is discovered that in a rare case customers get double invoiced (or some other crazy corner case). I'd like to be able to see what has happened over time to this procedure. Currently the way the development environment is setup here I don't have that, which I'm trying to change.
I can recommend DBPro which is part of Visual Studio Team Edition. Have been using it for a few months for storing all parts of the database in Team Foundation Server as well as for deployment and database compares, etc.
Of course, as someone else mentioned, it does depend on your environment and price range.
I wrote a utility for dumping all of the relevant parts of my db into a directory structure that I use SVN on. I never got around to trying to incorporate it into the Manager but, if you're interested, it's here: http://www.reluctantdba.com/dbas-and-programmers/sqltools/svnforsql2005.aspx
It's free and, since I regularly run it, you know any bugs get fixed quickly.
You can always try integrating SourceSafe with SQL Server. Here's a quick start : link . To work with it you've got to have Managment Studio Developers Edition.