Database queries as application healthchecks - management tool - database

Hey there fellow Stackoverflowers,
In our company we have several application stacks running on different types of databases (MySQL, PostgreSQL, MS SQL, Azure SQL,..). For monitoring purposes we use some scripted queries on the databases of all these application stacks, with Nagios reporting back the results in an email.
Now, since our support team would also like easy access to these queries in order to manually run them or modify them, we were considering building an application specifically designed to be able to store, run and modify queries that can be executed on any of the above listed database types and offering both a user-friendly webinterface and a REST API with JSON output for our new reporting stack based on SENSU, to be deployed in a few months.
My personal belief is that a tool like this must already be out there, since the use case for it is so generic. However, googling did not yield any results even closely resembling what I am looking for.
So my question to you is: Do you know of such a tool? If you had to build it yourself: what would your approach be? We're mostly a Java/C++ team, but are open to all options.

Some or may be all of this stuff can be done by an existing API called NAGIRA. Look it up on Google. This will definitely give you all the results in JSON format. Also i think it would allow you to run checks manually. So you can may be build a little front end and call this API to achieve what you want.

A little late of a reply, but check out http://cloudmonix.com -- it offers ability to create metrics based on custom SQL queries, supports SQL Azure, SQL Server, MySQL, and Oracle. Also integrates with Nagios (and Zabbix)

Related

Is there a generic oData provider for SQL Server?

I've created a gadget for our CRM consultants that allows them to present data from an oData source in CRM. At the moment, it will connect to any data source but for customer sites we need to develop an oData service using WCF each time for each data source.
Does anyone know if there's a decent generic tool out there that can retrieve data from SQL Server, present it (via IIS) as oData and that can be configured without Visual Studio by a non-developer?
We (the WCF Data Services team) have heard this ask a couple of times; what follows are a few of my thoughts in no particular order.
We haven't heard the ask a lot. There's a reasonable amount of work to do here, and without sufficient asks it's hard to justify. That said, there's nothing stopping the community from spinning up an effort to achieve this (hint, hint :)).
There's a number of questions you would need to answer. For instance, what sort of default limitations would the provider have? Would you really want to allow arbitrary expands on something that's probably a production database server? What about permissions? What about read/write?
What happens for mutable schemas? Is this a completely dynamic provider? How much overhead is there in scanning the database schema, and how frequently would the database schema need to be scanned?
How would clients take advantage of a dynamic OData service? Most clients use some form of code generation to make interacting with the service easier.
These thoughts aren't really intended to dissuade at all, but hopefully they give you some things to think about should you attempt to create a generic provider on your own. If you do so, I'd love to hear about it.

Best approach to database and UI development?

This question covers a number of areas, so may ultimately have to broken up into constituent parts. However, I don't want to do this, at least until I have some general idea of what direction I should be taking!
I want to create an online booking system under the .Net Framework. The GUI would have two separate vital areas. 1. would be the typical form GUI for setting up a user account. 2. would be a Calendar (much in the same vein as Google Calendar - if not actually Google Calendar) which would show users available times that users can book, and hopefully allow them to actually book by simply clicking on an available time as displayed on the calendar.
The available times that would be displayed on the calendar would depend on two things: the type of user the user is (would be specified in the signup form) and also (obviously) whether or not other users have already taken available slots.
One big question I have is whether it is possible to use Microsoft Access (any version) for this task. My preference is due to familiarity, and the fact that most other technologies being used in this development are Microsoft based. I know it is possible to migrate Access DBs with SQL Server - but would this include the vital front end (i.e. the form)?
Finally, The strategy for approaching the calendar aspect of the front-end is doing my head in a little! While it is certainly possible to integrate SQL Server and Google (docs/calendar) I feel that attempting to shoehorn Google Calendar into such a complex relational database would be ultimately infeasible - but I am struggling to think of an alternative without having to do my own development (which, in this case, I am seriously attempting to avoid!)
Any guidance on the above would be greatly appreciated!
My $.02: as #Luxspes said, Sql Server has no "forms" or "reports", but the SQL Server engine is better suited to web-based applications than Access's "Jet" engine. You could, however, develop your application in Access and then "upsize" your database, which tells Access "hey, those forms and reports are great, but now I want you to put the back-end database into SQL Server Express (free) and keep the front-end in Access." You can then publish your forms and reports to the web using Access's own tools, getting them "for free" while getting the better database engine on the back.
Be aware that doing this is not generally acceptable in the commercial world. An Access application looks like an Access application and most of the world will recognize it, and most don't like it: it will seem like you're cutting corners unless you can make the forms look really, really good. For in-house use it's fine because it has a fast turn-around time so the ROI is great. If you're making a commercial site, I'd take a look at the Microsoft MVC 3 framework: it will take a while to get your noodle around it, but once you're up to speed you can develop sites pretty quickly and robustly. If "robustly" is a word.
Is it possible to use Microsoft Access (any version) for this task, the task being using Access as the database for a online booking system under the .Net Framework: Yes. (But I would prefer to use MSSQL Express Edition)
I know it is possible to migrate Access DBs with SQL Server - but would this include the vital front end (i.e. the form)?: No, SQL Server has not concept of "forms". If you already have them in Access you would have to re-create them in, for example, ASP.NET. If you are looking for the easiest fastest way to do it, I would recommend you LightSwitch ( http://www.microsoft.com/visualstudio/en-us/lightswitch )
Finally, The strategy for approaching the calendar aspect of the front-end is doing my head in a little!: If you do no want to code it, you can find it in Google, just write ".NET Schedule control"

What considerations should be made when creating a reporting framework for a business?

It's a pretty classic problem. The company I work for has numerous business reports that are used to track sales, data feeds, and various other metrics. Of course this also means that there is a conglomerate of disparate frameworks, ASP.net pages, and areas where these reports can be found. There have been some attempts at consolidating these into a single entity, but nothing has stuck yet.
Since this is a common problem, and I am sure solved innumerable times, I wanted to see what others have done. For the most part these can be boiled down to the following pieces:
A SQL query against our database to gather data
A presentation of data, generally in a data grid
Filtering that can vary based on data types and the business needs
Some way to organize the reports, a single drop down gets long and unmanageable quickly
A method to download data to alter further, perhaps a csv file
My first thought was to create a framework in Silverlight with Linq to Sql. Mainly just because I like it and want to play with it which probably is not the best reason. I also thought the controls grant a lot of functionality like sorting, dragging columns, etc. I was also curious about the printing in Silverlight 4.
Which brings me around to my original question, what is the best way to do this? Is there a package out there I can just buy that will do it for me? The Silverlight approach seems pretty easy, after it's setup and templated, but maybe it's a bad idea and I can learn from someone else?
It may sound contrived, but we just used SSRS. Once it is installed, the /ReportServer web site does a decent job of managing and organizing reports, permissions, etc. You can also make wrappers in front of SSRS via ASP.Net controls or via SharePoint, etc. Cost = free. It works nicely via SQL Developer edition too.
If your SQL service is MS SQL Server 2005+ then I would definitely recommend SQL Service Reporting Services. It covers all the cases you outlined very well and is very easy to get into for someone already familiar with SQL.
myDBR reporting tool might be suitable for you. With myDBR you can create reports quite easily using the built in Query Builder. Once your reports are done you can organize them in categories and also specify access rights for users and user groups.
You can fully concentrate on creating your report content (using SQL) and myDBR will take care of the data presentation. For example creating charts from data is just a matter of adding one extra line to your stored procedure.
myDBR also provides Single-Sign-On Integration so that your users can continue to log in with their already existing username/password.
The tool is also very affordable, community version is free of charge and premium version is only 129 EUR / year.
We use FastReport .NET. It supports SQL Query, presentation of data in datagrid, filtering, export in many popular formats (PDF, DOC, XLS, CSV, DBF, etc.).

Advice on a DB that can be uploaded to a website by a smart client for collecting survey feedback

I'm hoping you can help.
I'm looking for a zero config multi-user datbase that my winforms application can easily upload to a webserver folder (together with 1 or 2 classic asp pages) and am looking for some suggestions/recommendations.
The idea is that the database will be used to collect feedback entered by people filling in the asp pages. The pages will write to the database using javascript.
The database will subsequently be downloaded again for processing once the responses are in.
In Summary:
It will mostly run in MS Windows environments.
I have a modest budget for this and do not mind paying for such a database.
No runtime licensing costs.
Should be xcopy - Once uploaded to a website folder it should be operational.
It should not have a dotnet CLR dependency.
It should support a resonable level of concurrent access. Average respondent count would be around 20-30 but one never knows.
Should be a reasonable size so that uploads/downloads to and from the site will be reasonably fast.
Would appreciate your suggestions/comments
Many thanks
Abz
To clarify - this is a desktop commercial application for feedback management in a vertical market. It uses SQL Server as the backing store.
The application currently provides feedback management from email and paper feedback. I now want to add web feedback capability. Getting users to to make their SQL servers accessible to a website is not at option at this time as I am want to make getting up and running as painless as possible.
I intend to release a web based implementation of the software in the near future but for now am looking at the above as a pragmatic way to provide web based feedback collection.
SQLite comes to mind. It meets all of your stated requirements, is open source, and has a liberal license (public domain).
http://sqlite.org/
I would use 'normal' database (say MySql, Postgresql, Firebird, etc.) on server. Instead of copying files to server your winforms application would create custom tables (or even custom databases). After collecting data you could just get it back to your application using plain old SQL.
why reinvent the wheel ? If you want to collect feedback and stuffs from users of your app and if they are connected to internet, it might be a better idea - and in the long term cheaper - to use a service like wufoo. We recently switched from homegrown setup to wufoo and are very pleased. Check it out.
Otherwise you might want to take a look at sqlite orfirebird. Both of them are very robust, and have ADO.NET providers. Firebird scales from a single user to full blown client server system and has no .NET dependency.
If you really don't want a DB/SQL Solution, you could try simple text files and ftp/xcopy files down and parse them into the back-office server as needed. ASP/VBScript or ASP.NET can create the files to store the basic feedback comments. Need to consider security of course!

Is there any way to impersonate a specific database engine while running another one?

This is something I would like to see while doing my day today programming works, But I've never seen such application yet. You input is highly appreciated.
Lets say we have an application that needs MSSQL server as DBMS. And suppose you just need to install it and do something. (i.e You are not going to deply it in production servers etc.)
In such a case it might be an overhead, of installing MSSQL first. I am suggesing something like a software bridge that can use another DBMS to store data. In other words the application "sees" an MSSQL instance but underneath that it might be Access. The bridge sholud do some sort of conversion.
Another example : You have MSSQL but a certain application needs Oracle. You have to purchase Oracle then. But with a something like a bridge, You can put information into your MSSQL DBMS. The bridge listens to port 1521 like Oracle so application "Thinks" there is an oracle installation.
Is it an idea that cannot be implemented?
Are there any such applications?
If so what are they?
Thanks... :)
Adding a Clarification : The application might be from a third party. You don't have any knowledge on internal architecture of that. you just know it uses a certain DBMS. I am trying to use a different DBMS other than the third party software needs.
Application usually don't depend on a specific database server, OR they depend on it for a reason.
If an application asks for oracle, or sql server, or whatever, it's because it relies on the implementation details of this specific vendor to run its SQL, its stored procedures, etc. There's no way you could emulate that with an access database, for example...
If your application just needs to run some very simple SQL (ie basic insert/select statements), it probably uses a standard driver (odbc, ado, etc.), and those drivers can accommodate every major sql database engines. In my experience, "simple applications" don't ask for a specific database vendor.
This is the problem that ODBC was supposed to solve :-) .
But in response to your questions:
Is it an idea that cannot be implemented?
It can be implemented.
It would be tedious and thankless work, and you would have a very limited audiance. In my opinion it's not worth doing.
Are there any such applications?
None that I know of.
If so what are they?
None that I know of.
......
Bringing in Chandrasekar's note in the comments section:
Have a look in a super user's perspective... He has a nice application but he can't use it without some DBMS. But still he is not a programmer to do something. So they need such a product
I agree it has applications, but it has a very limited audiance :) .
What you're proposing is something like the firefox plugin 'ietab', Only you won't have ie installed... so instead of embedding ie, you would need to entirely re-implement ie using firefox's rendering window.
Just my opinion : that's too much effort... It's simpler to just install a second database.
If this application uses ADO to connect to SQL Server and you can modify the connection string, then it's quite easy to use a different database: change the connection string! However, the other database must be able to support all features of SQL Server. Besides, the software was never tested on another database so the application might Crash & Burn.
If you can't change the connection string, or the application doesn't use ADO, things are more complicated and very close to impossible.
I've worked in the past on a project that needed to be reasonable database-independent. The database had to support stored procedures but there weren't any other restrictions. By default, we tried to support both SQL Server and Oracle. (We also supported Interbase but never advertised this.) While we did manage to keep it mostly database-independent, we did have to work around quite a few minor issues. Especially joins in our queries had some nasty problems which we just solved by adding more logic to stored procedures.
"This is the problem that ODBC was supposed to solve :-) ."
And it is the very same problem that SQL was intended to solve too.
It seems to me that the reason why this problem exists is that the world seemingly fails to agree sufficiently on what the data manipulation language/interface ought to look like.
I suspect that if this were solvable, it would already have been done.
The closest I've heard is EnterpriseDB where they have built a layer on top of Postgres so it looks more like Oracle.
But remember these databases have features covered by patents and copyright so there's a limit on how closely a competitor product can imitate the real thing.
It would probably be easier to imitate 'down' than up. For example, MS-Access wouldn't be able to imitate much of the functionality for Oracle or SQL Server, whereas there's a much better chance of SQL Server imitating a simpler DB like Access.
Applications usually DO depend on a specific database server. Every database implements things slightly differently - even MSSQL and Sybase, which have a common ancestor.
Any bridge, however well it attempts to abstract the differences, would leave some exposed. These would be likely to create subtle bugs in the application, which might appear initially to work, but then fail, or worse, corrupt data.
Moreover, the application vendor would not support you in such a case - they'd simply say they don't support that use case, and you should remove the bridge and install a proper instance of whatever database it was intended for.
In short, I don't think it's worth the risk of the application malfunctioning subtly, and being left without support, even if the application isn't especially important. If you dislike the underlying database the application uses, choose a different application.

Resources