Related
I was wondering if anyone could point me in the direction of some good resources (web sites, technical articles/journals, books, etc.) related to database monitoring/performance?
I am looking to write a paper that explores what kinds of statistics are useful in database monitoring, and to whom they are useful for.
So, I figured I'd ask the experts ...
(In my research, I will be focusing mostly on a PostgreSQL database, but I am looking to gather resources on many different databases).
Any suggestions would be appreciated, thanks!
ConSol-plugins are the de-facto standard when it comes to database monitoring with Nagios:
Oracle
MySQL
MS SQL Server
DB2
Not sure about PostgreSQL, but I've found www.sql-server-performance.com to be a wonderful resource on (MS SQL Server) database tuning.
Here's a page on that site that lists links to various performance-related topics.
http://www.sql-server-performance.com/tips/all_main.aspx
You may be able to apply some of the practices discussed here to PostgreSQL.
I don't think this is a question with a proper answer because your question tries to cover entire database programming domain in one question.
In most cases the database providers release a good set of tools for monitoring analysis purposes. Example : Oracle. You may use them as well.
And a simple google search gave me a hundreds of good resources that seems to be comply with your question.
Example
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
If someone knew little about databases and wanted to learn about them from scratch, which database would you recommend learning with and why?
MySQL seems ubiquitous, but are there others that are more modern that have learned lessons from the past, or others that are simply nicer or more logical to work with?
Universal compatibility/libraries is not a big concern, unless it is something truly obscure. Mac (Unix) compatibility is a must.
If you just want to be learning the SQL language, and not database administration, I would recommend working with SQLite. If you're on a Mac, it should already be installed. It is a much simpler system than most RDBMSes; there is no server to set up, and client to connect to the server. There are no directories of cryptic files, or anything of the sort. To get started, you can just type:
sqlite3 mydatabase.db
And start working with it. It's so much lighter weight and easier to set up and use than the other database systems that I think it's a good choice for a beginner.
Now, SQLite is a fairly small and lightweight language. If you need be getting into any kind of really complex queries and data mining, I would recommend PostgreSQL. It has a fairly advanced query optimizer, and a pretty long list of SQL features.
And if you want to learn a database as something to use for back-end storage for web programming or something of the sort, MySQL is what I'd choose. It's ubiquitous, supported by almost any web host, and it's pretty fast for very simple queries and updates, which is generally what you need for a web system. It has some real gotchas to avoid when setting it up; you have to choose between several different storage engines, and it can take a lot of work to convince it to actually work with Unicode data. But it's good to learn mostly for its ubiquity.
From what I've seen (at least on the web), MySQL and PostgreSQL are the most ubiquitous free database systems. If you're considering learning one of them, check this comparison out.
You may also want to consider learning SQLite, a "self-contained, embeddable, zero-configuration SQL database engine." It's really easy to get up and going, stored in a single file, and as its description says, has no complicated configuration. SQLite has proved enormously popular as a persistent data store for local apps on the desktop/iPhone. If you're going down this route on a Mac/iPhone, you may also want to check out Core Data, which is an abstraction layer Apple developed on top of SQLite(but can work with pretty much any DB), to simplify working with a database. As a bonus, Core Data includes a nice GUI for forming relationships and entities. You can check out this tutorial for more information.
If you really, truly, want to "learn from scratch", then theory is the first thing to learn. And that means : NOT products, not any. Not DB2, not MySQL, not oracle, not any-of-them.
Hugh Darwen has a freely available e-book entitled "An Introduction to Relational Database Theory". The material is quite "accessible" and quite unlike most other theory textbooks. It's also the accompanying textbook for his university course on database technology.
Chris Date has several books, of which "Introduction to database systems" is the most comprehensive, also the standard textbook in the field, but maybe a little too abstract for some.
If you think that all you need is "just to know a product" and that you can do equally well "without all that theory", then in that case, please disregard this response, because the wording of your question is dishonest.
Sad thing about databases is that each and every one works bit differently. I would most likely pick MySQL first and play with it a bit. Then get PostgreSQL and do the same.
If you need to use databases in corporate environment then I would aim to test also Oracle and SQL Server which both have express versions that can be installed free for yourself.
http://www.microsoft.com/express/sql/download/
http://www.oracle.com/technology/software/products/database/index.html
At start all databases are more or less confusing but I would pick MySQL as first because it can do most of the basic functionalities and has a lot of help available.
I'd go with mysql. It's easy to setup, easy to mess around with via with mysql client, and it's well documented. If you're just starting out, you probably won't need most of the features offered by other databases, like stored procedures and the like.
First of all, MySQL is both ubiquitous and modern.
ANSI SQL is more or less the same in all RDBMs, so you can learn any of them and you'll be good.
Once you've mastered ANSI SQL, then all you've got left is the localized solutions for each one of them, which won't be portable to other systems - and so, totally discouraged to use, unless they simplify your tasks in a way justifying it.
MySQL, PostgreSQL, SQLite - pick one. PostgreSQL is more like Oracle, and in my opinion a bit more mature. It's had stored procedures, triggers, and referential integrity longer than MySQL has. I'll admit that I have both installed, but I use MySQL more often because it's quick and easy.
But do be aware that non-SQL alternatives are out there and growing in importance. BigTable, object databases like db4o, are worth being aware of. "No SQL" is out there.
If you are just getting your feet wet, MYSQL is a great one to start with. Easy install on any platform, great community support and lots of free tools to work with (SQLYog is a favorite of mine).
I agree that theory is very important. Depending on how you learn best, digging in and tinkering may be the thing to do before you try to absorb 40 years of thought on relational systems.
Codd and Date are legends in the field and can help you understand the broader points of relational theory, but are hard to absorb before you have context for the topic.
If you are looking for more pragmatic/immediate guidance, I'd suggest a book like "Databases for Mere Mortals" and anything written by Joe Celko.
Once you get comfortable with the basics, there are lots of other platforms to explore as well. As mentioned above, SQLLite and PostGres are two other great choices for the Mac OS.
If you want to learn SQL : the best way is to choose database who implement more features of the SQL standard. So I would recommand Firebird or PostgreSQL
I might be "sidetracking" a bit with this answer, but I think we're in the same situation!
Check out the "The Manga Guide to Databases"! I haven't read it myself yet, but it's on the way in the mail as we speak! I've heard good things about it from friends and colleges, and it's got some good reviews as well. Albeit a bit "controversial," it's supposed to be a fun and surprisingly in-depth introduction to fundamental techniques and principles!
Alex wrote: "Reading a textbook without incrementally testing your knowledge on an actual database is not going to produce good results for the majority of people."
My book and my university course both use Dave Vooorhis's Rel for that very purpose.
Hugh
A database in the cloud: Amazon EC2, Google App Engine or Microsoft Azure
Since I took the basic undergrad course in databases design and SQL I haven't really touched anything like this.
So my question is - how would the database schema for a site like this one would usually look like? What are you generally expected to find? For instance, how are questions and answers stored?
Are there some tools which allow you to design it? or is it just something the devs come up with?
Stack Overflow used the MediaWiki database schema as a template before they went through a database refactoring a few weeks back.
There are few principles in database design which im sure you pretty much know if you took that course. They all say basically that your data should not be duplicated across multiple tables and all your columns should be integral to the table they appear in. Then there are common entities aplied in all software design like objects and relations. Managing them is easiest part because it comes instinctively. Then there is optimization for scalability and performance which shouldn't be hard if first steps are done right. And this is usually done together with software team which is writing code for your db.
You can use tools to design a database, but they are normally just templates for creating the right shapes in a diagram.
The logical structure will be designed in the same way arn architect would design a building, using their best knowledge and experiences.
Also, always "work in pencil" until you are happy.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
How do SQL developers go about keeping up on current techniques and trends in the SQL world? Are there any blogs, books, articles, techniques, etc that are being used to keep up to date and in the know?
There are a lot of opportunities out their for OO, procedural, and functional programmers to take part in a variety of open source projects, but it seems to me that the FOSS avenue is a bit more closed for SQL developers.
Thoughts?
Find challenging questions that test your TRANSACT-SQL knowledge ... personally I enjoy Joe Celko's SQL Puzzles and Answers.
Joe Celko's SQL Puzzles and Answers http://ecx.images-amazon.com/images/I/51DTJ099P7L._SL500_BO2,204,203,200_PIsitb-dp-500-arrow,TopRight,45,-64_OU01_AA240_SH20_.jpg
The thing about the SQL language is that it is pretty much a static target. Pretty soon you are looking at increasing your understanding of set theory and the problem domain itself rather than the details of the language.
The real meat is on either side of the language, in either the databases themselves (how to store, retrieve, and organize large data sets) or in the applications (with ORMs and such)
I skimmed the answers and apparently nobody has mentioned Stephane Faroult's work.
I strongly suggest you should consider "The Art of SQL" (http://www.amazon.com/Art-SQL-Stephane-Faroult/dp/0596008945), I found it really interesting and - amazingly enough, even fun to read.
How about http://sqlblog.com/
I improve by analyzing slow and complex queries and looking for ways to improve them. This can be done in SQL Server by analyzing the Query Plan tools and looking for bottlenecks. Also I find the Visual Quickstart Guide guide to be good for quick reference.
Joe Celko's SQL Puzzles and Answers and SQL for Smarties are the two best generic SQL books out there. Both are great sources to give you ideas for that tricky problem you used to think you needed a cursor or some client library to accomplish. For any truly interested SQL geek, these book are also pretty good for casual reading rather than as a mere desk reference. Two thumbs up.
While not a SQL Server expert, in general I find that community based events are great ways to keep up on current patterns. The underlying result of participating in a community of developers/DBAs/Marketing Pros/insert profession here is that you are learning new thought patterns and excercising critical thought. This is a great way to grow as whatever professional you are.
There aren't current techniques and trends in SQL. There's only that stuff you should already know but don't. The proper way to learn that stuff, is pain... so much pain.
Join a mailing list for the DB flavour you use...or lurk on stackoverflow ;)
Most "current" stuff is not SQL itself, but how the database stores the information, and how to retrieve it more quickly. Check out this other thread: What are some references, lessons and or best practices for SQL optimization training
The only real bleeding edge is in query planning, index structures, sort algorithms, things like that, not the SQL itself.
The fact that you asked this question is already a good sign. Avoiding complacency is "piece of advice #1". There is no substitute for writing and optimizing SQL. Practical use is the best way to stay sharp, but there is a risk of a "forest for the trees" scenario, where we tend to use what is comfortable and familiar. Trying new tactics, examining new approaches, and looking for new ways to train our brains to think about sets, SQL, relational theory, and staying on top of new developments in the particular dialects we employ are all hallmarks of good SQL developers.
There are many good blogs out there these days. I work mostly in the Microsoft arena, so I like SQLTeam.com.
Usenet is a good place to hang out and make a contribution. There are many SQL-related newsgroups. Often, you will find that working on someone else's problem helps you learn a new tactic or forces you to research a dusty corner of the language that you do not encounter every day. ISPs seem destined to shut all of the Usenet down, though, because of nefarious use, so this one may be going the way of the Dodo bird.
Also, some IRC servers have a vibrant sql channels where you can make the same sort of a difference (just take a thick skin with you).
Lastly, this very website might be another place to hang, where you can read over the answers to difficult questions, see how that might apply in your own world, practice the techniques, and internalize them. Contribute too, because seeing how others vote your solutions up or down is 100% pure honest feedback.
Of course, there are many wonderful books out there, too. Anything by Celko is a winner, and on the SQL Server side, Kalen Delaney and Ron Soukup have written some winners.
Best thing I've run into is working on other people's SQL code. Especially legacy business code. You want to test your skills against something, start changing some "voodoo code" that no one else understands. :)
Beyond that, I just try to keep an eye on changes with new releases of SQL and see if there's anything I can take advantage of.
gleam tips when using phpmyadmin
it's nice and verbose
Here is one with some interesting SSIS information.
http://blogs.conchango.com/jamiethomson/default.aspx
There is also some good information in the Wiki here:
http://wiki.lessthandot.com/index.php/Main_Page
For those who say SQL never changes, SQL Server 2005 and 2008 have some huge changes in the T-SQl that will help solve some difficult problems that were horrible to do in SQL Server 2000 and are much easier once you learn the new syntax, so yes there is stuff to keep up with.
Also performance tuning and SSIS are extremely complex subjects with much to learn.
I do find that developers who choose not to learn advanced SQL skills tend to write poorly performing SQL code and once the number of records grows in their databases, the applications they wrote tend to become glacially slow and very difficult to fix at that point. Right now I'm working with developers to fix some bad code they wrote that is causing timeouts on the site on virtually every query. Obviously, this is now an emergency and it would have been easy to write the code in a more efficeint manner at the start if the developer had better SQL skills.
I've never heard of the term "SQL developer." SQL should be a skill in your toolbox, like sorting, whatever framework you like, JavaScript, and so on. The best way to continue to improve your SQL skills to continue using it.
As a developer, and not a DBA, I keep an eye on various developer resources, and that often is DB related, but I don't specifically 'try to keep up'.
I know plenty, but I also know that there is so much more that there is to know. And in every project I have to learn something new. And every project also involves me taking a different approach to a similar task I'd encountered in the past.
Should I ever get to the stage where I think I'm doing the same things all the time, perhaps I'll make a concious effort to take specific steps. But currently, and for the foreseeable future, I'm learning organically, on-the-job, and as my projects dictate.
Reading:
Books - Celko (also read across to some Oracle-biased books)
Blogs - the above mentioned, plus SSWUG
Webinars and Conference - Best way to keep up with vendor-specific stuff like
SSIS/SSRS/SSAS
Practice:
Improving code (mine and others)
Refactoring
Mentoring/training other developers
Honestly, it's one of those things that you just get better at with time. Read as much as you can to know what's possible. Some things will take a while to really understand. I was scared off by sub queries for a long time until I pretty much had no choice but to use them.
When you get more experiance and need to do more complex things, you will just learn your way.
SqlServerCentral - great source of articles, scripts, advice
Unfortunately to access the articles you need to register (it is free though)
I guess one thing they could learn from StackOverflow is to remove login barrier
We've written a full tutorial, and you can test your SQL skills at a separate site (also created by our me, in the interests of full declaration).
SQL developers, or DBAs?
Aside from learning different dialects of SQL (Oracle, SQL Server, etc) in your day to day work, SQL doesn't actually change all that much. Sure you can bring in more advanced concepts as you develop your skills, work out where to implement stored procedures, etc, but in the end it's just SQL. The most important thing is to get your schema correct and maintainable.
Now administering the databases is a whole different thing, with a range of tools, and the database software itself getting updated every few years. Oracle at least have newsletters and websites and magazines that presumably include lots of information and examples and best practice scenarios.
To be honest, I don't see much a need for extreme SQL skills. Once I can create transactions (for DB consistency) and basic triggers (for cross-table consistency), I'm usually fine keeping program logic... in the program, and not putting it into whatever database I'm using. I've not found much depth to SQL worth investigating for a lifetime, unlike general programming, which keeps expanding in depth.
I am quite happy to code out tables by hand when making a database but it's not the easiest way to convey information about a database to someone else, especially someone that's not so comfortable coding the tables via a script and would instead use something such at phpMyAdmin.
Is there thus a free program (for me to use it it'll have to work on a Mac but feel free to suggest PC apps for others with the same Q) or script (preferably in PHP or Python) that allows you to design database structure and will then output either a basic diagram or the code as chosen by the user?
Well on the PC you can use MS Visio to produce a DB Entity diagram.
It will even reverse engineer one from an existing Database.
A pain to set-up the first time you use it, but quite handy thereafter.
Open System Architect has some potential. Its very similar to Visio.
I'm a big fan of ARGO UML from Tigris.org. Draws nice pictures using standard UML notation. It does some code generation, but mostly Java classes, which isn't SQL DDL, so that may not be close enough to what you want to do.
You can look at the Data Modeling Tools list and see if anything there is better than Argo UML. Many of the items on this list are free or cheap.
Also, if you're using Eclipse or NetBeans, there are many design plug-ins, some of which may have the features you're looking for.
I use the aptly named Database Design Tool. It's extremely simple and unfortunatly it's developed any more, however. It's the best tool I've come across that is free and at the end of designing your tables, it generates the T-SQL for you. It's also language independent.
You could try out MySQL Workbench which originates in the open source dbdesigner. There's a free community edition available. You can design the database via er-diagrams or reverse engineer an existing database.
MySQL Workbench is the best DB design tool that I've tried
I'm currently checking out SQL Power Architect (both w/ PostgreSQL and Mysql - but it also supports other vendors) and it definitely seems promising. Does both forward and backward SQL engineering. The Community Edition is open source and cross platform (Java). You can check it out yourself: http://code.google.com/p/power-architect/
When strictly dealing w/ MySQL so far I've otherwise used MySQL Workbench, http://wb.mysql.com/ which performed reliably.
I always have enjoyed Eclipse. There are a few plugins for it that look like they will do what you want.
SchemaBank (a web-based SaaS vendor) can turn your ER design into SQL statements for MySQL and PG. Can't do graphics export yet though. The nice thing is you don't need to install anything ('cos its browser-based) and it costs virtually nothing. You should be able to share your design to other people too.
SQLDeveloper from Oracle can work with Oracle and MySQL database.
http://www.oracle.com/us/corporate/press/020861