Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
My SQL Server database is getting bigger and bigger. Nowadays, with 2 GB, it's increasing a lot with many data. I have some table with a lot of data, like millions. These data are very important for SELECTS, like graphics and reports.
I expect that in one more year, I'll have about 5-6 million rows in one of the tables. I have indexes, the database is well organized... my unique worry is about the time that it will take to generate some reports and so on...
How to find data, SUM, COUNT, check 'n' variables based on columns, in so big tables?
What can you suggest? Is there a way to reorganize or split tables? I'm worried in the situation to use always the better manner and make everything look OK.
If it's an ordinary transactional database, you can go for a data warehousing solution for your reporting purposes.
Data warehouses are usually more efficient in these type of situations.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 days ago.
Improve this question
I'm designing quite simple database for storing the hotel rooms reservations. The Reservation object in the database has StartDate, EndDate, RoomId, and ClienId fields. The most frequent query to the database will be to retrieve reservations for certain period of time (e.g. find all reservations in December 2024). So, with time I assume that there will be quite alot of reservations (maybe 10000-25000), and of course I can just delete old reservations, but theoretically, is it possible for the database to swiftly execute this query in such a big data? As I understand, database will automatically organise all reservations by StartDate. Am I right? Should I even wary about that, or all modern databases will do all this "dirty work" for me? What articles/books can you recommend for me to read about databases performance?
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
Basically my application needs to dump data daily into a database. But for any data written down, there is no need to update.
Hence, is appending to csv or json file sufficient for the purpose. Or it will be more computationally efficient to write in standard SQL?
Edit
Use-Case Update
I am expecting to store one entry of for each particular activity count daily. There are about 6-8 activities.
It is exactly like a log in some sense. I would like to perform some analysis with the trend of activities for example. There is no relations between different activities though.
If say in some cases there might be a need for update, would that imply a proper database will be more suitable rather than text file?
It depends on the nature of the data, but there may be another style of database other than an SQL one which could be suitable, like MongoDB which essentially stores JSON objects.
SQL is great when you need entities to have relationships to each other, or if you can take advantage of the type of select queries it can provide you with.
Database systems do have some overhead and could have some gotchas you might not expect, like loading up a heap of crap into memory so it's ready to be searched.
But storing text files can have drawbacks, like it might become difficult to manage your data in the future.
It basically sounds like your use-case is similar to logging, in which case dumping it into a file is fine.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
Often we come across some small insignificant (debatable) tables left out as stand alone. Although they are used in joins (sparingly) but still developers don't bother to relate them.
May be too many References made the inserts slow.
This leads to this question :
As a thumb rule should we relate all the tables in the database ? If no then where to draw the line?
thanks
Foreign Keys are not always a negative impact to performance, they can be a positive impact as well. Database relationships do more than just ensure referential integrity, they also help teach SQL Server about the nature of your data. The fact that two fields are related can give clues as to the cardinality of your queries and thus the optimizer actually takes these relationships into consideration when it's estimating the cost of your query.
In my opinion, if two fields are related in your database, they should have a defined relationship. In general, the more you can teach SQL Server about your data (not just relationships, but CHECK constraints as well), the better it will be at generating efficient query plans. Of course like anything in SQL Server, there are exceptions to the rule, but if you want a rule of thumb, I would lean toward defining all the relationships.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I've been researching on SQL Cursors recently and a colleague of mine said that Cursors are best used for auditing. I tried to look for materials over in the internet but no luck.
Can anyone explain why Cursor is good for auditing despite its disadvantages?
Like any task, it's about picking the right tool for the job. Some disparage the use of cursors due to obviously bad examples of their use, but cursors have their place. They are particularly useful for subsetting data and for reducing code redundancy:
Primarily, I use cursors to perform tasks on subsets of very large datasets, ie, banking data. With billions of records there are some operations you wouldn't want to do all at once, so looping through by day is a good option. There are other methods of iterating through subsets, but a cursor performs well at this task, it's still set-based operations, just on smaller sets.
Cursors are also great for looping through multiple tables/fields in a database, no need to re-write a procedure for multiple tables if it's going to be doing the same thing in each table, or if you are consistently working on a variety of databases. For example, I had need to analyze a multitude of various log files generated by multiple systems, but they all had date and ip fields. Trivial to have a cursor loop through each of the tables and combine all relevant data into one spot.
I wouldn't use a cursor to perform row by row actions unless necessary, and while I can't think of a use-case off the top of my head I'm sure they exist.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I was at a .NET development group meeting a couple weeks ago and the speaker was extolling the virtues of NoSQL and how even relational data doesn't have to be stored relationally if you just index lots of data. So, my questions are: was he blowing smoke? How does one craft an index to be more efficient than the last? Does indexing just logically store the information in a table in a logical format i.e. alphabetically?
Well relational data is needed more for data integrity than indexing. Speed is not the only consideration when choosing a database. SQL Server and other enterpise databases can perform very well if they are designed by people who know what they are doing. Unforuntately most relational databases are designed by data amateurs and their performance reflects that.
NoSQL databases and relational database are used for different things. I would never consider putting a financial application in noSQL for instance because of the need for data integrity and internal controls to prevent fraud and ensure records are consistent and correct. However a website where data quality doesnt matter so much (think Google - who would notice if they failed to serve up every single website that mentions Bill Gates in a query) then yes it is a good choice.