As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
The National Park Service's Natural Sounds Program collects multiple terabytes of data each year measuring soundscapes. In your opinion, what is best available scripting language to manage massive amounts of files and file types? We would like to easily design and run efficient user-friendly scripts to search for and retrieve/create copies of files that may be located in different directories according a single static hierarchy. The OS will most likely be windows. Thanks!
Use the one your developers are most familiar with. The productivity gains you'll get from that will almost certainly beat out any advantages that one language may have over another.
Use Python. It's easy to learn. Everyone can easily convert.
The size of the files doesn't much matter when you're searching directories or searching for metadata outside the files. Even so, you rarely need to read an entire sound sample file to strip off the metadata.
Also, if you're doing this frequently, you might want to consider
Extract all the metadata to a relational database.
Use the relational database as a complex "index" to the sound sample files.
Each file add or change would be done through an application that synchronized file changes with database updates to assure that the database index actually matches the filesystem.
The bulk of your searches might become SQL queries.
I don't really know what your are going to be looking for in a scripting language, but Eric is right that you should use something all your developers are familiar with. However, if you don't have developers (yet) and are designing the project (and team) from the ground up, C++ or .Net (C# or VB).
While C++ offers more powerful programming and performance, C# and VB.Net offer quicker production. Regardless of .Net's production advantage, I would think that for massive amounts of files & file types, you will have the best overall satisfaction from C++. In my opinion, the best user friendly design requires very little user input other than clicking buttons or selecting options from a list.
Related
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Are there cases where database designers are not allowed to know the details of the data? I am looking for real-world examples to learn from — please.
I can't help but tell a story about database nightmares. One of the worst was when Amazon was first growing. Initially they only sold books, then expanded to music, and then to many other things.
For a period of about two years, Amazon would announce a new market every two or three months -- children's clothing, housewares, garden supplies, food, and so on. The database folks were tasked with developing and supporting the systems for the product lines. However, Amazon considered the new product announcements to be highly, highly secret.
In particular, the data warehouse people would be kept further from the loop. Sometimes, they would find out about a new line of business by reading news -- and then have to support it in the data warehouse.
So, they had to develop a flexible database to meet unannounced business needs.
In any business environment, there are new needs that arise. I would suggest a book such as Ralph Kimball's "Data Warehouse Toolkit" for more background on how to develop a fairly robust system.
I am currently working at a company that stores very private personal information. I am not allowed access to the production database. For our development and test environments, we replace all names, addresses, and other personal information with randomly generated information.
Yes, I've often seen databases allow for custom data to be defined by the user. The basic approach is to design a meta data system for your database. Then allow entities associations with custom fields. You wouldn't want to do this for all your data, otherwise you'll just end up with a database in a database, but for dynamically adding a number of custom fields this approach works well.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I would like to learn Database systems implementation in Depth. Is there an open-source simple implementation of database for educational purpose that I can go through the code? Like there are a lot of OS implementation (Minix, Pintos...). I am wondering if there are similar systems for database education as well.
I read a few textbooks and they are mainly focus on theory and concepts.
Thanks a lot!
Alfred
Then find some educational material :)
When i was learning db concept, my professor ask us to code a simple dbms. One important reference is the Redbase:
http://infolab.stanford.edu/~widom/cs346/
Hope that helps.
MySQL, PostgreSQL, SQlite are all opensource. You can find their source code and related documentation.
Also check NoSQL group of databases.
What makes you think implementing a database is simple?
What parts of the database interest you? Storage management? Indexing? Query Language? Query Planning? Transactions?
Modern (even "toy") Relational systems have all of those components, which makes them rather complex from the outset. Other DBs, such dbm based databases are much simpler. Then you have things like Lucene, which is a database for documents and free form text -- conceptually simple but put a lot of effort in to scaling.
You can look at implementations of SPARQL if you're curious about query languages, as they work against RDF triple stores (which aren't super complicated).
There's also things like Prevlayer, which is an in memory database using a concept called prevalence. Probably the simplest of all of them, really when you get down to it.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
So I am working on a mini-project for the summer to keep my coding skills sharp. I will be using the Qt4 and C++ to make a classroom management system for college professors. I just came up with the idea like 10 minutes ago so I don't have much.
One question I have is what is the best way to store student/class/assignment information so that the software could still be portable and used my different schools.
My first guess would be a MySQL database. I need a gurus opinion on this one though.
Since different sites have different database preferences you might wish to use a layer such as ActiveRecord or PDO or ODBC to abstract out the specific database that your end users want to use. This would allow people to deploy onto PostgreSQL or MySQL or whatever they prefer.
A good choice for single-process server systems could be SQLite3. It's not suitable for all systems, but if your system is designed to scale to a few dozen users at most, it'll probably work fine. (The amount of work you'd need to put into a server to make SQLite3 scale into the hundreds or thousands might argue for planning for a database server environment instead.)
http://www.sqlite.org/
might be a good option. It is embeddable so you don't need a specific database instance running wherever you deploy it
also, http://www.microsoft.com/sqlserver/2005/en/us/compact.aspx is an option
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
JavaFX is a new technology introduced by Sun. I would like to know what are the limitations and pitfalls of JavaFX and silverlight.........
The main problems I see with JavaFX are:
It is very immature technology. While everything, as others have noted, must have a start somewhere, JavaFX is very immature technology in a field that has several more mature competitors. That makes all the difference. Flex/AIR and Silverlight are out there already and in active use all over the place. To tackle an entrenched market you need tools that are far superior to what's out there, not just a little superior (if at all).
It is stacked on top of the rather large JRE. The JRE is available on every major platform (and many minor ones) but it is not ubiquitously installed. The same is true of Flex/AIR and Silverlight, of course, but Flash is a lighter-weight solution than the whole JRE+JavaFX and the latter is basically a default on the target platform anyway.
JavaFX is heavily reliant upon the huge infrastructure that surrounds Java. This is a strength, of course, given that if you need it there's a library for it somewhere, but it is also a weakness in that it involves even more bloat than the JRE+JavaFX and starts to introduce you to the wonderful world of JAR Hell.
The final reason to be wary of JavaFX: I prefer it to all of the other currently available RIA technology. (Yes, despite my critiques, I like it better.) If I like something, it is guaranteed to fail. Computing technology is positively littered with technologies I preferred over what was eventually chosen by the market. My liking something is a kiss of death for it.
One of those may not have been entirely serious....
Here is a description of JavaFX disadvantages taken from this website (which also gives JavaFX advantages as well)
JavaFX's lack of maturity is, at the moment, its weak point. For example, it is lacking a lot of JavaFX-native controls (for example, a tree control). Now, it is quite straight-forward to wrap Swing controls and subsequently use them in a JavaFX-project but it's obvious that that is a bit of a temporary measure. Sun has mentioned that they will address this issue in future versions.
Hope this helps.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
What's your favorite open source database design/modeling tool?
I'm looking for one that supports several databases, especially Firebird SQL but I can't find one on Google.
I've used DBDesigner before. It is an open source tool. You might check that out. Not sure if it fits your needs.
Best of luck!
Do you mean design as in 'graphic representation of tables' or just plain old 'engineering kind of design'. If it's the latter, use FlameRobin, version 0.9.0 has just been released.
If it's the former, then use DBDesigner. Yup, that uses Java.
Or maybe you meant something more like MS Access. Then Kexi should be right for you.
S.Lott inserted a comment, but it should be an answer: see the same question.
EDIT
Since it wasn't as obvious as I intended it to be, here follows a verbatim copy of S.Lott's answer in the other question:
I'm a big fan of ARGO UML from Tigris.org. Draws nice pictures
using standard UML notation. It does some code generation, but mostly
Java classes, which isn't SQL DDL, so that may not be close enough to
what you want to do.
You can look at the Data Modelling Tools list and see if anything
there is better than Argo UML. Many of the items on this list are
free or cheap.
Also, if you're using Eclipse or NetBeans, there are many
design plug-ins, some of which may have the features you're looking
for.
The DB Designer Fork project claims that it can generate FireBird sql scripts.
I like Clay Eclipse plugin. I've only used it with MySQL, but it claims Firebird support.
You may want to look at IBExpert Personal Edition. While not open source, this is a very good tool for designing, building, and administering Firebird and InterBase databases.
The Personal Edition is free, but some of the more advanced features are not available. Still, even without the slick extras, the free version is very powerful.