I'm new to both Yesod and Databases so please bear with me this basic question:
I plan to write SQL procedures and store them in .sql files and the Yesod web application just calls these sql file instead of writing queries directly in Haskell.
Is this common practice?
I assume that writing queries in SQL itself is more suitable than writing them in Haskell.
Any comments are welcome.
Is this common practice?
No. The closest you can get to this is use something like rawSql in persistent or use some other low level library. I would recommend you to avoid writing raw sql query unless you have a valid reason to.
I assume that writing queries in SQL itself is more suitable than writing them in Haskell.
No, the whole point of using persistent is to bring type safety to database queries. If you are gonna write them in SQL itself you lose that benefit.
Related
I've never done databases before but I know that you can create them on MySQL, the thing is that I want to be able to write/read a database locally and not have to send/receive any information online as it would be too slow for what I need.
Is there a way to create a stand alone database (possibly on MySQL then downloading it) and write/read from it using LUA?
Thanks for any information, as I say, never touched databases.
Yes it is.
You can download sqlite3, which is a simple SQL relational DB. Basically this type of DB is just a file (.db extension), so you can have it locally on you PC and also exchange it between many machines easily.
From a Lua standpoint, what you need is a library to access your DB and I think LuaSQLIte3 is the best option to go with. Check this SO post for a basic example on this.
IMO using lua in conjunction with sqlite3 is one of the best choices when one wants to have a (light) SQL-based DB locally, without the added "complexity" of a more commercially used DB (e.g.postgres).
I have installed Oracle XE. I build small database every day to practice from command prompt, but now I want to have more. I want to have a bigger database with a lot of different data to practice and make exercises.
So, is possible to get a big data file from somewhere and upload to XE database?
You can't get 'big' data for Oracle Express edition as it is limited to 4GB (10g) or 10GB (11g ).
That said, there are public datasets available. Personally I like the FAA data on registered aircraft owners/operators
As you are practicing with Oracle, perhaps a good solution (which will also generate exactly the data you need) would be to write your own stored procedures to generate your data in a loop (or similar construct).
You could then generate as much as you like whilst also practicing your handling of large datasets and writing of efficient PL/SQL and SQL code.
This way your data will match your current database structure too without having to build a new database matching whichever dataset you download from the web.
IIRC there are sample schemas as HR that can be enabled. See this.
As i couldnt get any satisfying answer to my Question it seems we have to write our own program for that, we are in the design phase and we are thinking which format shall we use to backup the data.
The program will be written in Delphi.
Needed is Exporting/Importing data between Oracle/Informix/Msserver, very important here is the Performance issue, as this program will run on a 1-2 GB Databases. Beside the normal data there are Blobs in the Database which have to be backuped.
We thought of Xml-Data or comma-separated data as both are transparent (which is nice to have), but Blobs must be considered here. Paradox format is not optinal in this case.
Can anybody recommend some performant formats?
Any other Ideas to achieve the same Goal are welcome.
Thanx in Advance.
I use an excellent program called OmegaSync for my backups, but it will only handle Informix via ODBC and not directly. If you find you can use OmegaSync, you'll find its performance to be excellent, because it compares the databases first, and then syncs only the differences. You might want to use this idea if you decide to do the programming yourself if efficiency is your number one goal.
But programming database conversion is very complex as others answers to your question have said. So why not just develop the SQL you need, and do the conversion that way. For example see: Convert Informix Schema to Oracle Schema Or Any Other RDBMS For moving the data, check out sources like: Moving non-informaix data between computers and dbspaces
You can optimize the SQL to what I'm sure will be an adequate speed if you dump and load your data smartly.
DbUnit is a popular tool which can extract and load data in XML format, see
http://www.dbunit.org/faq.html#extract
// partial database export
QueryDataSet partialDataSet = new QueryDataSet(connection);
partialDataSet.addTable("FOO", "SELECT * FROM TABLE WHERE COL='VALUE'");
partialDataSet.addTable("BAR");
FlatXmlDataSet.write(partialDataSet, new FileOutputStream("partial.xml"));
// full database export
IDataSet fullDataSet = connection.createDataSet();
FlatXmlDataSet.write(fullDataSet, new FileOutputStream("full.xml"));
Did you check ODI (Oracle Data Integrator) It has support for lots of source databases. It is able to capture changes from the source databases and integrate them in the target database. It is performant but has a price tag.
Ronald.
The new DBExpress framework give the possibility to exporting/importing data between many databases. you can check this CodeRage session Deep Dive into dbExpress by John Kaster
You should use your own binary format, integrated by (xml for text/streams for Blobs).
If you have to export metadata too and not only data, it could be very complex. There are many subtle (and not so subtle differences) among the databases you're going to use, that such a format should be general enough and the exporting/importing code should be able to translate and map metadata across databases, and because an external application can't write directly to the database internal structures, it would have to generate the db proper DDL to create the data structures.
As long as this is a proprietary format, IMHO its design is the least of your issues, if size and performance are important and the file is read sequentially it would not be difficult to design a binary format.
Anyway import/exports and backups are two different tasks. If you have to backup a database, use its facilities. They usually allow far more control, i.e. point-in-time recovery. If you have to move data across databases that's another issue - I would write just the code to move data, not metadata, pre-creating the required structure in the target database.
You could give Toad (Quest Software) a try.
It supports all your mentioned platforms and can do things like 'Export table data to INSERT statements' on your source platform which can then be run on the target platform.
IIRC there is even some Toad-internal backup-format which might be cross-platform.
Toad Communities:
Toad for ORACLE
Toad for SQL SERVER
Toad for OTHER RDMBS (including Informix)
Some videos about exporting, importing:
YouTube: Toad for Data Analysts v2.7 Export Enhancements
YouTube: Toad for Data Analysts v2.7 Import Enhancements
My database consists of 5 tables with ~10000 rows combined. It takes ~1Mb in SQL Server CE which is on shared folder. The database itself is hierarchical Country-Region-City-Street-Building. I am using Entity Framework 4.
Because the database is small users are able to explore and edit all 2000 Cities in a WPF ListView. But with every approach I tried so far the GUI is sluggish (because of many database round-trips, with dummy data GUI is lightfast). How can I load entire database into memory with one or few database round-trips?
I tried multiple Include() but I noted great performance penalty as described here
Should I write my own ORM-light? I could also use plain ascii CSV files instead of database but it would obviously exclude concurrency.
Honestly, I've done something like this myself, and the answer for me was to copy the whole database locally and work on it.
If you're looking not only to read but also to write, I'd definitely suggest ditching CE and installing one of the Express versions of Sql Server. They are designed for this kind of situation; CE is not*.
*SP1 is better for concurrent access, but over the network will never be performant for large datasets.
I re-asked this question on Microsoft forum and they were kind to give me some guidance:
Basically my question can be restated as following:
read only once from database on application start
do all subsequent queries from local data, not database (for performance)
write to both context and database each time an entity is being added or deleted.
With plain EF it is not possible because each query goes to the database. This implies that I must read data fast on start and then cache it.
Implementation details:
The best way seems to be using ESQL to import data fast and then cache it, for example using entities not connected to context. From my first experiments it seems to work well.
I'm developing an asp.net application with Database factory pattern which allows the application to support both Sql Server and Oracle. I've created an abstract class that has the methods common to Sql Server and Oracle, like the CreateConnection and CreateCommand methods. This class is implemented by SqlServer and Oracle classes. Now, is there an easy way to write in-line sql queries with parameters common to both Sql Server and Oracle. I mean, I understand that we use "#" symbol in Sql Server and ":" in Oracle for parameters. Just for this reason, I'm writing queries twice in each of the class. Is there a way to write such queries common to both the databases? (or interpret the parameters from one common query?)
Thanks.
The only way to write one query that will work for both Oracle and Sql Server is to use only the syntax that is common to both platforms. Once you use features that are different between the two languages (like parameters or joins), you either have to write two different queries or hack together a "translator" class that converts a query from one platform to the other.
I've done a lot of this type of programming (database-agnostic software), and with .Net a relatively pain-free way of doing this is to write your main application to work entirely with ADO.Net DataTables/DataSets, with a wrapper class that handles generating the DataTables from either Oracle or Sql Server tables under-the-hood, and also handles persisting changes made to the DataTables back into Oracle or Sql Server. This approach isolates your DB-specific code in one place, although it's not necessarily a viable approach if the data your application needs access to is large.
You could write some kind of translator, but I would suggest that in some cases you'll need to write db-specific code for performance reasons anyway, so you'll have to put up with the maintenance burden of two versions of some queries.
What is the point of using ORACLE and not using all its non standard functions (analytics, pivots etc) ? ORACLE is a powerful tool.
Other DBs have there own strenght also, so why use the lowest common denominator just to be able to work on ALL of them? You will just lose in performance.
Just pick one DB, and use it fully with all its functionalities !
Pardon my ignorance here, but can't something like an ORM (object relational mapper) work for both SQL and Oracle?
I had similar requirements, to support both Sql Server and Oracle, and summarized my two years of experience with such problems in these articles:
Writing ANSI Standard SQL is not practical.
Think ANSI Standard SQL Is Fully Portable Between Databases? Think Again.