Quick q, could be a silly one given my (lack of) findings on Google so far.
I have a Database. In this database is a Table with some Data. The Data is a large BLOB but can't be compressed (for reasons out of my control).
I have an Application that talks to this Database. I would really like to be able to ensure that the Data is compressed during transit.
As I understand it, the Database Provider would handle compression etc.
Is this the case? Are there settings on common ones, say SQL Server to enable compression?
For SQL Server, I found this "connect" entry, but no: I don't think TDS is currently compressed. You could (although I don't like it much) use SQL-CLR to compress it in .NET code, but it could have too much overhead.
I know it isn't an option in this case (from the question), but it is usually preferable to store BLOBs the way you want to get them. So if you want to get them compressed, store them compressed. SQL isn't a good tool for manipulating binary ;-p Such a strategy also means that you aren't using vendor-specific features - just the ability to store an opaque BLOB.
If your database access layer does not provide compression, you can set up a VPN link between the database server and the application host. Most serious VPN solutions compress data in transit. OpenVPN is a simple and easy to set up solution for quickly creating a tunnel. Data is compressed in transit. Probably won't be as efficient as a native compression, but it's a possible solution. And you get encryption thrown in for free :).
SQL Server 2008 is the first version of SQL Server to natively support compression of backups. Pre 2008, you need to do it with third party products.
Related
I want to build a wpf app (with mvvm dp) and I need to manage a list of customers (200 customers maximum)
What is the recommended way to handle this? sql (mySql, sqlServer) or other way (xml, excel, access)?
I can see advantage in xml because its not required addition installation
If you've got multiple users on different machines, and those machines have a persistent, shared network connection (they're on the same network, for instance), then you should prefer to use a central database, like (as you mention) SQL Server or MySQL. The latter is free, and the former is available in a free edition (Express) for light workloads.
If you don't need to share data between the different users of your application (you don't mind if their customer data get out of sync), then look at lightweight, embedded databases, like SQLite or SQL Server Compact Edition. Very simple and easy to set up and maintain, but the data stays present on a single machine.
You could build an infrastructure around manual synchronization between the different machines, sort of like how iTunes synchronizes data between your computer and iPhone whenever you plug the latter in, but trust me, that's a lot of work that's best avoided if possible. Databases have a lot of built-in multiple-users-doing-stuff-at-the-same-time logic that you want to leverage if at all possible.
If the only thing you need to persist is a list of 200the items and you don't see there as being any other persistent data requirements in the future, I would say that a flat file is plenty. However, this also depends on how often that small data will be retrieved.
I am a huge proponent of SQL Server, and Express is free. But you don't want to overengineer the problem. Unless I misunderstood your question, that is my evaluation.
Is there seriously no way of using a shared access non-server driven database file format without having to use an SQL Server? The Entity Framework is great, and it's not until I've completely finished designing my database model, getting SQL Server Compact Edition 4.0 to work with Visual Studio that I find out that it basically cannot be run off a network drive and be used by multiple users. I appreciate I should have done some research!
The only other way as far as I can tell is to have to set up an SQL server, something which I doubt I would be able to do. I'm searching for possible ways to use it with Access databases (which can be shared on a network drive) but this seems either difficult or impossible.
Would I have to go back to typed DataSets or even manually coding the SQL code?
Another alternative is to try using SQL
Install SQL Server express. Access is not supported by EF at all and my experience with file based databases (Access, SQL Server CE) is mostly:
If you need some very small mostly readonly data to persist in database you can use them (good for code tables but in the same time such data can be simply stored in XML).
If you expect some concurrent traffic and often writing into DB + larger data sets their performance and usability drops quickly. They are mostly useful for local storage for single user.
I'm not sure how this relates for example to SQLite. To generate database from model for SQLite you need special T4 template (using correct SQL syntax).
Have you tried SQLite? It has a SQL provider, and as far as I know EF supports any provider. Since it's file-based, that might be a plausible solution. It's also free.
We have stored all media in Sql Filestream, but now we'll need Video and Audio streaming... Will this be possible with Sql Filestream or will I have to take all of the Video and Audio out of the database?
Which technology would you use to enable Video/Audio Streaming?
WebORB
FluorineFX
Wowza (way better I think than the first two)
IIS Media (haven't looked into this yet)
When using IIS Media its not possible to store the data in a SQL Fielstream.
For further details check here.
It's possibly very similar with the rest of your suggested solutions, since all of them need to re encode the material to enable streaming (if its not in the necessary format already).
You actually have 2 problems:
Re encoding the videos into a format
that enables you to stream it via
the server platform you choose, just
for this part you need to extract
the files from the db since the
encoding tools can't be fed from a
database, even if its a SQL FileStream
Store the encoded files
somewhere the media servers can
access them, again they don't allow
a SQL Server as a data soure, they
probably have their own storing
infrastructure or use the file
system.
Conclusion:
The FileStream is extremely helpful when you have full control over server/client, but sadly not in your case.
You will probably have to extract all files from the DB.
The FileTable feature in SQL Server "Denali" (not yet released) is designed specifically for this scenario (amongst others).
There's a good overview link here: Using FileTables to Manage Unstructured FILESTREAM Data.
This will allow you to directly access and play these files through a provided UNC path without requiring any changes to the application, so you can use any of the above mentioned streaming servers.
I have a desktop application that needs to read data from a database, both installed on client computer.
That data in database must be encrypted, to deny client access, and protect data.
I need to know what kind of database to use that can supports encryption, because the amount of data in some tables will be very huge, I will need efficiency too.
I read some about this on web and perhaps SQL Server Compact Edition will be a good choice.
Can someone help me with this point?
Thanks
SQL Server 2008 includes a feature called Transparent Data Encryption which may do what you need. I don't know if Compact Edition contains this feature. A little googling with those keywords should get you started. I don't normally advocate using SQL Server, but in this case I'm not aware of any similar features from other DB vendors (but they must exist...).
That said, what do you hope to gain by encrypting the data? If this is entirely client-side, then the client is going to have the key to decrypt the data. At best, you're making a slight hurdle for people to get at your database contents. It won't be secure by any meaningful definition.
You cannot prevent a determined attacker from accessing the decryption key and accessing the database. This is effectively a copy-protection scheme, and they are all broken.
Update: The question states, "That data in database must be encrypted, to deny client access, and protect data." If the client has access to any application that can access the database, he has access to the key used by the application, and can bypass the application to access the database directly.
If the inescapable logic has no appeal, consider the anecdotal evidence of failed copy-protection schemes attempted to protect music, games, and other digital assets.
SQLite has an Encryption Extension (SEE) that allows an application to read and write encrypted database files: http://www.hwaci.com/sw/sqlite/see.html
This DB may be a good fit for a desktop application, and is widely used. For example, I believe FireFox uses it internally.
SQL Server CE has no support for encryption whatsoever. You can at best encrypt the database file using the host OS encryption facilities. The file level encryption does not work for databases because to read page X in the file one has to decrypt all pages 1...X-1 to get the encryption key in the appropriate state (reach the proper CBC block state). BitLocker on the other hand works fine as it can decrypt/encrypt pages in the file individually. But BitLocker is a partition level option, not file level. These general considerations apply to any plan to encrypt the entire database file, irrelevant of the product involved (SQL CE, SQL Express, MySQL, Access, anything).
SQL Server has database level encryption. The easiest to use is TDE, Transparent Data Encryption but this requires Enterprise Edition. The other option is to use the cryptographic functions and manage the encryption yourself. Hardly easy to use, but is available in the free Express edition.
Might I recommend SQLCipher? It's a free and open-source implementation of SQLite that supports transparent, page-level encryption. It's similar to SEE, it's under active development, and has experimental support for a number of different ciphers, as it uses OpenSSL for some of its implementation. Full disclosure: I'm one of the developers! We've got a tutorial on using it in iPhone applications that will give you a basic idea of how it works, and there's a comment thread that expands on some related topics. Obviously, using it in Visual Studio will be a tad different than XCode, but you should be able to hook up the linking and get it going in a Windows environment.
Is it possible to store Subversion meta data in SQL Server? We want a SQL Server based Subversion.
There isn't anyway to do that right now. Subversion lets you choose between FSFS and Berkeley DB, with FSFS being the better choice for most people. (Berkeley DB has some issues around getting corrupt).
Is there a reason you want to store the data in SQL Server?
I with James Avery - why do you want to store the data in an SQL Server? If you're wanting someway to interface with the underlying subversion data from a .NET library, check out SharpSVN: http://sharpsvn.open.collab.net/.
Subversion used to have BDB (Berkley Database) as its core storage. Most people moved away from that because it was hard to back up and corruption sometimes became an issue. I wouldn't recommend it but I am sure you could go look at the BDB code and have it hook up to a SQL Server.
Why use a backing store that you can so easily corrupt? Sure, you can corrupt the filesystem store, but most people have enough sense not to touch it.
As it is, future versions of SVN are looking to store revprop data in a sqlite db instead of files. If they do, then you should be able to access that in much the same way, but for the mutable revprops only. I'm not sure if they intend to restrict access to it to the svn libraries of allow any other process to read and write to it.
Of course, the biggest reason SQL server is not suitable for subversion is that it would completely break its ability to be cross-platform.