I've done quite some research now on HTML5, but I am still left wondering what would be my best guess to implement local data storage that is truly cross-platform (i.e., runs on all important mobile platforms + possibly on desktop), and can easily be queried?
I want an HTML5 web application (to reach all mobile/(desktop) platforms, and for its independence of third party frameworks/libraries), but using local/offline storage to mimic performance of native applications (and do not necessarily require connectivity). It creates/alters/manages certain records for a user (up to a couple of hundred records per year). Apart from data storage, as the app doesn't need any other access to the device, I think HTML5 would be a good option.
Some requirements on the data I want to store:
the best format would be some lightweight database like SQLite (due to performance reasons, and the ability to update single records without having to write a whole file (as in the case of XML))
disadvantage: I don't see any technology available across all platforms; WebSQL is deprecated, and IndexedDB is not available in too many browsers yet
the data records shall be easily exportable/downloadable in XML format (so that the user can read/modify it on his own)
therefore, XML would be a good way to go; I assume the datasize to be reasonably low for this option; 2 concerns though:
disadvantage 1: I need a query-language that allows me to easily select/sort/alter specific records (sthg like XQuery, but available in all browsers and running locally on the client)
disadvantage 2: as far as I have seen, HTML5 FileWriterAPI support is nowhere near mature - therefore, how would I be able to alter/save the XML data locally on the client? (ok, I have seen examples where the whole XML file is saved as a single key/value pair in local storage; but disadvantage 1 would still apply...)
What options do I have? Is HTML5 mature enough to do what I am longing for?
If not, what alternatives would meet my requirements? Couple of loose thoughts: some third party libraries (JQuery(?), JSON(?) or cross-platform frameworks (a la Phonegap - which I wanted to avoid in the first place, due to their limitations), or use some server-side storage (that is synced with local storage)?
I dont know what limitations of Phonegap are you talking about,
I would suggest your application needs to be a hybrid one.
According to you requirements, you need to use native SQLite in different operating systems.
For that you need to use Phonegap, where you can write your own plugin, in which javascript act as the interface and the implementation is in native code.
Otherwise you can always check out lawnchair
http://westcoastlogic.com/lawnchair/
Thanks
Gaurav Gupta
Paxcel
For others who happened upon this post. I am currently searching for a solution as well and ran into localForage which seems a pretty good choice.
https://github.com/mozilla/localForage
Related
In a os x app, which is the best way to store hundreds of user record with a lot of information for each record?
I know that i can use CoreData, SQLite, Realm, plist or other solution.
Which is the most compatible (i think for example for a web use) and clean/light/fast method/type of database to store data ?
thank
Core Data is written by Apple to be used in Cocoa based applications. It is already included on the operating system, has 10 years of development behind it and is used in hundreds of thousands of applications on the app store already.
Using a third party library has tremendous risks associated with it.
I always recommend using Core Data on iOS and OS X because it is written by Apple, has a fully supported team of developers behind it and is a mature code base.
Using raw SQLite is possible but you will not get the same performance benefits as you will with Core Data.
Realm is focusing on the wrong things. Their data migration system is scary, they focus on raw speed when they need to be focusing on the overall picture. They might be targeting mobile but I do not agree with their goals.
Hundreds of records is not hard for any database system. It is when you get to 10s of thousands of records on a device that many of them start to fall down due to memory pressure (even on OS X).
Core Data is designed to help with the memory pressure. I am not aware of any third part framework that will do so.
I think the most important thing is to compare Realm with the other database. Realm use its own core and the others are based on SQLite (CoreData use SQLite).
Realm is the first database designed for mobile use. So with your case (OSX) you must consider if your app have the same functionning as mobile App: device restrictions.
Realm:
Very fast methods: see example here
Rapid execution: see benchmark here, yes thousands of user records!
Light: the embeded database less than 1MB
But in beta state. This mean that they are constantlyy in development
SQLite:
the "lite" in the name mean it's a light database. You have many libraries available in github to help you, full documentation and full examples.
Personnaly, I use Realm for My OSX App and despite it being in a beta state, I have all the features I need. Realm is faster than CoreData and save me thousand of lines of code and months of work. :)
I hope that this will help you.
My application needs to use a database. I've read online that databases are supported in some browsers but they are now deprecated? It was very confusing. I need to use a database with HTML. Is it possible to use a database with HTML5?
UPDATE
It needs a database to store data when the user is offline. It needs to support the major browsers from IE 9 on up if possible on Mac and Win on desktop and possibly Linux. The application is a client side HTML editor. There is no server side yet. So I need to persistently store "files" on the client side. Also, when I do have a server side I still need to save the users work when they have no internet connection. Later, when they go online the application can sync up with the server side. I could make do with Local storage but it is too limited in space at 5MB. That's the maximum limit? The application allows you to use images and those will be saved to base 64 data URI resources so you will quickly use up that space with just a few projects.
A few people asked why go for a database on the client. Because it's appropriate for this situation and if browsers support it then I would like to use it.
You should not use a database in the browser, even if you could, because that probably is not standard conforming. However perhaps consider HTML5 local storage
What you can is develop some web service accessing a database; that access happens in the HTTP server (so the database is used in the server, not in the browser, which just indirectly gives access to it).
I would suggest to develop your web application in Opa but you could even use PHP for that. And you could make it a CGI or a FastCGI application.
BTW; you did not tell us which operating system you are using.
You could consider making your application some specialized HTTP server, e.g. using Wt or Ocsigen -or using some HTTP server library like libonion etc- and then it could access some database (and your application could run on e.g. localhost:8086 if you wanted to).
PS. I believe that having some web pages storing a large amount of data on the client side (in the browser) is simply not reasonable (and against the standard and the spirit of the web). You should store such large data on the server side (or have some URL to download it from the server). You could decide to make a "server" application running on localhost (of course, this is operating system specific).
PPS: If coding an HTML editor, I am not sure that standard browsers are the adequate tool. You may however use the contenteditable attribute.
Check in CanIUse website.
http://caniuse.com/#feat=sql-storage
It's not DEPRECATED, but it's not supported by all browsers so far. The reason is that it did not become a valid standard because all the approaches are based in SQLite, and this is way too limited to be considered a standard.
yes go for indexedDB. Its still in development phase but you can use it
http://www.html5rocks.com/en/tutorials/indexeddb/todo/
I've read online that databases are supported in some browsers but they are now deprecated?
You're right: The Web SQL Database spec has been abandoned. There are a few browsers that still support it, but not many, and it will likely disappear entirely in the very near future.
As an alternative, a newer spec called IndexedDB is now being promoted by the browser vendors. The down-side is that it is still relatively new, and thus you will probably have many users with have browsers that do not support it.
Depending on the size of the data you need to store, you might consider Web Storage instead. This is intended just for plain text, and can't store massive quantites of data, but it is a well-established standard, so you've got a much better chance of compatibility.
If you can get away with storing a relatively small data set (ie a few megabytes or less) then this is definitely the best option here. This would certainly be my advice; try to keep your offline data requirements as small as possible, and keep them in Web Storage.
Hope that helps.
For a detailed run-down of all available browser-side storage options, you should read this article. It gives a much more detailed description of each of the options I described above, plus a few others.
Given that a lot of people use content repositories. There must be a good reason. I'm building out a new web application that will need to store content. Can someone help me understanding this?
What are the advantages to using a content repository like Apache Jackrabbit as opposed to writing your own code/API to store images or text pages? Writing your own requires time etc. but so too does implementing and learning a new framework like the content repository API. A benefit to rolling your own seems to me that you know your code and have immediate expertise if you need to enhance or fix it. Using another framework you need to learn its foibles, and it is always easier to modify code you know that don't know... i.e. you don't know that underlying framework code as well as your own.
As I said a lot of people use them. There must be a reason. I can't see it as being just another "everyone is using them so, so should we." At least I hope it isn't that. :)
A JCR repository allows you to store all your content (from structured database-type data to large multimedia files) in a single place and with a single API, which is extremely convenient and makes your code simpler, avoiding the impedance mismatch between files and data that you usually have in content-based systems.
JCR also provides a lot of infrastructure functionality that you won't have to build or assemble yourself: search (including full-text), observation (callbacks when something changes) versioning, data types including multi-value, ordered nodes, etc...
If you allow a shameless plug, my "JCR - best of both worlds" article at http://java.dzone.com/articles/java-content-repository-best describes this in more detail and also provides a reading list for the JCR spec, that should allow you go get a good overview without reading the whole thing.
The article uses Apache Sling for its examples, which combined with a JCR repository provides a very nice (IMO, but as a Sling committer I'm biased ;-) platform for content-based applications.
My most recent projects have involved both choices: a custom-built data store (MySQL and image files) wtih a multi-level caching mechanism, and a JCR-based commercial repository.
A few thoughts:
In the short run, a DIY solution offers reduced complexity: you only have to build and learn what you need. And there is at least the opportunity to optimize
the data store for your particular application's needs -- more than likely speed of retrieval, but possibly storage footprint, security, or reliability concerns are foremost for you.
However, in the long run, you're looking at a significant increment of work to extend the home-grown system to a new content type (video, e.g.) or to provide new functionality (maybe,
versioning).
Also, it's difficult to separate the choice of a data store approach from the choice of tools that content providers will use to populate and maintain the data store. You'll have to give
your authors something more than an HTML form with a textarea and a submit button.
This is related to the advantages of standardization: compatibility and interchangeability. If everybody writes his own library and API, there is no compatibility and interchangeability, leading to higher cost.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I'm creating a web site that I think must have a client side database. The other option would be to stick everything on the server at the expense of increased complexity and decreased scalability. What options do I have? Must I build a plugin? Must I wait until everybody's HTML5 compliant?
Update There's been a lot of comments about why I would actually need this. Here are my thoughts. Tell me if I'm being silly:
The clients will have a large and complex state that will require something like a database to provide the data interaction that I need. Therefore (I think) cookies are out of the picture.
This data is transient, so the client won't care if it gets erased as soon as they close a session. However they will need to keep the data if they go to a different web page and then come back. Therefore (I think) somehow storing the data in some sort of a javascript SQL implementation will not work.
I can certainly do everything that I want to do on the server, and servers can scale to manage the load (Facebook). But (I think) I'd rather build a plugin than pay for the infrastructure to support this load. This is for a bare bones startup. (The richer the startup is, the barer my bones will be.)
Indexed Database (Can I use)
Web SQL (Can I use)
localStorage
I'm about 5 years late in answering this, but given that there are errors and outdated data in some of the existing answers, and unaddressed points in the original question, I figured i'd throw in my two cents.
First, contrary to what others have implied on here, localStorage is not a database. It is (or should be perceived as) a persistent, string-based key-value store...
...which may be perfectly fine for your needs (and brings me to my second point).
Do you need explicit or implicitly relationships between your data items?
How about the ability to query over said items?
Or more than 5 MB in space?
If you answered "no" to all all of the above, go with localStorage and save yourself from the headaches that are the WebSQL and IndexedDB APIs. Well, maybe just the latter headache, since the former has been deprecated.
There are also several other client-side storage facilities (native and non-native) you may want to look in to, some of which are deprecated* but still see support from some browsers:
userData*
The rest of webStorage (sessionStorage and globalStorage*)
HTML5 File System*
Flash Locally Shared Objects
Silverlight Isolated Storage
Check out BakedGoods if you want to utilize any of these facilities, and more, without having to write low-level storage operation code. With it, placing data in one (or more) of them, for example, is as simple as:
bakedGoods.set({
data: [{key: "key1", value: "val1"}, {key: "key2", value: "val2"}],
storageTypes: ["silverlight", "fileSystem", "localStorage"],
options: optionsObj,
complete: function(byStorageTypeStoredKeysObj, byStorageTypeErrorObj){}
});
Oh, and for the sake of complete transparency, BakedGoods is maintained by this guy right here :) .
Use PouchDB.
PouchDB is an open-source JavaScript database inspired by Apache CouchDB that is designed to run well within the browser.
It helps building applications that works online as well as offline.
Basically, it stores the last fetched data in the in-browser database (uses IndexedDB, WebSQL under the hood) and then syncs again when the network gets active.
I came across a JavaScript Database http://www.taffydb.com/ still trying it out myself, hope this helps.
If you are looking for a NoSQL-style db on the client you can check out http://www.forerunnerdb.com. It supports the same query language as MongoDB and has a data-binding module if you want your DOM to reflect changes to your data automatically.
It is also open source, is constantly being updated with new features and the community around it is growing rapidly.
Disclaimer, I'm the lead developer of the project.
If you feel like you need it then use it for the clients that support it and implement a server-side fallback for clients that don't.
An alternative is you can use Flash and Local Shared Objects which can store a lot more information than a cookie, will work in all browsers with Flash (which is pretty much all browsers), and store typed data. You don't have to do the whole app in Flash, you can just write a tiny utility to read/write LSO data. This can be done using straight ActionScript projects without any framework and will give you a tiny 5-15kb swf.
There are two API's you'll primarily need. SharedObject.getLocal() to get access to a LSO and read/write it's data, and ExternalInterface.addCallback which you can use to register an AS3 method as a callback to call your read/write LSO method.
SharedObject
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/SharedObject.html?filter_flex=4.1&filter_flashplayer=10.1&filter_air=2
ExternalInterface
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/external/ExternalInterface.html
These links are to Flex references but for this you can just create an ActionScript project with no need for the Flex framework and therefore greatly reduced swf size. There are a number of good IDEs including free open-source ones like FlashDevelop.
FlashDevelop
http://www.flashdevelop.org/
Check out HTML5 Local Storage:
http://people.w3.org/mike/localstorage.html
You may also find this helpful:
HTML5 database storage (SQL lite) - few questions
When Windows 98 first came out, there were a lot of us still stuck on MS-DOS 6.22. Naturally, there were really cool features on the new operating system that wouldn't run in MS-DOS.
There comes a time when some things must be left behind to make room for innovation. If your application is really innovative and will offer cool new functionality that uses the latest and greatest technologies, then some older browsers will naturally need to be left behind.
The advantage that you have is that, unlike upgrading an operating system, upgrading from IE7 to Chrome 8 or Firefox 3.6 is a more reachable goal for the average user of your app, especially if you provide a link and upgrade instructions.
I would try Mozilla's localForage. https://localforage.github.io/localForage/
I'm using SqlServer to drive a WPF application, I'm currently using NHibernate and pre-read all the data so it's cached for performance reasons. That works for a single client app, but I was wondering if there's an in memory database that I could use so I can share the information across multiple apps on the same machine. Ideally this would sit below my NHibernate stack, so my code wouldn't have to change. Effectively I'm looking to move my DB from it's traditional format on the server to be an in memory DB on the client.
Note I only need select functionality.
I would be incredibly surprised if you even need to load all your information in memory. I say this because, just as one example, I'm working on a Web app at the moment that (for various reasons) loads thousands of records on many pages. This is PHP + MySQL. And even so it can do it and render a page in well under 100ms.
Before you go down this route make sure that you have to. First make your database as performant as possible. Now obviously this includes things like having appropriate indexes and tuning your database but even though are putting the horse before the cart.
First and foremost you need to make sure you have a good relational data model: one that lends itself to performant queries. This is as much art as it is science.
Also, you may like NHibernate but ORMs are not always the best choice. There are some corner cases, for example, that hand-coded SQL will be vastly superior in.
Now assuming you have a good data model and assuming you've then optimized your indexes and database parameters and then you've properly configured NHibernate, then and only then should you consider storing data in memory if and only if performance is still an issue.
To put this in perspective, the only times I've needed to do this are on systems that need to perform millions of transactions per day.
One reason to avoid in-memory caching is because it adds a lot of complexity. You have to deal with issues like cache expiry, independent updates to the underlying data store, whether you use synchronous or asynchronous updates, how you give the client a consistent (if not up-to-date) view of your data, how you deal with failover and replication and so on. There is a huge complexity cost to be paid.
Assuming you've done all the above and you still need it, it sounds to me like what you need is a cache or grid solution. Here is an overview of Java grid/cluster solutions but many of them (eg Coherence, memcached) apply to .Net as well. Another choice for .Net is Velocity.
It needs to be pointed out and stressed that something like NHibernate is only consistent so long as nothing externally updates the database and that there is exactly one NHibernate-enabled process (barring clustered solutions). If two desktop apps on two different PCs are both updating the same database with NHibernate the caching simply won't work because the persistence units simply won't be aware of the changes the other is making.
http://www.db4o.com/ can be your friend!
Velocity is an out of process object caching server designed by Microsoft to do pretty much what you want although it's only in CTP form at the moment.
I believe there are also wrappers for memcached, which can also be used to cache objects.
You can use HANA, express edition. You can download it for free, it's in-memory, columnar and allows for further analytics capabilities such as text analytics, geospatial or predictive. You can also access with ODBC, JDBC, node.js hdb library, REST APIs among others.