I want to store some large offline data in user phone (more than 100 MB) in an encrypted database. If possible I also want to distribute the database pre-populated. I have also seen this.
I know about the webdatabase thing, but because it is depreciated, I am advised not to work with that.
I also have seen some third party plugins such as SQLite Plugin, but it works only for iOS and Android devices, but I target 4 platforms (ios, android, blackberry, windows)
Is there any other solution, other than writing down my own?
I made an app recently that required this, targetting the same OS's. You can use a combination of 2 databases :
1. LocalStorage ::
Check for localStorage
function supports_html5_storage() {
try {
return 'localStorage' in window && window['localStorage'] !== null;
} catch (e) {
return false;
}
}
Set an item into LocalStorage
localStorage.setItem("bar", foo);
or
localStorage["bar"] = foo;
Get an item from LocalStorage
var foo = localStorage.getItem("bar");
or
var foo = localStorage["bar"];
2. SQLite Database (more convenient, more persistive)
Set up your DB
var shortName = 'BHCAppDB';
var version = '1.0';
var displayName = 'BHCAppDB';
var maxSize = 65535;
if (!window.openDatabase){
alert('!! Databases are not supported in this Device !! \n\n We are sorry for the inconvenience and are currently working on a version that will work on your phone');
}
db = openDatabase(shortName, version, displayName,maxSize);
createAllTables(db);
Create your Tables
function createAllTables(db){
db.transaction(function(transaction){
transaction.executeSql("CREATE TABLE IF NOT EXISTS Profile(id INTEGER PRIMARY KEY AUTOINCREMENT,name TEXT, gender TEXT,age INTEGER)");
}
Execute an SQL Query
transaction(function(transaction){
var rowCount = 'SELECT * FROM Profile';
transaction.executeSql(rowCount,[],function(transaction,result){
if(result.rows.length == 0){
var sqlString = 'INSERT INTO Profile (name,gender,age) VALUES("自己","Female",18)';
transaction.executeSql(sqlString);
}
});
});
EDIT :: I forgot to add in the last option :)
3. Native Storage on all devices
This is the best part of Phonegap. You can call a native plugin class on all the devices using the Phonegap plugin call. During the call, you can pass parameters to the class, and the native class can store your data in the OS itself.
For example :: in iOS, you create a plugin .h & .m class and register it with the Cordova.plist file. Once that's done, you need to send a call to the class from JavaScript using Phonegap. Once the parameters have been received using NSDictionary or any other NSArray type, you can call a CoreData class to store UNLIMITED amounts of data. You'll never run out of memory .
This can be done in a similar fashion for all the rest of the OS's also :)
For Encryption try the following :: SQLCipher
Here is some additional information on working with an existing SQLite database. In this example encrypted.db is that brand new database you create and pragma.
ATTACH DATABASE 'encrypted.db' AS encrypted KEY 'secret'; -- create a new encrypted database
CREATE TABLE encrypted.t1(a,b); -- recreate the schema in the new database (you can inspect all objects using SELECT * FROM sqlite_master)
INSERT INTO encrypted.t1 SELECT * FROM t1; -- copy data from the existing tables to the new tables in the encrypted database
DETACH DATABASE encrypted;
In the W3C specification for webdatabase it is mentioned that the Web Applications Working Group continues work on two other storage-related specifications: Web Storage and Indexed Database API.
So the webdatabase specification is no longer active but the other two specifications are active.
The Web Storage can be used to store data locally within the user's browser. There are the following objects to achieve that:
localStorage which stores data without expiration date
sessionStorage which stores data for one session
The Web Storage is not recommended for your case (more than 100MB), because the W3C specification mentions that:
A mostly arbitrary limit of five megabytes per origin is recommended.
In my opinion SQLite is the best available option since it is a in-process library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. Moreover the SQLite limits seems to cover your needs:
The largest possible setting for SQLITE_MAX_PAGE_COUNT is 2147483646. When used with the maximum page size of 65536, this gives a maximum SQLite database size of about 140 terabytes.
Regarding your encryption requirements you should consider the SQLCipher which is an SQLite extension.
SQLCipher is an SQLite extension that provides transparent 256-bit AES encryption of database files. To date, it has been open-sourced, sponsored and maintained by Zetetic LLC. In the mobile space, SQLCipher has enjoyed widespread use in Apple’s iOS, as well as Nokia / QT for quite some time.
An alternative option is to encrypt and decrypt your data when writing and reading your database.
I hope this helps.
The mobile app I am working on has a similar requirement. It requires offline access to a parts table that contains nearly 500,000 different parts in it. The source for this table is extracted from the server by getting its JSON via a well defined GET URL.
I considered Indexed DB but the mobile browsers inside iOS and Android don't support this. Web local storage is not an option because of its hard 5 MB limit. So, I decided to use the Web SQL Database standard (http://www.w3.org/TR/webdatabase/) even though its deprecated. My experience so far with using Web SQL Database has been very good. Database operations perform very well and are very reliable on the mobile devices I support (iPad 2, iPad 3, Motorola Xyboard, Samsung Galaxy Tab 2). Plus, Phonegap exposes a JavaScript API to work with this standard (see http://docs.phonegap.com/en/2.5.0/cordova_storage_storage.md.html#Storage).
I wrote a Java utility that converts the downloaded JSON data into a SQLite database whose files are packaged as part of the Android APK or the iOS app package.
When my Phonegap mobile app starts, it uses native code to check the app's private data directory for the presence of the SQLite database files. If the files are not present, the native code copies the database files from the app package.
My implementation is based on the sample code I found at the link below. I hope this helps. Let me know if you have any questions about my particular implementation.
http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phonegap.html
I tried using LokiJS as a local database, and found it helpful in non-relational data. In my case I retrieve a data stored using MongoDB on the server, but it depends on the nature of your system
See those questions/answers:
Opening and storing encrypted documents offline in iOS
Does PhoneGap / Apache Cordova propose an API for encrypted SQLite database
When using Mono Touch can I also package for a standard Window application?
Related
Suppose I write
App.Current.Properties.Add("Crap", "123");
await App.Current.SavePropertiesAsync();
Where is my crap saved, on disk?
Properties are saved in Xamarin with
const string PropertyStoreFile = "PropertyStore.forms";
...
var stream = IsolatedStorageFileStream(PropertyStoreFile, FileMode.Create, isoStore)
Xamarin.Forms.Platform.WPF/Deserializer.cs
IsolatedStorage is a way for an application to store data that persists after closing, without having to specify where on disk it is saved. To find where yours is saved, go to C:\Users\$USER$\AppData\Local\IsolatedStorage and search for a files named PropertyStore.forms. It should be one of the results.
In Xamarin form because you project is a cross platform project you cant find one place on your disk for each platform is created by your compiler for example, in windows executable form you can put your Key/Value variable in multi different destination, Registry, App config, … or in android executable form, you can put your Key/Value on service that is provided by android OS and called "Preferences".
finally, where your Key/Value variables are saved depend on which platform you run your code.
if you want more detail, I suggest you watch this link https://github.com/aritchie/settings/tree/master/Acr.Settings/Platforms .
this link provide interface for store Key/Value Pair variable and implement in 4 different kind of OS that you prefer to run you project on it.
I supposed you need to understand how this is happen in UWP, because you tagged it. In this platform you can search for ApplicationDataContainer.
if you want to find out how this happen and where values saved on disk in android OS, you can watch this link that is provided by Microsoft docs
https://learn.microsoft.com/en-us/xamarin/essentials/preferences?tabs=android
I am interested in creating a video databse. My goal is to have a folder where my videos will be kept and each time I copy/delete a video the website that presents them should be updated to. the problem is I have no idea how to approach it.
Should I..
Use Sql and store a reference to each video location?
Have a script that checks all the time if new changes happen in that folder?
A package like joomla?
I am using ubuntu btw. I already have a simple html5 page, and I am presenting the videos using html5 video.
It depends on the size and the performance you want.
1.Way : use php to scan the folder and generate links on the fly
2.way : Use a database to store the file names and retrieve the names from the database and generate urls
pros and cons.
simple to implement , no changes in upload or download script. no database required.
You need have a database , little coding required for upload and also while genrating a page
You should make a db (format does not matter) and storing in it only file names of videos: the videos would be stored on hard drive.
Any operation on the web site will pass first on db for insert/update/delete videos records and then (maybe in a transaction context) on the file system.
This would be the standard approach to your question.
I am working on a new web application using Scala with Lift. I want to make it reusable so others might install it on their own servers for their own needs. I come out of a PHP background where it is common practice to create an install form asking for database connection details. This information is stored in a configuration file and used by the rest of the PHP application for connecting to the database. It is extremely convenient for the user because everything is contained within the directory storing the PHP files. They are free to define everything else. My Java/Scala background has all been enterprise work where an application was only intended to run on the database we setup for it. It was not meant to be installed on others' web servers or with different databases.
So my question is how is this typically done for the Java/Scala world? If there are open source applications implementing the mainstream solution, feel free to point me to those too.
I use this to set up the database:
val vendor =
new StandardDBVendor(
Props.get("db.driver") openOr "org.h2.Driver",
Props.get("db.url") openOr "jdbc:h2:mem:db;AUTO_SERVER=TRUE",
Props.get("db.user"),
Props.get("db.password"))
LiftRules.unloadHooks.append(vendor.closeAllConnections_! _)
DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
The 'Props' referred to will then be (by default) in the file default.props in the props directory in resources.
Updated: This is what I do on servers in production. With 'Props.whereToLook' you provide a function that retrieves an input stream of the configuration. This can be a file as in the example below or you could for example fetch it over network socket.
You will probably let the application to fail with an error dialog.
val localFile = () => {
val file = new File("/opt/jb/myapp/etc/myapp.props")
if (file.exists) Full(new FileInputStream(file)) else Empty
}
Props.whereToLook = () => (("local", localFile) :: Nil)
I am not sure if I am missing your points.
By default, Lift use Scala source file(Boot.scala) to configure all the settings, because Lift doesn't wanna introduce other language into the framework, however you can override some of the configurations using a .properties file.
In Java/Scala world, we use .properties file. It's just a plain text file used for configuration or localization etc,just like text configuration files in PHP.
Lift Framework has it's default support for the external database configuration files, you check out the code in Boot.scala, that's if a .properties file existed, the database will initialized using the connection configuration, if it doesn't, it will use the source file configuration.
A call to all Win32 developers... I'm developing an application in C using plain Win32. I wanted to ask about Windows development standards regarding these things:
Is there a standard Windows error log api? For example if my client uses my app and it crashes, I would like them to send me the error log and I would prefer this being a standard location so they can maybe access it with a standard Windows log utility.
My app needs to store settings information. I think the registry is the standard utility for this task. Is that right?
My app needs to store and retrieve files that it downloaded from the internet - images, executables etc. Is Application Data/myapp the standard location to store this type of information?
My app needs a very straight-forward database - I'm using CSV for this. I basically need to store and retrieve this type of data so I'm just serializing a .csv file from Application Data/myapp. Is there a better Windows standard way of doing this?
That's all for now :). Thanks!
Is there a standard Windows error log api?
There is the Windows Event Log, but I don't think you want a typical user having to go into it to extract your logged information.
You probably don't want to log by default, unless you're shipping questionable pre-release code. When a user is experiencing problems, then you have them turn logging on. In this case, I recommend placing the file somewhere that typical users have experience with, like My Documents.
By the way, if you're writing a standalone application and want the best possible information in the event of a crash, look into minidumps. Here is a Codeproject sample.
My app needs to store settings information
Yep, registry.
My app needs to store and retrieve files
Yes, App Data. Just be sure to use SHGetFolderPath and CSIDL_APPDATA.
My app needs a very straight-forward database
There's nothing wrong with CSV for simple data. You could store the data in XML and use MSXML to process it, if you prefer. I've used SQlite in the past when I needed fast, lightweight storage of more complicated data.
I have an Access 2007 database with an attachments facility. Currently the client may upload files locally but the files cannot be accessed elsewhere. I have been able to carry out a similar operation when developing on a web based system however I cannot seem to do it on an Access 2007 database and I am unsure as to whether it is even possible. Basically the system needs to connect to the SQL server online and upload the file although the database is not online itself. I would be grateful for any pointers!
I have faced this situation. Here are your choices:
Use Access attachment field in a shared ACCDB -- won't work "online" very well, but you could park the ACCDB on your LAN and make it a separate back-end ACCDB shared by all. Your post didn't say whether your users are either local or "online" -- and whether "online" meant web.
Use VarChar(Max) (aka BLOB) fields in SQL-Server to store the attachments. But, you can't populate these easily from Access. Assuming you control the server where SQL-Server is running, you can use ADO in Access to upload a VarChar(Max) using the bulkinsert T-SQL command. This works pretty well and it's easy.
Create an upload web page. Use iExplorer automation (i.e, create an iExplorer object) in VBA to navigate to that page, fill it in and press the upload button. For security reasons, you cannot use automation to fill in a file upload control, but you can use sendkeys. This doesn't work perfectly -- sometimes you have to repeat the process once or twice, but it works pretty well if it's invoked by a user who can validate it's working. This is what I did -- easiest solution.
Best solution probably is to create a web service using WCF to handle the upload. There are plenty of posts on how to encode and decode byte arrays to store files as VarChar(Max). It works extremely well. Unfortunately, Access cannot directly consume web services as far as I've been able to tell, so you would have to write a small vb.net program to do this and call it from Access.
You could store the files/attachments outside of SQL/Server - just on the server, and store only the links/URL's for those files in Access. You could make each one launchable. This is easy but harder to control the security.
You can use Sharepoint to store/share the attachments. That can work pretty well depending on the size of the attachments and your connectivity. It's built to support this.
Access allows multiple attachments in one record. SQL/Server doesn't support this. So, if you can split your ACCDB into a front-end for the programs only and back-end ACCDB that is sharable by your users to contain the data/attachments, that is by far the easiest answer.