Provide a database packaged with the .APK file or host it separately on a website? - database

Here is some background about my app:
I am developing an Android app that will display a random quote or verse to the user. For this I am using an SQLite database. The size of the DB would be approximately 5K to 10K records, possibly increasing to upto 1M in later versions as new quotes and verses are added. Thus the user would need to update the DB as and when newer versions are of the app or DB are released.
After reading through some forums online, there seem to be two feasible ways I could provide the DB:
1. Bundle it along with the .APK file of the app, or
2. Upload it to my app's website from where users will have to download it
I want to know which method would be better (if there is yet another approach other than these, please do let me know).
After pondering this problem for some time, I have these thoughts regarding the above approaches:
Approach 1:
Users will obtain the DB along with the app, and won't have to download it separately. Installation would thereby be easier. But, users will have to reinstall the app every time there is a new version of the DB. Also, if the DB is large, it will make the installable too cumbersome.
Approach 2:
Users will have to download the full DB from the website (although I can provide a small, sample version of the DB via Approach 1). But, the installer will be simpler and smaller in size. Also, I would be able to provide future versions of the DB easily for those who might not want newer versions of the app.
Could you please tell me from a technical and an administrative standpoint which approach would be the better one and why?
If there is a third or fourth approach better than either of these, please let me know.
Thank you!
Andruid

I built a similar app for Android which gets periodic updates with data from a government agency. It's fairly easy to build an Android compatible db off the device using perl or similar and download it to the phone from a website; and this works rather well, plus the user gets current data whenever they download the app. It's also supposed to be possible to throw the data onto the sdcard if you want to avoid using primary data storage space, which is a bigger concern for my app which has a ~6Mb database.
In order to make Android happy with the DB, I believe you have to do the following (I build my DB using perl).
$st = $db->prepare( "CREATE TABLE \"android_metadata\" (\"locale\" TEXT DEFAULT 'en_US')");
$st->execute();
$st = $db->prepare( "INSERT INTO \"android_metadata\" VALUES ('en_US')");
$st->execute();
I have an update activity which checks weather updates are available and if so presents an "update now" screen. The download process looks like this and lives in a DatabaseHelperClass.
public void downloadUpdate(final Handler handler, final UpdateActivity updateActivity) {
URL url;
try {
close();
File f = new File(getDatabasePath());
if (f.exists()) {
f.delete();
}
getReadableDatabase();
close();
url = new URL("http://yourserver.com/" + currentDbVersion + ".sqlite");
URLConnection urlconn = url.openConnection();
final int contentLength = urlconn.getContentLength();
Log.i(TAG, String.format("Download size %d", contentLength));
handler.post(new Runnable() {
public void run() {
updateActivity.setProgressMax(contentLength);
}
});
InputStream is = urlconn.getInputStream();
// Open the empty db as the output stream
OutputStream os = new FileOutputStream(f);
// transfer bytes from the inputfile to the outputfile
byte[] buffer = new byte[1024 * 1000];
int written = 0;
int length = 0;
while (written < contentLength) {
length = is.read(buffer);
os.write(buffer, 0, length);
written += length;
final int currentprogress = written;
handler.post(new Runnable() {
public void run() {
Log.i(TAG, String.format("progress %d", currentprogress));
updateActivity.setCurrentProgress(currentprogress);
}
});
}
// Close the streams
os.flush();
os.close();
is.close();
Log.i(TAG, "Download complete");
openDatabase();
} catch (Exception e) {
Log.e(TAG, "bad things", e);
}
handler.post(new Runnable() {
public void run() {
updateActivity.refreshState(true);
}
});
}
Also note that I keep a version number in the filename of the db files, and a pointer to the current one in a text file on the server.

It sounds like your app and your db are tightly bound -- that is, the db is useless without the database and the database is useless without the app, so I'd say go ahead and put them both in the same .apk.
That being said, if you expect the db to change very slowly over time, but the app to change quicker, and you don't want your users to have to download the db with each new app revision, then you might want to unbundle them. To make this work, you can do one of two things:
Install them as separate applications, but make sure they share the same userID using the sharedUserId tag in the AndroidManifest.xml file.
Install them as separate applications, and create a ContentProvider for the database. This way other apps could make use of your database as well (if that is useful).

If you are going to store the db on your website then I would recommend that you just make rpc calls to your webserver and get data that way, so the device will never have to deal with a local database. Using a cache manager to avoid multiple lookups will help as well so pages will not have to lookup data each time a page reloads. Also if you need to update the data you do not have to send out a new app every time. Using HttpClient is pretty straight forward, if you need any examples please let me know

Related

How to configure Ignite to work as a full distributed database?

I'm trying to manage a decentralized DB around a huge number of partial DB instances. Each instance has a subset of the whole data and they are all nodes and clients, thus asking for some data the query must be spread to every (group) instance and which one have it will return the data.
Due to avoid lost of data if one instance goes down, I figured out they must replicate its contents with some others. How this scenario can be configured with Ignite?
Supose I have a table with the name and last access datetime of users in a distributed application, like ...
class UserLogOns
{
string UserName;
DateTime LastAccess;
}
Now when the program starts I prepare Ingite for work as a decentralizad DB ...
static void Main(string[] args)
{
TcpCommunicationSpi commSpi = new TcpCommunicationSpi();
// Override local port.
commSpi.LocalPort = 44444;
commSpi.LocalPortRange = 0;
IgniteConfiguration cfg = new IgniteConfiguration();
// Override default communication SPI.
cfg.CommunicationSpi = commSpi;
using (var ignite = Ignition.Start(cfg))
{
var cfgCache = new CacheConfiguration("mio");
cfgCache.AtomicityMode = CacheAtomicityMode.Transactional;
var cache = ignite.GetOrCreateCache<string, UserLogOns>(cfgCache);
cache.Put(Environment.MachineName, new UserLogOns { UserName = Environment.MachineName, LastAccess = DateTime.UtcNow });
}
}
And now ... I want to get LastAccess of other "computerB" when ever it was ..
Is this correct? How can it be implemented?
It depends on the exact use-case that you want to implement. In general, Ignite provides out of the box everything that you mentioned here.
This is a good way to start with using SQL in Ignite: https://apacheignite-sql.readme.io/docs
Create table with "template=partitioned" instead of "replicated" as it is shown in the example here: https://apacheignite-sql.readme.io/docs/getting-started#section-creating-tables, configure number of backups and select a field to be affinity key (a field that is used to map specific entries to cluster node) and just run some queries.
Also check out the concept of baseline topology if you are going to use native persistence: https://apacheignite.readme.io/docs/baseline-topology.
In-memory mode will trigger rebalance between nodes on each server topology change (node that can store data in/out) automatically.

How do I append to a file in an Azure storage file share?

I want to write entries to a log file stored in Azure file storage. I currently have this:
var log = "My log entry";
var client = _storageAccount.CreateCloudFileClient();
var share = client.GetShareReference(Config.LogShare);
share.CreateIfNotExists();
var root = share.GetRootDirectoryReference();
var logfile = root.GetFileReference("log.txt");
if (!logfile.Exists()) logfile.Create(0);
// What goes here to append to the file...?
I can see plenty of examples of how to do this with Blobs, or how to upload an entire file, but how do I just append to an existing file?
I have tried this:
var buffer = Encoding.GetEncoding("UTF-8").GetBytes(log.ToCharArray());
using (var fileStream = logfile.OpenWrite(0)) {
fileStream.Write(buffer, (int)logfile.Properties.Length, buffer.Length);
}
But then I get this error:
The remote server returned an error: (416) The range specified is invalid for the current size of the resource..
I managed to work this out myself. You just need to increase the size of the file by the number of new bytes you want to write to it, and then write the new data to that new empty space at the end of the file, like this:
var client = _storageAccount.CreateCloudFileClient();
var share = client.GetShareReference(Config.LogShare);
share.CreateIfNotExists();
var root = share.GetRootDirectoryReference();
var logfile = root.GetFileReference("log.txt");
if (!logfile.Exists()) logfile.Create(0);
var buffer = Encoding.UTF8.GetBytes($"{log}\r\n");
logfile.Resize(logfile.Properties.Length + buffer.Length);
using (var fileStream = logfile.OpenWrite(null)) {
fileStream.Seek(buffer.Length * -1, SeekOrigin.End);
fileStream.Write(buffer, 0, buffer.Length);
}
You can do this with blobs https://blogs.msdn.microsoft.com/windowsazurestorage/2015/04/13/introducing-azure-storage-append-blob/
Shame it doesn't work with files too
Azure file storage REST API doesn't support appending to an existing file. To achieve this, please mount the file share to your machine as a drive, and append to the file just like simple local files.
Actually, I don't think you really need appending functionality per your code above. You can specify the file size in CloudFile.OpenWrite() / CloudFile.Create(), or try CloudFile.UploadFromStream() instead of CloudFile.OpenWrite().
This error could also be due to multi-threaded access.
I bet if you tried to lock the file before you access it, you will not face this problem.
There are many ways to update the file.
Since you already managed to get the share, the root, the folder and the file.. Here is a portion of my code that worked for me.
if (!fileLock.IsWriteLockHeld) fileLock.EnterWriteLock();
try
{
using (var stream = new MemoryStream(content, false))
{
file.UploadFromStream(stream, null, options);
}
}
catch (Exception ex)
{
File.AppendAllText(FileName, ex.ToString());
}
finally
{
if (fileLock.IsWriteLockHeld)
fileLock.ExitWriteLock();
}
Where fileLock is declared as:
protected ReaderWriterLockSlim fileLock = new ReaderWriterLockSlim();
Having said that, I am not saying that this is the best way ever to do it.
The two things I would like you to keep in mind :
1-Lock the resource that is likely to be accessed by more than one thread (That is so common in AZURE)
2- Get familiar with asynchronous methods that Azure provides.. use them when they suit well.
Coming back to your original problem about appending to the existing file..
All the methods of the CloudFile will overwrite the existing file. Cloud Files are not for frequent writing, and they indeed impact performance if you keep writing on them frequently, add the lock impact on performance, they will be horrible.
Cloud files are meant to store big bulk of data once and for all, if you want to add another bulk you have the choice of creating another file.
Have all your data with the client till they reach some size and create an algorith to select the file name and upload them all at once.

Issue with getting database via Sitecore API

We noticed a slight oddity in the Sitecore API code. The code is below for your reference. The code is trying to get a database by doing new Database(database). But randomly it was failing.
This code worked for a while with Database db = new Database(database); but started failing randomly yesterday. When we changed the code to Database db = Database.GetDatabase(database);, the code started working again. What is the difference between the two approaches and what is recommended by Sitecore?
I've seen this happen twice now - multiple times in production and a couple of times in my development environment.
public static void DeleteItem(string id, stringdatabase)
{
//get the database
Database db = new Database(database);
//get the item
item = db.GetItem(new ID(id));
if (item != null)
{
using(new Sitecore.SecurityModel.SecurityDisabler())|
{
//delete the item
item.Delete();
}
}
}
A common way you will see people get a specific database is:
Sitecore.Data.Database master = Sitecore.Configuration.Factory.GetDatabase("master");
This is equivalent to Sitecore.Data.Database.GetDatabase("master").
When you call either of these methods it will first check the cache for the database. If not found it will build up the database with all of the configuration values within the config file via reflection. Once the database is created it will be placed in the cache for future use.
When you use the constructor on the database it is simply creating a rather empty database object. I am rather suprised to hear it was working at all when you used this method.
The proper approach to get a specific database would be to use:
Sitecore.Configuration.Factory.GetDatabase("master");
// or
Sitecore.Data.Database.GetDatabase("master");
If you are looking to get the database used with the current request (aka context database) you can use Sitecore.Context.Database. You can also use Sitecore.Context.ContentDatabase.

Silverlight: Business Application Needs Access To Files To Print and Move

I have the following requirement for a business application:
(All of this could be on local or server)
Allow user to select folder location
Show contents of folder
Print selected items from folder (*.pdf)
Display which files have been printed
Potentially move printed files to new location (sub-folder of printed)
How can I make this happen in Silverlight?
Kind regards,
ribald
First of all, all but the last item can be done (the way you expect). Due to security protocols, silverlight cannot access the user's drive and manipulate it. The closest you can get is accessing silverlight's application storage which will be of no help to you whatsoever in this case. I will highlight how to do the first 4 items.
Allow user to select folder location & Show contents of folder
public void OnSelectPDF(object sender)
{
//create the open file dialog
OpenFileDialog ofg = new OpenFileDialog();
//filter to show only pdf files
ofg.Filter = "PDF Files|*.pdf";
ofg.ShowDialog();
byte[] _import_file = new byte[0];
//once a file is selected proceed
if (!object.ReferenceEquals(ofg.File, null))
{
try
{
fs = ofg.File.OpenRead();
_import_file = new byte[fs.Length];
fs.Read(_import_file, 0, (int)fs.Length);
}
catch (Exception ex)
{
}
finally
{
if (!object.ReferenceEquals(fs, null))
fs.Close();
}
//do stuff with file - such as upload the file to the server
};
}
If you noticed, in my example, once the file is retrieved, i suggest uploading it to a webserver or somewhere with temporary public access. I would recommend doing this via a web service. E.g
//configure the system file (customn class)
TSystemFile objFile = new TNetworkFile().Initialize();
//get the file description from the Open File Dialog (ofg)
objFile.Description = ofg.File.Extension.Contains(".") ? ofg.File.Extension : "." + ofg.File.Extension;
objFile.FileData = _import_file;
objFile.FileName = ofg.File.Name;
//upload the file
MasterService.ToolingInterface.UploadTemporaryFileAsync(objFile);
Once this file is uploaded, on the async result, most likely returning the temporary file name and upload location, I would foward the call to some javascript method in the browser for it to use the generic "download.aspx?fileName=givenFileName" technique to force a download on the users system which would take care of both saving to a new location and printing. Which is what your are seeking.
Example of the javascript technique (remember to include System.Windows.Browser):
public void OnInvokeDownload(string _destination)
{
//call the browser method/jquery method
//(I use constants to centralize the names of the respective browser methods)
try
{
HtmlWindow window = HtmlPage.Window;
//where BM_INVOKE_DOWNLOAD is something like "invokeDownload"
window.Invoke(Constants.TBrowserMethods.BM_INVOKE_DOWNLOAD, new object[] { _destination});
}
catch (Exception ex) { System.Diagnostics.Debug.WriteLine(ex.ToString()); }
}
Ensure you have the javascript method existing either in an included javaScript file or in the same hosting page as your silverlight app. E.g:
function invokeDownload(_destination) {
//some fancy jquery or just the traditional document.location change here
//open a popup window to http://www.myurl.com/downloads/download.aspx? fileName=_destination
}
The code for download.aspx is outside the scope of my answer, as it varies per need and would just lengthen this post (A LOT MORE). But from what I've given, it will "work" for what you're looking for, but maybe not in exactly the way you expected. However, remember that this is primarily due to silverlight restrictions. What this approach does is rather than forcing you to need a pluging to view pdf files in your app, it allows the user computer to play it's part by using the existing adobe pdf reader. In silverlight, most printing, at least to my knowledge is done my using what you call and "ImageVisual" which is a UIElement. To print a pdf directly from silverlight, you need to either be viewing that PDF in a silverlight control, or ask a web service to render the PDF as an image and then place that image in a control. Only then could you print directly. I presented this approach as a lot more clean and direct approach.
One note - with the temp directory, i would recommend doing a clean up by some timespan of the files on the server side everytime a file is being added. Saves you the work of running some task periodically to check the folder and remove old files. ;)

Can I download an SQLite db on /sdcard and access it from my Android app?

I already found out that there is no way to bundle files in an .apk and have them on /sdcard, the best option so far being to download the large files upon first run. I came accross a tutorial saying how to bundle an sqlite db with the apk and then copy it so that it can be accessed with SQLiteDatabase (thus doubling the space needed, and not using /sdcard at all).
http://developer.android.com/guide/topics/data/data-storage.html#db says all databases MUST be in /data/data/package_name/databases.
Is that really so? Is there a way to trick the framework into opening a database that is placed on the /sdcard partition? Is there a way to use another SQLite java wrapper/framework to access such databases?
If the answer to the above is 'No', what other options do I have? My data is very well represented in a relational model, but is just too big, plus, I want to be able to update it without the need to reinstall/upgrade the entire app.
Sure you can. The docs are a little conflicting about this as they also say that no limitations are imposed. I think they should say that relative paths are to the above location and dbs there ARE private. Here is the code you want:
File dbfile = new File("/sdcard/mydb.sqlite" );
SQLiteDatabase db = SQLiteDatabase.openOrCreateDatabase(dbfile, null);
System.out.println("Its open? " + db.isOpen());
Try this:
String dir = Environment.getExternalStorageDirectory().getPath()
File dbfile = new File(dir+"/mydb.sqlite");
SQLiteDatabase db = SQLiteDatabase.openOrCreateDatabase(dbfile, null);
System.out.println("Its open? " + db.isOpen());
Just an idea:
you can put your database.db into the assets folder.
if your database.db file is larger than 2Mb the system is unable to compress it, so you need other one options
you can rename your database.db for example database.jit or database.mp3 - which are not compressed, than at the first run you can rename it to database.db
check this out ...
storing android application data on SD Card
I share next code. Declare opcionesMenu Vector<String>
Vector < String > opcionesMenu = new Vector< String >();
// the database is SDCard. I saw the code to Blackberry (net.rim)
String rutaDB = "/storage/extSdCard/Android/databases/Offshore.db";
SQLiteDatabase db = SQLiteDatabase.openOrCreateDatabase(rutaDB, null);
Cursor cursor = db.rawQuery("SELECT Codigo, Nombre FROM Ruta ORDER BY Codigo, Nombre", null);
if (cursor.moveToFirst())
{
do
{
opcionesMenu.add(cursor.getString(0) + " - " + cursor.getString(1));
} while(cursor.moveToNext());
}
db.close();

Resources