Copy/Cache Azure database locally for offline use. - sql-server

I have a piece of software that connects to an Azure database to gather formulas for several varieties of colors to allow the user to follow a recipe to create their own product.
Basically there is just one big database pull when the application launches, that pulls down all the formulas, and from there the user can simply use, modify, or even delete the formulas as they wish.
The problem is that where this software is used, there are seldom constant internet connections, and the application so far is simply designed to shutdown if there isn't one present.
I am looking for a solution for being able to allow the application to BOTH connect to the database on application startup(If a connection is present) and save a copy locally, or if no connection is present, check for a locally saved copy to work with.
I have looked everywhere, but have been unable to find any methods to "programmatically" retrieve the pertinent(or all, if necessary) data, and either export it to a local file, or cache it somehow for offline use.
Any suggestions?

You don't mention what kind of technology you're using on the client side, and if you require access to a database on the client or just the data - but regardless there are several ways to do this.
Azure Mobile apps have a quickstart which will implements an Azure SQL <-> SQL Lite (mobile) database sync framework (table controllers on the server side). This lets you use libraries on the mobile side to use the local db for getting its data, and when you know you're online you can sync to/from the server. This is quite sophisticated and probably gives you more than just the caching that you're looking for.
I've used two other strategies for caching - one for HTML/javascript based applications (phonegap/cordova) and the other for Xamarin c# apps on iOS and Android. I'm assuming if it's a standard windows desktop app you know how to persist data so you can use whatever kind of cache/file system/db you like.
JavaScript/html - use the html5 localStorage functions to store the JSON output of the web server calls you're making. This is really easy then to abstract, where your app before is making an ajax call to the server to get some data, instead move that to a "liveorcache" class, which can determine whether to go to the server or just use the local storage. Code snippet for saving/loading json in localstorage below:
$scope.saveFixtures = function () {
localStorage["fixtures"] = JSON.stringify($scope.fixtures);
};
$scope.loadFixtures = function () {
if (localStorage["fixtures"] != undefined) {
$scope.fixtures = JSON.parse(localStorage["fixtures"]);
}
};
If you're writing your app in Xamarin you can do the same kind of thing, but using a PCL library - I used "PCLStorage" which works on Android and iOS. Same strategy though, in my code I just write the JSON data to a file with an appropriate filename, but usually wrap the object in another object that contains the cache write date/time. You then serialise the object to the file - something like below.
public class CacheProvider
{
public static async Task<CacheModel> ReadCache<T>(string filename)
{
IFolder rootFolder = FileSystem.Current.LocalStorage;
IFolder cache = await rootFolder.CreateFolderAsync("sportenzaCache", CreationCollisionOption.OpenIfExists);
try
{
IFile file = await cache.GetFileAsync(filename);
var data = await file.ReadAllTextAsync();
return JsonConvert.DeserializeObject<T>(data) as CacheModel;
}
catch(FileNotFoundException ex)
{
return null;
}
}
public static async void WriteCache<T>(string filename, CacheModel data)
{
IFolder rootFolder = FileSystem.Current.LocalStorage;
IFolder cache = await rootFolder.CreateFolderAsync("sportenzaCache", CreationCollisionOption.OpenIfExists);
IFile file = await cache.CreateFileAsync(filename, CreationCollisionOption.ReplaceExisting);
if (file != null)
{
data.CacheCreated = DateTime.Now;
string json = JsonConvert.SerializeObject(data);
await file.WriteAllTextAsync(json);
}
}
public static async void DeleteCache(string filename)
{
IFolder rootFolder = FileSystem.Current.LocalStorage;
IFolder cache = await rootFolder.CreateFolderAsync("sportenzaCache", CreationCollisionOption.OpenIfExists);
IFile file = await cache.GetFileAsync(filename);
if (file != null)
await file.DeleteAsync();
}
}

Related

Xamarin Forms - How do i use a Premade Local Database? [Solved] [duplicate]

I have started using the Xamarin plugin for Visual Studio to create an Android app.
I have a local SQL database, and I want to call it to display data. I don't see how I can do this. Is it possible?
After thinking this was a trivial thing to do, I was proven wrong when I tried setup a quick test project. This post will contain a full tutorial on setting up a DB for an Android App in Xamarin that will come in handy as a reference for future Xamarin users.
At a glance:
Add Sqlite.cs to your project.
Add your database file as an Asset.
Set your database file to build as an AndroidAsset.
Manually copy the database file out of your apk to another directory.
Open a database connetion using Sqlite.SqliteConnection.
Operate on the database using Sqlite.
Setting up a local database for a Xamarin Android project
1. Add Sqlite.cs to your project.
Start by going to this repository and downloading Sqlite.cs; this provides the Sqlite API that you can use to run queries against your db. Add the file to your project as a source file.
2. Add DB as asset.
Next, get your DB and copy it into the Assets directory of your Android project and then import it into your project so that it appears beneath the Assets folder within your solution:
I'm using the Chinook_Sqlite.sqlite database sample renamed to db.sqlite from this site throughout this example.
3. Set DB to build as AndroidAsset.
Right click on the DB file and set it to build action AndroidAsset. This will ensure that it is included into the assets directory of the APK.
4. Manually copy DB out of your APK.
As the DB is included as an Asset (packaged within the APK) you will need to extract it out.
You can do this with the following code:
string dbName = "db.sqlite";
string dbPath = Path.Combine (Android.OS.Environment.ExternalStorageDirectory.ToString (), dbName);
// Check if your DB has already been extracted.
if (!File.Exists(dbPath))
{
using (BinaryReader br = new BinaryReader(Android.App.Application.Context.Assets.Open(dbName)))
{
using (BinaryWriter bw = new BinaryWriter(new FileStream(dbPath, FileMode.Create)))
{
byte[] buffer = new byte[2048];
int len = 0;
while ((len = br.Read(buffer, 0, buffer.Length)) > 0)
{
bw.Write (buffer, 0, len);
}
}
}
}
This extracts the DB as a binary file from the APK and places it into the system external storage path. Realistically the DB can go wherever you want, I've just chosen to stick it here.
I also read that Android has a databases folder that will store databases directly; I couldn't get it to work so I've just ran with this method of using an existing DB.
5. Open DB Connection.
Now open a connection to the DB through the Sqlite.SqliteConnection class:
using (var conn = new SQLite.SQLiteConnection(dbPath))
{
// Do stuff here...
}
6. Operate on DB.
Lastly, as Sqlite.net is an ORM, you can operate on the database using your own data types:
public class Album
{
[PrimaryKey, AutoIncrement]
public int AlbumId { get; set; }
public string Title { get; set; }
public int ArtistId { get; set; }
}
// Other code...
using (var conn = new SQLite.SQLiteConnection(dbPath))
{
var cmd = new SQLite.SQLiteCommand (conn);
cmd.CommandText = "select * from Album";
var r = cmd.ExecuteQuery<Album> ();
Console.Write (r);
}
Summary
And that's how to add an existing Sqlite database to your Xamarin solution for Android! For more information check out the examples included with the Sqlite.net library, its unit tests and the examples in the Xamarin documentation.
Here is the one that I'm using and it's working
install the Sqlite plugin
create interface to access different platforms services
create a model for the table
implement the interface that you created earlier on all of the
platform you want to use
use the plugin to create, get, insert, etc on your table
for more detailed information check this

Flutter; save user generated data on phone

In my app I have a list that the user adds items to.
I want to be able to store this data (the list) locally, on the phone, so the list can show up next time the user opens the app.
How would I do that.
(the app is only for android phones)
you can use the Shared Preferences plugin which is a simple key-value storage
here is the link to the plugin https://pub.dev/packages/shared_preferences/install
here is an example of how u can store a list of items and also how to load them :
class PersonData {
final String id;
final String name;
final String lastName;
PersonData({this.id, this.name, this.lastName});
}
void saveItems(List<PersonData> items) async {
SharedPreferences sharedPreferences = await SharedPreferences.getInstance();
List<String> personList = [];
items.forEach((element) {
personList.add("${element.id},${element.name},${element.lastName}");
});
sharedPreferences.setStringList("personData", personList);
}
Future<List<PersonData>> loadItems() async {
SharedPreferences sharedPreferences = await SharedPreferences.getInstance();
List<String> personList = sharedPreferences.getStringList("personData");
List<PersonData> items = [];
personList.forEach((element) {
List<String> personData = element.split(",");
PersonData person = PersonData(
id: personData[0], name: personData[1], lastName: personData[2]);
items.add(person);
});
return items;
}
You can use different approaches,
Shared Preferences - Saving data into local cache.
Wraps platform-specific persistent storage for simple data (NSUserDefaults on iOS and macOS, SharedPreferences on Android, etc.). Data may be persisted to disk asynchronously, and there is no guarantee that writes will be persisted to disk after returning, so this plugin must not be used for storing critical data.
Sqflite - Using an SQLite local database.
SQLite plugin for Flutter. Supports iOS, Android and MacOS.
Support transactions and batches
Automatic version managment during open
Helpers for insert/query/update/delete queries
DB operation executed in a background thread on iOS and Android
If it's a small app i suggest to use Hive
Hive is a lightweight and blazing fast key-value database written in pure Dart
That's an example using Hive.
var box = Hive.box('myBox');
box.put('name', 'David');
var name = box.get('name');
print('Name: $name');
Of course there are many packages that use different ways to store data, you can search for the one that fits more your usage.

Pass byte array from WPF to WebApi

tl;dr What is the best way to pass binary data (up to 1MBish) from a WPF application to a WebAPI service method?
I'm currently trying to pass binary data from a WPF application to a WebAPI web service, with variable results. Small files (< 100k) generally work fine, but any larger and the odds of success reduce.
A standard OpenFileDialog, and then File.ReadAllBytes pass the byte[] parameter into the client method in WPF. This always succeeds, and I then post the data to WebAPI via a PostAsync call and a ByteArrayContent parameter.
Is this the correct way to do this? I started off with a PostJSONAsync call, and passed the byte[] into that, but thought the ByteArrayContent seemed more appropriate, but neither work reliably.
Client Method in WPF
public static async Task<bool> UploadFirmwareMCU(int productTestId, byte[] mcuFirmware)
{
string url = string.Format("productTest/{0}/mcuFirmware", productTestId);
ByteArrayContent bytesContent = new ByteArrayContent(mcuFirmware);
HttpResponseMessage response = await GetClient().PostAsync(url, bytesContent);
....
}
WebAPI Method
[HttpPost]
[Route("api/productTest/{productTestId}/mcuFirmware")]
public async Task<bool> UploadMcuFirmware(int productTestId)
{
bool result = false;
try
{
Byte[] mcuFirmwareBytes = await Request.Content.ReadAsByteArrayAsync();
....
}
Web Config Settings
AFAIK these limits in web.config should be sufficient to allow 1MB files through to the service?
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="1073741824" />
</requestFiltering>
</security>
<httpRuntime targetFramework="4.5" maxRequestLength="2097152"/>
I receive errors in WebAPI when calling ReadAsByteArrayAsync(). These vary, possibly due to the app pool in IIS Express having crashed / getting into a bad state, but they include the following (None of which have lead to any promising leads via google):
Specified argument was out of the range of valid values. Parameter name: offset
at System.Web.HttpInputStream.Seek(Int64 offset, SeekOrigin origin)\r\n
at System.Web.HttpInputStream.set_Position(Int64 value)\r\n at System.Web.Http.WebHost.SeekableBufferedRequestStream.SwapToSeekableStream()\r\n at System.Web.Http.WebHost.Seek
OR
Message = "An error occurred while communicating with the remote host. The error code is 0x800703E5."
InnerException = {"Overlapped I/O operation is in progress. (Exception from HRESULT: 0x800703E5)"}
at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)\r\n
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityCoreSync(Byte[] buffer, Int32 offset, Int32 size)\r\n
at System.Web.Hosting.IIS7WorkerRequ...
Initially I thought this was most likely down to IIS Express limitations (running on Windows 7 on my dev pc) but we've had the same issues on a staging server running Server 2012.
Any advice on how I might get this working would be great, or even just a basic example of uploading files to WebAPI from WPF would be great, as most of the code I've found out there relates to uploading files from multipart forms web pages.
Many thanks in advance for any help.
tl;dr It was a separate part of our code in the WebApi service that was causing it to go wrong, duh!
Ah, well, this is embarrassing.
It turns out our problem was down to a Request Logger class we'd registered in WebApiConfig.Register(HttpConfiguration config), and that I'd forgotten about.
It was reading the request content via async as StringContent, and then attempting to log it to the database in an ncarchar(max) field. This itself is probably OK, but I'm guessing all the weird problems started occurring when the LoggingHandler as well as the main WebApi controller, were both trying to access the Request content via async?
Removing the LoggingHandler fixed the problem immediately, and we're now able to upload files of up to 100MB without any problems. To fix it more permanently, I guess I rewrite of the LoggingHandler is required to set a limit on the maximum content size it tries to log / to ignore certain content types.
It's doubtful, but I hope this may be of use for someone one day!
public class LoggingHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
LogRequest(request);
return base.SendAsync(request, cancellationToken).ContinueWith(task =>
{
var response = task.Result;
// ToDo: Decide if/when we need to log responses
// LogResponse(response);
return response;
}, cancellationToken);
}
private void LogRequest(HttpRequestMessage request)
{
(request.Content ?? new StringContent("")).ReadAsStringAsync().ContinueWith(x =>
{
try
{
var callerId = CallerId(request);
var callerName = CallerName(request);
// Log request
LogEntry logEntry = new LogEntry
{
TimeStamp = DateTime.Now,
HttpVerb = request.Method.ToString(),
Uri = request.RequestUri.ToString(),
CorrelationId = request.GetCorrelationId(),
CallerId = callerId,
CallerName = callerName,
Controller = ControllerName(request),
Header = request.Headers.ToString(),
Body = x.Result
};
...........

Create persistent Sqlite db in Windows phone 8

I am trying my hands on Windows phone 8 applications and I am stuck into a weird situation here. I am using sqlite in order to create sqlite db and add values into the database. I am able to create the database and add the values in the database successfully but I am having a weird situation here.
Everytime I close the emulator and start the project again the database gets created again which should not be happening because I created the db the very first time I run the application.
Does anybody know why, and how I can prevent it from recreating the database each time?
public string DB_PATH = Path.Combine(Path.Combine(ApplicationData.Current.LocalFolder.Path, "aa.sqlite"));
private SQLiteConnection dtCon;
public MainPage()
{
InitializeComponent();
CreateDatabase();
dtCon = new SQLiteConnection(DB_PATH);
var tp = dtCon.Query<Contacts>("select * from contacts").ToList();
}
private async void CreateDatabase()
{
bool isDatabaseExisting = false;
//Checking if database already exists
try
{
Windows.Storage.StorageFile storagefile = await Windows.Storage.ApplicationData.Current.LocalFolder.GetFileAsync("aa.sqlite");
isDatabaseExisting = true;
}
catch
{
isDatabaseExisting = false;
}
//if not exists then creating database
if (!isDatabaseExisting)
{
String str = System.IO.Path.Combine(Windows.Storage.ApplicationData.Current.LocalFolder.Path, "sqlite.db");
AddDataToDB(DB_PATH);
}
}
private void AddDataToDB(string str)
{
// Create the database connection.
dtCon = new SQLiteConnection(str);
// Create the table Task, if it doesn't exist.
dtCon.CreateTable<Contacts>();
Contacts oContacts = new Contacts();
oContacts.Name = "dfgdf";
oContacts.Detail = "asdfsf";
dtCon.Insert(oContacts);
}
I'm pretty sure when you close your emulator and restart, you're basically just uninstalling the application. Which is why your files or not there anymore -- as it looks like you're storing your data in isolated storage. I do not know if there is anyway around this.
You can buy a very cheap Windows 8/8.1 Phone and the files will persist until you manually uninstall the test application.
As #Chubosaurus says, closing and re-opening the emulator will remove all the apps. You can generally keep it running as long as you want and keep re-deploying your app to the emulator (although obviously rebooting the host PC will kill it).
You can save and restore the data from your emulator image via the ISETool command. See more here
Try adding Console.WriteLine("True"); and Console.WriteLine("False"); into the expected places after checking isDatabaseExisting to see/understand what the code path really is.

Provide a database packaged with the .APK file or host it separately on a website?

Here is some background about my app:
I am developing an Android app that will display a random quote or verse to the user. For this I am using an SQLite database. The size of the DB would be approximately 5K to 10K records, possibly increasing to upto 1M in later versions as new quotes and verses are added. Thus the user would need to update the DB as and when newer versions are of the app or DB are released.
After reading through some forums online, there seem to be two feasible ways I could provide the DB:
1. Bundle it along with the .APK file of the app, or
2. Upload it to my app's website from where users will have to download it
I want to know which method would be better (if there is yet another approach other than these, please do let me know).
After pondering this problem for some time, I have these thoughts regarding the above approaches:
Approach 1:
Users will obtain the DB along with the app, and won't have to download it separately. Installation would thereby be easier. But, users will have to reinstall the app every time there is a new version of the DB. Also, if the DB is large, it will make the installable too cumbersome.
Approach 2:
Users will have to download the full DB from the website (although I can provide a small, sample version of the DB via Approach 1). But, the installer will be simpler and smaller in size. Also, I would be able to provide future versions of the DB easily for those who might not want newer versions of the app.
Could you please tell me from a technical and an administrative standpoint which approach would be the better one and why?
If there is a third or fourth approach better than either of these, please let me know.
Thank you!
Andruid
I built a similar app for Android which gets periodic updates with data from a government agency. It's fairly easy to build an Android compatible db off the device using perl or similar and download it to the phone from a website; and this works rather well, plus the user gets current data whenever they download the app. It's also supposed to be possible to throw the data onto the sdcard if you want to avoid using primary data storage space, which is a bigger concern for my app which has a ~6Mb database.
In order to make Android happy with the DB, I believe you have to do the following (I build my DB using perl).
$st = $db->prepare( "CREATE TABLE \"android_metadata\" (\"locale\" TEXT DEFAULT 'en_US')");
$st->execute();
$st = $db->prepare( "INSERT INTO \"android_metadata\" VALUES ('en_US')");
$st->execute();
I have an update activity which checks weather updates are available and if so presents an "update now" screen. The download process looks like this and lives in a DatabaseHelperClass.
public void downloadUpdate(final Handler handler, final UpdateActivity updateActivity) {
URL url;
try {
close();
File f = new File(getDatabasePath());
if (f.exists()) {
f.delete();
}
getReadableDatabase();
close();
url = new URL("http://yourserver.com/" + currentDbVersion + ".sqlite");
URLConnection urlconn = url.openConnection();
final int contentLength = urlconn.getContentLength();
Log.i(TAG, String.format("Download size %d", contentLength));
handler.post(new Runnable() {
public void run() {
updateActivity.setProgressMax(contentLength);
}
});
InputStream is = urlconn.getInputStream();
// Open the empty db as the output stream
OutputStream os = new FileOutputStream(f);
// transfer bytes from the inputfile to the outputfile
byte[] buffer = new byte[1024 * 1000];
int written = 0;
int length = 0;
while (written < contentLength) {
length = is.read(buffer);
os.write(buffer, 0, length);
written += length;
final int currentprogress = written;
handler.post(new Runnable() {
public void run() {
Log.i(TAG, String.format("progress %d", currentprogress));
updateActivity.setCurrentProgress(currentprogress);
}
});
}
// Close the streams
os.flush();
os.close();
is.close();
Log.i(TAG, "Download complete");
openDatabase();
} catch (Exception e) {
Log.e(TAG, "bad things", e);
}
handler.post(new Runnable() {
public void run() {
updateActivity.refreshState(true);
}
});
}
Also note that I keep a version number in the filename of the db files, and a pointer to the current one in a text file on the server.
It sounds like your app and your db are tightly bound -- that is, the db is useless without the database and the database is useless without the app, so I'd say go ahead and put them both in the same .apk.
That being said, if you expect the db to change very slowly over time, but the app to change quicker, and you don't want your users to have to download the db with each new app revision, then you might want to unbundle them. To make this work, you can do one of two things:
Install them as separate applications, but make sure they share the same userID using the sharedUserId tag in the AndroidManifest.xml file.
Install them as separate applications, and create a ContentProvider for the database. This way other apps could make use of your database as well (if that is useful).
If you are going to store the db on your website then I would recommend that you just make rpc calls to your webserver and get data that way, so the device will never have to deal with a local database. Using a cache manager to avoid multiple lookups will help as well so pages will not have to lookup data each time a page reloads. Also if you need to update the data you do not have to send out a new app every time. Using HttpClient is pretty straight forward, if you need any examples please let me know

Resources