How To Store Game Data Without An Online Database - database

I am working on a game that is intended to be played off line. For the past I had done games which required an online connection. We would store all relevant data online in a database, and I would fetch the database data, parse the data into objects, via JSON, and access them as needed.
For my offline environment, I'm not sure how I can best replicate storing all the types of game data. I really preferred the organization of the database and I don't want to create a whole bunch of game files and variables to manage everything. I'd prefer to mimick having a database, only it compiles with the game.
I considered using an excel file, but I feel that would be too easy to hack.
Do you have a suggestion on how I can mimick an offline database?
My specific engine is with Unity3D and I'm using C#.
Thanks

You can use a local database technology, they are used just like you would use an online database but the data is stored in a file stored on the local machine. If your in .NET two come to mind:
SQLite - http://www.sqlite.org/
This is the defaco local database technology, its open source and has very widespread use. There is even a library to connect unity to it: https://github.com/Busta117/SQLiteUnityKit
SQL CE
This is Micrsofts single file database. It is very similar to the features of SQL Server, its not open source but it has built in drivers already in the .NET framework so it maybe simpler to use. Issue with this could be if you want your game to run across platforms take a look here:
http://answers.unity3d.com/questions/26118/can-you-use-sql-compact-35-with-unity.html
I would recommend going with SQLite as it seems there is more support for it in Unity

Unity can do this for you in the form of PlayerPrefs. According to the Unity documentation, it:
Stores and accesses player preferences between game sessions.
However, it can be and often is used to store things such as highscores, tutorial information, and various other offline data.
Simple example:
// this gets the value with the key "score"
// defaults to 0 if it doesn't exist
PlayerPrefs.GetInt("score");
// this sets the value with the key "score"
// previous values will be overritten
PlayerPrefs.SetInt("score", 9001);
Be aware that the values are not encrypted in any form automatically, however you can wrap the function as follows:
public static int GetIntData(string key, int value)
{
if(PlayerPrefs.HasKey(key))
{
return YourDecryptionMethod(PlayerPrefs.GetInt(key));
}
else
{
return 0;
}
}
public static void SetIntData(string key, int value)
{
return PlayerPrefs.GetInt(key, YourEncryptionMethod(value));
}
You could take this even further and encrypt the keys if you wanted to. However for very large data sets, you should consider using a database instead (see this answer).
Additional
If you like storing data as JSON, XML or some other format, there's nothing to stop you using one of the many parsers available for popular data formats with the same encrypt/decrypt wrapper solution.
For example if you're using C# and want to store data as XML, you can use System.Xml.
Take a look here for information on using external DLLs in your project, should you choose not to go with a Mono/.NET class.

Avoid using PlayerPrefs for game state that you are concerned about being easily hacked. PlayerPrefs are stored in plain text, they really should only be used for storing configuration related values.
If you are looking for a native way to handle it (i.e. not SQLite), you can regular .NET file IO. Simply have your data structure in a serializable POCO object model. When you save, you can use something like:
BinaryFormatter bin = new BinaryFormatter();
FileStream file = File.Open(Application.persistentDataPath + "/savegame.dat", FileMode.Open);
bf.Serialize(file, mySerializablePoco);
file.Close();
That will write a binary file to disk that can be deserialized during a load process. You can add encryption if you want as well.
If you aren't caring to modify the data, but just want the structure of setting predefined objects, you can use ScriptableObject. I did one project where we did not want to use SQLite so we made something like:
public class MyRepository : ScriptableObject {
public List<Car> Cars;
public List<Driver> Drivers;
}
[Serializable]
public class Car {
public string Manufacturer;
}
... ect
ScriptableObject is created and designed during edit time. You actually create it with some editor code and it will write out an *.asset file into your project. At that point you can Resource.Load it at runtime or wire it up to a public variable on a MonoBehavior script.
Check out: http://unity3d.com/learn/tutorials/modules/beginner/live-training-archive/scriptable-objects

Related

Flink reference data advice/best practice

Looking for some advice on where to store/access Flink reference data. Use case here is really simple - I have a single column text file with a list of countries. I am streaming twitter data and then matching the countries from the text file based on the (parsed) Location field of the tweet. In the IDE (Eclipse) its all good as I have a static ArrayList populated when the routine fires up via a static Build method in my Flink Mapper (ie implements Flinks MapFunction). This class is now inner static as it gets shirt on serialization otherwise. Point is, when the overridden map function is invoked at runtime from within the stream, the static array of country data is their waiting, fully populated and ready to be matched against. Works a charm. BUT, when deployed into a Flink cluster ( and it took me to hell and back last week to actually get the code to FIND the text file), the array is only populated as part of the Build method. When it comes to being used the data has mysteriously disappeared and I am left with an array size of 0. (ergo, not a lot of matches get found. Thus, 2 questions - why does it work in Eclipse and not on deploy (renders a lot of Eclipse unit tests pointless as well). Or possibly just more generally, what is the right way to cross reference this kind of static, fixed reference data within Flink? (and in a way that it is found in both Eclipse and the cluster...)
The standard way to handle static reference data is to load the data in the open method of a RichMapFunction or RichFlatMapFunction. Rich functions have open and close methods that are useful for creating and finalizing local state, and can access the runtime context.

Kafka-Flink-Stream processing: Is there a way to reload input files into the variables being used in a streaming process?

We are planning to use Flink to process a stream of data from a kafka topic (Logs in Json format).
But for that processing, we need to use input files which change every day, and the information within can change completely (not the format, but the contents).
Each time one of those input files changes we will have to reload those files into the program and keep the stream processing going on.
Re-loading of the data could be done same way as it is done now:
DataSet<String> globalData = env.readTextFile("file:///path/to/file");
But so far I couldnt find examples or come up with a way to trigger that reload in a stream processing job.
As extra information, we wont be using HDFS but local filesystem on each node, so the reload will have to be done in each node, from the local file.
This is because the only reason why we would need HDFS would be for this input files, which are just 100 mb in total and using HDFS would be an overkill.
So far I have been experimenting with RichMapFunction, trying to find a kafka-topic that would provide this functionality (reload files) and trying to find examples of this with no luck.
Edit:
After reading a lot more, I found in several places that this is the way to go: DataArtisans examples.
Trying to make a simple code that would do a simple change in a stream from a control stream, I got the following code:
public class RichCoFlatMapExample extends EventTimeJoinHelper {
private String config_source_path = "NOT_INITIALIZED";
#Override
public void open(Configuration conf) {
config_source_path = "first_file_path";
}
public abstract void processElement1(String one, String two, Collector<String> out) {
config_source_path = one;
}
public abstract void processElement2(String one, String two, Collector<String> out) {
String three = two + config_source_path;
out.collect(three);
}
}
The problem Im having now is, no matter what I try, I get the following error:
Class 'RichCoFlatMapExample' must either be declared abstract or implement abstract method 'processElement1(String, String, Collector)' in 'RichCoFlatMapExample'
The problem is, the requested method is implemented, but I cant make them "abstract" in a non abstract class (I get an error from the IDE).
If I make the class RichCoFlatMapExample, I wont be able to call it from Flink methods (dataStream methods).
Im not sure what is happening but I think this must be close. I will keep trying and update if I make this work.
Flink can monitor a directory and ingest files when they are moved into that directory; maybe that's what you are looking for. See the PROCESS_CONTINUOUSLY option for readfile in the documentation.
However, if the data is in Kafka, it would be much more natural to use Flink's Kafka consumer to stream the data directly into Flink. There is also documentation about using the Kafka connector. And the Flink training includes an exercise on using Kafka with Flink.

Single User Application File/Data Storage Recommendations

I currently have a C# winform application in which you enter data that is ultimately relational. The amount of data being stored isn't huge. The original version used SQL CE to store the information. However, I found it to be quite slow. Also, I wanted to be able to save application files using my own extension.
I had changed my approach to basically keep my data loaded in-memory using class objects. To save, I simply serialize everything using ProtoBuf and deserialize when opening a file. This approach is lightning fast and changes are never persisted until a user clicks save. However, I find it a little cumbersome to query my hierarchical data. I query data using Linq-To-Objects. I'll have ClassA having a GUID key. I can reference ClassA in ClassB via the GUID. However, I can't really do an easy SQL join-type query to get ClassB properties along with ClassA properties. I get around it by creating a navigation property on ClassB to ClassA that simple returns ClassA via a LINQ query on the GUID. However, this results in a lot of collection scanning.
What options are out there that give me fast, single-user, relational file storage? I would still like to work in-memory where changes aren't persisted until a user uses File|Save. I would also like to be able to continue querying the data using LINQ. I'm looking at SQLite as an option. Are there better options or approaches out there for me?
UPDATE
I was unaware of the AsReference option in the ProtoMember attribute [ProtoMember(5, AsReference = true)]. If I abandon foreign keys in my classes and simply reference the related objects, then it looks like I'm able to serialize and deserialize using ProtoBuf while keeping my object references. Thus, I can easily use Linq-To-Objects to query my objects. I need to stop thinking from the database side of things.
If you have all your objects in some sort of hierarchical structure, you can also store the exact same objects in other structures at an overhead of 4 bytes/object (32bit machines).
Assuming you have a base object like:
public class HierarchyElement
{
public List<HierarchyElement> Children { get; set; }
public HierarchyElement Parent { get; set; }
}
So you have the root element in a local variable, which via the Children property, and the Children property of those first children, etc etc store an unknown number of objects in a hierarchy.
However, while you are building that object, or after deserialising it, you can add a reference to each HierarchyElement to a List (or other flat structure of your choice).
You can then use this flat list to do your Linq queries against.

How to get all the AutomationIDs of a WPF application in a file?

In automation of a WPF application (using UI Automation; VSTS 2010), we were adding all the Automation IDs in a Resource File manually and then access it one by one. Considering the application can expand any time, manually adding these IDs can become tedious.
So, is there any tool available which can create this for us? i.e. Get all the ids in a hierarchical format and store it in a file (xml or csv), and then we could parse it whenever required.
I was hoping for a tool like UISpy, which not only can spy all the elements but also export the same.
Do such tools exist? Or is there any alternate approach?
Any valuable feedback is highly appreciated.
Thanks!
I do like this:
public static class AutomationIds
{
public static readonly string MyDataGridId= Create();
private static string Create([CallerMemberName] string name = null)
{
return name;
}
}
<DataGrid AutomationProperties.AutomationId="{x:Static local:AutomationIds.MyDataGridId}"
... />
Then in tests
var dataGrid = window.Get<ListView>(AutomationIds.MyDataGridId);
Assign the automation IDs directly in XAML, then parse XAML files since they are XML after all...
Let's see...
First, I think that your data is not hierarchical - just because a control can be dynamically assigned to be a child of another.
If we change the problem to a subset: "how can we get a hierarchical view of the controls at a time t?" then we can answer this with MS UIA, and say, using a simple RawViewWalker (just a simple breadth-first search on the walker, starting from your main window will do - of course while the application is running so that UIA can reach and query it).
But this subset will not satisfy your initial question, because you'll probably see a portion of your whole ui collection (since some will be hidden / not activated yet at time t).
So it becomes very hard to use a UIA based tool (such as uispy) because then you'll have to set the application view to different states to reach all the controls in your application at different times t1, t2...
I would suggest parsing all your xmls at the same time and build a complete tree of the application's "static" control map, which I believe would be closest to what you're asking for.
Given that this is an old question, I doubt it matters anymore, but just wanted to make the distinctions here.

Grails - Where to store properties related to domains

This is something I have been struggling about for some time now. The thing is: I have many (20 or so) static arrays of values. I say static because that is how I'm actually storing them, as static arrays inside some domains. For example, if I have a list of known websites, I do:
class Website {
...
static websites = ["web1", "web2" ...]
}
But I do this just while developing, because I can easily change the arrays if needed, but what I'm going to do when the application is ready for deployment? In my project it is very probable that, at some point, these arrays of values change. I've been researching on that matter, one can store application properties inside an external .properties file, but it will be impossible to store an array, even futile, because if some array gets an additional value, the application can't recognize it until the name of the new property is added where needed.
Another approach is to store this information in the database, but for some reason it seems like a waste to add 20 or more tables that will have just two rows, an id and a name.
And the last option, as far as I know, would be an XML, but I'm not very experienced with those. It seems groovy has a way of creating and reading XML files relatively easy, but I don't know how difficult would be to modify an XML whose layout is predefined in the application.
Needless to say that storing them in the config.groovy is not an option since any change will require to recompile.
I haven't come across some "standard" (maybe a best practice?) way of dealing with these.
So the questions is: Where to store these arrays?
Use Enum for a fixed set of attributes. Do this, if you rely at some places in your code on some concrete values.
If you do not rely on the attributes within your code (which you obviously don't), use a String-type. In this case, if you need to provide a select box, just do a distinct-query on this property in your database.
Default: Use a domain class per dynamic set of attributes. Another table is OK.
For something as simple as arrays you should use groovy own type of property files. They allow you too define properties as proper groovy variables (not just strings) and obviously loading them would be done dinamically in a simple way by using ConfigSlurper. For an example on how to use this kind of file you can have a look at the following ConfigSlurper:
For defining properties:
my.property.array=[1,2,3,4]
For loading property files:
def config = new ConfigSlurper().parse(new File('myconfig.groovy').toURL())
assert [1,2,3,4] == config.my.property.array
Tip: When you want to access the property files you want to do it in a way that can work for any environment. To make paths environment-independent use the following path as the root path:
import org.codehaus.groovy.grails.commons.ApplicationHolder
def ctx = ApplicationHolder.application.mainContext.servletContext
def rootPath = ctx.contextPath

Resources