How does one work with string resources in python/gae in eclipse+pydev environment? looking for something that is the equivalent of this link. Also, what is the recommended approach for working with large strings? Create a text file as part of your app and use file I/O, or something else?
for those still seeking a solution for string resources file on GAE and Python (2.7), I used a YAML formated file:
create a file named "strings.txt" (for example), it's content can be something like this:
level_1_name: this is a text
# this is a comment
test:
- testtitle: test title
testtitlechild: testtitlechild
place this on the bottom of your main application file:
# strings file (YAML format) initialization
strings = yaml.load(open("strings.txt", "r"))
then you get 'strings' as dict, using it goes as follows:
strings["level_1_name"]
Assuming "string resource" is a fancy name for text-file, you have three choices;
If your large text file is read only then you can bundle it along with your other application files and access it as you would normally (via open() or similar method).
If your application needs to write lots of text-data then you will have to store it in the datastore using a TextProperty but beware there are limits on the amoount of data that can be written to the datastore in one go (currently 1MB) ... OR...
Use the Blobstore API to read/write files
Related
My question seems to be pretty straight forward but, I haven't been able to find any solutions to this online. I've looked at a number of different types of objects like DataTables and DataAssets only to realize they are for static data alone.
The goal of my project is to have data-driven configurable assets where we can choose different configurations for our different objects. I have been able to successfully pull JSON data down from the database at run-time but, I would like to save said data to something like a Data Asset or something similar that I can read and write to. So when we pull from said database later we only pull updates to our different configurations and not the entire database (every time at start-up).
On a side note: would this be possible/feasible using an .ini file or is this kind of thing considered too big for something like that (i.e 1000+ json objects)?
Any solutions to this problem would be greatly appreciated.
Like you say, DataTable isn't really usable here. You'll need to utilize UE4's various File IO API utilities.
Obtaining a Local Path
This function converts a path relative to your intended save directory, into one that's relative to the UE4 executable, which is the format expected throughout UE4's File IO.
//DataUtilities.cpp
FString DataUtilities::FullSavePath(const FString& SavePath) {
return FPaths::Combine(FPaths::ProjectSavedDir(), SavePath);
}
"Campaign/profile1.json" as input would result in something like:
"<game.exe>/game/Saved/Campaign/profile1.json".
Before you write anything locally, you should find the appropriate place to do it. Using ProjectSaveDir() results in saving files to <your_game.exe>/your_game/Saved/ in packaged builds, or in your project's Saved folder in development builds. Additionally, FPaths has other named Dir functions if ProjectSavedDir() doesn't suit your purpose.
Using FPaths::Combine to concatenate paths is less error-prone than trying to append strings with '/'.
Storing generated JSON Text Data on Disk
I'll assume you have a valid JSON-filled FString (as opposed to a FJSONObject), since generating valid JSON is fairly trivial.
You could just try to write directly to the location of the full path given by the above function, but if the directory tree to it doesn't exist (i.e., first-run), it'll fail. So, to generate that path tree, there's some path processing and PlatformFile usage.
//DataUtilities.cpp
void DataUtilities::WriteSaveFile(const FString& SavePath, const FString& Data) {
auto FullPath = FullSavePath(SavePath);
FString PathPart, Disregard;
FPaths::Split(FullPath, PathPart, Disregard, Disregard);
IPlatformFile& PlatformFile = FPlatformFileManager::Get().GetPlatformFile();
if (PlaftormFile.CreateDirectoryTree(*PathPart)){
FFileHelper::SaveStringToFile(Data, *FullPath);
}
}
If you're unsure what any of this does, read up on FPaths and FPlatformFileManager in the documentation section below.
As for generating a JSON string: Instead of using the Json module's DOM, I generate JSON strings directly from my FStructs when needed, so I don't have experience with using the Json module's serialization functionality. This answer seems to cover that pretty well, however, if you go that route.
Pulling Textual Data off the Disk
// DataUtilities.cpp
bool DataUtilities::SaveFileExists(const FString& SavePath) {
return IFileManager::Get().FileExists(*FullSavePath(SavePath));
}
FString DataUtilities::ReadSaveFile(const FString& SavePath) {
FString Contents;
if(SaveFileExists(SavePath)) {
FFileHelper::LoadFileToString(Contents, *FullSavePath(SavePath));
}
return Contents;
}
As is fairly obvious, this only works for string or string-like data, of which JSON qualifies.
You could consolidate SaveFileExists into ReadSaveFile, but I found benefit in having a simple "does-this-exist" probe for other methods. YMMV.
I assume if you're already pulling JSON off a server, you have a means of deserializing it into some form of traversable container. If you don't, this is an example from the UE4 Answer Hub of using the Json module to do so.
Relevant Documentation
FFileHelper
FFileHelper::LoadFileToString
FFileHelper::SaveStringToFile
IFileManager
FPlatformFileManager
FPaths
UE4 Json.h (which you may already be using)
To address your side note: I would suggest using an extension that matches the type of content saved, if for nothing other than clarity of intention. I.e., descriptive_name.json for files containing JSON. If you know ahead of time that you will be reading/needing all hundreds or thousands of JSON objects at once, it would likely be better to group as many as possible into fewer files, to minimize overhead.
Hello I'm coding a simple CRUDE application that runs perfectly using JSF and DB4O.
I can add and list all the entities normally without errors.
I've used this code to persist the entities
bd = Db4oEmbedded.openFile(configuration, "db.data");
bd.store(client);
bd.commit();
the problem is that I cant locate the db.data file. I,ve done a search in the whole PC and still cant find it.
soo where DB4O store those entities, how is it possible??
thank you.
It must be in your project execution path. Usually db4o file name is called with extension yap. Better you can use relative path to the database file like
String dbPath = "c:/db/project.yap"; // better read from project resource
EmbeddedObjectContainer openFile = Db4oEmbedded.openFile(dbPath);
I am trying to create an app which has a predefined set of data(currently a text file) and it reads from it, now how should i implement the data storage such that when i share the app with someone else, i don't have to pass the text file or any other data file(if possible, i can encrypt the contents and give it using a different extension) externally. I just want to give one exe file to the person and the data should be included inside the exe.
anyway to do it ??
Thanks in advance
Include the text file as a resource.
You do not need to change the build (if you are using Visual Studio that is). Visual Studio will embed the resource/file and generate a readonly property for the file so you can access it directly from within your code:
string fileContent = YourResourceFile.TheEmbeddedFile;
You could split the fileConent per linebreak but the previous line will load the entire file into memory.
string[] lines = fileContent.Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
If the file is too big to be read into memory at once, you could stream the resource as explained here.
You can change the build mode of the file to resource/embedded resource, reading from it works different then, this should be answered somewhere though.
I am trying to make a new file format for music. It needs to be a file that actually stores multiple audio files, for example a zip file. I am looking for a way to turn the zip file into this new file format. However, I still want to use id3 tags with these new files. I was wondering how I can make this new file format which is one file that holds multiple audio files, but still contains overall id3 tags for that one file, so that I can load it into my mobile applications.
Any help/recommendations would be appreciated.
Cheers,
AJ
The problem with creating your own new file format is that only you can use it. Until you convince lots of other people that it is a useful new format, no one else will have the tools to be able to do anything with the files you create.
For existing music player programs to be able to handle a new file format, you must write a CODEC for your file format in that player's plug-in style. Probably more than one plug-in as your file format is both a container of music and a catalog as well.
One alternative to creating a new file format is to put the MP3 files you have into a new MP3 file with each old file a new TRACK in the new file. Be sure to set each new tracks start time to be after the sum of all previous tracks play duration, so they don't step on each other. Merge the metadata about each file into the metadata of the new file. This might work OK for collections with lots of common metadata (like same artist), but might not work very well if the metadata is extremely varied.
Another alternative is to convert them to CDA format and put then into an Audio-CD image file, just as if you had burned them to a CD.
A third alternative is to put your files into an SQLite database file. Your metadata from each MP3 file fill in records, and you have your choice of leaving the MP3 file external and just linking to it, or storing the blob of your MP3 file in the DB as well. If you do store the blobs, then the SQLite database file is a single file that contains everything you put in it.
-Jesse
Don't create new formats unless you really really have a very good reason to do so.
Its sounds like Matroska can do anything you need. But in contrast to your own format you and other developers will have a bunch of ready to use tools to work with the format. This includes editors, players,... Additionally it you can leave making initial engineering errors to other people.
If you really really want to create your own format: Either just put your audio files that support id3 tags into your zip file, or create a meta file, for example in XML format, into your zip files as well, that contains the meta information that you want to be included.
I'm including a number of images as "Content" in my deployed XAP for Mango.
I'd like to enumerate these at runtime - is there any way to do this?
I've tried enumerating resources like:
foreach (string key in Application.Current.Resources.Keys)
{
Debug.WriteLine("Resource:" + key);
}
But the images aren't included in the list. I've also tried using embedded resources instead - but that didn't help. I can read the streams using Application.GetResourceStream(uri) but obviously I need to know the names in order to do this.
This is no API baked in to WP7 that allows you to enumerate the contents of the Xap. You need to know the name of the content items before you can retreive them.
There probably is some code floating around somewhere that is able to sniff out the Zip catalog in the XAP however I would strongly recommend that you don't bother. Instead include some sensible resource such as an Xml file or ResourceDictionary that lists them.
Having found no practical way to read the Content files from a XAP I build such a list at design time using T4.
See an example at https://github.com/mrlacey/phonegap-wp7/blob/master/WP7Gap/WP7Gap/MainPage.xaml.cs
This seems the right way to go as:
a) I'd rather build the list once at design time rather than on every phone which needs the code.
and
b) I shouldn't ever be building the XAP without being certain about what files I'm including anyway.
Plus it's a manual step to set the build action on all such files so adding a manual step to "Run Custom Tool" once for each build isn't an issue for me.
There is no way to enumerate the files set as "Content".
However, there is a way to enumerate files at runtime, if you set your files as "Embedded Resource".
Here is how you can do this:
Set the Build Action of your images as "Embedded Resource".
Use Assembly.GetCallingAssembly().GetManifestResourceNames() to
enumerate the resources names
Use
Assembly.GetCallingAssembly().GetManifestResourceStream(resName)
to get the file streams.
Here is the code:
public void Test()
{
foreach (String resName in GetResourcesNames())
{
Stream s = GetStreamFromEmbeddedResource(resName);
}
}
string[] GetResourcesNames()
{
return Assembly.GetCallingAssembly().GetManifestResourceNames();
}
Stream GetStreamFromEmbeddedResource(string resName)
{
return Assembly.GetCallingAssembly().GetManifestResourceStream(resName);
}
EDIT : As quetzalcoatl noted, the drawback of this solution is that images are embedded in the DLL, so if you a high volume of images, the app load time might take a hit.