I have no idea in what extend GAE is not easy to understand :(
My servlet manipulate a json string and then I'm trying to store it in datastore.
When I run the application I'm getting this output:
Jan 27, 2014 6:59:04 PM com.google.appengine.api.datastore.dev.LocalDatastoreService load
INFO: The backing store, D:\Android\IntelliJ IDEA\workspace\EyeBall\AppEngine\out\artifacts\AppEngine_war_exploded\WEB-INF\appengine-generated\local_db.bin, does not exist. It will be created.
1
2
3
4
5
7
***
***
***
***
***
8
9
Although it's mentioned that local_db.bin will be created but when I navigate to that directory the file is not there. Also, when I open http://localhost:8080/_ah/admin/datastore in browser nothing displays in Entity Kind drop down list.
So wtf happene to local_db.bin? Why it doesn't generates?
any suggestion would be appreciated. thanks.
==================
UPDATE:
I added my code based on request.
private static final String NO_DEVICE_ID = "FFFF0000";
private static final String SAMPLE_JSON = "{\"history\":[{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com\",\"url\":\"https://www.maybank2u.com.my/mbb/Mobile/info.do\",\"visits\":14},{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com\",\"url\":\"https://www.maybank2u.com.my/mbb/Mobile/adaptInfo.do\",\"visits\":4},{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com\",\"url\":\"http://www.maybank2u.com.my/mbb_info/m2u/public/personalBanking.do\",\"visits\":16},{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com Online Financial Services\",\"url\":\"https://www.maybank2u.com.my/mbb/m2u/common/M2ULogin.do?action=Login\",\"visits\":52},{\"date\":null,\"info\":null,\"title\":\"BBC\",\"url\":\"http://www.bbc.co.uk/persian/\",\"visits\":16}]}";
private static final String QUERY_HISTORY_DEVICE = "SELECT m FROM HistoryDeviceJPA m WHERE m.userUUID = :keyword ORDER BY m.domain ASC";
private static final String QUERY_HISTORY = "SELECT m FROM HistoryJPA m WHERE m.pageAddress = :keyword";
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// displayError(response, "The page doesn't support httpGet");
String deviceId = NO_DEVICE_ID;
String content = SAMPLE_JSON;
System.out.println("1");
HistoryBrowser historyBrowser = parseJson(content);
if(historyBrowser == null)
return;
System.out.println("2");
List<HistoryBrowser.BrowserInfo> historyList = historyBrowser.getHistory();
if(historyList == null)
return;
System.out.println("3");
List<HistoryDeviceJPA> historyDeviceJPAList = new ArrayList<HistoryDeviceJPA>(historyList.size());
for(int i=0; i<historyList.size(); i++) {
try {
HistoryBrowser.BrowserInfo browser = historyList.get(i);
HistoryDeviceJPA historyDeviceJPA = new HistoryDeviceJPA();
historyDeviceJPA.setUserUUID(deviceId);
historyDeviceJPA.setDomain(getDomainName(browser.getUrl()));
historyDeviceJPA.setPageAddress(browser.getUrl());
historyDeviceJPA.setPageTitle(browser.getTitle());
historyDeviceJPA.setPageVisits(browser.getVisits());
historyDeviceJPAList.add(historyDeviceJPA);
} catch (URISyntaxException e) {
System.out.println(e.getMessage());
}
}
System.out.println("4");
// get history of device from data store
EntityManager em = EMF.get().createEntityManager();
Query q = em.createQuery(QUERY_HISTORY_DEVICE).setParameter("keyword", deviceId);
#SuppressWarnings("unchecked")
List<HistoryDeviceJPA> dbList = (List<HistoryDeviceJPA>) q.getResultList();
System.out.println("5");
// If there is no result (shows there is no record for that device)
if(dbList == null)
addHistoryDeviceJPAToDs(historyDeviceJPAList);
else {
System.out.println("7");
// find each item in datastore and replace them if needed
// if current page visit is less ot equal than previous visit don't do anything (remove item form historyDeviceJPAList)
outerLoop:
for(int i=0; i<historyDeviceJPAList.size(); i++) {
HistoryDeviceJPA deviceItem = historyDeviceJPAList.get(i);
System.out.println("***");
for(int j=0; j<dbList.size(); j++) {
HistoryDeviceJPA dbItem = dbList.get(j);
if(deviceItem.getPageAddress().equalsIgnoreCase(dbItem.getPageAddress())) {
if(deviceItem.getPageVisits() > dbItem.getPageVisits()) {
long diff = deviceItem.getPageVisits() - dbItem.getPageVisits();
dbItem.setPageVisits(deviceItem.getPageVisits());
HistoryJPA historyJPA = findHistoryJPA(dbItem.getPageAddress());
historyJPA.setPageVisits(historyJPA.getPageVisits() + diff);
// update datastore
addHistoryDeviceJPAToDs(dbItem);
addHistoryJPAToDs(historyJPA);
// don't check other items of j list
break outerLoop;
}
}
}
}
System.out.println("8");
}
System.out.println("9");
// http://www.sohailaziz.com/2012/06/scheduling-activities-services-and.html
// https://dev.twitter.com/docs/api/1.1
// https://developers.google.com/appengine/docs/java/datastore/jdo/creatinggettinganddeletingdata?csw=1#Updating_an_Object
// http://en.wikibooks.org/wiki/Java_Persistence/Inheritance
}
and 6 is here:
private void addHistoryDeviceJPAToDs(List<HistoryDeviceJPA> list) {
System.out.println("6");
EntityManager em = EMF.get().createEntityManager();
try {
for (int i=0; i<list.size(); i++) {
System.out.println("=> " + i + " - " + list.get(i).toString());
em.getTransaction().begin();
em.persist(list.get(i));
em.getTransaction().commit();
}
} finally {
em.close();
}
}
after debug I found the problem is in this line:
List<HistoryDeviceJPA> dbList = (List<HistoryDeviceJPA>) q.getResultList();
if(dbList == null)
addHistoryDeviceJPAToDs(historyDeviceJPAList);
'dbList' is never null and it's size is 0 if there is nothing in datastore. That's why addHistoryDeviceJPAToDs method never invoked. By changing the code to following problem solved and local db created.
List<HistoryDeviceJPA> dbList = (List<HistoryDeviceJPA>) q.getResultList();
if(dbList == null)
return;
System.out.println("5");
// If there is no result (shows there is no record for that device)
if(dbList.size() == 0)
addHistoryDeviceJPAToDs(historyDeviceJPAList);
For other people who come across the same issue --
GAE will not create local_db.bin until you put data in the datastore. So if the file is not there, there is likely a bug in the application code.
Related
This is my first app on Android with Java and SQLite.
ISSUE:
I have a local SQLIte db on my app. I was very surprised to see how easy it is to get access to the db once you have installed the app (no need to be a programmer nor a hacker).
I tried adding SQLCipher to my app but it only worked for newer Android versions 11 & 12 and didn't work for Android 9 for example and it did make my app's size much bigger.
After researching more I found a better solution for my case which doesn"t involve crypting the db with SQLCipher but rather it consists of corrupting the first bytes of the db file then after each launch of the app the code will decorrupt the file and use the fixed file instead. This insures that anyone who decompiles the apk will only get access to a corrupt db file and will have to put more effort to fix it which is my goal.
I came across this solution in a reply [here][1] but I don't know how to implement it as I am new to Android and SQLite programming. Any help is much appreciated on how to actually do it.
These are the steps as mentioned by the user: farhad.kargaran which need more explanation as I don't get how to do it:
1- corrupt the db file (convert it to byte array and change some values)
2- copy it in asset folder
3- in first run fix corrupted file from asset and copy it in database
folder.
Change first 200 byte values like this:
int index = 0;
for(int i=0;i<100;i++)
{
byte tmp = b[index];
b[index] = b[index + 1];
b[index + 1] = tmp;
index += 2;
}
As only the first 200 bytes were replaced, the same code is used for fixing first 200 byte values.
Here is my code for the SQLiteOpenHelper if needed:
public class DatabaseHelper extends SQLiteOpenHelper {
private static final String TAG = DatabaseHelper.class.getSimpleName();
public static String DB_PATH;
public static String DB_NAME;
public SQLiteDatabase database;
public final Context context;
public SQLiteDatabase getDb() {
return database;
}
public DatabaseHelper(Context context, String databaseName, int db_version) {
super(context, databaseName, null, db_version);
this.context = context;
DB_PATH = getReadableDatabase().getPath();
DB_NAME = databaseName;
openDataBase();
// prepare if need to upgrade
int cur_version = database.getVersion();
if (cur_version == 0) database.setVersion(1);
Log.d(TAG, "DB version : " + db_version);
if (cur_version < db_version) {
try {
copyDataBase();
Log.d(TAG, "Upgrade DB from v." + cur_version + " to v." + db_version);
database.setVersion(db_version);
} catch (IOException e) {
Log.d(TAG, "Upgrade error");
throw new Error("Error upgrade database!");
}
}
}
public void createDataBase() {
boolean dbExist = checkDataBase();
if (!dbExist) {
this.getReadableDatabase();
this.close();
try {
copyDataBase();
} catch (IOException e) {
Log.e(TAG, "Copying error");
throw new Error("Error copying database!");
}
} else {
Log.i(this.getClass().toString(), "Database already exists");
}
}
private boolean checkDataBase() {
SQLiteDatabase checkDb = null;
try {
String path = DB_PATH + DB_NAME;
checkDb = SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READONLY);
} catch (SQLException e) {
Log.e(TAG, "Error while checking db");
}
if (checkDb != null) {
checkDb.close();
}
return checkDb != null;
}
private void copyDataBase() throws IOException {
InputStream externalDbStream = context.getAssets().open(DB_NAME);
String outFileName = DB_PATH + DB_NAME;
OutputStream localDbStream = new FileOutputStream(outFileName);
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = externalDbStream.read(buffer)) > 0) {
localDbStream.write(buffer, 0, bytesRead);
}
localDbStream.close();
externalDbStream.close();
}
public SQLiteDatabase openDataBase() throws SQLException {
String path = DB_PATH + DB_NAME;
if (database == null) {
createDataBase();
database = SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READWRITE);
}
return database;
}
#Override
public synchronized void close() {
if (database != null) {
database.close();
}
super.close();
}
Much appreciated.
[1]: https://stackoverflow.com/a/63637685/18684673
As part of the copyDatabase, correct and then write the corrupted data, then copy the rest.
Could be done various ways
e.g.
long buffersRead = 0; //<<<<< ADDED for detecting first buffer
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = externalDbStream.read(buffer)) > 0) {
if (bufferesRead++ < 1) {
//correct the first 200 bytes here before writing ....
}
localDbStream.write(buffer, 0, bytesRead);
}
I find the best way to save game data in Unity3D Game engine.
At first, I serialize objects using BinaryFormatter.
But I heard this way has some issues and is not suitable for save.
So, What is the best or recommended way for saving game state?
In my case, save format must be byte array.
But I heard this way has some issues and not suitable for save.
That's right. On some devices, there are issues with BinaryFormatter. It gets worse when you update or change the class. Your old settings might be lost since the classes non longer match. Sometimes, you get an exception when reading the saved data due to this.
Also, on iOS, you have to add Environment.SetEnvironmentVariable("MONO_REFLECTION_SERIALIZER", "yes"); or you will have problems with BinaryFormatter.
The best way to save is with PlayerPrefs and Json. You can learn how to do that here.
In my case, save format must be byte array
In this case, you can convert it to json then convert the json string to byte array. You can then use File.WriteAllBytes and File.ReadAllBytes to save and read the byte array.
Here is a Generic class that can be used to save data. Almost the-same as this but it does not use PlayerPrefs. It uses file to save the json data.
DataSaver class:
public class DataSaver
{
//Save Data
public static void saveData<T>(T dataToSave, string dataFileName)
{
string tempPath = Path.Combine(Application.persistentDataPath, "data");
tempPath = Path.Combine(tempPath, dataFileName + ".txt");
//Convert To Json then to bytes
string jsonData = JsonUtility.ToJson(dataToSave, true);
byte[] jsonByte = Encoding.ASCII.GetBytes(jsonData);
//Create Directory if it does not exist
if (!Directory.Exists(Path.GetDirectoryName(tempPath)))
{
Directory.CreateDirectory(Path.GetDirectoryName(tempPath));
}
//Debug.Log(path);
try
{
File.WriteAllBytes(tempPath, jsonByte);
Debug.Log("Saved Data to: " + tempPath.Replace("/", "\\"));
}
catch (Exception e)
{
Debug.LogWarning("Failed To PlayerInfo Data to: " + tempPath.Replace("/", "\\"));
Debug.LogWarning("Error: " + e.Message);
}
}
//Load Data
public static T loadData<T>(string dataFileName)
{
string tempPath = Path.Combine(Application.persistentDataPath, "data");
tempPath = Path.Combine(tempPath, dataFileName + ".txt");
//Exit if Directory or File does not exist
if (!Directory.Exists(Path.GetDirectoryName(tempPath)))
{
Debug.LogWarning("Directory does not exist");
return default(T);
}
if (!File.Exists(tempPath))
{
Debug.Log("File does not exist");
return default(T);
}
//Load saved Json
byte[] jsonByte = null;
try
{
jsonByte = File.ReadAllBytes(tempPath);
Debug.Log("Loaded Data from: " + tempPath.Replace("/", "\\"));
}
catch (Exception e)
{
Debug.LogWarning("Failed To Load Data from: " + tempPath.Replace("/", "\\"));
Debug.LogWarning("Error: " + e.Message);
}
//Convert to json string
string jsonData = Encoding.ASCII.GetString(jsonByte);
//Convert to Object
object resultValue = JsonUtility.FromJson<T>(jsonData);
return (T)Convert.ChangeType(resultValue, typeof(T));
}
public static bool deleteData(string dataFileName)
{
bool success = false;
//Load Data
string tempPath = Path.Combine(Application.persistentDataPath, "data");
tempPath = Path.Combine(tempPath, dataFileName + ".txt");
//Exit if Directory or File does not exist
if (!Directory.Exists(Path.GetDirectoryName(tempPath)))
{
Debug.LogWarning("Directory does not exist");
return false;
}
if (!File.Exists(tempPath))
{
Debug.Log("File does not exist");
return false;
}
try
{
File.Delete(tempPath);
Debug.Log("Data deleted from: " + tempPath.Replace("/", "\\"));
success = true;
}
catch (Exception e)
{
Debug.LogWarning("Failed To Delete Data: " + e.Message);
}
return success;
}
}
USAGE:
Example class to Save:
[Serializable]
public class PlayerInfo
{
public List<int> ID = new List<int>();
public List<int> Amounts = new List<int>();
public int life = 0;
public float highScore = 0;
}
Save Data:
PlayerInfo saveData = new PlayerInfo();
saveData.life = 99;
saveData.highScore = 40;
//Save data from PlayerInfo to a file named players
DataSaver.saveData(saveData, "players");
Load Data:
PlayerInfo loadedData = DataSaver.loadData<PlayerInfo>("players");
if (loadedData == null)
{
return;
}
//Display loaded Data
Debug.Log("Life: " + loadedData.life);
Debug.Log("High Score: " + loadedData.highScore);
for (int i = 0; i < loadedData.ID.Count; i++)
{
Debug.Log("ID: " + loadedData.ID[i]);
}
for (int i = 0; i < loadedData.Amounts.Count; i++)
{
Debug.Log("Amounts: " + loadedData.Amounts[i]);
}
Delete Data:
DataSaver.deleteData("players");
I know this post is old, but in case other users also find it while searching for save strategies, remember:
PlayerPrefs is not for storing game state. It is explicitly named "PlayerPrefs" to indicate its use: storing player preferences. It is essentially plain text. It can easily be located, opened, and edited by any player. This may not be a concern for all developers, but it will matter to many whose games are competitive.
Use PlayerPrefs for Options menu settings like volume sliders and graphics settings: things where you don't care that the player can set and change them at will.
Use I/O and serialization for saving game data, or send it to a server as Json. These methods are more secure than PlayerPrefs, even if you encrypt the data before saving.
This code has been working fine for years.
public PageReference chkinst() {
i=0;
integer j=0;
Set<String> aSearchSet = new Set<String>();
List<Lead> lList = ui;
for (Lead l : lList) {
aSearchSet.add(l.company);
}
Account[] accountToCreate = new Account[]{};
Map<String,Account> companyToAccountMap = new Map<String,Account> ();
for (Account a: [select id, Name from Account where name IN :aSearchSet])
companyToAccountMap.put(a.name,a);
for (Lead l : lList) {
if (!companyToAccountMap.containsKey(l.company)){
Account act = new Account(Name = l.Company , Country__c = l.Country__c );
accountToCreate.add(act);
j = j +1;
}
}
insert accountToCreate;
if(j == 0) {
ApexPages.addMessage(new ApexPages.message(ApexPages.severity.INFO,'All leads have correct institution names'));
} else{
ApexPages.addMessage(new ApexPages.message(ApexPages.severity.INFO,j + ' Institutions created'));
}
return null;
}
Today, it gave me the following error:
Maximum view state size limit (135KB) exceeded. Actual view state size for this page was 135.078KB
Does anyone know why?
Thanks,
It's hard to tell from this code snippet. But if I had to guess. It looks like your companyToAccountMap is a class level variable, and it's probably returning more records now then it used to. If you don't need that map anywhere else in the controller, I would move it to a method level or make it transient.
This question already has answers here:
how to read all cell value using Apache POI?
(2 answers)
Closed 7 years ago.
I had created my script to validate my actual result and expected result , .
As there is too many link to validated script will get too much of coding m so i need to convert this into Data Driven Case ,.
Where Webdriver will get URL , xpath ,expected value from excel .
But dont know how to proceed , .
A demo code is much appreciated
Here is my current script :
public void test() throws Exception
{
String home_logo_url="158321";
String enough_talk_promo="1057406";
System.out.println("Check for home_logo_url");
driver.get(baseUrl);
String SiteWindow = driver.getWindowHandle(); // get the current window handle
driver.findElement(By.xpath("//*[#id='logo']/a")).click();
for (String PromoWindow : driver.getWindowHandles())
{
driver.switchTo().window(PromoWindow); // switch focus of WebDriver to the next found window handle (that's your newly opened window)
}
String script = "return rlSerial;";
String value = (String) ((JavascriptExecutor)driver).executeScript(script);
Assert.assertEquals(value,home_logo_url);
driver.close();
driver.switchTo().window(SiteWindow);
System.out.println("Pass");
System.out.println("Check for enough_talk_promo");
driver.get(baseUrl + "/category/tournaments/");
driver.findElement(By.xpath("//*[#id='content']/div/div[4]/aside/div/div/p[1]/a")).click();
for (String PromoWindow : driver.getWindowHandles())
{
driver.switchTo().window(PromoWindow); // switch focus of WebDriver to the next found window handle (that's your newly opened window)
}
String sr_enough_talk_promo = (String) ((JavascriptExecutor)driver).executeScript(script);
Assert.assertEquals(sr_enough_talk_promo,enough_talk_promo);
driver.close();
driver.switchTo().window(SiteWindow);
System.out.println("Pass");
}
How to iterated to each rows and get my test case run !!!
It is much helpful , if some one can convert my existing code to work on excel sheet .
Thanks
i was working on a project in Spring that read from an excel (.xls) and this was my code if can help
private List<String> extraire (String fileName) throws IOException {
List<String> liste = new ArrayList();
FileInputStream fis = new FileInputStream(new File(fileName));
HSSFWorkbook workbook = new HSSFWorkbook(fis);
HSSFSheet spreadsheet = workbook.getSheetAt(0);
Iterator < Row > rowIterator = null;
rowIterator = spreadsheet.iterator();
while (rowIterator.hasNext()) {
int i = 0;
row = (HSSFRow) rowIterator.next();
Iterator < Cell > cellIterator = row.cellIterator();
while ( cellIterator.hasNext()) {
Cell cell = cellIterator.next();
i++;
/**
* For verifying if a line is empty
*/
if (i % 29 == 0 || i == 1) {
while ( cellIterator.hasNext() && cell.getCellType() == Cell.CELL_TYPE_BLANK) {
cell = cellIterator.next();
}
}
switch (cell.getCellType()) {
case Cell.CELL_TYPE_NUMERIC:
String cellule = String.valueOf(cell.getNumericCellValue());
liste.add(cellule);
break;
case Cell.CELL_TYPE_STRING:
liste.add(cell.getStringCellValue());
break;
case Cell.CELL_TYPE_BLANK:
cellule = " ";
liste.add(cellule);
break;
}
}
}
fis.close();
return liste;
}
in my controller :
List<String> liste = new ArrayList();
liste = extraire(modelnom);
for (int m=0, i=29;i<liste.size();i=i+29) {
if(i % 29 == 0) {
// i=i+29 : begin from the second line first coll
// m is line
m++;
}
String matricule = (String)liste.get(29*m).toString().trim();
float mat = Float.parseFloat(matricul); // From String to float
employe.setMatricule((int)mat); //reading mat as an int
// ... your Code
}
I am working with Admin SDK .when I update single user it's working fine but when I am trying to update bunch(1000) of users I got User Rate Limit Exceeded Exception. Please check below my code and tell me what am i missing ? or tell any suggestion ?
private Directory getDirectoryService(String adminEmailAddress){
Directory directoryService = null;
try {
Collection<String> SCOPES = new ArrayList<String>();
SCOPES.add("https://www.googleapis.com/auth/admin.directory.user");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.user.readonly");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.group");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.group.readonly");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.orgunit");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.orgunit.readonly");
SCOPES.add("https://www.googleapis.com/auth/userinfo.profile");
HttpTransport httpTransport = new NetHttpTransport();
JacksonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
.setServiceAccountScopes(SCOPES)
.setServiceAccountUser(adminEmailAddress)
.setServiceAccountPrivateKeyFromP12File(new java.io.File(SERVICE_ACCOUNT_PKCS12_FILE_PATH)).build();
directoryService = new Directory.Builder(httpTransport,jsonFactory, credential).setApplicationName("gdirectoryspring").build();
} catch(Exception e){
ErrorHandler.errorHandler(this.getClass().getName(), e);
}
return directoryService;
}
Below is Update User Code
Directory directoryService = getDirectoryService("adminEmailAddress");
User user = directoryService.users().get("userPrimaryEmail").execute();
List<UserOrganization> organizaionList = user.getOrganizations();
for (int j = 0; j < organizaionList.size(); j++)
{
UserOrganization singleOrg = organizaionList.get(j);
if (singleOrg != null)
{
if ("work".equalsIgnoreCase(singleOrg.getCustomType()) ||singleOrg.getPrimary() != null)
{
if (singleOrg.getTitle() != null)
{
singleOrg.setTitle(jobTitle);
}
}
}
user.setOrganizations(organizaionList);
}
Update update= directoryService.users().update(primaryEmail, user);
User userUpdated = update.execute();
and in admin console I increased my limit like below
Admin SDK 10.0 requests/second/user
but till I am getting User Rate Limit Exceeded Excepiton. can any one help me?
You are missing exponential backoff.
See here for an example (its for drive but can be adapted)
https://developers.google.com/drive/handle-errors#implementing_exponential_backoff in there you will see how it deals with errors like 'userRateLimitExceeded'