I try to store data in google app engine datastore with JPA and I have some troubles.
My code :
try {
for (int i = 1; i <= 10; i++) {
Employee emp = new Employee();
emp.setFirstName("John" + i);
emp.setLastName("Doe" + i);
emp.setAge(i);
em.persist(emp);
em.refresh(emp);
}
em.flush();
} catch (Exception e) {
e.printStackTrace();
} finally {
em.close();
}
When I launch it, data are stored but 2 errors occurs :
javax.persistence.TransactionRequiredException: This operation requires a transaction yet it is not active -> line em.flush();
and
java.lang.NullPointerException
at org.datanucleus.ObjectManagerImpl.flushInternalWithOrdering(ObjectManagerImpl.java:3887) -> line em.close();
Anyone know how to fix them?
Thanks.
Try:
em.getTransaction().begin();
//do all your persist logic
em.getTransaction().commit();
For more: https://developers.google.com/appengine/docs/java/datastore/transactions
Related
This is my first app on Android with Java and SQLite.
ISSUE:
I have a local SQLIte db on my app. I was very surprised to see how easy it is to get access to the db once you have installed the app (no need to be a programmer nor a hacker).
I tried adding SQLCipher to my app but it only worked for newer Android versions 11 & 12 and didn't work for Android 9 for example and it did make my app's size much bigger.
After researching more I found a better solution for my case which doesn"t involve crypting the db with SQLCipher but rather it consists of corrupting the first bytes of the db file then after each launch of the app the code will decorrupt the file and use the fixed file instead. This insures that anyone who decompiles the apk will only get access to a corrupt db file and will have to put more effort to fix it which is my goal.
I came across this solution in a reply [here][1] but I don't know how to implement it as I am new to Android and SQLite programming. Any help is much appreciated on how to actually do it.
These are the steps as mentioned by the user: farhad.kargaran which need more explanation as I don't get how to do it:
1- corrupt the db file (convert it to byte array and change some values)
2- copy it in asset folder
3- in first run fix corrupted file from asset and copy it in database
folder.
Change first 200 byte values like this:
int index = 0;
for(int i=0;i<100;i++)
{
byte tmp = b[index];
b[index] = b[index + 1];
b[index + 1] = tmp;
index += 2;
}
As only the first 200 bytes were replaced, the same code is used for fixing first 200 byte values.
Here is my code for the SQLiteOpenHelper if needed:
public class DatabaseHelper extends SQLiteOpenHelper {
private static final String TAG = DatabaseHelper.class.getSimpleName();
public static String DB_PATH;
public static String DB_NAME;
public SQLiteDatabase database;
public final Context context;
public SQLiteDatabase getDb() {
return database;
}
public DatabaseHelper(Context context, String databaseName, int db_version) {
super(context, databaseName, null, db_version);
this.context = context;
DB_PATH = getReadableDatabase().getPath();
DB_NAME = databaseName;
openDataBase();
// prepare if need to upgrade
int cur_version = database.getVersion();
if (cur_version == 0) database.setVersion(1);
Log.d(TAG, "DB version : " + db_version);
if (cur_version < db_version) {
try {
copyDataBase();
Log.d(TAG, "Upgrade DB from v." + cur_version + " to v." + db_version);
database.setVersion(db_version);
} catch (IOException e) {
Log.d(TAG, "Upgrade error");
throw new Error("Error upgrade database!");
}
}
}
public void createDataBase() {
boolean dbExist = checkDataBase();
if (!dbExist) {
this.getReadableDatabase();
this.close();
try {
copyDataBase();
} catch (IOException e) {
Log.e(TAG, "Copying error");
throw new Error("Error copying database!");
}
} else {
Log.i(this.getClass().toString(), "Database already exists");
}
}
private boolean checkDataBase() {
SQLiteDatabase checkDb = null;
try {
String path = DB_PATH + DB_NAME;
checkDb = SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READONLY);
} catch (SQLException e) {
Log.e(TAG, "Error while checking db");
}
if (checkDb != null) {
checkDb.close();
}
return checkDb != null;
}
private void copyDataBase() throws IOException {
InputStream externalDbStream = context.getAssets().open(DB_NAME);
String outFileName = DB_PATH + DB_NAME;
OutputStream localDbStream = new FileOutputStream(outFileName);
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = externalDbStream.read(buffer)) > 0) {
localDbStream.write(buffer, 0, bytesRead);
}
localDbStream.close();
externalDbStream.close();
}
public SQLiteDatabase openDataBase() throws SQLException {
String path = DB_PATH + DB_NAME;
if (database == null) {
createDataBase();
database = SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READWRITE);
}
return database;
}
#Override
public synchronized void close() {
if (database != null) {
database.close();
}
super.close();
}
Much appreciated.
[1]: https://stackoverflow.com/a/63637685/18684673
As part of the copyDatabase, correct and then write the corrupted data, then copy the rest.
Could be done various ways
e.g.
long buffersRead = 0; //<<<<< ADDED for detecting first buffer
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = externalDbStream.read(buffer)) > 0) {
if (bufferesRead++ < 1) {
//correct the first 200 bytes here before writing ....
}
localDbStream.write(buffer, 0, bytesRead);
}
I have no idea in what extend GAE is not easy to understand :(
My servlet manipulate a json string and then I'm trying to store it in datastore.
When I run the application I'm getting this output:
Jan 27, 2014 6:59:04 PM com.google.appengine.api.datastore.dev.LocalDatastoreService load
INFO: The backing store, D:\Android\IntelliJ IDEA\workspace\EyeBall\AppEngine\out\artifacts\AppEngine_war_exploded\WEB-INF\appengine-generated\local_db.bin, does not exist. It will be created.
1
2
3
4
5
7
***
***
***
***
***
8
9
Although it's mentioned that local_db.bin will be created but when I navigate to that directory the file is not there. Also, when I open http://localhost:8080/_ah/admin/datastore in browser nothing displays in Entity Kind drop down list.
So wtf happene to local_db.bin? Why it doesn't generates?
any suggestion would be appreciated. thanks.
==================
UPDATE:
I added my code based on request.
private static final String NO_DEVICE_ID = "FFFF0000";
private static final String SAMPLE_JSON = "{\"history\":[{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com\",\"url\":\"https://www.maybank2u.com.my/mbb/Mobile/info.do\",\"visits\":14},{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com\",\"url\":\"https://www.maybank2u.com.my/mbb/Mobile/adaptInfo.do\",\"visits\":4},{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com\",\"url\":\"http://www.maybank2u.com.my/mbb_info/m2u/public/personalBanking.do\",\"visits\":16},{\"date\":null,\"info\":null,\"title\":\"Maybank2u.com Online Financial Services\",\"url\":\"https://www.maybank2u.com.my/mbb/m2u/common/M2ULogin.do?action=Login\",\"visits\":52},{\"date\":null,\"info\":null,\"title\":\"BBC\",\"url\":\"http://www.bbc.co.uk/persian/\",\"visits\":16}]}";
private static final String QUERY_HISTORY_DEVICE = "SELECT m FROM HistoryDeviceJPA m WHERE m.userUUID = :keyword ORDER BY m.domain ASC";
private static final String QUERY_HISTORY = "SELECT m FROM HistoryJPA m WHERE m.pageAddress = :keyword";
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// displayError(response, "The page doesn't support httpGet");
String deviceId = NO_DEVICE_ID;
String content = SAMPLE_JSON;
System.out.println("1");
HistoryBrowser historyBrowser = parseJson(content);
if(historyBrowser == null)
return;
System.out.println("2");
List<HistoryBrowser.BrowserInfo> historyList = historyBrowser.getHistory();
if(historyList == null)
return;
System.out.println("3");
List<HistoryDeviceJPA> historyDeviceJPAList = new ArrayList<HistoryDeviceJPA>(historyList.size());
for(int i=0; i<historyList.size(); i++) {
try {
HistoryBrowser.BrowserInfo browser = historyList.get(i);
HistoryDeviceJPA historyDeviceJPA = new HistoryDeviceJPA();
historyDeviceJPA.setUserUUID(deviceId);
historyDeviceJPA.setDomain(getDomainName(browser.getUrl()));
historyDeviceJPA.setPageAddress(browser.getUrl());
historyDeviceJPA.setPageTitle(browser.getTitle());
historyDeviceJPA.setPageVisits(browser.getVisits());
historyDeviceJPAList.add(historyDeviceJPA);
} catch (URISyntaxException e) {
System.out.println(e.getMessage());
}
}
System.out.println("4");
// get history of device from data store
EntityManager em = EMF.get().createEntityManager();
Query q = em.createQuery(QUERY_HISTORY_DEVICE).setParameter("keyword", deviceId);
#SuppressWarnings("unchecked")
List<HistoryDeviceJPA> dbList = (List<HistoryDeviceJPA>) q.getResultList();
System.out.println("5");
// If there is no result (shows there is no record for that device)
if(dbList == null)
addHistoryDeviceJPAToDs(historyDeviceJPAList);
else {
System.out.println("7");
// find each item in datastore and replace them if needed
// if current page visit is less ot equal than previous visit don't do anything (remove item form historyDeviceJPAList)
outerLoop:
for(int i=0; i<historyDeviceJPAList.size(); i++) {
HistoryDeviceJPA deviceItem = historyDeviceJPAList.get(i);
System.out.println("***");
for(int j=0; j<dbList.size(); j++) {
HistoryDeviceJPA dbItem = dbList.get(j);
if(deviceItem.getPageAddress().equalsIgnoreCase(dbItem.getPageAddress())) {
if(deviceItem.getPageVisits() > dbItem.getPageVisits()) {
long diff = deviceItem.getPageVisits() - dbItem.getPageVisits();
dbItem.setPageVisits(deviceItem.getPageVisits());
HistoryJPA historyJPA = findHistoryJPA(dbItem.getPageAddress());
historyJPA.setPageVisits(historyJPA.getPageVisits() + diff);
// update datastore
addHistoryDeviceJPAToDs(dbItem);
addHistoryJPAToDs(historyJPA);
// don't check other items of j list
break outerLoop;
}
}
}
}
System.out.println("8");
}
System.out.println("9");
// http://www.sohailaziz.com/2012/06/scheduling-activities-services-and.html
// https://dev.twitter.com/docs/api/1.1
// https://developers.google.com/appengine/docs/java/datastore/jdo/creatinggettinganddeletingdata?csw=1#Updating_an_Object
// http://en.wikibooks.org/wiki/Java_Persistence/Inheritance
}
and 6 is here:
private void addHistoryDeviceJPAToDs(List<HistoryDeviceJPA> list) {
System.out.println("6");
EntityManager em = EMF.get().createEntityManager();
try {
for (int i=0; i<list.size(); i++) {
System.out.println("=> " + i + " - " + list.get(i).toString());
em.getTransaction().begin();
em.persist(list.get(i));
em.getTransaction().commit();
}
} finally {
em.close();
}
}
after debug I found the problem is in this line:
List<HistoryDeviceJPA> dbList = (List<HistoryDeviceJPA>) q.getResultList();
if(dbList == null)
addHistoryDeviceJPAToDs(historyDeviceJPAList);
'dbList' is never null and it's size is 0 if there is nothing in datastore. That's why addHistoryDeviceJPAToDs method never invoked. By changing the code to following problem solved and local db created.
List<HistoryDeviceJPA> dbList = (List<HistoryDeviceJPA>) q.getResultList();
if(dbList == null)
return;
System.out.println("5");
// If there is no result (shows there is no record for that device)
if(dbList.size() == 0)
addHistoryDeviceJPAToDs(historyDeviceJPAList);
For other people who come across the same issue --
GAE will not create local_db.bin until you put data in the datastore. So if the file is not there, there is likely a bug in the application code.
I am working with Admin SDK .when I update single user it's working fine but when I am trying to update bunch(1000) of users I got User Rate Limit Exceeded Exception. Please check below my code and tell me what am i missing ? or tell any suggestion ?
private Directory getDirectoryService(String adminEmailAddress){
Directory directoryService = null;
try {
Collection<String> SCOPES = new ArrayList<String>();
SCOPES.add("https://www.googleapis.com/auth/admin.directory.user");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.user.readonly");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.group");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.group.readonly");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.orgunit");
SCOPES.add("https://www.googleapis.com/auth/admin.directory.orgunit.readonly");
SCOPES.add("https://www.googleapis.com/auth/userinfo.profile");
HttpTransport httpTransport = new NetHttpTransport();
JacksonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
.setServiceAccountScopes(SCOPES)
.setServiceAccountUser(adminEmailAddress)
.setServiceAccountPrivateKeyFromP12File(new java.io.File(SERVICE_ACCOUNT_PKCS12_FILE_PATH)).build();
directoryService = new Directory.Builder(httpTransport,jsonFactory, credential).setApplicationName("gdirectoryspring").build();
} catch(Exception e){
ErrorHandler.errorHandler(this.getClass().getName(), e);
}
return directoryService;
}
Below is Update User Code
Directory directoryService = getDirectoryService("adminEmailAddress");
User user = directoryService.users().get("userPrimaryEmail").execute();
List<UserOrganization> organizaionList = user.getOrganizations();
for (int j = 0; j < organizaionList.size(); j++)
{
UserOrganization singleOrg = organizaionList.get(j);
if (singleOrg != null)
{
if ("work".equalsIgnoreCase(singleOrg.getCustomType()) ||singleOrg.getPrimary() != null)
{
if (singleOrg.getTitle() != null)
{
singleOrg.setTitle(jobTitle);
}
}
}
user.setOrganizations(organizaionList);
}
Update update= directoryService.users().update(primaryEmail, user);
User userUpdated = update.execute();
and in admin console I increased my limit like below
Admin SDK 10.0 requests/second/user
but till I am getting User Rate Limit Exceeded Excepiton. can any one help me?
You are missing exponential backoff.
See here for an example (its for drive but can be adapted)
https://developers.google.com/drive/handle-errors#implementing_exponential_backoff in there you will see how it deals with errors like 'userRateLimitExceeded'
I tried the example from "J2ME/Blackberry - how to read/write text file?". I want only the read functionality, the file I want to read is in CSV format as a .txt file placed in the /res/test.txt.
But I am having an issue with the FileConnection. I get the following error:
File system error (1003)
Any suggestions or advice on a better approach or as to how I can get this working?
public class FileDemo extends MainScreen {
public FileDemo() {
setTitle("My Page");
String str = readTextFile("file:///test.txt");
System.out.println("Contents of the file::::::: " + str);
}
public String readTextFile(String fName) {
String result = null;
FileConnection fconn = null;
DataInputStream is = null;
try {
fconn = (FileConnection) Connector.openInputStream(fName);
is = fconn.openDataInputStream();
byte[] data = IOUtilities.streamToBytes(is);
result = new String(data);
} catch (IOException e) {
System.out.println(e.getMessage());
} finally {
try {
if (null != is)
is.close();
if (null != fconn)
fconn.close();
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
return result;
}
}
try this
InputStream is = getClass().getResourceAsStream("/test.txt");
StringBuffer buff = new StringBuffer();
int ch;
try {
while ((ch = is.read()) != -1)
buff.append((char) ch);
} catch (Exception e) {
Log.Error(e, "Exception ");
}
String str = (buff.toString());
Same problem I also faced in my project. First check your simulator memory card is inserted or not. From simulator,
go to Options(Settings)-->Device-->Storage and Check the Memory card Storage.
If the memory card is not inserted, than it will show Media Card is not currently inserted in the device. So, you need to insert the memory card. From simulator menu bar, choose simulate-->Change SD Card...
You can add the SD card here. Than you try.
I think, This suggestion will help someone.
the version of my app engine is 1.4.0.the data dem.bil is under the /war/dem.bil directory.and these are my codes to fetch the data dem.bil that is 3M:
try{
URLConnection a = url.openConnection();
InputStream b = a.getInputStream();
int len = a.getContentLength();
if (len < 0) {
return null;
}
//System.out.println("Total: "+len);
byte[] c = new byte[len];
b.read(c,0,len);
return c;
}catch (Exception e) {
e.printStackTrace();
return null;
}
}
i know the version 1.4.0 increased the URLFetch response limit to 32MB,but when it goes to InputStream b = a.getInputStream(); it's debug is "com.google.appengine.api.urlfetch.ResponseTooLargeException: The response from url localhost:8888/dem.bil was too large. ".so can someone can tell me why?or some wrong with my codes?
Since the 1.4.0 version was released, the limit was raised to 32mb.
See: http://googleappengine.blogspot.com/2010/12/happy-holidays-from-app-engine-team-140.html