I'm Fetching Datastore Entities using PersistenceManager Query which has more than 10000 entities, I would like to fetch only the keys of the Entity, for that I'm using cursor of limit 1000, But the cursor is null after fetching
PersistenceManager pm = PMF.get().getPersistenceManager();
Query q = null;
List<String> idList = null;
int cursorLimit = 1000;
Map<String, Object> cursorMap = null;
Cursor cursorObj = null;
CommonUtil utility = new CommonUtil();
HashMap<String, Object> responseMap = null;
responseMap = new HashMap<String, Object>();
Query q = pm.newQuery("select id from " + Person.class.getName());
if( requestMap.containsKey("cursor") )
{
cursorMap = new HashMap<String,Object>();
cursorObj = Cursor.fromWebSafeString( String.valueOf( requestMap.get("cursor") ) );
cursorMap.put(JDOCursorHelper.CURSOR_EXTENSION, cursorObj);
q.setExtensions(cursorMap);
}
q.setRange(0, cursorLimit);
idList = (List<String>) q.execute();
cursorObj = JDOCursorHelper.getCursor(idList);
responseMap.put("contactIdList", idList);
if(idList.size() == cursorLimit && cursorObj != null)
responseMap.put("cursor", cursorObj.toWebSafeString());
else
responseMap.put("cursor", "");
q.closeAll();
pm.close();
But the cursorObj is always null, Has anyone come up with this issue or how to overcome it
The Cursor is working perfectly while fetching whole entity, But in keys only query alone its not working as expected
Try the keys-only query:
Query<Key> query = Query.newKeyQueryBuilder().setKind("Task").build();
I Resolved the above cursor issue with the below code on the same day, Thank you
final DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
FetchOptions fetchOptions = FetchOptions.Builder.withLimit(cursorLimit);
if( requestMap.containsKey("cursor"))
{
fetchOptions.startCursor(Cursor.fromWebSafeString(requestMap.get("cursor") ));
}
q = new com.google.appengine.api.datastore.Query("Contact").setKeysOnly();
PreparedQuery pq = datastore.prepare(q);
results = pq.asQueryResultList(fetchOptions);
cursorObj = results.getCursor();
This LowLevel API Resolves the problem
Related
I am trying to list all the PDF files from the external storage created by the app. How to list pdf files in the App after iterating through cursor?
private void getExternalPDFFiles() {
ContentResolver cr = getContentResolver();
Uri uri = MediaStore.Files.getContentUri("external");
// every column, although that is huge waste, you probably need
// BaseColumns.DATA (the path) only.
String[] projection = null;
// exclude media files, they would be here also.
String selection = MediaStore.Files.FileColumns.MEDIA_TYPE + "="
+ MediaStore.Files.FileColumns.MEDIA_TYPE;
String[] selectionArgs = null; // there is no ? in selection so null here
String sortOrder = null; // unordered
// Cursor allNonMediaFiles = cr.query(uri, projection, selection, selectionArgs, sortOrder);
// only pdf
String selectionMimeType = MediaStore.Files.FileColumns.MIME_TYPE + "=?";
String mimeType = MimeTypeMap.getSingleton().getMimeTypeFromExtension("pdf");
String[] selectionArgsPdf = new String[]{mimeType};
Cursor cursor = cr.query(uri, projection, selectionMimeType, selectionArgsPdf, sortOrder);
assert cursor != null;
if (cursor.moveToFirst()) {
while (!cursor.isAfterLast()) {
//your code to implement
Log.d("TAG", cursor.getColumnNames().toString());
cursor.moveToNext();
}
}
Log.d("TAG", cursor.getCount() + "");
Log.d("TAG", cursor.getColumnCount() + "");
cursor.close();
}
Get all file list with Uri and FileName in Android-10 and above:
class FileModel {
String displayName;
Uri fileUri;
FileModel(String displayName, Uri fileUri) {
this.displayName = displayName;
this.fileUri = fileUri;
}
}
...
private ArrayList<FileModel> getExternalPDFFileList() {
ContentResolver cr = getContentResolver();
Uri uri = MediaStore.Files.getContentUri("external");
String[] projection = {MediaStore.Files.FileColumns._ID, MediaStore.Files.FileColumns.DISPLAY_NAME};
String selection = MediaStore.Files.FileColumns.MEDIA_TYPE + "="
+ MediaStore.Files.FileColumns.MEDIA_TYPE;
String[] selectionArgs = null;
String selectionMimeType = MediaStore.Files.FileColumns.MIME_TYPE + "=?";
String mimeType = MimeTypeMap.getSingleton().getMimeTypeFromExtension("pdf");
String[] selectionArgsPdf = new String[]{mimeType};
Cursor cursor = cr.query(uri, projection, selectionMimeType, selectionArgsPdf, null);
assert cursor != null;
ArrayList<FileModel> uriList = new ArrayList<>();
for (cursor.moveToFirst(); !cursor.isAfterLast(); cursor.moveToNext()) {
int columnIndex = cursor.getColumnIndex(projection[0]);
long fileId = cursor.getLong(columnIndex);
Uri fileUri = Uri.parse(uri.toString() + "/" + fileId);
String displayName = cursor.getString(cursor.getColumnIndex(projection[1]));
uriList.add(new FileModel(displayName, fileUri));
}
cursor.close();
return uriList;
}
How can I set the safe area for pull to refresh? I have a form with with an infinite container. I set the IC with safe area ic.setSafeArea(true). When I rotate the screen the safe area is respected(see image 1). If i pull to refresh the safe area is not respected (see image 2). With the below code taken from the InfiniteContainerSample, how can I ensure the safe area is respected after pull to refresh?
image 1
public void showForm() {
Form hi = new Form("InfiniteContainer", new BorderLayout());
Style s = UIManager.getInstance().getComponentStyle("MultiLine1");
FontImage p = FontImage.createMaterial(FontImage.MATERIAL_PORTRAIT, s);
EncodedImage placeholder = EncodedImage.createFromImage(p.scaled(p.getWidth() * 3, p.getHeight() * 3), false);
InfiniteContainer ic = new InfiniteContainer() {
#Override
public Component[] fetchComponents(int index, int amount) {
java.util.List<Map<String, Object>> data = fetchPropertyData("Leeds");
MultiButton[] cmps = new MultiButton[data.size()];
for(int iter = 0 ; iter < cmps.length ; iter++) {
Map<String, Object> currentListing = data.get(iter);
if(currentListing == null) {
return null;
}
String thumb_url = (String)currentListing.get("thumb_url");
String guid = (String)currentListing.get("guid");
String summary = (String)currentListing.get("summary");
cmps[iter] = new MultiButton(summary);
cmps[iter].setIcon(URLImage.createToStorage(placeholder, guid, thumb_url));
}
return cmps;
}
};
ic.setUIID("Blue");
ic.setSafeArea(true);
ic.addComponent(new Label("This is a test"));
hi.add(BorderLayout.CENTER, ic);
hi.show();
}
int pageNumber = 1;
java.util.List<Map<String, Object>> fetchPropertyData(String text) {
try {
ConnectionRequest r = new ConnectionRequest();
r.setPost(false);
r.setUrl("http://api.nestoria.co.uk/api");
r.addArgument("pretty", "0");
r.addArgument("action", "search_listings");
r.addArgument("encoding", "json");
r.addArgument("listing_type", "buy");
r.addArgument("page", "" + pageNumber);
pageNumber++;
r.addArgument("country", "uk");
r.addArgument("place_name", text);
NetworkManager.getInstance().addToQueueAndWait(r);
Map<String,Object> result = new JSONParser().parseJSON(new InputStreamReader(new ByteArrayInputStream(r.getResponseData()), "UTF-8"));
Map<String, Object> response = (Map<String, Object>)result.get("response");
return (java.util.List<Map<String, Object>>)response.get("listings");
} catch(Exception err) {
Log.e(err);
return null;
}
}
Thanks for reporting this. This has been fixed on github. It will be included in the next release on Friday.
How do I get values read values line a DataReader to read values of fields returned by multi in the following code to set the values in the Class
OracleRefCursor m_Cursor=null;
Dapper.SqlMapper.GridReader multi =null;
using (m_Conn = new OracleConnection(m_ConnectionString))
{
try
{
m_Conn.Open();
String m_LastName = "S" + "%";
String m_Id = "";
String m_EmpId = "";
var p = new OracleDynamicParameters();
p.Add(":p_LastName", m_LastName);
p.Add(":p_Id", m_Id);
p.Add(":p_EmpId", m_EmpId);
p.Add( ":p_Cursor1", dbType: OracleDbType.RefCursor, direction: ParameterDirection.Output );
using (multi = m_Conn.QueryMultiple(m_ProcedureName, param: p, commandType: CommandType.StoredProcedure))
{
}
}
catch (Exception ex)
{
String m_Error = ex.ToString();
}
}//end of using statement
From Dapper docs:
var sql =
#"
select * from Customers where CustomerId = #id
select * from Orders where CustomerId = #id
select * from Returns where CustomerId = #id";
using (var multi = connection.QueryMultiple(sql, new {id=selectedId}))
{
var customer = multi.Read<Customer>().Single();
var orders = multi.Read<Order>().ToList();
var returns = multi.Read<Return>().ToList();
...
}
I need to make a generic logger to record certain insert/update statements so that my testers can verify the data being inserted is correct.
My first thought was that I would just use a function that accepted DynamicParameters and I would foreach through the DynamicParameters to generate a string to list the parameter's name and value and make them easier to read for the testers.
Unfortunately, Dapper.DynamicParameters does not contain a public definition for "GetEnumerator"
Here is basic example of what I was hoping to do:
string myFormattedListofParameters = "";
foreach (var p in myDynamicParameters)
{
myFormattedListofParameters += p.Name + "=" + p.Value.ToString();
}
Try:
var sb = new StringBuilder();
foreach (var name in p.ParameterNames)
{
var pValue = p.Get<dynamic>(name);
sb.AppendFormat("{0}={1}\n", name, pValue.ToString());
}
string ParametersToString(DynamicParameters parameters)
{
var result = new StringBuilder();
if (parameters != null)
{
var firstParam = true;
var parametersLookup = (SqlMapper.IParameterLookup)parameters;
foreach (var paramName in parameters.ParameterNames)
{
if (!firstParam)
{
result.Append(", ");
}
firstParam = false;
result.Append('#');
result.Append(paramName);
result.Append(" = ");
try
{
var value = parametersLookup[paramName];// parameters.Get<dynamic>(paramName);
result.Append((value != null) ? value.ToString() : "{null}");
}
catch
{
result.Append("unknown");
}
}
}
return result.ToString();
}
One line with Linq:
string.Join(", ", from pn in sprocParams.ParameterNames select string.Format("#{0}={1}", pn, (sprocParams as SqlMapper.IParameterLookup)[pn]))
I'm using that to log with log4net:
Log.InfoFormat("Exec {0} {1}", storedProc, string.Join(", ", from pn in sprocParams.ParameterNames select string.Format("#{0}={1}", pn, (sprocParams as SqlMapper.IParameterLookup)[pn])));
Just in case,
var pmaster = new Dapper.DynamicParameters();
SortedList l = new SortedList();
l.Add("param1", object1);
l.Add("param2", object2);
l.Add("param3", object3);
l.Add("param4", object4);
foreach(var key in l.Keys)
{
var val = l[key];
pmaster.Add(key.ToString(), val);
}
Worked for me.
I am working on an audit trail project where we have been told to following.
Track all the tables (200+) in our db with shadow tables just as Hibernate Envers does. It follows that we have create snapshot for a each transactions involving CUD.
In the past, I implemented audit solutions for finite sets of important data for each of my clients. For the current work, my questions are:
Does it make sense to audit every single table in the database?
How much of value would it be track the data like Envers does? Any application would want to have deltas for specific data points. Querying huge sets of data to figure out deltas seem to be unrealistic.
An Envers like solution requires tying up CUD actions with a transaction effectively ruling out triggers. This is because triggers run in their own transactions and hence the data in shadow tables can get out of sync in case of transaction rollbacks from the application. Anything I am missing here?
Does anybody suggest using NoSQL DBs for audit trail?
Fully implemented and can be improved more. I hope this may help someone :
public partial class Entity:DbContext
{
public enum AuditActions {I,U,D}
public override int SaveChanges( )
{
ChangeTracker.DetectChanges();
ObjectContext ctx = ((IObjectContextAdapter)this).ObjectContext;
// string UserName = WindowsIdentity.GetCurrent().Name;
IPrincipal principal = Thread.CurrentPrincipal;
IIdentity identity = principal == null ? null : principal.Identity;
string name = identity == null ? "" : identity.Name;
//Thread.CurrentPrincipal = new GenericPrincipal(new GenericIdentity((userName), roles);
List<ObjectStateEntry> objectStateEntryList =
ctx.ObjectStateManager.GetObjectStateEntries(EntityState.Added
| EntityState.Modified
| EntityState.Deleted)
.ToList();
List<DBAudit> AuditList = new List<DBAudit>();
string Audittable = string.Empty;
foreach (ObjectStateEntry entry in objectStateEntryList)
{
Audittable = entry.EntitySet.ToString();
if (!entry.IsRelationship && Audittable!="Audit table name")
{
//sIsAuditTble =entry.EntitySet="DBAudit"? true:false;
switch (entry.State)
{
case EntityState.Added:
AuditList= LogDetails(entry, name, AuditActions.I);
break;
case EntityState.Deleted:
AuditList= LogDetails(entry, name, AuditActions.D);
break;
case EntityState.Modified:
AuditList= LogDetails(entry, name, AuditActions.U);
break;
}
}
}
using (var context = new ProjectTrackerEntities())
{
for (int i = 0; i < AuditList.Count; i++)
{
context.DBAudits.Add(AuditList[i]);
context.SaveChanges();
}
}
return base.SaveChanges();
}
public List<DBAudit> LogDetails(ObjectStateEntry entry, string UserName, AuditActions action)
{
List<DBAudit> dbAuditList = new List<DBAudit>();
if (action == AuditActions.I)
{
var keyValues = new Dictionary<string, object>();
var currentValues = entry.CurrentValues;
// entry.Entity key = new EntityKey();
DBAudit audit = new DBAudit();
audit.AuditId = Guid.NewGuid().ToString();
audit.RevisionStamp = DateTime.Now;
audit.TableName = entry.EntitySet.Name;
audit.UserName = UserName;
audit.OldData = "";
audit.Actions = action.ToString();
for (int i = 0; i < currentValues.FieldCount; i++)
{
audit.ChangedColumns = audit.ChangedColumns + currentValues.GetName(i);
audit.NewData = audit.NewData + currentValues.GetValue(i);
audit.ChangedColumns = audit.ChangedColumns + ", ";
audit.NewData = audit.NewData + ", ";
}
dbAuditList.Add(audit);
//LogSave(audit);
}
else if (action == AuditActions.D)
{
var keyValues = new Dictionary<string, object>();
var DeletedValues = entry.OriginalValues;
// entry.Entity key = new EntityKey();
DBAudit audit = new DBAudit();
audit.AuditId = Guid.NewGuid().ToString();
audit.RevisionStamp = DateTime.Now;
audit.TableName = entry.EntitySet.Name;
audit.UserName = UserName;
audit.NewData = "";
audit.Actions = action.ToString();
for (int i = 0; i < DeletedValues.FieldCount; i++)
{
audit.ChangedColumns = audit.ChangedColumns + DeletedValues.GetName(i);
audit.OldData = audit.OldData + DeletedValues.GetValue(i);
audit.ChangedColumns = audit.ChangedColumns + ", ";
audit.OldData = audit.OldData + ", ";
}
dbAuditList.Add(audit);
}
else
{
foreach (string propertyName in entry.GetModifiedProperties())
{
DBAudit audit = new DBAudit();
DbDataRecord original = entry.OriginalValues;
string oldValue = original.GetValue(original.GetOrdinal(propertyName)).ToString();
CurrentValueRecord current = entry.CurrentValues;
string newValue = current.GetValue(current.GetOrdinal(propertyName)).ToString();
audit.AuditId = Guid.NewGuid().ToString();
audit.RevisionStamp = DateTime.Now;
audit.TableName = entry.EntitySet.Name;
audit.UserName = UserName;
audit.ChangedColumns = propertyName;
audit.OldData = oldValue;
audit.NewData = newValue;
audit.Actions = action.ToString();
dbAuditList.Add(audit);
//LogSave(audit);
}
}
return dbAuditList;
}
}
One option for a NoSQL database is RavenDB, using its "versioning bundle".
Though at this point it's probably too early, I recently listened to an interesting episode of Herding code where they talk with Eric Sink on about Veracity. As I understand, Veracity is part distributed version control system, and part NoSQL database. It is designed to be the backend to anything from a source control system to a wiki. It's been in development for a couple years, but is still effectively in pre-beta stages (as of Nov 2010).