I need to be able to retrieve all records available in the index. Seems like 1000 is the limit. Is there something else I can do?
I was also facing some similar issue in one of my projects so researched over internet and got one idea that instead of using the Search API, I created a workaround scenario. What I did is I have only one attribute in my table which needs to have a pattern based search. I am sharing my code as well here.
Objectify Entity Class
#Entity
public class NewsFeed {
#Id
#Index
private Long feedID;
private String title;
private Set<String> titleKeywords;
// getters and setter
}
Logic for storing the keyword in the same table. I have split all the title words for my entity into keywords, and stored them in a Set Object.
NewsFeed newsFeed = new NewsFeed();
newsFeed.setTitle(title);
newsFeed.setTitleKeywords(getKeywordsSet(newsTitle));
// save entity here
Method for extracting keywords from title(field to be searched)
public Set<String> getKeywordsSet(String title) {
Set<String> keywords = new HashSet<String>();
String titleNews = title.toLowerCase();
String[] array = titleNews.split(" ");
for (int i = 0; i < array.length; i++) {
// replacing all special characters here
String word = array[i].replaceAll("\\W", "");
keywords.add(word);
}
return keywords;
}
Listing all the feeds from our DB, and finally matching the parameter to be searched, by the logic below.
public List<NewsFeed> getFilterJsonArray(String param){
// Listing all the objects of entity
List<NewsFeed> list = newsFeedDao.listOrderedFeeds();
List<NewsFeed> matchedObject = new ArrayList<NewsFeed>();
for (NewsFeed newsFeed : list) {
/**
* main logic for pattern matched keywords
**/
if (isAnElementInSet(newsFeed.getTitleKeywords(), param.toLowerCase())) {
matchedObject.add(newsFeed);
}
}
return matchedObject;
}
public boolean isAnElementInSet(Set<String> keywords, String param) {
String []params = param.split(" ");
if (keywords.size() > 0) {
for(String splittedParam : params){
if (keywords.contains(splittedParam)) {
return true;
} else{
for (String keyword : keywords) {
if(keyword.contains(splittedParam)){
return true;
}
}
return false;
}
}
return true;
}else{
return false;
}
}
I know this cannot be the best solution for searching things but this solution worked very fine for me. I just shared it here so as to get improvements in this logic as well.
Related
How would you implement a matching system to check when two values are the same in one collection from different documents? This code has to call a function to run every-time it detects that the two values are matching.
As mentioned in the documentation :
Cloud Firestore provides powerful query functionality for specifying which documents you want to retrieve from a collection or collection group. These queries > can also be used with either get() or addSnapshotListener(), as described in Get Data and Get Realtime Updates.
I will recommend you to use a unique identifier for each entry and then you can add a listener that will match each data from the collections.
Syntax :
query.addListenerForSingleValueEvent(new ValueEventListener() {
#Override public void onDataChange(DataSnapshot dataSnapshot) {
if (dataSnapshot.exists()) {
// dataSnapshot is the "issue" node with all children with id 0
for (DataSnapshot issue : dataSnapshot.getChildren()) {
// do something with the individual "issues"
} } }
You can refer to the Stackoverflow answer where Frank has explained it briefly with the following function code.
public interface AlreadyBookedCallback {
void onCallback(boolean isAlreadyBooked);
}
private void alreadyBooked(final String boname, final String bodept, final String botime, AlreadyBookedCallback callback) {
CollectionReference cref=db.collection("bookingdetails");
Query q1=cref.whereEqualTo("time",botime).whereEqualTo("dept",bodept);
q1.get().addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
#Override
public void onSuccess(QuerySnapshot queryDocumentSnapshots) {
for (DocumentSnapshot ds : queryDocumentSnapshots) {
String rname, rdept, rtime;
rname = ds.getString("name");
rdept = ds.getString("dept");
rtime = ds.getString("time");
if (rdept.equals(botime)) {
if (rtime.equals(botime)) {
isExisting = true;
}
}
}
callback.onCallback(isExisting)
}
});
}
For more information you can also refer to this thread on how to check duplicate values in documents.
This question already has answers here:
Serialize and Deserialize Json and Json Array in Unity
(9 answers)
Closed 5 years ago.
While I could serialize/deserialize simple data with JsonUtility, when trying to load an array of objects into my game, Unity didn't respond well.
I do understand that the utility doesn't take arrays, so alternative solutions are needed.
I've tried using the wrapper posted here (Thanks!!), but it's still not doing what I'd expect it to do.
While I'm passing all the details into the JSON, when saving it, it's only saving instanceIDs, which are not very useful for persistency.
Here's the step by step.
1. Creating the object
private Customer[] _customerDeck;
Customer yidler = new GameObject().AddComponent<Customer>();
yidler.race = GameController.Race.Alien;
yidler.initial = true;
yidler.patience = 3;
yidler.dice = new ResourceGroup();
yidler.dice.diceRequired = 1;
yidler.dice.resources = new Dictionary<GameController.Resource, int>();
yidler.dice.resources.Add(GameController.Resource.Food, 3);
yidler.dice.resources.Add(GameController.Resource.Beverage, 0);
yidler.dice.resources.Add(GameController.Resource.Dessert, 1);
yidler.moneyAwarded = 6;
yidler.fidelityAwarded = 1;
yidler.starsAwarded = 0;
yidler.initial = true;
_customerDeck[0] = yidler;
JSONManager.SaveCustomers(_customerDeck);
2. Saving the JSON
public static void SaveCustomers(Customer[] customers)
{
string filePath = Path.Combine(Application.streamingAssetsPath, customerCardsFile);
if(File.Exists(filePath))
{
string data = JsonHelper.ToJson<Customer>(customers);
File.WriteAllText(filePath, data);
}
else
{
Supporting.Log("Cannot find Customers JSON", 1);
}
}
public static class JsonHelper
{
public static T[] FromJson<T>(string json)
{
Wrapper<T> wrapper = JsonUtility.FromJson<Wrapper<T>>(json);
return wrapper.Items;
}
public static string ToJson<T>(T[] array)
{
Wrapper<T> wrapper = new Wrapper<T>();
wrapper.Items = array;
return JsonUtility.ToJson(wrapper);
}
[System.Serializable]
private class Wrapper<T>
{
public T[] Items;
}
}
3. Additional Info
The Customer class is System.Serializable, same as the ResourceGroup referenced within it.
This is the resulting json
{"Items":[{"instanceID":-47734},{"instanceID":0},{"instanceID":0},{"instanceID":0},{"instanceID":0}]}
First of all try remove MonoBehaviour inheritance from Customer class. Also Unity's default serializer doesn't support Dictionary<> serialization. Try to change from Dictionary to a List of Serializable class. Also check that all inner Customer data fields are serializable too.
I am attempting to check for an existing string using a JDO query, in my attempt to prevent the insertion of a duplicate string.
My query to check for an existing string works fine, unless the two strings I am comparing have a comma in the value. If the commas exists, the comparison bombs using "==".
For example, if I query to see if "Architecture" exists, I get the right result (Horrray!).
If I attempt to see if "Architecture, Engineering, and Drafting" exists, and it does, the query comes back and says an identical value does not exist (Boo!).
The code I'm using is as follows:
Called from the RPC
public void addCommas()
{
final Industry e = new Industry();
e.setIndustryName("Architecture, Engineering, and Drafting");
persist(e);
}
public void addNoCommas()
{
final Industry e = new Industry();
e.setIndustryName("Architecture");
persist(e);
}
Persist Operation
private void persist(Industry industry)
{
if (industryNameExists(industry.getIndustryName()))
{
return;
}
final PersistenceManager pm = PMF.get().getPersistenceManager();
pm.currentTransaction().begin();
try
{
pm.makePersistent(industry);
pm.flush();
pm.currentTransaction().commit();
} catch (final Exception ex)
{
throw new RuntimeException(ex);
} finally
{
if (pm.currentTransaction().isActive())
{
pm.currentTransaction().rollback();
}
pm.close();
}
}
Query
public static boolean industryNameExists(final String industryName)
{
final PersistenceManager pm = PMF.get().getPersistenceManager();
Query q = null;
q = pm.newQuery(Industry.class);
q.setFilter("industryName == industryNameParam");
q.declareParameters(String.class.getName() + " industryNameParam");
final List<Industry> industry = (List<Industry>) q.execute(industryName.getBytes());
boolean exists = !industry.isEmpty();
if (q != null)
{
q.closeAll();
}
pm.close();
return exists;
}
JDO Entity
#PersistenceCapable(detachable = "true")
public class Industry implements StoreCallback
{
#NotNull(message = "Industry Name is required.")
private String industryName;
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
#PrimaryKey
private Key key;
public Industry()
{
super();
}
public Key getIndustryKey()
{
return key;
}
public String getIndustryName()
{
return industryName;
}
#Override
public void jdoPreStore()
{
if (industryName != null)
{
industryName = industryName.trim();
}
}
public void setIndustryName(final String industryName)
{
this.industryName = industryName;
}
}
Any thoughts on a resolution or pinpointing an oversight would be very much appreciated.
Cheerio.
So you're calling industryNameExists("Architecture, Engineering, and Drafting") and trying to match a JDO with industryName exactly equal "Architecture, Engineering, and Drafting"?
Assuming you don't have any typo or space difference the only thing suspect is the getBytes(). Try the following:
Query q = pm.newQuery(Industry.class, "this.industryName == :industryNameParam");
List<Industry> industry = (List<Industry>) q.execute(industryName);
You can also try variation filters like "this.industryName.equalsIgnoreCase(:industryNameParam)" and "this.industryName.startWith(:industryNameParam)" to troubleshoot.
If it still does not work, try logging the SQL generated for review and compare with a hand-written query that works.
I currently have a couple methods here:
public ADHelper()
{
connection = InitializeConnection();
}
private DirectoryEntry InitializeConnection()
{
DirectoryEntry ldapConnection = new DirectoryEntry("LDAP://servername.domain.com:389/DC=domain,DC=com");
ldapConnection.Username = "user"
ldapConnection.Password = "password";
ldapConnection.AuthenticationType = AuthenticationTypes.Secure;
return ldapConnection;
}
I'd like to create another method to check and see if an object exists within that domain. I'm currently doing that with the following:
public bool Exists(string objectPath)
{
bool found = DirectoryEntry.Exists("LDAP://" + objectPath);
return found;
}
But that forces me to specify an entire LDAP string. I'd like to simply extend the initial ldapConnection with an OU and maybe CN parameter within the Exists() method. Is there any way to make this happen without making the Initialize() method public?
Thanks so much!
Maybe something like this:
public bool AccountExists(string userEmail)
{
using (var root = GetLdapRoot())
{
using (var searcher = new DirectorySearcher(root))
{
searcher.Filter = string.Format("(&(objectClass=User)(mail={0}))", userEmail);
searcher.PropertiesToLoad.Add("email");
var result = searcher.FindAll();
return result.Count > 0;
}
}
}
private static DirectoryEntry GetLdapRoot()
{
return new DirectoryEntry("LDAP://DC=com"); //or whatever your root domain is. Set credentials if you need to
}
by setting a filter and being specific about what properties to load in, the search will be more efficient. By using the root as your LDAP:// string, you should be searching the entire directory.
I get the above error sometimes during the read. The exception originates from ASP.NET SqlDataReader whenever you try to read data before calling the Read() method. Since EF does all these internally, I am wondering what else can cause this error. could it be network (or) db connectivity?
thanks
Additional Bounty Info (GenericTypeTea):
I've got the same error after upgrading to EF Code First RC (4.1):
"Invalid attempt to read when no data
is present"
This is the code in question:
using (var context = GetContext())
{
var query = from item in context.Preferences
where item.UserName == userName
where item.PrefName == "TreeState"
select item;
// Error on this line
Preference entity = query.FirstOrDefault();
return entity == null ? null : entity.Value;
}
The table structure is as follows:
Preference
{
Username [varchar(50)]
PrefName [varchar(50)]
Value [varchar(max)] Nullable
}
The table is standalone and has no relationships. This is the DbModelBuilder code:
private void ConfigurePreference(DbModelBuilder builder)
{
builder.Entity<Preference>().HasKey(x => new { x.UserName, x.PrefName });
builder.Entity<Preference>().ToTable("RP_Preference");
}
Exactly the same code works perfectly in CTP5. I'm guessing this is an RC bug, but any ideas of how to fix it would be appreciated.
This error occurs when there is a large amount of data in the RC release. The difference between the RC and CTP5 is that you need to specify the [MaxLength] property that contains a large amount of data.
Are you re-using contexts? I would guess this is happening as a result of something you are doing within GetContext
If GetContext() provides a stale context, in which the DataReader is closed/corrupted, I could see the above happening.
I cannot reproduce your problem on EF4.1 RC1.
POCO:
public class Preference
{
public string UserName { get; set; }
public string PrefName { get; set; }
public string Value { get; set; }
}
Context:
public class PreferenceContext : DbContext
{
public DbSet<Preference> Preferences {get;set;}
public PreferenceContext()
: base("Data Source=localhost;Initial Catalog=_so_question_ef41_rc;Integrated Security=SSPI;") {
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
ConfigurePreference(modelBuilder);
base.OnModelCreating(modelBuilder);
}
private void ConfigurePreference(DbModelBuilder builder)
{
builder.Entity<Preference>().HasKey(x => new { x.UserName, x.PrefName });
builder.Entity<Preference>().ToTable("RP_Preference");
}
}
My little Console App:
class Program
{
static void Main(string[] args)
{
string userName = "Anon";
for (int i = 0; i < 10000; i++)
{
var p = GetPreference(userName);
}
}
private static string GetPreference(string userName)
{
using (var context = new PreferenceContext())
{
var query = from item in context.Preferences
where item.UserName == userName
where item.PrefName == "TreeState"
select item;
// Error on this line
Preference entity = query.FirstOrDefault();
return entity == null ? null : entity.Value;
}
}
}
I do 10,000 reads, and no error. You will need to post more complete code to continue.
Increase the CommandTimeout on the context.
I had the same issue with EF4 - In my case I was (trying to) return the list of entities within the using{} section. This is the same as you are doing in your question:
return entity == null ? null : entity.Value;
} // end using
I moved the return to after the } and it worked.
I think I had the problem because the code was in a function which had already queried the database in another using block, I suspect the table was locking but not reporting the error, ending the using block before the return released the database lock.
Steve