im using mongoDB alongside spring boot
i'm wonder is there any way to retrieve data from DBref collection in an objective way?
for example in my person class there is #DBref private UserAccountType userAccountType and in userAccount we have an enum calls Role. If in a method i get a role and want to find all persons with Role.Driver, i should first find all userAccountType that their role is driver and then find all persons who has that userAccountType id.
something like this
Query query = new Query();
query.addCriteria(Criteria.where("staticRole").in(StaticRole.DRIVER_BRONZE, StaticRole.DRIVER_SILVER));
query.fields().include("id");
List<UserAccountType> userAccountTypeList = mongoOperations.find(query, UserAccountType.class);
query = new Query();
Criteria criteria = new Criteria();
criteria.orOperator(
Criteria.where("phone").is(phone),
Criteria.where("phone2").is(phone)
);
query.addCriteria(criteria);
query.addCriteria(Criteria.where("userAccount.userAccountType").in(userAccountTypeList));
Person person = mongoOperations.findOne(query, Person.class);
and all this lookups make my app very slow. We know our database design is so bad. but we have no time. is there any way to do this operations simply?
Related
I am implementing a search feature for the users in our api. The search will query our db to fetch the first N users matching the search term. I want to order the user after "best match" so the most relevant user is on top.
What I'd like to do is something like:
var users = await _dbContext.Users
.IncludeUserData()
.Where(u => u.Name.Contains(searchTerm))
.OrderBy(u => u.Name.IndexOf(searchTerm)) <- This row is not possible
.ToListAsync();
Where basically a name that contains the search term early is ordered before a user whose name contains the term late.
E.g.
Simon Carlsson should come before Carl Simonsson if the searchTerm is "Simon"
Using SQL Server as the DB provider
How would I achieve an order by query where users with names better matching the searchTerm are sorted higher up in the list?
After some more searching this method of importing functions from the DB provider was found:
[DbFunction("CHARINDEX", IsBuiltIn = true)]
public static long CHARINDEX(string substring, string str)
{
throw new NotImplementedException();
}
Put this in user dbContext class. It will bind to the CHARINDEX function in SQL Server.
https://learn.microsoft.com/en-us/ef/core/querying/user-defined-function-mapping
Then use it to sort the query:
.sortBy(u => DbContext.CHARINDEX(searchTerm, u.name))
Have you tried the LIKE operator?
You may find this useful
Entity framework EF.Functions.Like vs string.Contains
When you use NHibernate to "fetch" a mapped object, it outputs a SELECT query to the database. It outputs this using parameters; so if I query a list of cars based on tenant ID and name, I get:
select Name, Location from Car where tenantID=#p0 and Name=#p1
This has the nice benefit of our database creating (and caching) a query plan based on this query and the result, so when it is run again, the query is much faster as it can load the plan from the cache.
The problem with this is that we are a multi-tenant database, and almost all of our indexes are partition aligned. Our tenants have vastly different data sets; one tenant could have 5 cars, while another could have 50,000. And so because NHibernate does this, it has the net effect of our database creating and caching a plan for the FIRST tenant that runs it. This plan is likely not efficient for subsequent tenants who run the query.
What I WANT to do is force NHibernate NOT to parameterize certain parameters; namely, the tenant ID. So I'd want the query to read:
select Name, Location from Car where tenantID=55 and Name=#p0
I can't figure out how to do this in the HBM.XML mapping. How can I dictate to NHibernate how to use parameters? Or can I just turn parameters off altogether?
OK everyone, I figured it out.
The way I did it was overriding the SqlClientDriver with my own custom driver that looks like this:
public class CustomSqlClientDriver : SqlClientDriver
{
private static Regex _partitionKeyReplacer = new Regex(#".PartitionKey=(#p0)", RegexOptions.Compiled);
public override void AdjustCommand(IDbCommand command)
{
var m = _tenantIDReplacer.Match(command.CommandText);
if (!m.Success)
return;
// replace the first parameter with the actual partition key
var parameterName = m.Groups[1].Value;
// find the parameter value
var tenantID = (IDbDataParameter ) command.Parameters[parameterName];
var valueOfTenantID = tenantID.Value;
// now replace the string
command.CommandText = _tenantIDReplacer.Replace(command.CommandText, ".TenantID=" + valueOfTenantID);
}
} }
I override the AdjustCommand method and use a Regex to replace the tenantID. This works; not sure if there's a better way, but I really didn't want to have to open up NHibernate and start messing with core code.
You'll have to register this custom driver in the connection.driver_class property of the SessionFactory upon initialization.
Hope this helps somebody!
I recently came across a number of articles pointing out to flatten the data for NoSQL databases. Coming from traditional SQL databases I realized I am replicating a SQL db bahaviour in GAE. So I started to refactor code where possible.
We have e.g. a social media site where users can become friends with each other.
class Friendship(ndb.Model):
from_friend = ndb.KeyProperty(kind=User)
to_friend = ndb.KeyProperty(kind=User)
Effectively the app creates a friendship instance between both users.
friendshipA = Friendship(from_friend = UserA, to_friend = userB)
friendshipB = Friendship(from_friend = UserB, to_friend = userA)
How could I now move this to the actual user model to flatten it. I thought maybe I could use a StructuredProperty. I know it is limited to 5000 entries, but that should be enough for friends.
class User(UserMixin, ndb.Model):
name = ndb.StringProperty()
friends = ndb.StructuredProperty(User, repeated=True)
So I came up with this, however User can't point to itself, so it seems. Because I get a NameError: name 'User' is not defined
Any idea how I could flatten it so that a single User instance would contain all its friends, with all their properties?
You can't create a StructuredProperty that references itself. Also, use of StructuredProperty to store a copy of User has additional problem of needing to perform a manual cascade update if a user ever modifies a property that is stored.
However, as KeyProperty accept String as kind, you can easily store the list of Users using KeyProperty as suggested by #dragonx. You can further optimise read by using ndb.get_multi to avoid multiple round-trip RPC calls when retrieving friends.
Here is a sample code:
class User(ndb.Model):
name = ndb.StringProperty()
friends = ndb.KeyProperty(kind="User", repeated=True)
userB = User(name="User B")
userB_key = userB.put()
userC = User(name="User C")
userC_key = userC.put()
userA = User(name="User A", friends=[userB_key, userC_key])
userA_key = userA.put()
# To retrieve all friends
for user in ndb.get_multi(userA.friends):
print "user: %s" % user.name
Use a KeyProperty that stores the key for the User instance.
I have an appengine app which has been running for about a year now, i have mainly been using JDO queries until now, but i am trying to collect stats and the queries are taking too long. I have the following entity (Device)
public class Device implements Serializable{
...
#Persistent
private Set<Key> feeds;// Key for the Feed entity
...
}
So I want to get a count of how many Devices have a certain Feed. I was doing it in JDOQL before as such (uses javax.jdo.Query):
Query query = pm.newQuery("select from Device where feeds.contains(:feedKey)");
Map<String, Object> paramsf = new HashMap<String, Object>();
paramsf.put("feedKey",feed.getId());
List<Device> results = (List<Device>) query.executeWithMap(paramsf);
Though this code times out now. I was trying to use the Datastore API so I could set chunk size,etc to see if i could speed the query up or use a cursor, but I am unsure how to search in a Set field. I was trying this (uses com.google.appengine.api.datastore.Query)
Query query = new Query("Device");
query.addFilter("feeds", FilterOperator.IN, feed.getId());
query.setKeysOnly();
final FetchOptions fetchOptions = FetchOptions.Builder.withPrefetchSize(100).chunkSize(100);
QueryResultList<Entity> results = dss.prepare(query).asQueryResultList(fetchOptions);
Essentially i am unsure how to search in the one-to-many filed (feeds) for a single key. Is it possible to index it somehow?
hope it makes sense....
Lists (and other things that are implemented as lists, like sets) are indexed individually. As a result, you can simply use an equality filter in your query, the same as if you were filtering on a single value rather than a list. A record will be returned if any of the items in the list match.
Let's say I've got a SQL 2008 database table with lots of records associated with two different customers, Customer A and Customer B.
I would like to build a fat client application that fetches all of the records that are specific to either Customer A or Customer B based on the credentials of the requesting user, then stores the fetched records in a temporary local table.
Thinking I might use the MS Sync Framework to accomplish this, I started reading about row filtering when I came across this little chestnut:
Do not rely on filtering for security.
The ability to filter data from the
server based on a client or user ID is
not a security feature. In other
words, this approach cannot be used to
prevent one client from reading data
that belongs to another client. This
type of filtering is useful only for
partitioning data and reducing the
amount of data that is brought down to
the client database.
So, is this telling me that the MS Sync Framework is only a good option when you want to replicate an entire table between point A and point B?
Doesn't that seem to be an extremely limiting characteristic of the framework? Or am I just interpreting this statement incorrectly? Or is there some other way to use the framework to achieve my purposes?
Ideas anyone?
Thanks!
No, it is only a security warning.
We use filtering extensively in our semi-connected app.
Here is some code to get you started:
//helper
void PrepareFilter(string tablename, string filter)
{
SyncAdapters.Remove(tablename);
var ab = new SqlSyncAdapterBuilder(this.Connection as SqlConnection);
ab.TableName = "dbo." + tablename;
ab.ChangeTrackingType = ChangeTrackingType.SqlServerChangeTracking;
ab.FilterClause = filter;
var cpar = new SqlParameter("#filterid", SqlDbType.UniqueIdentifier);
cpar.IsNullable = true;
cpar.Value = DBNull.Value;
ab.FilterParameters.Add(cpar);
var nsa = ab.ToSyncAdapter();
nsa.TableName = tablename;
SyncAdapters.Add(nsa);
}
// usage
void SetupFooBar()
{
var tablename = "FooBar";
var filter = "FooId IN (SELECT BarId FROM dbo.GetAllFooBars(#filterid))";
PrepareFilter(tablename, filter);
}