I currently have a query that returns all the documents in a collection using the findAll() method of MongoTemplate. I want to sort these results, but do not see any way to do so. I see that I can use find() with a Query argument and call .with(Sort sort), but in this scenario, how do I set the Query to return all documents? I would be okay with using either approach.
Query query = new Query();
query.with(new Sort(Sort.Direction.DESC, "_id"));
List<MyClass> myClassList= mongoTemplate.find(query, MyClass.class);
An empty Query will behave as findAll(). Like you can write in the mongo shell: db.myCollection.find({}) you can write an emypty Query in the java mongdb driver.
An working sample code would be:
public static void main(String[] args) throws UnknownHostException
{
ServerAddress address = new ServerAddress("localhost", 27017);
MongoClient client = new MongoClient(address);
SimpleMongoDbFactory simpleMongoDbFactory = new SimpleMongoDbFactory(client, "mydatabase");
MongoTemplate mongoTemplate = new MongoTemplate(simpleMongoDbFactory);
Query query = new Query().with(new Sort("_id", "-1"));
List<MyClass> allObjects = mongoTemplate.find(query, MyClass.class);
System.out.println(allObjects);
}
The syntax is now:
Query query = new Query();
query.with(Sort.by(Sort.Direction.DESC, "_id"));
List<MyClass> myClassList= mongoTemplate.find(query, MyClass.class);
Related
I'm writing a new ASP.NET Core Web API, and one of my requirements is to be able to leverage EF Core 3.1 to grab the next value of a sequence defined in my SQL Server as the ID for a record I need to store.
I'm struggling to find a way to do this - in EF 6.x, I used a method directly on the DbContext descendant like this:
public int GetNextSequenceValue()
{
var rawQuery = Database.SqlQuery<int>("SELECT NEXT VALUE FOR dbo.TestSequence;");
var task = rawQuery.SingleAsync();
int nextVal = task.Result;
return nextVal;
}
and for EF Core up to 2.1, I would have been able to use Database.ExecuteSqlCommand() to run a SQL snippet and get back results. But it seems, in EF Core 3.x, I'm out of luck....
I know there are the .FromSqlRaw() and .FromSqlInterpolated methods on the DbSet - but since I only need to return the next value of a sequence (an INT), that's not going to fly. And I also know these methods also exist on the context.Database level which looks like it would be really close to what I had in EF 6.x - but here, those methods will only return the number of rows affected - I haven't found a way to send back the new value from the SEQUENCE.
Can it really be that in EF Core 3.x, I have to actually resort back to way-old ADO.NET code to fetch that value?? Is there REALLY no way to execute an arbitrary SQL snippet and get back some results from the context??
If you want to run an arbitrary TSQL batch and return a scalar value, you can do it like this:
var p = new SqlParameter("#result", System.Data.SqlDbType.Int);
p.Direction = System.Data.ParameterDirection.Output;
context.Database.ExecuteSqlRaw("set #result = next value for some_seq", p);
var nextVal = (int)p.Value;
Looks like executing raw SQL is not priority for EF Core, so up to now (EF Core 3.1) it's providing publicly just few basic limited methods. FromSql requires entity type or keyless entity type, and ExecuteSqlRaw / ExecuteSqlInterpolated are the "modern" bridge to ADO.NET ExecuteNonQuery which returns the affected rows.
The good thing is that EF Core is built on top of a public service architecture, so it can be used to add some missing functionalities. For instance, services can be used to build the so called IRelationalCommand, which has all the DbCommand execute methods, in particular ExecuteScalar needed for SQL in question.
Since EF Core model supports sequences, there is also a service for building the IRelationalCommand needed to retrieve the next value (used internally by HiLo value generators).
With that being said, following is a sample implementation of the custom method in question using the aforementioned concepts:
using System;
using System.Globalization;
using Microsoft.EntityFrameworkCore.Diagnostics;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Storage;
using Microsoft.EntityFrameworkCore.Update;
namespace Microsoft.EntityFrameworkCore
{
public static partial class CustomExtensions
{
public static long GetNextSequenceValue(this DbContext context, string name, string schema = null)
{
var sqlGenerator = context.GetService<IUpdateSqlGenerator>();
var sql = sqlGenerator.GenerateNextSequenceValueOperation(name, schema ?? context.Model.GetDefaultSchema());
var rawCommandBuilder = context.GetService<IRawSqlCommandBuilder>();
var command = rawCommandBuilder.Build(sql);
var connection = context.GetService<IRelationalConnection>();
var logger = context.GetService<IDiagnosticsLogger<DbLoggerCategory.Database.Command>>();
var parameters = new RelationalCommandParameterObject(connection, null, null, context, logger);
var result = command.ExecuteScalar(parameters);
return Convert.ToInt64(result, CultureInfo.InvariantCulture);
}
}
}
In your fluent api configs you can create migration that set ID automatically to be next value from Sequence
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasSequence<int>("OrderNumbers");
modelBuilder.Entity<Order>()
.Property(o => o.OrderNo)
.HasDefaultValueSql("NEXT VALUE FOR shared.OrderNumbers");
}
For creating sequence:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasSequence<int>("OrderNumbers", schema: "shared")
.StartsAt(1000)
.IncrementsBy(5);
}
Read more from here: https://www.talkingdotnet.com/use-sql-server-sequence-in-entity-framework-core-primary-key/
For people suffering with oracle version of this problem, here's a solution:
var p = new OracleParameter("result", OracleDbType.Decimal, null, System.Data.ParameterDirection.Output);
Database.ExecuteSqlRaw($"BEGIN :result := my_seq.nextval; END;", p);
var nextVal = p.Value;
Ugly, but the best thing I could come up with:
var connection = repcontext.Database.GetDbConnection();
connection.Open();
using var cmd = connection.CreateCommand();
cmd.CommandText = "SELECT NEXT VALUE FOR AA.TransSeq;";
var obj = cmd.ExecuteScalar();
connection.Close();
seqnum = (int)obj;
This code should work in a variety of situations:
public static class DbSequence
{
private const string sqlCode = "SELECT NEXT VALUE FOR {0}.{1};";
public static T GetNextSeq<T>(this DbContext dbContext, string seqName)
{
var sqlCnn = dbContext.Database.GetDbConnection();
bool cnnClosed = sqlCnn.State != ConnectionState.Open;
if (cnnClosed) sqlCnn.Open();
try
{
using (var sqlCmd = sqlCnn.CreateCommand())
{
sqlCmd.Transaction = dbContext.Database.CurrentTransaction?.GetDbTransaction();
sqlCmd.CommandText = string.Format(sqlCode, "dbo", seqName);
var result = sqlCmd.ExecuteScalar();
if ((result == null) || (result == DBNull.Value)) throw new InvalidOperationException();
return (T)result;
}
}
finally
{
if (cnnClosed) sqlCnn.Close();
}
}
}
This code works when the connection is closed, opening it when needed and closing it after itself. It should also work when a transaction has been initiated. According to this source: https://learn.microsoft.com/en-us/sql/t-sql/statements/create-sequence-transact-sql?view=sql-server-ver16#general-remarks sequences run outside of transactions. Still, if it's available, I set up a transaction for the command. I also use generics and extension methods.
It's time to ask the community. I cannot find the answer anywhere.
I want to create a generic method that can trace all my repository queries and warn me if a query is not optimized (aka missing an index).
With Spring Data MongoDb v2.x and higher and with the introduction of the Document API, I cannot figure out how to access DBCursor and the explain() method.
The old way was to do it like this:
https://enesaltinkaya.com/java/how-to-explain-a-mongodb-query-in-spring/
Any advise on this is appreciated.
I know this is an old question but wanted to give input from a similar requirement I had in capacity planning for a cosmos Db project using Java Mongo API driver v2.X.
Summarizing Enes Altınkaya's blog post. With an #autowired MongoTemplate we use runCommand to execute server-side db queries by passing a Document object. Getting to an explain output we parse a Query or Aggregate object into a new Document object and add the entry {"executionStats": true}(or {"executionStatistics": true} for cosmos Db). Then wrap it in an another Document using "explain" as the propery.
For Example:
Query:
public static Document documentRequestStatsQuery(MongoTemplate mongoTemplate,
Query query, String collectionName) {
Document queryDocument = new Document();
queryDocument.put("find", collectionName);
queryDocument.put("filter", query.getQueryObject());
queryDocument.put("sort", query.getSortObject());
queryDocument.put("skip", query.getSkip());
queryDocument.put("limit", query.getLimit());
queryDocument.put("executionStatistics", true);
Document command = new Document();
command.put("explain", queryDocument);
Document explainResult = mongoTemplate.getDb().runCommand(command);
return explainResult;
}
Aggregate:
public static Document documentRequestStatsAggregate(MongoTemplate mongoTemplate,
Aggregation aggregate, String collection) {
Document explainAggDocument = Document.parse(aggregate.toString());
explainAggDocument.put("aggregate", collection);
explainAggDocument.put("executionStatistics", true);
Document command = new Document();
command.put("explain", explainAggDocument);
Document explainResult = mongoTemplate.getDb().runCommand(command);
return explainResult;
}
For the actual monitoring, since Service & Repository classes are MongoTemplate abstractions we can use Aspects to capture the query/aggregate execution details as the applications is running.
For Example:
#Aspect
#Component
#Slf4j
public class RequestStats {
#Autowired
MongoTemplate mongoTemplate;
#After("execution(* org.springframework.data.mongodb.core.MongoTemplate.aggregate(..))")
public void logTemplateAggregate(JoinPoint joinPoint) {
Object[] signatureArgs = joinPoint.getArgs();
Aggregation aggregate = (Aggregation) signatureArgs[0];
String collectionName = (String) signatureArgs[1];
Document explainAggDocument = Document.parse(aggregate.toString());
explainAggDocument.put("aggregate", collectionName);
explainAggDocument.put("executionStatistics", true);
Document dbCommand = new Document();
dbCommand.put("explain", explainAggDocument);
Document explainResult = mongoTemplate.getDb().runCommand(dbCommand);
log.info(explainResult.toJson());
}
}
Outputs something like below after each execution:
{
"queryMetrics": {
"retrievedDocumentCount": 101,
"retrievedDocumentSizeBytes": 202214,
"outputDocumentCount": 101,
"outputDocumentSizeBytes": 27800,
"indexHitRatio": 1.0,
"totalQueryExecutionTimeMS": 15.85,
"queryPreparationTimes": {
"queryCompilationTimeMS": 0.21,
"logicalPlanBuildTimeMS": 0.5,
"physicalPlanBuildTimeMS": 0.58,
"queryOptimizationTimeMS": 0.1
},
"indexLookupTimeMS": 10.43,
"documentLoadTimeMS": 0.93,
"vmExecutionTimeMS": 13.6,
"runtimeExecutionTimes": {
"queryEngineExecutionTimeMS": 1.56,
"systemFunctionExecutionTimeMS": 1.36,
"userDefinedFunctionExecutionTimeMS": 0
},
"documentWriteTimeMS": 0.68
}
// ...
I usually log this out into another collection or write to file.
I try to store in the gae datastore an array of string within an Entity. Something like that:
public class MyClass {
private String id;
private String name;
(...)
private List<String> tags;
(...)
}
Entity entity = new Entity("MyClass", "myId");
entity.setProperty("name", "My Name");
List<String> tags = new ArrayList<String>();
tags.add("gae");
tags.add("datastore");
entity.setProperty("tags", tags);
Is this approach the correct one and how to use the tags property in queries (for example, all elements that have the value gae in the tags list)?
Query query = new Query("MyClass");
query.addFilter(fieldName, FilterOperator.EQUAL, "gae");
Iterable<Entity> result = datastore.prepare(q).asIterable();
for (Entity entity : result) {
(...)
}
Thanks for your help!
Thierry
yes thats right, if your tags list doesn't get to big, (more then 10k, have a look at the field size limits)
you can query now you MyClass like this:
Query q = new Query("myClass")
.addFilter("tags",
Query.FilterOperator.EQUAL,
"gae");
I want to make a simple query with a getValueByLabel Method:
Here is my code:
public Config getValueByLabel(String label) throws EntityPersistException{
try {
Query query = em.createQuery("select id from config where config_label=:label",Long.class);
query.setParameter("label", label);
List<Long> config = query.getResultList();
return em.getReference(Config.class, config.get(0));
}
...
when I want to start the method I get:
org.hibernate.hql.internal.ast.QuerySyntaxException: config is not mapped [select id from config where config_label=:label]
Any ideas how to fix that?
UPDATE
I am using:
hibernate 4.0.1.Final
and a postgresql db 1.16.1
syntax of hql is case sensitive. please see if table/entity and column/instance variable names used in query are same as that of object.
I have the following query which I took from my URL
public static String query="pen&mq=pen&f=owners%5B%22abc%22%5D&f=application_type%5B%22cde%22%5D";
public static String q="pen";
I parsed my query string and took each facetname and facet value from it and stored in a map
String querydec = URLDecoder.decode(query, "UTF-8");
String[] facetswithval = querydec.split("&f=");
Map<String, String> facetMap = new HashMap<String, String>();
for (int i = 1; i < facetswithval.length; i++) {
String[] fsplit = facetswithval[i].split("\\[\"");
String[] value = fsplit[1].split("\"\\]");
facetMap.put(fsplit[0], value[0]);
}
Then i use the following code to query in solr using solrj
CommonsHttpSolrServer server = new CommonsHttpSolrServer("http://localhost:8983/solr/");
SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(q);
for (Iterator<String> iter = facetMap.keySet().iterator(); iter.hasNext();){
String key=iter.next();
System.out.println("key="+key+"::value="+facetMap.get(key));
solrQuery.setFilterQueries(key+":"+facetMap.get(key));
}
solrQuery.setRows(MAX_ROW_NUM);
QueryResponse qr = server.query(solrQuery);
SolrDocumentList sdl = qr.getResults();
But after running my code I found out that solrQuery.setFilterQuery method is setting filter for only last set facet. That means if i m running the loop and using this function three times it is taking the last set filter values only.
Can somebody please clarify this and tell me better approach for doing this. Also I am decoding url. So, if my facet contains some special character in the middle then i am not getting any result for that. I tried using it without encoding also but it didnt work. :(
There is also a addFilterQuery method, I would call that since you are setting the filter queries individually in your for loop.
Also, please see this post Filter query with special character using SolrJ client from the Solr Users Mailing List about the need to still escape special characters in queries.