While Importing the data into Solr using DataImportHandler, I am getting the below error. Please someone provide your suggestion.
org.apache.solr.common.SolrException: Document is missing mandatory uniqueKey field: id
at org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:92)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:717)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:557)
at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:235)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:512)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:331)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:239)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:464)
In the schema.xml file you have mentioned id as required field = true.
Also the document that you are trying to index in SOLR do not contain this id field and hence SOLR is throwing this error.
Solution
Either add id to all your documents
OR
Remove required = true form schema file for id field.
Please share your schema.xml file and the documents that you are trying to index into SOLR.
Also keep in mind if you want quick response try to provide as much details as you can.
this is probably where the error reporting should be improved for solr
for my case, I've defined a string field not related with KEY, and I am trying to put a null value to that field. I should probably indicate the field is nullable.
I had this problem too and the reason was in my data import configuration. If you are using a hsqldb connection to import your data, pay attention to the way you map the database columns to your solr fields. Hsqldb is case sensitive. So, for example, the following was producing an error.
<field column="item_id" name="id" />
While this is working perfectly:
<field column="ITEM_ID" name="id" />
I got this issue whereby I either had nulls or duplicates in the id field.
Check that there are no duplicates in your id field with:
SELECT MAX(c) FROM (SELECT COUNT(x) as c, if FROM TABLE GROUP BY id)
And check the response is = 1. If not, you can find the fields with:
SELECT COUNT(*) as c, id FROM <table> GROUP BY id HAVING c > 1
I had the same issue, even if everything was working fine.
I created new SolrDocument in each loop while indexing.
HttpSolrServer server =
new HttpSolrServer("http://localhost:8983/solr/shard1");
Collection<SolrInputDocument> docs = new ArrayList<>();
for (list<String> data: datas) {
SolrInputDocument doc = new SolrInputDocument();
String id = data("id");
String fileName = data("fileName");
String column2 = data("column2");
String column3 = data("column3");
i++;
doc.addField("id", id);
doc.addField("title", fileName);
doc.addField("size", getFileSizeInMb(fileEntry.length()));
doc.addField("column2", column2);
doc.addField("column3", column3);
System.out.println("doc: " + doc);
docs.add(doc);
if (i % 100 == 0) {
System.out.println("Committed :" + i);
server.add(docs);
docs.clear();
}
}
server.commit();
initializing this in each loop made my code work
SolrInputDocument doc = new SolrInputDocument();
Related
I am trying to enter unique values to the Contact object. Here is the code:
List<Contact> conList = new List<Contact> {
new Contact(FirstName='Joe',LastName='Smith',Department='Finance'),
new Contact(FirstName='Kathy',LastName='Smith',Department='Technology'),
new Contact(FirstName='Caroline',LastName='Roth',Department='Finance'),
new Contact()};
// Caroline Roth already exists so I want this code to update her record, not insert another Caroline Roth record
Database.UpsertResult[] srList = Database.upsert(conList, Contact.Fields.Name, false);
Salesforce's documentation states
"The upsert statement matches the sObjects with existing records by comparing values of one field. If you don’t specify a field when calling this statement, the upsert statement uses the sObject’s ID to match the sObject with existing records in Salesforce. Alternatively, you can specify a field to use for matching. For custom objects, specify a custom field marked as external ID. For standard objects, you can specify any field that has the idLookup property set to true. For example, the Email field of Contact or User has the idLookup property set."
I have two questions:
1) how can we see which fields on the Contact object have their idLookup property set to true
2) why am I getting the error in the subject line when I execute the code?
1:
Map<String, Schema.SObjectField> contacFieldsMap = Schema.getGlobalDescribe().get('Contact').getDescribe().fields.getMap();
for (Schema.SObjectField field : contacFieldsMap.values()) {
Schema.DescribeFieldResult fieldResult = field.getDescribe();
if (fieldResult.isIdLookup()) System.debug(fieldResult.getName() + ' IS idLookup');
}
2: System.debug(Contact.Name.getDescribe().isIdLookup()); // false
does Salesforce offer a way to obtain all Objects like Account, Contact etc and populate them in a SQL table with certain columns like
ObjectEntity, FieldName , FieldType ?
I'm pretty sure the only way to achieve this would be by using the Schema.sObjectType and Schema.sObjectField. Here is a link to the documentation for getting all sObjects. You will basically call the Schema.getGlobalDescribe() method which will return you a map of sObjectTypes with their sObject name as the key. Then you'll need to call getDesribe() on each sObjectType get the fields of the object from that Describe result. You'll again need to call getDescribe() on each sObjectField in order to have access to the columns you wanted (eg. FieldType). Here is the documentation on that You could save each DescribeFieldResult into a list that goes into a Map of > with the sObject name as the key, then you could do what you want with them... Put them in a table if you like. Keep in mind this is all going to be very expensive when it comes to CPU time. You may even run into some governor limits.
Here is a little example you can run using Execute Anonymous in the developer console where the sObject Name and all its field names and types are printed to the debug logs
Map<String, sObjectType> objects = Schema.getGlobalDescribe();
for(String objName : objects.keySet()){
system.debug('=======' + objName + '=========\n Fields: ');
sObjectType o = objects.get(objName);
DescribeSobjectResult oDRes = o.getDescribe();
Map<String, Schema.SObjectField> fields = dResult.fields.getMap();
for(Schema.SObjectField f : fields.values()){
DescribeFieldResult fDRes = f.getDescribe();
system.debug('name: ' + fDRes.getName() + ' | type: ' + fDRes.getType());
}
}
Hope this helps
I'm using dbContext and I am running a SQL query that is rather complex (just showing a simple example below), so to avoid having to run the query twice to get a count, I am using COUNT AS to return the total number of records as per other advice on this site.
But, I haven't been able to figure out how to access the resulting property:
using (var db = new DMSContext())
{
string queryString = "select *, COUNT(1) OVER() AS TotalRecords FROM DMSMetas";
var Metas = db.DMSMetas.SqlQuery(queryString).ToList();
for (int i = 0; i <= Metas.Count - 1; i++)
{
var Item = Metas[i];
if (i == 0)
{
//Want to do this, but TotalRecords not part of the DMSMeta class. How to access the created column?
Console.WriteLine("Total records found: " + Item.TotalRecords);
}
}
}
In the sample above, the SQL query generates the extra field TotalRecords. When I run the query in Management Studio, the results are as expected. But how do I access the TotalRecords field through dbContext?
I also tried including the TotalRecords field as part of the DMSMeta class, but then the SQL query fails with the error that the TotalRecords field is specified twice. I tried creating a partial class for DMSMeta containing the TotalRecords field, but then the value remains the default value and is not updated during the query.
I also tried the following:
db.Entry(Item).Property("TotalRecords").CurrentValue
But that generated an error too. Any help would be much appreciated - I am sure I am missing something obvious! All I want is to figure out a way to access the total number of records returned by the query
you have to create a new class (not an entity class but a pure DAO class) DMSMetaWithCount (self explanatory ?) and then
context.Database.SqlQuery<DMSMetaWithCount>("select *, COUNT(1) OVER() AS TotalRecords FROM DMSMetas");
please note that
imho, select * is ALWAYS a bad practice.
you will have no tracking on the not entity new class
Is there a way to fetch a list of all fields in a table in Salesforce? DESCRIBE myTable doesn't work, and SELECT * FROM myTable doesn't work.
From within Apex, you can get this by running the following Apex code snippet. If your table/object is named MyObject__c, then this will give you a Set of the API names of all fields on that object that you have access to (this is important --- even as a System Administrator, if certain fields on your table/object are not visible through Field Level Security to you, they will not show up here):
// Get a map of all fields available to you on the MyObject__c table/object
// keyed by the API name of each field
Map<String,Schema.SObjectField> myObjectFields
= MyObject__c.SObjectType.getDescribe().fields.getMap();
// Get a Set of the field names
Set<String> myObjectFieldAPINames = myObjectFields.keyset();
// Print out the names to the debug log
String allFields = 'ALL ACCESSIBLE FIELDS on MyObject__c:\n\n';
for (String s : myObjectFieldAPINames) {
allFields += s + '\n';
}
System.debug(allFields);
To finish this off, and achieve SELECT * FROM MYTABLE functionality, you would need to construct a dynamic SOQL query using these fields:
List<String> fieldsList = new List<String>(myObjectFieldAPINames);
String query = 'SELECT ';
// Add in all but the last field, comma-separated
for (Integer i = 0; i < fieldsList.size()-1; i++) {
query += fieldsList + ',';
}
// Add in the final field
query += fieldsList[fieldsList.size()-1];
// Complete the query
query += ' FROM MyCustomObject__c';
// Perform the query (perform the SELECT *)
List<SObject> results = Database.query(query);
the describeSObject API call returns all the metadata about a given object/table including its fields. Its available in the SOAP, REST & Apex APIs.
Try using Schema.FieldSet
Schema.DescribeSObjectResult d = Account.sObjectType.getDescribe();
Map<String, Schema.FieldSet> FsMap = d.fieldSets.getMap();
complete documentation
Have you tried DESC myTable?
For me it works fine, it's also in the underlying tips in italic. Look:
I have a table that contains some meta data in an XML field.
For example
<Meta>
<From>tst#test.com</From>
<To>
<Address>testing#123.com</Address>
<Address>2#2.com</Address>
</To>
<Subject>ESubject Goes Here</Subject>
</Meta>
I want to then be able to query this field to return the following results
From To Subject
tst#test.com testing#123.com Subject Goes Here
tst#test.com 2#2.com Subject Goes Here
I've written the following query
SELECT
MetaData.query('data(/Meta/From)') AS [From],
MetaData.query('data(/Meta/To/Address)') AS [To],
MetaData.query('data(/Meta/Subject)') AS [Subject]
FROM
Documents
However this only returns one record for that XML field. It combines both the 2 addresses into one result. Is it possible for split these on to separate records?
The result I'm getting is
From To Subject
tst#test.com testing#123.com 2#2.com Subject Goes Here
Thanks
Gav
You need to return the XML and then parse it using something like the following code:
StringReader stream = new StringReader(stringFromSQL);
XmlReader reader = XmlReader.Create(stream);
while (reader.Read())
{
// Do stuff
}
Where stringFromSQL is the whole string as read from your table.