Below is my actual log,
--2019-05-09 06:49:05.590 -TRACE 6293 --- [ntainer#0-0-C-1] c.s.s.service.MessageLogServiceImpl : [41a6811cbc1c66eda0e942712a12a003d6bf4654b3edb6d24bf159b592afc64f1557384545548] Event => Message Failure Identified : INVALID_STRUCTURE
My given grok filter pattern,
match => {
"message" => "--%{TIMESTAMP_ISO8601:logtime} -%{LOGLEVEL:level} (?<pid>\d+) --- \[(?<thread>[^\]]+)] (?<classname>[\w.]+)\s+: \[(?<token>[^\]]+)] Event \=> Message Failure Identified : (?<code>[\w]+)"
}
After doing some adding/removing desired filed below is my tokenized form,
{
"code" => "INVALID_STRUCTURE",
"event" => "message_failure",
"token" => "41a6811cbc1c66eda0e942712a12a003d6bf4654b3edb6d24bf159b592afc64f1557384545548",
"logtime" => "2019-05-09 06:49:05.590"
}
Now I want to send it to solr, but while sending this is giving me the warning,
[WARN ][logstash.outputs.solrhttp] An error occurred while indexing: undefined method `iso8601' for nil:NilClass
I think it related to "logtime" field since that is the only portion which deals with ISO8601. Nothing extra information found in the logs. What is the problem here?
Finally got the answer. Thanks
this blog
Logstash will use a ruby library called Rsolr for connecting to Solr. Logstash will install the latest version of Rsolr, but latest Rsolr has a bub with timestamp datatype which will prevent indexing to Solr. To prevent this we will have to manually install the working version of Rsolr.
Changed the default rsolr plugin as mentioned in the post and everything started working
Related
I'm setting up my first Solr server via docker using solr:8.11.1-slim. I am gonna use the schema API to set up the schema for my core whose name is 'products'.
While reading the docs there seems to be false info on the docs for field types:
https://solr.apache.org/guide/8_11/field-types-included-with-solr.html
vs.
https://solr.apache.org/guide/8_11/schema-api.html
I followed the first guide to get info on what field types I can specify and am trying to send requests based on the second doc such as this:
{ 'add-field': { "name":"latlong", "type":"LatLongPointSpatialField", "multiValued":False, "stored":True, 'indexed': True } },
but Solr gives me back errors such as:
org.apache.solr.api.ApiBag$ExceptionWithErrObject: error processing commands, errors: [{add-field={name=latlong, type=LatLongPointSpatialField, multiValued=false, stored=true, indexed=true}, errorMessages=[Field 'latlong': Field type 'LatLongPointSpatialField' not found
So what gives? Am I misreading the docs or are they wrong or is something wrong with the solr 8.11.1 image in docker? Why does it not accept the field types I'm providing?
Thanks for your help ahead of time.
I'm upgrading Solr from 6.x to 8.x. In the past, we used to build our request thusly in our PHP script:
$aPostData = array(
'stream.body' => '{"add": {"doc":{...stuff here...}}',
'commit' => 'true',
'collection' => 'mycollection',
'expandMacros' => 'false'
);
$oBody = new \http\Message\Body();
$oBody->addForm($aPostData);
sending it to our Solr server at /solr/mycollection/update/json. That worked just fine in 6.x but now that I've upgraded to 8.x, I'm receiving the following response from Solr
{
"responseHeader":{
"status":400,
"QTime":1
},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"missing content stream",
"code":400
}
}
Digging around I ran across the following
https://issues.apache.org/jira/browse/SOLR-10748
and
Solr error - Stream Body is disabled
I tried following the suggestions of both answers. For the first one, I now see a file called "configoverlay.json" in my ./conf directory and it has those settings. For the second answer, I set it up so my requestParsers node had those attributes. However, neither worked. I've searched around but at this point I'm at my wits end. How can I make it so that I can continue using "stream.body"? If I shouldn't be using "stream.body" is there some other request var that I can/should use when sending my data? I couldn't find anything in the documentation. Perhaps I was looking in the wrong place?
Any help would be greatly appreciated.
thnx,
Christoph
I'm using Solr extension with TYPO3 9.5.3 and I couldn't index the Pages, I get this error https://imgur.com/1e6LfIy
Failed to execute Page Indexer Request. Request ID: 5d78d130b8b4d
When I look at the Solr log, I see that Typo3 add &L=0 to the request URL, the pages with &L=0 return '404 page not found' error :
request url => 'http://example.com/index.php?id=5&L=0' (43 chars)
I added the following code to my TS setup, But that did not work and the request url always ends with &L=0
plugin.tx_solr.index.queue.pages.fields.url.typolink.additionalParams >
I'm not sure that's the only reason solr doesn't index the pages (news can be indexed without any problem), but first, how can I solve the problem and remove &L=0 from request URL in Solr ?
Can you check your TypoScript if you have a configuration like
config.defaultGetVars.L = 0
or if other old language settings exist
I
m not dure, but have you an older languge-Configuration where you Deine the language-Parameter deines?
I'd like to stop GAE instance from CloudFunctions(node.js 8)
I refered to the following documents.
https://cloud.google.com/appengine/docs/admin-api/reference/rest/v1beta5/apps.services.versions/patch?hl=JA
I wrote below code
var requestdata = {
appsId: PROJECT_NAME,
servicesId: SERVICE_ID,
versionsId: VERSION_ID,
auth: authClient,
automaticScaling: {
standardSchedulerSettings:
{
maxInstances: 0,
minInstances: 0
}
},
}
appengine.apps.services.versions.patch(requestdata);
But it is not work well.
I encounter this error message.
Error: function crashed. Details:
Invalid JSON payload received. Unknown name "automaticScaling[standardSchedulerSettings][maxInstances]": Cannot bind query parameter. Field 'automaticScaling[standardSchedulerSettings][maxInstances]' could not be found in request message.
Invalid JSON payload received. Unknown name "automaticScaling[standardSchedulerSettings][minInstances]": Cannot bind query parameter. Field 'automaticScaling[standardSchedulerSettings][minInstances]' could not be found in request message.
I do not know how to solve the problem.
If you have any advice, please let me know.
This is because standardSchedulerSettings is not a valid parameter as it does not exist in v1beta5.
As of January 2019, the Admin API was upgraded from v1beta -> V1.
The v1beta4 and v1beta5 versions of the API are no longer supported and scheduled for shut down on January 14, 2019.
To resolve this just update any old dependencies you may have to the latest version and make sure to follow the latest V1 apps.services.versions.patch documentation .
This worked for me.
i use solr 4.3 for my website ,
whie i query one data with morelike this function ,
sometime this exception goes out :
org.apache.solr.common.SolrException: parsing error
at org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:43)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
parsing error??
how was it ?
why it occures ?
thanks for your reply .
this code :
the code :
public MoreLikeThisQueryResponse(QueryResponse queryResponse) {
this.queryResponse = queryResponse;
NamedList<Object> res = this.queryResponse.getResponse();
for (int i = 0; i < res.size(); i++) {
String name = res.getName(i);
if ("match".equals(name)) {
this.matchResult = (SolrDocumentList) res.getVal(i);
}
}
}
This type of "parsing error" in the BinaryResponseParser.processResponse method typically indicates that the server is returning a raw HTML text response rather than a Solr javabin response. Typically that means that the server container (Tomcat or Jetty) is detecting and reporting an error before Solr gets a chance to handle the error and obey the wt parameter which sets the response format to javabin.
Check the server log for the actual error.
I had a similar problem.
It seems that there might be a compatibility error is your are not using the EXACT compatible versions of Solr and Solrj.
To resolve this, you have to specify
server.setParser(new XMLResponseParser());
Quoting from Solrj
SolrJ generally maintains backwards compatibility, so you can use a newer SolrJ with an older Solr, or an older SolrJ with a newer Solr. There are some minor exceptions to this general rule:
If you're mixing 1.x and a later major version, you must set the response parser to XML, because the two versions use incompatible versions of javabin.