How to build stream expression with StreamFactory - solr

I am trying to build let expression for solr stream but it gives error saying
Caused by: java.io.IOException: Invalid stream expression col(samples,score) - function 'col' is unknown (not mapped to a valid TupleStream)
StreamFactory = new StreamFactory().withDefaultZkHost(zookeeper)
.withFunctionName("search", CloudSolrStream.class)
.withFunctionName("select", SelectStream.class)
.withFunctionName("merge", MergeStream.class)
.withFunctionName("sort", SortStream.class)
.withFunctionName("tuple", TupStream.class)
.withFunctionName("rollup", RollupStream.class)
.withFunctionName("hashJoin", HashJoinStream.class)
.withFunctionName("count", CountMetric.class)
.withFunctionName("facet", FacetStream.class)
.withFunctionName("sum", SumMetric.class)
.withFunctionName("unique", UniqueStream.class)
.withFunctionName("significantTerms", SignificantTermsStream.class)
.withFunctionName("stats", StatsStream.class)
.withFunctionName("innerJoin", InnerJoinStream.class)
.withFunctionName("issnMerge", IssnMergeStream.class)
.withFunctionName("intersect", IntersectStream.class)
.withFunctionName("boostByEra", BoostBooksByEraExpression.class)
.withFunctionName("determineRRec", DetermineRepresentativeRecord.class)
.withFunctionName("plist", ParallelListStream.class)
.withFunctionName("let", LetStream.class);
My expression is as below:
let(samples=search(wc_art,qt="/select",q="al:next AND al:generation",fq="_delete:false",fq="_namespace:default",fl="score,numdocs(),id",rows=10,fq="{!lucene q.op=OR}pa:(medline)",defType=edismax,q.op=AND,sort="score desc"),responses=col(samples,score),summary=describe(responses))

The col function was added in Solr 7, so if your Solr installation is older than that the function won't be available. Upgrade to a more recent version.

Related

Mautic and Salesforces integration

I have this error with Mautic and Salesforces
Fetching contacts…
In AbstractIntegration.php line 2002:
Typed property Mautic\PluginBundle\Integration\AbstractIntegration::$adminU
sers must not be accessed before initialization
mautic:integration:fetchleads [-i|–integration INTEGRATION] [-d|–start-date START-DATE] [-t|–end-date END-DATE] [–fetch-all] [-a|–time-interval [TIME-INTERVAL]] [-l|–limit [LIMIT]] [-f|–force] [-h|–help] [-q|–quiet] [-v|vv|vvv|–verbose] [-V|–version] [–ansi] [–no-ansi] [-n|–no-interaction] [-e|–env ENV] [–no-debug] [–]
Any idea how to resolve this?
Thanks

SQL compilation error: Format argument for function 'TO_TIMESTAMP_TZ' needs to be a string

I am querying data from a internally stage file in snowflake as below -
I get the following error:
SQL compilation error: Format argument for function 'TO_TIMESTAMP_TZ' needs to be a string
Any idea what could be going wrong here?
Thanks
SELECT
TO_VARCHAR(stg.$10),
TO_VARCHAR(stg.$45),
TO_NUMBER(stg.$1,20),
TO_NUMBER(stg.$18,20),
TO_VARCHAR(stg.$8),
TO_VARCHAR(stg.$42),
TO_NUMBER(stg.$19,20),
TO_VARCHAR(stg.$11),
TO_VARCHAR(stg.$16),
TO_VARCHAR(stg.$49),
TO_TIMESTAMP_TZ(stg.$47::STRING,'21'),
TO_TIMESTAMP_TZ(stg.$50::STRING,10),
TO_VARCHAR(stg.$48),
TO_NUMBER(stg.$36,19,6),
TO_NUMBER(stg.$27,19,6),TO_VARCHAR(stg.$12),
TO_TIMESTAMP_TZ(stg.$13::STRING,10),
TO_TIMESTAMP_TZ(stg.$5::STRING,10),
TO_NUMBER(stg.$22,20),
TO_NUMBER(stg.$21,19,6),
TO_NUMBER(stg.$20,19,6),
TO_TIMESTAMP_TZ(stg.$2::STRING,10),
TO_NUMBER(stg.$39,19,6),TO_VARCHAR(stg.$35),
TO_VARCHAR(stg.$4),
TO_NUMBER(stg.$40,19,6),
TO_NUMBER(stg.$32,19,6),
TO_NUMBER(stg.$33,19,6),
TO_VARCHAR(stg.$34),
TO_VARCHAR(stg.$37),
TO_VARCHAR(stg.$38),
TO_VARCHAR(stg.$17),
TO_VARCHAR(stg.$23),
TO_NUMBER(stg.$14,19,6),
TO_NUMBER(stg.$28,19,6),
TO_NUMBER(stg.$29,19,6),
TO_NUMBER(stg.$41,6,2),
TO_VARCHAR(stg.$31),
TO_NUMBER(stg.$30,19,6),
TO_VARCHAR(stg.$24),
TO_VARCHAR(stg.$25),
TO_NUMBER(stg.$26,19,6),
TO_VARCHAR(stg.$9),
TO_TIMESTAMP_TZ(stg.$3::STRING,10),
TO_VARCHAR(stg.$15),
TO_VARCHAR(stg.$44),
TO_VARCHAR(stg.$43),
TO_VARCHAR(stg.$53),
TO_TIMESTAMP_TZ(stg.$51::STRING,10),
TO_TIMESTAMP_TZ(stg.$50::STRING,10),
TO_VARCHAR(stg.$52),
TO_TIMESTAMP_TZ(stg.$7::STRING,10),
TO_TIMESTAMP_TZ(stg.$6::STRING,10),
'T_RPDB_POLICY_1_0_0',
TO_TIMESTAMP_TZ(CURRENT_TIMESTAMP::STRING)
FROM '#INTERNAL_POLICY_STAGE/T_RPDB_POLICY.CSV.gz' (file_format => '"JVCO"."STAGING".CSV') stg;
Try
select to_timestamp_tz(CURRENT_TIMESTAMP::string)
Second argument is format(varchar type) and you are passing number.
select to_timestamp_tz('04/05/2013 01:02:03', 'mm/dd/yyyy hh24:mi:ss');
https://docs.snowflake.com/en/sql-reference/functions/to_timestamp.html

ijson.common.IncompleteJSONError: lexical error: invalid char in json text

hi all ijson newbie I have a very large .json file 168 (GB) I want to get all possible keys, but in the file some values are written as NaN. ijson creates a generator and outputs dictionaries, in My code value. When a specific item is returned, it throws an error. How can you get a string instead of a dictionary instead of value? Tried **parser = ijson.items (input_file, '', multiple_values = True, map_type = str) **, didn't help.
def parse_json(json_filename):
with open('max_data_error.txt', 'w') as outfile:
with open(json_filename, 'r') as input_file:'''
# outfile.write('[ '
parser = ijson.items(input_file, '', multiple_values=True)
cont = 0
max_keys_list = list()
for value in parser:
for i in json.loads(json.dumps(value, ensure_ascii=False, default=str)) :
if i not in max_keys_list:
max_keys_list.append(i)
print(value)
print(max_keys_list)
for keys_item in max_keys_list:
outfile.write(keys_item + '\n')
if __name__ == '__main__':
parse_json('./email/emailrecords.bson.json')
Traceback (most recent call last):
File "panda read.py", line 29, in <module>
parse_json('./email/emailrecords.bson.json')
File "panda read.py", line 17, in parse_json
for value in parser:
ijson.common.IncompleteJSONError: lexical error: invalid char in json text.
litecashwire.com","lastname":NaN,"firstname":"Mia","zip":"87
(right here) ------^
Your file I not valid JSON (NaN is not a valid JSON value); therefore any JSON parsing library will complain about this, one way or another, unless they have an extension to handle this non-standard content.
The ijson FAQ found in the project description has a question about invalid UTF-8 characters and how to deal with them. Those same answers apply here, so I would suggest you go and try one of those.

invalid sign in external "numeric" value error from Npgsql Binary copy

I get this error on using Npgsql Binary copy (see my implementation at the bottom):
invalid sign in external "numeric" value
The RetailCost column on which it fails has the following PostgreSQL properties:
type: numeric(19,2)
nullable: not null
storage: main
default: 0.00
The PostgreSQL log looks like this:
2019-07-15 13:24:05.131 ACST [17856] ERROR: invalid sign in external "numeric" value
2019-07-15 13:24:05.131 ACST [17856] CONTEXT: COPY products_temp, line 1, column RetailCost
2019-07-15 13:24:05.131 ACST [17856] STATEMENT: copy products_temp (...) from stdin (format binary)
I don't think it should matter, but there are only zero or positive RetailCost values (no negative or null ones)
My implementation looks like this:
using (var importer = conn.BeginBinaryImport($"copy {tempTableName} ({dataColumns}) from stdin (format binary)"))
{
foreach (var product in products)
{
importer.StartRow();
importer.Write(product.ManufacturerNumber, NpgsqlDbType.Text);
if (product.LastCostDateTime == null)
importer.WriteNull();
else
importer.Write((DateTime)product.LastCostDateTime, NpgsqlDbType.Timestamp);
importer.Write(product.LastCost, NpgsqlDbType.Numeric);
importer.Write(product.AverageCost, NpgsqlDbType.Numeric);
importer.Write(product.RetailCost, NpgsqlDbType.Numeric);
if (product.TaxPercent == null)
importer.WriteNull();
else
importer.Write((decimal)product.TaxPercent, NpgsqlDbType.Numeric);
importer.Write(product.Active, NpgsqlDbType.Boolean);
importer.Write(product.NumberInStock, NpgsqlDbType.Bigint);
}
importer.Complete();
}
Any suggestions would be welcome
I caused this problem when using MapBigInt, not realizing the column was accidentally numeric(18). Changing the column to bigint fixed my problem.
The cause of the problem was that the importer.Write statements was not in the same order as the dataColumns

Error in rowSums(x) : 'x' must be an array of at least two dimensions (Vegan:oecosimu)

I am trying to use the application oecosimu in Vegan package.
library(sna)
library(permute)
library(lattice)
library(vegan)
library(bipartite)
bio<-read.csv("/home/vicar66/Raw_data/Jan12/98percents_April_16/otu_table/Chapter1/no_bloom_bio_data_L3.txt")
rownames(bio)<-bio[,1]
bio[,1]<-NULL
bio_m<-as.matrix(bio)
a<-oecosimu(bio_m,bipartite::C.score,"swap")
but I keep having this error message:
Attaching package: 'bipartite'
The following object is masked from 'package:vegan':
nullmodel
Error in rowSums(x) : 'x' must be an array of at least two dimensions
Calls: oecosimu -> nullmodel -> rowSums
Execution halted
demo data:
Ciliophora Dinoflagellates MALVs Choanoflagellida
DNAHIN5m 0.062469804 0.826323018 0.031084701 0.000323747
DNAHIN35m 0.045216826 0.595750636 0.187010719 0.000917053
DNAHIN150m 0.018434224 0.204865296 0.531016032 0.017009618
DNAHIN240m 0.016211333 0.889640227 0.04191846 0.03087407
**first column first row is empty. First row are rownames
Anyone have encountered this problem yet?
Thanks!

Resources