Migrate Documents to Another Collection - solr

I have a solr collections on the merger, we would like to ask the next question is as follows
I have two collections, c1 and c2,
C1 colleciton there are 10 data, id is from c1_0 to c1_9,
C2 colleciton also has 10 data, id is from c2_0 to c2_9,
I now want to c1 id c1_ format data into the c2, I implemented the following order, it seems no effect, and why?
I c1 designated in the new router.field=id
http://localhost:8081/solr/admin/collections?action=CREATE&name=c1&numShards=3&replicationFactor=3&maxShardsPerNode=3&collection.configName=myconf&router.field=id
I refer to https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api12
Solr version 6.3.0
I have a problem? Or understanding wrong?

Related

How to select the matching records when the where clause contains an array column?

In a project I have a table called documents from where I have to select all the records as documents.id where I have a documents.recipients.
The resulting record I need to have is:
documentId
recipentId
1
gmiG_duuBQOX6WblPXpUk
1
TQ7o1lBsrfDPtBeqGnyYt
2
gmiG_duuBQOX6WblPXpUk
2
TQ7o1lBsrfDPtBeqGnyYt
The problem is as you see from table screenshot that the recipients are an array and I have no idea how to make this matching.
The document can have many recipients, not only 2 as showed in this dummy table, and I have to match all the documents to one recipient ID.
E.G. [{r1,d1},{r1,d2},{r2,d3}]
The result I need to be able to construct a payload in an API later where I have to send an OBJ as e.g.
{
"phone": null,
"email": "foo#bar.com",
"recipients": [{r1,d1},{r1,d2},{r2,d3}]
}
However, the above OBJ was added only for a competition of information.
The help I need is to construct a PostgreSQL query which can result on what I described before and resolving that issue I have with the recipients be an array.
I've been trying as follow but not really understood how to do it:
select
id,
recipients
from
documents d
where
recipients -> '0' = 'gmiG_duuBQOX6WblPXpUk'
As edit added the screenshot of properties of this table
Use the jsonb containment operator #> to find all lines where the recipient contains a certain recipient:
SELECT d.id, 'gmiG_duuBQOX6WblPXpUk'
FROM documents d
WHERE d.recipients #> ARRAY['gmiG_duuBQOX6WblPXpUk'];

Query archaic DB with entity framework

I have inherited a very old DB and I'm just starting out with .NET Core 3 (preview) to see if I can create a proof-of-concept WebApi that uses EF Core to query the DB.
So my problem is that in this DB, much of the data is untrimmed (so has lots of white space around values). There's a mix of varchar(n) and char(n) fields. And no, I'm not in a position to re-engineer this.
So my challenge is that the WebApi will receive a query which will contain a List<People>, and I need to return all relevant data about those people.
If I were doing this in SQL I guess I would do something like:
SELECT *
FROM Table1
INNER JOIN Table2
ON RTRIM(LTRIM(Table1.[P1])) = RTRIM(LTRIM(Table2.[P9]))
I need to be able to do something similar using Entity Framework where I would join the DB table with the .NET List<People>. However, I'm a) not sure how best to go about this with the necessary trims and b) the number of rows in this DB is very large and so it has to be a set-based operation performed on the SQL Server - I can't bring everything back to the DB and join in-memory.
[EDIT (24th June)]
To add some examples of what I've tried, and to re-focus the question onto the approach I should be employing:
1 - I attempted this using a JOIN
var y = from a in query
join b in request.Items
on new { a.Prop1, a.Prop2 } equals new { b.Prop1, b.Prop2 }
select new { a.Prop1, a.Prop2, a.Prop3 };
return await y.ToListAsync().ConfigureAwait(false);
The compile error returned is:
The type of one of the expressions in the join clause is incorrect.
Type inference failed in the call to 'Join'.
Not sure why, but I suspect that this is because I'm trying to join strings to DB char fields.
Also tried:
var y = from a in query
from b in request.Items
where a.Prop1 == b.Prop1 && a.Prop2 == b.Prop2
select new {a.Prop1, a.Prop2, a.Prop3};
return await y.ToListAsync().ConfigureAwait(false);
This compiles, but gives run-time error:
System.InvalidOperationException: 'collection selector was not
NavigationExpansionExpression'
So, yes, I do need to be able to ensure that I'm trimming the data when doing the "join", but I think the main problem is that this needs to be a server-side operation (because of the size of the data sets) but I'd doing a join with a DB table against an in-memory List on the Client.
Not sure what the best approach is here (with EF Core 3).

Single Solrj call for adding and deleting docs

I am using org.apache.solr.client.solrj.impl.HttpSolrServer.HttpSolrServer for calling solr.
For sequential delete and add operations , I am hitting solr like
solr.addBeans(<solrDocs>);
solr.deleteByQuery(<Query>)
solr.commit();
Is there anyway, I can achieve same in one solr call, like solr.execute(addbean, deleteByQuery1)?
I know that multiple commands may be contained in one message as per solr wiki. I what to know that how to achieve same in solrj and any other java library.
What I want to achieve by this ?
Atomic opertion.
Lets have a case:There are two process(or thread) P1 and P2. Each perform Add(corresponding A1 and A2) and Delete(D1 and D2) operation. Let the sequence be like this :
D1 (Deletion of docs by process P1)
D2 (Deletion of docs by process P2)
A2 (Addition of docs by process P2)
P2.commit -> (This will make D1 commited in Solr too)
A1 (Addition of docs by process P1) : Now even if it failed, D1 is not going to rollback (beacuse of P2.commit)
What I want is to rollback P1.D1

Golang GAE - Change a variable name in a structure in Datastore

I am programming a Google Apps Engine Go application and I would like to change a variable name inside a structure that is stored in the datastore.
Say I have a struct:
type AA struct{
A string
BB string
}
And would like to change BB into B. If I try just changing BB into B, the datastore will start giving me errors when it would try to assign stored BB values to new struct AA that does not have that variable. I can add B and still keep BB, but then the struct would start getting messy.
How can I neatly change variable structure in GAE Go datastore without resorting to temporary copying over the entire database and wiping a lot of data?
You can have your AA implement PropertyLoadSaver as described in the Datastore docs, an then
in Load method copy BB into B
in Save method just save A and B
Take a look at the App Engine documentation about Updating Your Model's Schema tha describes the flow you need to follow in order to update your schema and then delete obsolete properties.
Hope this helps.

In Grails... How to union two tables in a single controller?

I have two tables, but they are installed as plugin in Grails.
Columns in T1 are: a1, b1, c1, d1
Columns in T2 are: a2, b2, c2, d2
I need to select columns a*, b*, c*, d* (=1,2) from both tables in a controller as union and sort all of them by the column d, how can I do that?
Furthermore, how can the pagination work as treating about result as a single table?
Pls help. Appreciate!!
Not sure if this is helpful or not. We hit a similar (though not exact) problem in the past, and we solved it by creating a view. You can create the view to do your union and select, and then create the new domain class that maps to the view.
You won't be able to use grails auto create for the table, which is another limitation.
On pagination:
groovy.sql.Sql rows() and eachRow() have 2nd and 3rd parameters max and offset that you can paginate with all like in ordinary list.
Retrieve total count with another SQL query or some other way. Construct a PagedResultList from a data page rows() and int totalCount - and you got an object that you can use as a model.

Resources