Mongodb find and delete records matching in another collection - database

Question

I have two collections in my database.
First collection's name is init_records. A record document looks like,
if automatically created:
{
"_id": 0,
"email": "a#xxx.com",
"source":"campaign"
}
else if manually created by the user:
{
"_id": 1,
"email": "a#xxx.com",
"type":"typeA",
"isActive": false,
"process":"waiting"
}
You can see, auto created records have field "source" as record's source. But they don't have fields "type", "isActive" or "process"; team_id and isActive must be selected by the user manually, after user confirms the record, process field is added and set to "waiting". User may create a init_record self or update a automatically created record (like _id:0, some necessary fields are missing, they will be added by the user) and set for process.
Second collection's name is confirmed_records. If the user confirms an init record, a new confirmed record is created, for example, the above init_record with _id:1
{
"_id": "aa1",
"email": "a#xxx.com",
"type": "typeA",
"isActive": false
}
and the record in init_records collection (_id:1) is deleted. But if some error occurs in this process, no confirmed_record is created, init_record is not deleted but "process" field is set from "waiting" to "error".
Well, now I have decided to clean up some init_records, because users are usually ignoring automatically created (init records with "source" field) records and they manually create records with the same email. That means meanwhile they have a record for an email,
{
"_id": 10,
"email": "count#xxx.com",
"source":"commercial"
}
they ignore that record and create another record,
{
"_id": 11,
"email": "count#xxx.com",
"type":"typeB",
"isActive": false
"process":"waiting"
}
(As you can see, automatically created records have "source" field but no "process" field. Manually created ones have "process" and "isActive" but no "source")
And then after they confirm that record they have created, a confirmed_record is created,
{
"_id": "aa85",
"email": "count#xxx.com",
"type":"typeB",
"isActive": false
}
init_record _id:11 is deleted but init_record _id:10 still remains there. When you check an email (count#xxx.com in this example) they see both a confirmed record and an init (not confirmed) record and this is confusing.
Now I need to delete some init_records because some user's are very unhappy.
For an email:
If there is only an init_record (automatically or manually created), it shall keep remaining.
If there is an automatically created init_record and user has ignored it and created a second init_record (like the above example, _id: 10 and 11) the automatically created init_record (which has no "process" or "isActive" field) must be deleted
If there is an automatically created init_record but user has ignored it and created a second init_record (_id: 10 and 11) and then confirmed it (like the above example, _id:11 is deleted and confirmed_record _id:"aa85" is created), the remaining init_record (_id: 11) must be deleted.
If there is an automatically created init_record but user has ignored it and created a second init_record (_id: 10 and 11) and then -ignored the newly created one- and confirmed the automatically created one (this time _id:10 is deleted and a confirmed_record is created), the remaining init_record (this time _id: 10) must be deleted.
I created an example with mongo playgound with sample data:
https://mongoplayground.net/p/gdHsOPF-Z9F
Shortly:
init_record a#xxx.com also exists in confirmed_records, but this record is created by a user(have type and isActive fields) should remain there
init_record c#xxx.com (Automatically created) also exists in confirmed_records, init_record must be deleted
init_record d#xxx.com (Automatically created) also exists in confirmed_records (there are 2 confirmed_records with different types), init_record must be deleted
init_record f#xxx.com does not exist in confirmed_records, should remain there
init_record g#xxx.com does not exist in confirmed_records, should remain there
There are 2 init_records for h#xxx.com, one of them is automatically created, the other is manually created by a user. Only the automatically created (missing fields) one must be deleted.
(_ids 3, 6, 13 must be deleted, 10, 12, 14, 17 should remain)
So, if I check the records with the same email in both collections,
db.init_records.aggregate([
{
"$lookup": {
"from": "confirmed_records",
"localField": "email",
"foreignField": "email",
"as": "relation"
}
},
{
"$match": {
"$and": [
{
"relation._id": {
"$exists": true
}
},
{
"type": {
"$exists": false
}
}
]
}
}
])
3 records are listed with this aggregate, these are the emails which have both init and confirmed records, and also do not have "process" fields which means they are not confirmed by the user. But I don't know how to make this query to delete records, deleteMany method deletes records matching a filter, not an aggregate pipeline (as far as I know).
My real problem here is I also have to find and delete duplicate init_records, (if there is more than one init_records with same "email" and if at least one of them is manually created = the automatically created records must be removed), I have no clue what to do here. I can detect _ids 3 and 6 (these emails also exist in confirmed_records and they do not have "type" field) but I cannot detect _id:13 which is an automatically created record and has a manually created duplicate.
I have a feeling that I am missing something here, is there an easy way to handle this (finding both existing records in the other collection and also finding duplicate records (with some conditions and also deleting these unwanted records obtained with aggregate). Any ideas are appreciated, I have already spent many hours but couldn't find a solution yet. If you guys think this thing cannot be done via a few simple queries, I have to write some code to work with live data (Maybe a long batch process to check all the records one by one but still doable for me). Thank you for reading this.