I have a "hackathon" model that looks like this:
I need to paginate the data, but right now I'm querying the database based on the amountInPrizes range and a list of tags set by the user on the frontend first, and then filtering the array manually based on the rest of the fields specified by the user (since firestore doesn't allow multiple inequality queries on different fields and I couldn't find another way around it)
Here's what the route looks like:
const {minPrize, maxPrize, inPerson, online, startDateString, endDateString, tags} = req.query;
try {
let hackathonQuery = db.collection("hackathons");
if (minPrize && minPrize !== "") {
hackathonQuery = hackathonQuery.where("amountInPrizes", ">=", Number.parseInt(minPrize));
}
if (maxPrize && maxPrize !== "") {
hackathonQuery = hackathonQuery.where("amountInPrizes", "<=", Number.parseInt(maxPrize));
}
if (tags && tags !== "") {
const tagsList = tags.split(",");
if (tagsList.length >= 1) {
hackathonQuery = hackathonQuery.where("tags", "array-contains-any", tagsList);
}
}
// this is where the pagination should be, but there's still a bunch of manual filters after, so the pagination wouldn't be accurate
const hackathonsSnapshot = await hackathonQuery.orderBy("amountInPrizes", "desc").get();
const hackathons = [];
hackathonsSnapshot.forEach(doc => {
const hackathon = doc.data();
if (startDateString && startDateString !== "") {
const startDate = new Date(startDateString);
if (new Date(hackathon.startDate).getTime() < startDate.getTime()) {
return;
}
}
if (endDateString && endDateString !== "") {
const endDate = new Date(endDateString);
if (new Date(hackathon.endDate).getTime() > endDate.getTime()) {
return;
}
}
if (inPerson && inPerson === "true") {
// to make it so that if both are selected it queries correctly
if (online !== "true") {
if (hackathon.location == "Online") {
return;
}
}
}
if (online && online === "true") {
// to make it so that if both are selected it queries correctly
if (inPerson !== "true") {
if (hackathon.location !== "Online") {
return;
}
}
}
hackathons.push({
id: doc.id,
...hackathon,
})
})
return res.status(200).json({hackathons});
} catch (err) {
console.log(err.message);
res.status(400).send("Server error");
}
If I were to add a .limit() on my initial query, an inconsistent number of documents would be filtered out by the rest of the manual filters on each page of hackathons. Is there a way around this or a better way to do this whole thing?
(Sorry if that explanation was confusing)
If you really need full flexibility on the user input, then there is no way around it.
If you can change the model and change the user interface a bit there are some things you can do:
add onlineLocation: boolean to the model and when saving the document set it to true/false according to the data. This will eliminate your online filtering and will allow you to add a where... to the query (you will not need to use the inequality).
Date range - as I understand the user can select a start date. If it is acceptable to change the interface to allow the user to select from predefined values (like: last week, last month, last year... etc.) what you can do is save in addition to the start date:
startDayIndex = Math.floor(startDate.getTime() / millisecondsInDay)
this results in the day index since Jan 1st 1970
startWeekIndex = Math.floor(startDate.getTime() / millisecondsInWeek)
this results in the week index since Jan 1st 1970
etc...
this will allow you to query firestore with equality query (get all documents that the start date is 3 weeks ago)
I'm not sure if this is applicable for your use case.
If the date range is important to keep as is, you can change the amountInPrizes to be ranges instead of numbers. This way the user can get all documents that the amountInPrizes is between 1000-2000 for example.
again you will need to add a field to your model, say: prizeRange, and query that field with equality query, leaving the date in range query
Related
In my flutter app. I have to fetch data from firebase using multiple filters.
I've done as below. It worked but the problem is that some filter is null so I need to skip it in the firebase query, therefore, even I have only 2 fields (name, age) I have to create 3 indexes for supporting my query. 1st index is for: name, 2nd index is for: age, 3rd index is for: name and age.
Future<List<Transac>> getTrans(TransFilter filter, Transac? lastTrans) async {
const limit = 10;
var result =
_collection.orderBy(filter.orderBy, descending: true).limit(limit);
if (filter.directionId != null) {
result = result.where(directionIdKey, isEqualTo: filter.directionId);
}
if (filter.flag != null) {
result = result.where(Transac.flagKey, isEqualTo: filter.flag);
}
if (filter.officeId != null) {
result = result.where(officeIdKey, isEqualTo: filter.officeId);
}
if (lastTrans != null) {
result = result.startAfter([lastTrans.createdAt.millisecondsSinceEpoch]);
}
final _result = await result.get().then(
(value) => value.docs.map((e) => Transac.fromSnapshot(e)).toList());
return _result;
}
I have tried something as below because I think I just need to create all indexs at once, but it throws an error because I cannot use isNotEqualTo on the field that not use in the first orderby.
Future<List<Transac>> getTrans(TransFilter filter, Transac? lastTrans) async {
const limit = 10;
var result =
_collection.orderBy(filter.orderBy, descending: true).limit(limit);
if (filter.directionId != null) {
result = result.where(directionIdKey, isEqualTo: filter.directionId);
}else {
result = result.where(directionIdKey, isNotEqualTo: '');
}
if (filter.officeId != null) {
result = result.where(officeIdKey, isEqualTo: filter.officeId);
}else {
result = result.where(officeIdKey, isNotEqualTo: '');
}
if (lastTrans != null) {
result = result.startAfter([lastTrans.createdAt.millisecondsSinceEpoch]);
}
final _result = await result.get().then(
(value) => value.docs.map((e) => Transac.fromSnapshot(e)).toList());
return _result;
}
Any solution or suggestion? Your help will be much appreciated, Thank you
You have 3 parameters that you want to filterBy directionIdKey, Transac.flagKey, officeIdKey. You want to orderBy: filter.orderBy.
I believe these parameters are holding the field names.
We need a field that will be present in all queries. We use your orderBy field.
So create 3 composite indexes.
filter.orderBy and directionIdKey ASC
filter.orderBy and Transac.flagKey ASC
filter.orderBy and officeIdKey ASC
This should handle your queries.
How can I make a summary row like this using a material table?
Please help me, thank you.
If by "Summary row" you're referring to table title, that's a prop "title" you just add to the <MaterialTable /> component.
However, I suspect you need the row with Total result, which I couldn't find in the examples, either.
Here's a custom function you could use to calculate a total by your own, add it to your data set and achieve similar result:
const addTotal = (data, byColumn) => {
let keys = Object.keys(data[0]);
let total = data.reduce((acc, el) => {
return acc += +(el[byColumn]);
}, 0);
let totalRow = {};
let emptyRow = {};
for (let key of keys) {
if (key === keys[0]) {
totalRow[key] = 'Total';
} else if (key === byColumn) {
totalRow[key] = total;
} else {
totalRow[key] = '';
}
emptyRow[key] = '';
}
return [...data, emptyRow, totalRow];
}
This will add an empty row and a total with the argument you put as byColumn. You need to be careful about the values you are summing (i.e. add type checking or validate the column name with hasOwnProperty).
I might be a little late, but here’s another question that reminded me that material table happen to have a footer row api.
You can use it alongside a specific method to sum values (or else) that you can call before setting the datasource for example, or on demand.
I know that firestore doesn't support full text search and it giving us solution to use third party services. However I found a simple solution to simple "full text search" and I think this might help others who doesn't want to use third party services as me for such a simple task.
I'm trying to search for company name which is saved in firestore collection under my companyName which can be in any format for example "My Awesome Company". When adding new company with companyName or updating a value in companyName I'm also saving searchName with it which is the same value as company name but in lower case without spaces
searchName: removeSpace(companyName).toLowerCase()
removeSpace is my simple custom function which remove all spaces from a text
export const removeSpace = (string) => {
return string.replace(/\s/g, '');
}
That turns our company name to myawesomecompany which is saved in searchName
Now I've got a firestore function to search for company which indexing through searchName and returning companyName. Minumum search value is a searched value without last character and maximum search value is a searched value with added "zzzzzzzzzzzzzzzzzzzzzzzz" transformed to lower case. That means if you search for My Aw then min value will be mya and max value will be myawzzzzzzzzzzzzzzzzzzzzzzz
exports.handler = ((data) => {
const searchValue = data.value.replace(/\s/g, '').toLowerCase()
const minName = searchValue.substr(0, searchName.length-1)
const maxName = searchValue + "zzzzzzzzzzzzzzzzzzzzzzzz"
let list = []
const newRef = db.collection("user").where("profile.searchName", ">=", minName).where("profile.searchName", "<=", maxName)
return newRef.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
list.push({ name: doc.data().profile.companyName})
})
return list
})
})
I didn't have time to fully test it but so far it works without any problems. Please let me know if you spot anything wrong with it. Now the question is
Is "z" character the highest value character in firestore or is there any other more decent way to add into the search value maximum amount without adding "zzzzzzzzzzzzz"?
I like your decision to preprocess the text so that it can be queried, but you could provide for a more flexible search by storing lowercase keywords with the users and searching those. In other words, transform:
"My Awesome Company"
to...
{ my: true, awesome: true, company: true }
...and test against that.
When adding/updating the property:
// save keywords on the user
let keywords = {}
companyName.split(' ').forEach(word => keywords[word.toLowerCase()] = true)
When querying:
let searchKeywords = userInputString.split(' ').map(word => word.toLowerCase())
let collection = db.collection("user")
searchKeywords.forEach(keyword => {
collection = collection.where(`keywords.${keyword}` , '==' , true);
});
With a little modification of previous answer I have made another simple text search. I'm saving keyword to an array instead of saving it in object like this
nameIndex: textIndexToArray(companyName)
where textIndexToArray is my custom function
export const textIndexToArray = (str) => {
const string = str.trim().replace(/ +(?= )/g,'')
let arr = []
for (let i = 0; i < string.trim().length; i++) {
arr.push(string.substr(0,i+1).toLowerCase());
}
return arr
}
which transfer a text into array. For example
"My Company"
will return
[m, my, my , my c, my co, my com, my comp, my compa, my compan, my company]
with nameIndex saved in firestore we can simply query the data thorough nameIndex and return companyName
exports.handler = ((data) => {
const searchValue = data.value.toLowerCase()
let list = []
const newRef = db.collection("user").where("nameIndex", "array-contains", searchValue)
return newRef.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
list.push({ name: doc.data().companyName, })
})
return list
})
})
I have a question about dynamicly filter array.. I making a table and I have different filtering options. For example ; Only selected column filter, MultiSelect dropdown menu filter and ı want filter multiple columns . my table
For Example ;
Write a filter word in Brand filter. This code line is filtered my data. (veri = data. (in Turkish) ).
public filter(str, i) {
const collName = this.bizimKolon[i].Baslik; // Column Name
this.veri = this.yedekVeri.filter(x => x[collName].toString().toLowerCase().includes(str.toLowerCase())); }
I want to filter the filtered data by year but is not working. And I have a more problem. We were suppose filtered Brand and Year. if the filter word we wrote is in the array, update my array and display into table. OK. No problem. We select colors in dropdown menu. After we want filter by colors ( White and Green ). How we make do this? I don't know how many data will come to filter. in this point we need to dynamic filter. Please help me. thank you..
x[colName] should work.
#Pipe({name: 'dataListFilterPipe'})
export class DataListFilterPipe implements PipeTransform {
transform(data: any, col: string, value: string): any {
if (data !== undefined) {
return data.filter(x => x[col] == value);
}
}
}
I created a pipe in which list is filtered and it works. This is in angular 6
here is my example for filtering by many inputs:
applyFilter() {
let custs = this.ch.customers.filter((cust: Customer) => {
return (
((cust.name.toLowerCase().trim().indexOf(this.filters[0]) !== -1) || (!this.filters[0])) &&
((!this.filters[5]) || (cust.user.username.toLowerCase().trim().indexOf(this.filters[5]) !== -1)) &&
((!this.filters[2]) || (cust.state.name.toLowerCase().trim().indexOf(this.filters[2]) !== -1)) &&
((!this.filters[1]) || (cust.description.toLowerCase().trim().indexOf(this.filters[1]) !== -1)) &&
((!this.filters[3]) || (cust.last_mess.toLowerCase().trim().indexOf(this.filters[3]) !== -1))
);
});
this.dataSource.data = custs;
this.ch.customers_filter = this.dataSource.data;
if (this.dataSource.paginator) {
this.dataSource.paginator.firstPage();
}
}
Variable Filter is array of string.
I'm currently tying to compare 2 dates (and time): the date limit my action has to be completed and the current date.
I receive the date and time in this format: 2017-05-29T15:30:17.983Z.
I then try to compare the 2 dates using this function:
function checkExceedLimit (props) {
exceedLimit = 0;
props.items.map((item) => {
var dateLimit = moment(item.limit)
var now = moment()
if (item.limit != null || item != ' ' && now > dateLimit) {
exceedLimit++;
console.log(now)
console.log(dateLimit)
}
})
}
Basically I want to compare the limit from each item and add +1 to exceedLimit when the current date is passed. Yet it returns +1 for each limit even though not all of them are passed.
Thank you for your help
Simply use isBefore and isAfter, in your case you can do:
if (item.limit != null || item != ' ' && now.isAfter(dateLimit) ) {
First of all you should proper create your instance of moment.
Like this:
moment("2017-05-29T15:30:17.983Z", 'YYYY-MM-DDTHH:mm:ss.SSSZ')
You can check this link for more details: https://momentjs.com/docs/#/parsing/string-format/
And then you can compare date using the moment's API methods on https://momentjs.com/docs/#/query/
So your code should look like this:
function checkExceedLimit (props) {
exceedLimit = 0;
props.items.map((item) => {
const dateLimit = moment(item.limit, 'YYYY-MM-DDTHH:mm:ss.SSSZ');
const now = moment()
if (dateLimit.isValid() && now.isAfter(dateLimit)) {
exceedLimit++;
console.log(now.toString())
console.log(dateLimit.toString())
}
})
}