I am looking to add a simple search field, would like to use something like
collectionRef.where('name', 'contains', 'searchTerm')
I tried using where('name', '==', '%searchTerm%'), but it didn't return anything.
I agree with #Kuba's answer, But still, it needs to add a small change to work perfectly for search by prefix. here what worked for me
For searching records starting with name queryText
collectionRef
.where('name', '>=', queryText)
.where('name', '<=', queryText+ '\uf8ff')
The character \uf8ff used in the query is a very high code point in the Unicode range (it is a Private Usage Area [PUA] code). Because it is after most regular characters in Unicode, the query matches all values that start with queryText.
Full-Text Search, Relevant Search, and Trigram Search!
UPDATE - 2/17/21 - I created several new Full Text Search Options.
See Code.Build for details.
Also, side note, dgraph now has websockets for realtime... wow, never saw that coming, what a treat! Cloud Dgraph - Amazing!
--Original Post--
A few notes here:
1.) \uf8ff works the same way as ~
2.) You can use a where clause or start end clauses:
ref.orderBy('title').startAt(term).endAt(term + '~');
is exactly the same as
ref.where('title', '>=', term).where('title', '<=', term + '~');
3.) No, it does not work if you reverse startAt() and endAt() in every combination, however, you can achieve the same result by creating a second search field that is reversed, and combining the results.
Example: First you have to save a reversed version of the field when the field is created. Something like this:
// collection
const postRef = db.collection('posts')
async function searchTitle(term) {
// reverse term
const termR = term.split("").reverse().join("");
// define queries
const titles = postRef.orderBy('title').startAt(term).endAt(term + '~').get();
const titlesR = postRef.orderBy('titleRev').startAt(termR).endAt(termR + '~').get();
// get queries
const [titleSnap, titlesRSnap] = await Promise.all([
titles,
titlesR
]);
return (titleSnap.docs).concat(titlesRSnap.docs);
}
With this, you can search the last letters of a string field and the first, just not random middle letters or groups of letters. This is closer to the desired result. However, this won't really help us when we want random middle letters or words. Also, remember to save everything lowercase, or a lowercase copy for searching, so case won't be an issue.
4.) If you have only a few words, Ken Tan's Method will do everything you want, or at least after you modify it slightly. However, with only a paragraph of text, you will exponentially create more than 1MB of data, which is bigger than firestore's document size limit (I know, I tested it).
5.) If you could combine array-contains (or some form of arrays) with the \uf8ff trick, you might could have a viable search that does not reach the limits. I tried every combination, even with maps, and a no go. Anyone figures this out, post it here.
6.) If you must get away from ALGOLIA and ELASTIC SEARCH, and I don't blame you at all, you could always use mySQL, postSQL, or neo4Js on Google Cloud. They are all 3 easy to set up, and they have free tiers. You would have one cloud function to save the data onCreate() and another onCall() function to search the data. Simple...ish. Why not just switch to mySQL then? The real-time data of course! When someone writes DGraph with websocks for real-time data, count me in!
Algolia and ElasticSearch were built to be search-only dbs, so there is nothing as quick... but you pay for it. Google, why do you lead us away from Google, and don't you follow MongoDB noSQL and allow searches?
There's no such operator, allowed ones are ==, <, <=, >, >=.
You can filter by prefixes only, for example for everything that starts between bar and foo you can use
collectionRef
.where('name', '>=', 'bar')
.where('name', '<=', 'foo')
You can use external service like Algolia or ElasticSearch for that.
While Kuba's answer is true as far as restrictions go, you can partially emulate this with a set-like structure:
{
'terms': {
'reebok': true,
'mens': true,
'tennis': true,
'racket': true
}
}
Now you can query with
collectionRef.where('terms.tennis', '==', true)
This works because Firestore will automatically create an index for every field. Unfortunately this doesn't work directly for compound queries because Firestore doesn't automatically create composite indexes.
You can still work around this by storing combinations of words but this gets ugly fast.
You're still probably better off with an outboard full text search.
While Firebase does not explicitly support searching for a term within a string,
Firebase does (now) support the following which will solve for your case and many others:
As of August 2018 they support array-contains query. See: https://firebase.googleblog.com/2018/08/better-arrays-in-cloud-firestore.html
You can now set all of your key terms into an array as a field then query for all documents that have an array that contains 'X'. You can use logical AND to make further comparisons for additional queries. (This is because firebase does not currently natively support compound queries for multiple array-contains queries so 'AND' sorting queries will have to be done on client end)
Using arrays in this style will allow them to be optimized for concurrent writes which is nice! Haven't tested that it supports batch requests (docs don't say) but I'd wager it does since its an official solution.
Usage:
collection("collectionPath").
where("searchTermsArray", "array-contains", "term").get()
Per the Firestore docs, Cloud Firestore doesn't support native indexing or search for text fields in documents. Additionally, downloading an entire collection to search for fields client-side isn't practical.
Third-party search solutions like Algolia and Elastic Search are recommended.
I'm sure Firebase will come out with "string-contains" soon to capture any index[i] startAt in the string...
But
I’ve researched the webs and found this solution thought of by someone else
set up your data like this
state = { title: "Knitting" };
// ...
const c = this.state.title.toLowerCase();
var array = [];
for (let i = 1; i < c.length + 1; i++) {
array.push(c.substring(0, i));
}
firebase
.firestore()
.collection("clubs")
.doc(documentId)
.update({
title: this.state.title,
titleAsArray: array
});
query like this
firebase.firestore()
.collection("clubs")
.where(
"titleAsArray",
"array-contains",
this.state.userQuery.toLowerCase()
)
As of today (18-Aug-2020), there are basically 3 different workarounds, which were suggested by the experts, as answers to the question.
I have tried them all. I thought it might be useful to document my experience with each one of them.
Method-A: Using: (dbField ">=" searchString) & (dbField "<=" searchString + "\uf8ff")
Suggested by #Kuba & #Ankit Prajapati
.where("dbField1", ">=", searchString)
.where("dbField1", "<=", searchString + "\uf8ff");
A.1 Firestore queries can only perform range filters (>, <, >=, <=) on a single field. Queries with range filters on multiple fields are not supported. By using this method, you can't have a range operator in any other field on the db, e.g. a date field.
A.2. This method does NOT work for searching in multiple fields at the same time. For example, you can't check if a search string is in any of the fileds (name, notes & address).
Method-B: Using a MAP of search strings with "true" for each entry in the map, & using the "==" operator in the queries
Suggested by #Gil Gilbert
document1 = {
'searchKeywordsMap': {
'Jam': true,
'Butter': true,
'Muhamed': true,
'Green District': true,
'Muhamed, Green District': true,
}
}
.where(`searchKeywordsMap.${searchString}`, "==", true);
B.1 Obviously, this method requires extra processing every time data is saved to the db, and more importantly, requires extra space to store the map of search strings.
B.2 If a Firestore query has a single condition like the one above, no index needs to be created beforehand. This solution would work just fine in this case.
B.3 However, if the query has another condition, e.g. (status === "active",) it seems that an index is required for each "search string" the user enters. In other words, if a user searches for "Jam" and another user searches for "Butter", an index should be created beforehand for the string "Jam", and another one for "Butter", etc. Unless you can predict all possible users' search strings, this does NOT work - in case of the query has other conditions!
.where(searchKeywordsMap["Jam"], "==", true); // requires an index on searchKeywordsMap["Jam"]
.where("status", "==", "active");
**Method-C: Using an ARRAY of search strings, & the "array-contains" operator
Suggested by #Albert Renshaw & demonstrated by #Nick Carducci
document1 = {
'searchKeywordsArray': [
'Jam',
'Butter',
'Muhamed',
'Green District',
'Muhamed, Green District',
]
}
.where("searchKeywordsArray", "array-contains", searchString);
C.1 Similar to Method-B, this method requires extra processing every time data is saved to the db, and more importantly, requires extra space to store the array of search strings.
C.2 Firestore queries can include at most one "array-contains" or "array-contains-any" clause in a compound query.
General Limitations:
None of these solutions seems to support searching for partial strings. For example, if a db field contains "1 Peter St, Green District", you can't search for the string "strict."
It is almost impossible to cover all possible combinations of expected search strings. For example, if a db field contains "1 Mohamed St, Green District", you may NOT be able to search for the string "Green Mohamed", which is a string having the words in a different order than the order used in the DB field.
There is no one solution that fits all. Each workaround has its limitations. I hope the information above can help you during the selection process between these workarounds.
For a list of Firestore query conditions, please check out the documentation https://firebase.google.com/docs/firestore/query-data/queries.
I have not tried https://fireblog.io/blog/post/firestore-full-text-search, which is suggested by #Jonathan.
Late answer but for anyone who's still looking for an answer, Let's say we have a collection of users and in each document of the collection we have a "username" field, so if want to find a document where the username starts with "al" we can do something like
FirebaseFirestore.getInstance()
.collection("users")
.whereGreaterThanOrEqualTo("username", "al")
I used trigram just like Jonathan said it.
trigrams are groups of 3 letters stored in a database to help with searching. so if I have data of users and I let' say I want to query 'trum' for donald trump I have to store it this way
and I just to recall this way
onPressed: () {
//LET SAY YOU TYPE FOR 'tru' for trump
List<String> search = ['tru', 'rum'];
Future<QuerySnapshot> inst = FirebaseFirestore.instance
.collection("users")
.where('trigram', arrayContainsAny: search)
.get();
print('result=');
inst.then((value) {
for (var i in value.docs) {
print(i.data()['name']);
}
});
that will get correct result no matter what
EDIT 05/2021:
Google Firebase now has an extension to implement Search with Algolia. Algolia is a full text search platform that has an extensive list of features. You are required to have a "Blaze" plan on Firebase and there are fees associated with Algolia queries, but this would be my recommended approach for production applications. If you prefer a free basic search, see my original answer below.
https://firebase.google.com/products/extensions/firestore-algolia-search
https://www.algolia.com
ORIGINAL ANSWER:
The selected answer only works for exact searches and is not natural user search behavior (searching for "apple" in "Joe ate an apple today" would not work).
I think Dan Fein's answer above should be ranked higher. If the String data you're searching through is short, you can save all substrings of the string in an array in your Document and then search through the array with Firebase's array_contains query. Firebase Documents are limited to 1 MiB (1,048,576 bytes) (Firebase Quotas and Limits) , which is about 1 million characters saved in a document (I think 1 character ~= 1 byte). Storing the substrings is fine as long as your document isn't close to 1 million mark.
Example to search user names:
Step 1: Add the following String extension to your project. This lets you easily break up a string into substrings. (I found this here).
extension String {
var length: Int {
return count
}
subscript (i: Int) -> String {
return self[i ..< i + 1]
}
func substring(fromIndex: Int) -> String {
return self[min(fromIndex, length) ..< length]
}
func substring(toIndex: Int) -> String {
return self[0 ..< max(0, toIndex)]
}
subscript (r: Range<Int>) -> String {
let range = Range(uncheckedBounds: (lower: max(0, min(length, r.lowerBound)),
upper: min(length, max(0, r.upperBound))))
let start = index(startIndex, offsetBy: range.lowerBound)
let end = index(start, offsetBy: range.upperBound - range.lowerBound)
return String(self[start ..< end])
}
Step 2: When you store a user's name, also store the result of this function as an array in the same Document. This creates all variations of the original text and stores them in an array. For example, the text input "Apple" would creates the following array: ["a", "p", "p", "l", "e", "ap", "pp", "pl", "le", "app", "ppl", "ple", "appl", "pple", "apple"], which should encompass all search criteria a user might enter. You can leave maximumStringSize as nil if you want all results, however, if there is long text, I would recommend capping it before the document size gets too big - somewhere around 15 works fine for me (most people don't search long phrases anyway).
func createSubstringArray(forText text: String, maximumStringSize: Int?) -> [String] {
var substringArray = [String]()
var characterCounter = 1
let textLowercased = text.lowercased()
let characterCount = text.count
for _ in 0...characterCount {
for x in 0...characterCount {
let lastCharacter = x + characterCounter
if lastCharacter <= characterCount {
let substring = textLowercased[x..<lastCharacter]
substringArray.append(substring)
}
}
characterCounter += 1
if let max = maximumStringSize, characterCounter > max {
break
}
}
print(substringArray)
return substringArray
}
Step 3: You can use Firebase's array_contains function!
[yourDatabasePath].whereField([savedSubstringArray], arrayContains: searchText).getDocuments....
I just had this problem and came up with a pretty simple solution.
String search = "ca";
Firestore.instance.collection("categories").orderBy("name").where("name",isGreaterThanOrEqualTo: search).where("name",isLessThanOrEqualTo: search+"z")
The isGreaterThanOrEqualTo lets us filter out the beginning of our search and by adding a "z" to the end of the isLessThanOrEqualTo we cap our search to not roll over to the next documents.
I actually think the best solution to do this within Firestore is to put all substrings in an array, and just do an array_contains query. This allows you to do substring matching. A bit overkill to store all substrings but if your search terms are short it's very very reasonable.
If you don't want to use a third-party service like Algolia, Firebase Cloud Functions are a great alternative. You can create a function that can receive an input parameter, process through the records server-side and then return the ones that match your criteria.
This worked for me perfectly but might cause performance issues.
Do this when querying firestore:
Future<QuerySnapshot> searchResults = collectionRef
.where('property', isGreaterThanOrEqualTo: searchQuery.toUpperCase())
.getDocuments();
Do this in your FutureBuilder:
return FutureBuilder(
future: searchResults,
builder: (context, snapshot) {
List<Model> searchResults = [];
snapshot.data.documents.forEach((doc) {
Model model = Model.fromDocumet(doc);
if (searchQuery.isNotEmpty &&
!model.property.toLowerCase().contains(searchQuery.toLowerCase())) {
return;
}
searchResults.add(model);
})
};
Following code snippet takes input from user and acquires data starting with the typed one.
Sample Data:
Under Firebase Collection 'Users'
user1: {name: 'Ali', age: 28},
user2: {name: 'Khan', age: 30},
user3: {name: 'Hassan', age: 26},
user4: {name: 'Adil', age: 32}
TextInput: A
Result:
{name: 'Ali', age: 28},
{name: 'Adil', age: 32}
let timer;
// method called onChangeText from TextInput
const textInputSearch = (text) => {
const inputStart = text.trim();
let lastLetterCode = inputStart.charCodeAt(inputStart.length-1);
lastLetterCode++;
const newLastLetter = String.fromCharCode(lastLetterCode);
const inputEnd = inputStart.slice(0,inputStart.length-1) + lastLetterCode;
clearTimeout(timer);
timer = setTimeout(() => {
firestore().collection('Users')
.where('name', '>=', inputStart)
.where('name', '<', inputEnd)
.limit(10)
.get()
.then(querySnapshot => {
const users = [];
querySnapshot.forEach(doc => {
users.push(doc.data());
})
setUsers(users); // Setting Respective State
});
}, 1000);
};
2021 Update
Took a few things from other answers. This one includes:
Multi word search using split (acts as OR)
Multi key search using flat
A bit limited on case-sensitivity, you can solve this by storing duplicate properties in uppercase. Ex: query.toUpperCase() user.last_name_upper
// query: searchable terms as string
let users = await searchResults("Bob Dylan", 'users');
async function searchResults(query = null, collection = 'users', keys = ['last_name', 'first_name', 'email']) {
let querySnapshot = { docs : [] };
try {
if (query) {
let search = async (query)=> {
let queryWords = query.trim().split(' ');
return queryWords.map((queryWord) => keys.map(async (key) =>
await firebase
.firestore()
.collection(collection)
.where(key, '>=', queryWord)
.where(key, '<=', queryWord + '\uf8ff')
.get())).flat();
}
let results = await search(query);
await (await Promise.all(results)).forEach((search) => {
querySnapshot.docs = querySnapshot.docs.concat(search.docs);
});
} else {
// No query
querySnapshot = await firebase
.firestore()
.collection(collection)
// Pagination (optional)
// .orderBy(sortField, sortOrder)
// .startAfter(startAfter)
// .limit(perPage)
.get();
}
} catch(err) {
console.log(err)
}
// Appends id and creates clean Array
const items = [];
querySnapshot.docs.forEach(doc => {
let item = doc.data();
item.id = doc.id;
items.push(item);
});
// Filters duplicates
return items.filter((v, i, a) => a.findIndex(t => (t.id === v.id)) === i);
}
Note: the number of Firebase calls is equivalent to the number of words in the query string * the number of keys you're searching on.
Same as #nicksarno but with a more polished code that doesn't need any extension:
Step 1
func getSubstrings(from string: String, maximumSubstringLenght: Int = .max) -> [Substring] {
let string = string.lowercased()
let stringLength = string.count
let stringStartIndex = string.startIndex
var substrings: [Substring] = []
for lowBound in 0..<stringLength {
for upBound in lowBound..<min(stringLength, lowBound+maximumSubstringLenght) {
let lowIndex = string.index(stringStartIndex, offsetBy: lowBound)
let upIndex = string.index(stringStartIndex, offsetBy: upBound)
substrings.append(string[lowIndex...upIndex])
}
}
return substrings
}
Step 2
let name = "Lorenzo"
ref.setData(["name": name, "nameSubstrings": getSubstrings(from: name)])
Step 3
Firestore.firestore().collection("Users")
.whereField("nameSubstrings", arrayContains: searchText)
.getDocuments...
With Firestore you can implement a full text search but it will still cost more reads than it would have otherwise, and also you'll need to enter and index the data in a particular way, So in this approach you can use firebase cloud functions to tokenise and then hash your input text while choosing a linear hash function h(x) that satisfies the following - if x < y < z then h(x) < h (y) < h(z). For tokenisation you can choose some lightweight NLP Libraries in order to keep the cold start time of your function low that can strip unnecessary words from your sentence. Then you can run a query with less than and greater than operator in Firestore.
While storing your data also, you'll have to make sure that you hash the text before storing it, and store the plain text also as if you change the plain text the hashed value will also change.
Typesense service provide substring search for Firebase Cloud Firestore database.
https://typesense.org/docs/guide/firebase-full-text-search.html
Following is the relevant codes of typesense integration for my project.
lib/utils/typesense.dart
import 'dart:convert';
import 'package:flutter_instagram_clone/model/PostModel.dart';
import 'package:http/http.dart' as http;
class Typesense {
static String baseUrl = 'http://typesense_server_ip:port/';
static String apiKey = 'xxxxxxxx'; // your Typesense API key
static String resource = 'collections/postData/documents/search';
static Future<List<PostModel>> search(String searchKey, int page, {int contentType=-1}) async {
if (searchKey.isEmpty) return [];
List<PostModel> _results = [];
var header = {'X-TYPESENSE-API-KEY': apiKey};
String strSearchKey4Url = searchKey.replaceFirst('#', '%23').replaceAll(' ', '%20');
String url = baseUrl +
resource +
'?q=${strSearchKey4Url}&query_by=postText&page=$page&sort_by=millisecondsTimestamp:desc&num_typos=0';
if(contentType==0)
{
url += "&filter_by=isSelling:false";
} else if(contentType == 1)
{
url += "&filter_by=isSelling:true";
}
var response = await http.get(Uri.parse(url), headers: header);
var data = json.decode(response.body);
for (var item in data['hits']) {
PostModel _post = PostModel.fromTypeSenseJson(item['document']);
if (searchKey.contains('#')) {
if (_post.postText.toLowerCase().contains(searchKey.toLowerCase()))
_results.add(_post);
} else {
_results.add(_post);
}
}
print(_results.length);
return _results;
}
static Future<List<PostModel>> getHubPosts(String searchKey, int page,
{List<String>? authors, bool? isSelling}) async {
List<PostModel> _results = [];
var header = {'X-TYPESENSE-API-KEY': apiKey};
String filter = "";
if (authors != null || isSelling != null) {
filter += "&filter_by=";
if (isSelling != null) {
filter += "isSelling:$isSelling";
if (authors != null && authors.isNotEmpty) {
filter += "&&";
}
}
if (authors != null && authors.isNotEmpty) {
filter += "authorID:$authors";
}
}
String url = baseUrl +
resource +
'?q=${searchKey.replaceFirst('#', '%23')}&query_by=postText&page=$page&sort_by=millisecondsTimestamp:desc&num_typos=0$filter';
var response = await http.get(Uri.parse(url), headers: header);
var data = json.decode(response.body);
for (var item in data['hits']) {
PostModel _post = PostModel.fromTypeSenseJson(item['document']);
_results.add(_post);
}
print(_results.length);
return _results;
}
}
lib/services/hubDetailsService.dart
import 'package:flutter/material.dart';
import 'package:flutter_instagram_clone/model/PostModel.dart';
import 'package:flutter_instagram_clone/utils/typesense.dart';
class HubDetailsService with ChangeNotifier {
String searchKey = '';
List<String>? authors;
bool? isSelling;
int nContentType=-1;
bool isLoading = false;
List<PostModel> hubResults = [];
int _page = 1;
bool isMore = true;
bool noResult = false;
Future initSearch() async {
isLoading = true;
isMore = true;
noResult = false;
hubResults = [];
_page = 1;
List<PostModel> _results = await Typesense.search(searchKey, _page, contentType: nContentType);
for(var item in _results) {
hubResults.add(item);
}
isLoading = false;
if(_results.length < 10) isMore = false;
if(_results.isEmpty) noResult = true;
notifyListeners();
}
Future nextPage() async {
if(!isMore) return;
_page++;
List<PostModel> _results = await Typesense.search(searchKey, _page);
hubResults.addAll(_results);
if(_results.isEmpty) {
isMore = false;
}
notifyListeners();
}
Future refreshPage() async {
isLoading = true;
notifyListeners();
await initSearch();
isLoading = false;
notifyListeners();
}
Future search(String _searchKey) async {
isLoading = true;
notifyListeners();
searchKey = _searchKey;
await initSearch();
isLoading = false;
notifyListeners();
}
}
lib/ui/hub/hubDetailsScreen.dart
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:flutter_instagram_clone/constants.dart';
import 'package:flutter_instagram_clone/main.dart';
import 'package:flutter_instagram_clone/model/MessageData.dart';
import 'package:flutter_instagram_clone/model/SocialReactionModel.dart';
import 'package:flutter_instagram_clone/model/User.dart';
import 'package:flutter_instagram_clone/model/hubModel.dart';
import 'package:flutter_instagram_clone/services/FirebaseHelper.dart';
import 'package:flutter_instagram_clone/services/HubService.dart';
import 'package:flutter_instagram_clone/services/helper.dart';
import 'package:flutter_instagram_clone/services/hubDetailsService.dart';
import 'package:flutter_instagram_clone/ui/fullScreenImageViewer/FullScreenImageViewer.dart';
import 'package:flutter_instagram_clone/ui/home/HomeScreen.dart';
import 'package:flutter_instagram_clone/ui/hub/editHubScreen.dart';
import 'package:provider/provider.dart';
import 'package:smooth_page_indicator/smooth_page_indicator.dart';
class HubDetailsScreen extends StatefulWidget {
final HubModel hub;
HubDetailsScreen(this.hub);
#override
_HubDetailsScreenState createState() => _HubDetailsScreenState();
}
class _HubDetailsScreenState extends State<HubDetailsScreen> {
late HubDetailsService _service;
List<SocialReactionModel?> _reactionsList = [];
final fireStoreUtils = FireStoreUtils();
late Future<List<SocialReactionModel>> _myReactions;
final scrollController = ScrollController();
bool _isSubLoading = false;
#override
void initState() {
// TODO: implement initState
super.initState();
_service = Provider.of<HubDetailsService>(context, listen: false);
print(_service.isLoading);
init();
}
init() async {
_service.searchKey = "";
if(widget.hub.contentWords.length>0)
{
for(var item in widget.hub.contentWords) {
_service.searchKey += item + " ";
}
}
switch(widget.hub.contentType) {
case 'All':
break;
case 'Marketplace':
_service.isSelling = true;
_service.nContentType = 1;
break;
case 'Post Only':
_service.isSelling = false;
_service.nContentType = 0;
break;
case 'Keywords':
break;
}
for(var item in widget.hub.exceptWords) {
if(item == 'Marketplace') {
_service.isSelling = _service.isSelling != null?true:false;
} else {
_service.searchKey += "-" + item + "";
}
}
if(widget.hub.fromUserType == 'Followers') {
List<User> _followers = await fireStoreUtils.getFollowers(MyAppState.currentUser!.userID);
_service.authors = [];
for(var item in _followers)
_service.authors!.add(item.userID);
}
if(widget.hub.fromUserType == 'Selected') {
_service.authors = widget.hub.fromUserIds;
}
_service.initSearch();
_myReactions = fireStoreUtils.getMyReactions()
..then((value) {
_reactionsList.addAll(value);
});
scrollController.addListener(pagination);
}
void pagination(){
if(scrollController.position.pixels ==
scrollController.position.maxScrollExtent) {
_service.nextPage();
}
}
#override
Widget build(BuildContext context) {
Provider.of<HubDetailsService>(context);
PageController _controller = PageController(
initialPage: 0,
);
return Scaffold(
backgroundColor: Colors.white,
body: RefreshIndicator(
onRefresh: () async {
_service.refreshPage();
},
child: CustomScrollView(
controller: scrollController,
slivers: [
SliverAppBar(
centerTitle: false,
expandedHeight: MediaQuery.of(context).size.height * 0.25,
pinned: true,
backgroundColor: Colors.white,
title: Row(
mainAxisAlignment: MainAxisAlignment.spaceBetween,
children: [
InkWell(
onTap: (){
Navigator.pop(context);
},
child: Container(
width: 35, height: 35,
decoration: BoxDecoration(
color: Colors.white,
borderRadius: BorderRadius.circular(20)
),
child: Center(
child: Icon(Icons.arrow_back),
),
),
),
if(widget.hub.user.userID == MyAppState.currentUser!.userID)
InkWell(
onTap: () async {
var _hub = await push(context, EditHubScreen(widget.hub));
if(_hub != null) {
Navigator.pop(context, true);
}
},
child: Container(
width: 35, height: 35,
decoration: BoxDecoration(
color: Colors.white,
borderRadius: BorderRadius.circular(20)
),
child: Center(
child: Icon(Icons.edit, color: Colors.black, size: 20,),
),
),
),
],
),
automaticallyImplyLeading: false,
flexibleSpace: FlexibleSpaceBar(
collapseMode: CollapseMode.pin,
background: Container(color: Colors.grey,
child: Stack(
children: [
PageView.builder(
controller: _controller,
itemCount: widget.hub.medias.length,
itemBuilder: (context, index) {
Url postMedia = widget.hub.medias[index];
return GestureDetector(
onTap: () => push(
context,
FullScreenImageViewer(
imageUrl: postMedia.url)),
child: displayPostImage(postMedia.url));
}),
if (widget.hub.medias.length > 1)
Padding(
padding: const EdgeInsets.only(bottom: 30.0),
child: Align(
alignment: Alignment.bottomCenter,
child: SmoothPageIndicator(
controller: _controller,
count: widget.hub.medias.length,
effect: ScrollingDotsEffect(
dotWidth: 6,
dotHeight: 6,
dotColor: isDarkMode(context)
? Colors.white54
: Colors.black54,
activeDotColor: Color(COLOR_PRIMARY)),
),
),
),
],
),
)
),
),
_service.isLoading?
SliverFillRemaining(
child: Center(
child: CircularProgressIndicator(),
),
):
SliverList(
delegate: SliverChildListDelegate([
if(widget.hub.userId != MyAppState.currentUser!.userID)
_isSubLoading?
Center(
child: Padding(
padding: EdgeInsets.all(5),
child: CircularProgressIndicator(),
),
):
Padding(
padding: EdgeInsets.symmetric(horizontal: 5),
child: widget.hub.shareUserIds.contains(MyAppState.currentUser!.userID)?
ElevatedButton(
onPressed: () async {
setState(() {
_isSubLoading = true;
});
await Provider.of<HubService>(context, listen: false).unsubscribe(widget.hub);
setState(() {
_isSubLoading = false;
widget.hub.shareUserIds.remove(MyAppState.currentUser!.userID);
});
},
style: ElevatedButton.styleFrom(
primary: Colors.red
),
child: Text(
"Unsubscribe",
),
):
ElevatedButton(
onPressed: () async {
setState(() {
_isSubLoading = true;
});
await Provider.of<HubService>(context, listen: false).subscribe(widget.hub);
setState(() {
_isSubLoading = false;
widget.hub.shareUserIds.add(MyAppState.currentUser!.userID);
});
},
style: ElevatedButton.styleFrom(
primary: Colors.green
),
child: Text(
"Subscribe",
),
),
),
Padding(
padding: EdgeInsets.all(15,),
child: Text(
widget.hub.name,
style: TextStyle(
color: Colors.black,
fontSize: 18,
fontWeight: FontWeight.bold
),
),
),
..._service.hubResults.map((e) {
if(e.isAuction && (e.auctionEnded || DateTime.now().isAfter(e.auctionEndTime??DateTime.now()))) {
return Container();
}
return PostWidget(post: e);
}).toList(),
if(_service.noResult)
Padding(
padding: EdgeInsets.all(20),
child: Text(
'No results for this hub',
style: TextStyle(
fontSize: 18,
fontWeight: FontWeight.bold
),
),
),
if(_service.isMore)
Center(
child: Container(
padding: EdgeInsets.all(5),
child: CircularProgressIndicator(),
),
)
]),
)
],
),
)
);
}
}
You can try using 2 lambdas and S3. These resources are very cheap and you will only be charged once the app has extreme usage ( if the business model is good then high usage -> higher income).
The first lambda will be used to push a text-document mapping to an S3 json file.
the second lambda will basically be your search api, you will use it to query the JSON in s3 and return the results.
The drawback will probably be the latency from s3 to lambda.
I use this with Vue js
query(collection(db,'collection'),where("name",">=",'searchTerm'),where("name","<=","~"))
I also couldn't manage to create a search function to Firebase using the suggestions and Firebase tools so I created my own "field-string contains search-string(substring) check", using the .contains() Kotlin function:
firestoreDB.collection("products")
.get().addOnCompleteListener { task->
if (task.isSuccessful){
val document = task.result
if (!document.isEmpty) {
if (document != null) {
for (documents in document) {
var name = documents.getString("name")
var type = documents.getString("type")
if (name != null && type != null) {
if (name.contains(text, ignoreCase = true) || type.contains(text, ignoreCase = true)) {
// do whatever you want with the document
} else {
showNoProductsMsg()
}
}
}
}
binding.progressBarSearch.visibility = View.INVISIBLE
} else {
showNoProductsMsg()
}
} else{
showNoProductsMsg()
}
}
First, you get ALL the documents in the collection you want, then you filter them using:
for (documents in document) {
var name = documents.getString("name")
var type = documents.getString("type")
if (name != null && type != null) {
if (name.contains(text, ignoreCase = true) || type.contains(text, ignoreCase = true)) {
//do whatever you want with this document
} else {
showNoProductsMsg()
}
}
}
In my case, I filtered them all by the name of the product and its type, then I used the boolean name.contains(string, ignoreCase = true) OR type.contains(string, ignoreCase = true, string is the text I got in the search bar of my app and I recommend you to use ignoreCase = true. With this setence being true, you can do whatever you want with the document.
I guess this is the best workaround since Firestore only supports number and exacts strings queries, so if your code didn't work doing this:
collection.whereGreaterThanOrEqualTo("name", querySearch)
collection.whereLessThanOrEqualTo("name", querySearch)
You're welcome :) because what I did works!
Firebase suggests Algolia or ElasticSearch for Full-Text search, but a cheaper alternative might be MongoDB. The cheapest cluster (approx US$10/mth) allows you to index for full-text.
We can use the back-tick to print out the value of a string. This should work:
where('name', '==', `${searchTerm}`)
Related
Say I have this minimal database stored in Cloud Firestore. How could I retrieve the names of subCollection1 and subCollection2?
rootCollection {
aDocument: {
someField: { value: 1 },
anotherField: { value: 2 }
subCollection1: ...,
subCollection2: ...,
}
}
I would expect to be able to just read the ids off of aDocument, but only the fields show up when I get() the document.
rootRef.doc('aDocument').get()
.then(doc =>
// only logs [ "someField", "anotherField" ], no collections
console.log( Object.keys(doc.data()) )
)
It is not currently supported to get a list of (sub)collections from Firestore in the client SDKs (Web, iOS, Android).
In server-side SDKs this functionality does exist. For example, in Node.js you'll be after the ListCollectionIds method:
var firestore = require('firestore.v1beta1');
var client = firestore.v1beta1({
// optional auth parameters.
});
// Iterate over all elements.
var formattedParent = client.anyPathPath("[PROJECT]", "[DATABASE]", "[DOCUMENT]", "[ANY_PATH]");
client.listCollectionIds({parent: formattedParent}).then(function(responses) {
var resources = responses[0];
for (var i = 0; i < resources.length; ++i) {
// doThingsWith(resources[i])
}
})
.catch(function(err) {
console.error(err);
});
It seems like they have added a method called getCollections() to Node.js:
firestore.doc(`/myCollection/myDocument`).getCollections().then(collections => {
for (let collection of collections) {
console.log(`Found collection with id: ${collection.id}`);
}
});
This example prints out all subcollections of the document at /myCollection/myDocument
Isn't this detailed in the documentation?
/**
* Delete a collection, in batches of batchSize. Note that this does
* not recursively delete subcollections of documents in the collection
*/
function deleteCollection(db, collectionRef, batchSize) {
var query = collectionRef.orderBy('__name__').limit(batchSize);
return new Promise(function(resolve, reject) {
deleteQueryBatch(db, query, batchSize, resolve, reject);
});
}
function deleteQueryBatch(db, query, batchSize, resolve, reject) {
query.get()
.then((snapshot) => {
// When there are no documents left, we are done
if (snapshot.size == 0) {
return 0;
}
// Delete documents in a batch
var batch = db.batch();
snapshot.docs.forEach(function(doc) {
batch.delete(doc.ref);
});
return batch.commit().then(function() {
return snapshot.size;
});
}).then(function(numDeleted) {
if (numDeleted <= batchSize) {
resolve();
return;
}
// Recurse on the next process tick, to avoid
// exploding the stack.
process.nextTick(function() {
deleteQueryBatch(db, query, batchSize, resolve, reject);
});
})
.catch(reject);
}
This answer is in the docs
Sadly the docs aren't clear what you import.
Based on the docs, my code ended up looking like this:
import admin, { firestore } from 'firebase-admin'
let collections: string[] = null
const adminRef: firestore.DocumentReference<any> = admin.firestore().doc(path)
const collectionRefs: firestore.CollectionReference[] = await adminRef.listCollections()
collections = collectionRefs.map((collectionRef: firestore.CollectionReference) => collectionRef.id)
This is of course Node.js server side code. As per the docs, this cannot be done on the client.
I am working with Flutter and am currently trying to create a graph. I am looking to parse this JSON Array from the link below. My issue is that the information provided in the "prices" object, the values are all inside arrays themselves. I want to get those values and split them into an X and Y list but I have no idea how to accomplish this. I posted a snippet of the JSON data below.
https://api.coingecko.com/api/v3/coins/bitcoin/market_chartvs_currency=usd&days=1
I am only familiar with parsing data by creating a class and constructor. Then create a fromJSON(Map<String, dynamic> json) class and putting the data into a list, as shown in the code snippet below that I created from another URL with object values. How could I go about parsing this array JSON data into two list data?
CODE TO PARSE JSON
List<Coins> _coins = List<Coins>();
Future<List<Coins>> fetchCoins() async {
var url = 'URL';
var response = await http.get(url);
var coins = List<Coins>();
if (response.statusCode == 200) {
var coinsJSON = json.decode(response.body);
for (var coinJSON in coinsJSON) {
coins.add(Coins.fromJson(coinJSON));
}
}
return coins;
}
#override
void initState() {
fetchCoins().then((value) {
setState(() {
_coins.addAll(value);
});
});
super.initState();
}
class Coins{
String symbol;
String name;
Coins(this.symbol, this.name);
Coins.fromJson(Map<String, dynamic> json) {
symbol = json['symbol'];
name = json['name'];
JSON DATA SNIPPET
{
"prices":[
[
1566344769277,
10758.856131083012
],
[
1566345110646,
10747.91694691537
],
[
1566345345922,
10743.789313302059
],
]
}
EDIT: SOLVED WITH THE HELP OF #EJABU.
class HistoricalData {
List prices;
List<num> listX = [];
List<num> listY = [];
HistoricalData(this.prices,this.listX, this.listY);
HistoricalData.fromJson(Map<String, dynamic> json) {
prices = json['prices'];
for (var price in prices) {
listX.add(price[0]);
listY.add(price[1]);
}
}
You may try this...
New class Coins definition:
class Coins {
List<num> listX = [];
List<num> listY = [];
Coins(this.listX, this.listY);
Coins.fromJson(Map<String, dynamic> json) {
List<List<num>> prices = json['prices'];
for (var price in prices) {
listX.add(price[0]);
listY.add(price[1]);
}
}
}
Then later you can fetch it by these lines :
// Future<List<Coins>> fetchCoins() async { // Remove This
Future<Coins> fetchCoins() async {
var url = 'URL';
var response = await http.get(url);
// var coins = List<Coins>(); // Remove This
Coins coins;
if (response.statusCode == 200) {
var coinsJSON = json.decode(response.body);
// Remove This
// for (var coinJSON in coinsJSON) {
// coins.add(Coins.fromJson(coinJSON));
// }
//
coins = Coins.fromJSON(coinsJSON);
}
return coins;
}
Accessing Data in Widget
In Widgets , our expected variable resides as property inside Coins class.
For example, if you use FutureBuilder, you may use these lines:
child: FutureBuilder(
future: fetchCoins(),
builder: (_, __) {
return SomeChartWidget(
listX: coins.listX,
listY: coins.listY,
);
},
),
Generating Serializers automatically
I suggest you take a look at https://pub.dev/packages/json_serializable, which is a package that does the boilerplate code generation for you. Although it might me a bit overkill to add something like this to your code or your workflow, automatically generating serializers is very convenient.
Not that in order to have custom sub-classes, they need to provide serialization as well.
If you want to extend your knowledge even further, you can also have a look at https://pub.dev/packages/built_value and https://pub.dev/packages/built_value_generator
I know that firestore doesn't support full text search and it giving us solution to use third party services. However I found a simple solution to simple "full text search" and I think this might help others who doesn't want to use third party services as me for such a simple task.
I'm trying to search for company name which is saved in firestore collection under my companyName which can be in any format for example "My Awesome Company". When adding new company with companyName or updating a value in companyName I'm also saving searchName with it which is the same value as company name but in lower case without spaces
searchName: removeSpace(companyName).toLowerCase()
removeSpace is my simple custom function which remove all spaces from a text
export const removeSpace = (string) => {
return string.replace(/\s/g, '');
}
That turns our company name to myawesomecompany which is saved in searchName
Now I've got a firestore function to search for company which indexing through searchName and returning companyName. Minumum search value is a searched value without last character and maximum search value is a searched value with added "zzzzzzzzzzzzzzzzzzzzzzzz" transformed to lower case. That means if you search for My Aw then min value will be mya and max value will be myawzzzzzzzzzzzzzzzzzzzzzzz
exports.handler = ((data) => {
const searchValue = data.value.replace(/\s/g, '').toLowerCase()
const minName = searchValue.substr(0, searchName.length-1)
const maxName = searchValue + "zzzzzzzzzzzzzzzzzzzzzzzz"
let list = []
const newRef = db.collection("user").where("profile.searchName", ">=", minName).where("profile.searchName", "<=", maxName)
return newRef.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
list.push({ name: doc.data().profile.companyName})
})
return list
})
})
I didn't have time to fully test it but so far it works without any problems. Please let me know if you spot anything wrong with it. Now the question is
Is "z" character the highest value character in firestore or is there any other more decent way to add into the search value maximum amount without adding "zzzzzzzzzzzzz"?
I like your decision to preprocess the text so that it can be queried, but you could provide for a more flexible search by storing lowercase keywords with the users and searching those. In other words, transform:
"My Awesome Company"
to...
{ my: true, awesome: true, company: true }
...and test against that.
When adding/updating the property:
// save keywords on the user
let keywords = {}
companyName.split(' ').forEach(word => keywords[word.toLowerCase()] = true)
When querying:
let searchKeywords = userInputString.split(' ').map(word => word.toLowerCase())
let collection = db.collection("user")
searchKeywords.forEach(keyword => {
collection = collection.where(`keywords.${keyword}` , '==' , true);
});
With a little modification of previous answer I have made another simple text search. I'm saving keyword to an array instead of saving it in object like this
nameIndex: textIndexToArray(companyName)
where textIndexToArray is my custom function
export const textIndexToArray = (str) => {
const string = str.trim().replace(/ +(?= )/g,'')
let arr = []
for (let i = 0; i < string.trim().length; i++) {
arr.push(string.substr(0,i+1).toLowerCase());
}
return arr
}
which transfer a text into array. For example
"My Company"
will return
[m, my, my , my c, my co, my com, my comp, my compa, my compan, my company]
with nameIndex saved in firestore we can simply query the data thorough nameIndex and return companyName
exports.handler = ((data) => {
const searchValue = data.value.toLowerCase()
let list = []
const newRef = db.collection("user").where("nameIndex", "array-contains", searchValue)
return newRef.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
list.push({ name: doc.data().companyName, })
})
return list
})
})
I have looked around at a few answers/questions regarding this issue but yet to find a solution.
I have a collection with documents (simplified) as such:
{
"id": 123
"stuff": "abc"
"array":[
{
"id2":456
"properties": [
{
"id3": 789
"important": true
}
]
}
]
}
I want to check for each document in my collection, for each array object within array, for each properties, if it has important: true for example. Then return:
"id": 123
"id2": 456
"id3": 789
I have tried using:
client.queryDocuments(self.collection._self, querySpec).toArray(function(err, results) {
if (err) {
callback(err);
} else {
callback(null, results[0]);
}
});
But the issue is an array has a maximum character limit. If my collection has millions of documents, this would presumably be exceeded. (Javascript Increase max array size)
Or, am I misunderstanding the above question? Is it talking about the number of objects in an array (of which, each can have unlimited object character length?)
Thus I am looking a for loop-esque solution, where each document is returned, I do my analysis, then move to then next/do them in parallel.
Any insight would be greatly appreciated.
But the issue is an array has a maximum character limit. If my
collection has millions of documents, this would presumably be
exceeded. (Javascript Increase max array size)
Based on my research,the longest possible array in js could have 232-1 = 4,294,967,295 = 4.29 billion elements. However, it is perfectly enough to meet your millions data volume requirements. In addition,you can't query such huge volume data directly surely,that's impossible you do that.
Whether about throughput constraints(RUs settings) or query efficiency factors, you should consider batching large volumes of data anyway.
Thus I am looking a for loop-esque solution, where each document is
returned, I do my analysis, then move to then next/do them in
parallel.
Maybe you could use v2 js sdk for cosmos db sql api.Please refer to the sample code:
const cosmos = require('#azure/cosmos');
const CosmosClient = cosmos.CosmosClient;
const endpoint = "https://***.documents.azure.com:443/"; // Add your endpoint
const masterKey = "***"; // Add the masterkey of the endpoint
const client = new CosmosClient({ endpoint, auth: { masterKey } });
const databaseId = "db";
const containerId = "coll";
async function run() {
const { container, database } = await init();
const querySpec = {
query: "SELECT r.id,r._ts FROM root r"
};
const queryOptions = {
maxItemCount : -1
}
const queryIterator = await container.items.query(querySpec,queryOptions);
while (queryIterator.hasMoreResults()) {
const { result: results, headers } = await queryIterator.executeNext();
console.log(results)
console.log(headers)
//do what you want to do
if (results === undefined) {
// no more results
break;
}
}
}
async function init() {
const { database } = await client.databases.createIfNotExists({ id: databaseId });
const { container } = await database.containers.createIfNotExists({ id: containerId });
return { database, container };
}
run().catch(err => {
console.error(err);
});
More details about continuation token ,please refer to my previous case.Any concern,please let me know.
I am using Cosmos DB SQL API Node.js library. I am unable to find the Continuation Token from this library so that I can return it to client. The idea is to get it back from the client for the next pagination request.
I have a working code which iterates multiple times to get all the documents. What changes will be required here to get the continuation token?
function queryCollectionPaging() {
return new Promise((resolve, reject) => {
function executeNextWithRetry(iterator, callback) {
iterator.executeNext(function (err, results, responseHeaders) {
if (err) {
return callback(err, null);
}
else {
documents = documents.concat(results);
if (iterator.hasMoreResults()) {
executeNextWithRetry(iterator, callback);
}
else {
callback();
}
}
});
}
let options = {
maxItemCount: 1,
enableCrossPartitionQuery: true
};
let documents = []
let iterator = client.queryDocuments( collectionUrl, 'SELECT r.partitionkey, r.documentid, r._ts FROM root r WHERE r.partitionkey in ("user1", "user2") ORDER BY r._ts', options);
executeNextWithRetry(iterator, function (err, result) {
if (err) {
reject(err)
}
else {
console.log(documents);
resolve(documents)
}
});
});
};
I have the following mutation:
export default class AddTaskMutation extends Relay.Mutation {
static fragments = {
classroom: () => Relay.QL`
fragment on Classroom {
id,
}`,
};
getMutation() {
return Relay.QL`mutation { addTask }`;
}
getFatQuery() {
return Relay.QL`
fragment on AddTaskPayload {
classroom {
taskList,
},
taskEdge,
}
`;
}
getConfigs() {
let rangeBehaviors = {};
let member_id = 'hasNewMessages(true) member_id(' + Global.fromGlobalId(this.props.createdByMember)['id'] + ')';
rangeBehaviors[''] = 'append';
rangeBehaviors['hasToDo(true)'] = 'append';
rangeBehaviors['hasToDo(false)'] = 'append';
rangeBehaviors['isStart(true)'] = 'ignore';
rangeBehaviors['isStart(false)'] = 'append';
rangeBehaviors[`${member_id}`] = 'append';
rangeBehaviors['hasNewMessages(true) member_id(null)'] = 'ignore';
return [
{
type: 'RANGE_ADD',
parentName: 'classroom',
parentID: this.props.classroomId,
connectionName: 'taskList',
edgeName: 'taskEdge',
rangeBehaviors,
}
];
}
getVariables() {
return {
title: this.props.title,
instruction: this.props.instruction,
start_date: this.props.start_date,
end_date: this.props.end_date,
is_archived: this.props.is_archived,
is_published: this.props.is_published,
is_preview: this.props.is_preview,
productId: this.props.productId,
classroomId: this.props.classroomId,
createdByMember: this.props.createdByMember,
subTasks: this.props.subTasks,
students: this.props.students,
};
}
}
When running my application I get the following 2 warnings:
warning.js:44 Warning: RelayMutation: The connection taskList{hasNewMessages:true,member_id:null} on the mutation field classroom that corresponds to the ID Classroom:35 did not match any of the rangeBehaviors specified in your RANGE_ADD config. This means that the entire connection will be refetched. Configure a range behavior for this mutation in order to fetch only the new edge and to enable optimistic mutations or use refetch to squelch this warning.
and
warning.js:44 Warning: Using null as a rangeBehavior value is deprecated. Use ignore to avoid refetching a range.
Since the other rangeBehaviors work, I assume there must be a syntactical error when declaring 2 variables in one behavior - in this case hasNewMessages and memeber_id.
I've looked for an answer for this, but I just cannot find any. The docs don't seem to cover this edge case either.
EDIT: I also tried rangeBehaviors['hasNewMessages(true),member_id(null)'] = 'ignore'; (comma as a separator) but with no success.
After inspecting the source code of Relay (file RelayMutationQuery.js) I could see what array key it was searching for in the array of all rangeBehaviors. I could then update my code to the correct formatting.
Since I haven't found anything on the web about this edge case, I'll post my solution here - perhaps someone will find it helpful in the future.
When having 2 (or more) variables for a rangeBehavior, you need to separate them with a period (.). Also, when passing null you don't pass it explicitly - just omit it from its' variable.
For example:
Right:
rangeBehaviors['hasNewMessages(true).member_id()'] = 'ignore';
Wrong:
rangeBehaviors['hasNewMessages(true),member_id(null)'] = 'ignore';