Perform Asynchronous Decorations in DraftJS? - reactjs

I'm trying to perform real-time Named Entity Recognition highlighting in a WYSIWYG editor, which requires me to make a request to my back-end in between each keystroke.
After spending about a week on ProseMirror I gave up on it and decided to try DraftJS. I have searched the repository and docs and haven't found any asynchronous examples using Decorations. (There are some examples with Entities, but they seem like a bad fit for my problem.)
Here is the stripped down Codepen of what I'd like to solve.
It boils down to me wanting to do something like this:
const handleStrategy = (contentBlock, callback, contentState) => {
const text = contentBlock.getText();
let matchArr, start;
while ((matchArr = properNouns.exec(text)) !== null) {
start = matchArr.index;
setTimeout(() => {
// THROWS ERROR: Cannot read property '0' of null
callback(start, start + matchArr[0].length);
}, 200) // to simulate API request
}
};
I expected it to asynchronously call the callback once the timeout resolved but instead matchArr is empty, which just confuses me.
Any help is appreciated!

ok, one possible solution, a example, simple version (may not be 100% solid) :
write a function take editor's string, send it to server, and resolve the data get from server, you need to figure out send the whole editor string or just one word
getServerResult = data => new Promise((resolve, reject) => {
...
fetch(link, {
method: 'POST',
headers: {
...
},
// figure what to send here
body: this.state.editorState.getCurrentContent().getPlainText(),
})
.then(res => resolve(res))
.catch(reject);
});
determine when to call the getServerResult function(i.e when to send string to server and get entity data), from what I understand from your comment, when user hit spacebar key, send the word before to server, this can done by draftjs Key Bindings or react SyntheticEvent. You will need to handle case what if user hit spacebar many times continuously.
function myKeyBindingFn(e: SyntheticKeyboardEvent): string {
if (e.keyCode === 32) {
return 'send-server';
}
return getDefaultKeyBinding(e);
}
async handleKeyCommand(command: string): DraftHandleValue {
if (command === 'send-server') {
// you need to manually add a space char to the editorState
// and get result from server
...
// entity data get from server
const result = await getServerResult()
return 'handled';
}
return 'not-handled';
}
add entity data get from server to specific word using ContentState.createEntity()
async handleKeyCommand(command: string): DraftHandleValue {
if (command === 'send-server') {
// you need to manually add a space char to the editorState
// and get result from server
...
// entity data get from server
const result = await getServerResult()
const newContentState = ContentState.createEntity(
type: 'string',
mutability: ...
data: result
)
const entityKey = contentStateWithEntity.getLastCreatedEntityKey();
// you need to figure out the selectionState, selectionState mean add
// the entity data to where
const contentStateWithEntity = Modifier.applyEntity(
newContentState,
selectionState,
entityKey
);
// create a new EditorState and use this.setState()
const newEditorState = EditorState.push(
...
contentState: contentStateWithEntity
)
this.setState({
editorState: newEditorState
})
return 'handled';
}
return 'not-handled';
}
create different decorators find words with specific entity data, and return different style or whatever you need to return
...
const compositeDecorator = new CompositeDecorator([
strategy: findSubjStrategy,
component: HandleSubjSpan,
])
function findSubjStrategy(contentBlock, callback, contentState) {
// search whole editor content find words with subj entity data
// if the word's entity data === 'Subj'
// pass the start index & end index of the word to callback
...
if(...) {
...
callback(startIndex, endIndex);
}
}
// this function handle what if findSubjStrategy() find any word with subj
// entity data
const HandleSubjSpan = (props) => {
// if the word with subj entity data, it font color become red
return <span {...props} style={{ color: 'red' }}>{props.children}</span>;
};

Related

SvelteKit form get request

I am trying to set up a simple endpoint in SvelteKit that reads the input given in a form and returns the result of an SQL query. In my first attempt I used the form actions and wrote the following code in the +page.server.js file:
export const actions = {
default: async (event) => {
let form_input = await event.request.formData();
let query = {
text: "select * from ux_return_shipments($1, $2, $3, $4)",
values: [form_input.get('beg-dep-date') || null, form_input.get('end-dep-date') || null, form_input.get('beg-arr-date') || null, form_input.get('end-arr-date') || null
}
try {
const result = await event.locals.pool.query(query);
return result.rows;
} catch (e) {
console.log(e);
}
}
};
I am now trying to set up the same process using a GET request instead of a POST one but I am having difficulties setting up the endpoint. I tried to replace the above code with this template but it looks like the endpoint is not getting activated since I see no activity server side:
export function GET({ url }) {
console.log(url);
return new Response("Test response");
};
What am I doing wrong? I see that using this code for an API endpoint (+server.js file) works correctly. I also checked the form element and the URL looks correct.
In case someone has the same problem I managed to solve it using this template:
export const load = async (event) => {
return ...
};
Using the load function I was able to set up a get endpoint and pass some data to the frontend using a return.

how to turn off buffering in react-player?

when I try to play the next video it does not start and I guess the problem is buffering.
P.S my url is video.m3u8 files
It works fine, but when i change url nothing happens, i would like to know how can i stop current video and load a new one whe, url changes ?
here's my rewind function
const showVideo = async () => {
sessionStorage.setItem("sPlayerLinkId", params.id);
const body = new FormData();
const mac = window.TvipStb.getMainMacAddress();
body.append("link_id", params.id);
body.append("mac", mac);
let response = await fetch(getVideo, {
method: "POST",
body: body,
});
let data = await response.json();
if (data.error) {
openDialog("crush");
return 0;
}
if (_isMounted.current) setVideoLink(data.response.url); };
var goToNext = function () {
playerRef.current.seekTo(0, "seconds");
setVideoLink(null);
if (playerInfo.next_id) {
params.id = playerInfo.next_id;
showVideo();
} else navigate(-1);};
<ReactPlayer
url={videoLink}
playing={isPlaying}
ref={playerRef}
key={params.id}
onProgress={() => {
current();
}}
config={{
file: {
forceHLS: true,
},
}}
/>
I would suggest you build your own player from scratch using just react and a style library.
I had similar issues using react-player and I had to resort to building my own custom player in which I could now ensure that buffering is handled the way I expect it to.
I handled buffering using the progress event as follows
const onProgress = () => {
if (!element.buffered) return;
const bufferedEnd = element.buffered.end(element.buffered.length - 1);
const duration = element.duration;
if (bufferRef && duration > 0) {
bufferRef.current!.style.width = (bufferedEnd / duration) * 100 + "%";
}
};
element.buffered represents a collection of buffered time ranges.
element.buffered.end(element.buffered.length - 1) gets the time at the end of the buffer range. With this value, I was able to compute the current buffer range and update the buffer progress accordingly.
I ended up writing an article that would help others learn to build an easily customizable player from scratch using just React and any style library (in this case charkra UI was used).

Querying persisted React WYSIWYG data from MongoDB

thanks in advance!
In summary, I am using React's WYSIWYG rich text editor, and saving the text written in the editor to a MongoDB, data is sent to a server which does the insertion. My issue is that I am unable, after following recommended code, to retrieve the stored data back successfully to display it on my page. This is for a prospective blog post site.
Below I've provided all relevant code:
My Component which sends the data to the server to insert it into MongoDB, (not in order, only relevant code):
<Editor
editorState={editorState}
onEditorStateChange={handleEditorChange}
wrapperClassName="wrapper-class"
editorClassName="editor-class"
toolbarClassName="toolbar-class"
/>
const Practice = () => {
const [editorState, setEditorState] = useState(
() => EditorState.createEmpty(),
);
const [convertedContent, setConvertedContent] = useState(null);
const handleEditorChange = (state) => {
setEditorState(state);
convertContentToRaw();
}
const convertContentToRaw = () => {
const contentState = editorState.getCurrentContent();
setEditorState(editorState: {convertToRaw(contentState)});
}
const stateToSend = JSON.stringify(editorState);
try {
const response = await axios.post('http://localhost:8080/api/insert', {
content: stateToSend
})
} catch(error) {
}
In MongoDB, I've initialized 1 column for storing the WYSIWYG data, I've initialized as an empty JS object:
const wysiwygtest = new mongoose.Schema({
content: {
type: {}
}
});
As a result, my data is inserted into MongoDB as such, with everything desired clearly in data type such as RGBA etc. correct me if I'm wrong but I believe Mongo uses BSON, a form of binary based JSON, so this looks doable for retrieval:
Lastly, the code which is not working correctly, the retrieval. For this, I have no interest just yet in placing the data back into the text editor. Rather, I'd like to display it on the page like a typical blog post. However, I'm unable to even log to the console as of yet.
I am parsing the data back to JSON using JSON.parse, converting JSON to JS object using createFromRaw and using EdiorState (even though I don't have the text editor in this component but this seems to be needed to convert the data fully..) to convert fully:
useEffect( async () => {
try {
const response = await axios.get('http://localhost:8080/api/query', {
_id: '60da9673b996f54d507dbfc5'
});
const content = response;
if(content) {
const convertedContent =
EditorState.createWithContent(convertFromRaw(JSON.parse(content)));
console.log('convertedContent - ', convertedContent);
}
console.log('response - ', content);
} catch(error) {
console.log('error!', error);
}
}, [])
My result for the past day and last night has been the following:
"SyntaxError: Unexpected token o in JSON at position 1" and so I'm unsure what I'm doing wrong in the data retrieval, and possibly even the insertion.
Any ideas? Thanks again!
Edit: For more reference, here is what the data looks like when output to the console without a JSON.stringify, this is the full tree of data. I can see all of the relevant data is there, but how do I convert this data and display it into a div or paragraph tag, for example?
More or less figured this out, see my solution below given the aforementioned implementation:
Firstly, I think my biggest mistake was using JSON.parse(); I did away with this with success. My guess as to why this does not work (even though I inserted into MongoDB as JSON) is because we ultimately need the draft-js.Editor Object to convert the data from the DB into an object type it can understand, in order to subsequently convert into HTML successfully, with all properties.
Below is the code with captions/descriptions:
Retrieve data (in useEffect before React component is rendered:
useEffect( async () => {
console.log('useeffect');
try {
const response = await axios.get('http://localhost:8080/api/query', {
_id: '60da9673b996f54d507dbfc5' //hard-coded id from DB for testing
});
const content = response.data; //get JSON data from MongoDB
if(content) {
const rawContent = convertFromRaw(content); //convert from JSON to contentstate understood by DraftJS, for EditorState obj to use
setEditorState(EditorState.createWithContent(rawContent)); //create EditorState based on JSON data from DB and set into component state
let currentContentAsHTML = draftToHtml(convertToRaw(editorState.getCurrentContent())); //create object which converts contentstate understood by DraftJS into a regular vanilla JS object, then take THAT and convert into HTML with "draftToHtml" function. Save that into our 2nd state titled "convertedContent" to be displayed on page for blog post
setConvertedContent(currentContentAsHTML);
}
} catch(error) {
console.log('error retrieving!', error);
} },[convertedContent]) //ensure dependency with with convertedContent state, DB/server calls take time...
In component render, return HTML which sets the innerHTML in the DOM using/passing the convertedContent state which we converted to proper HTML format in step 1.
return (
<div className="blog-container" dangerouslySetInnerHTML={createMarkup(convertedContent)}></div>
</div>
);
In step 2, we called a function entitled, "createMarkup"; here is that method. It essentially returns HTML object using the HTML converted data originally from our database. This is a bit vulnerable it terms of malicious users being able to intercept that HTML in the DOM, however, so we use a method, "purify" from "DOMPurify" class from 'isomorphic-dompurify" library. I'm using this instead of regular DOMPurify because I am using Next JS and NEXT runs on the server side as well, and DOMPurify only expects client side:
const createMarkup = (html) => {
return {
__html: DOMPurify.sanitize(html)
}
}

How to get a count of number of documents in a collection with Cloud Firestore [duplicate]

This question already has answers here:
Cloud Firestore collection count
(29 answers)
Closed 10 months ago.
In Firestore, how can I get the total number of documents in a collection?
For instance if I have
/people
/123456
/name - 'John'
/456789
/name - 'Jane'
I want to query how many people I have and get 2.
I could do a query on /people and then get the length of the returned results, but that seems a waste, especially because I will be doing this on larger datasets.
You currently have 3 options:
Option 1: Client side
This is basically the approach you mentioned. Select all from collection and count on the client side. This works well enough for small datasets but obviously doesn't work if the dataset is larger.
Option 2: Write-time best-effort
With this approach, you can use Cloud Functions to update a counter for each addition and deletion from the collection.
This works well for any dataset size, as long as additions/deletions only occur at the rate less than or equal to 1 per second. This gives you a single document to read to give you the almost current count immediately.
If need need to exceed 1 per second, you need to implement distributed counters per our documentation.
Option 3: Write-time exact
Rather than using Cloud Functions, in your client you can update the counter at the same time as you add or delete a document. This means the counter will also be current, but you'll need to make sure to include this logic anywhere you add or delete documents.
Like option 2, you'll need to implement distributed counters if you want to exceed per second
Aggregations are the way to go (firebase functions looks like the recommended way to update these aggregations as client side exposes info to the user you may not want exposed) https://firebase.google.com/docs/firestore/solutions/aggregation
Another way (NOT recommended) which is not good for large lists and involves downloading the whole list: res.size like this example:
db.collection("logs")
.get()
.then((res) => console.log(res.size));
If you use AngulareFire2, you can do (assuming private afs: AngularFirestore is injected in your constructor):
this.afs.collection(myCollection).valueChanges().subscribe( values => console.log(values.length));
Here, values is an array of all items in myCollection. You don't need metadata so you can use valueChanges() method directly.
Be careful counting number of documents for large collections with a cloud function. It is a little bit complex with firestore database if you want to have a precalculated counter for every collection.
Code like this doesn't work in this case:
export const customerCounterListener =
functions.firestore.document('customers/{customerId}')
.onWrite((change, context) => {
// on create
if (!change.before.exists && change.after.exists) {
return firestore
.collection('metadatas')
.doc('customers')
.get()
.then(docSnap =>
docSnap.ref.set({
count: docSnap.data().count + 1
}))
// on delete
} else if (change.before.exists && !change.after.exists) {
return firestore
.collection('metadatas')
.doc('customers')
.get()
.then(docSnap =>
docSnap.ref.set({
count: docSnap.data().count - 1
}))
}
return null;
});
The reason is because every cloud firestore trigger has to be idempotent, as firestore documentation say: https://firebase.google.com/docs/functions/firestore-events#limitations_and_guarantees
Solution
So, in order to prevent multiple executions of your code, you need to manage with events and transactions. This is my particular way to handle large collection counters:
const executeOnce = (change, context, task) => {
const eventRef = firestore.collection('events').doc(context.eventId);
return firestore.runTransaction(t =>
t
.get(eventRef)
.then(docSnap => (docSnap.exists ? null : task(t)))
.then(() => t.set(eventRef, { processed: true }))
);
};
const documentCounter = collectionName => (change, context) =>
executeOnce(change, context, t => {
// on create
if (!change.before.exists && change.after.exists) {
return t
.get(firestore.collection('metadatas')
.doc(collectionName))
.then(docSnap =>
t.set(docSnap.ref, {
count: ((docSnap.data() && docSnap.data().count) || 0) + 1
}));
// on delete
} else if (change.before.exists && !change.after.exists) {
return t
.get(firestore.collection('metadatas')
.doc(collectionName))
.then(docSnap =>
t.set(docSnap.ref, {
count: docSnap.data().count - 1
}));
}
return null;
});
Use cases here:
/**
* Count documents in articles collection.
*/
exports.articlesCounter = functions.firestore
.document('articles/{id}')
.onWrite(documentCounter('articles'));
/**
* Count documents in customers collection.
*/
exports.customersCounter = functions.firestore
.document('customers/{id}')
.onWrite(documentCounter('customers'));
As you can see, the key to prevent multiple execution is the property called eventId in the context object. If the function has been handled many times for the same event, the event id will be the same in all cases. Unfortunately, you must have "events" collection in your database.
Please check below answer I found on another thread. Your count should be atomic. Its required to use FieldValue.increment() function in such case.
https://stackoverflow.com/a/49407570/3337028
firebase-admin offers select(fields) which allows you to only fetch specific fields for documents within your collection. Using select is more performant than fetching all fields. However, it is only available for firebase-admin and firebase-admin is typically only used server side.
select can be used as follows:
select('age', 'name') // fetch the age and name fields
select() // select no fields, which is perfect if you just want a count
select is available for Node.js servers but I am not sure about other languages:
https://googleapis.dev/nodejs/firestore/latest/Query.html#select
https://googleapis.dev/nodejs/firestore/latest/CollectionReference.html#select
Here's a server side cloud function written in Node.js which uses select to count a filtered collection and to get the IDs of all resulting documents. Its written in TS but easily converted to JS.
import admin from 'firebase-admin'
// https://stackoverflow.com/questions/46554091/cloud-firestore-collection-count
// we need to use admin SDK here as select() is only available for admin
export const videoIds = async (req: any): Promise<any> => {
const id: string = req.query.id || null
const group: string = req.query.group || null
let processed: boolean = null
if (req.query.processed === 'true') processed = true
if (req.query.processed === 'false') processed = false
let q: admin.firestore.Query<admin.firestore.DocumentData> = admin.firestore().collection('videos')
if (group != null) q = q.where('group', '==', group)
if (processed != null) q = q.where('flowPlayerProcessed', '==', processed)
// select restricts returned fields such as ... select('id', 'name')
const query: admin.firestore.QuerySnapshot<admin.firestore.DocumentData> = await q.orderBy('timeCreated').select().get()
const ids: string[] = query.docs.map((doc: admin.firestore.QueryDocumentSnapshot<admin.firestore.DocumentData>) => doc.id) // ({ id: doc.id, ...doc.data() })
return {
id,
group,
processed,
idx: id == null ? null : ids.indexOf(id),
count: ids.length,
ids
}
}
The cloud function HTTP request completes within 1 second for a collection of 500 docs where each doc contains a lot of data. Not amazingly performant but much better than not using select. Performance could be improved by introducing client side caching (or even server side caching).
The cloud function entry point looks like this:
exports.videoIds = functions.https.onRequest(async (req, res) => {
const response: any = await videoIds(req)
res.json(response)
})
The HTTP request URL would be:
https://SERVER/videoIds?group=my-group&processed=true
Firebase functions detail where the server is located on deployment.
Following Dan Answer: You can have a separated counter in your database and use Cloud Functions to maintain it. (Write-time best-effort)
// Example of performing an increment when item is added
module.exports.incrementIncomesCounter = collectionRef.onCreate(event => {
const counterRef = event.data.ref.firestore.doc('counters/incomes')
counterRef.get()
.then(documentSnapshot => {
const currentCount = documentSnapshot.exists ? documentSnapshot.data().count : 0
counterRef.set({
count: Number(currentCount) + 1
})
.then(() => {
console.log('counter has increased!')
})
})
})
This code shows you the complete example of how to do it:
https://gist.github.com/saintplay/3f965e0aea933a1129cc2c9a823e74d7
Get a new write batch
WriteBatch batch = db.batch();
Add a New Value to Collection "NYC"
DocumentReference nycRef = db.collection("cities").document();
batch.set(nycRef, new City());
Maintain a Document with Id as Count and initial Value as total=0
During Add Operation perform like below
DocumentReference countRef= db.collection("cities").document("count");
batch.update(countRef, "total", FieldValue.increment(1));
During Delete Operation perform like below
DocumentReference countRef= db.collection("cities").document("count");
batch.update(countRef, "total", FieldValue.increment(-1));
Always get Document count from
DocumentReference nycRef = db.collection("cities").document("count");
I created an NPM package to handle all counters:
First install the module in your functions directory:
npm i adv-firestore-functions
then use it like so:
import { eventExists, colCounter } from 'adv-firestore-functions';
functions.firestore
.document('posts/{docId}')
.onWrite(async (change: any, context: any) => {
// don't run if repeated function
if (await eventExists(context)) {
return null;
}
await colCounter(change, context);
}
It handles events, and everything else.
If you want to make it a universal counter for all functions:
import { eventExists, colCounter } from 'adv-firestore-functions';
functions.firestore
.document('{colId}/{docId}')
.onWrite(async (change: any, context: any) => {
const colId = context.params.colId;
// don't run if repeated function
if (await eventExists(context) || colId.startsWith('_')) {
return null;
}
await colCounter(change, context);
}
And don't forget your rules:
match /_counters/{document} {
allow read;
allow write: if false;
}
And of course access it this way:
const collectionPath = 'path/to/collection';
const colSnap = await db.doc('_counters/' + collectionPath).get();
const count = colSnap.get('count');
Read more: https://code.build/p/9DicAmrnRoK4uk62Hw1bEV/firestore-counters
GitHub: https://github.com/jdgamble555/adv-firestore-functions
Use Transaction to update the count inside the success listener of your database write.
FirebaseFirestore.getInstance().runTransaction(new Transaction.Function<Long>() {
#Nullable
#Override
public Long apply(#NonNull Transaction transaction) throws FirebaseFirestoreException {
DocumentSnapshot snapshot = transaction
.get(pRefs.postRef(forumHelper.getPost_id()));
long newCount;
if (b) {
newCount = snapshot.getLong(kMap.like_count) + 1;
} else {
newCount = snapshot.getLong(kMap.like_count) - 1;
}
transaction.update(pRefs.postRef(forumHelper.getPost_id()),
kMap.like_count, newCount);
return newCount;
}
});

Caching in React

In my react App I have a input element. The search query should be memoized, which means that if the user has previously searched for 'John' and the API has provided me valid results for that query, then next time when the user types 'Joh', there should be suggestion for the user with the previously memoized values(in this case 'John' would be suggested).
I am new to react and am trying caching for the first time.I read a few articles but couldn't implement the desired functionality.
You don't clarify which API you're using nor which stack; the solution would vary somewhat depending on if you are using XHR requests or something over GraphQL.
For an asynchronous XHR request to some backend API, I would do something like the example below.
Query the API for the search term
_queryUserXHR = (searchTxt) => {
jQuery.ajax({
type: "GET",
url: url,
data: searchTxt,
success: (data) => {
this.setState({previousQueries: this.state.previousQueries.concat([searchTxt])
}
});
}
You would run this function whenever you want to do the check against your API. If the API can find the search string you query, then insert that data into a local state array variable (previousQueries in my example).
You can either return the data to be inserted from the database if there are unknowns to your view (e.g database id). Above I just insert the searchTxt which is what we send in to the function based on what the user typed in the input-field. The choice is yours here.
Get suggestions for previously searched terms
I would start by adding an input field that runs a function on the onKeyPress event:
<input type="text" onKeyPress={this._getSuggestions} />
then the function would be something like:
_getSuggestions = (e) => {
let inputValue = e.target.value;
let {previousQueries} = this.state;
let results = [];
previousQueries.forEach((q) => {
if (q.toString().indexOf(inputValue)>-1) {
result.push(a);
}
}
this.setState({suggestions: results});
}
Then you can output this.state.suggestions somewhere and add behavior there. Perhaps some keyboard navigation or something. There are many different ways to implement how the results are displayed and how you would select one.
Note: I haven't tested the code above
I guess you have somewhere a function that queries the server, such as
const queryServer = function(queryString) {
/* access the server */
}
The trick would be to memorize this core function only, so that your UI thinks its actually accessing the server.
In javascript it is very easy to implement your own memorization decorator, but you could use existing ones. For example, lru-memoize looks popular on npm. You use it this way:
const memoize = require('lru-memoize')
const queryServer_memoized = memoize(100)(queryServer)
This code keeps in memory the last 100 request results. Next, in your code, you call queryServer_memoized instead of queryServer.
You can create a memoization function:
const memo = (callback) => {
// We will save the key-value pairs in the following variable. It will be our cache storage
const cache = new Map();
return (...args) => {
// The key will be used to identify the different arguments combination. Same arguments means same key
const key = JSON.stringify(args);
// If the cache storage has the key we are looking for, return the previously stored value
if (cache.has(key)) return cache.get(key);
// If the key is new, call the function (in this case fetch)
const value = callback(...args);
// And save the new key-value pair to the cache
cache.set(key, value);
return value;
};
};
const memoizedFetch = memo(fetch);
This memo function will act like a key-value cache. If the params (in our case the URL) of the function (fetch) are the same, the function will not be executed. Instead, the previous result will be returned.
So you can just use this memoized version memoizedFetch in your useEffect to make sure network request are not repeated for that particular petition.
For example you can do:
// Place this outside your react element
const memoizedFetchJson = memo((...args) => fetch(...args).then(res => res.json()));
useEffect(() => {
memoizedFetchJson(`https://pokeapi.co/api/v2/pokemon/${pokemon}/`)
.then(response => {
setPokemonData(response);
})
.catch(error => {
console.error(error);
});
}, [pokemon]);
Demo integrated in React

Resources