I have a text editor built on Tiptap that sends its content as JSON to a API which saves it to a mongoDB database. I have a timeout on it so it only sends a request after the user has stopped typing for 1000ms.
Currently it sends the entire document in the request body making it very taxing performance-wise so I'm trying to figure out a way to identify the parts that have been changed, and only send the parts that have been changed in my request body.
Because the content is being saved in JSON I've been trying to find the index of the node so I can update said node, but I haven't been able to find a clear cut way of doing it. I've also been looking into Steps from the prosemirror docs but I can't figure out a way of implementing steps into the JSON content saved to the DB
I've tried assigning uuids to each node as an attribute, to later iterate through the JSON saved in the database but it seems highly inefficient and I'd really love some input as to how:
this could be achieved
OR in the case that my approach is entirely out of convention, how content is usually saved to a database
if you want save your content real time you have to do more efficient that i will say some solutions for this:
create a de-bouncer to save your content in data base after while user stop typing or create save button.
use UDP instead of TCP it's faster but its not safe then maybe u will lost some data
use RDB instead of Mongo like Redis to store data faster if you have prob in saving data then you can save you data after a while in you main database (mongoDB)
in order to get real time chat and and real time saving data base use web-socket instead of sending request and waiting for handshake with this approach you can show the text data with web socket in client side and also send data into Redis to save temporary in RDB then save in Mongo db for permanently store.
for more performance and more safety is better to combine all these struct er together.
these solution are my personal experience and i do it before.
I'm new to Tiptap and ProseMirror but :
You might want to look at Tiptap's Collaborative editing.
You could intercept the transactions beeing sent to the socket server (It should only contain the updated part, I guess its only sends Prose Mirror transaction). And I guess that, since the socket server is able to send full document to new connected user, It is always able to reconstruct the full document. So you potentially could replace your system, with hocuspocus.
PS: it is beeing developped right now, so you have to pay a little to have access to it. But it could save you time.
Related
I am trying to save data as it arrives in a streaming fashion (with the least amount of delay) to my database which is InfluxDB. Currently I save it in batches.
Current setup - interval based
Currently I have an Airflow instance where I read the data from a REST API every 5min and then save it to the InfluxDB.
Desired setup - continuous
Instead of saving data every 5 min, I would like to establish a connection via a Web-socket (I guess) and save the data as it arrives. I have never done this before and I am confusing how actually it is done? Some question I have are:
One I write the code for it, do I keep it up like a daemon?
Do I need to use something like Telegraf for this or that's not really the case (example article)
Instead of Airflow (since it is for batch processing) do I need to use something like Apache Beam or Spark?
As you can see, I am quite lost on where to start, what to read and how to make sense from all this. Any advise on direction and/or guidance for a set-up would be very appreciated.
If I understand correctly, you are keen to code a java service which would process the incoming data, so one of the solution is to implement a websocket with for example jetty.
From there you receive the data in json format for example and you process the data using the influxdb-java framework with which you fill the database. Influxdb-java will allow you to create and manage the data.
I don't know airflow, and how you produce the data, so maybe there is built-in tools (influxdb sinks) that can save you some work in your context.
I hope that this can give you some guide lines to start digging more.
I need help to hide/disable local-storage data from end-user.
In my application i set data into local-storage but i need to hide those data to see anyone by debugging using Chrome/IE/Firefox.
window.localStorage.setItem('testValue', 'JSONarrayData');
You cannot have data on users machine that they cannot see. The only option is that you can encrypt it, but then you cannot read this data on client (since there is no key there or no point in encrypting it if key is present). Then you will need to decrypt this data on the server and if you have a call to do that, then it's almost the same as having it available.
So the question is: why do you really want to hide some data from a user? I think you start thinking about it in other direction. E.g. if your goal is to not allow user manipulation of data, then you can use signing.
Hi Im a newbie in stackoverflow!
As mentioned on the question title, I've been storing the email's image path into the db via localhost. Once the email is sent and received, my outlook automatically block the image download and I will need to manually download it (Not a big issue here).
Then I started to wonder what if my website/server is down? If it is down, the email will not be able to locate and download the image at all. So I'm wondering if there is any alternative ways to display the image without worrying bout the availability of my server.
Thanks in advance for any incoming advises/replies!
Since your primary concern seems to be about failure mitigation and not actual coding, I'll direct you to this question.
Your current method isn't actually embedding the images and is making a link. What you want to do is add the images as linked resources. This WILL make your emails larger in size and slower to send, but as long as you aren't a spammer, you should be OK.
Alternatively, you could have an enterprise level failure plan where your server would go offline and a mirrored server in a different location would begin serving up the data/images.
I am designing a program that will run on a mailserver. It is intended to monitor email sent to a particular username and act on input received through the email messages.
My idea is to run this program from a cron job every X minutes, check for new email, act on the email if it's present, and delete the email.
Of course, I could easily open and read /var/spool/mail/username directly as a regular text file, then truncate the file once I've read through it. But what's the proper way deal with the situation without stepping on sendmail? Another email might show up for that user either while I'm still reading the file or while I'm truncating it.
Generally what you're trying to do is better accomplished through server-side filtering as the mail arrives rather than trying to search through a mailbox every so often. It's complex and if you get it wrong, you end up losing mail.
Instead, look to server side filtering like procmail or similar to accomplish what you want.
How would one create a REST web service to write a row into a databse table. Use the follwoing scenario:
The table is called Customer - the
data to be inserted into the row would
be the name, addresss, telephone
number, email.
I think its impossible to describe the whole thing end to end in Java or C#, and I would never expect that, but here are the questions I have popping into my head as I prepare for coding:
How the URI would look (eg. if you use this URL - http://www.example.com/)?
What info would go into the HTTP envelope?
Would I use POST when writing to the database in this way?
Do I use a resource to store the posted data from the client? Is this even necessary if the data is being written to a database anyway?
When the data to be writeen into the db is recieved by the server - how do I physically insert it into the database - do I call some method on the server to actually write the data (in Java)? - this doesn't seem to fit with truely REST architecture - shunning RPC calls.
Should I even be bothering writing to a DB - should I be storing my data as a resource?
As you can see I need a few issues clearing in my head. Any help much appreciated.
First of all, I'm not either java nor c# expert and I don't exactly know what means do these languages have to support REST design, but in general:
http://www.example.com/customers - customers is a collection of resources and you want to add a new resource to this collection
It depends on various things - you should probably set the content-type header (according to the data format in which you are sending the representation) and set some authentication headers if you need it.
Yes, you always use POST to create a new entry in a collection of resources.
I don't fully understand this question, to be honest. What do you mean by "inmediately writing data into the database"?
REST is primarily just a style of communication between server and a client. It doesn't say anything about how you should handle the data received by using it. The usual way how modern web approaches (MVC style frameworks) solve it, is by routing every REST action to a method of some class (usually a controller instance) where you handle the received parameters (eg. save them to the database) and generate a response to be sent back.
For a very brief and very clear introduction to REST have a look at this short video.
RESTful Web Services, published by O'Reilly and Associates, seems to fit the bill you're looking for.
As far as doing it in Java, Sun has a page on it.