I have a backend data management console written with CakePHP that allows a user to manage some hierarchical data. Every time a user makes a change to the data I want to regenerate a data file (JSON in this case) that's used on the site's frontend.
The rebuild can take some time and I'd like the backend UI to be a bit more responsive. My idea is to have the JSON rebuild take place after the new page (the "your changes have been saved" page) is rendered to the user. I've got some code within the afterFilter() call back in my app_controller.php, but the page doesn't actually render in the browser until the JSON rebuild completes.
I've found code examples for plain-vanilla PHP that does things like send a Connection: Close HTTP header and/or uses output buffer flushing to tell the browser that the server's done while processing continues, but these techniques don't (as far as I can tell) work with CakePHP's structure and its own output buffering.
What I would like is a technique that would allow me to completely render a view for the user and then, once the user has their page loaded, continue executing the JSON rebuild in the background.
I realize there may be situation/setup specific concerns that could affect things, so please let me know if you've got questions about my particular application.
Thanks in advance.
those cakephp queue plugins are invented for this exact purpose:
https://github.com/MSeven/cakephp_queue
they decouple the frontend from the backend services like those file generations.
Here is a suggestion. This is how I ran long running processes from the UI for scrubbing files. First create a SHELL for doing the processing. http://book.cakephp.org/2.0/en/console-and-shells.html#Shell
This will provide the code that will run the background process you are looking to run.
Then, setup the method in the UI side to call the shell. Grab the PID and save it to the database (so you can tell when it finished).
$PID = shell_exec("/path/to/cake/console/cake SHELLNAME SHELLMETHOD");
$this->Jobs->query("UPDATE `jobs` SET `pid` = $pid WHERE `id` = $job_id");
Then you can always check to see if the process is running by checking the /proc/$PID.
Related
I am looking for a pattern that would allow me to better the UX for my users. I have a REST server running behind CloudFront being consumed from a plain React application on the frontend.
I'll simplify my example to illustrate my issue.
I have an endpoint called GET /posts/<id>. When the browser asks for it, it comes with a max=age=180 which means it would get stored in the browser's cache and any subsequent call to GET /posts/<id> will be served from the browser's cache for the duration of those 180 seconds, after which it will hit the CDN again to try and obtain a fresh copy.
That is okay for most users. I don't mind if updates to any post to delay up to 3 minutes before they're propagated to all the users. But there is one user who's the author of this post. That user can make changes to this post using PATCH /posts/<id>. Let's call that user The Editor.
Here's a scenario I have right now:
The Editor loads up the post page which then calls GET /posts/5
The CDN serves the latest copy to the front end.
the Editor then makes a change to the post and submits it to be back end via PATCH /posts/5.
The editor then refreshes his browser tab using Command-R (or CTRL-R).
As a result, the front end then requests GET /posts/5 again -- but gets the stale copy from before the changes because 180 seconds haven't passed yet since the last GET and the GET issued after the PATCH
What I'd like the experience to be is:
The Editor loads up the post page which then calls GET /posts/5
The CDN serves the latest copy to the front end.
The editor then makes a change to the post and submits it to be back end via PATCH /posts/5.
After a Command-R browser tab refresh the GET /posts/5 brings back a copy of the data with the changes the editor made with PATCH right away, regardless of the 180 seconds of ttl before a fresh copy can be obtained.
As for the rest of the users, it's perfectly okay for them to wait up to 180 seconds before the change in the post propagates to them when the GET /posts/5
I am using Axios, but I do not that SWR and React-Query support mutations. To my understanding this would allow the editor to declare a mutation for the object he just PATCH'ed on the server, so that any subsequent calls he makes to GET /posts/5 will be served from there, until a fresher version can be obtained from the backend.
My questions are:
Can SWR with "mutations" serve the mutated object via the GET /posts/5 transparently?
Will the mutation survive a hard browser tab refresh? or a browser closure, re-opening and subsequent /GET posts/5?
Is there another pattern/best practice to solve that?
TL;DR: Just append a harmless, gibberish querystring to the end of the request GET /posts/<id>?version=whatever
Good question. I must admit I don't know the full answer to this problem, but I want to share one well-known technique among frontend devs.
The technique is called cache busting. I'm not sure if this is the best practice, but I'm pretty sure it's widely practiced, since it's so straight-forward to understand.
Idea is simple. When you add a changed querystring to the end, you effectively change the URL, thus no cache is hit, you evade the whole cache problem.
So the detail steps to a solution for your particular use case would go like this:
Normally you'll just request GET /posts/<id> for all users
When a user logs in, a hash key is generated from whatever algorithm. For simplicity let's just use increasing integer and call it version. You store this version in localStorage so it can survive through page refresh.
Now you need to distinguish scenario when the user is viewing his own posts or other's posts. When guy is viewing his own, you always use GET /posts/<id>?version=n
Whenever the user edits his post and hits save button, you bump version from n to n+1
Next time he goes to post view page, the app requests GET /posts/<id>?version=n+1 which is not cached, and would retrieve the up-to-date content.
One last thing, make sure your server safely ignores that ?version=n querystring.
I'm sure there're other solutions to this problem. I'm no expert of server config and HTTP headers so I'm not getting into that topic, but there must be something to look for.
As of pure frontend solution, there's Serivce Worker API for you to consider. The main point of this API is to enable devs to programmatically control cache strategies.
With this API, you could leave your current app code as-is, just install a service worker, then you could use the same cache busting technique in the background to fetch new content, or just delete the cache (using Cache API) when user edits, or even fake a response for the GET /posts/<id> from the PATCH /posts/<id> that user just send.
Depending on what CDN you use, you can invalidate a cache manually when publishing updates to a post. For example cloudfront lets you specify which path you want to fetch fresh on the next request.
For sites with lots of traffic but few updates this works pretty well, and is quite simple to implement. For sites with a lot of authors and frequently changing content you would need to get more creative though.
One strategy I've used in the past is using a technique called object versioning, where instead of invalidating the cache to an object you just publish a version of it with a timestamp. This would also mean you need to publish a manifest file when your frontend loads. The manifest contains the latest timestamps of all the content the page needs to load, and is on a much shorter TTL than the rest of the content. When you publish a new version of a post you would update the timestamp in the manifest, and the frontend pulls the latest version of it the next time the page loads.
I am using Angular SPA with DTM.Using custom event based rules, I am able to get all my data including pageName, v41,v42 as correct. Now inside adobe editor, i am storing pagename to s.pageName and some hard-coded value to s.server. I have verified that all my data is correctly populating using OMNIBUG tool as server,pageName, v41 and v42.
Problem is coming in Omniture reporting, as server and page data are not coming through. Page-name data only showing SPA homepage in all page visits and server also coming as default from s.code and not the one i am passing from s.server. eVar/prop are all coming fine.Even if I do prop40=s.pageName/prop41=s.server, then in omniutre reporting i am seeing correct data populating in prop40/prop41 but not under Page and server. And again I cant use prop40/prop41 for pagename/server as its not a correct way to follow and PAGE-VISITS are ZERO in that case.
Any help how to get data in page/server in omniture for SPA or anything wrong in my implementation? Thanks in advance!!
If you really do see the correct values in Omnibug (or more specifically, network request to Adobe collection server), then the issue is not in the code.
Check against another AA hit debugger. Possible Omnibug is somehow bugging out. There are a ton of alternatives out there. Adobe Experience Cloud Debugger. Observepoint. Charles Proxy. Fiddler. Or just use the browser dev tool network tab (what I usually do as a backup).
Make sure you are looking in the correct report suite. Perhaps your data is being sent to a dev report suite, and you are looking at prod report suite, or visa versa?
Check to see if you have any Processing Rules that are overriding your values.
Contact your Adobe Rep to check if there are any VISTA Rules present for the report suite, that are overriding your values.
If you have verified none of the above is the case, then sorry, but it sounds like the issue must really be in your code, but there is a problem with your QA method (e.g. maybe you are looking at the wrong AA request, or something).
Update:
Based on your comment:
Earlier, i was making s.tl() call, but replacing it with s.t() call
resolved my problem for data was not populating
pageName/server/page-views in Omniture and now it is. But the current
problem is we need PageName on all SPA clicks (can be achieved by
s.t() call ) , but the page-Views are not needed on all clicks. So,
its like link-tracking needed only but with PageName data. I am
struggling not to populate page-views on a s.t() call or vice-versa
how to get PageName populated on s.tl() call. Again, omnibug shows all
requests just fine but the issue comes in reports in omniture
When Adobe processes a hit, it wipes pageName for s.tl calls, as that's how it determines whether to count the request as a page view or not. If you want to see page name even for s.tl calls, the common practice is to dupe the pageName value to a prop or eVar and send in with the s.tl call, and look at that report. In fact, most clients I work with don't even use the native pages report, and instead use the (usually eVar) report.
I have a an application that uses Angular for the frontend and communicates with an API running Django RF.
Now, let me try and outline what I'm trying to achieve in hopes of finding an easy solution.
When a user runs a report, a worker version of the API generates the report to prevent the main API from bogging down. The report runs for a couple seconds/minutes. The user refreshes the page and voila, their report is there.
What I'm trying to achieve is the elimination of the whole "user refreshes the page" portion of this process.
My goal is to do this via websockets.
I literally want something as simple as:
WEB: "Hey API, I want this report. I'll wait."
API: "Yo dog, you're reports done."
WEB: "Cool, let me refresh my report list. Thanks bud."
Now, we start to venture into an area I'm unfamiliar with. Websockets can do this, right? I just need to create a connection and wait for the all-clear to be sent by the worker.
Now, here's where it gets hairy.
I've spent the better part of the day going through libraries and just can't find what I need. The closest I've come is this, but it clashes with restframework. I get hit with tons of 404 errors and I think it has to do with the way rf manages urls.
I literally need a simple event listener. There's got to be a better way, right? To clarify, I don't want to do something brute-force like silently ping the API for report status. That gets a tad hinky. I want the API to tell me when it's ready.
In a basic way, can use something like django-websocket-redis and use Django signals to pass the messages around. ws4redis handles alot of the tricky bit. However, websockets are weird and honestly I doubt you need them. You could just poll some route that has the job state. If you need to get it done fast, I would go that route.
I had the idea to make a SPA application using angularJS and then just sending AJAX updates to the server when I need.
My initial idea would be make the client application fly, but if I have to do an AJAX round trip to the server, I think the time would be approximately the same as to request a single web page.
Requesting a page just has more bytes of data, is not like I'm requesting 20 resources like in this article: https://community.compuwareapm.com/community/display/PUB/Best+Practices+on+Network+Requests+and+Roundtrips
I would be requesting a page or resource per request.
So in the end even if I create my client side application as a SPA using angularJS, these requests (would have to be synchronous and show a please wait message while they don't return, as I don't want to user to take more actions before I make sure his request passes validation and is processed correctly) would take some time and make user wait, just about the same time as requesting a full page.
I think SPA pages would be very useful if I have like a wizard on my app with multiple pages/steps and at the end, submit the results of wizard, to the server, which I don't.
Also found this article:
https://help.optimizely.com/hc/en-us/articles/203326524-AngularJS-Backbone-js-and-other-Single-Page-Applications
One of the biggest advantages of Single Page Apps is that they reduce
data transfer. As a result, pages after the initial loading usually
can be displayed faster and seem more interactive.
But I don't believe this last quote is really true.
Am I right, or is there a way that I'm not seeing to build an application that would look like it's executing locally?
I know how guys will start saying "depends on what you want", but lets focus on this scenario where there's no wizards.
What ever you said is right. But most of the frameworks(Angular,BackBone) you take they are going to cache the templates of html on the browser so the rendering would be pretty fast compared to the normal applications. Traditional apps will have to fetch the html from the server for each request which is a time consuming one.
Hope this helps you!!!
If you are wanting to go through that syncronous server side validation step for each page request, then there is probably no big advantage to using AngularJS.
If you are requesting a page and then manipulating that page's contents once it's loaded you might want to consider AngularJS. A good example would be requesting a page that displays a list of items. Now let's say we want to search that list or order it in different ways. Rather than using AJAX to call the server to filter the list and then re-render it, it could be much faster to user AngularJS to filter and re-render the list without making any further requests to the server.
Symptom: I'm running a Google Website Optimizer A-B test on a web page, which is reached after the user fills out a form. The form data appears to arrive successfully when GWO does not fire a redirect (when the default page is reached after submitting the web form). The data does not appear to arrive, however, when the alternate page is reached (i.e. when GWO redirects the user).
Details: I wouldn't attribute this to GWO, except that in my development environment, where I was obliged to comment out the GWO javascript, everything works as it should do. (The GWO code had to be omitted, of course, to prevent my browser redirecting me to the production environment, i.e. the live version of my site.) (Naturally, without the GWO code in place, I had to hard code a change to my form's action attribute in order to verify that my query data was handled successfully by the alternate destination page.)
Really, it looks as though GWO is just failing to pass on the requisite query parameters when it performs its redirect, but that seems unlikely. Can you give me any advice?
Fixed (rather): I changed the form to pass GET (not POST) data, and indeed, GWO was able to collect that query data and pass it on to the new URL. (I find that surprising, since the url is stipulated, and I should have thought it easier to accept and pass on POST data.)