Nginx config for serving snapshots to the Google bot - angularjs

Question

I have an AngularJS app which I'd like to get indexed properly on Google.
I wrote a client that scrapes the sites for links and then downloads the pages with Phantomjs making snapshots. This all works fine. What I'm having a problem with is serving those snapshots to the Google bot.
For some reason, the Google bot appends ?_escaped_fragment= to my URLs. As an example, http://me.com/about gets changed to http://me.com/about?_escaped_fragment=. I've verified this in the access logs.
I'm trying to catch this request and serve the Google bot the snapshot with this config:
location / {
if ($args ~ "_escaped_fragment_=") {
rewrite ^ /snapshots/$1;
}
}
However, requesting this URL: http://me.com/about?_escaped_fragment= always results in a 404. Same with the other pages.
The snapshots are stored in /snapshots, relative to the root of the website. They're named after their pages, following directory structure, so http://me.com/business/register has a snapshot in /snapshots/business/register.html.
What can I do to get these snapshots to work?
Thanks.

Answer 1

Ok first let me explain why google uses ?_escaped_fragment_, This is used for websites that rely on ajax, and mark their page with hashes, like for example if you have http://example.com/gallery/#!image1 and each time the user changes to the next image you update the hash to image2, image3, but if the user goes directly to http://example.com/gallery/#!image50 your javascript uses that hash to load the 50th image directly instead of image1 ( servers can't see the hash part, only javascript can ).
So google uses this _excaped_fragment_ to tell the server which page it's trying to cache.
For more explanation use this link
As for why you get a 404 error, I think because you used a $1 without using a capturing block, The right rule would be something like this
location / {
if ($args ~ "_escaped_fragment_=(.*)") {
rewrite ^ /snapshots/$1;
}
}
But I don't think this will fix your problem, because according to your example, you didn't use hashes, you used the uri of the page, so i would rewrite the rule to something like this
location / {
# try snapshot, if not found try direct file.
try_files snapshots$request_uri.html $uri;
}

Answer 2

Here is what I have in nginx and it is working fine, you might need to add a special one for index.html (i.e. when accessing the root of your website)
if ($args ~ "_escaped_fragment_=/(.+)/?") {
set $path $1;
rewrite ^ /snapshots/$path.html;
break;
}
location /snapshots/ {
internal;
alias /var/www/snapshots/;
}
So http://me.com/?_escaped_fragment_=/about will access /var/www/snaphots/about.html
Don't forget this meta tag as well in your page if you use html pushstate instead of hashbangs:
meta(name="fragment", content="!")