I have an AngularJS SPA site which I wanted to test using google's "Fetch as Google" feature in webmaster tools. I am a little confused about the results. The screenshot from Googlebot looks correct however the response doesn't include any of the contents inside the "ui-view" (ui-router)... can someone explain what is happening here? Is google indexing the site properly since the screenshot is correct? Or is google not able to execute the JS properly for indexing?
This is a mixed bag. From some tests I've seen the GoogleBot is able to index some of the AJAX fetched content in some cases. A safe bet though to make all the search engines happy is to use prerender.io or download their open source stuff (uses PhantomJS) to have your site be easily indexable. Basically what this does is saves the version of your site after async operations have completed for a given URL and then you setup a redirect on your server that points any of the potential bots for search engines over to the preprocessed page. It sounds pretty complicated but following the instructions on the site it's not too hard to setup, and if you don't want to pay for prerender.io to serve cached copies of your pages to search engines you can run the server component yourself too.
Related
I have a NextJS site which is statically generated at build (SSG).
There are two things I need to implement next
Google Analytics
GDPR compliant opt-in cookie options
The first one is easy enough to do, however i'm struggling with making this GDPR compliant.
The issue is I don't have access to cookies at server side when my site is statically generated. This means that without knowing whether the user has consented to cookies at the server, I can't serve (or not serve) the analytics script along with the rest of the page.
Possible solutions:
Handle everything at client side - ask for consent, then dynamically add the GA tag to the <head>. However i'm worried this will negatively effect the analytics, or break it altogether. Does anyone know?
Change my site to be server-side rendered (SSR). I'd love to avoid this if possible. I'm really happy with how fast the site is running with SSG. It's essentially just a basic blog so would be a shame to have to convert for the sake of analytics.
Any other ideas?...
If anyone has experience with this, whether they used Next or Nuxt, etc, your input would be greatly appreciated!
Thanks in advance
Use Google Tag Manager to manage everything, your GA integration and your cookie integration using something like CookieHub for example (How to set up Google Analytics through Google Tag Manager for Next-Js?)
GTM will allow you to trigger the GA script only if the user specified he accepts analytics cookies.
Eitherway you could use Vercel.com built in analytics since your website is using Next.js wich is Vercel's framework.
how should I put Angular url's in the sitemap.xml?
Now I've added this:
https://www.domain.com/user-area#!/logon
but I think Google doesn't like it.
I've read about the _escaped_fragment_ but I don't understand what that means.
The _escaped_fragment_ was recently depreciated by google. See their blog post here:
http://googlewebmastercentral.blogspot.com/2015/10/deprecating-our-ajax-crawling-scheme.html
Anyways, from what you wrote, it seems like your server structure isn't prepared for the _escaped_fragment_. I won't go into too much detail about it here, since it was depreciated after all.
Anyways, Google's bots weren't always able to process websites with AJAX content (content rendered via Javascript). To create a workaround, Google proposed adding the hashbang #! to all AJAX sites. Bots would be able to detect the hashbang and know that the website had content rendered through AJAX. The bots would then request a pre-rendered version of the AJAX pages by replace the hashbang with the _escaped_fragment_. However, this required the server hosting the AJAX pages to know about the _escaped_fragment_ and be able to serve up a pre-rendered page, which was a difficult process to set up and execute.
Now, according to the depreciation blogpost, the URL you entered in your sitemap.xml should be fine, since Google's bots should be able to "crawl, render, and index the #! URLs". If you really want to know if Google can understand your website, I'd recommend using their webmaster console, located at https://www.google.com/intl/en/webmasters/ . Using that tool, you can register your site with google, observe how google indexes your site, and be notified if any problems arise with your site.
In general though, getting Google to index an AJAX site is a pain. I'd strongly recommend using the webmaster console and referring to it frequently. It does help.
I have a PHP website which is currently not using AngularJS. I'd like to change its design and while I'm at it, start using AngularJS to make a SPA website. This website contains programming tutorials, so its position in google search result is crucial. I've been searching recently to see if AngularJS can be used in that case and the most interesting post I found is this one : Google bot crawling on AngularJS site with HTML5 Mode routes
I'd like to know if my website will still be referenced the same way if I use AngularsJS. I'll pay attention that URL stays the same, but will the Google bot be able to crawl my website the same way it does now ? I need to be sure because I don't want to loose all my traffic because Google is no longer able to crawl my site.
Google now runs JavaScript and can index AngularJS sites.
Source
Some caveats though: most top ranked search results are still server-rendered, so how the Google algorithm ranks your site is unclear. Also other search engines such as Bing may not be able to read your site.
I am working in Angula JS and Drupal in this i am facing two issue SEO and page preview when we paste url in Google,Facbook,Twitter and many more social site
i have done case for _escaped_fragment_ in url by this help url(https://developers.google.com/webmasters/ajax-crawling/docs/getting-started)
But the challange is coming when i paste my url like
http://example.com/#!/test/a/1235
then no preview is generated
How will i show preview in social sites.
Any help is appreciated.
Thanks
Welcome to Javascript application! :)
_escaped_fragment_ is not standard & Social platforms do not support it
Google developed the _escaped_fragment_ system but this not a standard.
There is many bots on the web and most of them do not understand the _escaped_fragment_ solution and do not understand Javascript applications as AngluarJS ones.
As far as I know (I worked on many JS application websites), social platforms do not use the _escaped_fragment_ system.
Moreover, some Google services do not support it yet.
URL is standard and is supported by every bots
If you want every robots to be able to crawl your website, the only way for now is to use classic URLs.
For now, you need to make sure your content is delivered on classic URLs. It is the only way to be sure that it will be interpreted by every bots of the web
This can be a bit challenging and requires a two-fold answer.
Headless Browser
The majority of crawlers and bots cannot parse the javascript in your Single Page App (SPA). Therefore, you will need some sort of headless browser to generate what those bots see. I have used PhantomJS and it works well for me. Once your headless browser is up and running you can create rewrite conditions for the _escaped_fragment_. For example in Apache:
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
I would suggest you create rewrite conditions based on user agents as well. This would be particularly useful when detecting "FaceBot", Facebook's preview crawler, and others.
Social Applications
The other part of the solution is to read the developer docs on exactly how to manipulate the generation of these previews. Here are a couple resources for this (sorry I can't find twitter's):
Facebook's Sharing Best Practices
Google's Article Rendering
When checking you page for Facebook, they have a neat little tool that will help you troubleshoot your site/page for preview rendering: https://developers.facebook.com/tools/debug
I'm developing a web site with Google App Engine, and I want to have a search feature for user submitted stuff. Since this project is just a toy and I don't control the server, I'd like to just use Google to handle search. However, since the content is stored in the database, I don't think Google can discover the dynamic urls. Unless maybe I create a page that links the last N submissions and hope it gets crawled frequently. Thoughts?
Absolutely. As long as the database is exposed in a web page which can be crawled, Google will crawl it (unless told not to).
The best way to make it all accessible is decent navigation between pages. However, lacking that, a site map page linked from the home page should suffice.
This is an excellent candidate for a sitemap.
You can generate the XML any way you want, and give it to Google. The best part is, it is a "private" XML file; no need to have ugly listings of dynamic URLs for users to see.