I'm deciding whether it's safe to develop my client-facing app in AngularJS using pushState.
I've read that when using pushState in an AngularJS app, we don't need to worry about Googlebot because it can now execute enough JS to produce an HTML snippet for itself. But then I wonder about Bing, Facebook and other bots and scrapers. The tutorials I've seen for making AngularJS SEO-friendly all deal with apps that use hashbangs (#!). These don't apply to me since I'm not using hashbangs.
Does anyone have insight into this problem? What are some methods for ensuring an AngularJS app that uses pushState is SEO-friendly and Social-scraper-friendly? If you use a service like Seo4Ajax or prerender.io I'd appreciate your thoughts on it.
Note: As I understand it, when developing single page apps in the last couple of years it has been necessary to send HTML snippets to SEO crawlers. This was accomplished by using hashbangs and a meta tag that let Google, Bing and Facebook know that it needed to replace the bang (!) with an _escaped_string when making a request. On the server you'd listed for requests with _escaped_string and deliver the appropriate HTML snippet using a tool to generate HTML snippets like phantomJS.
Now that we have pushState, I don't see how we indicate to javascript-less bots what part of the URL to rewrite with an _escaped_string or even if it's necessary. I'm having trouble finding any information beyond "you're site will be okay with google ;)".
Here are some other SO questions that are similar but have gone unanswered.
Angularjs vs SEO vs pushState
.htaccess for SEO bots crawling single page applications without hashbangs
Here's a solution I posted in that question and am considering for myself in case I want to send HTML snippets to bots. This would be a solution for a Symfony2 backend:
Use prerender or another service to generate static snippets of all your pages. Store them somewhere accessible by your router.
In your Symfony2 routing file, create a route that matches your SPA. I have a test SPA running at localhost.com/ng-test/, so my route would look like this:
# Adding a trailing / to this route breaks it. Not sure why.
# This is also not formatting correctly in StackOverflow. This is yaml.
NgTestReroute:
----path: /ng-test/{one}/{two}/{three}/{four}
----defaults:
--------_controller: DriverSideSiteBundle:NgTest:ngTestReroute
--------'one': null
--------'two': null
--------'three': null
--------'four': null
----methods: [GET]
In your Symfony2 controller, check user-agent to see if it's googlebot or bingbot. You should be able to do this with the code below, and then use this list to target the bots you're interested in (http://www.searchenginedictionary.com/spider-names.shtml)...
if(strstr(strtolower($_SERVER['HTTP_USER_AGENT']), "googlebot"))
{
// what to do
}
If your controller finds a match to a bot, send it the HTML snippet. Otherwise, as in the case with my AngularJS app, just send the user to the index page and Angular will correctly do the rest.
Supposedly, Bing also supports pushState. For Facebook, make sure your website takes advantage of Open Graph META tags.
Related
As far as I know the hash symbol(#) is the key when implementing routing in Angular. The web server only takes care about the part of the URL which is before the hash, and Angular takes care of the rest.
I´ve read some articles that explain how to remove the hash from URL. But if I remove the hash(#) from URL: Which routing works first?
OK, it is MVC. In that case we have to edit the MVC Route in order the server to understand the URL. But we are at the beginning again. Does it make any sense to use Angular Routing and MVC together ? Is not enough with MVC Routing?
Maybe I´m missing something. I hope you can help me.
Does it make any sense to use Angular Routing and MVC together ? Is
not enough with MVC Routing?
TL;DR;
I've rarely use both. The only time I use both is when I need to authenticate the user for some routes.
Long answer
1. Authentication
As you already figured out, Angular routing is great when you want to navigate to another page without the roundtrip to the server. It's usually a SPA. But there might be a scenario when you need to authenticate the user before sending the HTML, then MVC routing will be handy. I wrote an answer about it here. Note the difference between sending HTML and sending DATA to to the client. If you have no server routes the html-pages (or templates) will be fully accessible (unless you limit access in web.config or some other way). Some times the HTML-pages can contain some sensitive information as well...
The most common scenario is if you have a public site with an admin-part. But in my experience you can handle this on client side with client-side-routing only. It's usually the data that is sensitive, not the templates.
2. Server-side logging
The other scenario is when you want to do some logging on server side. For example if you want to log every page request. This can often be done on the client as well... Look at Google Analytics. But you might want to log the request even if the browser has javascript turned off.
3. SEO
There might be some SEO-issues when using client-side-routing. But this is only when we render the html with client side templates and if we compare to completely rendered views with MVC.Net. Do not confuse me posting the link with me actually agreeing with the content...
4. WCAG
In my country all government sites need to follow WCAG. One of the rules are - no javascript. Or at least that the site should be fully accessible without javascript. Without javascript client-side-routes are simply very difficult. ;)
These are some examples when you might need both server-side and client-side routes. But to sum up, in most cases client-side is enough.
I tried using OWASP in my angularjs web app, however.. none of my routing pages ever get lists in the spider.. can someone give me a detailed explanation on how to do this? I just want to see the vulnerabilites of all routing URLS in my angularjs app
ZAP will spider from the base URL you give it. Make sure that it can get to your routing pages by traversing the paths of available links that start on your base URL. If ZAP can't get to them by traversing your site from the base URL, create a simple sitemap or listing of all of your URLs and link/insert that on your home page.
Also, this is a handy way to get ZAP up and running via Docker.
https://github.com/zaproxy/zaproxy/wiki/Docker
HTH
AngularJS makes heavy use of JavaScript, which the 'traditional' spider doesnt handle very well.
Instead use the ajax spider: https://github.com/zaproxy/zap-core-help/wiki/HelpAddonsSpiderAjaxConcepts this is slower but will handle any JS as it launches browsers to crawl the site.
I am developing an app with an AngularJS front end and a Laravel API backend. The Laravel backend just listens for requests, process them and returns an answer. Front and back end are different independent apps... so far. I did it in such a way in order to be able to develop a mobile app later on which could consume the API. Im using also JWT to authenticate users, so Im not using Laravel's sessions at all. At this point I only require a webapp. I built a webapp which uses angular ui-router.
So far so good. The front end and the back end work well. However some of the front end views will be public and require share buttons, they also require to be indexable by Google.
I've read there are some alternatives.
Make some hack using the apache mod_rewrite in order to serve the angular app for people and a static version of it served directly by the backend. I think this would not be very difficult using Laravel.
Using Prerender.js. Which as far as Im concerned does pretty much the same job than option 1 but in a more complete manner.
However I am thinking about using a third alternative. Given that I only require the webapp now and the API is working I am thinking about using Laravel's built-in webapp functionality. I can use the controllers, directives and factories from angular and let Laravel handle the webapp routing.
An advantage of this is that I can render the meta tags using Blade (this fixes completely the SEO issues) and serve the rest of the contents using angular and the API.
Do anyone of you can see drawacks of such a solution or do you know a better way to accomplish SEO purposes using angular and Laravel?
Render Site :
You will have to render the pages from server side if you want to make it SEO friendly. But yes, what if you serve the server rendered pages to bots only. And for real users client side rendering will work.
Read about the Rendertron which will detect the user agent and accordingly serve the html.
Meta Tags :
Use Sluggable and replace id with slug in urls. You can follow https://oodavid.com/article/angularjs-meta-tags-management/
Is it possible for Google to render SPAs (angular in particular) without the use of a headless browser?
I have built a service purely in angular but I would like to make it possible to apply SEO.
Now Google crawlers are able to render AJAX (Angular) sites: http://googlewebmastercentral.blogspot.com/2015/10/deprecating-our-ajax-crawling-scheme.html
But you can implement AJAX crawling scheme (deprecated): https://developers.google.com/webmasters/ajax-crawling/docs/learn-more
First of all, no, you can't render an Angular page without the aid of an actual browser. It's something that is coming in Angular 2 but for now we are stuck using PhantomJS or some headless Chrome or something if we want to create HTML snapshots.
However, as Alex already pointed out that is no longer needed for SEO reasons!
But, wait, there's more
The reason I write my own answer is to point out that it IS still needed for Facebook, Twitter, LinkedIn etc :(
If a user shares a link to your site on Facebook (or Twitter, etc) their crawler will try to read your page and generate a preview. Unlike search engines these crawlers will not run Javascript, so the preview of your Angular page will look all broken.
If you want your previews to be dynamic you will still need the html snapshots for now.
I'm getting really frustrated with configuring the Routing on our app, which is using sailsJS and angularJS.
The problem is, that the browser doesn't know about angular, so any request like /login returns a 404 Error from sails. I need a solution, to keep the sails routes from the angular ones,
One solution would be to disable html5Mode, but i really don't like the look of URLs with the /#/ which is typical for angular.
I have researched a lot on this and haven't yet found a good answer or maybe a working project for this.
Is what I am trying to do even possible right now?
If you're using HTML5 mode with Angular, then you need to configure your web server (in this case SailsJS) to respond with your index.html file for requests to /login or any arbitrary routes.
If you navigate directly to http://localhost:3000/login in your web browser (assuming you're running Sails on localhost:3000), Sails needs to respond with your index.html so that your Angular app can bootstrap and then display the appropriate route. Then, subsequent links that the user clicks on in your app will be intercepted directly by the Angular router instead of Sails directly.
Angular has documentation about making HTML5 mode work correctly here.
Using this mode requires URL rewriting on server side, basically you have to rewrite all your links to entry point of your application (e.g. index.html). Requiring a <base> tag is also important for this case, as it allows Angular to differentiate between the part of the url that is the application base and the path that should be handeled by the application.