How to do search engine optimization (SEO) for the website built with angular js.
please explain how can we do it.
To my knowledge there is only one way to do this:
By making html snapshots to serve up rendered JavaScript pages
Now there are different ways to achieve this.
The easy way
The most easiest way to do this would be to pay to a third party. I am familiar with Prerender.io. I haven't used it myself, but when I needed SEO for a site built with AngularJS this was one of the options I have considered. Just browse the internet a bit and read up on their service, you will find some pretty good reviews.
The alternative
The alternative is Angular-SEO by steeve. Basically it does the same as prerender.io but it required you to launch the PhantomJS server on your own server (to which you need sudo access/rights). This, together with a bit of server configuration / tweaking in the htaccess file in my case did it for me. You basically detect the useragent / robot / scraper that is accessing your site and you redirect them to the location of your Angular-SEO server (address:port), which then returns a snapshot of the required page.
I decided to take the Angular-SEO= route, since it was an application for a client and I did not want to pay for the prerender service myself nor asking them to pay an extra monthly fee for a problem that I caused (I chose to use AngularJS without having investigated the SEO part (regret that))
The other day I saw that DivShot has partnered with Prerender.io which makes it an interesting choice to consider for hosting
Related
As far as I know the hash symbol(#) is the key when implementing routing in Angular. The web server only takes care about the part of the URL which is before the hash, and Angular takes care of the rest.
I´ve read some articles that explain how to remove the hash from URL. But if I remove the hash(#) from URL: Which routing works first?
OK, it is MVC. In that case we have to edit the MVC Route in order the server to understand the URL. But we are at the beginning again. Does it make any sense to use Angular Routing and MVC together ? Is not enough with MVC Routing?
Maybe I´m missing something. I hope you can help me.
Does it make any sense to use Angular Routing and MVC together ? Is
not enough with MVC Routing?
TL;DR;
I've rarely use both. The only time I use both is when I need to authenticate the user for some routes.
Long answer
1. Authentication
As you already figured out, Angular routing is great when you want to navigate to another page without the roundtrip to the server. It's usually a SPA. But there might be a scenario when you need to authenticate the user before sending the HTML, then MVC routing will be handy. I wrote an answer about it here. Note the difference between sending HTML and sending DATA to to the client. If you have no server routes the html-pages (or templates) will be fully accessible (unless you limit access in web.config or some other way). Some times the HTML-pages can contain some sensitive information as well...
The most common scenario is if you have a public site with an admin-part. But in my experience you can handle this on client side with client-side-routing only. It's usually the data that is sensitive, not the templates.
2. Server-side logging
The other scenario is when you want to do some logging on server side. For example if you want to log every page request. This can often be done on the client as well... Look at Google Analytics. But you might want to log the request even if the browser has javascript turned off.
3. SEO
There might be some SEO-issues when using client-side-routing. But this is only when we render the html with client side templates and if we compare to completely rendered views with MVC.Net. Do not confuse me posting the link with me actually agreeing with the content...
4. WCAG
In my country all government sites need to follow WCAG. One of the rules are - no javascript. Or at least that the site should be fully accessible without javascript. Without javascript client-side-routes are simply very difficult. ;)
These are some examples when you might need both server-side and client-side routes. But to sum up, in most cases client-side is enough.
I tried using OWASP in my angularjs web app, however.. none of my routing pages ever get lists in the spider.. can someone give me a detailed explanation on how to do this? I just want to see the vulnerabilites of all routing URLS in my angularjs app
ZAP will spider from the base URL you give it. Make sure that it can get to your routing pages by traversing the paths of available links that start on your base URL. If ZAP can't get to them by traversing your site from the base URL, create a simple sitemap or listing of all of your URLs and link/insert that on your home page.
Also, this is a handy way to get ZAP up and running via Docker.
https://github.com/zaproxy/zaproxy/wiki/Docker
HTH
AngularJS makes heavy use of JavaScript, which the 'traditional' spider doesnt handle very well.
Instead use the ajax spider: https://github.com/zaproxy/zap-core-help/wiki/HelpAddonsSpiderAjaxConcepts this is slower but will handle any JS as it launches browsers to crawl the site.
I'm deciding whether it's safe to develop my client-facing app in AngularJS using pushState.
I've read that when using pushState in an AngularJS app, we don't need to worry about Googlebot because it can now execute enough JS to produce an HTML snippet for itself. But then I wonder about Bing, Facebook and other bots and scrapers. The tutorials I've seen for making AngularJS SEO-friendly all deal with apps that use hashbangs (#!). These don't apply to me since I'm not using hashbangs.
Does anyone have insight into this problem? What are some methods for ensuring an AngularJS app that uses pushState is SEO-friendly and Social-scraper-friendly? If you use a service like Seo4Ajax or prerender.io I'd appreciate your thoughts on it.
Note: As I understand it, when developing single page apps in the last couple of years it has been necessary to send HTML snippets to SEO crawlers. This was accomplished by using hashbangs and a meta tag that let Google, Bing and Facebook know that it needed to replace the bang (!) with an _escaped_string when making a request. On the server you'd listed for requests with _escaped_string and deliver the appropriate HTML snippet using a tool to generate HTML snippets like phantomJS.
Now that we have pushState, I don't see how we indicate to javascript-less bots what part of the URL to rewrite with an _escaped_string or even if it's necessary. I'm having trouble finding any information beyond "you're site will be okay with google ;)".
Here are some other SO questions that are similar but have gone unanswered.
Angularjs vs SEO vs pushState
.htaccess for SEO bots crawling single page applications without hashbangs
Here's a solution I posted in that question and am considering for myself in case I want to send HTML snippets to bots. This would be a solution for a Symfony2 backend:
Use prerender or another service to generate static snippets of all your pages. Store them somewhere accessible by your router.
In your Symfony2 routing file, create a route that matches your SPA. I have a test SPA running at localhost.com/ng-test/, so my route would look like this:
# Adding a trailing / to this route breaks it. Not sure why.
# This is also not formatting correctly in StackOverflow. This is yaml.
NgTestReroute:
----path: /ng-test/{one}/{two}/{three}/{four}
----defaults:
--------_controller: DriverSideSiteBundle:NgTest:ngTestReroute
--------'one': null
--------'two': null
--------'three': null
--------'four': null
----methods: [GET]
In your Symfony2 controller, check user-agent to see if it's googlebot or bingbot. You should be able to do this with the code below, and then use this list to target the bots you're interested in (http://www.searchenginedictionary.com/spider-names.shtml)...
if(strstr(strtolower($_SERVER['HTTP_USER_AGENT']), "googlebot"))
{
// what to do
}
If your controller finds a match to a bot, send it the HTML snippet. Otherwise, as in the case with my AngularJS app, just send the user to the index page and Angular will correctly do the rest.
Supposedly, Bing also supports pushState. For Facebook, make sure your website takes advantage of Open Graph META tags.
I want to use Symfony2 as back end to create a REST API and use AngularJS as front end. The 2 are completely separated ie Symfony 2 will not render anything, it'll just send json data to AngularJS.
I'm not sure on how to configure my web server (nginx).
The Symfony documentation gives the configuration but it's intended for a site that only uses Symfony, and so everything outside of the /web/ folder is not accessible.
I can see several possibilities:
Create 2 different directories (eg /path/frontend and /path/backend) and a corresponding website for both. I would then have 2 different addresses to access the front end and the back end (eg http://myfrontend.com and http://mybackend.com). The problem I see is that I probably won't be able to directly use AJAX calls in AngularJS.
Create 2 different directories (eg /website/frontend and /website/backend) and only one website. I would then probably need to access the front end and back end with something like http://example.com/frontend and http://example.com/backend. I'm not sure how to configure the web server though (issue with root /website/backend/web).
Put the AngularJS directory inside the web folder of Symfony, but then I'd need to also change the configuration so that nginx doesn't only server app.php, app_dev.php and config.php.
Put the AngularJS directory in the src folder of Symfony, and have Symfony handle the routing. I don't know if it will mess with AngularJS' one routing. Also I will probably have a few other php that should be accessible, so I'd need to route them through Symfony also.
What would you suggest and why? Maybe I'm missing something obivous?
I guess you could accomplish your task using any of those methods. It would come down to how you want to structure you application and what it's objectives are. For large scale projects the first method (having the API separate from the AngularJS) would serve you well. Twitter really made that software model big.
So I would suggest going with method one. All you would have to do is specify an Nginx header in your server block that allows cross domain access to another domain. So you would specify the following directive in your frontendangular.com site:
add_header Access-Control-Allow-Origin backendsymfony.com;
This way every time a page request comes in on your front end app Nginx tells the browser that it is safe to access another domain (your symfony setup).
These are 2 frameworks that both have powerful routing capabilities, and it looks like you are going for a best of both worlds. There are many pros and cons to any setup, so I'll list a few that come to mind:
Angular routing / templating is great but it will leave you with SEO and meta issues to solve . It's probably better to manage your major pages with symfony2 and any routing within each page with angular. This would allow you to still make dynamic pages w/out compromising your meta and SEO control. Access Control seems flexible but probably not necessary, I would just put all calls to REST API under http://www.thesite.com/api and if I need another setup something like https://api.thesite.com, nginx can route or proxypass this without leaving the domain.
Locating partials gets a little wonky but that's probably fine for a large application. Just note that you will probably need to search the js location object for [host] / [path] /web/bundles/someBundle/public/blah.... Or you can setup a '/partials' path in nginx.
Twig and Angular tpl may end up a confusing mix as they both use {{foo}}. This alone would make me reconsider mixing the 2, and I might look to go with a frontend server, like node with ejs where I could also benefit from the streaming transfer of the data sent from the API.
You can get around it easy enough with but it's still concerning:
angular.module('myApp', []).config(function($interpolateProvider){
$interpolateProvider.startSymbol('[[').endSymbol(']]');
}
);
You do get the benefit of angular partials as symfony twig, which can be good or bad depending on how you see the flexibility being used. I have seen guys making examples of forms that fill out values with symfony data, but they are just undermining the power of angulars binding.
Don't get me wrong, I actually do really like the idea of the two harmonizing.
Just food for thought - cheers
I'm doing some due diligence on backbone for a single page app and wonder if it is possible to re-use the same views/routes from the client on the server side, so that when google visits a pushstate URL, or it is accessed directly, the server can generate the exact same HTML that would be generated by backbone in the client.
It would be a pain to have to maintain two separate sets of views/routes, one on the client and one on the server. I have seen the backnode project on github however this seems to miss the point a bit and you end up having to write the same backbone router twice.
Just wondering how people are generally handling the case when using pushstate urls in backbone and needing to serve the same view from the server? Are people duplicating code or is there a better way?
I haven't tried this yet but these ideas using node.js and backbone might help:
http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
http://bennolan.com/2010/08/13/pushstate-and-nodejs.html
Basically the only way to make it so your not writing the same thing twice is to have both your node.js server and frontend client share the same routing/model code. If you are using something else on the server side (like Ruby) you would have to place node.js in front of your app server. Then make node.js proxy for new clients (push state and client js) and do actually rendering work for old/bot clients.
The other option is doing what jQuery Mobile does which is the Hijax method. The idea to make tons of HTML5 pages (you'll have to look at its routing to see).
And for completeness you should be aware of how google crawls AJAX: http://code.google.com/web/ajaxcrawling/docs/getting-started.html
This project does exactly what you are trying to do. It might interest you to check it out.
https://github.com/developmentseed/bones