I am new to SEO and just want to get the idea about how it works for Single Page Application with dynamic content.
In my case, I have a single page application (powered by AngularJS, using router to show different state) that provides some location-based search functionalities, similar to Zillow, Redfin, or Yelp. On mt site, user can type in a location name, and the site will return some results based on the location.
I am trying to figure out a way to make it work well with Google. For example, if I type in "Apartment San Francisco" in Google, the results will be:
And when user click on these links, the sites will display the correct result. I am thinking about having similar SEO like these for my site.
The question is, the page content is purely depending on user's query. User can search by city name, state name, zip code, etc, to show different results, and it's not possible to put them all into sitemap. How google can crawl the content for these kind of dynamic page results?
I don't have experience with SEO and not sure how to do it for my site. Please share some experience or pointers to help me get started. Thanks a lot!
===========
Follow up question:
I saw Googlebot can now run Javascript. I want to understand a bit more of this. When a specific url of my SPA app is opened, it will do some network query (XHR request) for a few seconds and then the page content will be displayed. In this case, will GoogleBot wait for the http response?
I saw some tutorial says we need to prepare static html specifically for Search Engines. If I only want to deal with Google, does it mean I don't have to serve static html anymore because Google can run Javascript?
Thanks again.
If a search engine should come across your JavaScript application then we have the permission to redirect the search engine to another URL that serves the fully rendered version of the page.
For this job
You can either use this tool by Thomas Davis available on github
SEOSERVER
Or
you can use the code below which does the same job as above this code is also available here
Implementation using Phantom.js
We can setup a node.js server that given a URL, it will fully render the page content. Then we will redirect bots to this server to retrieve the correct content.
We will need to install node.js and phantom.js onto a box. Then start up this server below. There are two files, one which is the web server and the other is a phantomjs script that renders the page.
// web.js
// Express is our web server that can handle request
var express = require('express');
var app = express();
var getContent = function(url, callback) {
var content = '';
// Here we spawn a phantom.js process, the first element of the
// array is our phantomjs script and the second element is our url
var phantom = require('child_process').spawn('phantomjs',['phantom-server.js', url]);
phantom.stdout.setEncoding('utf8');
// Our phantom.js script is simply logging the output and
// we access it here through stdout
phantom.stdout.on('data', function(data) {
content += data.toString();
});
phantom.on('exit', function(code) {
if (code !== 0) {
console.log('We have an error');
} else {
// once our phantom.js script exits, let's call out call back
// which outputs the contents to the page
callback(content);
}
});
};
var respond = function (req, res) {
// Because we use [P] in htaccess we have access to this header
url = 'http://' + req.headers['x-forwarded-host'] + req.params[0];
getContent(url, function (content) {
res.send(content);
});
}
app.get(/(.*)/, respond);
app.listen(3000);
The script below is phantom-server.js and will be in charge of fully rendering the content. We don't return the content until the page is fully rendered. We hook into the resources listener to do this.
var page = require('webpage').create();
var system = require('system');
var lastReceived = new Date().getTime();
var requestCount = 0;
var responseCount = 0;
var requestIds = [];
var startTime = new Date().getTime();
page.onResourceReceived = function (response) {
if(requestIds.indexOf(response.id) !== -1) {
lastReceived = new Date().getTime();
responseCount++;
requestIds[requestIds.indexOf(response.id)] = null;
}
};
page.onResourceRequested = function (request) {
if(requestIds.indexOf(request.id) === -1) {
requestIds.push(request.id);
requestCount++;
}
};
// Open the page
page.open(system.args[1], function () {});
var checkComplete = function () {
// We don't allow it to take longer than 5 seconds but
// don't return until all requests are finished
if((new Date().getTime() - lastReceived > 300 && requestCount === responseCount) || new Date().getTime() - startTime > 5000) {
clearInterval(checkCompleteInterval);
console.log(page.content);
phantom.exit();
}
}
// Let us check to see if the page is finished rendering
var checkCompleteInterval = setInterval(checkComplete, 1);
Once we have this server up and running we just redirect bots to the server in our client's web server configuration.
Redirecting bots
If you are using apache we can edit out .htaccess such that Google requests are proxied to our middle man phantom.js server.
RewriteEngine on
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
RewriteRule (.*) http://webserver:3000/%1? [P]
We could also include other RewriteCond, such as user agent to redirect other search engines we wish to be indexed on.
Though Google won't use _escaped_fragment_ unless we tell it to by either including a meta tag; <meta name="fragment" content="!">or using #! URLs in our links.
You will most likely have to use both.
This has been tested with Google Webmasters fetch tool. Make sure you include #! on your URLs when using the fetch tool.
Related
Using Next.js , I currently have an app with a single entry point in the form of /pages/[...slug]/index.ts
It contains a getServerSideProps function which analyses the slug and decide upon a redirection
In some cases a redirection is needed, but it will always be towards a page that can be statically rendered. Example: redirect /fr/uid towards /fr/blog/uid which can be static.
In other cases the slug already is the url of a page that can be static.
How can I mix this dynamic element with a static generation of all pages?
Thanks a lot for your help!
If I understood you problem correctly, you cannot use getServerSideProps if you are going to export a static site.
You have two solutions:
Configure your redirection rules in your web hosting solution (i.e. Amazon S3/CloudFront).
Create client-side redirects (when _app.tsx mounts you can check if router.asPath matches any of the redirection you would like to have configured.
Please remember that the first solution is more correct (as 301 redirects from the browser) for SEO purposes.
EDIT: #juliomalves rightly pointed out OP is looking at two different things: redirection, and hybrid builds.
However, question should be clarified a bit more to really be able to solve his problem.
Because you will need to host a web-server for SSR, you can leverage Next.js 9.5 built-in redirection system to have permanent server-side redirects.
When it comes to SSR vs SSG, Next.js allows you to adopt a hybrid approach, by giving you the possibility of choosing with Data Fetching strategy to adopt.
In case you are using AWS CloudFront, then you can redirect with CloudFront Functions.
CloudFront Functions is ideal for lightweight, short-running functions for use cases like the following:
URL redirects or rewrites – You can redirect viewers to other pages based on information in the request, or rewrite all requests from one path to another.
Here is what we are using to redirect clients (e.g. Native App, Google search index, etc.) to new location when NextJS page was moved or removed.
// NOTE: Choose "viewer request" for event trigger when you associate this function with CloudFront distribution.
function makeRedirectResponse(location) {
var response = {
statusCode: 301,
statusDescription: 'Moved Permanently',
headers: {
'location': { value: location }
}
};
return response;
}
function handler(event) {
var mappings = [
{ from: "/products/decode/app.html", to: '/products/decode.html' },
{ from: "/products/decode/privacy/2021_01_25.html", to: '/products/decode/privacy.html' }
];
var request = event.request;
var uri = request.uri;
for (var i = 0; i < mappings.length; i++) {
var mapping = mappings[i]
if (mapping.from === uri) {
return makeRedirectResponse(mapping.to)
}
}
return request;
}
I am trying to implement google custom search in an angular js website.
When I click on the search button it does not display me anything, but the url is updated to the url.
I have followed the steps mentioned in the documentation by google.
I am not sure what I am doing wrong?
My search bar is located on the home page as -
<gcse:searchbox-only enableAutoComplete="true" resultsUrl="#/searchresult" lr="lang_en" queryParameterName="search"></gcse:searchbox-only>
my search result has -
<gcse:searchresults-only lr="lang_en"></gcse:searchresults-only>
Any input is much appreciated.
Thanks,
You may have more than one problem happening at the same time...
1. Query Parameter mismatch
Your searchresults-only does not match the queryParameterName specified on gcse:searchbox-only.
Index.html
<gcse:searchresults-only queryParameterName="search"></gcse:searchresults-only>
Search.html
<gcse:searchresults-only queryParameterName="search"></gcse:searchresults-only>
2. Angular.js is blocking the flow of Google CSE
Under normal circumstances, Google Search Element will trigger an HTTP GET with the search parameter. However, since you are dealing with a one-page application, you may not see the query parameter. If that suspicion is true when you target resultsUrl="#/searchresult", then you have two options:
Force a HTTP GET on resultsUrl="http://YOURWEBSITE/searchresult". You may have to match routes, or something along those lines in order to catch the REST request (Ember.js is really easy to do so, but I haven't done in Angular.js yet.)
Use JQuery alongside Angular.js to get the input from the user on Index.html and manually trigger a search on search.html. How would you do it? For the index.html you would do something like below and for the results you would implement something like I answered in another post.
Index.html
<div>GSC SEARCH BUTTON HOOK: <strong><div id="search_button_hook">NOT ACTIVATED.</div></strong></div>
<div>GSC SEARCH TEXT: <strong><div id="search_text_hook"></div></strong></div>
<gcse:search ></gcse:search>
Index.js
//Hook a callback into the rendered Google Search. From my understanding, this is possible because the outermost rendered div has id of "___gcse_0".
window.__gcse = {
callback: googleCSELoaded
};
//When it renders, their initial customized function cseLoaded() is triggered which adds more hooks. I added comments to what each one does:
function googleCSELoaded() {
$(".gsc-search-button").click(function() {
$("#search_button_hook").text('HOOK ACTIVATED');
});
$("#gsc-i-id1").keydown(function(e) {
if (e.which == 13) {
$("#enter_keyboard_hook").text('HOOK ACTIVATED');
}
else{
$("#search_text_hook").text($("#gsc-i-id1").val());
}
});
}
(function() {
var cx = '001386805071419863133:cb1vfab8b4y';
var gcse = document.createElement('script');
gcse.type = 'text/javascript';
gcse.async = true;
gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(gcse, s);
})();
I have a live version of the index.html code, but I don't make promises that will be permanently live since it is hosted in my NDSU FTP.
I'm working on an AngularJS project with the Play Framework 2.2. I'm supposed to develop a mobile version for the web application (not responsive, its part of a given uni project). For the desktop version I'm loading the index page with:
def index(any: String) = Assets.at(path = "/public", file = "app/html/index.html")
which works fine. Detection of the mobile browser works as well by examining the user agent in a Scala Action.
I changed the above code as follows to get the request header:
def index(any: String) = Action { implicit request: RequestHeader =>
if(isMobile(request)) {
// result for mobile version
}
else //result for desktop version
}
However, I don't know how to serve the different asset files as result type.
Any help is appreciated.
If I understand your question correctly, you wish to serve different files from Assets.at() based on your isMobile test, but can't work out how to get the types to line up?
Assets.at() returns an Action[AnyContent] which is at its simplest a function from Request[AnyContent] to Future[Result].
So knowing this, we just need a couple of tweaks to your index function and everything fits:
def index(any: String) = Action.async { request: Request[AnyContent] =>
if(isMobile(request)) {
Assets.at(path = "/public", file = "mobile.html").apply(request)
} else {
Assets.at(path = "/public", file = "desktop.html")(request)
}
}
Explanations:
The inner call returns a Future[Result] so we've become an Action.async
implicit is not needed here so I dropped it
An Action needs to be given a Request not a RequestHeader so I changed that
I'm showing both .apply(request) and just (request) - they are exactly the same
I am new to dart and I have been trying to figure out how to use the googleapis library to update a calendars events, then display the calendar/events on a webpage.
So far I have this code that I was hoping would just change the #text id's text to a list of events from the selected calendars ID:
import 'dart:html';
import 'package:googleapis/calendar/v3.dart';
import 'package:googleapis_auth/auth_io.dart';
final _credentials = new ServiceAccountCredentials.fromJson(r'''
{
"private_key_id": "myprivatekeyid",
"private_key": "myprivatekey",
"client_email": "myclientemail",
"client_id": "myclientid",
"type": "service_account"
}
''');
const _SCOPES = const [CalendarApi.CalendarScope];
void main() {
clientViaServiceAccount(_credentials, _SCOPES).then((http_client) {
var calendar = new CalendarApi(http_client);
String adminPanelCalendarId = 'mycalendarID';
var event = calendar.events;
var events = event.list(adminPanelCalendarId);
events.then((showEvents) {
querySelector("#text2").text = showEvents.toString();
});
});
}
But nothing displays on the webpage. I think I am misunderstanding how to use client-side and server-side code in dart... Do I break up the file into multiple files? How would I go about updating a calendar and displaying it on a web page with dart?
I'm familiar with the browser package, but this is the first time I have written anything with server-side libraries(googleapis uses dart:io so I assume it's server-side? I cannot run the code in dartium).
If anybody could point me in the right direction, or provide an example as to how this could be accomplished, I would really appreciate it!
What you might be looking for is the hybrid flow. This produces two items
access credentials (for client side API access)
authorization code (for server side API access using the user credentials)
From the documentation:
Use case: A web application might want to get consent for accessing data on behalf of a user. The client part is a dynamic webapp which wants to open a popup which asks the user for consent. The webapp might want to use the credentials to make API calls, but the server may want to have offline access to user data as well.
The page Google+ Sign-In for server-side apps describes how this flow works.
Using the following code you can display the events of a calendar associated with the logged account. In this example i used createImplicitBrowserFlow ( see the documentation at https://pub.dartlang.org/packages/googleapis_auth ) with id and key from Google Cloud Console Project.
import 'dart:html';
import 'package:googleapis/calendar/v3.dart';
import 'package:googleapis_auth/auth_browser.dart' as auth;
var id = new auth.ClientId("<yourID>", "<yourKey>");
var scopes = [CalendarApi.CalendarScope];
void main() {
auth.createImplicitBrowserFlow(id, scopes).then((auth.BrowserOAuth2Flow flow) {
flow.clientViaUserConsent().then((auth.AuthClient client) {
var calendar = new CalendarApi(client);
String adminPanelCalendarId = 'primary';
var event = calendar.events;
var events = event.list(adminPanelCalendarId);
events.then((showEvents) {
showEvents.items.forEach((Event ev) { print(ev.summary); });
querySelector("#text2").text = showEvents.toString();
});
client.close();
flow.close();
});
});
}
So I have this web-app using angularJS and nodeJS. I don't want to just use localhost to demo my project because it doesn't looks cool at all when I type "node server.js" and then go to localhost.....
Since I intend to use Firebase for the data, I have noticed that Firebase provides hosting. I tried it, but it seems to only host the index.html and not through/using server.js. I have customized files for the server to use/update. So, how can I tell Firebase Hosting to use my server and related files when hosting?
Is it possible to tell Firebase, hey, run "node server.js" to host my index.html?
I'm guessing by the way you are wording the question you want to see this site from "the internet".
Two routes you could go here.
a) Serve your index through Firebase hosting. Firebase only hosts assets. If your Angular app is being served through Node then you will need to change your architecture to be more SPA-ish
SPA-ish would be like an index bootstrap that interacts with the backend purely through API's.
You would host the API server on something more appropriate like through Nodejitsu.
b) Serve the whole thing through something like Nodejitsu (hosting platform) or your very own VM managed by a different kind of hosting company like BuyVM.net.
Another idea, is if your nodejs app is independent of the angularjs app (however they use shared data, and perform operations on that data model) you could separate the two and connect them only via firebase.
Firebase hosting -> index.html and necessary angularjs files.
Locally (your PC) -> server.js which just connects to firebase and trigger on changed data.
I have done this for a few projects and it's a handy way to access the outside world (internet) while maintaining some semblence of security by not opening ports blindly.
I was able to do this to control a chromecast at my house while at a friends house
Here's an example from my most recent project (I'm trying to make a DVR).
https://github.com/onaclov2000/webdvr/blob/master/app.js
var FB_URL = '';
var Firebase = require('firebase');
var os = require('os')
var myRootRef = new Firebase(FB_URL);
var interfaces = os.networkInterfaces();
var addresses = [];
for (k in interfaces) {
for (k2 in interfaces[k]) {
var address = interfaces[k][k2];
if (address.family == 'IPv4' && !address.internal) {
addresses.push(address.address)
}
}
}
// Push my IP to firebase
// Perhaps a common "devices" location would be handy
var ipRef = myRootRef.push({
"type": "local",
"ip": addresses[0]
});
myRootRef.on('child_changed', function(childSnapshot, prevChildName) {
// code to handle child data changes.
var data = childSnapshot.val();
var localref = childSnapshot.ref();
if (data["commanded"] == "new") {
console.log("New Schedule Added");
var schedule = require('node-schedule');
var date = new Date(data["year"], data["month"], data["day"], data["hh"], data["mm"], 0);
console.log(date);
var j = schedule.scheduleJob(date, function(channel, program, length){
console.log("Recording Channel " + channel + " and program " + program + " for " + length + "ms");
}.bind(null, data["channel"], data["program"], data["length"]));
localref.update({"commanded" : "waiting"});
}
});
When I change my "commanded" data at the FB_URL, to "new" (which can be accomplished by angularjs VERY Simply, using an ng-click operation for example) it'll schedule a recording for a particular date and time (not all actually functional at the moment).
I might be late but since 3 years have passed there is an solution available now from Firebase in the form of cloud functions
Its not straight forward but looks promising if one can refactor their code a bit