nginx, prerender and angular routing issue - angularjs

I am developing a dynamic angular website (using html5mode to remove the #). It uses Elasticsearch to grab some data and render the contents. nginx is responsible for the mapping to Elasticsearch and my website root.
In order to get the whole site SEO-capable, I am trying to integrate prerender, using a local instance (http://localhost:3000).
I am desperately trying to do apply the prerender magic in combination with an nginx location /angular_test. The default nginx prerender middleware config does not work for locations. Find below my nginx configration:
upstream elasticsearch {
zone elasticsearch 64K;
server localhost:9200;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name angular_test_server;
location /angular_test {
rewrite_log on;
error_log /var/log/nginx/localhost.error_log notice;
set $prerender 0;
if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~ "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff)") {
set $prerender 0;
}
if ($prerender = 1) {
rewrite .* /$scheme://$host$request_uri?;
proxy_pass http://localhost:3000;
break;
}
# support html5mode of angular (fallback to index.html and let angular do the routing)
try_files $uri $uri/ /angular_test/index.html;
}
location /es/ {
rewrite /es/(.*) /$1 break;
proxy_pass http://elasticsearch;
proxy_redirect http://elasticsearch /es/;
}
}
When I access http://localhost/angular_test/search?_escaped_fragment_= I get a 404 because nginx tries to load the file /var/html/www/angular_test/search. This should be covered by angulars routing (see try_files doing a try on the index.html). http://localhost/angular_test?_escaped_fragment_= works just fine
When I use something like the following (as propsed here https://gist.github.com/thoop/8165802), it does not even get into the #prerender location part.
location /angular_test {
# support html5mode of angular (fallback to index.html and let angular do the routing)
try_files $uri $uri/ /angular_test/index.html #prerender;
}
Is there any way to get this setup running with prerender? I have the feeling that I do not use nginx in the right way here...

I figured out the solutions. If is really evil in nginx locations!
It works when the if(...) logic is moved out of the location and to the server. Here is my final configuration:
upstream elasticsearch {
zone elasticsearch 64K;
server localhost:9200;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name angular_test_server;
set $prerender 0;
if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~ "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff)") {
set $prerender 0;
}
location /angular_test {
rewrite_log on;
error_log /var/log/nginx/localhost.error_log notice;
#resolve using Google's DNS server to force DNS resolution and prevent caching of IPs
resolver 8.8.8.8;
if ($prerender = 1) {
#setting prerender as a variable forces DNS resolution since nginx caches IPs and doesnt play well with load balancing
#set $prerender "$host:3000";
rewrite .* /$scheme://$host$request_uri? break;
proxy_pass http://localhost:3000;
}
# support html5mode of angular (fallback to index.html and let angular do the routing)
try_files $uri $uri/ /angular_test/index.html;
}
location /es/ {
rewrite /es/(.*) /$1 break;
proxy_pass http://elasticsearch;
proxy_redirect http://elasticsearch /es/;
}
}

Related

nginx - serving backend and frontend on different endpoints and same domain

I have installed nginx on OS X with php7.1, mysql and so on... and the basic testing example is working.
When I try to configure nginx to serve Laravel backend on user.domain.dev/api/... and Angular frontend on user.domain.dev/... I am getting 404 or 403. Nginx error log is mostly
/Users/name/Projects/domain.dev/api.domain.dev/" is forbidden
or
/Users/name/Projects/domain.dev/frontend.domain.dev/build/index.php" failed (2: No such file or directory)
I can't seem to understand nginx's location directive and so they're pointing to a wrong directories. Thanks for any advices or guidance.
My config is:
nginx.conf
user name staff;
worker_processes auto;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
ssl_certificate ssl/nginx.crt;
ssl_certificate_key ssl/nginx.key;
access_log /usr/local/var/log/nginx/access.log;
error_log /usr/local/var/log/nginx/error.log;
sendfile on;
keepalive_timeout 5;
gzip on;
include servers/*.conf;
}
servers/domain.dev.conf
server {
listen 80;
listen 443 ssl http2 default_server;
server_name domain.dev *.domain.dev www.domain.dev;
charset utf-8;
# removes trailing slashes (prevents SEO duplicate content issues)
if (!-d $request_filename)
{
rewrite ^/(.+)/$ /$1 permanent;
}
# enforce NO www
if ($host ~* ^www\.(.*))
{
set $host_without_www $1;
rewrite ^/(.*)$ $scheme://$host_without_www/$1 permanent;
}
##
## Backend HTTP server
##
location /api {
root /Users/name/Projects/domain.dev/api.domain.dev;
index index.php;
try_files $uri $uri/ /index.php?$query_string;
location ~ \.php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
try_files $fastcgi_script_name =404;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi.conf;
}
}
##
## Frontend HTTP server
##
location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ {
access_log off;
expires max;
}
location = /favicon.ico {
access_log off;
log_not_found off;
}
location = /robots.txt {
access_log off;
log_not_found off;
}
location / {
root /Users/name/Projects/domain.dev/frontend.domain.dev/build;
index index.html;
}
location ~ /\. {
deny all;
}
}
The main problem is an inconsistent use of the root directive. You have two document roots, one for each application, but only specified in two of your locations. There is no root specified at the server { ... } block level, therefore if (!-d $request_filename) is meaningless, location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ will always result in a 404 response, etc.
Read this document to understand how nginx processes a request.
But you should probably set the frontend root in the server block and override it with the backend root in the location /api block.
server {
root /Users/name/Projects/domain.dev/frontend.domain.dev/build;
location / { ... }
location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ { ... }
location ^~ /api {
root /Users/name/Projects/domain.dev/api.domain.dev;
...
}
}
This would place the backend document root at: /Users/name/Projects/domain.dev/api.domain.dev/api -- note that the URI is always appended to the root.
Note also, the ^~ modifier is used to make the prefix location take precedence over regular expression locations at the same level. See this document for details.
I do not like the naked if blocks in your design. The if (!-d $request_filename) should probably be replaced with a try_files directive, and if ($host ~* ^www\.(.*)) should probably be replaced by using a separate server block. See this document for more.
The try_files statement in your location /api block contains an incorrect default URI, it should probably be /api/index.php?$query_string.

Prerender.io conflicting with CRUD in AngularJs application

I am using prerender.io to make my AngularJS website crawlable.
However, since I started using prerender.io, the majority of my CRUD requests (e.g. update, delete) return an nginx 405 Not Allowed response.
Original (working) nginx location block (before using prerender.io):
location / {
try_files $uri $uri/ /index.php?$query_string;
}
Current location block (Fetch as Google correctly displays static html, CRUD does not work):
location / {
try_files $uri #prerender;
}
Current prerender block
location #prerender {
proxy_set_header X-Prerender-Token D3Ft1yU7Ho3nNMwvgQQO;
set $prerender 0;
if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~ "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff)") {
set $prerender 0;
}
resolver 8.8.8.8;
if ($prerender = 1) {
set $prerender "service.prerender.io";
rewrite .* /$scheme://$host$request_uri? break;
proxy_pass http://$prerender;
}
if ($prerender = 0) {
rewrite .* / break;
}
}
I believe that the problem is caused either by my current location block or my rewrite statement.
I tried several alternatives for both. e.g.:
#CRUD works, FETCH as google does not work
try_files $uri $uri/ #prerender /index.php?$query_string;
#CRUD works, FETCH as google does not work
try_files $uri #prerender $uri/ /index.php?$query_string;
#CRUD does not work, FETCH does work
try_files $uri $uri/ /index.php?$query_string #prerender;
rewrite .* /index.html break;
rewrite .* /?$query_string break;
rewrite .* /index.php?$query_string break;
etc.
However, in all these configurations EITHER crud works OR prerendering the HTML correctly, never both. Any ideas how I can fix this?
Surprisingly, after the latest Ubuntu update, the last rewrite mentioned in my question (rewrite .* /index.php?$query_string;) did the trick. I am not sure why.

CakePHP 2 in a subdirectory with ngix - 301 redirect

I am having a really hard time getting nginx working with CakePHP in a subdirectory.
I'm using nginx because I am only familiar with Apache and I want to learn something new. My goal is to host CakePHP in a subdirectory.
My /etc/nginx/sites-available/default file looks like this:
server {
listen 0.0.0.0:80;
root /var/www/;
index index.html index.php;
include /etc/nginx/include/php;
error_page 404 = /404.html;
location / {
root index.html;
index index.php index.html index.htm;
autoindex on;
try_files $uri $uri/ /index.php;
}
location /randodb {
if (!-e $request_filename) {
rewrite ^/randodb(.+)$ /randodb/app/webroot/$1 last;
break;
}
}
location /randodb/app/webroot {
if (!-e $request_filename) {
rewrite ^/randodb/app/webroot/(.+)$ /randodb/app/webroot/index.php?url=$1 last;
break;
}
}
}
server {
listen 0.0.0.0:443;
root /var/www/;
index index.html index.php;
fastcgi_param HTTPS on;
include /etc/nginx/include/ssl;
include /etc/nginx/include/php;
error_page 404 = /404.html;
}
No matter what I do, any requests in the /randodb folder get a 301 redirect.
It always redirects me to http://nginx-php-fastcgi/randodb/
If you want to see this behavior in action: http://www.matgilbert.com/randodb
First this is the only block you need
location ~ /randodb(.*) {
root /var/www/randodb/app/webroot;
try_files $1 $1/ /index.php?url=$1;
}
above in the / block, you specified root index.html; what is this supposed to do, cause I believe this is incorrect, root is supposed to be a directory.
About this block
fastcgi_param HTTPS on;
include /etc/nginx/include/ssl;
include /etc/nginx/include/php;
is php working ? never seen it before, are u using something different than fast-cgi or fpm?
then the ssl block, here's my ssl block on my server
server {
listen 443 ssl;
ssl_certificate /etc/ssl/name.crt;
ssl_certificate_key /etc/ssl/name.key;
server_name example.com
location / {
# the rest of my config
}
}

how to configure cakephp in nginx

Currently i am working on cakephp with nginx. I setup a cakephp environment on a Centos server running Nginx with Fact CGI. The problem is that I cannot get the rewrite rules to setup correct in my vhost so that cake renders pages correctly i.e. with styling and so on.
My .conf file is as -
#The default server
server {
listen 80 default_server;
server_name 1 23.123.123.123;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.php index.html index.htm;
}
error_page 404 /404.html;
location = /404.html {
root /usr/share/nginx/html;
}
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ \.php$ {
root html;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /usr/share/nginx/html$fastcgi_script_name;
include fastcgi_params;
}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
location ~ /\.ht {
deny all;
}
}
Luchomolina's answer below got me started in the right direction. However, CSS and JS files stopped working on URLs that were not top-level URLs, e.g. /posts worked but /posts/title123 didn't.
This is what ended up working for me:
location / {
if (-f $request_filename) {
break;
}
if (!-f $request_filename) {
rewrite ^/(.+)$ /index.php?url=$1 last;
break;
}
set $new_uri $uri;
}
use the blow code
location / {
if (-f $request_filename) {
break;
}
if (!-f $request_filename) {
rewrite ^/(.+)$ /index.php?url=$1 last;
break;
}
set $new_uri $uri;
}
This works for me:
server {
listen 80 ;
server_name example.com;
root /www/example.com/app/webroot;
index index.html index.php;
location / {
try_files $uri $uri/ /index.php?$uri&$args;
set $new_uri $uri;
}
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
# NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param PATH_INFO $new_uri;
}
location ~ /(\.ht|\.git|\.svn) {
deny all;
}
}
The $new_uri rules were added to make some special routes work on my site, maybe that won't apply to you, but note that CakePHP assumes PATH_INFO is set, so those rules won't harm leaving it there.
Hope it helps.

CakePHP in a subdirectory using nginx (Rewrite rules?)

I managed to get this to work a while back, but on returning to the cakephp project I had started it seems that whatever changes I've made to nginx recently (or perhaps a recent update) have broken my rewrite rules.
Currently I have:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name localhost;
location / {
root html;
index index.php index.html index.htm;
}
location /basic_cake/ {
index index.php;
if (-f $request_filename) {
break;
}
if (!-f $request_filename) {
rewrite ^/basic_cake/(.+)$ /basic_cake/index.php?url=$1 last;
break;
}
}
location /cake_test/ {
index index.php;
if (-f $request_filename) {
break;
}
if (!-f $request_filename) {
rewrite ^/cake_test/(.+)$ /cake_test/index.php?url=$1 last;
break;
}
}
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ \.php$ {
root html;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
}
server {
listen 8081;
server_name localhost;
root /srv/http/html/xsp;
location / {
index index.html index.htm index.aspx default.aspx;
}
location ~ \.(aspx|asmx|ashx|asax|ascx|soap|rem|axd|cs|config|dll)$ {
fastcgi_pass 127.0.0.1:9001;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
}
}
The problem that I have is that the css and images will not load from the webroot. Instead if I visit http://localhost/basic_cake/css/cake.generic.css, I get a page which tells me:
CakePHP: the rapid development php
framework Missing Controller
Error: CssController could not be
found.
Error: Create the class CssController
below in file:
app/controllers/css_controller.php
Notice: If you want to customize this
error message, create
app/views/errors/missing_controller.ctp
CakePHP: the rapid development php
framework
Does anybody have any ideas on how to fix this?
(I posted this on ServerFault, but I have this feeling that not many people check that site in comparison to this one, so I probably shouldn't have bothered...)
There is a much easier way to do this using alias and try_files directives. I've started with a simple configuration with working PHP and added a Cake project in the path /cakeproject on the server:
root /var/www;
index index.php;
location /cakeproject {
alias /var/www/cakeproject/app/webroot;
try_files $uri $uri/ /cakeproject/app/webroot/index.php;
}
location ~ \.htaccess {
deny all;
}
location ~ \.php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include /etc/nginx/fastcgi_params;
}
The cake project now works perfectly from http://thedomain.com/cakeproject/
I've managed to solve this by adding App.base parameter to the cakephp configuration instead of using the App.baseUrl (looked the code in dispatcher.php).
So let's say I have the cakephp copy located in /var/www/html/cakeprj and my WWWROOT is /var/www/html:
nginx host configuration
# that's for all other content on the web host
location / {
root /var/www/html;
autoindex off;
index index.php index.html index.htm;
...
}
# that's for cakephp
location /cakeprj {
rewrite ^/cakeprj$ /cakeprj/ permanent;
rewrite ^/cakeprj/(.+)$ /$1 break;
root /var/www/html/cakeprj/app/webroot;
try_files $uri /$uri/ #cakephp;
}
# that's for all other php scripts on the web host
location ~ \.php$ {
root /var/www/html;
fastcgi_pass unix:/var/lib/fcgi/php-fcgi.socket;
...
include /etc/nginx/fastcgi_params;
}
# that's for cakephp execution
location #cakephp {
set $q $request_uri;
if ($request_uri ~ "^/cakeprj(.+)$") {
set $q $1;
}
fastcgi_param SCRIPT_FILENAME /var/www/html/cakeprj/app/webroot/index.php;
fastcgi_param QUERY_STRING url=$q;
fastcgi_pass unix:/var/lib/fcgi/php-fcgi.socket;
include /etc/nginx/fastcgi_params;
}
cakephp configuration in app/config/core.php
Configure::write('App.base', '/cakeprj');
Configure::write('App.baseUrl', '/cakeprj/'); // seems like it doesn't matter anymore
...and voila - you get nginx serving cakephp static files correctly, url passing to the cakephp dispatcher correctly and cakephp generating urls correctly as well.
P.S. if your nginx doesn't support try_files I believe its configuration can be rewritten with if condition and another rewrite.
In my case, what I did was the following:
location /cakeproj {
rewrite_log on;
error_log /var/log/nginx/notice.log notice; # just for debuggin
if (-f $request_filename) {
break;
}
# Avoid recursivity
if ($request_uri ~ /webroot/index.php) {
break;
}
rewrite ^/cakeproj$ /cakeproj/ permanent;
rewrite ^/cakeproj/app/webroot/(.*) /cakeproj/app/webroot/index.php?url=$1 last;
rewrite ^/cakeproj/(.*)$ /cakeproj/app/webroot/$1 last;
}
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
# NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
fastcgi_pass unix:/var/run/php5-fpm.sock;
# Edit listen directive in /etc/php5/fpm/pool.d/www.conf
fastcgi_index index.php;
include fastcgi_params;
}
And it worked!!!
Right now I'm experiencing slowness, but I think is PHP and not nginx related
Cheers
Found a solution:
location /cakeprj {
rewrite ^/cakeprj(.+)$ /cakeprj/app/webroot$1 break;
try_files $uri $uri/ /cakeprj/index.php?$args;
}
where cakeprj is a cakephp directory.
Ref: http://jamesmcdonald.id.au/it-tips/cakephp-in-a-subdirectory-nginx

Resources