I have a a nginx server deployed in GoDaddy XLarge Cloud Server with 8GB RAM and 4 CPU. My setup for nginx is to proxy request to a Google App Engine application.
The problem is the nginx serves the static files too slow, sometimes breaking the connection rendering the website full or broken images, CSS and JS files. Now accessing the GAE app directly the static files are served really quick.
Here is my server nginx.conf file:
user www-data;
worker_processes 1;
worker_rlimit_nofile 20480; # worker_connections * 4
pid /run/nginx.pid;
events {
use epoll;
worker_connections 4096;
multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
client_body_timeout 12;
client_header_timeout 12;
keepalive_timeout 15;
send_timeout 10;
types_hash_max_size 2048;
# server_tokens off;
##
# Tweaks
# https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration
##
client_body_buffer_size 10K;
client_header_buffer_size 1k;
client_max_body_size 8m;
large_client_header_buffers 2 1k;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
##
## Proxy Settings
##
proxy_buffering off;
##
# nginx-naxsi config
##
# Uncomment it if you installed nginx-naxsi
##
#include /etc/nginx/naxsi_core.rules;
##
# nginx-passenger config
##
# Uncomment it if you installed nginx-passenger
##
#passenger_root /usr;
#passenger_ruby /usr/bin/ruby;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
Update
Here the Network graph for the application server:
Here the Network graph for the NGINX server (very slow):
What could be causing the slowness of nginx in this configuration?
Is this because GoDaddy Cloud Servers are slow? or something is really wrong with the NGINX configuration?
What configuration can make the proxy work fast?
Try optimise these:
1.worker process
since your have a 4 core CPU and you are serving quite a lot of files in one request, So the value of it should be at least 4, or the value of this
grep processor /proc/cpuinfo | wc -l
2.use CDN for common used js files.
I see you use some common library(jquery-1.10.2.min.js, Angular-1.4.3.js, fontawesome-webfont.wotf2 etc) directly served by GAE. These files take seconds to load. you should try to serve these files through CDN instead.
3.Do a test with Google PageSpeed Tools, It's very helpful.
Related
Build is working perfectly on my local PC with pm2, no errors at all. Every page loads perfectly, there are no 404 or 500 errors in fetching files. It's great! This is EXACTLY how I want it to run.
But when I try and deploy this on Ubuntu with pm2 I am getting two sets of errors:
I'll put screenshots here:
https://i.imgur.com/IdnEH7r.png
Written form:
Script_app-a44cfb7405f734c3.js
Script_buildManifest.js
Script_ssgManifest.js
Script_middlewareManifest.js
(And others) all are giving me a 500 Internal Server Error no matter what I do.
Attempted Solutions
I've tried many approaches and all of them end with this error/failure when I am navigating to my deployed website.
Upload manually with filezilla.
Git clone from my repository, build on the server (no build errors) and then deploy with pm2. No errors with pm2 either! But then I am given 404/500 errors.
I've tried this in different folders, I've tried it with a host of different commands. I am completely out of ideas and I've uploaded and tried to get my files on there and install packages and more.
Nginx error?
This might be nginx error? But the nginx settings work perfectly fine for a brand new "npx create-next-app#latest" Following this exact tutorial to the letter: https://www.youtube.com/watch?v=x6ci2iCckWc&t=658s&ab_channel=DigitalCEO
My nginx file
"server {
server_name specialservername.com;
gzip on;
gzip_proxied any;
gzip_types application/javascript application/x-javascript text/css text/javascript;
gzip_comp_level 5;
gzip_buffers 16 8k;
gzip_min_length 256;
location /_next/static/ {
alias /var/www/frontend/.next/static/;
expires 365d;
access_log off;
}
#EDITS
location ~ ^/_next/static/(.*)$ {
root /.next;
try_files "/static/$1" "/server/static/o$1" #proxy_pass;
}
#END EDITS
location / {
proxy_pass http://127.0.0.1:3000; #change to 3001 for second app, but make sure second nextjs app starts on new port in packages.json "start": "next start -p 3001",
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
add_header Access-Control-Allow-Origin *;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/specialservername.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/specialservername.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host =specialservername.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name specialservername.com;
return 404; # managed by Certbot
}"
What I was Expecting
The NextJS build to be deployed on this server no different than it is on my local machine. On my local machine it's BEAUTIFUL!
If you're seeing an "Internal Server Error" when trying to access your Next.js application on Ubuntu with nginx, it's likely that there's an issue with your configuration.
Here are a few things you can try:
Check your nginx error logs: Look in your nginx error logs (typically located in /var/log/nginx/error.log) for any error messages that might indicate what's causing the issue.
Check your Next.js logs: You should also check your Next.js logs (usually located in the .next directory of your application) for any error messages that might indicate what's causing the issue.
Check your Next.js configuration: Make sure your Next.js configuration is set up correctly for production deployment. You should make sure that your next.config.js file has the necessary settings for production deployment, such as setting target: 'server', configuring your build options, and setting your asset prefix if necessary.
Check your environment variables: Make sure any environment variables that your application depends on are set correctly on your Ubuntu server.
Check for permission: Make sure file, build files on server has enough permissions.
Also if everything from above works fine than try dockerizing your application with nginx and run on local then simply mimic the same on server(ubuntu) that would definatly give you some clue.
and lastly, don't panic. π
I used vite to create a react app with typescript and I follow a tutorial to get a good starter (video). I have another application in angular and it works fine with this approach I'm deploying it to kubernetes using ngix, but with vite i'm facing this error and I don't know the cause:
plint2dev.linguaserve.net/:16 Refused to apply style from 'https://plint2dev.linguaserve.net/assets/index.2518dafb.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
index.f2ba2231.js:1
Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
My nginx config and the repository code can be revised here. I tried all the alternatives provided here, but no one works for me.
My actual nginx.conf:
pid /run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 65535;
events {
multi_accept on;
worker_connections 65535;
}
http {
client_max_body_size 100M;
charset utf-8;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
server_tokens off;
# MIME
include mime.types;
default_type application/octet-stream;
# logging
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log warn;
# load configs
include /etc/nginx/conf.d/*.conf;
# linguaserve.net
server {
listen 80;
listen [::]:80;
server_name .linguaserve.net;
set $base /usr/share/nginx/html;
# security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header X-Request-ID $request_id;
location / {
try_files $uri $uri/ /index.html;
}
# . files
location ~ /\.(?!well-known) {
deny all;
}
# logging
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log warn;
# favicon.ico
location = /favicon.ico {
log_not_found off;
access_log off;
}
# gzip
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css text/xml application/json application/javascript application/rss+xml application/atom+xml image/svg+xml;
}
}
From here, I'm totally lost. Any help would be appreciated.
My dockerfile
FROM nginx
## Remove default Nginx website
RUN rm -rf /usr/share/nginx/html/*
## Copy our default nginx config
COPY nginx/default.conf /etc/nginx/nginx.conf
COPY /dist/* /usr/share/nginx/html/
RUN cd /usr/share/nginx
RUN ln -s /usr/share/nginx/html /usr/share/nginx/www
RUN ln -s /usr/share/nginx/html /etc/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
Finally, i solved it. It was a silly mistake. In my dockerfile, I copied all the files from dist, but no the directory structure.
Before:
dist
β index.12321.js
β index.12321.css
β index.html
β vite.svg.
Now:
dist
ββββassests
β β index.12321.js
β β index.12321.css
β index.html
β vite.svg.
The solution was change this command
COPY /dist/* /usr/share/nginx/html/
to this
COPY /dist/ /usr/share/nginx/html/
Removing the * solved the pain.
Ok so I have a stackoverflow ticket created as described here: Using nginx reverse proxy, how do you BLOCK all request except if request comes from my application
As an overview, it was having a frontend PHP app with Django rest api backend. To serve Django backend, nginx reverse proxy is placed in front of it.
Status: I have made this to work as I had described in my comment section.
Now my problem is, I have another project in which the frontend is ReactJS and the backend is Django which is fronted with an nginx reverse proxy. The problem I have is that, that nginx configuration works in PHP because PHP is server side, it however doesnt work with ReactJS because it is using the client computer IP accessing the website and not from where ReactJS is hosted. I need to filter this so that the backend api cant be used externally other than the server where ReactJS/node is hosted. Can anyone help me on this?
Here is my current configuration now (removing all those allow/deny parts in nginx):
server {
listen 80;
server_name ecommerce.abakada.tech;
location /static {
alias /vol/static;
}
location / {
uwsgi_pass ecommerce-backend:8000;
include /etc/nginx/uwsgi_params;
}
add_header Strict-Transport-Security max-age=31536000 always;
gzip on;
gzip_comp_level 2;
gzip_min_length 1024;
gzip_proxied expired no-cache no-store private auth;
gzip_types application/x-javascript application/javascript application/xml application/json text/xml text/css text;
client_max_body_size 10M;
client_body_timeout 12;
client_header_timeout 12;
reset_timedout_connection on;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
expires 1y;
access_log off;
log_not_found off;
}
I am using Nginx for my ReactJs project and used the below configurations to allow browser to cache only images not other files(HTML, JS & CSS). It is working fine for me. But some clients are facing cache issues. Latest html is loaded immediately. But they are getting old JS bundle file. I am using webpack to generate a production bundle. In the html, the bundle code is reflected immediately. Please check the sample HTML & Nginx configuration files below.
HTML:
<!DOCTYPE html>
<html lang="en">
<head>
...
<link href="css/main.css?76m85vt7qo00000" rel="stylesheet">
...
</head>
<body>
<div id="app"></div>
<script type="text/javascript" src="js/bundle.js?438400c3459a72d63b87"></script>
</body>
</html>
NGINX CONF:
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
##
# Virtual Host Configs
##
server {
listen 80;
listen [::]:80;
root /var/www/html;
index index.html index.htm;
server_name *.mydomain.com;
location / {
add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
add_header Pragma 'no-cache';
expires off;
add_header Last-Modified $date_gmt;
if_modified_since off;
etag off;
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri /index.html;
}
location ~* \.(jpg|jpeg|png|gif|ico)$ {
expires 30d;
log_not_found off;
add_header Pragma public;
add_header Cache-Control "public";
}
location ~* \.(js|css)$ {
expires -1;
add_header Cache-Control 'no-store';
add_header Last-Modified $date_gmt;
}
}
}
RESPONSE HEADER:
Note: This problem is not happening for all the time for the all users. This problem is resolved automatically by itself after some minutes or hours.
I have cleared the browser cache and tried again. No luck. Failed to get the latest version. How to instruct the Nginx to always serve the latest JS file? I have researched a lot in google but failed to get the reason & solution. Please anyone provide me the way to resolve this. Thanks in advance...
I was led down the "Nginx Cache Webpack Bundle" rabbit hole, only to realize that I had configured nginx to serve the build from an old build directory. So, even though I had the updated bundle and assets compiling correctly, it was still pointing to the old versions of them in the old directory.
It's not the direct solution to the OP's cache invalidation question (looks like unregistering service workers worked there), but an easy sanity-check for anyone else who stumbles up on this
I am using Nginx Server for surfing AngularJS based website (with HTML mode True to avoid # in URLs). Apart from this I have configured Varnish to reduce load on server. So Varnish runs on port 80 and Nginx runs on port 81. I am using prerender.io for SEO of AngularJS website.
The problem is, when prerender tries to recache the angular URLs, Nginx returns 404 error code.
My nginx.conf is as follow:
server {
listen 81 default_server;
listen [::]:81 default_server;
server_name abcxyzabc.com;
root /var/www/html/abcxyzabc/;
location / {
proxy_set_header X-Prerender-Token ****************;
set $prerender 0;
if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~* "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") {
set $prerender 0;
}
#resolve using Google's DNS server to force DNS resolution and prevent caching of IPs
resolver 8.8.8.8;
if ($prerender = 1) {
#setting prerender as a variable forces DNS resolution since nginx caches IPs and doesnt play well with load balancing
set $prerender "service.prerender.io";
rewrite .* /$scheme://$host$request_uri? break;
proxy_pass http://$prerender;
}
location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
expires 30d;
}
location /status {
# Turn on nginx stats
stub_status on;
# I do not need logs for stats
access_log off;
# Security: Only allow access from localhost IP #
allow 127.0.0.1;
# Send rest of the world to /dev/null #
deny all;
}
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_min_length 256;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/vnd.ms-fontobject application/x-font-ttf font/opentype image/svg+xml image/x-icon;
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
location ~ /\. {
deny all;
}
}
server {
listen 81;
server_name www.xyzabcxyz.com;
return 301 $scheme://xyzabcxyz.com$request_uri;
proxy_set_header X-Prerender-Token ***************;
}
With the above nginx configuration file, when prerender tries to recache the URLs, Nginx returns 404 error page. And after some time the same URL works fine. This case introduced after inserting prerender code in nginx configuration. Before the everything was good. Please help me out. Any solution/suggestion will be helpful.