I set up prerender.io for CRA and it works well, but when bot hits URL without parameters it puts in the end of URL - string ".var"
I tried variations of (.*) but it seems not working. Any ideas?
Here is .htaccess file
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://%1/$1 [R=301,L]
RewriteCond %{HTTPS} !on
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
<IfModule mod_headers.c>
RequestHeader set X-Prerender-Token "TOKEN"
RequestHeader set X-Prerender-Version "prerender-apache#2.0.0"
</IfModule>
<IfModule mod_rewrite.c>
RewriteEngine On
<IfModule mod_proxy_http.c>
RewriteCond %{HTTP_USER_AGENT} googlebot|bingbot|yandex|baiduspider|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|whatsapp [NC,OR]
RewriteCond %{QUERY_STRING} _escaped_fragment_
RewriteCond %{REQUEST_URI} ^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff|\.svg))
RewriteRule ^(index\.html|index\.php)?(.*) https://service.prerender.io/%{REQUEST_SCHEME}://%{HTTP_HOST}/$2 [P,END]
</IfModule>
</IfModule>
Lately #MrWhite gave us another, better and simple solution - just add DirectoryIndex index.html to .htaccess file will do the same.
From the beginning I wrote that DirectoryIndex is working but NO!
It seems it's working when you try prerender.io, but in reality it was showing website like this:
and I had to remove it.
So it was not issue with .htaccess file, it was coming from the server.
What I did was I went into WHM->Apache Configurations->DirectoryIndex Priority and I saw this list
and yes that was it!
To fix I just moved index.html to the very top second comes index.html.var and after rest of them.
I don't know what index.html.var is for, but I did not risk just to remove it. Hope it helps someone who struggled as me.
Related
I have deployed a React app with React Router to my Bluehost server, and need to configure the htaccess file to redirect all of my routed URLs (/portfolio, /about, etc) to index.html instead of trying to fetch a new file from the server and throwing a 404.
I have read about countless similar problems in which the solution seems to be to add this into your htaccess file:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.html$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-l
RewriteRule . /index.html [L]
</IfModule>
I tried this, but I am still getting 404's when I try to visit any page of my site directly that isn't the homepage. I'm wondering if there is anything else in my existing htaccess file that is preventing the above code from working?
There was some code already in there from Bluehost, and I see another IfModule statement, so I'm wondering if that one is overwriting the first one. However I am afraid to edit it and break something, as it clearly says "do not edit." Here is my full htaccess code:
Header always set Content-Security-Policy: upgrade-insecure-requests
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.html$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-l
RewriteRule . /index.html [L]
</IfModule>
# php -- BEGIN cPanel-generated handler, do not edit
# Set the “ea-php74” package as the default “PHP” programming language.
<IfModule mime_module>
AddHandler application/x-httpd-ea-php74 .php .php7 .phtml
</IfModule>
# php -- END cPanel-generated handler, do not edit
# BEGIN WordPress
# The directives (lines) between "BEGIN WordPress" and "END WordPress" are
# dynamically generated, and should only be modified via WordPress filters.
# Any changes to the directives between these markers will be overwritten.
# END WordPress
Any ideas? I've double-checked that my BrowserRouter is set up correctly and also tried a few other htaccess configurations. I want to avoid using HashRouter or Node if possible but am getting frustrated. I can provide my React code as well if needed, but I'm pretty sure the error is not with the React setup.
You can create a virtual host file in the /etc/apache/sites-available folder and add this:
<VirtualHost *:8080>
ServerName example.com
DocumentRoot /var/www/httpd/example.com
<Directory "/var/www/httpd/example.com">
...
RewriteEngine on
# Don't rewrite files or directories
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
# Rewrite everything else to index.html to allow html5 state links
RewriteRule ^ index.html [L]
</Directory>
</VirtualHost>
This worked for me
I need to make some changes on an existing CakePHP project, but I can't run it on localhost because every time I try to access http://localhost:8080/{name-of-project}/ it redirects me to http://www.{name-of-project}.com.
Any help would be appreciated.
I also encountered this one when Im still using cakephp2 and still duable using cakephp3
from live .htaccess
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.mydomain\.com$ [NC]
RewriteCond %{HTTPS}s ^on(s)|off
RewriteRule ^(.*)$ http%1://www.mydomain.com/$1 [R=301,QSA,L]
RewriteRule ^$ webroot/ [L]
RewriteRule (.*) webroot/$1 [L]
</IfModule>
I change it to this one
.htaccess local
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^$ webroot/ [L]
RewriteRule (.*) webroot/$1 [L]
</IfModule>
Most likely in your .htaccess file there is a non-www to www redirect installed.
As Adam notes, it may be useful to clear your browser cache when working with redirects. A cached redirect would no longer hit your server configuration so any changes you do will not be visible.
I am trying to setup an apache2 redirect from https://www.dolgins.com to https://dolgins.com and cannot get it working correctly. I followed this guide and have the A records pointed correctly.
Here is a screenshot of my apache config
Here is my .htacess file located in the DocumentRoot folder:
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://%1/$1 [R=301,L]
Any ideas? Also, I want the URL to change. What would I need to do?
Thank you
Joseph
I have tried commenting this line present in congif/core.php,
// Configure::write('App.baseUrl', env('SCRIPT_NAME'));
But when I hit my url it gives me 404 error.
Make sure you are loading mod_rewrite correctly. You should see something like
#uncomment it by removing leading #
LoadModule rewrite_module libexec/apache2/mod_rewrite.so
Verify that your .htaccess files are actually in the right directories. Some operating systems treat files that start with ‘.’ as hidden and therefore won’t copy them.
#CAKEPHP root .htaccess
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^$ app/webroot/ [L]
RewriteRule (.*) app/webroot/$1 [L]
</IfModule>
#Cakephp App directory .htacess
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^$ webroot/ [L]
RewriteRule (.*) webroot/$1 [L]
</IfModule>
#Cakephp webroot directory .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ index.php [QSA,L]
</IfModule>
URL_REWRITING
You may fix it with your web-sersver (.htaccess rules for apache, rules in configuration file for nginx).
I am trying to set up Angular 1.5 app for server side rendering for the crawlers by using Prerender service.
And everything works fine for the inner pages but there is a problem with the main page's rendering - the crawler sees the 404 page instead of the main page.
I suppose there is a problem with some other rules in my .htaccess - except the rules for the Prerender, I use two other rules for all the pages:
rewriting urls without trailing slashes onto the urls with trailing slashes
rewriting urls with www on the urls without www
Will be appreciate for any tips!
Here is my .htaccess file for Apache serveer
RequestHeader set X-Prerender-Token "MyToken"
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www.example.com$ [NC]
RewriteRule ^(.*)$ http://example.com/$1 [R=301,L]
# If an existing asset or directory is requested go to it as it is
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d
RewriteRule ^ - [L]
RewriteCond %{REQUEST_URI} ^/$
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$
RewriteRule ^(.*)$ /snapshots/%1? [NC,L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*[^/])$ /$1/ [L,R=301]
<IfModule mod_proxy_http.c>
RewriteCond %{HTTP_USER_AGENT} Googlebot|bingbot|Googlebot-Mobile|Baiduspider|Yahoo|YahooSeeker|DoCoMo|Twitterbot|TweetmemeBot|Twikle|Netseer|Daumoa|SeznamBot|Ezooms|MSNBot|Exabot|MJ12bot|sogou\sspider|YandexBot|bitlybot|ia_archiver|proximic|spbot|ChangeDetection|NaverBot|MetaJobBot|magpie-crawler|Genieo\sWeb\sfilter|Qualidator.com\sBot|Woko|Vagabondo|360Spider|ExB\sLanguage\sCrawler|AddThis.com|aiHitBot|Spinn3r|BingPreview|GrapeshotCrawler|CareerBot|ZumBot|ShopWiki|bixocrawler|uMBot|sistrix|linkdexbot|AhrefsBot|archive.org_bot|SeoCheckBot|TurnitinBot|VoilaBot|SearchmetricsBot|Butterfly|Yahoo!|Plukkie|yacybot|trendictionbot|UASlinkChecker|Blekkobot|Wotbox|YioopBot|meanpathbot|TinEye|LuminateBot|FyberSpider|Infohelfer|linkdex.com|Curious\sGeorge|Fetch-Guess|ichiro|MojeekBot|SBSearch|WebThumbnail|socialbm_bot|SemrushBot|Vedma|alexa\ssite\saudit|SEOkicks-Robot|Browsershots|BLEXBot|woriobot|AMZNKAssocBot|Speedy|oBot|HostTracker|OpenWebSpider|WBSearchBot|FacebookExternalHit [NC,OR]
RewriteCond %{QUERY_STRING} _escaped_fragment_
# Only proxy the request to Prerender if it's a request for HTML
RewriteRule ^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff))(.*) http://service.prerender.io/http://example.com/$2 [P,L]
</IfModule>
# If the requested resource doesn't exist, use index.html
RewriteRule ^ /index.html
You have this section:
RewriteCond %{REQUEST_URI} ^/$
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$
RewriteRule ^(.*)$ /snapshots/%1? [NC,L]
Which will try to serve files from your /snapshots/ directory if _escaped_fragment_ is in the URL. That doesn't have anything to do with Prerender.io so you'll probably want to remove that section, as it could be the cause of the 404.
You're also checking Googlebot and Bingbot by their user agents which is a bad idea because they could penalize you for cloaking.