.htaccess для роботов доменов и поддоменов для предотвращения сканирования - PullRequest
Новая
       7

.htaccess для роботов доменов и поддоменов для предотвращения сканирования

0 голосов
/ 02 мая 2018

domain.com - живой сервер

dev.domain.com - сервер разработки

Оба с: 

/public/robots.txt #which is for domain.com
/public/robots-dev.txt #which is for dev.domain.com
.htaccess

Как использовать тот же .htaccess для выбора определенных роботов, в зависимости от того, включен он в live или dev ?

Я пробовал это, но это не работает (часть htaccess - перенаправить не-https & www на https с не-www, пожалуйста, игнорируйте это): 

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{HTTPS} off [OR]
    RewriteCond %{HTTP_HOST} ^www\. [NC]
    RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC]
    RewriteRule ^ https://%1%{REQUEST_URI} [L,NE,R=301]
    RewriteRule ^(.*)$ public/$1 [L]
    RewriteRule ^robots\.txt$ /robots-%1\.txt [L] #this one
</IfModule>

Обновление: ниже публичный / .htaccess 

# GLOBAL RULES
# ============
# Directories Index protection
Options -Indexes

# Remove index.php from URL
<IfModule mod_rewrite.c>
    RewriteCond %{THE_REQUEST} ^GET.*index\.php [NC]
    RewriteRule (.*?)index\.php/*(.*) /$1$2 [R=301,NE,L]
</IfModule>

# Default Apache rules
<IfModule mod_rewrite.c>
    <IfModule mod_negotiation.c>
        Options -MultiViews
    </IfModule>
    RewriteEngine On

    # Redirect Trailing Slashes If Not A Folder...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)/$ /$1 [L,R=301]

    # Handle Front Controller...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^ index.php [L]

    # Handle Authorization Header
    RewriteCond %{HTTP:Authorization} .
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
</IfModule>

# Robots.txt
RewriteRule  ^(robots\.txt)$ /robots [NC,L]

# Iframe calling
<IfModule mod_headers.c>
    Header unset X-Frame-Options
</IfModule>

# GTMetrix - Enable gzip compression
<IfModule mod_deflate.c>
    # Compress HTML, CSS, JavaScript, Text, XML and fonts
    AddOutputFilterByType DEFLATE application/javascript
    AddOutputFilterByType DEFLATE application/rss+xml
    AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
    AddOutputFilterByType DEFLATE application/x-font
    AddOutputFilterByType DEFLATE application/x-font-opentype
    AddOutputFilterByType DEFLATE application/x-font-otf
    AddOutputFilterByType DEFLATE application/x-font-truetype
    AddOutputFilterByType DEFLATE application/x-font-ttf
    AddOutputFilterByType DEFLATE application/x-javascript
    AddOutputFilterByType DEFLATE application/xhtml+xml
    AddOutputFilterByType DEFLATE application/xml
    AddOutputFilterByType DEFLATE application/json
    AddOutputFilterByType DEFLATE font/opentype
    AddOutputFilterByType DEFLATE font/otf
    AddOutputFilterByType DEFLATE font/ttf
    AddOutputFilterByType DEFLATE image/svg+xml
    AddOutputFilterByType DEFLATE image/x-icon
    AddOutputFilterByType DEFLATE text/css
    AddOutputFilterByType DEFLATE text/html
    AddOutputFilterByType DEFLATE text/javascript
    AddOutputFilterByType DEFLATE text/plain
    AddOutputFilterByType DEFLATE text/xml

    # Remove browser bugs (only needed for really old browsers)
    BrowserMatch ^Mozilla/4 gzip-only-text/html
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
    Header append Vary User-Agent
</IfModule>
# GTMetrix - Enable gzip compression

# GTMetrix Browser Caching
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType image/jpg "access plus 1 week"
    ExpiresByType image/jpeg "access plus 1 week"
    ExpiresByType image/gif "access plus 1 week"
    ExpiresByType image/png "access plus 1 week"
    ExpiresByType image/svg "access plus 1 month"
    ExpiresByType text/css "access plus 1 week"
    ExpiresByType text/javascript "access plus 1 week"
    ExpiresByType application/pdf "access plus 1 month"
    ExpiresByType application/json "access plus 1 week"
    ExpiresByType application/javascript "access plus 1 week"
    ExpiresByType text/x-javascript "access plus 1 week"
    ExpiresByType application/x-shockwave-flash "access plus 1 month"
    ExpiresByType image/x-icon "access plus 1 year"
    ExpiresDefault "access plus 1 month"
</IfModule>
# GTMetrix Browser Caching

1 Ответ

0 голосов
/ 02 мая 2018

Вы можете использовать эти правила: 

RewriteEngine On

RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC]
RewriteRule ^ https://%1%{REQUEST_URI} [L,NE,R=301]

RewriteCond %{HTTP_HOST} ^dev\. [NC]
RewriteRule ^robots\.txt$ public/robots-dev.txt [L,NC]

RewriteRule ^(?!public/)(.*)$ public/$1 [L,NC]
...