Quantcast
Channel: Plesk Forum
Viewing all articles
Browse latest Browse all 45718

Block selected UserAgents for all vhosts

$
0
0
I want block a few bots and crawlers, which not respect robots.txt.
I think the best way is to create a new apache config-file in "/etc/httpd/conf.d/block_useragents.conf" and reload apache?

Content of block_useragents.conf:
<Directory "/var/www/vhosts/*">
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*(Agent1|Agent2|Agent3).*$ [NC]
RewriteCond %{REQUEST_URI} !^robots.txt
RewriteRule .* - [F]
</Directory>
Would this work for all vhosts? Is...

Block selected UserAgents for all vhosts

Viewing all articles
Browse latest Browse all 45718

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>