允许Google只访问一个文件夹并阻止其他网站

Question

EDIT: Doing this with robots.txt isn't good enough for me.

This code block entire site from google. I do not know how to keep this code functional but allow google to access one folder on the site.

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteRule ^(.*)$ - [F,L]

Does this works if i had 2 conditions?

Options +FollowSymLinks -MultiViews
# Turn mod_rewrite on
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_USER_AGENT} yahoo [OR]
RewriteCond %{REQUEST_URI} !^/allowed-folder/ [NC]
RewriteRule ^ - [F,L]

Answer 1

As suggested in comments create a file robots.txt under DOCUMENT_ROOT with following content:

User-Agent: googlebot
Disallow: /
Allow: /allowed-folder/

If you really want to handle this via mod_rewrite then enable mod_rewrite and .htaccess through httpd.conf and then put this code in your .htaccess under DOCUMENT_ROOT directory:

Options +FollowSymLinks -MultiViews
# Turn mod_rewrite on
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} Googlebot [NC]
RewriteCond %{REQUEST_URI} !^/allowed-folder/ [NC]
RewriteRule ^ - [F,L]