Файл robots.txt для Drupal

Необходимые правила в файле robots.txt для CMS Drupal

1. Скопировать содержимое в файл robots.txt,
2. example.com переименовать на название вашего домена,
3. Загрузить полученный файл в корневой каталог вашего сайта.

User-agent: Yandex
Disallow: /includes
Disallow: /misc/
Disallow: /modules
Disallow: /profiles
Disallow: /scripts
Disallow: /themes/
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /admin
Disallow: /comment/
Disallow: /filter/
Disallow: /node/add/
Disallow: /search
Disallow: /user*
Disallow: /archive/all/20*
Disallow: /archive/all$
Disallow: /?q=admin/
Disallow: /?q=comment/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/
Disallow: /*?sort*
Disallow: /*&sort*
Disallow: /tracker?
Host: example.com

User-agent: *
Disallow: /includes
Disallow: /misc/
Disallow: /modules
Disallow: /profiles
Disallow: /scripts
Disallow: /themes/
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /admin
Disallow: /comment/
Disallow: /filter/
Disallow: /node/add/
Disallow: /search
Disallow: /user*
Disallow: /archive/all/20*
Disallow: /archive/all$
Disallow: /?q=admin/
Disallow: /?q=comment/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/
Disallow: /*?sort*
Disallow: /*&sort*
Disallow: /tracker?
Sitemap: http://example.com/sitemap.xml