diff options
| author | Adam T. Carpenter <atc@53hor.net> | 2022-09-26 15:05:02 -0400 |
|---|---|---|
| committer | Adam T. Carpenter <atc@53hor.net> | 2022-09-26 15:05:02 -0400 |
| commit | 48db6b7f53bd73a7d2d89acadb18f28b7351119f (patch) | |
| tree | a7efb24d9ca65571f3197dbcad736095efadb669 | |
| parent | 32b884758d7f1c678f74e7f0c993c89fcd44d5d8 (diff) | |
| download | 53hor-48db6b7f53bd73a7d2d89acadb18f28b7351119f.tar.xz 53hor-48db6b7f53bd73a7d2d89acadb18f28b7351119f.zip | |
fix: default deny for crawlers, allow index only
| -rw-r--r-- | robots.txt | 10 |
1 files changed, 2 insertions, 8 deletions
@@ -1,10 +1,4 @@ User-agent: * Crawl-delay: 120 -Disallow: /.git/ -Disallow: /cv.html -Disallow: /cv.php -Disallow: /drafts/ -Disallow: /feed.php -Disallow: /includes/ -Disallow: /payments/ -Disallow: /contact/ +Disallow: / +Allow: /index.php |