User-agent: * Disallow: /administrator/ Disallow: /cache/ Disallow: /components/ Disallow: /editor/ Disallow: /help/ Disallow: /images/ Disallow: /includes/ Disallow: /language/ Disallow: /mambots/ Disallow: /media/ Disallow: /modules/ Disallow: /templates/ Disallow: /installation/ # Don't crawl the bookmarks actual links Disallow: /web-links/view/ # This is the old bookmarks SEF urls, don't crawl Disallow: /component/option,com_bookmarks/ # Disallow anything that is not sef # So all the PDF / email links that we don't want # to be crawlled and give duplicates Disallow: /index.php? Disallow: /index2.php