I can't speculate about future features, but robots.txt is probably something that's worth taking the time to set up correctly (especially if there are areas of your content that you don't want crawled). Even if Google did implement some sort of "robots.txt cross-submission", there are still many other crawlers and search engines that only look for robots.txt at the root of your site (since that's what the spec specifies). |