收录日期:2020/10/28 11:22:20 时间:2013-08-29 12:40:04 标签:php,.htaccess,search,web-crawler

I want to block search engines like Google and Yahoo from crawling user sub.domains like user.example.com, how can i do it?

Use robots.txt file in your web server.

So, in your subdomain put a robots.txt file that looks like this:

User-agent: *
Disallow: /

All sites have something specifically specified because that's what they don't want crawled by search engines...