robots.txt只能阻止目录中的一种文件吗？

Question

I have a directory with 3 kind of files

1. thisdir/some/phpfile.php

2. thisdir/some/after-url-rewrite/1 (all numbers)

3. thisdir/something.htm (block any file end with .htm in thisdir/

I just want to block the 3rd type of files, since there are not existing any more, but search engine spider keep asking for them

Thanks for your help.

Answer 1

You could write in your robots.txt

disallow: thisdir/*.htm$

https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt