Checks two things: the note your website leaves for robots, and whether your web host turns them away at the door. About a third of blocks hide in the second one, which the usual robots.txt trick never shows you.