Well, in my opinion, you shouldn't need to test anything for the web crawlers. If it is related to search engine optimization then this is a funciton of marketing, not QA.
If you wanted a basic, bare bones, check of a web page then I would say you can check the META Keywords and description and make sure it is related to the content on the page (is the keywords must appear on the page). Also, ensure that the keywords appear prominently in the first 1-2 paragraphs of the article and the last paragraph of the article, also, if possible.
There is so much more to know, but unless they want you to become an SEO expert, and pay you as such, then I would keep it simple. Otherwise, make sure to score yourself some free training on the subject. [stupid is as stupid does] to your own skillset along the way. Who knows when it might become useful.
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.