在Python中遍历HTMLDOM-GoingthroughHTMLDOMinPython

6  

There are many different modules you could use. For example, lxml or BeautifulSoup.

您可以使用许多不同的模块。例如，lxml或BeautifulSoup。

Here's an lxml example:

这里有一个lxml的例子:

import lxml.html

mysite = urllib.request.urlopen('http://www.google.com').read()
lxml_mysite = lxml.html.fromstring(mysite)

description = lxml_mysite.xpath("//meta[@name='description']")[0] # meta tag description
text = description.get('content') # content attribute of the tag

>>> print(text)
"Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for."

And a BeautifulSoup example:

和一个BeautifulSoup的例子:

from bs4 import BeautifulSoup

mysite = urllib.request.urlopen('http://www.google.com').read()
soup_mysite = BeautifulSoup(mysite)

description = soup_mysite.find("meta", {"name": "description"}) # meta tag description
text = description['content'] # text of content attribute

>>> print(text)
u"Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for."

Notice how BeautifulSoup returns a unicode string, while lxml does not. This can be useful/hurtful depending on what is needed.

注意BeautifulSoup如何返回unicode字符串，而lxml则不返回。这可能是有用的/有害的，取决于需要什么。

1  

Check out the BeautifulSoup module.

看看美丽的汤模块。

from bs4 import BeautifulSoup
import urllib                                       
soup = BeautifulSoup(urllib.urlopen("http://google.com").read())

for link in soup.find_all('a'):
    print(link.get('href'))