最简单的选择是to 并且找到所有内容：extract //body//text()join
''.join(sel.select("//body//text()").extract()).strip()
这里sel是一个Selector实例。
另一种选择是使用nltk的clean_html()：
>>> import nltk
>>> html = """
... <div class="post-text" itemprop="description">
...
... <p>I would like to have all the text visible from a website, after the HTML is rendered. I'm working in Python with Scrapy framework.
... With <code>xpath('//body//text()')</code> I'm able to get it, but with the HTML tags, and I only want the text. Any solution for this? Thanks !</p>
...
... </div>"""
>>> nltk.clean_html(html)
"I would like to have all the text visible from a website, after the HTML is rendered. I'm working in Python with Scrapy framework.\nWith xpath('//body//text()') I'm able to get it, but with the HTML tags, and I only want the text. Any solution for this? Thanks !"
另一种选择是使用BeautifulSoup的get_text()：
get_text()
如果只需要文档或标签的文本部分，则可以使用该get_text()方法。它以单个Unicode字符串的形式返回文档中或标签下的所有文本。
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup(html)
>>> print soup.get_text().strip()
I would like to have all the text visible from a website, after the HTML is rendered. I'm working in Python with Scrapy framework.
With xpath('//body//text()') I'm able to get it, but with the HTML tags, and I only want the text. Any solution for this? Thanks !
另一种选择是使用lxml.html的text_content()：
.text_content()
>>> import lxml.html
>>> tree = lxml.html.fromstring(html)
>>> print tree.text_content().strip()
I would like to have all the text visible from a website, after the HTML is rendered. I'm working in Python with Scrapy framework.
With xpath('//body//text()') I'm able to get it, but with the HTML tags, and I only want the text. Any solution for this? Thanks !
602392714
清零编程群