刚刚学习了Python没几天,看了《Python网络数据采集》这本书,准备今天在网上试验着爬一个数据,网站是UTF-8编码的,可以在网站的文件头可以看出来

Python:UnicodeEncodeError: 'gbk' codec can't encode character '\xbb' in position 12305,以及中文乱码的解决方案-StubbornHuang Blog

所以我就按照书上的代码照着写了几行代码:

#__author__ = 'Administrat
#coding=utf-8
from urllib.request import  urlopen
from urllib  import request
from bs4 import BeautifulSoup
import requests
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}
req=request.Request("网址",headers=headers)
html=urlopen(req)
bsObj=BeautifulSoup(html.read())
print(bsObj.body)

结果马上出现UnicodeEncodeError: 'gbk' codec can't encode character '\xbb' in position 12305: illegal multibyte sequence.的这个错误,所以在网上搜索教程后发现了问题的所在,附上网址http://blog.csdn.net/jim7424994/article/details/22675759点击打开链接

结果按照上述的解决方案将代码改成,添加了:
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='utf-8')
整个代码修改为:

#__author__ = 'Administrat
#coding=utf-8
from urllib.request import  urlopen
from urllib  import request
from bs4 import BeautifulSoup
import requests
import sys
import io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='utf-8')
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}
req=request.Request("网址",headers=headers)
html=urlopen(req)
bsObj=BeautifulSoup(html.read())
print(bsObj.body)

改了代码之后,虽然没有报之前的那个错误,但是打印出来文字,英文文字没有出现乱码,但是中文出现乱码的情况:
Python:UnicodeEncodeError: 'gbk' codec can't encode character '\xbb' in position 12305,以及中文乱码的解决方案-StubbornHuang Blog
所以在详细查看了刚刚的博客之后,将上述代码中的:
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='utf-8')
改成
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='gb18030')
然后就完美中文显示