环境:Python 3.8.5

在使用pymysql去数据库查询数据时,出现了如下错误:

Exception ignored in: <function MysqlPoolClient.__del__ at 0x000001107810DC10>
Traceback (most recent call last):
  File "E:\test\work\myspider\utils\mysql_pool.py", line 39, in __del__
    self.dispose()
  File "E:\test\work\myspider\utils\mysql_pool.py", line 206, in dispose
    self.end('commit')
  File "E:\test\work\myspider\utils\mysql_pool.py", line 196, in end
    self._conn.commit()
  File "E:\test\venv1\lib\site-packages\dbutils\steady_db.py", line 435, in commit
    raise error  # re-raise the original error
  File "E:\test\venv1\lib\site-packages\dbutils\steady_db.py", line 426, in commit
    self._con.commit()
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 422, in commit
    self._read_ok_packet()
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 396, in _read_ok_packet
    pkt = self._read_packet()
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 658, in _read_packet
    raise err.InternalError(
pymysql.err.InternalError: Packet sequence number wrong - got 5 expected 1

最开始查找的相关资料,都是说多线程未加锁导致类似问题的出现,例如:python pymysql 多线程 读写数据库 报错 Packet sequence number wrong

但实际上,mysql_pool.py是基于pymysql封装的模块,已经注意了多线程锁的问题,所以考虑实际问题不是因为未加锁。

重新使用traceback模块打印异常,发现了如下错误:

Traceback (most recent call last):
  File "E:/test/work/myspider/spider_files/tianyancha_new/mysql_to_es.py", line 52, in get_one_from_company
    result = client.getOne(sql)
  File "E:\test\work\myspider\utils\mysql_pool.py", line 88, in getOne
    count = self._cursor.execute(sql)
  File "E:\test\venv1\lib\site-packages\dbutils\steady_db.py", line 598, in tough_method
    result = method(*args, **kwargs)  # try to execute
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\cursors.py", line 163, in execute
    result = self._query(query)
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\cursors.py", line 321, in _query
    conn.query(q)
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 505, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 724, in _read_query_result
    result.read()
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 1076, in read
    self._read_result_packet(first_packet)
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 1146, in _read_result_packet
    self._read_rowdata_packet()
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 1184, in _read_rowdata_packet
    rows.append(self._read_row_from_packet(packet))
  File "C:\Users\admin\AppData\Roaming\Python\Python38\site-packages\pymysql\connections.py", line 1203, in _read_row_from_packet
    data = converter(data)
TypeError: conversion from bytes to Decimal is not supported

看见“Decimal”想起来,要查询的数据中,有个字段是Decimal类型,然后继续查找相关资料:

pymysql TypeError: conversion from bytes to Decimal is not supported,这位大佬直接去把pymysql源码做了修改,应该也能解决问题(未实验)

后来参考了这个:python-Django錯誤:不支持從字節到十進位的轉換,大体意思是,由于use_unicode设置为False导致。

use_unicode是连接mysql时设置的一个参数,在python3中,该参数默认是True,但是在封装的mysql_pool.py中,use_unicode设置的为False,所以产生了上面的错误。重新将use_unicode改为True,即可避免上述问题的出现了。

参考:PyMySQL 官方文档

另外,当use_unicode设置为False时,查询结果里的文本对象都是字节类型;设置为True后,获取到的文本对象是字符串类型的数据。

Logo

为开发者提供学习成长、分享交流、生态实践、资源工具等服务,帮助开发者快速成长。

更多推荐