python - mongodb做为数据库不稳定,请问怎么怎么分析原因?

 手机用户2502877051 发布于 2022-10-30 10:30

问题描述:pyspider运行一段时间(一天到三天不等)后调度器自动退出.重启pyspider能继续运行
运行环境:linux mint 17.2 通过pip安装的pyspider.使用mongodb做为数据库.
启动配置

{
  "taskdb": "mongodb+taskdb://localhost/taskdb",
  "projectdb": "mongodb+projectdb://localhost/projectdb",
  "resultdb": "mongodb+resultdb://localhost/resultdb",
  "webui": {
    "username": "admin",
    "password": "123456",
    "need-auth": true
  }
}

终端报错

[E 160522 09:37:38 scheduler:499] localhost:27017: [Errno -2] Name or service not known
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/pyspider/scheduler/scheduler.py", line 494, in run
        self.run_once()
      File "/usr/local/lib/python2.7/dist-packages/pyspider/scheduler/scheduler.py", line 1061, in run_once
        super(ThreadBaseScheduler, self).run_once()
      File "/usr/local/lib/python2.7/dist-packages/pyspider/scheduler/scheduler.py", line 478, in run_once
        self._update_projects()
      File "/usr/local/lib/python2.7/dist-packages/pyspider/scheduler/scheduler.py", line 93, in _update_projects
        for project in self.projectdb.check_update(self._last_update_project):
      File "/usr/local/lib/python2.7/dist-packages/pyspider/database/mongodb/projectdb.py", line 62, in check_update
        for project in self.get_all(fields=('updatetime', 'name')):
      File "/usr/local/lib/python2.7/dist-packages/pyspider/database/mongodb/projectdb.py", line 50, in get_all
        for each in self.collection.find({}, fields):
      File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 1090, in next
        if len(self.__data) or self._refresh():
      File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 1012, in _refresh
        self.__read_concern))
      File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 850, in __send_message
        **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 798, in _send_message_with_response
        exhaust)
      File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 809, in _reset_on_error
        return func(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/pymongo/server.py", line 88, in send_message_with_response
        with self.get_socket(all_credentials, exhaust) as sock_info:
      File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
        return self.gen.next()
      File "/usr/local/lib/python2.7/dist-packages/pymongo/server.py", line 152, in get_socket
        with self.pool.get_socket(all_credentials, checkout) as sock_info:
      File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
        return self.gen.next()
      File "/usr/local/lib/python2.7/dist-packages/pymongo/pool.py", line 539, in get_socket
        sock_info = self._get_socket_no_auth()
      File "/usr/local/lib/python2.7/dist-packages/pymongo/pool.py", line 573, in _get_socket_no_auth
        sock_info, from_pool = self.connect(), False
      File "/usr/local/lib/python2.7/dist-packages/pymongo/pool.py", line 512, in connect
        _raise_connection_failure(self.address, error)
      File "/usr/local/lib/python2.7/dist-packages/pymongo/pool.py", line 65, in _raise_connection_failure
        raise AutoReconnect(msg)
    AutoReconnect: localhost:27017: [Errno -2] Name or service not known
[I 160522 09:37:38 scheduler:505] scheduler exiting...
1 个回答
  • 应该是pyspider连接不上mangodb导致的。
    Name or service not known,可能是pyspider识别不了mangodb的HOST。

    1. 查看是否mangodb的原因,查看mangodb的日志,看是否有异常

    2. 查看是否网络原因,1. 把localhost换成127.0.0.1或者本机IP 2. 参考http://blog.csdn.net/sahenzhang/article/details/8181685

    2022-10-31 21:21 回答
撰写答案
今天,你开发时遇到什么问题呢?
立即提问
热门标签
PHP1.CN | 中国最专业的PHP中文社区 | PNG素材下载 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有