Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

部分ID爬取错误 #589

Open
ciiio opened this issue Jun 6, 2024 · 1 comment
Open

部分ID爬取错误 #589

ciiio opened this issue Jun 6, 2024 · 1 comment
Labels
failed 程序运行出错

Comments

@ciiio
Copy link

ciiio commented Jun 6, 2024

为了更好的解决问题,请认真回答下面的问题。等到问题解决,请及时关闭本issue。

  • 问:请您指明哪个版本运行出错(github版/PyPi版/全部)?

答:github

  • 问:您使用的是否是最新的程序(是/否)?

答:是

  • 问:爬取任意用户都会运行出错吗(是/否)?

答:否

  • 问:若只有爬特定微博时才出错,能否提供出错微博的weibo_id或url(非必填)?

答:5724107516

  • 问:若您已提供出错微博的weibo_id或url,可忽略此内容,否则能否提供出错账号的user_id及您配置的since_date,方便我们定位出错微博(非必填)?

答:

  • 问:如果方便,请您描述出错详情,最好附上错误提示。

答:我尝试在爬两个ID,上面提供的这个ID没法正常爬取,但是另一个ID可以正常爬取
报错如下:
(base) liangzhi@zhiliang14 ~ % python3 -m weibo_spider --output_dir="/Users/liangzhi/Desktop/微博爬虫"
'NoneType' object has no attribute 'xpath'
Traceback (most recent call last):
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/parser/index_parser.py", line 33, in get_user
user_id = self._get_user_id()
^^^^^^^^^^^^^^^^^^^
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/parser/index_parser.py", line 20, in _get_user_id
url_list = self.selector.xpath("//div[@Class='u']//a")
^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'xpath'
None


'NoneType' object has no attribute 'nickname'
Traceback (most recent call last):
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/spider.py", line 226, in _get_filepath
dir_name = self.user.nickname
^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'nickname'
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/writer/csv_writer.py", line 25, in init
with open(self.file_path, 'a', encoding='utf-8-sig',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
'NoneType' object has no attribute 'nickname'
Traceback (most recent call last):
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/spider.py", line 226, in _get_filepath
dir_name = self.user.nickname
^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'nickname'
'NoneType' object has no attribute 'dict'
Traceback (most recent call last):
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/spider.py", line 313, in get_one_user
self.write_user(self.user)
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/spider.py", line 137, in write_user
writer.write_user(user)
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/writer/txt_writer.py", line 29, in write_user
[v + ':' + str(self.user.dict[k]) for k, v in self.user_desc])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/liangzhi/anaconda3/lib/python3.11/site-packages/weibo_spider/writer/txt_writer.py", line 29, in
[v + ':' + str(self.user.dict[k]) for k, v in self.user_desc])
^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'dict'

期待您的回复~谢谢

@ciiio ciiio added the failed 程序运行出错 label Jun 6, 2024
@dataabc
Copy link
Owner

dataabc commented Jun 6, 2024

应该是被暂时限制了,一般来说,过一段时间限制会自动解除。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed 程序运行出错
Projects
None yet
Development

No branches or pull requests

2 participants