Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeEncodeError for the character "ō" #671

Closed
2 tasks done
waivek opened this issue Sep 19, 2022 · 1 comment
Closed
2 tasks done

UnicodeEncodeError for the character "ō" #671

waivek opened this issue Sep 19, 2022 · 1 comment

Comments

@waivek
Copy link

waivek commented Sep 19, 2022

Description

When launching vimspector or using the balloon eval feature, the character "ō" causes a UnicodeEncodeError in python.

vimspector_encode v01

Works in vim?

Reproducable in Vim

Works with clean config?

Yes

Sample project works?

Reproducable with sample project

Minimal Reproduction

Open a python file with the contents:

japanese_name = "Yoshifumi Kondō"
raise Exception

Launch vimspector using:
:call vimspector#LaunchWithSettings({ 'configuration': 'run' })

Expected Behaviour

Expected no error messages during startup / balloon eval.

Actual Behaviour

A vim error trace is produced with the following output

Error detected while processing function <SNR>156__OnServerData:
line    6:
--- Logging error ---
Traceback (most recent call last):
  File "C:\Program Files\Python310\Lib\logging\__init__.py", line 1101, in emit
    stream.write(msg + self.terminator)
  File "C:\Program Files\Python310\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u014d' in position 336: character maps to <undefined>
Call stack:
  File "<string>", line 1, in <module>
  File "C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\debug_session.py", line 463, in OnChannelData
    self._connection.OnData( data )
  File "C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\debug_adapter_connection.py", line 179, in OnData
    self._ReadBody()
  File "C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\debug_adapter_connection.py", line 257, in _ReadBody
    self._logger.debug( 'Message received: {0}'.format( message ) )
Message: 'Message received: {\'seq\': 24, \'type\': \'response\', \'request_seq\': 15, \'success\': True, \'command\': \'variables\', \'body\': {\'variables\': [{\'name\': \'special variables\', \'value\': \'\', \'type\': \'\', \'evaluateName\': \'special variables\', \'variablesReference\': 7}, {\'name\': \'japanese_name\', \'value\': "\'Yoshifumi Kondō\'", \'type\': \'str\', \'evaluateName\': \'japanese_name\', \'variablesReference\': 0, \'presentationHint\': {\'attributes\': [\'rawString\']}}]}}'
Arguments: ()

Additional information

No response

Installation Type

Plugin Manager (git clone)

Vimspector version

3378018

Debug Info

Vimspector Debug Info
--------------------------------------------------------------------------------
ConnectionType: job
Adapter: 
--------------------------------------------------------------------------------
{
  "command": [
    "C:\\Program Files\\Python310\\python.exe",
    "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows/debugpy/build/lib/debugpy/adapter"
  ],
  "configuration": {
    "python": "C:\\Program Files\\Python310\\python.exe",
    "subProcess": false
  },
  "name": "debugpy",
  "env": {},
  "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python"
}
--------------------------------------------------------------------------------
Configuration: 
--------------------------------------------------------------------------------
{
  "adapter": "debugpy",
  "configuration": {
    "request": "launch",
    "type": "python",
    "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python",
    "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py",
    "stopOnEntry": false,
    "console": "integratedTerminal",
    "justMyCode": false,
    "purpose": [
      "debug-in-terminal"
    ]
  },
  "breakpoints": {
    "exception": {
      "raised": "N",
      "uncaught": "",
      "userUnhandled": ""
    }
  }
}
--------------------------------------------------------------------------------
API Prefix: 
Launch/Init: True / True
Workspace Root: C:\Users\vivek\vimfiles\plugged\vimspector\support\test\python\simple_python
Launch Config: 
--------------------------------------------------------------------------------
{
  "python": "C:\\Program Files\\Python310\\python.exe",
  "subProcess": false,
  "request": "launch",
  "type": "python",
  "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python",
  "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py",
  "stopOnEntry": false,
  "console": "integratedTerminal",
  "justMyCode": false,
  "purpose": [
    "debug-in-terminal"
  ],
  "name": "test"
}
--------------------------------------------------------------------------------
Server Capabilities: 
--------------------------------------------------------------------------------
{
  "supportsCompletionsRequest": true,
  "supportsConditionalBreakpoints": true,
  "supportsConfigurationDoneRequest": true,
  "supportsDebuggerProperties": true,
  "supportsDelayedStackTraceLoading": true,
  "supportsEvaluateForHovers": true,
  "supportsExceptionInfoRequest": true,
  "supportsExceptionOptions": true,
  "supportsFunctionBreakpoints": true,
  "supportsHitConditionalBreakpoints": true,
  "supportsLogPoints": true,
  "supportsModulesRequest": true,
  "supportsSetExpression": true,
  "supportsSetVariable": true,
  "supportsValueFormattingOptions": true,
  "supportsTerminateDebuggee": true,
  "supportsGotoTargetsRequest": true,
  "supportsClipboardContext": true,
  "exceptionBreakpointFilters": [
    {
      "filter": "raised",
      "label": "Raised Exceptions",
      "default": false,
      "description": "Break whenever any exception is raised."
    },
    {
      "filter": "uncaught",
      "label": "Uncaught Exceptions",
      "default": true,
      "description": "Break when the process is exiting due to unhandled exception."
    },
    {
      "filter": "userUnhandled",
      "label": "User Uncaught Exceptions",
      "default": false,
      "description": "Break when exception escapes into library code."
    }
  ],
  "supportsStepInTargetsRequest": true
}
--------------------------------------------------------------------------------
Line Breakpoints: 
--------------------------------------------------------------------------------
{
  "C:\\Users\\vivek\\Documents\\samples\\letterboxd_ten.py": [],
  "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\dbg.py": [],
  "C:\\Users\\vivek\\vimfiles\\ide.vim": [],
  "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py": []
}
--------------------------------------------------------------------------------
Func Breakpoints: 
--------------------------------------------------------------------------------
[]
--------------------------------------------------------------------------------
Ex Breakpoints: 
--------------------------------------------------------------------------------
{
  "filters": [
    "uncaught"
  ],
  "exceptionOptions": []
}
--------------------------------------------------------------------------------

Log file

2022-09-19 20:56:54,273 - INFO - **** INITIALISING NEW VIMSPECTOR SESSION ****
2022-09-19 20:56:54,274 - INFO - API is: 
2022-09-19 20:56:54,274 - INFO - VIMSPECTOR_HOME = C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\..\..
2022-09-19 20:56:54,404 - INFO - gadgetDir = C:\Users\vivek\vimfiles\plugged\vimspector\gadgets\windows
2022-09-19 20:56:54,405 - INFO - User requested start debug session with {}
2022-09-19 20:56:54,408 - DEBUG - Reading configurations from: None
2022-09-19 20:56:54,409 - DEBUG - Reading configurations from: C:\Users\vivek\vimfiles\plugged\vimspector\support\test\python\simple_python\.vimspector.json
2022-09-19 20:56:54,773 - DEBUG - Reading gadget config: C:\Users\vivek\vimfiles\plugged\vimspector\gadgets\windows\.gadgets.json
2022-09-19 20:56:54,776 - DEBUG - Reading gadget config: None
2022-09-19 20:56:56,479 - INFO - Configuration: {"adapter": "debugpy", "configuration": {"request": "launch", "type": "python", "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python", "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py", "stopOnEntry": false, "console": "integratedTerminal", "justMyCode": false, "purpose": ["debug-in-terminal"]}, "breakpoints": {"exception": {"raised": "N", "uncaught": "", "userUnhandled": ""}}}
2022-09-19 20:56:56,479 - INFO - Adapter: {"command": ["C:\\Program Files\\Python310\\python.exe", "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows/debugpy/build/lib/debugpy/adapter"], "configuration": {"python": "C:\\Program Files\\Python310\\python.exe", "subProcess": false}, "name": "debugpy"}
2022-09-19 20:56:56,857 - DEBUG - min_width/height: 149/50, actual: 172/38 - result: horizontal
2022-09-19 20:56:57,408 - DEBUG - LAUNCH!
2022-09-19 20:56:57,417 - INFO - Starting debug adapter with: {"command": ["C:\\Program Files\\Python310\\python.exe", "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows/debugpy/build/lib/debugpy/adapter"], "configuration": {"python": "C:\\Program Files\\Python310\\python.exe", "subProcess": false}, "name": "debugpy"}
2022-09-19 20:56:57,417 - DEBUG - Connection Type: job
2022-09-19 20:56:57,546 - INFO - Debug Adapter Started
2022-09-19 20:56:57,547 - DEBUG - Sending Message: {"command": "initialize", "arguments": {"adapterID": "debugpy", "clientID": "vimspector", "clientName": "vimspector", "linesStartAt1": true, "columnsStartAt1": true, "locale": "en_GB", "pathFormat": "path", "supportsVariableType": true, "supportsVariablePaging": false, "supportsRunInTerminalRequest": true, "supportsMemoryReferences": true}, "seq": 0, "type": "request"}
2022-09-19 20:56:58,243 - DEBUG - Message received: {'seq': 1, 'type': 'event', 'event': 'output', 'body': {'category': 'telemetry', 'output': 'ptvsd', 'data': {'packageVersion': '1.6.3'}}}
2022-09-19 20:56:58,244 - DEBUG - Message received: {'seq': 2, 'type': 'event', 'event': 'output', 'body': {'category': 'telemetry', 'output': 'debugpy', 'data': {'packageVersion': '1.6.3'}}}
2022-09-19 20:56:58,244 - DEBUG - Message received: {'seq': 3, 'type': 'response', 'request_seq': 0, 'success': True, 'command': 'initialize', 'body': {'supportsCompletionsRequest': True, 'supportsConditionalBreakpoints': True, 'supportsConfigurationDoneRequest': True, 'supportsDebuggerProperties': True, 'supportsDelayedStackTraceLoading': True, 'supportsEvaluateForHovers': True, 'supportsExceptionInfoRequest': True, 'supportsExceptionOptions': True, 'supportsFunctionBreakpoints': True, 'supportsHitConditionalBreakpoints': True, 'supportsLogPoints': True, 'supportsModulesRequest': True, 'supportsSetExpression': True, 'supportsSetVariable': True, 'supportsValueFormattingOptions': True, 'supportsTerminateDebuggee': True, 'supportsGotoTargetsRequest': True, 'supportsClipboardContext': True, 'exceptionBreakpointFilters': [{'filter': 'raised', 'label': 'Raised Exceptions', 'default': False, 'description': 'Break whenever any exception is raised.'}, {'filter': 'uncaught', 'label': 'Uncaught Exceptions', 'default': True, 'description': 'Break when the process is exiting due to unhandled exception.'}, {'filter': 'userUnhandled', 'label': 'User Uncaught Exceptions', 'default': False, 'description': 'Break when exception escapes into library code.'}], 'supportsStepInTargetsRequest': True}}
2022-09-19 20:56:58,246 - DEBUG - Sending Message: {"command": "launch", "arguments": {"python": "C:\\Program Files\\Python310\\python.exe", "subProcess": false, "request": "launch", "type": "python", "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python", "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py", "stopOnEntry": false, "console": "integratedTerminal", "justMyCode": false, "purpose": ["debug-in-terminal"], "name": "test"}, "seq": 1, "type": "request"}
2022-09-19 20:56:58,257 - DEBUG - Message received: {'seq': 4, 'type': 'request', 'command': 'runInTerminal', 'arguments': {'kind': 'integrated', 'title': 'Python Debug Console', 'args': ['C:\\Program Files\\Python310\\python.exe', 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows\\debugpy\\build\\lib\\debugpy\\adapter/../..\\debugpy\\launcher', '2607', '--', 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py'], 'env': {}, 'cwd': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python'}}
2022-09-19 20:56:58,622 - DEBUG - Sending Message: {"seq": 2, "type": "response", "request_seq": 4, "command": "runInTerminal", "body": {"processId": 10532}, "success": true}
2022-09-19 20:57:01,291 - DEBUG - Message received: {'seq': 5, 'type': 'event', 'event': 'initialized'}
2022-09-19 20:57:01,291 - DEBUG - Sending Message: {"command": "setFunctionBreakpoints", "arguments": {"breakpoints": []}, "seq": 3, "type": "request"}
2022-09-19 20:57:01,292 - DEBUG - Sending Message: {"command": "setExceptionBreakpoints", "arguments": {"filters": ["uncaught"], "exceptionOptions": []}, "seq": 4, "type": "request"}
2022-09-19 20:57:01,302 - DEBUG - Message received: {'seq': 6, 'type': 'response', 'request_seq': 3, 'success': True, 'command': 'setFunctionBreakpoints', 'body': {'breakpoints': []}}
2022-09-19 20:57:01,303 - DEBUG - Message received: {'seq': 7, 'type': 'response', 'request_seq': 4, 'success': True, 'command': 'setExceptionBreakpoints'}
2022-09-19 20:57:01,303 - DEBUG - Sending Message: {"command": "configurationDone", "seq": 5, "type": "request"}
2022-09-19 20:57:01,314 - DEBUG - Message received: {'seq': 8, 'type': 'response', 'request_seq': 5, 'success': True, 'command': 'configurationDone'}
2022-09-19 20:57:01,315 - DEBUG - Message received: {'seq': 9, 'type': 'response', 'request_seq': 1, 'success': True, 'command': 'launch'}
2022-09-19 20:57:01,316 - DEBUG - Sending Message: {"command": "threads", "seq": 6, "type": "request"}
2022-09-19 20:57:01,316 - DEBUG - Message received: {'seq': 10, 'type': 'event', 'event': 'process', 'body': {'startMethod': 'launch', 'isLocalProcess': True, 'systemProcessId': 10928, 'name': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py', 'pointerSize': 64}}
2022-09-19 20:57:01,317 - INFO - User Msg: The debuggee was started: C:\Users\vivek\vimfiles\plugged\vimspector\support\test\python\simple_python\encode_error.py
2022-09-19 20:57:01,325 - DEBUG - Message received: {'seq': 11, 'type': 'event', 'event': 'thread', 'body': {'reason': 'started', 'threadId': 1}}
2022-09-19 20:57:01,327 - DEBUG - Message received: {'seq': 12, 'type': 'response', 'request_seq': 6, 'success': True, 'command': 'threads', 'body': {'threads': [{'id': 1, 'name': 'MainThread'}]}}
2022-09-19 20:57:01,328 - DEBUG - Sending Message: {"command": "threads", "seq": 7, "type": "request"}
2022-09-19 20:57:01,353 - DEBUG - Message received: {'seq': 13, 'type': 'response', 'request_seq': 7, 'success': True, 'command': 'threads', 'body': {'threads': [{'id': 1, 'name': 'MainThread'}]}}
2022-09-19 20:57:01,431 - DEBUG - Message received: {'seq': 14, 'type': 'event', 'event': 'stopped', 'body': {'reason': 'exception', 'description': 'exception: no description', 'threadId': 1, 'preserveFocusHint': False, 'text': 'Exception       (note: full exception trace is shown but execution is paused at: _run_module_as_main)', 'allThreadsStopped': True}}
2022-09-19 20:57:01,431 - WARNING - User Msg: Paused in thread 1 due to exception: no description(exception): Exception       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
2022-09-19 20:57:01,559 - DEBUG - Sending Message: {"command": "threads", "seq": 8, "type": "request"}
2022-09-19 20:57:01,564 - DEBUG - Message received: {'seq': 15, 'type': 'response', 'request_seq': 8, 'success': True, 'command': 'threads', 'body': {'threads': [{'id': 1, 'name': 'MainThread'}]}}
2022-09-19 20:57:01,564 - DEBUG - Sending Message: {"command": "stackTrace", "arguments": {"threadId": 1}, "seq": 9, "type": "request"}
2022-09-19 20:57:01,588 - DEBUG - Message received: {'seq': 16, 'type': 'response', 'request_seq': 9, 'success': True, 'command': 'stackTrace', 'body': {'stackFrames': [{'id': 2, 'name': '<module>', 'line': 2, 'column': 1, 'source': {'path': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py', 'sourceReference': 0}}, {'id': 3, 'name': '_run_code', 'line': 86, 'column': 1, 'source': {'path': 'C:\\Program Files\\Python310\\Lib\\runpy.py', 'sourceReference': 0}, 'presentationHint': 'subtle'}, {'id': 4, 'name': '_run_module_as_main (Current frame)', 'line': 196, 'column': 1, 'source': {'path': 'C:\\Program Files\\Python310\\Lib\\runpy.py', 'sourceReference': 0}, 'presentationHint': 'subtle'}], 'totalFrames': 3}}
2022-09-19 20:57:01,611 - DEBUG - Sending Message: {"command": "scopes", "arguments": {"frameId": 2}, "seq": 10, "type": "request"}
2022-09-19 20:57:01,614 - DEBUG - Message received: {'seq': 17, 'type': 'event', 'event': 'module', 'body': {'reason': 'new', 'module': {'id': 0, 'name': '__main__', 'path': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py'}}}
2022-09-19 20:57:01,614 - DEBUG - Message received: {'seq': 18, 'type': 'event', 'event': 'module', 'body': {'reason': 'new', 'module': {'id': 1, 'name': 'runpy', 'path': 'C:\\Program Files\\Python310\\Lib\\runpy.py'}}}
2022-09-19 20:57:01,673 - DEBUG - Message received: {'seq': 19, 'type': 'response', 'request_seq': 10, 'success': True, 'command': 'scopes', 'body': {'scopes': [{'name': 'Locals', 'variablesReference': 5, 'expensive': False, 'presentationHint': 'locals', 'source': {}}, {'name': 'Globals', 'variablesReference': 6, 'expensive': False, 'source': {}}]}}
2022-09-19 20:57:01,674 - DEBUG - Sending Message: {"command": "variables", "arguments": {"variablesReference": 5}, "seq": 11, "type": "request"}
2022-09-19 20:57:02,285 - DEBUG - Message received: {'seq': 21, 'type': 'event', 'event': 'output', 'body': {'output': 'pydevd warning: Computing repr of special variables () was slow (took 0.42s)\n', 'category': 'important', 'source': {}}}

Vim version

VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Sep 11 2022 22:03:01)
MS-Windows 64-bit GUI version with OLE support
Included patches: 1-447
Compiled by appveyor@APPVYR-WIN
Huge version with GUI.  Features included (+) or not (-):
+acl                +ex_extra           +mzscheme/dyn       -tcl
+arabic             +extra_search       +netbeans_intg      -termguicolors
+autocmd            -farsi              +num64              +terminal
+autochdir          +file_in_path       +ole                -termresponse
+autoservername     +find_in_path       +packages           +textobjects
+balloon_eval       +float              +path_extra         +textprop
-balloon_eval_term  +folding            +perl/dyn           -tgetent
+browse             -footer             +persistent_undo    +timers
++builtin_terms     +gettext/dyn        +popupwin           +title
+byte_offset        -hangul_input       -postscript         +toolbar
+channel            +iconv/dyn          +printer            +user_commands
+cindent            +insert_expand      +profile            +vartabs
+clientserver       +ipv6               +python/dyn         +vertsplit
+clipboard          +job                +python3/dyn        +vim9script
+cmdline_compl      +jumplist           +quickfix           +viminfo
+cmdline_hist       +keymap             +reltime            +virtualedit
+cmdline_info       +lambda             +rightleft          +visual
+comments           +langmap            +ruby/dyn           +visualextra
+conceal            +libcall            +scrollbind         +vreplace
+cryptv             +linebreak          +signs              -vtp
+cscope             +lispindent         +smartindent        +wildignore
+cursorbind         +listcmds           +sodium/dyn         +wildmenu
+cursorshape        +localmap           +sound              +windows
+dialog_con_gui     +lua/dyn            +spell              +writebackup
+diff               +menu               +startuptime        -xfontset
+digraphs           +mksession          +statusline         -xim
+directx            +modify_fname       -sun_workshop       +xpm_w32
-dnd                +mouse              +syntax             -xterm_save
-ebcdic             +mouseshape         +tag_binary         
+emacs_tags         +multi_byte_ime/dyn -tag_old_static     
+eval               +multi_lang         -tag_any_white      
   system vimrc file: "$VIM\vimrc"
     user vimrc file: "$HOME\_vimrc"
 2nd user vimrc file: "$HOME\vimfiles\vimrc"
 3rd user vimrc file: "$VIM\_vimrc"
      user exrc file: "$HOME\_exrc"
  2nd user exrc file: "$VIM\_exrc"
  system gvimrc file: "$VIM\gvimrc"
    user gvimrc file: "$HOME\_gvimrc"
2nd user gvimrc file: "$HOME\vimfiles\gvimrc"
3rd user gvimrc file: "$VIM\_gvimrc"
       defaults file: "$VIMRUNTIME\defaults.vim"
    system menu file: "$VIMRUNTIME\menu.vim"
Compilation: cl -c /W3 /GF /nologo -I. -Iproto -DHAVE_PATHDEF -DWIN32  -DFEAT_CSCOPE -DFEAT_TERMINAL 
-DFEAT_SOUND -DFEAT_NETBEANS_INTG -DFEAT_JOB_CHANNEL -DFEAT_IPV6   -DFEAT_XPM_W32 
-DHAVE_SODIUM -DDYNAMIC_SODIUM -DDYNAMIC_SODIUM_DLL=\"libsodium.dll\" /I "C:\libsodium\include"

Python version

No response

Neovim diagnostics

No response

Operating System

OS: Windows 10 20H2

Declaration

@puremourning
Copy link
Owner

Thanks. This is covered by #96

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants