Repository navigation
create_subprocess_shell on Windows corrupts embedded quotes #153
Description
Activity
@naivedyakhare Fixing subprocess is much more of a bothersome annoyance than you think. We got rid of the shell.pyx module for a reason in order to sync up with uvloop which merging it in is an ongoing effort in order to have more contributors to fix these kinds of bugs however, feel free to do whatever means necessary, I'll even accept vibecoded at this point since this problem has plagued the library since the very beginning. I have many other things to attend to at the moment. I'll be a lot more lax with how this problem is solved this time due to this annoyance of finding the right balance.
@naivedyakhare Changed up your code a little bit but I made a better change to the debug
import asyncio import winloop CMD = 'python -c "import sys; print(\'hi\')"' async def run_shell(): proc = await asyncio.create_subprocess_shell( CMD, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, ) out, err = await proc.communicate() return out, err, proc.returncode if __name__ == "__main__": print("asyncio:", asyncio.run(run_shell())) print("winloop:", winloop.run(run_shell()))
- asyncio succeeds winloop fails.
My Other Observations
- removing /c from function
subprocess_shell()triggers winloop to run in limbo. (keep that in mind)
@cython.iterable_coroutine async def subprocess_shell(self, protocol_factory, cmd, *, shell=True, **kwargs): cdef list args if not shell: raise ValueError("shell must be True") if not system.PLATFORM_IS_WINDOWS: args = [cmd] if shell: args = [b'/bin/sh', b'-c'] + args else: # SEE: https://github.com/libuv/libuv/pull/2627 # See subprocess.py for the mirror of this code. comspec = os_environ.get("ComSpec") if not comspec: system_root = os_environ.get("SystemRoot", '') comspec = os_path_join(system_root, 'System32', 'cmd.exe') if not os_path_isabs(comspec): raise FileNotFoundError('shell not found: neither %ComSpec% nor %SystemRoot% is set') args = [comspec] # args.append('/c') args.append(cmd) return await self.__subprocess_run(protocol_factory, args, shell=True, **kwargs)
@naivedyakhare I guess I was in the wrong for procrastination of this bug let me make a few edits to this issue for you. I'll update as I diagnose this bug a bit deeper...
It should be noted that on windows it quotes everything immediately. This is what was observed on python 3.11.15 (which is the version that I use for all of my development of all my libraries currently).
if shell: startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW startupinfo.wShowWindow = _winapi.SW_HIDE if not executable: # gh-101283: without a fully-qualified path, before Windows # checks the system directories, it first looks in the # application directory, and also the current directory if # NeedCurrentDirectoryForExePathW(ExeName) is true, so try # to avoid executing unqualified "cmd.exe". comspec = os.environ.get('ComSpec') if not comspec: system_root = os.environ.get('SystemRoot', '') comspec = os.path.join(system_root, 'System32', 'cmd.exe') if not os.path.isabs(comspec): raise FileNotFoundError('shell not found: neither %ComSpec% nor %SystemRoot% is set') if os.path.isabs(comspec): executable = comspec else: comspec = executable args = '{} /c "{}"'.format (comspec, args)
@naivedyakhare Would you be all for me attempting to make this section of code more 1:1 with subprocess or is there something else I'm not catching onto?
- linked a pull request that will close this issuebring back shlex parser due to shell related difficulties #154
on Jul 11, 2026 @naivedyakhare Here comes a fix #154, It's not what I wanted but it's at least better than what was done previously. I'm leaving the job of optimizing it when I finish the merge with uvloop so that I have more maintainers to review it.
I want to thank you @naivedyakhare for your time, you've given me a creative idea, I'll probably come up with a cythonic speedup of the shlex library in the future.
Thanks a lot for a quick fix!! I am still learning so thanks for the tips you gave above. I can confirm everything works perfectly from my end now!
On Windows,
create_subprocess_shellre-escapes quotes inside the command, soanything with double quotes reaches the child corrupted. The same command works on
the stdlib asyncio loop and on uvloop (Linux).
This commonly breaks python -c commands.I feel like this was intentionally left. I would love to know the reasoning if it was.
Reproduce Bug
Result
hi(rc 0)-cargument and fails withSyntaxError: unterminated string literal(rc 1)I ran into this while building an agentic AI app. The agent often generates shell
commands like python -c with quoted code strings, and those almost always include
quotes. With winloop (through uvicorn) those commands fail or produce no output.
The same commands work fine with the stdlib asyncio loop.
Root cause
Winloop passes the shell command to cmd.exe as separate arguments, and libuv
re-escapes the quotes. asyncio builds one command string (cmd.exe /c plus the
full command wrapped in quotes) instead, so the quotes are not mangled.
Environment