Context Navigation

subprocess.py@ 1281

Last change on this file since 1281 was 1281, checked in by dmik, 8 years ago

python: Implement subprocess.Popen using LIBCx spawn2.

Closes #280.

Property svn:eol-style set to native

File size: 62.5 KB

Line
1	# subprocess - Subprocesses with accessible I/O streams
2	#
3	# For more information about this module, see PEP 324.
4	#
5	# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
6	#
7	# Licensed to PSF under a Contributor Agreement.
8	# See http://www.python.org/2.4/license for licensing details.
9
10	r"""subprocess - Subprocesses with accessible I/O streams
11
12	This module allows you to spawn processes, connect to their
13	input/output/error pipes, and obtain their return codes. This module
14	intends to replace several other, older modules and functions, like:
15
16	os.system
17	os.spawn*
18	os.popen*
19	popen2.*
20	commands.*
21
22	Information about how the subprocess module can be used to replace these
23	modules and functions can be found below.
24
25
26
27	Using the subprocess module
28	===========================
29	This module defines one class called Popen:
30
31	class Popen(args, bufsize=0, executable=None,
32	stdin=None, stdout=None, stderr=None,
33	preexec_fn=None, close_fds=False, shell=False,
34	cwd=None, env=None, universal_newlines=False,
35	startupinfo=None, creationflags=0):
36
37
38	Arguments are:
39
40	args should be a string, or a sequence of program arguments. The
41	program to execute is normally the first item in the args sequence or
42	string, but can be explicitly set by using the executable argument.
43
44	On UNIX, with shell=False (default): In this case, the Popen class
45	uses os.execvp() to execute the child program. args should normally
46	be a sequence. A string will be treated as a sequence with the string
47	as the only item (the program to execute).
48
49	On UNIX, with shell=True: If args is a string, it specifies the
50	command string to execute through the shell. If args is a sequence,
51	the first item specifies the command string, and any additional items
52	will be treated as additional shell arguments.
53
54	On Windows: the Popen class uses CreateProcess() to execute the child
55	program, which operates on strings. If args is a sequence, it will be
56	converted to a string using the list2cmdline method. Please note that
57	not all MS Windows applications interpret the command line the same
58	way: The list2cmdline is designed for applications using the same
59	rules as the MS C runtime.
60
61	bufsize, if given, has the same meaning as the corresponding argument
62	to the built-in open() function: 0 means unbuffered, 1 means line
63	buffered, any other positive value means use a buffer of
64	(approximately) that size. A negative bufsize means to use the system
65	default, which usually means fully buffered. The default value for
66	bufsize is 0 (unbuffered).
67
68	stdin, stdout and stderr specify the executed programs' standard
69	input, standard output and standard error file handles, respectively.
70	Valid values are PIPE, an existing file descriptor (a positive
71	integer), an existing file object, and None. PIPE indicates that a
72	new pipe to the child should be created. With None, no redirection
73	will occur; the child's file handles will be inherited from the
74	parent. Additionally, stderr can be STDOUT, which indicates that the
75	stderr data from the applications should be captured into the same
76	file handle as for stdout.
77
78	If preexec_fn is set to a callable object, this object will be called
79	in the child process just before the child is executed.
80
81	If close_fds is true, all file descriptors except 0, 1 and 2 will be
82	closed before the child process is executed.
83
84	if shell is true, the specified command will be executed through the
85	shell.
86
87	If cwd is not None, the current directory will be changed to cwd
88	before the child is executed.
89
90	If env is not None, it defines the environment variables for the new
91	process.
92
93	If universal_newlines is true, the file objects stdout and stderr are
94	opened as a text files, but lines may be terminated by any of '\n',
95	the Unix end-of-line convention, '\r', the Macintosh convention or
96	'\r\n', the Windows convention. All of these external representations
97	are seen as '\n' by the Python program. Note: This feature is only
98	available if Python is built with universal newline support (the
99	default). Also, the newlines attribute of the file objects stdout,
100	stdin and stderr are not updated by the communicate() method.
101
102	The startupinfo and creationflags, if given, will be passed to the
103	underlying CreateProcess() function. They can specify things such as
104	appearance of the main window and priority for the new process.
105	(Windows only)
106
107
108	This module also defines some shortcut functions:
109
110	call(popenargs, *kwargs):
111	Run command with arguments. Wait for command to complete, then
112	return the returncode attribute.
113
114	The arguments are the same as for the Popen constructor. Example:
115
116	retcode = call(["ls", "-l"])
117
118	check_call(popenargs, *kwargs):
119	Run command with arguments. Wait for command to complete. If the
120	exit code was zero then return, otherwise raise
121	CalledProcessError. The CalledProcessError object will have the
122	return code in the returncode attribute.
123
124	The arguments are the same as for the Popen constructor. Example:
125
126	check_call(["ls", "-l"])
127
128	check_output(popenargs, *kwargs):
129	Run command with arguments and return its output as a byte string.
130
131	If the exit code was non-zero it raises a CalledProcessError. The
132	CalledProcessError object will have the return code in the returncode
133	attribute and output in the output attribute.
134
135	The arguments are the same as for the Popen constructor. Example:
136
137	output = check_output(["ls", "-l", "/dev/null"])
138
139
140	Exceptions
141	----------
142	Exceptions raised in the child process, before the new program has
143	started to execute, will be re-raised in the parent. Additionally,
144	the exception object will have one extra attribute called
145	'child_traceback', which is a string containing traceback information
146	from the child's point of view.
147
148	The most common exception raised is OSError. This occurs, for
149	example, when trying to execute a non-existent file. Applications
150	should prepare for OSErrors.
151
152	A ValueError will be raised if Popen is called with invalid arguments.
153
154	check_call() and check_output() will raise CalledProcessError, if the
155	called process returns a non-zero return code.
156
157
158	Security
159	--------
160	Unlike some other popen functions, this implementation will never call
161	/bin/sh implicitly. This means that all characters, including shell
162	metacharacters, can safely be passed to child processes.
163
164
165	Popen objects
166	=============
167	Instances of the Popen class have the following methods:
168
169	poll()
170	Check if child process has terminated. Returns returncode
171	attribute.
172
173	wait()
174	Wait for child process to terminate. Returns returncode attribute.
175
176	communicate(input=None)
177	Interact with process: Send data to stdin. Read data from stdout
178	and stderr, until end-of-file is reached. Wait for process to
179	terminate. The optional input argument should be a string to be
180	sent to the child process, or None, if no data should be sent to
181	the child.
182
183	communicate() returns a tuple (stdout, stderr).
184
185	Note: The data read is buffered in memory, so do not use this
186	method if the data size is large or unlimited.
187
188	The following attributes are also available:
189
190	stdin
191	If the stdin argument is PIPE, this attribute is a file object
192	that provides input to the child process. Otherwise, it is None.
193
194	stdout
195	If the stdout argument is PIPE, this attribute is a file object
196	that provides output from the child process. Otherwise, it is
197	None.
198
199	stderr
200	If the stderr argument is PIPE, this attribute is file object that
201	provides error output from the child process. Otherwise, it is
202	None.
203
204	pid
205	The process ID of the child process.
206
207	returncode
208	The child return code. A None value indicates that the process
209	hasn't terminated yet. A negative value -N indicates that the
210	child was terminated by signal N (UNIX only).
211
212
213	Replacing older functions with the subprocess module
214	====================================================
215	In this section, "a ==> b" means that b can be used as a replacement
216	for a.
217
218	Note: All functions in this section fail (more or less) silently if
219	the executed program cannot be found; this module raises an OSError
220	exception.
221
222	In the following examples, we assume that the subprocess module is
223	imported with "from subprocess import *".
224
225
226	Replacing /bin/sh shell backquote
227	---------------------------------
228	output=`mycmd myarg`
229	==>
230	output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
231
232
233	Replacing shell pipe line
234	-------------------------
235	output=`dmesg \| grep hda`
236	==>
237	p1 = Popen(["dmesg"], stdout=PIPE)
238	p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
239	output = p2.communicate()[0]
240
241
242	Replacing os.system()
243	---------------------
244	sts = os.system("mycmd" + " myarg")
245	==>
246	p = Popen("mycmd" + " myarg", shell=True)
247	pid, sts = os.waitpid(p.pid, 0)
248
249	Note:
250
251	* Calling the program through the shell is usually not required.
252
253	* It's easier to look at the returncode attribute than the
254	exitstatus.
255
256	A more real-world example would look like this:
257
258	try:
259	retcode = call("mycmd" + " myarg", shell=True)
260	if retcode < 0:
261	print >>sys.stderr, "Child was terminated by signal", -retcode
262	else:
263	print >>sys.stderr, "Child returned", retcode
264	except OSError, e:
265	print >>sys.stderr, "Execution failed:", e
266
267
268	Replacing os.spawn*
269	-------------------
270	P_NOWAIT example:
271
272	pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
273	==>
274	pid = Popen(["/bin/mycmd", "myarg"]).pid
275
276
277	P_WAIT example:
278
279	retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
280	==>
281	retcode = call(["/bin/mycmd", "myarg"])
282
283
284	Vector example:
285
286	os.spawnvp(os.P_NOWAIT, path, args)
287	==>
288	Popen([path] + args[1:])
289
290
291	Environment example:
292
293	os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
294	==>
295	Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
296
297
298	Replacing os.popen*
299	-------------------
300	pipe = os.popen("cmd", mode='r', bufsize)
301	==>
302	pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout
303
304	pipe = os.popen("cmd", mode='w', bufsize)
305	==>
306	pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin
307
308
309	(child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
310	==>
311	p = Popen("cmd", shell=True, bufsize=bufsize,
312	stdin=PIPE, stdout=PIPE, close_fds=True)
313	(child_stdin, child_stdout) = (p.stdin, p.stdout)
314
315
316	(child_stdin,
317	child_stdout,
318	child_stderr) = os.popen3("cmd", mode, bufsize)
319	==>
320	p = Popen("cmd", shell=True, bufsize=bufsize,
321	stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
322	(child_stdin,
323	child_stdout,
324	child_stderr) = (p.stdin, p.stdout, p.stderr)
325
326
327	(child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
328	bufsize)
329	==>
330	p = Popen("cmd", shell=True, bufsize=bufsize,
331	stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
332	(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
333
334	On Unix, os.popen2, os.popen3 and os.popen4 also accept a sequence as
335	the command to execute, in which case arguments will be passed
336	directly to the program without shell intervention. This usage can be
337	replaced as follows:
338
339	(child_stdin, child_stdout) = os.popen2(["/bin/ls", "-l"], mode,
340	bufsize)
341	==>
342	p = Popen(["/bin/ls", "-l"], bufsize=bufsize, stdin=PIPE, stdout=PIPE)
343	(child_stdin, child_stdout) = (p.stdin, p.stdout)
344
345	Return code handling translates as follows:
346
347	pipe = os.popen("cmd", 'w')
348	...
349	rc = pipe.close()
350	if rc is not None and rc % 256:
351	print "There were some errors"
352	==>
353	process = Popen("cmd", 'w', shell=True, stdin=PIPE)
354	...
355	process.stdin.close()
356	if process.wait() != 0:
357	print "There were some errors"
358
359
360	Replacing popen2.*
361	------------------
362	(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
363	==>
364	p = Popen(["somestring"], shell=True, bufsize=bufsize
365	stdin=PIPE, stdout=PIPE, close_fds=True)
366	(child_stdout, child_stdin) = (p.stdout, p.stdin)
367
368	On Unix, popen2 also accepts a sequence as the command to execute, in
369	which case arguments will be passed directly to the program without
370	shell intervention. This usage can be replaced as follows:
371
372	(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize,
373	mode)
374	==>
375	p = Popen(["mycmd", "myarg"], bufsize=bufsize,
376	stdin=PIPE, stdout=PIPE, close_fds=True)
377	(child_stdout, child_stdin) = (p.stdout, p.stdin)
378
379	The popen2.Popen3 and popen2.Popen4 basically works as subprocess.Popen,
380	except that:
381
382	* subprocess.Popen raises an exception if the execution fails
383	* the capturestderr argument is replaced with the stderr argument.
384	* stdin=PIPE and stdout=PIPE must be specified.
385	* popen2 closes all filedescriptors by default, but you have to specify
386	close_fds=True with subprocess.Popen.
387	"""
388
389	import sys
390	mswindows = (sys.platform == "win32")
391
392	import os
393	import types
394	import traceback
395	import gc
396	import signal
397	import errno
398
399	os2 = (os.name == "os2")
400
401	import sysconfig
402	SHELL = sysconfig.get_config_var('SHELL') or '/bin/sh'
403
404	# Exception classes used by this module.
405	class CalledProcessError(Exception):
406	"""This exception is raised when a process run by check_call() or
407	check_output() returns a non-zero exit status.
408	The exit status will be stored in the returncode attribute;
409	check_output() will also store the output in the output attribute.
410	"""
411	def __init__(self, returncode, cmd, output=None):
412	self.returncode = returncode
413	self.cmd = cmd
414	self.output = output
415	def __str__(self):
416	return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
417
418
419	if mswindows:
420	import threading
421	import msvcrt
422	import _subprocess
423	class STARTUPINFO:
424	dwFlags = 0
425	hStdInput = None
426	hStdOutput = None
427	hStdError = None
428	wShowWindow = 0
429	class pywintypes:
430	error = IOError
431	elif os2:
432	import threading
433	import fcntl
434	import time
435	else:
436	import select
437	_has_poll = hasattr(select, 'poll')
438	import fcntl
439	import pickle
440
441	# When select or poll has indicated that the file is writable,
442	# we can write up to _PIPE_BUF bytes without risk of blocking.
443	# POSIX defines PIPE_BUF as >= 512.
444	_PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
445
446
447	__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call",
448	"check_output", "CalledProcessError"]
449
450	if mswindows:
451	from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
452	STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
453	STD_ERROR_HANDLE, SW_HIDE,
454	STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
455
456	__all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
457	"STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
458	"STD_ERROR_HANDLE", "SW_HIDE",
459	"STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
460	try:
461	MAXFD = os.sysconf("SC_OPEN_MAX")
462	except:
463	MAXFD = 256
464
465	_active = []
466
467	def _cleanup():
468	for inst in _active[:]:
469	res = inst._internal_poll(_deadstate=sys.maxint)
470	if res is not None:
471	try:
472	_active.remove(inst)
473	except ValueError:
474	# This can happen if two threads create a new Popen instance.
475	# It's harmless that it was already removed, so ignore.
476	pass
477
478	PIPE = -1
479	STDOUT = -2
480
481
482	def _eintr_retry_call(func, *args):
483	while True:
484	try:
485	return func(*args)
486	except (OSError, IOError) as e:
487	if e.errno == errno.EINTR:
488	continue
489	raise
490
491
492	# XXX This function is only used by multiprocessing and the test suite,
493	# but it's here so that it can be imported when Python is compiled without
494	# threads.
495
496	def _args_from_interpreter_flags():
497	"""Return a list of command-line arguments reproducing the current
498	settings in sys.flags and sys.warnoptions."""
499	flag_opt_map = {
500	'debug': 'd',
501	# 'inspect': 'i',
502	# 'interactive': 'i',
503	'optimize': 'O',
504	'dont_write_bytecode': 'B',
505	'no_user_site': 's',
506	'no_site': 'S',
507	'ignore_environment': 'E',
508	'verbose': 'v',
509	'bytes_warning': 'b',
510	'hash_randomization': 'R',
511	'py3k_warning': '3',
512	}
513	args = []
514	for flag, opt in flag_opt_map.items():
515	v = getattr(sys.flags, flag)
516	if v > 0:
517	args.append('-' + opt * v)
518	for opt in sys.warnoptions:
519	args.append('-W' + opt)
520	return args
521
522
523	def call(popenargs, *kwargs):
524	"""Run command with arguments. Wait for command to complete, then
525	return the returncode attribute.
526
527	The arguments are the same as for the Popen constructor. Example:
528
529	retcode = call(["ls", "-l"])
530	"""
531	return Popen(popenargs, *kwargs).wait()
532
533
534	def check_call(popenargs, *kwargs):
535	"""Run command with arguments. Wait for command to complete. If
536	the exit code was zero then return, otherwise raise
537	CalledProcessError. The CalledProcessError object will have the
538	return code in the returncode attribute.
539
540	The arguments are the same as for the Popen constructor. Example:
541
542	check_call(["ls", "-l"])
543	"""
544	retcode = call(popenargs, *kwargs)
545	if retcode:
546	cmd = kwargs.get("args")
547	if cmd is None:
548	cmd = popenargs[0]
549	raise CalledProcessError(retcode, cmd)
550	return 0
551
552
553	def check_output(popenargs, *kwargs):
554	r"""Run command with arguments and return its output as a byte string.
555
556	If the exit code was non-zero it raises a CalledProcessError. The
557	CalledProcessError object will have the return code in the returncode
558	attribute and output in the output attribute.
559
560	The arguments are the same as for the Popen constructor. Example:
561
562	>>> check_output(["ls", "-l", "/dev/null"])
563	'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
564
565	The stdout argument is not allowed as it is used internally.
566	To capture standard error in the result, use stderr=STDOUT.
567
568	>>> check_output(["/bin/sh", "-c",
569	... "ls -l non_existent_file ; exit 0"],
570	... stderr=STDOUT)
571	'ls: non_existent_file: No such file or directory\n'
572	"""
573	if 'stdout' in kwargs:
574	raise ValueError('stdout argument not allowed, it will be overridden.')
575	process = Popen(stdout=PIPE, popenargs, *kwargs)
576	output, unused_err = process.communicate()
577	retcode = process.poll()
578	if retcode:
579	cmd = kwargs.get("args")
580	if cmd is None:
581	cmd = popenargs[0]
582	raise CalledProcessError(retcode, cmd, output=output)
583	return output
584
585
586	def list2cmdline(seq):
587	"""
588	Translate a sequence of arguments into a command line
589	string, using the same rules as the MS C runtime:
590
591	1) Arguments are delimited by white space, which is either a
592	space or a tab.
593
594	2) A string surrounded by double quotation marks is
595	interpreted as a single argument, regardless of white space
596	contained within. A quoted string can be embedded in an
597	argument.
598
599	3) A double quotation mark preceded by a backslash is
600	interpreted as a literal double quotation mark.
601
602	4) Backslashes are interpreted literally, unless they
603	immediately precede a double quotation mark.
604
605	5) If backslashes immediately precede a double quotation mark,
606	every pair of backslashes is interpreted as a literal
607	backslash. If the number of backslashes is odd, the last
608	backslash escapes the next double quotation mark as
609	described in rule 3.
610	"""
611
612	# See
613	# http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
614	# or search http://msdn.microsoft.com for
615	# "Parsing C++ Command-Line Arguments"
616	result = []
617	needquote = False
618	for arg in seq:
619	bs_buf = []
620
621	# Add a space to separate this argument from the others
622	if result:
623	result.append(' ')
624
625	needquote = (" " in arg) or ("\t" in arg) or not arg
626	if needquote:
627	result.append('"')
628
629	for c in arg:
630	if c == '\\':
631	# Don't know if we need to double yet.
632	bs_buf.append(c)
633	elif c == '"':
634	# Double backslashes.
635	result.append('\\' * len(bs_buf)*2)
636	bs_buf = []
637	result.append('\\"')
638	else:
639	# Normal char
640	if bs_buf:
641	result.extend(bs_buf)
642	bs_buf = []
643	result.append(c)
644
645	# Add remaining backslashes, if any.
646	if bs_buf:
647	result.extend(bs_buf)
648
649	if needquote:
650	result.extend(bs_buf)
651	result.append('"')
652
653	return ''.join(result)
654
655
656	class Popen(object):
657	def __init__(self, args, bufsize=0, executable=None,
658	stdin=None, stdout=None, stderr=None,
659	preexec_fn=None, close_fds=False, shell=False,
660	cwd=None, env=None, universal_newlines=False,
661	startupinfo=None, creationflags=0,
662	threadsafe=None):
663	"""Create new Popen instance."""
664	_cleanup()
665
666	self._child_created = False
667	if not isinstance(bufsize, (int, long)):
668	raise TypeError("bufsize must be an integer")
669
670	if mswindows or os2:
671	if preexec_fn is not None:
672	raise ValueError("preexec_fn is not supported on Windows "
673	"and OS/2 platforms")
674	if not os2 and close_fds and (stdin is not None or stdout is not None or
675	stderr is not None):
676	raise ValueError("close_fds is not supported on Windows "
677	"platforms if you redirect stdin/stdout/stderr")
678	if not mswindows:
679	# POSIX
680	if startupinfo is not None:
681	raise ValueError("startupinfo is only supported on Windows "
682	"platforms")
683	if creationflags != 0:
684	raise ValueError("creationflags is only supported on Windows "
685	"platforms")
686	if os2:
687	if threadsafe is None:
688	threadsafe = False
689	else:
690	if threadsafe is not None:
691	raise ValueError("threadsafe is only supported on OS/2 "
692	"platforms")
693
694	self.stdin = None
695	self.stdout = None
696	self.stderr = None
697	self.pid = None
698	self.returncode = None
699	self.universal_newlines = universal_newlines
700
701	# Input and output objects. The general principle is like
702	# this:
703	#
704	# Parent Child
705	# ------ -----
706	# p2cwrite ---stdin---> p2cread
707	# c2pread <--stdout--- c2pwrite
708	# errread <--stderr--- errwrite
709	#
710	# On POSIX, the child objects are file descriptors. On
711	# Windows, these are Windows file handles. The parent objects
712	# are file descriptors on both platforms. The parent objects
713	# are None when not using PIPEs. The child objects are None
714	# when not redirecting.
715
716	(p2cread, p2cwrite,
717	c2pread, c2pwrite,
718	errread, errwrite), to_close = self._get_handles(stdin, stdout, stderr)
719
720	try:
721	self._execute_child(args, executable, preexec_fn, close_fds,
722	cwd, env, universal_newlines,
723	startupinfo, creationflags, threadsafe, shell, to_close,
724	p2cread, p2cwrite,
725	c2pread, c2pwrite,
726	errread, errwrite)
727	except Exception:
728	# Preserve original exception in case os.close raises.
729	exc_type, exc_value, exc_trace = sys.exc_info()
730
731	for fd in to_close:
732	try:
733	if mswindows:
734	fd.Close()
735	else:
736	os.close(fd)
737	except EnvironmentError:
738	pass
739
740	raise exc_type, exc_value, exc_trace
741
742	if mswindows:
743	if p2cwrite is not None:
744	p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
745	if c2pread is not None:
746	c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
747	if errread is not None:
748	errread = msvcrt.open_osfhandle(errread.Detach(), 0)
749
750	if p2cwrite is not None:
751	self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)
752	if c2pread is not None:
753	if universal_newlines:
754	self.stdout = os.fdopen(c2pread, 'rU', bufsize)
755	else:
756	self.stdout = os.fdopen(c2pread, 'rb', bufsize)
757	if errread is not None:
758	if universal_newlines:
759	self.stderr = os.fdopen(errread, 'rU', bufsize)
760	else:
761	self.stderr = os.fdopen(errread, 'rb', bufsize)
762
763
764	def _translate_newlines(self, data):
765	data = data.replace("\r\n", "\n")
766	data = data.replace("\r", "\n")
767	return data
768
769
770	def __del__(self, _maxint=sys.maxint, _active=_active):
771	# If __init__ hasn't had a chance to execute (e.g. if it
772	# was passed an undeclared keyword argument), we don't
773	# have a _child_created attribute at all.
774	if not getattr(self, '_child_created', False):
775	# We didn't get to successfully create a child process.
776	return
777	# In case the child hasn't been waited on, check if it's done.
778	self._internal_poll(_deadstate=_maxint)
779	if self.returncode is None and _active is not None:
780	# Child is still running, keep us alive until we can wait on it.
781	_active.append(self)
782
783
784	def communicate(self, input=None):
785	"""Interact with process: Send data to stdin. Read data from
786	stdout and stderr, until end-of-file is reached. Wait for
787	process to terminate. The optional input argument should be a
788	string to be sent to the child process, or None, if no data
789	should be sent to the child.
790
791	communicate() returns a tuple (stdout, stderr)."""
792
793	# Optimization: If we are only using one pipe, or no pipe at
794	# all, using select() or threads is unnecessary.
795	if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
796	stdout = None
797	stderr = None
798	if self.stdin:
799	if input:
800	try:
801	self.stdin.write(input)
802	except IOError as e:
803	if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
804	raise
805	self.stdin.close()
806	elif self.stdout:
807	stdout = _eintr_retry_call(self.stdout.read)
808	self.stdout.close()
809	elif self.stderr:
810	stderr = _eintr_retry_call(self.stderr.read)
811	self.stderr.close()
812	self.wait()
813	return (stdout, stderr)
814
815	return self._communicate(input)
816
817
818	def poll(self):
819	return self._internal_poll()
820
821
822	if mswindows:
823	#
824	# Windows methods
825	#
826	def _get_handles(self, stdin, stdout, stderr):
827	"""Construct and return tuple with IO objects:
828	p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
829	"""
830	to_close = set()
831	if stdin is None and stdout is None and stderr is None:
832	return (None, None, None, None, None, None), to_close
833
834	p2cread, p2cwrite = None, None
835	c2pread, c2pwrite = None, None
836	errread, errwrite = None, None
837
838	if stdin is None:
839	p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
840	if p2cread is None:
841	p2cread, _ = _subprocess.CreatePipe(None, 0)
842	elif stdin == PIPE:
843	p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
844	elif isinstance(stdin, int):
845	p2cread = msvcrt.get_osfhandle(stdin)
846	else:
847	# Assuming file-like object
848	p2cread = msvcrt.get_osfhandle(stdin.fileno())
849	p2cread = self._make_inheritable(p2cread)
850	# We just duplicated the handle, it has to be closed at the end
851	to_close.add(p2cread)
852	if stdin == PIPE:
853	to_close.add(p2cwrite)
854
855	if stdout is None:
856	c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
857	if c2pwrite is None:
858	_, c2pwrite = _subprocess.CreatePipe(None, 0)
859	elif stdout == PIPE:
860	c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
861	elif isinstance(stdout, int):
862	c2pwrite = msvcrt.get_osfhandle(stdout)
863	else:
864	# Assuming file-like object
865	c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
866	c2pwrite = self._make_inheritable(c2pwrite)
867	# We just duplicated the handle, it has to be closed at the end
868	to_close.add(c2pwrite)
869	if stdout == PIPE:
870	to_close.add(c2pread)
871
872	if stderr is None:
873	errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
874	if errwrite is None:
875	_, errwrite = _subprocess.CreatePipe(None, 0)
876	elif stderr == PIPE:
877	errread, errwrite = _subprocess.CreatePipe(None, 0)
878	elif stderr == STDOUT:
879	errwrite = c2pwrite
880	elif isinstance(stderr, int):
881	errwrite = msvcrt.get_osfhandle(stderr)
882	else:
883	# Assuming file-like object
884	errwrite = msvcrt.get_osfhandle(stderr.fileno())
885	errwrite = self._make_inheritable(errwrite)
886	# We just duplicated the handle, it has to be closed at the end
887	to_close.add(errwrite)
888	if stderr == PIPE:
889	to_close.add(errread)
890
891	return (p2cread, p2cwrite,
892	c2pread, c2pwrite,
893	errread, errwrite), to_close
894
895
896	def _make_inheritable(self, handle):
897	"""Return a duplicate of handle, which is inheritable"""
898	return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
899	handle, _subprocess.GetCurrentProcess(), 0, 1,
900	_subprocess.DUPLICATE_SAME_ACCESS)
901
902
903	def _find_w9xpopen(self):
904	"""Find and return absolut path to w9xpopen.exe"""
905	w9xpopen = os.path.join(
906	os.path.dirname(_subprocess.GetModuleFileName(0)),
907	"w9xpopen.exe")
908	if not os.path.exists(w9xpopen):
909	# Eeek - file-not-found - possibly an embedding
910	# situation - see if we can locate it in sys.exec_prefix
911	w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
912	"w9xpopen.exe")
913	if not os.path.exists(w9xpopen):
914	raise RuntimeError("Cannot locate w9xpopen.exe, which is "
915	"needed for Popen to work with your "
916	"shell or platform.")
917	return w9xpopen
918
919
920	def _execute_child(self, args, executable, preexec_fn, close_fds,
921	cwd, env, universal_newlines,
922	startupinfo, creationflags, threadsafe, shell, to_close,
923	p2cread, p2cwrite,
924	c2pread, c2pwrite,
925	errread, errwrite):
926	"""Execute program (MS Windows version)"""
927
928	if not isinstance(args, types.StringTypes):
929	args = list2cmdline(args)
930
931	# Process startup details
932	if startupinfo is None:
933	startupinfo = STARTUPINFO()
934	if None not in (p2cread, c2pwrite, errwrite):
935	startupinfo.dwFlags \|= _subprocess.STARTF_USESTDHANDLES
936	startupinfo.hStdInput = p2cread
937	startupinfo.hStdOutput = c2pwrite
938	startupinfo.hStdError = errwrite
939
940	if shell:
941	startupinfo.dwFlags \|= _subprocess.STARTF_USESHOWWINDOW
942	startupinfo.wShowWindow = _subprocess.SW_HIDE
943	comspec = os.environ.get("COMSPEC", "cmd.exe")
944	args = '{} /c "{}"'.format (comspec, args)
945	if (_subprocess.GetVersion() >= 0x80000000 or
946	os.path.basename(comspec).lower() == "command.com"):
947	# Win9x, or using command.com on NT. We need to
948	# use the w9xpopen intermediate program. For more
949	# information, see KB Q150956
950	# (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
951	w9xpopen = self._find_w9xpopen()
952	args = '"%s" %s' % (w9xpopen, args)
953	# Not passing CREATE_NEW_CONSOLE has been known to
954	# cause random failures on win9x. Specifically a
955	# dialog: "Your program accessed mem currently in
956	# use at xxx" and a hopeful warning about the
957	# stability of your system. Cost is Ctrl+C wont
958	# kill children.
959	creationflags \|= _subprocess.CREATE_NEW_CONSOLE
960
961	def _close_in_parent(fd):
962	fd.Close()
963	to_close.remove(fd)
964
965	# Start the process
966	try:
967	hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
968	# no special security
969	None, None,
970	int(not close_fds),
971	creationflags,
972	env,
973	cwd,
974	startupinfo)
975	except pywintypes.error, e:
976	# Translate pywintypes.error to WindowsError, which is
977	# a subclass of OSError. FIXME: We should really
978	# translate errno using _sys_errlist (or similar), but
979	# how can this be done from Python?
980	raise WindowsError(*e.args)
981	finally:
982	# Child is launched. Close the parent's copy of those pipe
983	# handles that only the child should have open. You need
984	# to make sure that no handles to the write end of the
985	# output pipe are maintained in this process or else the
986	# pipe will not close when the child process exits and the
987	# ReadFile will hang.
988	if p2cread is not None:
989	_close_in_parent(p2cread)
990	if c2pwrite is not None:
991	_close_in_parent(c2pwrite)
992	if errwrite is not None:
993	_close_in_parent(errwrite)
994
995	# Retain the process handle, but close the thread handle
996	self._child_created = True
997	self._handle = hp
998	self.pid = pid
999	ht.Close()
1000
1001	def _internal_poll(self, _deadstate=None,
1002	_WaitForSingleObject=_subprocess.WaitForSingleObject,
1003	_WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
1004	_GetExitCodeProcess=_subprocess.GetExitCodeProcess):
1005	"""Check if child process has terminated. Returns returncode
1006	attribute.
1007
1008	This method is called by __del__, so it can only refer to objects
1009	in its local scope.
1010
1011	"""
1012	if self.returncode is None:
1013	if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
1014	self.returncode = _GetExitCodeProcess(self._handle)
1015	return self.returncode
1016
1017
1018	def wait(self):
1019	"""Wait for child process to terminate. Returns returncode
1020	attribute."""
1021	if self.returncode is None:
1022	_subprocess.WaitForSingleObject(self._handle,
1023	_subprocess.INFINITE)
1024	self.returncode = _subprocess.GetExitCodeProcess(self._handle)
1025	return self.returncode
1026
1027
1028	def _readerthread(self, fh, buffer):
1029	buffer.append(fh.read())
1030
1031
1032	def _communicate(self, input):
1033	stdout = None # Return
1034	stderr = None # Return
1035
1036	if self.stdout:
1037	stdout = []
1038	stdout_thread = threading.Thread(target=self._readerthread,
1039	args=(self.stdout, stdout))
1040	stdout_thread.setDaemon(True)
1041	stdout_thread.start()
1042	if self.stderr:
1043	stderr = []
1044	stderr_thread = threading.Thread(target=self._readerthread,
1045	args=(self.stderr, stderr))
1046	stderr_thread.setDaemon(True)
1047	stderr_thread.start()
1048
1049	if self.stdin:
1050	if input is not None:
1051	try:
1052	self.stdin.write(input)
1053	except IOError as e:
1054	if e.errno != errno.EPIPE:
1055	raise
1056	self.stdin.close()
1057
1058	if self.stdout:
1059	stdout_thread.join()
1060	if self.stderr:
1061	stderr_thread.join()
1062
1063	# All data exchanged. Translate lists into strings.
1064	if stdout is not None:
1065	stdout = stdout[0]
1066	if stderr is not None:
1067	stderr = stderr[0]
1068
1069	# Translate newlines, if requested. We cannot let the file
1070	# object do the translation: It is based on stdio, which is
1071	# impossible to combine with select (unless forcing no
1072	# buffering).
1073	if self.universal_newlines and hasattr(file, 'newlines'):
1074	if stdout:
1075	stdout = self._translate_newlines(stdout)
1076	if stderr:
1077	stderr = self._translate_newlines(stderr)
1078
1079	self.wait()
1080	return (stdout, stderr)
1081
1082	def send_signal(self, sig):
1083	"""Send a signal to the process
1084	"""
1085	if sig == signal.SIGTERM:
1086	self.terminate()
1087	elif sig == signal.CTRL_C_EVENT:
1088	os.kill(self.pid, signal.CTRL_C_EVENT)
1089	elif sig == signal.CTRL_BREAK_EVENT:
1090	os.kill(self.pid, signal.CTRL_BREAK_EVENT)
1091	else:
1092	raise ValueError("Unsupported signal: {}".format(sig))
1093
1094	def terminate(self):
1095	"""Terminates the process
1096	"""
1097	try:
1098	_subprocess.TerminateProcess(self._handle, 1)
1099	except OSError as e:
1100	# ERROR_ACCESS_DENIED (winerror 5) is received when the
1101	# process already died.
1102	if e.winerror != 5:
1103	raise
1104	rc = _subprocess.GetExitCodeProcess(self._handle)
1105	if rc == _subprocess.STILL_ACTIVE:
1106	raise
1107	self.returncode = rc
1108
1109	kill = terminate
1110
1111	else:
1112	#
1113	# POSIX methods
1114	#
1115	def _get_handles(self, stdin, stdout, stderr):
1116	"""Construct and return tuple with IO objects:
1117	p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1118	"""
1119	to_close = set()
1120	p2cread, p2cwrite = None, None
1121	c2pread, c2pwrite = None, None
1122	errread, errwrite = None, None
1123
1124	if stdin is None:
1125	pass
1126	elif stdin == PIPE:
1127	p2cread, p2cwrite = self.pipe_cloexec()
1128	to_close.update((p2cread, p2cwrite))
1129	elif isinstance(stdin, int):
1130	p2cread = stdin
1131	else:
1132	# Assuming file-like object
1133	p2cread = stdin.fileno()
1134
1135	if stdout is None:
1136	pass
1137	elif stdout == PIPE:
1138	c2pread, c2pwrite = self.pipe_cloexec()
1139	to_close.update((c2pread, c2pwrite))
1140	elif isinstance(stdout, int):
1141	c2pwrite = stdout
1142	else:
1143	# Assuming file-like object
1144	c2pwrite = stdout.fileno()
1145
1146	if stderr is None:
1147	pass
1148	elif stderr == PIPE:
1149	errread, errwrite = self.pipe_cloexec()
1150	to_close.update((errread, errwrite))
1151	elif stderr == STDOUT:
1152	errwrite = c2pwrite
1153	elif isinstance(stderr, int):
1154	errwrite = stderr
1155	else:
1156	# Assuming file-like object
1157	errwrite = stderr.fileno()
1158
1159	return (p2cread, p2cwrite,
1160	c2pread, c2pwrite,
1161	errread, errwrite), to_close
1162
1163
1164	def _set_cloexec_flag(self, fd, cloexec=True):
1165	try:
1166	cloexec_flag = fcntl.FD_CLOEXEC
1167	except AttributeError:
1168	cloexec_flag = 1
1169
1170	old = fcntl.fcntl(fd, fcntl.F_GETFD)
1171	if cloexec:
1172	fcntl.fcntl(fd, fcntl.F_SETFD, old \| cloexec_flag)
1173	else:
1174	fcntl.fcntl(fd, fcntl.F_SETFD, old & ~cloexec_flag)
1175
1176
1177	def pipe_cloexec(self):
1178	"""Create a pipe with FDs set CLOEXEC."""
1179	# Pipes' FDs are set CLOEXEC by default because we don't want them
1180	# to be inherited by other subprocesses: the CLOEXEC flag is removed
1181	# from the child's FDs by _dup2(), between fork() and exec().
1182	# This is not atomic: we would need the pipe2() syscall for that.
1183	r, w = os.pipe()
1184	self._set_cloexec_flag(r)
1185	self._set_cloexec_flag(w)
1186	return r, w
1187
1188
1189	def _close_fds(self, but):
1190	if hasattr(os, 'closerange'):
1191	os.closerange(3, but)
1192	os.closerange(but + 1, MAXFD)
1193	else:
1194	for i in xrange(3, MAXFD):
1195	if i == but:
1196	continue
1197	try:
1198	os.close(i)
1199	except:
1200	pass
1201
1202
1203	def _execute_child(self, args, executable, preexec_fn, close_fds,
1204	cwd, env, universal_newlines,
1205	startupinfo, creationflags, threadsafe, shell, to_close,
1206	p2cread, p2cwrite,
1207	c2pread, c2pwrite,
1208	errread, errwrite):
1209	"""Execute program (POSIX version)"""
1210
1211	if isinstance(args, types.StringTypes):
1212	args = [args]
1213	else:
1214	args = list(args)
1215
1216	if shell:
1217	args = [SHELL, "-c"] + args
1218	if executable:
1219	args[0] = executable
1220
1221	if executable is None:
1222	executable = args[0]
1223
1224	def _close_in_parent(fd):
1225	os.close(fd)
1226	to_close.remove(fd)
1227
1228	if os2:
1229	# We use spawn on OS/2 instead of fork because fork is very
1230	# inefficient there (mostly due to the lack of COW page support
1231	# in the kernel).
1232	mode = os.P_NOWAIT
1233
1234	if close_fds:
1235	mode \|= os.P_2_NOINHERIT
1236	if threadsafe:
1237	mode \|= os.P_2_THREADSAFE
1238	stdfds = [ p2cread, c2pwrite, errwrite ]
1239
1240	pid = os.spawn2(mode, executable, args, cwd, env, stdfds)
1241
1242	# Child is launched. Close the parent's copy of those pipe
1243	# handles that only the child should have open. You need
1244	# to make sure that no handles to the write end of the
1245	# output pipe are maintained in this process or else the
1246	# pipe will not close when the child process exits and the
1247	# ReadFile will hang.
1248	if p2cread is not None and p2cwrite is not None:
1249	_close_in_parent(p2cread)
1250	if c2pwrite is not None and c2pread is not None:
1251	_close_in_parent(c2pwrite)
1252	if errwrite is not None and errread is not None:
1253	_close_in_parent(errwrite)
1254
1255	self._child_created = True
1256	self.pid = pid
1257
1258	else:
1259	# For transferring possible exec failure from child to parent
1260	# The first char specifies the exception type: 0 means
1261	# OSError, 1 means some other error.
1262	errpipe_read, errpipe_write = self.pipe_cloexec()
1263	try:
1264	try:
1265	gc_was_enabled = gc.isenabled()
1266	# Disable gc to avoid bug where gc -> file_dealloc ->
1267	# write to stderr -> hang. http://bugs.python.org/issue1336
1268	gc.disable()
1269	try:
1270	self.pid = os.fork()
1271	except:
1272	if gc_was_enabled:
1273	gc.enable()
1274	raise
1275	self._child_created = True
1276	if self.pid == 0:
1277	# Child
1278	try:
1279	# Close parent's pipe ends
1280	if p2cwrite is not None:
1281	os.close(p2cwrite)
1282	if c2pread is not None:
1283	os.close(c2pread)
1284	if errread is not None:
1285	os.close(errread)
1286	os.close(errpipe_read)
1287
1288	# When duping fds, if there arises a situation
1289	# where one of the fds is either 0, 1 or 2, it
1290	# is possible that it is overwritten (#12607).
1291	if c2pwrite == 0:
1292	c2pwrite = os.dup(c2pwrite)
1293	if errwrite == 0 or errwrite == 1:
1294	errwrite = os.dup(errwrite)
1295
1296	# Dup fds for child
1297	def _dup2(a, b):
1298	# dup2() removes the CLOEXEC flag but
1299	# we must do it ourselves if dup2()
1300	# would be a no-op (issue #10806).
1301	if a == b:
1302	self._set_cloexec_flag(a, False)
1303	elif a is not None:
1304	os.dup2(a, b)
1305	_dup2(p2cread, 0)
1306	_dup2(c2pwrite, 1)
1307	_dup2(errwrite, 2)
1308
1309	# Close pipe fds. Make sure we don't close the
1310	# same fd more than once, or standard fds.
1311	closed = { None }
1312	for fd in [p2cread, c2pwrite, errwrite]:
1313	if fd not in closed and fd > 2:
1314	os.close(fd)
1315	closed.add(fd)
1316
1317	if cwd is not None:
1318	os.chdir(cwd)
1319
1320	if preexec_fn:
1321	preexec_fn()
1322
1323	# Close all other fds, if asked for - after
1324	# preexec_fn(), which may open FDs.
1325	if close_fds:
1326	self._close_fds(but=errpipe_write)
1327
1328	if env is None:
1329	os.execvp(executable, args)
1330	else:
1331	os.execvpe(executable, args, env)
1332
1333	except:
1334	exc_type, exc_value, tb = sys.exc_info()
1335	# Save the traceback and attach it to the exception object
1336	exc_lines = traceback.format_exception(exc_type,
1337	exc_value,
1338	tb)
1339	exc_value.child_traceback = ''.join(exc_lines)
1340	os.write(errpipe_write, pickle.dumps(exc_value))
1341
1342	# This exitcode won't be reported to applications, so it
1343	# really doesn't matter what we return.
1344	os._exit(255)
1345
1346	# Parent
1347	if gc_was_enabled:
1348	gc.enable()
1349	finally:
1350	# be sure the FD is closed no matter what
1351	os.close(errpipe_write)
1352
1353	# Wait for exec to fail or succeed; possibly raising exception
1354	# Exception limited to 1M
1355	data = _eintr_retry_call(os.read, errpipe_read, 1048576)
1356	finally:
1357	if p2cread is not None and p2cwrite is not None:
1358	_close_in_parent(p2cread)
1359	if c2pwrite is not None and c2pread is not None:
1360	_close_in_parent(c2pwrite)
1361	if errwrite is not None and errread is not None:
1362	_close_in_parent(errwrite)
1363
1364	# be sure the FD is closed no matter what
1365	os.close(errpipe_read)
1366
1367	if data != "":
1368	try:
1369	_eintr_retry_call(os.waitpid, self.pid, 0)
1370	except OSError as e:
1371	if e.errno != errno.ECHILD:
1372	raise
1373	child_exception = pickle.loads(data)
1374	raise child_exception
1375
1376
1377	def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1378	_WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1379	_WEXITSTATUS=os.WEXITSTATUS):
1380	# This method is called (indirectly) by __del__, so it cannot
1381	# refer to anything outside of its local scope."""
1382	if _WIFSIGNALED(sts):
1383	self.returncode = -_WTERMSIG(sts)
1384	elif _WIFEXITED(sts):
1385	self.returncode = _WEXITSTATUS(sts)
1386	else:
1387	# Should never happen
1388	raise RuntimeError("Unknown child exit status!")
1389
1390
1391	def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1392	_WNOHANG=os.WNOHANG, _os_error=os.error, _ECHILD=errno.ECHILD):
1393	"""Check if child process has terminated. Returns returncode
1394	attribute.
1395
1396	This method is called by __del__, so it cannot reference anything
1397	outside of the local scope (nor can any methods it calls).
1398
1399	"""
1400	if self.returncode is None:
1401	try:
1402	pid, sts = _waitpid(self.pid, _WNOHANG)
1403	if pid == self.pid:
1404	self._handle_exitstatus(sts)
1405	except _os_error as e:
1406	if _deadstate is not None:
1407	self.returncode = _deadstate
1408	if e.errno == _ECHILD:
1409	# This happens if SIGCLD is set to be ignored or
1410	# waiting for child processes has otherwise been
1411	# disabled for our process. This child is dead, we
1412	# can't get the status.
1413	# http://bugs.python.org/issue15756
1414	self.returncode = 0
1415	return self.returncode
1416
1417
1418	def wait(self):
1419	"""Wait for child process to terminate. Returns returncode
1420	attribute."""
1421	while self.returncode is None:
1422	try:
1423	pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1424	except OSError as e:
1425	if e.errno != errno.ECHILD:
1426	raise
1427	# This happens if SIGCLD is set to be ignored or waiting
1428	# for child processes has otherwise been disabled for our
1429	# process. This child is dead, we can't get the status.
1430	pid = self.pid
1431	sts = 0
1432	# Check the pid and loop as waitpid has been known to return
1433	# 0 even without WNOHANG in odd situations. issue14396.
1434	if pid == self.pid:
1435	self._handle_exitstatus(sts)
1436	return self.returncode
1437
1438
1439	def _communicate(self, input):
1440	if os2:
1441	def _readerthread(file, buffer):
1442	while True:
1443	data = _eintr_retry_call(file.read)
1444	if data == "":
1445	try:
1446	file.close()
1447	except IOError as e:
1448	# kLIBC close may fail with EBADF or even Error 0,
1449	# ignore for now
1450	pass
1451	break
1452	buffer.append(data)
1453
1454	stdout = None # Return
1455	stderr = None # Return
1456
1457	if self.stdout:
1458	stdout = []
1459	stdout_thread = threading.Thread(target=_readerthread,
1460	args=(self.stdout, stdout))
1461	stdout_thread.setDaemon(True)
1462	stdout_thread.start()
1463
1464	if self.stderr:
1465	stderr = []
1466	stderr_thread = threading.Thread(target=_readerthread,
1467	args=(self.stderr, stderr))
1468	stderr_thread.setDaemon(True)
1469	stderr_thread.start()
1470
1471	if self.stdin:
1472	if input is not None:
1473	try:
1474	_eintr_retry_call(self.stdin.write, input)
1475	except IOError as e:
1476	if e.errno != errno.EPIPE:
1477	raise
1478	try:
1479	self.stdin.close()
1480	except IOError as e:
1481	# kLIBC close may fail with EBADF or even Error 0,
1482	# ignore for now
1483	pass
1484
1485	if self.stdout:
1486	stdout_thread.join()
1487	if self.stderr:
1488	stderr_thread.join()
1489
1490	else:
1491	if self.stdin:
1492	# Flush stdio buffer. This might block, if the user has
1493	# been writing to .stdin in an uncontrolled fashion.
1494	self.stdin.flush()
1495	if not input:
1496	self.stdin.close()
1497
1498	if _has_poll:
1499	stdout, stderr = self._communicate_with_poll(input)
1500	else:
1501	stdout, stderr = self._communicate_with_select(input)
1502
1503	# All data exchanged. Translate lists into strings.
1504	if stdout is not None:
1505	stdout = ''.join(stdout)
1506	if stderr is not None:
1507	stderr = ''.join(stderr)
1508
1509	# Translate newlines, if requested. We cannot let the file
1510	# object do the translation: It is based on stdio, which is
1511	# impossible to combine with select (unless forcing no
1512	# buffering).
1513	if self.universal_newlines and hasattr(file, 'newlines'):
1514	if stdout:
1515	stdout = self._translate_newlines(stdout)
1516	if stderr:
1517	stderr = self._translate_newlines(stderr)
1518
1519	self.wait()
1520	return (stdout, stderr)
1521
1522
1523	def _communicate_with_poll(self, input):
1524	stdout = None # Return
1525	stderr = None # Return
1526	fd2file = {}
1527	fd2output = {}
1528
1529	poller = select.poll()
1530	def register_and_append(file_obj, eventmask):
1531	poller.register(file_obj.fileno(), eventmask)
1532	fd2file[file_obj.fileno()] = file_obj
1533
1534	def close_unregister_and_remove(fd):
1535	poller.unregister(fd)
1536	fd2file[fd].close()
1537	fd2file.pop(fd)
1538
1539	if self.stdin and input:
1540	register_and_append(self.stdin, select.POLLOUT)
1541
1542	select_POLLIN_POLLPRI = select.POLLIN \| select.POLLPRI
1543	if self.stdout:
1544	register_and_append(self.stdout, select_POLLIN_POLLPRI)
1545	fd2output[self.stdout.fileno()] = stdout = []
1546	if self.stderr:
1547	register_and_append(self.stderr, select_POLLIN_POLLPRI)
1548	fd2output[self.stderr.fileno()] = stderr = []
1549
1550	input_offset = 0
1551	while fd2file:
1552	try:
1553	ready = poller.poll()
1554	except select.error, e:
1555	if e.args[0] == errno.EINTR:
1556	continue
1557	raise
1558
1559	for fd, mode in ready:
1560	if mode & select.POLLOUT:
1561	chunk = input[input_offset : input_offset + _PIPE_BUF]
1562	try:
1563	input_offset += os.write(fd, chunk)
1564	except OSError as e:
1565	if e.errno == errno.EPIPE:
1566	close_unregister_and_remove(fd)
1567	else:
1568	raise
1569	else:
1570	if input_offset >= len(input):
1571	close_unregister_and_remove(fd)
1572	elif mode & select_POLLIN_POLLPRI:
1573	data = os.read(fd, 4096)
1574	if not data:
1575	close_unregister_and_remove(fd)
1576	fd2output[fd].append(data)
1577	else:
1578	# Ignore hang up or errors.
1579	close_unregister_and_remove(fd)
1580
1581	return (stdout, stderr)
1582
1583
1584	def _communicate_with_select(self, input):
1585	read_set = []
1586	write_set = []
1587	stdout = None # Return
1588	stderr = None # Return
1589
1590	if self.stdin and input:
1591	write_set.append(self.stdin)
1592	if self.stdout:
1593	read_set.append(self.stdout)
1594	stdout = []
1595	if self.stderr:
1596	read_set.append(self.stderr)
1597	stderr = []
1598
1599	input_offset = 0
1600	while read_set or write_set:
1601	try:
1602	rlist, wlist, xlist = select.select(read_set, write_set, [])
1603	except select.error, e:
1604	if e.args[0] == errno.EINTR:
1605	continue
1606	raise
1607
1608	if self.stdin in wlist:
1609	chunk = input[input_offset : input_offset + _PIPE_BUF]
1610	try:
1611	bytes_written = os.write(self.stdin.fileno(), chunk)
1612	except OSError as e:
1613	if e.errno == errno.EPIPE:
1614	self.stdin.close()
1615	write_set.remove(self.stdin)
1616	else:
1617	raise
1618	else:
1619	input_offset += bytes_written
1620	if input_offset >= len(input):
1621	self.stdin.close()
1622	write_set.remove(self.stdin)
1623
1624	if self.stdout in rlist:
1625	data = os.read(self.stdout.fileno(), 1024)
1626	if data == "":
1627	self.stdout.close()
1628	read_set.remove(self.stdout)
1629	stdout.append(data)
1630
1631	if self.stderr in rlist:
1632	data = os.read(self.stderr.fileno(), 1024)
1633	if data == "":
1634	self.stderr.close()
1635	read_set.remove(self.stderr)
1636	stderr.append(data)
1637
1638	return (stdout, stderr)
1639
1640
1641	def send_signal(self, sig):
1642	"""Send a signal to the process
1643	"""
1644	os.kill(self.pid, sig)
1645
1646	def terminate(self):
1647	"""Terminate the process with SIGTERM
1648	"""
1649	self.send_signal(signal.SIGTERM)
1650
1651	def kill(self):
1652	"""Kill the process with SIGKILL
1653	"""
1654	self.send_signal(signal.SIGKILL)
1655
1656
1657	def _demo_posix():
1658	#
1659	# Example 1: Simple redirection: Get process list
1660	#
1661	plist = Popen(["ps"], stdout=PIPE).communicate()[0]
1662	print "Process list:"
1663	print plist
1664
1665	#
1666	# Example 2: Change uid before executing child
1667	#
1668	if os.getuid() == 0:
1669	p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1670	p.wait()
1671
1672	#
1673	# Example 3: Connecting several subprocesses
1674	#
1675	print "Looking for 'hda'..."
1676	p1 = Popen(["dmesg"], stdout=PIPE)
1677	p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
1678	print repr(p2.communicate()[0])
1679
1680	#
1681	# Example 4: Catch execution error
1682	#
1683	print
1684	print "Trying a weird file..."
1685	try:
1686	print Popen(["/this/path/does/not/exist"]).communicate()
1687	except OSError, e:
1688	if e.errno == errno.ENOENT:
1689	print "The file didn't exist. I thought so..."
1690	print "Child traceback:"
1691	print e.child_traceback
1692	else:
1693	print "Error", e.errno
1694	else:
1695	print >>sys.stderr, "Gosh. No error."
1696
1697
1698	def _demo_windows():
1699	#
1700	# Example 1: Connecting several subprocesses
1701	#
1702	print "Looking for 'PROMPT' in set output..."
1703	p1 = Popen("set", stdout=PIPE, shell=True)
1704	p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
1705	print repr(p2.communicate()[0])
1706
1707	#
1708	# Example 2: Simple execution of program
1709	#
1710	print "Executing calc..."
1711	p = Popen("calc")
1712	p.wait()
1713
1714
1715	def _demo_os2():
1716	#
1717	# Example 1: Simple redirection: Get current process data
1718	#
1719	pdata = Popen(["pstat", "/p:%s" % hex(os.getpid())[2:]], stdout=PIPE, stderr=PIPE).communicate()[0]
1720	print "Current process data:"
1721	print pdata
1722	#
1723	# Example 2: Connecting several subprocesses
1724	#
1725	p1 = Popen(["pstat", "/p:%s" % hex(os.getpid())[2:]], stdout=PIPE)
1726	p2 = Popen(["grep", "\.EXE"], stdin=p1.stdout, stdout=PIPE)
1727	print "Current .EXE:"
1728	print p2.communicate()[0]
1729
1730
1731	if __name__ == "__main__":
1732	if mswindows:
1733	_demo_windows()
1734	elif os2:
1735	_demo_os2()
1736	else:
1737	_demo_posix()

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: python/trunk/Lib/subprocess.py@ 1281

Download in other formats: