$Id: Fork.os2 1279 2004-02-25 03:26:45Z bird $ Fork Design Draft -------------------- 1.0 Intro ---------- blah. 1.1 The SuS fork() Description ------------------------------ NAME fork - create a new process SYNOPSIS #include pid_t fork(void); DESCRIPTION The fork() function shall create a new process. The new process (child process) shall be an exact copy of the calling process (parent process) except as detailed below: * The child process shall have a unique process ID. * The child process ID also shall not match any active process group ID. * The child process shall have a different parent process ID, which shall be the process ID of the calling process. * The child process shall have its own copy of the parent's file descriptors. Each of the child's file descriptors shall refer to the same open file description with the corresponding file descriptor of the parent. * The child process shall have its own copy of the parent's open directory streams. Each open directory stream in the child process may share directory stream positioning with the corresponding directory stream of the parent. * [XSI] The child process shall have its own copy of the parent's message catalog descriptors. * The child process' values of tms_utime, tms_stime, tms_cutime, and tms_cstime shall be set to 0. * The time left until an alarm clock signal shall be reset to zero, and the alarm, if any, shall be canceled; see alarm() . * [XSI] All semadj values shall be cleared. * File locks set by the parent process shall not be inherited by the child process. * The set of signals pending for the child process shall be initialized to the empty set. * [XSI] Interval timers shall be reset in the child process. * [SEM] Any semaphores that are open in the parent process shall also be open in the child process. * [ML] The child process shall not inherit any address space memory locks established by the parent process via calls to mlockall() or mlock(). * [MF|SHM] Memory mappings created in the parent shall be retained in the child process. MAP_PRIVATE mappings inherited from the parent shall also be MAP_PRIVATE mappings in the child, and any modifications to the data in these mappings made by the parent prior to calling fork() shall be visible to the child. Any modifications to the data in MAP_PRIVATE mappings made by the parent after fork() returns shall be visible only to the parent. Modifications to the data in MAP_PRIVATE mappings made by the child shall be visible only to the child. * [PS] For the SCHED_FIFO and SCHED_RR scheduling policies, the child process shall inherit the policy and priority settings of the parent process during a fork() function. For other s cheduling policies, the policy and priority settings on fork() are implementation-defined. * [TMR] Per-process timers created by the parent shall not be inherited by the child process. * [MSG] The child process shall have its own copy of the message queue descriptors of the parent. Each of the message descriptors of the child shall refer to the same open message queue description as the corresponding message descriptor of the parent. * [AIO] No asynchronous input or asynchronous output operations shall be inherited by the child process. * A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called. [THR] Fork handlers may be established by means of the pthread_atfork() function in order to maintain application invariants across fork() calls. When the application calls fork() from a signal handler and any of the fork handlers registered by pthread_atfork() calls a function that is not asynch-signal-safe, the behavior is undefined. * [TRC TRI] If the Trace option and the Trace Inherit option are both supported: If the calling process was being traced in a trace stream that had its inheritance policy set to POSIX_TRACE_INHERITED, the child process shall be traced into that trace stream, and the child process shall inherit the parent's mapping of trace event names to trace event type identifiers. If the trace stream in which the calling process was being traced had its inheritance policy set to POSIX_TRACE_CLOSE_FOR_CHILD, the child process shall not be traced into that trace stream. The inheritance policy is set by a call to the posix_trace_attr_setinherited() function. * [TRC] If the Trace option is supported, but the Trace Inherit option is not supported: The child process shall not be traced into any of the trace streams of its parent process. * [TRC] If the Trace option is supported, the child process of a trace controller process shall not control the trace streams controlled by its parent process. * [CPT] The initial value of the CPU-time clock of the child process shall be set to zero. * [TCT] The initial value of the CPU-time clock of the single thread of the child process shall be set to zero. All other process characteristics defined by IEEE Std 1003.1-2001 shall be the same in the parent and child processes. The inheritance of process characteristics not defined by IEEE Std 1003.1-2001 is unspecified by IEEE Std 1003.1-2001. After fork(), both the parent and the child processes shall be capable of executing independently before either one terminates. RETURN VALUE Upon successful completion, fork() shall return 0 to the child process and shall return the process ID of the child process to the parent process. Both processes shall continue to execute from the fork() function. Otherwise, -1 shall be returned to the parent process, no child process shall be created, and errno shall be set to indicate the error. ERRORS The fork() function shall fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. The fork() function may fail if: [ENOMEM] Insufficient storage space is available. 2.0 Requirements and Assumptions Of The Implementation ------------------------------------------------------ The Innotek LIBC fork() implementation will require the following features in LIBC to work: 1. A shared process management internal to LIBC for communication to the child that a fork() is in progress. 2. A very generalized and varied set of fork helper functions to archive maximum flexibility of the implementation. 3. Extended versions of some memory related OS/2 APIs must be implemented. The implemenetation will further make the following assumption about the operation of OS/2: 1. DosExecPgm will not return till all DLLs are initated successfully. 3.0 The Shared Process Management --------------------------------- The fork() implementation requires a method of telling the child process that it's being forked and must take a very different startup route. For some other LIBC apis there is need for parent -> child and child -> parent information exchange. More specifically, the inheritance of sockets, signals, the different scheduler actions of a posix_spawn[p]() call, and possibly some process group stuff related to posix_spawn too if we get it figured out eventually. All this was parent -> child during spawn/fork. A need exist also for child -> parent notification and possibly exchange for process termination. It might be necessary to reimplement the different wait apis and implement SIGCHLD, it's likely that those tasks will make such demands. The choice is now whether or not to make this shared process management specific to each LIBC version or try to make it survive normal LIBC updates. Making is specific have advantages in code size and memory footprint (no reserved field), however it have certain disadvantages when LIBC is updated. The other option is to use a named shared memory object, defining the content with reserved space for later extensions so several versions of LIBC with more or less features implemented can co use the memory space. The latter option is prefered since it allows more applications to interoperate, it causes less shared memory waste, the shared memory can be located in high memory and it would be possible to fork processes using multiple versions of LIBC. The shared memory must be named \SHAREMEM\INNOTEKLIBC.V01, the version number being the one of the shared memory layout and contents, it will only be increased when incompatible changes are made. The shared memory will be protected by an standard OS/2 mutex semaphore. It will not use any fast R3 semaphore since the the usage frequency is low and the result of a messup may be disastrous. Care must be take for avoiding creation races and owner died scenarios. The memory will have a fixed size, since adding segments is very hard. Thus the size must be large enough to cope with a great deal of processes, but bearing in mind that OS/2 normally doesn't support more than a 1000 processes, with a theoritical max of some 4000 (being the max thread count). A very simplistic allocation scheme will be implemented. Practically speaking a fixed block size pool would do fine for the process structure, while for the misc structures like socket lists a linked list based heap would do fine. The process blocks will be rounded up to in size adding a reasonable amount of space resevered for future extensions. Reserved space must be all zeroed. The fork() specific members of the process block will be a pointer to the shared memory object for the fork operation (the fork handle) and list of forkable modules. The fork handle will it self contain information indicating whether or not another LIBC version have already started fork() handling in the child. The presense of the fork handle means that the child is being forked and normal dll init and startup will not be executed, but a registered callback will be called to do the forking of each module. (more details in section 4.0) The parent will before spawn, fork and exec (essentially before DosExecPgm or DosStartSession) create a process block for the child to be born and link it into an embryo list in the shared memory block. The child will find the process block by looking searching an embryo list using the parent pid as key. All DosExecPgm and DosStartSession calls are serialized within one LIBC version. (If some empty headed programmer manages to link together a program which may end up using two or more LIBC versions and having two or more thread doing DosExecPgm at the very same time, well then he really deserves what ever trouble he gets! At least don't blame me!) Process blocks will have to stay around after the process terminated (for child -> parent term exchange), a cleanup mechanism will be invoked whenever a free memory threshold is reached. All processes will register exit list handlers to mark the process block as zombie (and later perhaps setting error codes and notifying waiters/child-listeners). 4.0 The fork() Implementation ----------------------------- The implementation will be based on a fork handle and a set of primitives. The fork handle is a pointer to an shared memory object allocated for the occation and which will be freed before fork() returns. The primitives all operates on this handle and will be provided using a callback table in order to fully support multiple LIBC versions. 4.1 Forkable Executable and DLLs -------------------------------- The support for fork() is an optional feature of LIBC. The default executable produced with LIBC and GCC will not be forkable. The fork support will be based on registration of the DLLs and EXEs in their LIBC supplied startup code (crt0/dll0). A set of fork versions of these modules will be made. The big differnece between the ordinary crt0/dll0 and the forkable crt0/dll0 is a per module structure, a call to register this, and the handling of the return code of that call. The structure will contain these fields: - chain pointer. - data segment base address. - data segment end address. - fork callback function. The fork callback function is called _atfork_callback, it takes the fork handle, module structure, and an operation enum as arguments. LIBC will contain a default implementation of _atfork_callback() which simply duplicates the data segment. The register call, __libc_ForkRegisterModule(), will return: - 0 if normal process startup. no forking. - 1 if fork() is in progress. The crt0/dll0 code will then not call any standard initiation code, but let the _atfork_callback() do all necessary stuff. 4.2 Fork Primitives ------------------- These primitives are provided by the fork implementation in the fork handle structure. We will define a set of these primitives now, if later new ones are added the users of these must check that they are actually present. Example: rc = pForkHandle->pOps->pfnDuplicatePages(pModule->pvDataBase, pModule->pvDataEnd, __LIBC_FORK_ONLY_DIRTY); if (rc) return rc; /* failure */ Prototypes: /** * Duplicating a number of pages from pvStart to pvEnd. * @returns 0 on success. * @returns appropriate non-zero error code on failure. * @param pForkHandle Handle of the current fork operation. * @param pvStart Pointer to start of the pages. Rounded down. * @param pvEnd Pointer to end of the pages. Rounded up. * @param fFlags __LIBC_FORK_ONLY_DIRTY means checking whether the * pages are actually dirty before bothering touching * and copying them. (Using the partically broken * DosQueryMemState() API.) * __LIBC_FORK_ALL means not to bother checking, but * just go ahead copying all the pages. */ int pfnDuplicatePages(__LIBC_FORKHANDLE *pForkHandle, void *pvStart, void *pvEnd, unsigned fFlags); /** * Invoke a function in the child process giving it an chunk of input. * The function is invoked the next time the fork buffer is flushed, * call pfnFlush() if the return code is desired. * * @returns 0 on success. * @returns appropriate non-zero error code on failure. * @param pForkHandle Handle of the current fork operation. * @param pfn Pointer to the function to invoke in the child. * The function gets the fork handle, pointer to * the argument memory chunk and the size of that. * The function must return 0 on success, and non-zero * on failure. * @param pvArg Pointer to a block of memory of size cbArg containing * input to be copied to the child and given to pfn upon * invocation. */ int pfnInvoke(int *(pfn)(__LIBC_FORKHANDLE *pForkHandle, void *pvArg, size_t cbArg), void *pvArg, size_t cbArg); /** * Flush the fork() buffer. Meaning taking what ever is in the fork buffer * and let the child process it. * This might be desired to get the result of a pfnInvoke() in a near * synchornous way. * @returns 0 on success. * @returns appropriate non-zero error code on failure. * @param pForkHandle Handle of the current fork operation. */ int pfnFlush(__LIBC_FORKHANDLE *pForkHandle); ...