[2] | 1 |
|
---|
| 2 | :mod:`mmap` --- Memory-mapped file support
|
---|
| 3 | ==========================================
|
---|
| 4 |
|
---|
| 5 | .. module:: mmap
|
---|
| 6 | :synopsis: Interface to memory-mapped files for Unix and Windows.
|
---|
| 7 |
|
---|
| 8 |
|
---|
| 9 | Memory-mapped file objects behave like both strings and like file objects.
|
---|
| 10 | Unlike normal string objects, however, these are mutable. You can use mmap
|
---|
| 11 | objects in most places where strings are expected; for example, you can use
|
---|
| 12 | the :mod:`re` module to search through a memory-mapped file. Since they're
|
---|
| 13 | mutable, you can change a single character by doing ``obj[index] = 'a'``, or
|
---|
| 14 | change a substring by assigning to a slice: ``obj[i1:i2] = '...'``. You can
|
---|
| 15 | also read and write data starting at the current file position, and
|
---|
| 16 | :meth:`seek` through the file to different positions.
|
---|
| 17 |
|
---|
| 18 | A memory-mapped file is created by the :class:`mmap` constructor, which is
|
---|
| 19 | different on Unix and on Windows. In either case you must provide a file
|
---|
| 20 | descriptor for a file opened for update. If you wish to map an existing Python
|
---|
| 21 | file object, use its :meth:`fileno` method to obtain the correct value for the
|
---|
| 22 | *fileno* parameter. Otherwise, you can open the file using the
|
---|
| 23 | :func:`os.open` function, which returns a file descriptor directly (the file
|
---|
| 24 | still needs to be closed when done).
|
---|
| 25 |
|
---|
[391] | 26 | .. note::
|
---|
| 27 | If you want to create a memory-mapping for a writable, buffered file, you
|
---|
| 28 | should :func:`~io.IOBase.flush` the file first. This is necessary to ensure
|
---|
| 29 | that local modifications to the buffers are actually available to the
|
---|
| 30 | mapping.
|
---|
| 31 |
|
---|
[2] | 32 | For both the Unix and Windows versions of the constructor, *access* may be
|
---|
| 33 | specified as an optional keyword parameter. *access* accepts one of three
|
---|
| 34 | values: :const:`ACCESS_READ`, :const:`ACCESS_WRITE`, or :const:`ACCESS_COPY`
|
---|
| 35 | to specify read-only, write-through or copy-on-write memory respectively.
|
---|
| 36 | *access* can be used on both Unix and Windows. If *access* is not specified,
|
---|
| 37 | Windows mmap returns a write-through mapping. The initial memory values for
|
---|
| 38 | all three access types are taken from the specified file. Assignment to an
|
---|
| 39 | :const:`ACCESS_READ` memory map raises a :exc:`TypeError` exception.
|
---|
| 40 | Assignment to an :const:`ACCESS_WRITE` memory map affects both memory and the
|
---|
| 41 | underlying file. Assignment to an :const:`ACCESS_COPY` memory map affects
|
---|
| 42 | memory but does not update the underlying file.
|
---|
| 43 |
|
---|
| 44 | .. versionchanged:: 2.5
|
---|
| 45 | To map anonymous memory, -1 should be passed as the fileno along with the
|
---|
| 46 | length.
|
---|
| 47 |
|
---|
| 48 | .. versionchanged:: 2.6
|
---|
| 49 | mmap.mmap has formerly been a factory function creating mmap objects. Now
|
---|
| 50 | mmap.mmap is the class itself.
|
---|
| 51 |
|
---|
| 52 | .. class:: mmap(fileno, length[, tagname[, access[, offset]]])
|
---|
| 53 |
|
---|
| 54 | **(Windows version)** Maps *length* bytes from the file specified by the
|
---|
| 55 | file handle *fileno*, and creates a mmap object. If *length* is larger
|
---|
| 56 | than the current size of the file, the file is extended to contain *length*
|
---|
| 57 | bytes. If *length* is ``0``, the maximum length of the map is the current
|
---|
| 58 | size of the file, except that if the file is empty Windows raises an
|
---|
| 59 | exception (you cannot create an empty mapping on Windows).
|
---|
| 60 |
|
---|
| 61 | *tagname*, if specified and not ``None``, is a string giving a tag name for
|
---|
| 62 | the mapping. Windows allows you to have many different mappings against
|
---|
| 63 | the same file. If you specify the name of an existing tag, that tag is
|
---|
| 64 | opened, otherwise a new tag of this name is created. If this parameter is
|
---|
| 65 | omitted or ``None``, the mapping is created without a name. Avoiding the
|
---|
| 66 | use of the tag parameter will assist in keeping your code portable between
|
---|
| 67 | Unix and Windows.
|
---|
| 68 |
|
---|
| 69 | *offset* may be specified as a non-negative integer offset. mmap references
|
---|
| 70 | will be relative to the offset from the beginning of the file. *offset*
|
---|
| 71 | defaults to 0. *offset* must be a multiple of the ALLOCATIONGRANULARITY.
|
---|
| 72 |
|
---|
| 73 |
|
---|
| 74 | .. class:: mmap(fileno, length[, flags[, prot[, access[, offset]]]])
|
---|
| 75 | :noindex:
|
---|
| 76 |
|
---|
| 77 | **(Unix version)** Maps *length* bytes from the file specified by the file
|
---|
| 78 | descriptor *fileno*, and returns a mmap object. If *length* is ``0``, the
|
---|
| 79 | maximum length of the map will be the current size of the file when
|
---|
| 80 | :class:`mmap` is called.
|
---|
| 81 |
|
---|
| 82 | *flags* specifies the nature of the mapping. :const:`MAP_PRIVATE` creates a
|
---|
| 83 | private copy-on-write mapping, so changes to the contents of the mmap
|
---|
| 84 | object will be private to this process, and :const:`MAP_SHARED` creates a
|
---|
| 85 | mapping that's shared with all other processes mapping the same areas of
|
---|
| 86 | the file. The default value is :const:`MAP_SHARED`.
|
---|
| 87 |
|
---|
| 88 | *prot*, if specified, gives the desired memory protection; the two most
|
---|
| 89 | useful values are :const:`PROT_READ` and :const:`PROT_WRITE`, to specify
|
---|
| 90 | that the pages may be read or written. *prot* defaults to
|
---|
| 91 | :const:`PROT_READ \| PROT_WRITE`.
|
---|
| 92 |
|
---|
| 93 | *access* may be specified in lieu of *flags* and *prot* as an optional
|
---|
| 94 | keyword parameter. It is an error to specify both *flags*, *prot* and
|
---|
| 95 | *access*. See the description of *access* above for information on how to
|
---|
| 96 | use this parameter.
|
---|
| 97 |
|
---|
| 98 | *offset* may be specified as a non-negative integer offset. mmap references
|
---|
| 99 | will be relative to the offset from the beginning of the file. *offset*
|
---|
| 100 | defaults to 0. *offset* must be a multiple of the PAGESIZE or
|
---|
| 101 | ALLOCATIONGRANULARITY.
|
---|
| 102 |
|
---|
[391] | 103 | To ensure validity of the created memory mapping the file specified
|
---|
| 104 | by the descriptor *fileno* is internally automatically synchronized
|
---|
| 105 | with physical backing store on Mac OS X and OpenVMS.
|
---|
| 106 |
|
---|
[2] | 107 | This example shows a simple way of using :class:`mmap`::
|
---|
| 108 |
|
---|
| 109 | import mmap
|
---|
| 110 |
|
---|
| 111 | # write a simple example file
|
---|
| 112 | with open("hello.txt", "wb") as f:
|
---|
| 113 | f.write("Hello Python!\n")
|
---|
| 114 |
|
---|
| 115 | with open("hello.txt", "r+b") as f:
|
---|
| 116 | # memory-map the file, size 0 means whole file
|
---|
[391] | 117 | mm = mmap.mmap(f.fileno(), 0)
|
---|
[2] | 118 | # read content via standard file methods
|
---|
[391] | 119 | print mm.readline() # prints "Hello Python!"
|
---|
[2] | 120 | # read content via slice notation
|
---|
[391] | 121 | print mm[:5] # prints "Hello"
|
---|
[2] | 122 | # update content using slice notation;
|
---|
| 123 | # note that new content must have same size
|
---|
[391] | 124 | mm[6:] = " world!\n"
|
---|
[2] | 125 | # ... and read again using standard file methods
|
---|
[391] | 126 | mm.seek(0)
|
---|
| 127 | print mm.readline() # prints "Hello world!"
|
---|
[2] | 128 | # close the map
|
---|
[391] | 129 | mm.close()
|
---|
[2] | 130 |
|
---|
| 131 |
|
---|
| 132 | The next example demonstrates how to create an anonymous map and exchange
|
---|
| 133 | data between the parent and child processes::
|
---|
| 134 |
|
---|
| 135 | import mmap
|
---|
| 136 | import os
|
---|
| 137 |
|
---|
[391] | 138 | mm = mmap.mmap(-1, 13)
|
---|
| 139 | mm.write("Hello world!")
|
---|
[2] | 140 |
|
---|
| 141 | pid = os.fork()
|
---|
| 142 |
|
---|
| 143 | if pid == 0: # In a child process
|
---|
[391] | 144 | mm.seek(0)
|
---|
| 145 | print mm.readline()
|
---|
[2] | 146 |
|
---|
[391] | 147 | mm.close()
|
---|
[2] | 148 |
|
---|
| 149 |
|
---|
| 150 | Memory-mapped file objects support the following methods:
|
---|
| 151 |
|
---|
| 152 |
|
---|
| 153 | .. method:: close()
|
---|
| 154 |
|
---|
[391] | 155 | Closes the mmap. Subsequent calls to other methods of the object will
|
---|
| 156 | result in a ValueError exception being raised. This will not close
|
---|
| 157 | the open file.
|
---|
[2] | 158 |
|
---|
| 159 |
|
---|
| 160 | .. method:: find(string[, start[, end]])
|
---|
| 161 |
|
---|
| 162 | Returns the lowest index in the object where the substring *string* is
|
---|
| 163 | found, such that *string* is contained in the range [*start*, *end*].
|
---|
| 164 | Optional arguments *start* and *end* are interpreted as in slice notation.
|
---|
| 165 | Returns ``-1`` on failure.
|
---|
| 166 |
|
---|
| 167 |
|
---|
| 168 | .. method:: flush([offset, size])
|
---|
| 169 |
|
---|
| 170 | Flushes changes made to the in-memory copy of a file back to disk. Without
|
---|
| 171 | use of this call there is no guarantee that changes are written back before
|
---|
| 172 | the object is destroyed. If *offset* and *size* are specified, only
|
---|
| 173 | changes to the given range of bytes will be flushed to disk; otherwise, the
|
---|
| 174 | whole extent of the mapping is flushed.
|
---|
| 175 |
|
---|
| 176 | **(Windows version)** A nonzero value returned indicates success; zero
|
---|
| 177 | indicates failure.
|
---|
| 178 |
|
---|
| 179 | **(Unix version)** A zero value is returned to indicate success. An
|
---|
| 180 | exception is raised when the call failed.
|
---|
| 181 |
|
---|
| 182 |
|
---|
| 183 | .. method:: move(dest, src, count)
|
---|
| 184 |
|
---|
| 185 | Copy the *count* bytes starting at offset *src* to the destination index
|
---|
| 186 | *dest*. If the mmap was created with :const:`ACCESS_READ`, then calls to
|
---|
[391] | 187 | move will raise a :exc:`TypeError` exception.
|
---|
[2] | 188 |
|
---|
| 189 |
|
---|
| 190 | .. method:: read(num)
|
---|
| 191 |
|
---|
| 192 | Return a string containing up to *num* bytes starting from the current
|
---|
| 193 | file position; the file position is updated to point after the bytes that
|
---|
| 194 | were returned.
|
---|
| 195 |
|
---|
| 196 |
|
---|
| 197 | .. method:: read_byte()
|
---|
| 198 |
|
---|
| 199 | Returns a string of length 1 containing the character at the current file
|
---|
| 200 | position, and advances the file position by 1.
|
---|
| 201 |
|
---|
| 202 |
|
---|
| 203 | .. method:: readline()
|
---|
| 204 |
|
---|
| 205 | Returns a single line, starting at the current file position and up to the
|
---|
| 206 | next newline.
|
---|
| 207 |
|
---|
| 208 |
|
---|
| 209 | .. method:: resize(newsize)
|
---|
| 210 |
|
---|
| 211 | Resizes the map and the underlying file, if any. If the mmap was created
|
---|
| 212 | with :const:`ACCESS_READ` or :const:`ACCESS_COPY`, resizing the map will
|
---|
[391] | 213 | raise a :exc:`TypeError` exception.
|
---|
[2] | 214 |
|
---|
| 215 |
|
---|
| 216 | .. method:: rfind(string[, start[, end]])
|
---|
| 217 |
|
---|
| 218 | Returns the highest index in the object where the substring *string* is
|
---|
| 219 | found, such that *string* is contained in the range [*start*, *end*].
|
---|
| 220 | Optional arguments *start* and *end* are interpreted as in slice notation.
|
---|
| 221 | Returns ``-1`` on failure.
|
---|
| 222 |
|
---|
| 223 |
|
---|
| 224 | .. method:: seek(pos[, whence])
|
---|
| 225 |
|
---|
| 226 | Set the file's current position. *whence* argument is optional and
|
---|
| 227 | defaults to ``os.SEEK_SET`` or ``0`` (absolute file positioning); other
|
---|
| 228 | values are ``os.SEEK_CUR`` or ``1`` (seek relative to the current
|
---|
| 229 | position) and ``os.SEEK_END`` or ``2`` (seek relative to the file's end).
|
---|
| 230 |
|
---|
| 231 |
|
---|
| 232 | .. method:: size()
|
---|
| 233 |
|
---|
| 234 | Return the length of the file, which can be larger than the size of the
|
---|
| 235 | memory-mapped area.
|
---|
| 236 |
|
---|
| 237 |
|
---|
| 238 | .. method:: tell()
|
---|
| 239 |
|
---|
| 240 | Returns the current position of the file pointer.
|
---|
| 241 |
|
---|
| 242 |
|
---|
| 243 | .. method:: write(string)
|
---|
| 244 |
|
---|
| 245 | Write the bytes in *string* into memory at the current position of the
|
---|
| 246 | file pointer; the file position is updated to point after the bytes that
|
---|
| 247 | were written. If the mmap was created with :const:`ACCESS_READ`, then
|
---|
[391] | 248 | writing to it will raise a :exc:`TypeError` exception.
|
---|
[2] | 249 |
|
---|
| 250 |
|
---|
| 251 | .. method:: write_byte(byte)
|
---|
| 252 |
|
---|
| 253 | Write the single-character string *byte* into memory at the current
|
---|
| 254 | position of the file pointer; the file position is advanced by ``1``. If
|
---|
| 255 | the mmap was created with :const:`ACCESS_READ`, then writing to it will
|
---|
[391] | 256 | raise a :exc:`TypeError` exception.
|
---|