[2] | 1 | *******************************
|
---|
| 2 | HOWTO Use Python in the web
|
---|
| 3 | *******************************
|
---|
| 4 |
|
---|
| 5 | :Author: Marek Kubica
|
---|
| 6 |
|
---|
| 7 | .. topic:: Abstract
|
---|
| 8 |
|
---|
[391] | 9 | This document shows how Python fits into the web. It presents some ways
|
---|
| 10 | to integrate Python with a web server, and general practices useful for
|
---|
[2] | 11 | developing web sites.
|
---|
| 12 |
|
---|
| 13 |
|
---|
[391] | 14 | Programming for the Web has become a hot topic since the rise of "Web 2.0",
|
---|
[2] | 15 | which focuses on user-generated content on web sites. It has always been
|
---|
| 16 | possible to use Python for creating web sites, but it was a rather tedious task.
|
---|
[391] | 17 | Therefore, many frameworks and helper tools have been created to assist
|
---|
| 18 | developers in creating faster and more robust sites. This HOWTO describes
|
---|
| 19 | some of the methods used to combine Python with a web server to create
|
---|
| 20 | dynamic content. It is not meant as a complete introduction, as this topic is
|
---|
| 21 | far too broad to be covered in one single document. However, a short overview
|
---|
| 22 | of the most popular libraries is provided.
|
---|
[2] | 23 |
|
---|
| 24 | .. seealso::
|
---|
| 25 |
|
---|
[391] | 26 | While this HOWTO tries to give an overview of Python in the web, it cannot
|
---|
| 27 | always be as up to date as desired. Web development in Python is rapidly
|
---|
| 28 | moving forward, so the wiki page on `Web Programming
|
---|
| 29 | <http://wiki.python.org/moin/WebProgramming>`_ may be more in sync with
|
---|
[2] | 30 | recent development.
|
---|
| 31 |
|
---|
| 32 |
|
---|
[391] | 33 | The Low-Level View
|
---|
[2] | 34 | ==================
|
---|
| 35 |
|
---|
[391] | 36 | When a user enters a web site, their browser makes a connection to the site's
|
---|
| 37 | web server (this is called the *request*). The server looks up the file in the
|
---|
[2] | 38 | file system and sends it back to the user's browser, which displays it (this is
|
---|
[391] | 39 | the *response*). This is roughly how the underlying protocol, HTTP, works.
|
---|
[2] | 40 |
|
---|
[391] | 41 | Dynamic web sites are not based on files in the file system, but rather on
|
---|
| 42 | programs which are run by the web server when a request comes in, and which
|
---|
| 43 | *generate* the content that is returned to the user. They can do all sorts of
|
---|
| 44 | useful things, like display the postings of a bulletin board, show your email,
|
---|
| 45 | configure software, or just display the current time. These programs can be
|
---|
| 46 | written in any programming language the server supports. Since most servers
|
---|
| 47 | support Python, it is easy to use Python to create dynamic web sites.
|
---|
[2] | 48 |
|
---|
[391] | 49 | Most HTTP servers are written in C or C++, so they cannot execute Python code
|
---|
| 50 | directly -- a bridge is needed between the server and the program. These
|
---|
| 51 | bridges, or rather interfaces, define how programs interact with the server.
|
---|
| 52 | There have been numerous attempts to create the best possible interface, but
|
---|
| 53 | there are only a few worth mentioning.
|
---|
[2] | 54 |
|
---|
[391] | 55 | Not every web server supports every interface. Many web servers only support
|
---|
| 56 | old, now-obsolete interfaces; however, they can often be extended using
|
---|
| 57 | third-party modules to support newer ones.
|
---|
[2] | 58 |
|
---|
| 59 |
|
---|
| 60 | Common Gateway Interface
|
---|
| 61 | ------------------------
|
---|
| 62 |
|
---|
[391] | 63 | This interface, most commonly referred to as "CGI", is the oldest, and is
|
---|
| 64 | supported by nearly every web server out of the box. Programs using CGI to
|
---|
| 65 | communicate with their web server need to be started by the server for every
|
---|
| 66 | request. So, every request starts a new Python interpreter -- which takes some
|
---|
| 67 | time to start up -- thus making the whole interface only usable for low load
|
---|
| 68 | situations.
|
---|
[2] | 69 |
|
---|
[391] | 70 | The upside of CGI is that it is simple -- writing a Python program which uses
|
---|
| 71 | CGI is a matter of about three lines of code. This simplicity comes at a
|
---|
| 72 | price: it does very few things to help the developer.
|
---|
[2] | 73 |
|
---|
[391] | 74 | Writing CGI programs, while still possible, is no longer recommended. With
|
---|
| 75 | :ref:`WSGI <WSGI>`, a topic covered later in this document, it is possible to write
|
---|
| 76 | programs that emulate CGI, so they can be run as CGI if no better option is
|
---|
| 77 | available.
|
---|
[2] | 78 |
|
---|
| 79 | .. seealso::
|
---|
| 80 |
|
---|
| 81 | The Python standard library includes some modules that are helpful for
|
---|
| 82 | creating plain CGI programs:
|
---|
| 83 |
|
---|
| 84 | * :mod:`cgi` -- Handling of user input in CGI scripts
|
---|
[391] | 85 | * :mod:`cgitb` -- Displays nice tracebacks when errors happen in CGI
|
---|
[2] | 86 | applications, instead of presenting a "500 Internal Server Error" message
|
---|
| 87 |
|
---|
| 88 | The Python wiki features a page on `CGI scripts
|
---|
| 89 | <http://wiki.python.org/moin/CgiScripts>`_ with some additional information
|
---|
| 90 | about CGI in Python.
|
---|
| 91 |
|
---|
| 92 |
|
---|
| 93 | Simple script for testing CGI
|
---|
| 94 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
---|
| 95 |
|
---|
| 96 | To test whether your web server works with CGI, you can use this short and
|
---|
| 97 | simple CGI program::
|
---|
| 98 |
|
---|
| 99 | #!/usr/bin/env python
|
---|
| 100 | # -*- coding: UTF-8 -*-
|
---|
| 101 |
|
---|
| 102 | # enable debugging
|
---|
| 103 | import cgitb
|
---|
| 104 | cgitb.enable()
|
---|
| 105 |
|
---|
| 106 | print "Content-Type: text/plain;charset=utf-8"
|
---|
| 107 | print
|
---|
| 108 |
|
---|
| 109 | print "Hello World!"
|
---|
| 110 |
|
---|
[391] | 111 | Depending on your web server configuration, you may need to save this code with
|
---|
| 112 | a ``.py`` or ``.cgi`` extension. Additionally, this file may also need to be
|
---|
| 113 | in a ``cgi-bin`` folder, for security reasons.
|
---|
[2] | 114 |
|
---|
| 115 | You might wonder what the ``cgitb`` line is about. This line makes it possible
|
---|
| 116 | to display a nice traceback instead of just crashing and displaying an "Internal
|
---|
| 117 | Server Error" in the user's browser. This is useful for debugging, but it might
|
---|
[391] | 118 | risk exposing some confidential data to the user. You should not use ``cgitb``
|
---|
| 119 | in production code for this reason. You should *always* catch exceptions, and
|
---|
[2] | 120 | display proper error pages -- end-users don't like to see nondescript "Internal
|
---|
| 121 | Server Errors" in their browsers.
|
---|
| 122 |
|
---|
| 123 |
|
---|
| 124 | Setting up CGI on your own server
|
---|
| 125 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
---|
| 126 |
|
---|
| 127 | If you don't have your own web server, this does not apply to you. You can
|
---|
[391] | 128 | check whether it works as-is, and if not you will need to talk to the
|
---|
| 129 | administrator of your web server. If it is a big host, you can try filing a
|
---|
| 130 | ticket asking for Python support.
|
---|
[2] | 131 |
|
---|
[391] | 132 | If you are your own administrator or want to set up CGI for testing purposes on
|
---|
| 133 | your own computers, you have to configure it by yourself. There is no single
|
---|
| 134 | way to configure CGI, as there are many web servers with different
|
---|
| 135 | configuration options. Currently the most widely used free web server is
|
---|
| 136 | `Apache HTTPd <http://httpd.apache.org/>`_, or Apache for short. Apache can be
|
---|
| 137 | easily installed on nearly every system using the system's package management
|
---|
| 138 | tool. `lighttpd <http://www.lighttpd.net>`_ is another alternative and is
|
---|
| 139 | said to have better performance. On many systems this server can also be
|
---|
| 140 | installed using the package management tool, so manually compiling the web
|
---|
| 141 | server may not be needed.
|
---|
[2] | 142 |
|
---|
[391] | 143 | * On Apache you can take a look at the `Dynamic Content with CGI
|
---|
[2] | 144 | <http://httpd.apache.org/docs/2.2/howto/cgi.html>`_ tutorial, where everything
|
---|
| 145 | is described. Most of the time it is enough just to set ``+ExecCGI``. The
|
---|
| 146 | tutorial also describes the most common gotchas that might arise.
|
---|
[391] | 147 |
|
---|
[2] | 148 | * On lighttpd you need to use the `CGI module
|
---|
[391] | 149 | <http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModCGI>`_\ , which can be configured
|
---|
[2] | 150 | in a straightforward way. It boils down to setting ``cgi.assign`` properly.
|
---|
| 151 |
|
---|
| 152 |
|
---|
| 153 | Common problems with CGI scripts
|
---|
| 154 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
---|
| 155 |
|
---|
[391] | 156 | Using CGI sometimes leads to small annoyances while trying to get these
|
---|
| 157 | scripts to run. Sometimes a seemingly correct script does not work as
|
---|
| 158 | expected, the cause being some small hidden problem that's difficult to spot.
|
---|
[2] | 159 |
|
---|
[391] | 160 | Some of these potential problems are:
|
---|
[2] | 161 |
|
---|
[391] | 162 | * The Python script is not marked as executable. When CGI scripts are not
|
---|
| 163 | executable most web servers will let the user download it, instead of
|
---|
[2] | 164 | running it and sending the output to the user. For CGI scripts to run
|
---|
[391] | 165 | properly on Unix-like operating systems, the ``+x`` bit needs to be set.
|
---|
| 166 | Using ``chmod a+x your_script.py`` may solve this problem.
|
---|
| 167 |
|
---|
| 168 | * On a Unix-like system, The line endings in the program file must be Unix
|
---|
| 169 | style line endings. This is important because the web server checks the
|
---|
| 170 | first line of the script (called shebang) and tries to run the program
|
---|
| 171 | specified there. It gets easily confused by Windows line endings (Carriage
|
---|
| 172 | Return & Line Feed, also called CRLF), so you have to convert the file to
|
---|
| 173 | Unix line endings (only Line Feed, LF). This can be done automatically by
|
---|
| 174 | uploading the file via FTP in text mode instead of binary mode, but the
|
---|
| 175 | preferred way is just telling your editor to save the files with Unix line
|
---|
| 176 | endings. Most editors support this.
|
---|
| 177 |
|
---|
| 178 | * Your web server must be able to read the file, and you need to make sure the
|
---|
| 179 | permissions are correct. On unix-like systems, the server often runs as user
|
---|
| 180 | and group ``www-data``, so it might be worth a try to change the file
|
---|
| 181 | ownership, or making the file world readable by using ``chmod a+r
|
---|
| 182 | your_script.py``.
|
---|
| 183 |
|
---|
| 184 | * The web server must know that the file you're trying to access is a CGI script.
|
---|
| 185 | Check the configuration of your web server, as it may be configured
|
---|
| 186 | to expect a specific file extension for CGI scripts.
|
---|
| 187 |
|
---|
| 188 | * On Unix-like systems, the path to the interpreter in the shebang
|
---|
| 189 | (``#!/usr/bin/env python``) must be correct. This line calls
|
---|
| 190 | ``/usr/bin/env`` to find Python, but it will fail if there is no
|
---|
| 191 | ``/usr/bin/env``, or if Python is not in the web server's path. If you know
|
---|
| 192 | where your Python is installed, you can also use that full path. The
|
---|
| 193 | commands ``whereis python`` and ``type -p python`` could help you find
|
---|
| 194 | where it is installed. Once you know the path, you can change the shebang
|
---|
| 195 | accordingly: ``#!/usr/bin/python``.
|
---|
| 196 |
|
---|
[2] | 197 | * The file must not contain a BOM (Byte Order Mark). The BOM is meant for
|
---|
[391] | 198 | determining the byte order of UTF-16 and UTF-32 encodings, but some editors
|
---|
| 199 | write this also into UTF-8 files. The BOM interferes with the shebang line,
|
---|
| 200 | so be sure to tell your editor not to write the BOM.
|
---|
[2] | 201 |
|
---|
[391] | 202 | * If the web server is using :ref:`mod-python`, ``mod_python`` may be having
|
---|
| 203 | problems. ``mod_python`` is able to handle CGI scripts by itself, but it can
|
---|
| 204 | also be a source of issues.
|
---|
[2] | 205 |
|
---|
[391] | 206 |
|
---|
[2] | 207 | .. _mod-python:
|
---|
| 208 |
|
---|
| 209 | mod_python
|
---|
| 210 | ----------
|
---|
| 211 |
|
---|
| 212 | People coming from PHP often find it hard to grasp how to use Python in the web.
|
---|
[391] | 213 | Their first thought is mostly `mod_python <http://www.modpython.org/>`_\ ,
|
---|
| 214 | because they think that this is the equivalent to ``mod_php``. Actually, there
|
---|
| 215 | are many differences. What ``mod_python`` does is embed the interpreter into
|
---|
| 216 | the Apache process, thus speeding up requests by not having to start a Python
|
---|
| 217 | interpreter for each request. On the other hand, it is not "Python intermixed
|
---|
| 218 | with HTML" in the way that PHP is often intermixed with HTML. The Python
|
---|
| 219 | equivalent of that is a template engine. ``mod_python`` itself is much more
|
---|
| 220 | powerful and provides more access to Apache internals. It can emulate CGI,
|
---|
| 221 | work in a "Python Server Pages" mode (similar to JSP) which is "HTML
|
---|
| 222 | intermingled with Python", and it has a "Publisher" which designates one file
|
---|
| 223 | to accept all requests and decide what to do with them.
|
---|
[2] | 224 |
|
---|
[391] | 225 | ``mod_python`` does have some problems. Unlike the PHP interpreter, the Python
|
---|
| 226 | interpreter uses caching when executing files, so changes to a file will
|
---|
| 227 | require the web server to be restarted. Another problem is the basic concept
|
---|
| 228 | -- Apache starts child processes to handle the requests, and unfortunately
|
---|
| 229 | every child process needs to load the whole Python interpreter even if it does
|
---|
| 230 | not use it. This makes the whole web server slower. Another problem is that,
|
---|
| 231 | because ``mod_python`` is linked against a specific version of ``libpython``,
|
---|
| 232 | it is not possible to switch from an older version to a newer (e.g. 2.4 to 2.5)
|
---|
| 233 | without recompiling ``mod_python``. ``mod_python`` is also bound to the Apache
|
---|
| 234 | web server, so programs written for ``mod_python`` cannot easily run on other
|
---|
| 235 | web servers.
|
---|
[2] | 236 |
|
---|
[391] | 237 | These are the reasons why ``mod_python`` should be avoided when writing new
|
---|
| 238 | programs. In some circumstances it still might be a good idea to use
|
---|
| 239 | ``mod_python`` for deployment, but WSGI makes it possible to run WSGI programs
|
---|
| 240 | under ``mod_python`` as well.
|
---|
[2] | 241 |
|
---|
| 242 |
|
---|
| 243 | FastCGI and SCGI
|
---|
| 244 | ----------------
|
---|
| 245 |
|
---|
| 246 | FastCGI and SCGI try to solve the performance problem of CGI in another way.
|
---|
| 247 | Instead of embedding the interpreter into the web server, they create
|
---|
[391] | 248 | long-running background processes. There is still a module in the web server
|
---|
| 249 | which makes it possible for the web server to "speak" with the background
|
---|
| 250 | process. As the background process is independent of the server, it can be
|
---|
| 251 | written in any language, including Python. The language just needs to have a
|
---|
| 252 | library which handles the communication with the webserver.
|
---|
[2] | 253 |
|
---|
| 254 | The difference between FastCGI and SCGI is very small, as SCGI is essentially
|
---|
[391] | 255 | just a "simpler FastCGI". As the web server support for SCGI is limited,
|
---|
[2] | 256 | most people use FastCGI instead, which works the same way. Almost everything
|
---|
[391] | 257 | that applies to SCGI also applies to FastCGI as well, so we'll only cover
|
---|
[2] | 258 | the latter.
|
---|
| 259 |
|
---|
[391] | 260 | These days, FastCGI is never used directly. Just like ``mod_python``, it is only
|
---|
[2] | 261 | used for the deployment of WSGI applications.
|
---|
| 262 |
|
---|
| 263 | .. seealso::
|
---|
| 264 |
|
---|
| 265 | * `FastCGI, SCGI, and Apache: Background and Future
|
---|
| 266 | <http://www.vmunix.com/mark/blog/archives/2006/01/02/fastcgi-scgi-and-apache-background-and-future/>`_
|
---|
[391] | 267 | is a discussion on why the concept of FastCGI and SCGI is better than that
|
---|
[2] | 268 | of mod_python.
|
---|
| 269 |
|
---|
| 270 |
|
---|
| 271 | Setting up FastCGI
|
---|
| 272 | ^^^^^^^^^^^^^^^^^^
|
---|
| 273 |
|
---|
[391] | 274 | Each web server requires a specific module.
|
---|
[2] | 275 |
|
---|
[391] | 276 | * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid
|
---|
| 277 | <http://httpd.apache.org/mod_fcgid/>`_. ``mod_fastcgi`` is the original one, but it
|
---|
| 278 | has some licensing issues, which is why it is sometimes considered non-free.
|
---|
| 279 | ``mod_fcgid`` is a smaller, compatible alternative. One of these modules needs
|
---|
[2] | 280 | to be loaded by Apache.
|
---|
[391] | 281 |
|
---|
[2] | 282 | * lighttpd ships its own `FastCGI module
|
---|
[391] | 283 | <http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModFastCGI>`_ as well as an
|
---|
| 284 | `SCGI module <http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModSCGI>`_.
|
---|
[2] | 285 |
|
---|
[391] | 286 | * `nginx <http://nginx.org/>`_ also supports `FastCGI
|
---|
| 287 | <http://wiki.nginx.org/NginxSimplePythonFCGI>`_.
|
---|
| 288 |
|
---|
[2] | 289 | Once you have installed and configured the module, you can test it with the
|
---|
| 290 | following WSGI-application::
|
---|
| 291 |
|
---|
| 292 | #!/usr/bin/env python
|
---|
| 293 | # -*- coding: UTF-8 -*-
|
---|
| 294 |
|
---|
| 295 | from cgi import escape
|
---|
| 296 | import sys, os
|
---|
| 297 | from flup.server.fcgi import WSGIServer
|
---|
| 298 |
|
---|
| 299 | def app(environ, start_response):
|
---|
| 300 | start_response('200 OK', [('Content-Type', 'text/html')])
|
---|
| 301 |
|
---|
| 302 | yield '<h1>FastCGI Environment</h1>'
|
---|
| 303 | yield '<table>'
|
---|
| 304 | for k, v in sorted(environ.items()):
|
---|
| 305 | yield '<tr><th>%s</th><td>%s</td></tr>' % (escape(k), escape(v))
|
---|
| 306 | yield '</table>'
|
---|
| 307 |
|
---|
| 308 | WSGIServer(app).run()
|
---|
| 309 |
|
---|
| 310 | This is a simple WSGI application, but you need to install `flup
|
---|
| 311 | <http://pypi.python.org/pypi/flup/1.0>`_ first, as flup handles the low level
|
---|
| 312 | FastCGI access.
|
---|
| 313 |
|
---|
| 314 | .. seealso::
|
---|
| 315 |
|
---|
| 316 | There is some documentation on `setting up Django with FastCGI
|
---|
[391] | 317 | <http://docs.djangoproject.com/en/dev/howto/deployment/fastcgi/>`_, most of
|
---|
| 318 | which can be reused for other WSGI-compliant frameworks and libraries.
|
---|
| 319 | Only the ``manage.py`` part has to be changed, the example used here can be
|
---|
| 320 | used instead. Django does more or less the exact same thing.
|
---|
[2] | 321 |
|
---|
| 322 |
|
---|
| 323 | mod_wsgi
|
---|
| 324 | --------
|
---|
| 325 |
|
---|
[391] | 326 | `mod_wsgi <http://code.google.com/p/modwsgi/>`_ is an attempt to get rid of the
|
---|
| 327 | low level gateways. Given that FastCGI, SCGI, and mod_python are mostly used to
|
---|
| 328 | deploy WSGI applications, mod_wsgi was started to directly embed WSGI applications
|
---|
| 329 | into the Apache web server. mod_wsgi is specifically designed to host WSGI
|
---|
| 330 | applications. It makes the deployment of WSGI applications much easier than
|
---|
| 331 | deployment using other low level methods, which need glue code. The downside
|
---|
| 332 | is that mod_wsgi is limited to the Apache web server; other servers would need
|
---|
[2] | 333 | their own implementations of mod_wsgi.
|
---|
| 334 |
|
---|
[391] | 335 | mod_wsgi supports two modes: embedded mode, in which it integrates with the
|
---|
| 336 | Apache process, and daemon mode, which is more FastCGI-like. Unlike FastCGI,
|
---|
| 337 | mod_wsgi handles the worker-processes by itself, which makes administration
|
---|
[2] | 338 | easier.
|
---|
| 339 |
|
---|
| 340 |
|
---|
| 341 | .. _WSGI:
|
---|
| 342 |
|
---|
| 343 | Step back: WSGI
|
---|
| 344 | ===============
|
---|
| 345 |
|
---|
[391] | 346 | WSGI has already been mentioned several times, so it has to be something
|
---|
| 347 | important. In fact it really is, and now it is time to explain it.
|
---|
[2] | 348 |
|
---|
[391] | 349 | The *Web Server Gateway Interface*, or WSGI for short, is defined in
|
---|
| 350 | :pep:`333` and is currently the best way to do Python web programming. While
|
---|
| 351 | it is great for programmers writing frameworks, a normal web developer does not
|
---|
| 352 | need to get in direct contact with it. When choosing a framework for web
|
---|
| 353 | development it is a good idea to choose one which supports WSGI.
|
---|
[2] | 354 |
|
---|
[391] | 355 | The big benefit of WSGI is the unification of the application programming
|
---|
| 356 | interface. When your program is compatible with WSGI -- which at the outer
|
---|
| 357 | level means that the framework you are using has support for WSGI -- your
|
---|
| 358 | program can be deployed via any web server interface for which there are WSGI
|
---|
| 359 | wrappers. You do not need to care about whether the application user uses
|
---|
| 360 | mod_python or FastCGI or mod_wsgi -- with WSGI your application will work on
|
---|
| 361 | any gateway interface. The Python standard library contains its own WSGI
|
---|
| 362 | server, :mod:`wsgiref`, which is a small web server that can be used for
|
---|
| 363 | testing.
|
---|
[2] | 364 |
|
---|
[391] | 365 | A really great WSGI feature is middleware. Middleware is a layer around your
|
---|
| 366 | program which can add various functionality to it. There is quite a bit of
|
---|
| 367 | `middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already
|
---|
| 368 | available. For example, instead of writing your own session management (HTTP
|
---|
| 369 | is a stateless protocol, so to associate multiple HTTP requests with a single
|
---|
| 370 | user your application must create and manage such state via a session), you can
|
---|
| 371 | just download middleware which does that, plug it in, and get on with coding
|
---|
| 372 | the unique parts of your application. The same thing with compression -- there
|
---|
| 373 | is existing middleware which handles compressing your HTML using gzip to save
|
---|
| 374 | on your server's bandwidth. Authentication is another a problem easily solved
|
---|
| 375 | using existing middleware.
|
---|
[2] | 376 |
|
---|
[391] | 377 | Although WSGI may seem complex, the initial phase of learning can be very
|
---|
| 378 | rewarding because WSGI and the associated middleware already have solutions to
|
---|
| 379 | many problems that might arise while developing web sites.
|
---|
[2] | 380 |
|
---|
| 381 |
|
---|
| 382 | WSGI Servers
|
---|
| 383 | ------------
|
---|
| 384 |
|
---|
| 385 | The code that is used to connect to various low level gateways like CGI or
|
---|
[391] | 386 | mod_python is called a *WSGI server*. One of these servers is ``flup``, which
|
---|
| 387 | supports FastCGI and SCGI, as well as `AJP
|
---|
[2] | 388 | <http://en.wikipedia.org/wiki/Apache_JServ_Protocol>`_. Some of these servers
|
---|
[391] | 389 | are written in Python, as ``flup`` is, but there also exist others which are
|
---|
[2] | 390 | written in C and can be used as drop-in replacements.
|
---|
| 391 |
|
---|
[391] | 392 | There are many servers already available, so a Python web application
|
---|
| 393 | can be deployed nearly anywhere. This is one big advantage that Python has
|
---|
| 394 | compared with other web technologies.
|
---|
[2] | 395 |
|
---|
| 396 | .. seealso::
|
---|
| 397 |
|
---|
[391] | 398 | A good overview of WSGI-related code can be found in the `WSGI homepage
|
---|
| 399 | <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers
|
---|
| 400 | <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application
|
---|
[2] | 401 | supporting WSGI.
|
---|
| 402 |
|
---|
| 403 | You might be interested in some WSGI-supporting modules already contained in
|
---|
| 404 | the standard library, namely:
|
---|
| 405 |
|
---|
| 406 | * :mod:`wsgiref` -- some tiny utilities and servers for WSGI
|
---|
| 407 |
|
---|
| 408 |
|
---|
| 409 | Case study: MoinMoin
|
---|
| 410 | --------------------
|
---|
| 411 |
|
---|
[391] | 412 | What does WSGI give the web application developer? Let's take a look at
|
---|
| 413 | an application that's been around for a while, which was written in
|
---|
| 414 | Python without using WSGI.
|
---|
[2] | 415 |
|
---|
[391] | 416 | One of the most widely used wiki software packages is `MoinMoin
|
---|
| 417 | <http://moinmo.in/>`_. It was created in 2000, so it predates WSGI by about
|
---|
| 418 | three years. Older versions needed separate code to run on CGI, mod_python,
|
---|
| 419 | FastCGI and standalone.
|
---|
[2] | 420 |
|
---|
[391] | 421 | It now includes support for WSGI. Using WSGI, it is possible to deploy
|
---|
| 422 | MoinMoin on any WSGI compliant server, with no additional glue code.
|
---|
| 423 | Unlike the pre-WSGI versions, this could include WSGI servers that the
|
---|
| 424 | authors of MoinMoin know nothing about.
|
---|
[2] | 425 |
|
---|
[391] | 426 |
|
---|
| 427 | Model-View-Controller
|
---|
[2] | 428 | =====================
|
---|
| 429 |
|
---|
[391] | 430 | The term *MVC* is often encountered in statements such as "framework *foo*
|
---|
| 431 | supports MVC". MVC is more about the overall organization of code, rather than
|
---|
| 432 | any particular API. Many web frameworks use this model to help the developer
|
---|
| 433 | bring structure to their program. Bigger web applications can have lots of
|
---|
| 434 | code, so it is a good idea to have an effective structure right from the beginning.
|
---|
| 435 | That way, even users of other frameworks (or even other languages, since MVC is
|
---|
| 436 | not Python-specific) can easily understand the code, given that they are
|
---|
| 437 | already familiar with the MVC structure.
|
---|
[2] | 438 |
|
---|
| 439 | MVC stands for three components:
|
---|
| 440 |
|
---|
[391] | 441 | * The *model*. This is the data that will be displayed and modified. In
|
---|
| 442 | Python frameworks, this component is often represented by the classes used by
|
---|
| 443 | an object-relational mapper.
|
---|
| 444 |
|
---|
[2] | 445 | * The *view*. This component's job is to display the data of the model to the
|
---|
[391] | 446 | user. Typically this component is implemented via templates.
|
---|
| 447 |
|
---|
[2] | 448 | * The *controller*. This is the layer between the user and the model. The
|
---|
[391] | 449 | controller reacts to user actions (like opening some specific URL), tells
|
---|
| 450 | the model to modify the data if necessary, and tells the view code what to
|
---|
| 451 | display,
|
---|
[2] | 452 |
|
---|
| 453 | While one might think that MVC is a complex design pattern, in fact it is not.
|
---|
| 454 | It is used in Python because it has turned out to be useful for creating clean,
|
---|
| 455 | maintainable web sites.
|
---|
| 456 |
|
---|
| 457 | .. note::
|
---|
| 458 |
|
---|
| 459 | While not all Python frameworks explicitly support MVC, it is often trivial
|
---|
| 460 | to create a web site which uses the MVC pattern by separating the data logic
|
---|
| 461 | (the model) from the user interaction logic (the controller) and the
|
---|
| 462 | templates (the view). That's why it is important not to write unnecessary
|
---|
[391] | 463 | Python code in the templates -- it works against the MVC model and creates
|
---|
| 464 | chaos in the code base, making it harder to understand and modify.
|
---|
[2] | 465 |
|
---|
| 466 | .. seealso::
|
---|
| 467 |
|
---|
[391] | 468 | The English Wikipedia has an article about the `Model-View-Controller pattern
|
---|
| 469 | <http://en.wikipedia.org/wiki/Model-view-controller>`_. It includes a long
|
---|
| 470 | list of web frameworks for various programming languages.
|
---|
[2] | 471 |
|
---|
| 472 |
|
---|
[391] | 473 | Ingredients for Websites
|
---|
| 474 | ========================
|
---|
[2] | 475 |
|
---|
[391] | 476 | Websites are complex constructs, so tools have been created to help web
|
---|
| 477 | developers make their code easier to write and more maintainable. Tools like
|
---|
| 478 | these exist for all web frameworks in all languages. Developers are not forced
|
---|
| 479 | to use these tools, and often there is no "best" tool. It is worth learning
|
---|
| 480 | about the available tools because they can greatly simplify the process of
|
---|
| 481 | developing a web site.
|
---|
[2] | 482 |
|
---|
| 483 |
|
---|
| 484 | .. seealso::
|
---|
| 485 |
|
---|
[391] | 486 | There are far more components than can be presented here. The Python wiki
|
---|
| 487 | has a page about these components, called
|
---|
[2] | 488 | `Web Components <http://wiki.python.org/moin/WebComponents>`_.
|
---|
| 489 |
|
---|
| 490 |
|
---|
| 491 | Templates
|
---|
| 492 | ---------
|
---|
| 493 |
|
---|
[391] | 494 | Mixing of HTML and Python code is made possible by a few libraries. While
|
---|
[2] | 495 | convenient at first, it leads to horribly unmaintainable code. That's why
|
---|
| 496 | templates exist. Templates are, in the simplest case, just HTML files with
|
---|
[391] | 497 | placeholders. The HTML is sent to the user's browser after filling in the
|
---|
[2] | 498 | placeholders.
|
---|
| 499 |
|
---|
[391] | 500 | Python already includes two ways to build simple templates::
|
---|
[2] | 501 |
|
---|
[391] | 502 | >>> template = "<html><body><h1>Hello %s!</h1></body></html>"
|
---|
| 503 | >>> print template % "Reader"
|
---|
| 504 | <html><body><h1>Hello Reader!</h1></body></html>
|
---|
[2] | 505 |
|
---|
[391] | 506 | >>> from string import Template
|
---|
| 507 | >>> template = Template("<html><body><h1>Hello ${name}</h1></body></html>")
|
---|
| 508 | >>> print template.substitute(dict(name='Dinsdale'))
|
---|
| 509 | <html><body><h1>Hello Dinsdale!</h1></body></html>
|
---|
[2] | 510 |
|
---|
[391] | 511 | To generate complex HTML based on non-trivial model data, conditional
|
---|
| 512 | and looping constructs like Python's *for* and *if* are generally needed.
|
---|
| 513 | *Template engines* support templates of this complexity.
|
---|
[2] | 514 |
|
---|
[391] | 515 | There are a lot of template engines available for Python which can be used with
|
---|
| 516 | or without a `framework`_. Some of these define a plain-text programming
|
---|
| 517 | language which is easy to learn, partly because it is limited in scope.
|
---|
| 518 | Others use XML, and the template output is guaranteed to be always be valid
|
---|
| 519 | XML. There are many other variations.
|
---|
[2] | 520 |
|
---|
[391] | 521 | Some `frameworks`_ ship their own template engine or recommend one in
|
---|
| 522 | particular. In the absence of a reason to use a different template engine,
|
---|
| 523 | using the one provided by or recommended by the framework is a good idea.
|
---|
[2] | 524 |
|
---|
[391] | 525 | Popular template engines include:
|
---|
[2] | 526 |
|
---|
[391] | 527 | * `Mako <http://www.makotemplates.org/>`_
|
---|
| 528 | * `Genshi <http://genshi.edgewall.org/>`_
|
---|
| 529 | * `Jinja <http://jinja.pocoo.org/2/>`_
|
---|
[2] | 530 |
|
---|
| 531 | .. seealso::
|
---|
| 532 |
|
---|
[391] | 533 | There are many template engines competing for attention, because it is
|
---|
| 534 | pretty easy to create them in Python. The page `Templating
|
---|
[2] | 535 | <http://wiki.python.org/moin/Templating>`_ in the wiki lists a big,
|
---|
[391] | 536 | ever-growing number of these. The three listed above are considered "second
|
---|
| 537 | generation" template engines and are a good place to start.
|
---|
[2] | 538 |
|
---|
| 539 |
|
---|
| 540 | Data persistence
|
---|
| 541 | ----------------
|
---|
| 542 |
|
---|
[391] | 543 | *Data persistence*, while sounding very complicated, is just about storing data.
|
---|
| 544 | This data might be the text of blog entries, the postings on a bulletin board or
|
---|
| 545 | the text of a wiki page. There are, of course, a number of different ways to store
|
---|
| 546 | information on a web server.
|
---|
[2] | 547 |
|
---|
[391] | 548 | Often, relational database engines like `MySQL <http://www.mysql.com/>`_ or
|
---|
| 549 | `PostgreSQL <http://www.postgresql.org/>`_ are used because of their good
|
---|
| 550 | performance when handling very large databases consisting of millions of
|
---|
| 551 | entries. There is also a small database engine called `SQLite
|
---|
| 552 | <http://www.sqlite.org/>`_, which is bundled with Python in the :mod:`sqlite3`
|
---|
| 553 | module, and which uses only one file. It has no other dependencies. For
|
---|
| 554 | smaller sites SQLite is just enough.
|
---|
[2] | 555 |
|
---|
[391] | 556 | Relational databases are *queried* using a language called `SQL
|
---|
| 557 | <http://en.wikipedia.org/wiki/SQL>`_. Python programmers in general do not
|
---|
| 558 | like SQL too much, as they prefer to work with objects. It is possible to save
|
---|
| 559 | Python objects into a database using a technology called `ORM
|
---|
| 560 | <http://en.wikipedia.org/wiki/Object-relational_mapping>`_ (Object Relational
|
---|
| 561 | Mapping). ORM translates all object-oriented access into SQL code under the
|
---|
| 562 | hood, so the developer does not need to think about it. Most `frameworks`_ use
|
---|
| 563 | ORMs, and it works quite well.
|
---|
[2] | 564 |
|
---|
[391] | 565 | A second possibility is storing data in normal, plain text files (some
|
---|
| 566 | times called "flat files"). This is very easy for simple sites,
|
---|
| 567 | but can be difficult to get right if the web site is performing many
|
---|
| 568 | updates to the stored data.
|
---|
[2] | 569 |
|
---|
[391] | 570 | A third possibility are object oriented databases (also called "object
|
---|
| 571 | databases"). These databases store the object data in a form that closely
|
---|
| 572 | parallels the way the objects are structured in memory during program
|
---|
| 573 | execution. (By contrast, ORMs store the object data as rows of data in tables
|
---|
| 574 | and relations between those rows.) Storing the objects directly has the
|
---|
| 575 | advantage that nearly all objects can be saved in a straightforward way, unlike
|
---|
| 576 | in relational databases where some objects are very hard to represent.
|
---|
[2] | 577 |
|
---|
[391] | 578 | `Frameworks`_ often give hints on which data storage method to choose. It is
|
---|
| 579 | usually a good idea to stick to the data store recommended by the framework
|
---|
| 580 | unless the application has special requirements better satisfied by an
|
---|
| 581 | alternate storage mechanism.
|
---|
| 582 |
|
---|
[2] | 583 | .. seealso::
|
---|
| 584 |
|
---|
| 585 | * `Persistence Tools <http://wiki.python.org/moin/PersistenceTools>`_ lists
|
---|
[391] | 586 | possibilities on how to save data in the file system. Some of these
|
---|
| 587 | modules are part of the standard library
|
---|
| 588 |
|
---|
[2] | 589 | * `Database Programming <http://wiki.python.org/moin/DatabaseProgramming>`_
|
---|
[391] | 590 | helps with choosing a method for saving data
|
---|
| 591 |
|
---|
| 592 | * `SQLAlchemy <http://www.sqlalchemy.org/>`_, the most powerful OR-Mapper
|
---|
| 593 | for Python, and `Elixir <http://elixir.ematia.de/>`_, which makes
|
---|
| 594 | SQLAlchemy easier to use
|
---|
| 595 |
|
---|
[2] | 596 | * `SQLObject <http://www.sqlobject.org/>`_, another popular OR-Mapper
|
---|
[391] | 597 |
|
---|
[2] | 598 | * `ZODB <https://launchpad.net/zodb>`_ and `Durus
|
---|
| 599 | <http://www.mems-exchange.org/software/durus/>`_, two object oriented
|
---|
| 600 | databases
|
---|
| 601 |
|
---|
| 602 |
|
---|
| 603 | .. _framework:
|
---|
| 604 |
|
---|
| 605 | Frameworks
|
---|
| 606 | ==========
|
---|
| 607 |
|
---|
[391] | 608 | The process of creating code to run web sites involves writing code to provide
|
---|
| 609 | various services. The code to provide a particular service often works the
|
---|
| 610 | same way regardless of the complexity or purpose of the web site in question.
|
---|
| 611 | Abstracting these common solutions into reusable code produces what are called
|
---|
| 612 | "frameworks" for web development. Perhaps the most well-known framework for
|
---|
| 613 | web development is Ruby on Rails, but Python has its own frameworks. Some of
|
---|
| 614 | these were partly inspired by Rails, or borrowed ideas from Rails, but many
|
---|
| 615 | existed a long time before Rails.
|
---|
[2] | 616 |
|
---|
[391] | 617 | Originally Python web frameworks tended to incorporate all of the services
|
---|
| 618 | needed to develop web sites as a giant, integrated set of tools. No two web
|
---|
| 619 | frameworks were interoperable: a program developed for one could not be
|
---|
| 620 | deployed on a different one without considerable re-engineering work. This led
|
---|
| 621 | to the development of "minimalist" web frameworks that provided just the tools
|
---|
| 622 | to communicate between the Python code and the http protocol, with all other
|
---|
| 623 | services to be added on top via separate components. Some ad hoc standards
|
---|
| 624 | were developed that allowed for limited interoperability between frameworks,
|
---|
| 625 | such as a standard that allowed different template engines to be used
|
---|
| 626 | interchangeably.
|
---|
[2] | 627 |
|
---|
[391] | 628 | Since the advent of WSGI, the Python web framework world has been evolving
|
---|
| 629 | toward interoperability based on the WSGI standard. Now many web frameworks,
|
---|
| 630 | whether "full stack" (providing all the tools one needs to deploy the most
|
---|
| 631 | complex web sites) or minimalist, or anything in between, are built from
|
---|
| 632 | collections of reusable components that can be used with more than one
|
---|
| 633 | framework.
|
---|
[2] | 634 |
|
---|
[391] | 635 | The majority of users will probably want to select a "full stack" framework
|
---|
| 636 | that has an active community. These frameworks tend to be well documented,
|
---|
| 637 | and provide the easiest path to producing a fully functional web site in
|
---|
| 638 | minimal time.
|
---|
[2] | 639 |
|
---|
| 640 |
|
---|
| 641 | Some notable frameworks
|
---|
| 642 | -----------------------
|
---|
| 643 |
|
---|
[391] | 644 | There are an incredible number of frameworks, so they cannot all be covered
|
---|
| 645 | here. Instead we will briefly touch on some of the most popular.
|
---|
[2] | 646 |
|
---|
| 647 |
|
---|
| 648 | Django
|
---|
| 649 | ^^^^^^
|
---|
| 650 |
|
---|
| 651 | `Django <http://www.djangoproject.com/>`_ is a framework consisting of several
|
---|
| 652 | tightly coupled elements which were written from scratch and work together very
|
---|
[391] | 653 | well. It includes an ORM which is quite powerful while being simple to use,
|
---|
| 654 | and has a great online administration interface which makes it possible to edit
|
---|
| 655 | the data in the database with a browser. The template engine is text-based and
|
---|
| 656 | is designed to be usable for page designers who cannot write Python. It
|
---|
| 657 | supports template inheritance and filters (which work like Unix pipes). Django
|
---|
| 658 | has many handy features bundled, such as creation of RSS feeds or generic views,
|
---|
| 659 | which make it possible to create web sites almost without writing any Python code.
|
---|
[2] | 660 |
|
---|
[391] | 661 | It has a big, international community, the members of which have created many
|
---|
| 662 | web sites. There are also a lot of add-on projects which extend Django's normal
|
---|
[2] | 663 | functionality. This is partly due to Django's well written `online
|
---|
| 664 | documentation <http://docs.djangoproject.com/>`_ and the `Django book
|
---|
| 665 | <http://www.djangobook.com/>`_.
|
---|
| 666 |
|
---|
| 667 |
|
---|
| 668 | .. note::
|
---|
| 669 |
|
---|
[391] | 670 | Although Django is an MVC-style framework, it names the elements
|
---|
[2] | 671 | differently, which is described in the `Django FAQ
|
---|
[391] | 672 | <http://docs.djangoproject.com/en/dev/faq/general/#django-appears-to-be-a-mvc-framework-but-you-call-the-controller-the-view-and-the-view-the-template-how-come-you-don-t-use-the-standard-names>`_.
|
---|
[2] | 673 |
|
---|
| 674 |
|
---|
| 675 | TurboGears
|
---|
| 676 | ^^^^^^^^^^
|
---|
| 677 |
|
---|
[391] | 678 | Another popular web framework for Python is `TurboGears
|
---|
| 679 | <http://www.turbogears.org/>`_. TurboGears takes the approach of using already
|
---|
| 680 | existing components and combining them with glue code to create a seamless
|
---|
| 681 | experience. TurboGears gives the user flexibility in choosing components. For
|
---|
| 682 | example the ORM and template engine can be changed to use packages different
|
---|
| 683 | from those used by default.
|
---|
[2] | 684 |
|
---|
| 685 | The documentation can be found in the `TurboGears wiki
|
---|
| 686 | <http://docs.turbogears.org/>`_, where links to screencasts can be found.
|
---|
| 687 | TurboGears has also an active user community which can respond to most related
|
---|
| 688 | questions. There is also a `TurboGears book <http://turbogearsbook.com/>`_
|
---|
| 689 | published, which is a good starting point.
|
---|
| 690 |
|
---|
[391] | 691 | The newest version of TurboGears, version 2.0, moves even further in direction
|
---|
| 692 | of WSGI support and a component-based architecture. TurboGears 2 is based on
|
---|
| 693 | the WSGI stack of another popular component-based web framework, `Pylons
|
---|
| 694 | <http://pylonshq.com/>`_.
|
---|
[2] | 695 |
|
---|
| 696 |
|
---|
[391] | 697 | Zope
|
---|
| 698 | ^^^^
|
---|
| 699 |
|
---|
| 700 | The Zope framework is one of the "old original" frameworks. Its current
|
---|
| 701 | incarnation in Zope2 is a tightly integrated full-stack framework. One of its
|
---|
| 702 | most interesting feature is its tight integration with a powerful object
|
---|
| 703 | database called the `ZODB <https://launchpad.net/zodb>`_ (Zope Object Database).
|
---|
| 704 | Because of its highly integrated nature, Zope wound up in a somewhat isolated
|
---|
| 705 | ecosystem: code written for Zope wasn't very usable outside of Zope, and
|
---|
| 706 | vice-versa. To solve this problem the Zope 3 effort was started. Zope 3
|
---|
| 707 | re-engineers Zope as a set of more cleanly isolated components. This effort
|
---|
| 708 | was started before the advent of the WSGI standard, but there is WSGI support
|
---|
| 709 | for Zope 3 from the `Repoze <http://repoze.org/>`_ project. Zope components
|
---|
| 710 | have many years of production use behind them, and the Zope 3 project gives
|
---|
| 711 | access to these components to the wider Python community. There is even a
|
---|
| 712 | separate framework based on the Zope components: `Grok
|
---|
| 713 | <http://grok.zope.org/>`_.
|
---|
| 714 |
|
---|
| 715 | Zope is also the infrastructure used by the `Plone <http://plone.org/>`_ content
|
---|
| 716 | management system, one of the most powerful and popular content management
|
---|
| 717 | systems available.
|
---|
| 718 |
|
---|
| 719 |
|
---|
[2] | 720 | Other notable frameworks
|
---|
| 721 | ^^^^^^^^^^^^^^^^^^^^^^^^
|
---|
| 722 |
|
---|
[391] | 723 | Of course these are not the only frameworks that are available. There are
|
---|
| 724 | many other frameworks worth mentioning.
|
---|
[2] | 725 |
|
---|
| 726 | Another framework that's already been mentioned is `Pylons`_. Pylons is much
|
---|
[391] | 727 | like TurboGears, but with an even stronger emphasis on flexibility, which comes
|
---|
[2] | 728 | at the cost of being more difficult to use. Nearly every component can be
|
---|
| 729 | exchanged, which makes it necessary to use the documentation of every single
|
---|
[391] | 730 | component, of which there are many. Pylons builds upon `Paste
|
---|
[2] | 731 | <http://pythonpaste.org/>`_, an extensive set of tools which are handy for WSGI.
|
---|
| 732 |
|
---|
| 733 | And that's still not everything. The most up-to-date information can always be
|
---|
| 734 | found in the Python wiki.
|
---|
| 735 |
|
---|
| 736 | .. seealso::
|
---|
| 737 |
|
---|
| 738 | The Python wiki contains an extensive list of `web frameworks
|
---|
| 739 | <http://wiki.python.org/moin/WebFrameworks>`_.
|
---|
| 740 |
|
---|
| 741 | Most frameworks also have their own mailing lists and IRC channels, look out
|
---|
[391] | 742 | for these on the projects' web sites. There is also a general "Python in the
|
---|
[2] | 743 | Web" IRC channel on freenode called `#python.web
|
---|
| 744 | <http://wiki.python.org/moin/PoundPythonWeb>`_.
|
---|