Secrets of a WSGI master

Secrets of 
a WSGI master.
Graham Dumpleton 
Graham.Dumpleton@gmail.com
@GrahamDumpleton
https://www.slideshare.net/GrahamDumpleton/secrets-of-a-wsgi-master

What is WSGI?
Web Browser
Web Browser
Web Browser
Web Server
HTTP
HTTP
HTTP
File System
(Static Files)
Python
Web Application
WSGI
WSGI == Web Server Gateway Interface (PEP 3333)

WSGI is a speciﬁcation for an
Application Programming Interface
WSGI is NOT a wire protocol
WSGI is NOT an implementation
of any anything

def application(environ, start_response):
status = '200 OK'
output = b'Hello World!'
response_headers = [
('Content-Type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [output]

Friends don’t let friends
use raw WSGI

You still need a way to host a
WSGI application
The development servers builtin to
a framework are not good enough

Installing mod_wsgi the easy way
pip install mod_wsgi
https://pypi.python.org/pypi/mod_wsgi

Run mod_wsgi from the command line
mod_wsgi-express start-server wsgi.py

No Apache conﬁguration required
$ mod_wsgi-express start-server wsgi.py
Server URL : http://localhost:8000/
Server Root : /tmp/mod_wsgi-localhost:8000:502
Server Conf : /tmp/mod_wsgi-localhost:8000:502/httpd.conf
Error Log File : /tmp/mod_wsgi-localhost:8000:502/error_log (warn)
Request Capacity : 5 (1 process * 5 threads)
Request Timeout : 60 (seconds)
Startup Timeout : 15 (seconds)
Queue Backlog : 100 (connections)
Queue Timeout : 45 (seconds)
Server Capacity : 20 (event/worker), 20 (prefork)
Server Backlog : 500 (connections)
Locale Setting : en_AU.UTF-8

Django framework integration
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'mod_wsgi.server',
] Add mod_wsgi.server

Run with Django management command
python manage.py runmodwsgi

Automatic code reloading
python manage.py runmodwsgi --reload-on-changes

Interactive debugger
python manage.py runmodwsgi --enable-debugger

Production conﬁguration #1
python manage.py runmodwsgi
--server-root /etc/wsgi-port-80
--user www-data --group www-data
--port 80 --setup-only

Production conﬁguration #2
mod_wsgi-express start-server wsgi.py
--server-root /etc/wsgi-port-80
--user www-data --group www-data
--port 80 --url-alias /static static
--setup-only

Running in production
/etc/wsgi-port-80/apachectl start
/etc/wsgi-port-80/apachectl restart
/etc/wsgi-port-80/apachectl stop

Build a container image
FROM python:3
RUN apt-get update &&
apt-get install -y --no-install-recommends apache2 apache2-dev locales &&
apt-get clean && rm -r /var/lib/apt/lists/*
RUN echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
RUN pip install --no-cache-dir mod_wsgi
WORKDIR /opt/app-root
COPY . /opt/app-root
EXPOSE 80
CMD [ "mod_wsgi-express", "start-server", "--port", "80", "--user",
"www-data", "--group", "www-data", "--log-to-terminal", "wsgi.py" ]
Install Apache
Fix Unicode Problems
Run mod_wsgi-express

Building the container
$ docker build -t mypyapp .
Sending build context to Docker daemon 3.584kB
Step 1/9 : FROM python:3
---> 968120d8cbe8
Step 2/9 : WORKDIR /opt/app-root
---> Using cache
---> 003096a40d39
Step 3/9 : .......

Starting the container
$ docker run --rm -p 80:80 mypyapp

run containers as root

use Python without a
Python virtual environment

A better container image #1
FROM python:3
RUN apt-get update &&
apt-get install -y --no-install-recommends apache2 apache2-dev locales &&
apt-get clean &&
rm -r /var/lib/apt/lists/*
RUN adduser --disabled-password --gecos "Warp Drive" --uid 1001
--gid 0 --home /opt/app-root warpdrive &&
chmod g+w /etc/passwd
RUN echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8
PATH=/opt/app-root/bin:$PATH
HOME=/opt/app-root
Create non root user

A better container image #2
RUN pip install --no-cache-dir virtualenv &&
virtualenv /opt/app-root &&
. /opt/app-root/bin/activate &&
pip install --no-cache-dir warpdrive &&
warpdrive fixup /opt/app-root
WORKDIR /opt/app-root
COPY . /opt/app-root/src
RUN warpdrive fixup /opt/app-root/src
USER 1001
RUN warpdrive build &&
warpdrive fixup /opt/app-root
EXPOSE 8080
CMD [ "warpdrive", "start" ]
Create a Python 
virtual environment
Use warpdrive,
it's magicUse non
root user
Non
privileged
port

Running as non root
$ docker run mypyapp warpdrive exec id
uid=1001(warpdrive) gid=0(root) groups=0(root)
$ docker run -u 100001 mypyapp warpdrive exec id
uid=100001(warpdrive) gid=0(root) groups=0(root)
Default to assigned non root user
Can be run as arbitrary high user ID

Same tools for development
$ warpdrive project mypyapp
(warpdrive+mypyapp) $ warpdrive build
(warpdrive+mypyapp) $ warpdrive start
https://pypi.python.org/pypi/warpdrive

Generate image with no Dockerﬁle
$ warpdrive image mypyapp
$ docker run --rm -p 80:8080 mypyapp

Source-to-Image
$ s2i build https://github.com/GrahamDumpleton/warpdrive-django-modwsgi
getwarped/warp0-debian8-python35 mypyapp
https://github.com/openshift/source-to-image

OpenShift
$ oc new-app --image-stream getwarped/warp0-debian8-python35
--code https://github.com/GrahamDumpleton/warpdrive-django-modwsgi
--name mypyapp
https://www.openshift.com

Manual Apache conﬁguration
$ mod_wsgi-express module-config
LoadModule wsgi_module "/.../venv/.../mod_wsgi/server/mod_wsgi-py27.so"
WSGIPythonHome "/.../venv"

use embedded mode
of mod_wsgi

Manual daemon mode conﬁguration
WSGIRestrictEmbedded On
WSGIDaemonProcess mypyapp python-home=/.../env
WSGIScriptAlias / /.../src/wsgi.py
process-group=mypyapp application-group=%{GLOBAL}
<Directory /.../src>
<Files wsgi.py>
Require all granted
</Files>
</Directory>

The missing options
• display-name='%{GROUP}'
• lang='en_US.UTF-8'
• locale='en_US.UTF-8'
• startup-timeout=15
• connect-timeout=15
• socket-timeout=60
• queue-timeout=45
• request-timeout=60
• inactivity-timeout=0
• restart-interval=0
• maximum-requests=0
• shutdown-timeout=5
• deadlock-timeout=60
• graceful-timeout=15
• eviction-timeout=0
http://modwsgi.readthedocs.io/en/develop/conﬁguration-directives/WSGIDaemonProcess.html

Failed application loading
• startup-timeout=15 
 
Deﬁnes the maximum number of seconds allowed to pass waiting to
see if a WSGI script ﬁle can be loaded successfully by a daemon
process. When the timeout is passed, the process will be restarted.

Connection timeouts
• connect-timeout=15 
 
Defines the maximum amount of time for an Apache child process to wait trying to get a
successful connection to the mod_wsgi daemon processes. This defaults to 15 seconds.
• socket-timeout=60 
 
Defines the timeout on individual reads/writes on the socket connection between the Apache
child processes and the mod_wsgi daemon processes. If not specified, the number of seconds
specified by the Apache Timeout directive will be used instead. See also response-socket-
timeout if need to control this only for writing back content of the response.
• queue-timeout=45 
 
Defines the timeout on how long to wait for a mod_wsgi daemon process to accept a request for
processing. Not enabled by default.

Time triggered restart
• request-timeout=60 
 
Deﬁnes the maximum number of seconds that a request is allowed
to run before the daemon process is restarted.
• inactivity-timeout=0 
 
Deﬁnes the maximum number of seconds allowed to pass before
the daemon process is shutdown and restarted when the daemon
process has entered an idle state. To restart on stuck requests use
request-timeout instead.

Request Monitoring
import mod_wsgi
def event_handler(name, **kwargs):
if name == 'request_started':
...
elif name == 'request_finished':
environ = kwargs['request_environ']
response_time = kwargs.get('response_time')
cpu_user_time = kwargs.get('cpu_user_time')
cpu_system_time = kwargs.get('cpu_system_time')
...
elif name == 'request_exception':
...
mod_wsgi.subscribe_events(event_handler)

Resources
• mod_wsgi - http://modwsgi.readthedocs.io
• warpdrive - http://warpdrive.readthedocs.io
• Source-to-Image - https://github.com/openshift/source-to-image
• OpenShift - https://www.openshift.com

Friends don't let friends
use Windows for running
Python web applications

Friends don't let friends use the
mod_wsgi which comes packaged
with the operating system

Friends don't let friends use
those other WSGI servers

Friends don't let friends
make things too complicated,
simple is good

Graham.Dumpleton@gmail.com
@GrahamDumpleton
https://www.slideshare.net/GrahamDumpleton/secrets-of-a-wsgi-master

Secrets of a WSGI master

Recommended

Recommended

More Related Content

More from Graham Dumpleton

More from Graham Dumpleton (8)

Recently uploaded

Recently uploaded (20)

Secrets of a WSGI master