docs/source/siteadmin/production-deployments.rst

   1 .. MediaGoblin Documentation
   2
   3    Written in 2011, 2012 by MediaGoblin contributors
   4
   5    To the extent possible under law, the author(s) have dedicated all
   6    copyright and related and neighboring rights to this software to
   7    the public domain worldwide. This software is distributed without
   8    any warranty.
   9
  10    You should have received a copy of the CC0 Public Domain
  11    Dedication along with this software. If not, see
  12    <http://creativecommons.org/publicdomain/zero/1.0/>.
  13
  14 =========================================
  15 Considerations for Production Deployments
  16 =========================================
  17
  18 This document contains a number of suggestions for deploying
  19 MediaGoblin in actual production environments. Consider
  20 ":doc:`deploying`" for a basic overview of how to deploy MediaGoblin.
  21
  22 Deploy with Paste
  23 -----------------
  24
  25 The MediaGoblin WSGI application instance you get with ``./lazyserver.sh`` is
  26 not ideal for a production MediaGoblin deployment. Ideally, you should be able
  27 to use an "init" or "control" script to launch and restart the MediaGoblin
  28 process.
  29
  30 Use the following command as the basis for such a script:
  31
  32 .. code-block:: bash
  33
  34     CELERY_ALWAYS_EAGER=true \
  35      /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
  36      /srv/mediagoblin.example.org/mediagoblin/paste.ini \
  37      --pid-file=/var/run/mediagoblin.pid \
  38      --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
  39
  40 The above configuration places MediaGoblin in "always eager" mode
  41 with Celery, this means that submissions of content will be processed
  42 synchronously, and the user will advance to the next page only after
  43 processing is complete. If we take Celery out of "always eager mode,"
  44 the user will be able to immediately return to the MediaGoblin site
  45 while processing is ongoing. In these cases, use the following command
  46 as the basis for your script:
  47
  48 .. code-block:: bash
  49
  50     CELERY_ALWAYS_EAGER=false \
  51      /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
  52      /srv/mediagoblin.example.org/mediagoblin/paste.ini \
  53      --pid-file=/var/run/mediagoblin.pid \
  54      --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
  55
  56 Separate Celery
  57 ---------------
  58
  59 MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can
  60 be launched in two ways:
  61
  62 1.  Embedded in the MediaGoblin WSGI application [#f-mediagoblin-wsgi-app]_.
  63     This is the way ``./lazyserver.sh`` does it for you. It's simple as you
  64     only have to run one process. The only bad thing with this is that the
  65     heavy and long-running tasks will run *in* the webserver, keeping the user
  66     waiting each time some heavy lifting is needed as in for example processing
  67     a video. This could lead to problems as an aborted connection will halt any
  68     processing and since most front-end web servers *will* terminate your
  69     connection if it doesn't get any response from the MediaGoblin WSGI
  70     application in a while.
  71
  72 2.  As a separate process communicating with the MediaGoblin WSGI application
  73     via a `broker`_. This offloads the heavy lifting from the MediaGoblin WSGI
  74     application and users will be able to continue to browse the site while the
  75     media is being processed in the background.
  76
  77 .. _`broker`: http://docs.celeryproject.org/en/latest/getting-started/brokers/
  78 .. _`celery`: http://www.celeryproject.org/
  79
  80
  81 .. [#f-mediagoblin-wsgi-app] The MediaGoblin WSGI application is the part that
  82     of MediaGoblin that processes HTTP requests.
  83
  84 To launch Celery separately from the MediaGoblin WSGI application:
  85
  86 1.  Make sure that the ``CELERY_ALWAYS_EAGER`` environment variable is unset or
  87     set to ``false`` when launching the MediaGoblin WSGI application.
  88 2.  Start the ``celeryd`` main process with
  89
  90     .. code-block:: bash
  91
  92         CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd
  93
  94 .. _sentry:
  95
  96 Set up sentry to monitor exceptions
  97 -----------------------------------
  98
  99 We have a plugin for `raven`_ integration, see the ":doc:`/plugindocs/raven`"
 100 documentation.
 101
 102 .. _`raven`: http://raven.readthedocs.org
 103
 104
 105 .. _init-script:
 106
 107 Use an Init Script
 108 ------------------
 109
 110 Look in your system's ``/etc/init.d/`` or ``/etc/rc.d/`` directory for
 111 examples of how to build scripts that will start, stop, and restart
 112 MediaGoblin and Celery. These scripts will vary by
 113 distribution/operating system.
 114
 115 These are scripts provided by the MediaGoblin community:
 116
 117 Debian
 118   * `GNU MediaGoblin init scripts
 119     <https://github.com/joar/mediagoblin-init-scripts>`_
 120     by `Joar Wandborg <http://wandborg.se>`_
 121
 122 Arch Linux
 123   * `MediaGoblin - ArchLinux rc.d scripts
 124     <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_
 125     by `Jeremy Pope <http://jpope.org/>`_
 126   * `Mediagoblin init script on Archlinux
 127     <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_
 128     by `Chimo <http://chimo.chromic.org/>`_
 129
 130 .. TODO insert init script here
 131 .. TODO are additional concerns ?
 132    .. Other Concerns
 133    .. --------------