docs/source/siteadmin/production-deployments.rst

   1 .. MediaGoblin Documentation
   2
   3    Written in 2011, 2012 by MediaGoblin contributors
   4
   5    To the extent possible under law, the author(s) have dedicated all
   6    copyright and related and neighboring rights to this software to
   7    the public domain worldwide. This software is distributed without
   8    any warranty.
   9
  10    You should have received a copy of the CC0 Public Domain
  11    Dedication along with this software. If not, see
  12    <http://creativecommons.org/publicdomain/zero/1.0/>.
  13
  14 =========================================
  15 Considerations for Production Deployments
  16 =========================================
  17
  18 This document contains a number of suggestions for deploying
  19 MediaGoblin in actual production environments. Consider
  20 ":doc:`deploying`" for a basic overview of how to deploy MediaGoblin.
  21
  22 Deploy with Paste
  23 -----------------
  24
  25 The instance configured with ``./lazyserver.sh`` is not ideal for a
  26 production MediaGoblin deployment. Ideally, you should be able to use
  27 an "init" or "control" script to launch and restart the MediaGoblin
  28 process.
  29
  30 Use the following command as the basis for such a script: ::
  31
  32     CELERY_ALWAYS_EAGER=true \
  33      /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
  34      /srv/mediagoblin.example.org/mediagoblin/paste.ini \
  35      --pid-file=/var/run/mediagoblin.pid \
  36      --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
  37
  38 The above configuration places MediaGoblin in "always eager" mode
  39 with Celery, this means that submissions of content will be processed
  40 synchronously, and the user will advance to the next page only after
  41 processing is complete. If we take Celery out of "always eager mode,"
  42 the user will be able to immediately return to the MediaGoblin site
  43 while processing is ongoing. In these cases, use the following command
  44 as the basis for your script: ::
  45
  46     CELERY_ALWAYS_EAGER=false \
  47      /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
  48      /srv/mediagoblin.example.org/mediagoblin/paste.ini \
  49      --pid-file=/var/run/mediagoblin.pid \
  50      --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
  51
  52 Separate Celery
  53 ---------------
  54
  55 While the ``./lazyserer.sh`` configuration provides an efficient way to
  56 start using a MediaGoblin instance, it is not suitable for production
  57 deployments for several reasons:
  58
  59 In nearly every scenario, work on the Celery queue will need to
  60 balance with the demands of other processes, and cannot proceed
  61 synchronously. This is a particularly relevant problem if you use
  62 MediaGoblin to host video content. Processing with Celery ought to be
  63 operationally separate from the MediaGoblin application itself, this
  64 simplifies management and support better workload distribution.
  65
  66 Basically, if you're doing anything beyond a trivial workload, such as
  67 image hosting for a small set of users, or have limited media types
  68 such as "ASCII art" or icon sharing, you will need to run ``celeryd``
  69 as a separate process.
  70
  71 Build an :ref:`init script <init-script>` around the following
  72 command::
  73
  74     CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd
  75
  76 Modify your existing MediaGoblin and application init scripts, if
  77 necessary, to prevent them from starting their own ``celeryd``
  78 processes.
  79
  80 .. _init-script:
  81
  82 Use an Init Script
  83 ------------------
  84
  85 Look in your system's ``/etc/init.d/`` or ``/etc/rc.d/`` directory for
  86 examples of how to build scripts that will start, stop, and restart
  87 MediaGoblin and Celery. These scripts will vary by
  88 distribution/operating system.
  89
  90 These are scripts provided by the MediaGoblin community:
  91
  92 Debian
  93   * `GNU MediaGoblin init scripts
  94     <https://github.com/jwandborg/mediagoblin-init-scripts>`_
  95     by `Joar Wandborg <http://wandborg.se>`_
  96
  97 Arch Linux
  98   * `MediaGoblin - ArchLinux rc.d scripts
  99     <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_
 100     by `Jeremy Pope <http://jpope.org/>`_
 101   * `Mediagoblin init script on Archlinux
 102     <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_
 103     by `Chimo <http://chimo.chromic.org/>`_
 104
 105 .. TODO insert init script here
 106 .. TODO are additional concerns ?
 107    .. Other Concerns
 108    .. --------------