docs/source/production-deployments.rst

   1 =========================================
   2 Considerations for Production Deployments
   3 =========================================
   4
   5 This document contains a number of suggestions for deploying
   6 MediaGoblin in actual production environments. Consider
   7 ":doc:`deploying`" for a basic overview of how to deploy MediaGoblin.
   8
   9 Deploy with Paste
  10 -----------------
  11
  12 The instance configured with ``./lazyserver.sh`` is not ideal for a
  13 production MediaGoblin deployment. Ideally, you should be able to use
  14 an "init" or "control" script to launch and restart the MediaGoblin
  15 process.
  16
  17 Use the following command as the basis for such a script: ::
  18
  19     CELERY_ALWAYS_EAGER=true \
  20      /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
  21      /srv/mediagoblin.example.org/mediagoblin/paste.ini \
  22      --pid-file=/var/run/mediagoblin.pid \
  23      --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
  24
  25 The above configuration places MediaGoblin in "always eager" mode
  26 with Celery, this means that submissions of content will be processed
  27 synchronously, and the user will advance to the next page only after
  28 processing is complete. If we take Celery out of "always eager mode,"
  29 the user will be able to immediately return to the MediaGoblin site
  30 while processing is ongoing. In these cases, use the following command
  31 as the basis for your script: ::
  32
  33     CELERY_ALWAYS_EAGER=false \
  34      /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
  35      /srv/mediagoblin.example.org/mediagoblin/paste.ini \
  36      --pid-file=/var/run/mediagoblin.pid \
  37      --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
  38
  39 Separate Celery
  40 ---------------
  41
  42 While the ``./lazyserer.sh`` configuration provides an efficient way to
  43 start using a MediaGoblin instance, it is not suitable for production
  44 deployments for several reasons:
  45
  46 In nearly every scenario, work on the Celery queue will need to
  47 balance with the demands of other processes, and cannot proceed
  48 synchronously. This is a particularly relevant problem if you use
  49 MediaGoblin to host video content. Processing with Celery ought to be
  50 operationally separate from the MediaGoblin application itself, this
  51 simplifies management and support better workload distribution.
  52
  53 Basically, if you're doing anything beyond a trivial workload, such as
  54 image hosting for a small set of users, or have limited media types
  55 such as "ASCII art" or icon sharing, you will need to run ``celeryd``
  56 as a separate process.
  57
  58 Build an :ref:`init script <init-script>` around the following
  59 command::
  60
  61     CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd
  62
  63 Modify your existing MediaGoblin and application init scripts, if
  64 necessary, to prevent them from starting their own ``celeryd``
  65 processes.
  66
  67 .. _init-script:
  68
  69 Use an Init Script
  70 ------------------
  71
  72 Look in your system's ``/etc/init.d/`` or ``/etc/rc.d/`` directory for
  73 examples of how to build scripts that will start, stop, and restart
  74 MediaGoblin and Celery. These scripts will vary by
  75 distribution/operating system. In the future, MediaGoblin will provide
  76 example scripts as examples.
  77
  78 .. TODO insert init script here
  79 .. TODO are additional concerns ?
  80    .. Other Concerns
  81    .. --------------