1 .. MediaGoblin Documentation
3 Written in 2011, 2012, 2013 by MediaGoblin contributors
5 To the extent possible under law, the author(s) have dedicated all
6 copyright and related and neighboring rights to this software to
7 the public domain worldwide. This software is distributed without
10 You should have received a copy of the CC0 Public Domain
11 Dedication along with this software. If not, see
12 <http://creativecommons.org/publicdomain/zero/1.0/>.
14 .. _deploying-chapter:
20 GNU MediaGoblin is fairly new, and so at the time of writing there aren't
21 easy package-manager-friendly methods to install it. However, doing a basic
22 install isn't too complex in and of itself. Following this deployment guide
23 will take you step-by-step through setting up your own instance of MediaGoblin.
25 Of course, when it comes to setting up web applications like MediaGoblin,
26 there's an almost infinite way to deploy things, so for now, we'll keep it
27 simple with some assumptions. We recommend a setup that combines MediaGoblin +
28 virtualenv + Waitress + Nginx on a .deb- or .rpm-based GNU/Linux distro.
30 Other deployment options (e.g., deploying on FreeBSD, Arch Linux, using
31 Apache, etc.) are possible, though! If you'd prefer a different deployment
33 `Deployment wiki page <http://wiki.mediagoblin.org/Deployment>`_.
37 These tools are for site administrators wanting to deploy a fresh
38 install. If you want to join in as a contributor, see our
39 `Hacking HOWTO <http://wiki.mediagoblin.org/HackingHowto>`_ instead.
43 Throughout the documentation we use the ``sudo`` command to indicate that
44 an instruction requires elevated user privileges to run. You can issue
45 these commands as the ``root`` user if you prefer.
47 If you need help configuring ``sudo``, see the
48 `Debian wiki <https://wiki.debian.org/sudo/>`_ or the
49 `Fedora Project wiki <https://fedoraproject.org/wiki/Configuring_Sudo/>`_.
58 MediaGoblin has the following core dependencies:
60 - Python 2.7 or Python 3.4+
61 - `python3-lxml <http://lxml.de/>`_
62 - `git <http://git-scm.com/>`_
63 - `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_
64 - `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL)
65 - `virtualenv <http://www.virtualenv.org/>`_
66 - `nodejs <https://nodejs.org>`_
68 On a DEB-based system (e.g Debian, gNewSense, Trisquel, \*buntu, and
69 derivatives) issue the following command::
71 sudo apt-get install git-core python python3-dev python3-lxml \
72 python3-imaging python3-virtualenv npm nodejs-legacy automake \
75 On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the
78 sudo yum install python3-paste-deploy python3-paste-script \
79 git-core python python3-devel python3-lxml python3-imaging \
80 python3-virtualenv npm automake nginx rabbitmq-server
82 (Note: MediaGoblin now uses Python 3 by default. To use Python 2, you may instead
83 substitute from "python3" to "python" for most package names in the
84 Debian instructions and this should cover dependency installation.
85 These instructions have not yet been tested on Fedora.)
87 (Note: you might have to include additional repositories to a RPM-
88 based system, because rabbitmq-server might be not included in
89 official repositories. As an alternative, you can try installing
90 redis-server and configure it as celery broker)
97 MediaGoblin currently supports PostgreSQL and SQLite. The default is a
98 local SQLite database. This will "just work" for small deployments.
100 For medium to large deployments we recommend PostgreSQL.
102 If you don't want/need PostgreSQL, skip this section.
104 These are the packages needed for Debian Jessie (stable)::
106 sudo apt-get install postgresql postgresql-client python3-psycopg2
108 These are the packages needed for an RPM-based system::
110 sudo yum install postgresql postgresql-server python3-psycopg2
112 An rpm-based system also requires that you initialize and start the
113 PostgreSQL database with a few commands. The following commands are
114 not needed on a Debian-based platform, however::
116 sudo /usr/bin/postgresql-setup initdb
117 sudo systemctl enable postgresql
118 sudo systemctl start postgresql
120 The installation process will create a new *system* user named ``postgres``,
121 which will have privileges sufficient to manage the database. We will create a
122 new database user with restricted privileges and a new database owned by our
123 restricted database user for our MediaGoblin instance.
125 In this example, the database user will be ``mediagoblin`` and the database
126 name will be ``mediagoblin`` too.
128 We'll add these entities by first switching to the *postgres* account::
132 This will change your prompt to a shell prompt, such as *-bash-4.2$*. Enter
133 the following *createuser* and *createdb* commands at that prompt. We'll
134 create the *mediagoblin* database user first::
136 # this command and the one that follows are run as the ``postgres`` user:
137 createuser -A -D mediagoblin
139 Then we'll create the database where all of our MediaGoblin data will be stored::
141 createdb -E UNICODE -O mediagoblin mediagoblin
143 where the first ``mediagoblin`` is the database owner and the second
144 ``mediagoblin`` is the database name.
146 Type ``exit`` to exit from the 'postgres' user account.::
150 .. caution:: Where is the password?
152 These steps enable you to authenticate to the database in a password-less
153 manner via local UNIX authentication provided you run the MediaGoblin
154 application as a user with the same name as the user you created in
157 More on this in :ref:`Drop Privileges for MediaGoblin <drop-privileges-for-mediagoblin>`.
160 .. _drop-privileges-for-mediagoblin:
162 Drop Privileges for MediaGoblin
163 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
165 MediaGoblin does not require special permissions or elevated
166 access to run. As such, the preferred way to run MediaGoblin is to
167 create a dedicated, unprivileged system user for the sole purpose of running
168 MediaGoblin. Running MediaGoblin processes under an unprivileged system user
169 helps to keep it more secure.
171 The following command (entered as root or with sudo) will create a
172 system account with a username of ``mediagoblin``. You may choose a different
173 username if you wish.
175 If you are using a Debian-based system, enter this command::
177 sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g www-data mediagoblin
179 If you are using an RPM-based system, enter this command::
181 sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g nginx mediagoblin
183 This will create a ``mediagoblin`` user and assign it to a group that is
184 associated with the web server. This will ensure that the web server can
185 read the media files (images, videos, etc.) that users upload.
187 We will also create a ``mediagoblin`` group and associate the mediagoblin
188 user with that group, as well::
190 sudo groupadd mediagoblin && sudo usermod --append -G mediagoblin mediagoblin
192 No password will be assigned to this account, and you will not be able
193 to log in as this user. To switch to this account, enter::
195 sudo su mediagoblin -s /bin/bash
197 To return to your regular user account after using the system account, type
200 .. _create-mediagoblin-directory:
202 Create a MediaGoblin Directory
203 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
205 You should create a working directory for MediaGoblin. This document
206 assumes your local git repository will be located at
207 ``/srv/mediagoblin.example.org/mediagoblin/``.
208 Substitute your preferred local deployment path as needed.
210 Setting up the working directory requires that we first create the directory
211 with elevated privileges, and then assign ownership of the directory
212 to the unprivileged system account.
214 To do this, enter the following command, changing the defaults to suit your
215 particular requirements. On a Debian-based platform you will enter this::
217 sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:www-data /srv/mediagoblin.example.org
219 On an RPM-based distribution, enter this command::
221 sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:nginx /srv/mediagoblin.example.org
225 Unless otherwise noted, the remainder of this document assumes that all
226 operations are performed using this unprivileged account.
229 Install MediaGoblin and Virtualenv
230 ----------------------------------
232 We will now switch to our 'mediagoblin' system account, and then set up
233 our MediaGoblin source code repository and its necessary services.
234 You should modify these commands to suit your own environment.
236 Change to the MediaGoblin directory that you just created::
238 sudo su mediagoblin -s /bin/bash # to change to the 'mediagoblin' account
239 $ cd /srv/mediagoblin.example.org
241 Clone the MediaGoblin repository and set up the git submodules::
243 $ git clone https://git.savannah.gnu.org/git/mediagoblin.git -b stable
245 $ git submodule init && git submodule update
249 The MediaGoblin repository used to be on gitorious.org, but since
250 gitorious.org shut down, we had to move. We are presently on
251 Savannah. You may need to update your git repository location::
253 $ git remote set-url origin https://git.savannah.gnu.org/git/mediagoblin.git
255 Set up the hacking environment::
257 $ ./bootstrap.sh && ./configure && make
259 (Note that if you'd prefer to run MediaGoblin with Python 2, pass in
260 `--without-python3` to the `./configure` command.)
262 Create and set the proper permissions on the ``user_dev`` directory.
263 This directory will be used to store uploaded media files::
265 $ mkdir user_dev && chmod 750 user_dev
267 This concludes the initial configuration of the MediaGoblin
268 environment. In the future, when you update your
269 codebase, you should also run::
271 $ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate
275 Note: If you are running an active site, depending on your server
276 configuration, you may need to stop it first or the dbupdate command
277 may hang (and it's certainly a good idea to restart it after the
281 Deploy MediaGoblin Services
282 ---------------------------
284 Edit site configuration
285 ~~~~~~~~~~~~~~~~~~~~~~~
287 A few basic properties must be set before MediaGoblin will work. First
288 make a copy of ``paste.ini`` for editing so the original
289 config files aren't lost (you likely won't need to edit the paste configuration,
290 but we'll make a local copy of it just in case)::
292 $ cp -av paste.ini paste_local.ini
294 ``mediagoblin.ini`` does not need to be copied, because original config is
295 stored in ``mediagoblin.example.ini``.
297 Then edit ``mediagoblin.ini``:
298 - Set ``email_sender_address`` to the address you wish to be used as
299 the sender for system-generated emails
300 - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if
301 your mediagoblin directory is not the root directory of your
305 Configure MediaGoblin to use the PostgreSQL database
306 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
308 If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your
309 ``mediagoblin.ini`` and put in::
311 sql_engine = postgresql:///mediagoblin
313 if you are running the MediaGoblin application as the same 'user' as the
317 Update database data structures
318 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
320 Before you start using the database, you need to run::
324 to populate the database with the MediaGoblin data structures.
330 At this point MediaGoblin should be properly installed. You can
331 test the deployment with the following command::
333 $ ./lazyserver.sh --server-name=broadcast
335 You should be able to connect to the machine on port 6543 in your
336 browser to confirm that the service is operable.
338 The next series of commands will need to be run as a privileged user. Type
339 exit to return to the root/sudo account.::
343 .. _webserver-config:
349 This configuration example will use Nginx, however, you may
350 use any webserver of your choice. If you do not already have
351 a web server, consider Nginx, as the configuration files may
352 be more clear than the alternatives.
354 Create a configuration file at
355 ``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link
356 into a directory that will be included in your ``nginx`` configuration
357 (e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with
358 one of the following commands.
360 On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and
361 derivatives) issue the following commands::
363 sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/
364 sudo systemctl enable nginx
366 On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the
369 sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/
370 sudo systemctl enable nginx
372 You can modify these commands and locations depending on your preferences and
373 the existing configuration of your Nginx instance. The contents of
374 this ``nginx.conf`` file should be modeled on the following::
377 #################################################
378 # Stock useful config options, but ignore them :)
379 #################################################
380 include /etc/nginx/mime.types;
383 default_type application/octet-stream;
388 gzip_min_length 1024;
390 gzip_types text/plain application/x-javascript text/javascript text/xml text/css;
392 #####################################
393 # Mounting MediaGoblin stuff
394 # This is the section you should read
395 #####################################
397 # Change this to update the upload size limit for your users
398 client_max_body_size 8m;
400 # prevent attacks (someone uploading a .txt file that the browser
401 # interprets as an HTML file, etc.)
402 add_header X-Content-Type-Options nosniff;
404 server_name mediagoblin.example.org www.mediagoblin.example.org;
405 access_log /var/log/nginx/mediagoblin.example.access.log;
406 error_log /var/log/nginx/mediagoblin.example.error.log;
408 # MediaGoblin's stock static files: CSS, JS, etc.
409 location /mgoblin_static/ {
410 alias /srv/mediagoblin.example.org/mediagoblin/mediagoblin/static/;
413 # Instance specific media:
414 location /mgoblin_media/ {
415 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/media/public/;
418 # Theme static files (usually symlinked in)
419 location /theme_static/ {
420 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/theme_static/;
423 # Plugin static files (usually symlinked in)
424 location /plugin_static/ {
425 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/;
428 # Forward requests to the MediaGoblin app server.
430 proxy_pass http://127.0.0.1:6543;
434 The first four ``location`` directives instruct Nginx to serve the
435 static and uploaded files directly rather than through the MediaGoblin
436 process. This approach is faster and requires less memory.
440 The user who owns the Nginx process, normally ``www-data`` or ``nginx``,
441 requires execute permission on the directories ``static``,
442 ``public``, ``theme_static`` and ``plugin_static`` plus all their
443 parent directories. This user also requires read permission on all
444 the files within these directories. This is normally the default.
446 Nginx is now configured to serve the MediaGoblin application. Perform a quick
447 test to ensure that this configuration works::
451 If you encounter any errors, review your Nginx configuration files, and try to
452 resolve them. If you do not encounter any errors, you can start your Nginx
453 server with one of the following commands (depending on your environment)::
455 sudo /etc/init.d/nginx restart
456 sudo /etc/rc.d/nginx restart
457 sudo systemctl restart nginx
459 Now start MediaGoblin. Use the following command sequence as an
462 cd /srv/mediagoblin.example.org/mediagoblin/
463 su mediagoblin -s /bin/bash
464 ./lazyserver.sh --server-name=main
466 Visit the site you've set up in your browser by visiting
467 <http://mediagoblin.example.org>. You should see MediaGoblin!
471 The configuration described above is sufficient for development and
472 smaller deployments. However, for larger production deployments
473 with larger processing requirements, see the
474 ":doc:`production-deployments`" documentation.
480 Instructions and scripts for running MediaGoblin on an Apache server
481 can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_.
484 Should I Keep Open Registration Enabled?
485 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
487 Unfortunately, in this current release of MediaGoblin we are suffering
488 from spammers registering to public instances en masse. As such, you
491 a) Disable registration on your instance and just make
492 accounts for people you know and trust (eg via the `gmg adduser`
493 command). You can disable registration in your mediagoblin.ini
497 allow_registration = false
499 b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA
500 plugins exist, for various reasons we do not have any general
501 recommendations we can make at this point.
503 We hope to have a better solution to this situation shortly. We
504 apologize for the inconvenience in the meanwhile.
507 Security Considerations
508 ~~~~~~~~~~~~~~~~~~~~~~~
512 The directory ``user_dev/crypto/`` contains some very
514 Especially the ``itsdangeroussecret.bin`` is very important
515 for session security. Make sure not to leak its contents anywhere.
516 If the contents gets leaked nevertheless, delete your file
517 and restart the server, so that it creates a new secret key.
518 All previous sessions will be invalidated.