1 .. MediaGoblin Documentation
3 Written in 2011, 2012, 2013 by MediaGoblin contributors
5 To the extent possible under law, the author(s) have dedicated all
6 copyright and related and neighboring rights to this software to
7 the public domain worldwide. This software is distributed without
10 You should have received a copy of the CC0 Public Domain
11 Dedication along with this software. If not, see
12 <http://creativecommons.org/publicdomain/zero/1.0/>.
14 .. _deploying-chapter:
20 GNU MediaGoblin is fairly new, and so at the time of writing there aren't
21 easy package-manager-friendly methods to install it. However, doing a basic
22 install isn't too complex in and of itself. Following this deployment guide
23 will take you step-by-step through setting up your own instance of MediaGoblin.
25 Of course, when it comes to setting up web applications like MediaGoblin,
26 there's an almost infinite way to deploy things, so for now, we'll keep it
27 simple with some assumptions. We recommend a setup that combines MediaGoblin +
28 virtualenv + FastCGI + Nginx on a .deb- or .rpm-based GNU/Linux distro.
30 Other deployment options (e.g., deploying on FreeBSD, Arch Linux, using
31 Apache, etc.) are possible, though! If you'd prefer a different deployment
33 `Deployment wiki page <http://wiki.mediagoblin.org/Deployment>`_.
37 These tools are for site administrators wanting to deploy a fresh
38 install. If you want to join in as a contributor, see our
39 `Hacking HOWTO <http://wiki.mediagoblin.org/HackingHowto>`_ instead.
43 Throughout the documentation we use the ``sudo`` command to indicate that
44 an instruction requires elevated user privileges to run. You can issue
45 these commands as the ``root`` user if you prefer.
47 If you need help configuring ``sudo``, see the
48 `Debian wiki <https://wiki.debian.org/sudo/>`_ or the
49 `Fedora Project wiki <https://fedoraproject.org/wiki/Configuring_Sudo/>`_.
58 MediaGoblin has the following core dependencies:
60 - Python 2.7 or Python 3.4+
61 - `python-lxml <http://lxml.de/>`_
62 - `git <http://git-scm.com/>`_
63 - `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_
64 - `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL)
65 - `virtualenv <http://www.virtualenv.org/>`_
66 - `nodejs <https://nodejs.org>`_
68 On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and
69 derivatives) issue the following command::
71 sudo apt-get install git-core python python-dev python-lxml \
72 python-imaging python-virtualenv npm nodejs-legacy automake \
75 On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the
78 sudo yum install python-paste-deploy python-paste-script \
79 git-core python python-devel python-lxml python-imaging \
80 python-virtualenv npm automake nginx
82 (Note: MediaGoblin now officially supports Python 3. You may instead
83 substitute from "python" to "python3" for most package names in the
84 Debian instructions and this should cover dependency installation.
85 These instructions have not yet been tested on Fedora.)
92 MediaGoblin currently supports PostgreSQL and SQLite. The default is a
93 local SQLite database. This will "just work" for small deployments.
95 For medium to large deployments we recommend PostgreSQL.
97 If you don't want/need PostgreSQL, skip this section.
99 These are the packages needed for Debian Jessie (stable)::
101 sudo apt-get install postgresql postgresql-client python-psycopg2
103 These are the packages needed for an RPM-based system::
105 sudo yum install postgresql postgresql-server python-psycopg2
107 An rpm-based system also requires that you initialize and start the
108 PostgreSQL database with a few commands. The following commands are
109 not needed on a Debian-based platform, however::
111 sudo /usr/bin/postgresql-setup initdb
112 sudo systemctl enable postgresql
113 sudo systemctl start postgresql
115 The installation process will create a new *system* user named ``postgres``,
116 which will have privileges sufficient to manage the database. We will create a
117 new database user with restricted privileges and a new database owned by our
118 restricted database user for our MediaGoblin instance.
120 In this example, the database user will be ``mediagoblin`` and the database
121 name will be ``mediagoblin`` too.
123 We'll add these entities by first switching to the *postgres* account::
127 This will change your prompt to a shell prompt, such as *-bash-4.2$*. Enter
128 the following *createuser* and *createdb* commands at that prompt. We'll
129 create the *mediagoblin* database user first::
131 # this command and the one that follows are run as the ``postgres`` user:
132 createuser -A -D mediagoblin
134 Then we'll create the database where all of our MediaGoblin data will be stored::
136 createdb -E UNICODE -O mediagoblin mediagoblin
138 where the first ``mediagoblin`` is the database owner and the second
139 ``mediagoblin`` is the database name.
141 Type ``exit`` to exit from the 'postgres' user account.::
145 .. caution:: Where is the password?
147 These steps enable you to authenticate to the database in a password-less
148 manner via local UNIX authentication provided you run the MediaGoblin
149 application as a user with the same name as the user you created in
152 More on this in :ref:`Drop Privileges for MediaGoblin <drop-privileges-for-mediagoblin>`.
155 .. _drop-privileges-for-mediagoblin:
157 Drop Privileges for MediaGoblin
158 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
160 MediaGoblin does not require special permissions or elevated
161 access to run. As such, the preferred way to run MediaGoblin is to
162 create a dedicated, unprivileged system user for the sole purpose of running
163 MediaGoblin. Running MediaGoblin processes under an unprivileged system user
164 helps to keep it more secure.
166 The following command (entered as root or with sudo) will create a
167 system account with a username of ``mediagoblin``. You may choose a different
168 username if you wish.
170 If you are using a Debian-based system, enter this command::
172 sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g www-data mediagoblin
174 If you are using an RPM-based system, enter this command::
176 sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g nginx mediagoblin
178 This will create a ``mediagoblin`` user and assign it to a group that is
179 associated with the web server. This will ensure that the web server can
180 read the media files (images, videos, etc.) that users upload.
182 We will also create a ``mediagoblin`` group and associate the mediagoblin
183 user with that group, as well::
185 sudo groupadd mediagoblin && sudo usermod --append -G mediagoblin mediagoblin
187 No password will be assigned to this account, and you will not be able
188 to log in as this user. To switch to this account, enter::
190 sudo su mediagoblin -s /bin/bash
192 To return to your regular user account after using the system account, type
195 .. _create-mediagoblin-directory:
197 Create a MediaGoblin Directory
198 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
200 You should create a working directory for MediaGoblin. This document
201 assumes your local git repository will be located at
202 ``/srv/mediagoblin.example.org/mediagoblin/``.
203 Substitute your preferred local deployment path as needed.
205 Setting up the working directory requires that we first create the directory
206 with elevated privileges, and then assign ownership of the directory
207 to the unprivileged system account.
209 To do this, enter the following command, changing the defaults to suit your
210 particular requirements. On a Debian-based platform you will enter this::
212 sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:www-data /srv/mediagoblin.example.org
214 On an RPM-based distribution, enter this command::
216 sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:nginx /srv/mediagoblin.example.org
220 Unless otherwise noted, the remainder of this document assumes that all
221 operations are performed using this unprivileged account.
224 Install MediaGoblin and Virtualenv
225 ----------------------------------
227 We will now switch to our 'mediagoblin' system account, and then set up
228 our MediaGoblin source code repository and its necessary services.
229 You should modify these commands to suit your own environment.
231 Change to the MediaGoblin directory that you just created::
233 sudo su mediagoblin -s /bin/bash # to change to the 'mediagoblin' account
234 $ cd /srv/mediagoblin.example.org
236 Clone the MediaGoblin repository and set up the git submodules::
238 $ git clone git://git.savannah.gnu.org/mediagoblin.git -b stable
240 $ git submodule init && git submodule update
244 The MediaGoblin repository used to be on gitorious.org, but since
245 gitorious.org shut down, we had to move. We are presently on
246 Savannah. You may need to update your git repository location::
248 $ git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git
250 Set up the hacking environment::
252 $ ./bootstrap.sh && ./configure && make
254 (Note that if you'd prefer to run MediaGoblin with Python 3, pass in
255 `--with-python3` to the `./configure` command.)
257 Create and set the proper permissions on the ``user_dev`` directory.
258 This directory will be used to store uploaded media files::
260 $ mkdir user_dev && chmod 750 user_dev
262 Assuming you are going to deploy with FastCGI, you should also install
265 $ ./bin/easy_install flup
267 (Note, if you're running Python 2, which you probably are at this
268 point in MediaGoblin's development, you'll need to run:)
270 $ ./bin/easy_install flup==1.0.3.dev-20110405
272 The above provides an in-package install of ``virtualenv``. While this
273 is counter to the conventional ``virtualenv`` configuration, it is
274 more reliable and considerably easier to configure and illustrate. If
275 you're familiar with Python packaging you may consider deploying with
276 your preferred method.
280 What if you don't want an in-package ``virtualenv``? Maybe you
281 have your own ``virtualenv``, or you are building a MediaGoblin
282 package for a distribution. There's no need necessarily for the
283 virtualenv produced by ``./configure && make`` by default other
284 than attempting to simplify work for developers and people
285 deploying by hiding all the virtualenv and bower complexity.
287 If you want to install all of MediaGoblin's libraries
288 independently, that's totally fine! You can pass the flag
289 ``--without-virtualenv`` which will skip this step.
290 But you will need to install all those libraries manually and make
291 sure they are on your ``PYTHONPATH`` yourself! (You can still use
292 ``python setup.py develop`` to install some of those libraries,
293 but note that no ``./bin/python`` will be set up for you via this
294 method, since no virtualenv is set up for you!)
296 This concludes the initial configuration of the MediaGoblin
297 environment. In the future, when you update your
298 codebase, you should also run::
300 $ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate
304 Note: If you are running an active site, depending on your server
305 configuration, you may need to stop it first or the dbupdate command
306 may hang (and it's certainly a good idea to restart it after the
310 Deploy MediaGoblin Services
311 ---------------------------
313 Edit site configuration
314 ~~~~~~~~~~~~~~~~~~~~~~~
316 A few basic properties must be set before MediaGoblin will work. First
317 make a copy of ``mediagoblin.ini`` and ``paste.ini`` for editing so the original
318 config files aren't lost (you likely won't need to edit the paste configuration,
319 but we'll make a local copy of it just in case)::
321 $ cp -av mediagoblin.ini mediagoblin_local.ini && cp -av paste.ini paste_local.ini
323 Then edit mediagoblin_local.ini:
324 - Set ``email_sender_address`` to the address you wish to be used as
325 the sender for system-generated emails
326 - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if
327 your mediagoblin directory is not the root directory of your
331 Configure MediaGoblin to use the PostgreSQL database
332 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
334 If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your
335 ``mediagoblin_local.ini`` and put in::
337 sql_engine = postgresql:///mediagoblin
339 if you are running the MediaGoblin application as the same 'user' as the
343 Update database data structures
344 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
346 Before you start using the database, you need to run::
350 to populate the database with the MediaGoblin data structures.
356 At this point MediaGoblin should be properly installed. You can
357 test the deployment with the following command::
359 $ ./lazyserver.sh --server-name=broadcast
361 You should be able to connect to the machine on port 6543 in your
362 browser to confirm that the service is operable.
364 The next series of commands will need to be run as a privileged user. Type
365 exit to return to the root/sudo account.::
369 .. _webserver-config:
375 This configuration example will use Nginx, however, you may
376 use any webserver of your choice as long as it supports the FastCGI
377 protocol. If you do not already have a web server, consider Nginx, as
378 the configuration files may be more clear than the
381 Create a configuration file at
382 ``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link
383 into a directory that will be included in your ``nginx`` configuration
384 (e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with
385 one of the following commands.
387 On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and
388 derivatives) issue the following commands::
390 sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/
391 sudo systemctl enable nginx
393 On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the
396 sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/
397 sudo systemctl enable nginx
399 You can modify these commands and locations depending on your preferences and
400 the existing configuration of your Nginx instance. The contents of
401 this ``nginx.conf`` file should be modeled on the following::
404 #################################################
405 # Stock useful config options, but ignore them :)
406 #################################################
407 include /etc/nginx/mime.types;
410 default_type application/octet-stream;
415 gzip_min_length 1024;
417 gzip_types text/plain application/x-javascript text/javascript text/xml text/css;
419 #####################################
420 # Mounting MediaGoblin stuff
421 # This is the section you should read
422 #####################################
424 # Change this to update the upload size limit for your users
425 client_max_body_size 8m;
427 # prevent attacks (someone uploading a .txt file that the browser
428 # interprets as an HTML file, etc.)
429 add_header X-Content-Type-Options nosniff;
431 server_name mediagoblin.example.org www.mediagoblin.example.org;
432 access_log /var/log/nginx/mediagoblin.example.access.log;
433 error_log /var/log/nginx/mediagoblin.example.error.log;
435 # MediaGoblin's stock static files: CSS, JS, etc.
436 location /mgoblin_static/ {
437 alias /srv/mediagoblin.example.org/mediagoblin/mediagoblin/static/;
440 # Instance specific media:
441 location /mgoblin_media/ {
442 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/media/public/;
445 # Theme static files (usually symlinked in)
446 location /theme_static/ {
447 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/theme_static/;
450 # Plugin static files (usually symlinked in)
451 location /plugin_static/ {
452 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/;
455 # Mounting MediaGoblin itself via FastCGI.
457 fastcgi_pass 127.0.0.1:26543;
458 include /etc/nginx/fastcgi_params;
460 # our understanding vs Nginx's handling of script_name vs
461 # path_info don't match :)
462 fastcgi_param PATH_INFO $fastcgi_script_name;
463 fastcgi_param SCRIPT_NAME "";
467 The first four ``location`` directives instruct Nginx to serve the
468 static and uploaded files directly rather than through the MediaGoblin
469 process. This approach is faster and requires less memory.
473 The user who owns the Nginx process, normally ``www-data`` or ``nginx``,
474 requires execute permission on the directories ``static``,
475 ``public``, ``theme_static`` and ``plugin_static`` plus all their
476 parent directories. This user also requires read permission on all
477 the files within these directories. This is normally the default.
479 Nginx is now configured to serve the MediaGoblin application. Perform a quick
480 test to ensure that this configuration works::
484 If you encounter any errors, review your Nginx configuration files, and try to
485 resolve them. If you do not encounter any errors, you can start your Nginx
486 server with one of the following commands (depending on your environment)::
488 sudo /etc/init.d/nginx restart
489 sudo /etc/rc.d/nginx restart
490 sudo systemctl restart nginx
492 Now start MediaGoblin. Use the following command sequence as an
495 cd /srv/mediagoblin.example.org/mediagoblin/
496 su mediagoblin -s /bin/bash
497 ./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
499 Visit the site you've set up in your browser by visiting
500 <http://mediagoblin.example.org>. You should see MediaGoblin!
504 The configuration described above is sufficient for development and
505 smaller deployments. However, for larger production deployments
506 with larger processing requirements, see the
507 ":doc:`production-deployments`" documentation.
513 Instructions and scripts for running MediaGoblin on an Apache server
514 can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_.
517 Should I Keep Open Registration Enabled?
518 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
520 Unfortunately, in this current release of MediaGoblin we are suffering
521 from spammers registering to public instances en masse. As such, you
524 a) Disable registration on your instance and just make
525 accounts for people you know and trust (eg via the `gmg adduser`
526 command). You can disable registration in your mediagoblin.ini
530 allow_registration = false
532 b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA
533 plugins exist, for various reasons we do not have any general
534 recommendations we can make at this point.
536 We hope to have a better solution to this situation shortly. We
537 apologize for the inconvenience in the meanwhile.
540 Security Considerations
541 ~~~~~~~~~~~~~~~~~~~~~~~
545 The directory ``user_dev/crypto/`` contains some very
547 Especially the ``itsdangeroussecret.bin`` is very important
548 for session security. Make sure not to leak its contents anywhere.
549 If the contents gets leaked nevertheless, delete your file
550 and restart the server, so that it creates a new secret key.
551 All previous sessions will be invalidated.