1 .. MediaGoblin Documentation
3 Written in 2012 by MediaGoblin contributors
5 To the extent possible under law, the author(s) have dedicated all
6 copyright and related and neighboring rights to this software to
7 the public domain worldwide. This software is distributed without
10 You should have received a copy of the CC0 Public Domain
11 Dedication along with this software. If not, see
12 <http://creativecommons.org/publicdomain/zero/1.0/>.
20 This chapter has important information for releases in it.
21 If you're upgrading from a previous release, please read it
22 carefully, or at least skim over it.
26 ALWAYS do backups before upgrading, especially before
27 running migrations! That way if something goes wrong, we can fix
30 And be sure to shut down your current mediagoblin/celery processes
35 The MediaGoblin repository used to be on gitorious.org, but since
36 gitorious.org shut down, we had to move. We are presently on
37 Savannah. You may need to update your git repository location::
39 git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git
45 This release is a security and bugfix release. We recommend you upgrade as
48 **Do this to upgrade**
50 0. If you haven't already, switch the git remote URL:
51 ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git``
52 1. Update to the latest release. If checked out from git, run:
53 ``git fetch && git checkout -q v0.8.1``
55 ``./bootstrap.sh && ./configure && make``
57 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
59 (Please check intermediate release steps as well if not upgrading from
62 **Bugfixes/improvements:**
64 Most importantly, there is an **important security fix**:
66 Quoting here a portion of the
67 `release blogpost <http://mediagoblin.org/news/mediagoblin-0.8.1-security-release.html>`_::
69 We have had a security problem in our OAuth implementation reported to
70 us privately and have taken steps to address it. The security problem
71 affects all versions of GNU MediaGoblin since 0.5.0. I have created a patch
72 for this and released a minor version 0.8.1. It's strongly advised
73 that everyone upgrade as soon as they can.
75 In order to exploit the security issue, an attacker must have had
76 access to a logged in session to your GNU MediaGoblin account. If you
77 have kept your username and password secret, logging in only over
78 HTTPS and you've not left yourself logged in on publicly accessible
79 computers, you should be safe. However it's still advised all users
80 take the following precautions, listed below.
82 Users should check their authorized clients. Any client which looks
83 unfamiliar to you, you should deauthorize. To check this:
85 1) Log in to the GNU MediaGoblin instance
86 2) Click the drop down arrow in the upper right
87 3) Click "Change account settings"
88 4) At the bottom click the "Deauthorize applications" link
90 If you are unsure of any of these, click "Deauthorize".
92 There are other bugfixes, but they are fairly minor.
98 This release has a number of changes related to the way we recommend
99 building MediaGoblin; upgrade steps are below, but if you run into
100 trouble, consider pinging the MediaGoblin list or IRC channel.
102 **Do this to upgrade**
104 0. If you haven't already, switch the git remote URL:
105 ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git``
106 1. If you don't have node.js installed, you'll need it for handling
107 MediaGoblin's static web dependencies. Install this via your
108 distribution! (In the glorious future MediaGoblin will be simply
109 packaged for your distribution so you won't have to worry about
111 2. Update to the latest release. If checked out from git, run:
112 ``git fetch && git checkout -q v0.8.0``
114 ``./bootstrap.sh && ./configure && make``
116 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
118 Please note the important steps of 0 and 2, which have not appeared in
119 prior upgrade guides!
123 - Are you using audio or video media types? In that case, you'll need
124 to update your Gstreamer instance to 1.0.
125 - The Pump API needs some data passed through to the WSGI application,
126 so if you are using apache with mod_wsgi you should be sure to make
127 sure to add "WSGIPassAuthorization On" to your config. (Using the
128 default MediaGoblin documnetation and config, things should work
132 **Bugfixes/improvements:**
134 - Preliminary / experimental support for Python 3!
135 - Footer forced to the bottom of page
136 - Massive improvements to Pump API support
137 - Able to run on multiple existing Pump clients! Including Pumpa
139 - much cleaner ./configure && make support; it's now the default
140 - Clearer documentation on permissions and installation
141 - Switched from Transifex, which had become proprietary, to an
142 instance of Pootle hosted for GNU
143 - Moved to Gstreamer 1.0! This also adds a new thumbnailer which
144 gives much better results in
145 - Removed terrible check-javascript-dependencies-into-your-application
146 setup, now using Bower for dependency tracking
147 - Put some scaffolding in place for Alembic, which will be used for
148 future mitration work
149 - Automatically create a fresh mediagoblin.ini from
150 mediagoblin.ini.example
151 - no more need for mediagoblin_local.ini (though it's still supported)
152 - Fix lowercasing of username in auth steps
153 - Slowly moving towards removing global state (a source of many bugs)
158 This is a purely bugfix release. Important changes happened since
159 0.7.0; if running MediaGoblin 0.7.0, an upgrade is highly recommended;
160 see below. This release is especially useful if you have been running
161 postgres and have been receiving seemingly random database transaction
164 **Do this to upgrade**
166 1. Update to the latest release. If checked out from git, run:
167 ``git fetch && git checkout -q v0.7.1 && git submodule init && git submodule update``
169 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
171 That's it, probably! If you run into problems, don't hesitate to
172 `contact us <http://mediagoblin.org/pages/join.html>`_
175 **Bugfixes/improvements:**
177 - The *MOST IMPORTANT* change in this release:
178 Disabling a couple of non-critical features that were causing
179 database transaction issues. (These should be back by 0.8.0.)
181 + Disabled the "checking if the database is up to date at
182 mediagoblin startup" feature
183 + Disabled the garbage collection stuff by default for now
184 (You can set garbage_collection under the config mediagoblin
185 header to something other than 0 to turn it back on for now, but
186 it's potentially risky for the moment.)
188 - Some fixes to the 0.7.0 docs
189 - Fixed Sandy 70s speedboat navbar by updating git submodule
190 - Added support for cr2 files in raw_image media type
191 - Added a description to setup.py
192 - Collection and CollectionItem objects now have nicer in-python representations
193 - Fixed unicode error with raw image mediatype logging
194 - Fixed #945 "Host metadata does not confirm to spec (/.well-known/meta-data)"
196 + Add XRD+XML formatting for /.well-known/host-meta
197 + Add /.well-known/webfinger API to lookup user hrefs
199 - deleteuser gmg subcommand now fails gracefully
200 - Removed a false download link from setup.py
205 **Do this to upgrade**
207 1. Update to the latest release. If checked out from git, run:
208 ``git fetch && git checkout -q v0.7.0 && git submodule init && git submodule update``
210 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
212 (NOTE: earlier versions of the 0.7.0 release instructions left out the
213 ``git submodule init`` step! If you did an upgrade earlier based on
214 these instructions and your theme looks weirdly aligned, try running
217 ``git submodule init && git submodule update``
219 That's it, probably! If you run into problems, don't hesitate to
220 `contact us <http://mediagoblin.org/pages/join.html>`_
225 - New mobile upload API making use of the
226 `Pump API <https://github.com/e14n/pump.io/blob/master/API.md>`_
227 (which will be the foundation for MediaGoblin's federation)
228 - New theme: Sandy 70s Speedboat!
230 - Metadata features! We also now have a json-ld context.
232 - Many improvements for archival institutions, including metadata
233 support and featuring items on the homepage. With the (new!)
234 archivalook plugin enabled, featuring media is possible.
235 Additionally, metadata about the particular media item will show up
238 In the future these plugins may be separated, but for now they have
239 come together as part of the same plugin.
241 - There is a new gmg subcommand called batchaddmedia that allows for
242 uploading many files at once. This is aimed to be useful for
243 archival institutions and groups where there is an already existing
244 and large set of available media that needs to be included.
245 - Speaking of, the call to postgres in the makefile is fixed.
246 - We have a new, generic media-page context hook that allows for
247 adding context depending on the type of media.
248 - Tired of video thumbnails breaking during processing all the time?
249 Good news, everyone! Video thumbnail generation should not fail
250 frequently anymore. (We think...)
251 - You can now set default permissions for new users in the config.
253 - bootstrap.sh / gnu configuration stuff still exists, but moves to be
254 experimental-bootstrap.sh so as to not confuse newcomers. There are
255 some problems currently with the autoconf stuff that we need to work
256 out... we still have interest in supporting it, though help is
259 - MediaGoblin now checks whether or not the database is up to date
261 - Switched to `Skeleton <http://www.getskeleton.com/>`_ as a system for
263 - New gmg subcommands for administrators:
264 - A "deletemedia" command
265 - A "deleteuser" command
266 - We now have a blogging media type... it's very experimental,
267 however. Use with caution!
268 - We have switched to exifread as an external library for reading EXIF
269 data. It's basically the same thing as before, but packaged
270 separately from MediaGoblin.
271 - Many improvements to internationalization. Also (still rudimentary,
272 but existant!) RTL language support!
275 - The host-meta is now json by default; in the spec it should be xml by
276 default. We have done this because of compatibility with the pump
277 API. We are checking with upstream to see if there is a way to
278 resolve this discrepancy.
284 This is a short, bugfix release.
286 **Do this to upgrade**
288 1. Update to the latest release. If checked out from git, run:
289 ``git fetch && git checkout -q v0.6.1``
291 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
293 This release switches the default terms of service to be off by
294 default and corrects some mistakes in the default terms of service.
296 Turning the terms of service on is very easy, just set ``show_tos`` in
297 the ``[mediagoblin]`` section of your config to ``true``.
303 **Do this to upgrade**
305 1. Update to the latest release. If checked out from git, run:
306 ``git fetch && git checkout -q v0.6.0``
308 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
310 That's it, probably! If you run into problems, don't hesitate to
311 `contact us <http://mediagoblin.org/pages/join.html>`_
314 This tool has a lot of new tools for administrators, hence the
315 nickname "Lore of the Admin"!
319 - New tools to control how much users can upload, both as a general
320 user limit, or per file.
322 You can set this with the following options in your mediagoblin
323 config file: `upload_limit` and `max_file_size`. Both are integers
326 There is an option to control how much each individual user can
327 upload too, though an interface for this is not yet exposed. See
328 the "uploaded" field on the core__users table.
330 - MediaGoblin now contains an authentication plugin for ldap! You
331 can turn on the mediagoblin.plugins.ldap plugin to make use of
332 this. See the documentation: :ref:`ldap-plugin`
334 - There's a new command line upload tool! At long last! See
335 `./bin/gmg addmedia --help` for info on how to use this.
337 - There's now a terms of service document included in MediaGoblin.
338 It's turned on by default, but you can turn it off if you prefer,
339 just set the configuration option of `show_tos` in the [mediagoblin]
340 section of your config to false.
342 Alternately, you can override the template for the terms of service
343 document to set up your own.
345 - We have a lot of new administrative tooling features!
346 - There's a built-in privileges/permissions system now.
347 Administrators are given access to modifying these parameters
348 from a user administration panel.
349 - Users can submit reports about other problematic users or media
350 and administrators are given tools to resolve said reports and
351 ban/unban users if needed.
353 - New version of video.js is included with MediaGoblin. Slight
354 amount of skinning to match the MediaGoblin look, otherwise also
355 uses the new default skin.
357 Developer-oriented changes:
359 - New developer tool for quickly setting up a development environment
360 in `devtools/make_example_database.sh`. Requires doing a checkout
361 of our other tool `mg_dev_environments <https://gitorious.org/mediagoblin/mg-dev-environments/>`_
362 (probably in the parent Directory) though!
363 - A "foundations" framework has entered into the codebase.
364 This is mostly just relevant to coders, but it does mean that it's
365 much easier to add database structures that need some entries filled
366 automatically by default.
367 - Refactoring to the authentication code and the reprocessing code
373 v0.5.1 is a bugfix release... the steps are the same as for 0.5.1.
377 - python 2.6 compatibility restored
378 - Fixed last release's release notes ;)
384 **NOTE:** If using the API is important to you, we're in a state of
385 ransition towards a new API via the Pump API. As such, though the old
386 API still probably works, some changes have happened to the way oauth
387 works to make it more Pump-compatible. If you're heavily using
388 clients using the old API, you may wish to hold off on upgrading for
389 now. Otherwise, jump in and have fun! :)
391 **Do this to upgrade**
394 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
397 .. mention something about new, experimental configure && make support
399 2. Note that a couple of things have changed with ``mediagoblin.ini``. First
400 we have a new Authentication System. You need to add
401 ``[[mediagoblin.plugins.basic_auth]]`` under the ``[plugins]`` section of
402 your config file. Second, media types are now plugins, so you need to add
403 each media type under the ``[plugins]`` section of your config file.
406 3. We have made a script to transition your ``mediagoblin_local.ini`` file for
407 you. This script can be found at:
409 http://mediagoblin.org/download/0.5.0_config_converter.py
411 If you run into problems, don't hesitate to
412 `contact us <http://mediagoblin.org/pages/join.html>`_
417 * As mentioned above, we now have a plugable Authentication system. You can
418 use any combination of the multiple authentication systems
419 (:ref:`basic_auth-chapter`, :ref:`persona-chapter`, :ref:`openid-chapter`)
421 * Media types are now plugins! This means that new media types will
422 be able to do new, fancy things they couldn't in the future.
423 * We now have notification support! This allows you to subscribe to media
424 comments and to be notified when someone comments on your media.
425 * New reprocessing framework! You can now reprocess failed uploads, and
426 send already processed media back to processing to re-transcode or resize
429 * Users now have the ability to change their email associated with their
431 * New oauth code as we move closer to federation support.
432 * Experimental pyconfigure support for GNU-style configue and makefile
434 * Database foundations! You can now pre-populate the database models.
435 * Way faster unit test run-time via in-memory database.
436 * All mongokit stuff has been cleaned up.
437 * Fixes for non-ascii filenames.
438 * The option to stay logged in.
439 * Mediagoblin has been upgraded to use the latest `celery <http://celeryproject.org/>`_
441 * You can now add jinja2 extensions to your config file to use in custom
443 * Fixed video permission issues.
444 * Mediagoblin docs are now hosted with multiple versions.
445 * We removed redundent tooltips from the STL media display.
446 * We are now using itsdangerous for verification tokens.
452 This is a bugfix release for 0.4.0. This only implements one major
453 fix in the newly released document support which prevented the
454 "conversion via libreoffice" feature.
456 If you were running 0.4.0 you can upgrade to v0.4.1 via a simple
457 switch and restarting mediagoblin/celery with no other actions.
459 Otherwise, follow 0.4.0 instructions.
465 **Do this to upgrade**
468 ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
470 2. See "For Theme authors" if you have a custom theme.
471 3. Note that ``./bin/gmg theme assetlink`` is now just
472 ``./bin/gmg assetlink`` and covers both plugins and assets.
473 Keep on reading to hear more about new plugin features.
474 4. If you want to take advantage of new plugins that have statically
475 served assets, you are going to need to add the new "plugin_static"
476 section to your nginx config. Basically the following for nginx::
478 # Plugin static files (usually symlinked in)
479 location /plugin_static/ {
480 alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/;
483 Similarly, if you've got a modified paste config, you may want to
484 borrow the app:plugin_static section from the default paste.ini
486 5. We now use itsdangerous for sessions; if you had any references to
487 beaker in your paste config you can remove them. Again, see the
488 default paste.ini config
489 6. We also now use git submodules. Please do:
490 ``git submodule init && git submodule update``
491 You will need to do this to use the new PDF support.
493 **For theme authors**
495 If you have your own theme or you have any "user modified templates",
496 please note the following:
498 * mediagoblin/bits/ files above-content.html, body-end.html,
499 body-start.html now are renamed... they have underscores instead of
500 dashes in the filenames now :)
501 * There's a new file: ``mediagoblin/bits/frontpage_welcome.html``.
502 You can easily customize this to give a welcome page appropriate to
509 * Improved plugin system. More flexible, better documented, with a
510 new plugin authoring section of the docs.
511 * itsdangerous based sessions. No more beaker!
512 * New, experimental Piwigo-based API. This means you should be able
513 to use MediaGoblin with something like Shotwell. (Again, a word of
514 caution: this is *very experimental*!)
515 * Human readable timestamps, and the option to display the original
516 date of an image when available (available as the
517 "original_date_visible" variable)
518 * Moved unit testing system from nosetests to py.test so we can better
519 handle issues with sqlalchemy exploding with different database
520 configurations. Long story :)
521 * You can now disable the ability to post comments.
522 * Tags now can be up to length 255 characters by default.
528 **Do this to upgrade**
530 1. Make sure to run ``bin/gmg dbupdate`` after upgrading.
531 2. OpenStreetMap is now a plugin, so if you want to use it, add the
532 following to your config file:
537 [[mediagoblin.plugins.geolocation]]
539 If you have your own theme, you may need to make some adjustments to
540 it as some theme related things may have changed in this release. If
541 you run into problems, don't hesitate to
542 `contact us <http://mediagoblin.org/pages/join.html>`_
547 * New dropdown menu for accessing various features.
549 * Significantly improved URL generation. Now mediagoblin won't give
550 up on making a slug if it looks like there will be a duplicate;
551 it'll try extra hard to generate a meaningful one instead.
553 Similarly, linking to an id no longer can possibly conflict with
554 linking to a slug; /u/username/m/id:35/ is the kind of reference we
555 now use to linking to entries with ids. However, old links with
556 entries that linked to ids should work just fine with our migration.
557 The only urls that might break in this release are ones using colons
560 * New template hooks for plugin authoring.
562 * As a demonstration of new template hooks for plugin authoring,
563 openstreetmap support now moved to a plugin!
565 * Method to add media to collections switched from icon of paperclip
566 to button with "add to collection" text.
568 * Bug where videos often failed to produce a proper thumbnail fixed!
570 * Copying around files in MediaGoblin now much more efficient, doesn't
571 waste gobs of memory.
573 * Video transcoding now optional for videos that meet certain
574 criteria. By default, MediaGoblin will not transcode webm videos
575 that are smaller in resolution than the MediaGoblin defaults, and
576 MediaGoblin can also be configured to allow theora files to not be
579 * Per-user license preference option; always want your uploads to be
580 BY-SA and tired of changing that field? You can now set your
581 license preference in your user settings.
583 * Video player now responsive; better for mobile!
585 * You can now delete your account from the user preferences page if
590 * Plugin writers: Internal restructuring led to mediagoblin.db.sql* be
591 mediagoblin.db.* starting from 0.3.3
593 * Dependency list has been reduced not requiring the "webob" package anymore.
595 * And many small fixes/improvements, too numerous to list!
601 This will be the last release that is capable of converting from an earlier
602 MongoDB-based MediaGoblin instance to the newer SQL-based system.
604 **Do this to upgrade**
606 # directory of your mediagoblin install
607 cd /srv/mediagoblin.example.org
609 # copy source for this release
611 git checkout tags/v0.3.2
613 # perform any needed database updates
616 # restart your servers however you do that, e.g.,
617 sudo service mediagoblin-paster restart
618 sudo service mediagoblin-celeryd restart
623 * **3d model support!**
625 You can now upload STL and OBJ files and display them in
626 MediaGoblin. Requires a recent-ish Blender; for details see:
627 :ref:`deploying-chapter`
629 * **trim_whitespace**
631 We bundle the optional plugin trim_whitespace which reduces the size
632 of the delivered html output by reducing redundant whitespace.
634 See :ref:`core-plugin-section` for plugin documentation
638 It isn't well documented yet but we do have an API. There is an
639 `android application in progress <https://gitorious.org/mediagoblin/mediagoblin-android>`_
640 which makes use of it, and there are some demo applications between
641 `automgtic <https://github.com/jwandborg/automgtic>`_, an
642 automatic media uploader for your desktop
643 and `OMGMG <https://github.com/jwandborg/omgmg>`_, an example of
644 a web application hooking up to the API.
646 This is a plugin, so you have to enable it in your mediagoblin
647 config file by adding a section under [plugins] like::
650 [[mediagoblin.plugins.api]]
652 Note that the API works but is not nailed down... the way it is
653 called may change in future releases.
655 * **OAuth login support**
657 For applications that use OAuth to connect to the API.
659 This is a plugin, so you have to enable it in your mediagoblin
660 config file by adding a section under [plugins] like::
663 [[mediagoblin.plugins.oauth]]
667 We now have user-curated collections support. These are arbitrary
668 galleries that are customizable by users. You can add media to
669 these by clicking on the paperclip icon when logged in and looking
672 * **OpenStreetMap licensing display improvements**
674 More accurate display of OSM licensing, and less disruptive: you
675 click to "expand" the display of said licensing.
677 Geolocation is also now on by default.
679 * **Miscelaneous visual improvements**
681 We've made a number of small visual improvements including newer and
682 nicer looking thumbnails and improved checkbox placement.
689 **Do this to upgrade**
691 1. Make sure to run ``bin/gmg dbuptdate`` after upgrading.
693 2. If you set up your server config with an older version of
694 mediagoblin and the mediagoblin docs, it's possible you don't
695 have the "theme static files" alias, so double check to make
696 sure that section is there if you are having problems.
702 * **theming support**
704 MediaGoblin now also includes theming support, which you can
705 read about in the section :ref:`theming-chapter`.
709 MediaGoblin has a flatpages plugin allowing you to add pages that
710 are aren't media-related like "About this site...", "Terms of
713 See :ref:`core-plugin-section` for plugin documentation
719 This release has one important change. You need to act when
720 upgrading from a previous version!
722 This release changes the database system from MongoDB to
723 SQL(alchemy). If you want to setup a fresh instance, just
724 follow the instructions in the deployment chapter. If on
725 the other hand you want to continue to use one instance,
728 To convert your data from MongoDB to SQL(alchemy), you need
729 to follow these steps:
731 1. Make sure your MongoDB is still running and has your
732 data, it's needed for the conversion.
734 2. Configure the ``sql_engine`` URI in the config to represent
735 your target database (see: :ref:`deploying-chapter`)
737 3. You need an empty database.
739 4. Then run the following command::
741 bin/gmg [-cf mediagoblin_config.ini] convert_mongo_to_sql
743 5. Start your server and investigate.