+.. _hacking-howto:
+
===============
Hacking HOWTO
===============
-So you want to hack on GNU MediaGoblin
-======================================
-First thing to do is check out the Web site where we list all the
-project infrastructure including:
+So you want to hack on GNU MediaGoblin?
+=======================================
+
+First thing to do is check out the `Web site
+<http://mediagoblin.org/join/>`_ where we list all the project
+infrastructure including:
-* the mailing list
* the IRC channel
-* the bug tracker
+* the mailing list
+* the issue tracker
Additionally, we have information on how to get involved, who to talk
to, what needs to be worked on, and other things besides!
+Second thing to do is take a look at :ref:`codebase-chapter` where
+we've started documenting how GNU MediaGoblin is built and how to add
+new things.
-How to set up an environment for hacking
-========================================
+Third you'll need to :ref:`get the requirements
+<get-requirements-section>`.
-The following assumes you have these things installed:
+Fourth, you'll need to build a development environment. For this step, there are two options:
-1. virtualenv:
+1. :ref:`buildout and bootstrap <hacking-with-buildout>` (easier) OR
+2. :ref:`virtualenv <hacking-with-virtualenv>` (more flexible, but harder)
- http://pypi.python.org/pypi/virtualenv
+Pick one---don't do both!
-2. virtualenv wrapper:
- http://www.doughellmann.com/projects/virtualenvwrapper/
+.. _get-requirements-section:
-3. git:
+Getting requirements
+====================
- http://git-scm.com/
+First, you need to have the following installed before you can build
+an environment for hacking on GNU MediaGoblin:
+* Python 2.6 or 2.7 - http://www.python.org/
-Follow these steps:
+ You'll need Python as well as the dev files for building modules.
-1. clone the repository::
+* python-lxml - http://lxml.de/
+* git - http://git-scm.com/
+* MongoDB - http://www.mongodb.org/
- git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
+If you're running Debian GNU/Linux or a Debian-derived distribution
+such as Mint or Ubuntu, running the following should install these
+requirements::
-2. create a virtual environment::
+ sudo apt-get install mongodb git-core python python-dev \
+ python-lxml
- mkvirtualenv mediagoblin
+.. YouCanHelp::
-3. if that doesn't put you in the virtual environment you created,
- then do::
+ If you have instructions for other GNU/Linux distributions to set
+ up requirements, let us know!
- workon mediagoblin
-4. run::
+.. _hacking-with-buildout:
+
+How to set up and maintain an environment for hacking with buildout
+===================================================================
+
+.. Note::
+
+ Either follow the instructions in this section OR follow the ones
+ in :ref:`hacking-with-virtualenv`. But don't do both!
+
+
+**Requirements**
+
+No additional requirements.
+
+
+**Create a development environment**
+
+After installing the requirements, follow these steps:
+
+1. Clone the repository::
+
+ git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
+
+2. Bootstrap and run buildout::
+
+ cd mediagoblin
+ python bootstrap.py && ./bin/buildout
+
+
+That's it! Using this method, buildout should create a ``user_dev``
+directory, in which certain things will be stored (media, beaker
+session stuff, etc). You can change this, but for development
+purposes this default should be fine.
+
+
+**Updating for dependency changes**
+
+While hacking on GNU MediaGoblin over time, you'll eventually have to
+update your development environment because the dependencies have
+changed. To do that, run::
+
+ ./bin/buildout
+
+
+**Updating for code changes**
+
+You don't need to do anything---code changes are automatically
+available.
+
+
+**Deleting your buildout**
+
+At some point, you may want to delete your buildout. Perhaps it's to
+start over. Perhaps it's to test building development environments
+with buildout.
+
+To do this, do::
+
+ rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev
+
+Usually buildout works pretty great and is super easy, but if you get
+problems with python-dateutil conflicts on your system, you may need
+to use virtualenv instead.
- python setup.py develop
+.. _hacking-with-virtualenv:
-When you want to work on GNU MediaGoblin, make sure to enter your
-virtual environment::
+How to set up and maintain an environment for hacking with virtualenv
+=====================================================================
+
+.. Note::
+
+ Either follow the instructions in this section OR follow the ones
+ in :ref:`hacking-with-buildout`. But don't do both!
+
+
+**Requirements**
+
+* virtualenv: http://pypi.python.org/pypi/virtualenv
+* virtualenv wrapper:
+ http://www.doughellmann.com/projects/virtualenvwrapper/ (be sure to
+ read the `install instructions
+ <http://www.doughellmann.com/docs/virtualenvwrapper/install.html>`_)
+
+
+**Create a development environment**
+
+1. Clone the repository::
+
+ git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
+
+2. Create a virtual environment::
+
+ mkvirtualenv --no-site-packages mediagoblin
+
+3. If that doesn't put you in the virutal environment you just
+ created, then do::
+
+ workon mediagoblin
+
+4. Run::
+
+ python setup.py develop
+
+That's it!
+
+
+**Activating a virtual environment**
+
+When you want to work on GNU MediaGoblin, you need to activate the
+virtual environment like this::
workon mediagoblin
-Any changes you make to the code will show up in your virtual
-environment--there's no need to continuously run ``python setup.py
-develop``.
+
+**Deactivating a virtual environment**
+
+If you want to deactivate it, you can do this::
+
+ deactivate
+
+
+**Updating a virtual environment with dependency changes**
+
+1. Enter the virtual environment.
+
+2. Run::
+
+ python setup.py develop
+
+
+**Updating a virtual environment with code changes**
+
+You don't need to do anything---code changes are automatically
+available.
+
+
+**Deleting a virtual environment**
+
+At some point you may want to delete your virtual environment.
+Perhaps it's to start over. Perhaps it's so you can test building
+development environments with virtualenv.
+
+To do this, do::
+
+ rmvirtualenv mediagoblin
+
+
+Running the server
+==================
+
+If you did virtualenv, run::
+
+ paster serve mediagoblin.ini --reload
+
+If you did buildout, run::
+
+ ./bin/paster serve mediagoblin.ini --reload
+
+
+Running celeryd
+===============
+
+You need to do this if you want your media to process and actually
+show up. It's probably a good idea in development to have the web
+server (above) running in one terminal and celeryd in another window.
+
+If you did virtualenv, run::
+
+ CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery celeryd
+
+If you did buildout, run::
+
+ CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery ./bin/celeryd
Running the test suite
======================
-Run::
+If you did virtualenv, run::
+
+ nosetests
+
+If you did buildout, run::
+
+ ./bin/nosetests
+
+
+Troubleshooting
+===============
+
+pymongo.errors.AutoReconnect: could not find master/primary
+-----------------------------------------------------------
+
+If you see this::
+
+ pymongo.errors.AutoReconnect: could not find master/primary
+
+then make sure mongodb is installed and running.
+
+If it's installed, check the mongodb log. On my machine, that's ``/var/log/mongodb/mongodb.log``. If you see something like::
+
+ old lock file: /var/lib/mongodb/mongod.lock. probably means...
+
+Then delete the lock file and relaunch mongodb.
+
+
+Wiping your user data
+=====================
+
+.. Note::
+
+ Unless you're doing development and working on and testing creating
+ a new instance, you will probably never have to do this. Will
+ plans to do this work and thus he documented it.
+
+.. YouCanHelp::
+
+ If you're familiar with MongoDB, we'd love to get a `script that
+ removes all the GNU MediaGoblin data from an existing instance
+ <http://bugs.foocorp.net/issues/296>`_. Let us know!
+
+
+Quickstart for Django programmers
+=================================
+
+We're not using Django, but the codebase is very Django-like in its
+structure.
+
+* ``routing.py`` is like ``urls.py`` in Django
+* ``models.py`` has mongokit ORM definitions
+* ``views.py`` is where the views go
+
+We're using MongoDB. Basically, instead of a relational database with
+tables, you have a big JSON structure which acts a lot like a Python
+dict.
+
+
+.. YouCanHelp::
+
+ If there are other things that you think would help orient someone
+ new to GNU MediaGoblin but coming from Django, let us know!
+
+
+Bite-sized bugs to start with
+=============================
+
+**May 3rd, 2011**: We don't have a list of bite-sized bugs, yet, but
+this is important to us. If you're interested in things to work on,
+let us know on `the mailing list <http://mediagoblin.org/join/>`_ or
+on the `IRC channel <http://mediagoblin.org/join/>`_.
+
+
+Tips for people new to coding
+=============================
+
+Learning Python
+---------------
+
+GNU MediaGoblin is written using a programming language called `Python
+<http://python.org/>`_.
+
+There are two different incompatible iterations of Python which I'll
+refer to as Python 2 and Python 3. GNU MediaGoblin is written in
+Python 2 and requires Python 2.6 or 2.7. At some point, we might
+switch to Python 3, but that's a future thing.
+
+You can learn how to code in Python 2 from several excellent books
+that are freely available on the Internet:
+
+* `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_
+* `Dive Into Pyton <http://diveintopython.org/>`_
+* `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_
+* `A Byte of Python <http://www.swaroopch.com/notes/Python>`_
+
+These are all excellent texts.
+
+.. YouCanHelp::
+
+ If you know of other good quality Python tutorials and Python
+ tutorial videos, let us know!
+
+
+Learning Libraries GNU MediaGoblin uses
+---------------------------------------
+
+GNU MediaGoblin uses a variety of libraries in order to do what it
+does. These libraries are listed in the :ref:`codebase-chapter`
+along with links to the project Web sites and documentation for the
+libraries.
+
+There are a variety of Python-related conferences every year that have
+sessions covering many aspects of these libraries. You can find them
+at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_.
+
+.. [0] This is a shameless plug. Will Kahn-Greene runs Python Miro
+ Community.
+
+If you have questions or need help, find us on the mailing list and on
+IRC.
- python setup.py test
+.. _hacking-howto-git:
-Creating a new file
-===================
+Learning git
+------------
-All new files need to have license/copyright information.
+git is an interesting and very powerful tool. Like all powerful
+tools, it has a learning curve.
-The following kinds of files get the GNU AGPL header:
+If you're new to git, we highly recommend the following resources for
+getting the hang of it:
-* Python files
-* JavaScript files
-* templates
-* other files with code in them
+* `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub
+ intro to git
+* `Pro Git <http://progit.org/book/>`_ --- fantastic book
+* `Git casts <http://gitcasts.com/>`_ --- screencast covering git
+ usage
+* `Git Reference <http://gitref.org/>`_ --- Git reference that makes
+ it easier to get the hang of git if you're coming from other version
+ control systems
-The following files get a CC BY header:
-* CSS files
+Learning other utilities
+------------------------
-The following files don't get a header because that's hard, but are
-under the CC BY license:
+The `OpenHatch <http://openhatch.org/>`_ site has a series of
+`training missions <http://openhatch.org/missions/>`_ which are
+designed to help you learn how to use these tools.
-* image files
-* video files
+If you're new to tar, diff and patch, we highly recommend you sign up
+with OpenHatch and do the missions.