docs/hackinghowto.rst

   1 .. _hacking-howto:
   2
   3 ===============
   4  Hacking HOWTO
   5 ===============
   6
   7 .. contents:: Sections
   8    :local:
   9
  10
  11 So you want to hack on GNU MediaGoblin?
  12 =======================================
  13
  14 First thing to do is check out the `Web site
  15 <http://mediagoblin.org/join/>`_ where we list all the project
  16 infrastructure including:
  17
  18 * the IRC channel
  19 * the mailing list
  20 * the issue tracker
  21
  22 Additionally, we have information on how to get involved, who to talk
  23 to, what needs to be worked on, and other things besides!
  24
  25 Second thing to do is take a look at :ref:`codebase-chapter` where
  26 we've started documenting how GNU MediaGoblin is built and how to add
  27 new things.
  28
  29 Third you'll need to :ref:`get the requirements
  30 <get-requirements-section>`.
  31
  32 Fourth, you'll need to build a development environment.  We use buildout,
  33 but if you want to use virtualenv, there's a set of mediocre not-very-supported
  34 steps in the `wiki <https://gitorious.org/mediagoblin/pages/Home>`_.
  35
  36
  37 .. _get-requirements-section:
  38
  39 Getting requirements
  40 ====================
  41
  42 First, you need to have the following installed before you can build
  43 an environment for hacking on GNU MediaGoblin:
  44
  45 * Python 2.6 or 2.7  - http://www.python.org/
  46
  47   You'll need Python as well as the dev files for building modules.
  48
  49 * python-lxml        - http://lxml.de/
  50 * git                - http://git-scm.com/
  51 * MongoDB            - http://www.mongodb.org/
  52
  53 If you're running Debian GNU/Linux or a Debian-derived distribution
  54 such as Mint or Ubuntu, running the following should install these
  55 requirements::
  56
  57     sudo apt-get install mongodb git-core python python-dev \
  58          python-lxml
  59
  60 .. YouCanHelp::
  61
  62    If you have instructions for other GNU/Linux distributions to set
  63    up requirements, let us know!
  64
  65
  66 .. _hacking-with-buildout:
  67
  68
  69 How to set up and maintain an environment for hacking with buildout
  70 ===================================================================
  71
  72 **Requirements**
  73
  74 No additional requirements.
  75
  76
  77 **Create a development environment**
  78
  79 After installing the requirements, follow these steps:
  80
  81 1. Clone the repository::
  82
  83        git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
  84
  85 2. Bootstrap and run buildout::
  86
  87        cd mediagoblin
  88        python bootstrap.py && ./bin/buildout
  89
  90
  91 That's it!  Using this method, buildout should create a ``user_dev``
  92 directory, in which certain things will be stored (media, beaker
  93 session stuff, etc).  You can change this, but for development
  94 purposes this default should be fine.
  95
  96
  97 **Updating for dependency changes**
  98
  99 While hacking on GNU MediaGoblin over time, you'll eventually have to
 100 update your development environment because the dependencies have
 101 changed.  To do that, run::
 102
 103     ./bin/buildout && ./bin/gmg migrate
 104
 105
 106 **Updating for code changes**
 107
 108 You don't need to do anything---code changes are automatically
 109 available.
 110
 111
 112 **Deleting your buildout**
 113
 114 At some point, you may want to delete your buildout.  Perhaps it's to
 115 start over.  Perhaps it's to test building development environments
 116 with buildout.
 117
 118 To do this, do::
 119
 120     rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev
 121
 122
 123 Running the server
 124 ==================
 125
 126 If you want to get things running quickly and without hassle, just
 127 run::
 128
 129     ./lazyserver.sh
 130
 131 This will start up a python server where you can begin playing with
 132 mediagoblin.  It will also run celery in "always eager" mode so you
 133 don't have to start a separate process for it.
 134
 135 This is fine in development, but if you want to actually run celery
 136 separately for testing (or deployment purposes), you'll want to run
 137 the server independently::
 138
 139     ./bin/paster serve server.ini --reload
 140
 141
 142 Running celeryd
 143 ===============
 144
 145 If you aren't using ./lazyserver.sh or otherwise aren't running celery
 146 in always eager mode, you'll need to do this if you want your media to
 147 process and actually show up.  It's probably a good idea in
 148 development to have the web server (above) running in one terminal and
 149 celeryd in another window.
 150
 151 Run::
 152
 153     CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery ./bin/celeryd
 154
 155
 156 Running the test suite
 157 ======================
 158
 159 Run::
 160
 161     ./runtests.sh
 162
 163
 164 Running a shell
 165 ===============
 166
 167 If you want a shell with your database pre-setup and an instantiated
 168 application ready and at your fingertips....
 169
 170 Run::
 171
 172     ./bin/gmg shell
 173
 174
 175 Troubleshooting
 176 ===============
 177
 178 pymongo.errors.AutoReconnect: could not find master/primary
 179 -----------------------------------------------------------
 180
 181 If you see this::
 182
 183     pymongo.errors.AutoReconnect: could not find master/primary
 184
 185 then make sure mongodb is installed and running.
 186
 187 If it's installed, check the mongodb log.  On my machine, that's
 188 ``/var/log/mongodb/mongodb.log``.  If you see something like::
 189
 190     old lock file: /var/lib/mongodb/mongod.lock.  probably means...
 191
 192 Then delete the lock file and relaunch mongodb.
 193
 194
 195 Wiping your user data
 196 =====================
 197
 198 .. Note::
 199
 200    Unless you're doing development and working on and testing creating
 201    a new instance, you will probably never have to do this.  Will
 202    plans to do this work and thus he documented it.
 203
 204 .. YouCanHelp::
 205
 206    If you're familiar with MongoDB, we'd love to get a `script that
 207    removes all the GNU MediaGoblin data from an existing instance
 208    <http://bugs.foocorp.net/issues/296>`_.  Let us know!
 209
 210
 211 Quickstart for Django programmers
 212 =================================
 213
 214 We're not using Django, but the codebase is very Django-like in its
 215 structure.
 216
 217 * ``routing.py`` is like ``urls.py`` in Django
 218 * ``models.py`` has mongokit ORM definitions
 219 * ``views.py`` is where the views go
 220
 221 We're using MongoDB.  Basically, instead of a relational database with
 222 tables, you have a big JSON structure which acts a lot like a Python
 223 dict.
 224
 225
 226 .. YouCanHelp::
 227
 228    If there are other things that you think would help orient someone
 229    new to GNU MediaGoblin but coming from Django, let us know!
 230
 231
 232 Bite-sized bugs to start with
 233 =============================
 234
 235 **May 3rd, 2011**:  We don't have a list of bite-sized bugs, yet, but
 236 this is important to us.  If you're interested in things to work on,
 237 let us know on `the mailing list <http://mediagoblin.org/join/>`_ or
 238 on the `IRC channel <http://mediagoblin.org/join/>`_.
 239
 240
 241 Tips for people new to coding
 242 =============================
 243
 244 Learning Python
 245 ---------------
 246
 247 GNU MediaGoblin is written using a programming language called `Python
 248 <http://python.org/>`_.
 249
 250 There are two different incompatible iterations of Python which I'll
 251 refer to as Python 2 and Python 3.  GNU MediaGoblin is written in
 252 Python 2 and requires Python 2.6 or 2.7.  At some point, we might
 253 switch to Python 3, but that's a future thing.
 254
 255 You can learn how to code in Python 2 from several excellent books
 256 that are freely available on the Internet:
 257
 258 * `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_
 259 * `Dive Into Pyton <http://diveintopython.org/>`_
 260 * `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_
 261 * `A Byte of Python <http://www.swaroopch.com/notes/Python>`_
 262
 263 These are all excellent texts.
 264
 265 .. YouCanHelp::
 266
 267    If you know of other good quality Python tutorials and Python
 268    tutorial videos, let us know!
 269
 270
 271 Learning Libraries GNU MediaGoblin uses
 272 ---------------------------------------
 273
 274 GNU MediaGoblin uses a variety of libraries in order to do what it
 275 does.  These libraries are listed in the :ref:`codebase-chapter`
 276 along with links to the project Web sites and documentation for the
 277 libraries.
 278
 279 There are a variety of Python-related conferences every year that have
 280 sessions covering many aspects of these libraries.  You can find them
 281 at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_.
 282
 283 .. [0] This is a shameless plug.  Will Kahn-Greene runs Python Miro
 284    Community.
 285
 286 If you have questions or need help, find us on the mailing list and on
 287 IRC.
 288
 289
 290 .. _hacking-howto-git:
 291
 292 Learning git
 293 ------------
 294
 295 git is an interesting and very powerful tool.  Like all powerful
 296 tools, it has a learning curve.
 297
 298 If you're new to git, we highly recommend the following resources for
 299 getting the hang of it:
 300
 301 * `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub
 302   intro to git
 303 * `Pro Git <http://progit.org/book/>`_ --- fantastic book
 304 * `Git casts <http://gitcasts.com/>`_ --- screencast covering git
 305   usage
 306 * `Git Reference <http://gitref.org/>`_ --- Git reference that makes
 307   it easier to get the hang of git if you're coming from other version
 308   control systems
 309
 310 There's also a git mission at `OpenHatch <http://openhatch.org/>`_.
 311
 312
 313 Learning other utilities
 314 ------------------------
 315
 316 The `OpenHatch <http://openhatch.org/>`_ site has a series of
 317 `training missions <http://openhatch.org/missions/>`_ which are
 318 designed to help you learn how to use these tools.
 319
 320 If you're new to tar, diff, patch and git, we highly recommend you sign
 321 up with OpenHatch and do the missions.