docs/hackinghowto.rst

   1 .. _hacking-howto:
   2
   3 ===============
   4  Hacking HOWTO
   5 ===============
   6
   7 .. contents:: Sections
   8    :local:
   9
  10
  11 So you want to hack on GNU MediaGoblin?
  12 =======================================
  13
  14 First thing to do is check out the `Web site
  15 <http://mediagoblin.org/join/>`_ where we list all the project
  16 infrastructure including:
  17
  18 * the IRC channel
  19 * the mailing list
  20 * the issue tracker
  21
  22 Additionally, we have information on how to get involved, who to talk
  23 to, what needs to be worked on, and other things besides!
  24
  25 Second thing to do is take a look at :ref:`codebase-chapter` where
  26 we've started documenting how GNU MediaGoblin is built and how to add
  27 new things.
  28
  29 Third you'll need to :ref:`get the requirements
  30 <get-requirements-section>`.
  31
  32 Fourth, you'll need to build a development environment.  We use buildout,
  33 but if you want to use virtualenv, there's a set of mediocre not-very-supported
  34 steps in the `wiki <https://gitorious.org/mediagoblin/pages/Home>`_.
  35
  36
  37 .. _get-requirements-section:
  38
  39 Getting requirements
  40 ====================
  41
  42 First, you need to have the following installed before you can build
  43 an environment for hacking on GNU MediaGoblin:
  44
  45 * Python 2.6 or 2.7  - http://www.python.org/
  46
  47   You'll need Python as well as the dev files for building modules.
  48
  49 * python-lxml        - http://lxml.de/
  50 * git                - http://git-scm.com/
  51 * MongoDB            - http://www.mongodb.org/
  52
  53 If you're running Debian GNU/Linux or a Debian-derived distribution
  54 such as Mint or Ubuntu, running the following should install these
  55 requirements::
  56
  57     sudo apt-get install mongodb git-core python python-dev \
  58          python-lxml
  59
  60 On Fedora::
  61
  62     yum install mongodb-server python-paste-deploy python-paste-script \
  63          git-core python python-devel python-lxml
  64
  65 .. YouCanHelp::
  66
  67    If you have instructions for other GNU/Linux distributions to set
  68    up requirements, let us know!
  69
  70
  71 .. _hacking-with-buildout:
  72
  73
  74 How to set up and maintain an environment for hacking with buildout
  75 ===================================================================
  76
  77 **Requirements**
  78
  79 No additional requirements.
  80
  81
  82 **Create a development environment**
  83
  84 After installing the requirements, follow these steps:
  85
  86 1. Clone the repository::
  87
  88        git clone git://gitorious.org/mediagoblin/mediagoblin.git
  89
  90 2. Bootstrap and run buildout::
  91
  92        cd mediagoblin
  93        python bootstrap.py && ./bin/buildout
  94
  95
  96 That's it!  Using this method, buildout should create a ``user_dev``
  97 directory, in which certain things will be stored (media, beaker
  98 session stuff, etc).  You can change this, but for development
  99 purposes this default should be fine.
 100
 101
 102 **Updating for dependency changes**
 103
 104 While hacking on GNU MediaGoblin over time, you'll eventually have to
 105 update your development environment because the dependencies have
 106 changed.  To do that, run::
 107
 108     ./bin/buildout && ./bin/gmg migrate
 109
 110
 111 **Updating for code changes**
 112
 113 You don't need to do anything---code changes are automatically
 114 available.
 115
 116
 117 **Deleting your buildout**
 118
 119 At some point, you may want to delete your buildout.  Perhaps it's to
 120 start over.  Perhaps it's to test building development environments
 121 with buildout.
 122
 123 To do this, do::
 124
 125     rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev
 126
 127
 128 Running the server
 129 ==================
 130
 131 If you want to get things running quickly and without hassle, just
 132 run::
 133
 134     ./lazyserver.sh
 135
 136 This will start up a python server where you can begin playing with
 137 mediagoblin.  It will also run celery in "always eager" mode so you
 138 don't have to start a separate process for it.
 139
 140 This is fine in development, but if you want to actually run celery
 141 separately for testing (or deployment purposes), you'll want to run
 142 the server independently::
 143
 144     ./bin/paster serve paste.ini --reload
 145
 146
 147 Running celeryd
 148 ===============
 149
 150 If you aren't using ./lazyserver.sh or otherwise aren't running celery
 151 in always eager mode, you'll need to do this if you want your media to
 152 process and actually show up.  It's probably a good idea in
 153 development to have the web server (above) running in one terminal and
 154 celeryd in another window.
 155
 156 Run::
 157
 158     CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd
 159
 160
 161 Running the test suite
 162 ======================
 163
 164 Run::
 165
 166     ./runtests.sh
 167
 168
 169 Running a shell
 170 ===============
 171
 172 If you want a shell with your database pre-setup and an instantiated
 173 application ready and at your fingertips....
 174
 175 Run::
 176
 177     ./bin/gmg shell
 178
 179
 180 Troubleshooting
 181 ===============
 182
 183 pymongo.errors.AutoReconnect: could not find master/primary
 184 -----------------------------------------------------------
 185
 186 If you see this::
 187
 188     pymongo.errors.AutoReconnect: could not find master/primary
 189
 190 then make sure mongodb is installed and running.
 191
 192 If it's installed, check the mongodb log.  On my machine, that's
 193 ``/var/log/mongodb/mongodb.log``.  If you see something like::
 194
 195     old lock file: /var/lib/mongodb/mongod.lock.  probably means...
 196
 197 in that case you might have had an unclean shutdown.  Try::
 198
 199     sudo mongod --repair
 200
 201 If that didn't work, just delete the lock file and relaunch mongodb.
 202
 203 Anyway, then start the mongodb server in whatever way is appropriate
 204 for your distro / OS.
 205
 206
 207 pkg_resources.DistributionNotFound: distribute
 208 ----------------------------------------------
 209
 210 If you get this while running buildout::
 211
 212    pkg_resources.DistributionNotFound: distribute
 213
 214 Try this commmand instead::
 215
 216    python bootstrap.py --distribute && ./bin/buildout
 217
 218
 219 Wiping your user data
 220 =====================
 221
 222 .. Note::
 223
 224    Unless you're doing development and working on and testing creating
 225    a new instance, you will probably never have to do this.  Will
 226    plans to do this work and thus he documented it.
 227
 228 .. YouCanHelp::
 229
 230    If you're familiar with MongoDB, we'd love to get a `script that
 231    removes all the GNU MediaGoblin data from an existing instance
 232    <http://bugs.foocorp.net/issues/296>`_.  Let us know!
 233
 234
 235 Quickstart for Django programmers
 236 =================================
 237
 238 We're not using Django, but the codebase is very Django-like in its
 239 structure.
 240
 241 * ``routing.py`` is like ``urls.py`` in Django
 242 * ``models.py`` has mongokit ORM definitions
 243 * ``views.py`` is where the views go
 244
 245 We're using MongoDB.  Basically, instead of a relational database with
 246 tables, you have a big JSON structure which acts a lot like a Python
 247 dict.
 248
 249
 250 .. YouCanHelp::
 251
 252    If there are other things that you think would help orient someone
 253    new to GNU MediaGoblin but coming from Django, let us know!
 254
 255
 256 Bite-sized bugs to start with
 257 =============================
 258
 259 **May 3rd, 2011**:  We don't have a list of bite-sized bugs, yet, but
 260 this is important to us.  If you're interested in things to work on,
 261 let us know on `the mailing list <http://mediagoblin.org/join/>`_ or
 262 on the `IRC channel <http://mediagoblin.org/join/>`_.
 263
 264
 265 Tips for people new to coding
 266 =============================
 267
 268 Learning Python
 269 ---------------
 270
 271 GNU MediaGoblin is written using a programming language called `Python
 272 <http://python.org/>`_.
 273
 274 There are two different incompatible iterations of Python which I'll
 275 refer to as Python 2 and Python 3.  GNU MediaGoblin is written in
 276 Python 2 and requires Python 2.6 or 2.7.  At some point, we might
 277 switch to Python 3, but that's a future thing.
 278
 279 You can learn how to code in Python 2 from several excellent books
 280 that are freely available on the Internet:
 281
 282 * `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_
 283 * `Dive Into Pyton <http://diveintopython.org/>`_
 284 * `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_
 285 * `A Byte of Python <http://www.swaroopch.com/notes/Python>`_
 286
 287 These are all excellent texts.
 288
 289 .. YouCanHelp::
 290
 291    If you know of other good quality Python tutorials and Python
 292    tutorial videos, let us know!
 293
 294
 295 Learning Libraries GNU MediaGoblin uses
 296 ---------------------------------------
 297
 298 GNU MediaGoblin uses a variety of libraries in order to do what it
 299 does.  These libraries are listed in the :ref:`codebase-chapter`
 300 along with links to the project Web sites and documentation for the
 301 libraries.
 302
 303 There are a variety of Python-related conferences every year that have
 304 sessions covering many aspects of these libraries.  You can find them
 305 at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_.
 306
 307 .. [0] This is a shameless plug.  Will Kahn-Greene runs Python Miro
 308    Community.
 309
 310 If you have questions or need help, find us on the mailing list and on
 311 IRC.
 312
 313
 314 .. _hacking-howto-git:
 315
 316 Learning git
 317 ------------
 318
 319 git is an interesting and very powerful tool.  Like all powerful
 320 tools, it has a learning curve.
 321
 322 If you're new to git, we highly recommend the following resources for
 323 getting the hang of it:
 324
 325 * `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub
 326   intro to git
 327 * `Pro Git <http://progit.org/book/>`_ --- fantastic book
 328 * `Git casts <http://gitcasts.com/>`_ --- screencast covering git
 329   usage
 330 * `Git Reference <http://gitref.org/>`_ --- Git reference that makes
 331   it easier to get the hang of git if you're coming from other version
 332   control systems
 333
 334 There's also a git mission at `OpenHatch <http://openhatch.org/>`_.
 335
 336
 337 Learning other utilities
 338 ------------------------
 339
 340 The `OpenHatch <http://openhatch.org/>`_ site has a series of
 341 `training missions <http://openhatch.org/missions/>`_ which are
 342 designed to help you learn how to use these tools.
 343
 344 If you're new to tar, diff, patch and git, we highly recommend you sign
 345 up with OpenHatch and do the missions.