docs/hackinghowto.rst

   1 .. _hacking-howto:
   2
   3 ===============
   4  Hacking HOWTO
   5 ===============
   6
   7
   8 So you want to hack on GNU MediaGoblin?
   9 =======================================
  10
  11 First thing to do is check out the `Web site
  12 <http://mediagoblin.org/join/>`_ where we list all the project
  13 infrastructure including:
  14
  15 * the IRC channel
  16 * the mailing list
  17 * the issue tracker
  18
  19 Additionally, we have information on how to get involved, who to talk
  20 to, what needs to be worked on, and other things besides!
  21
  22 Second thing to do is take a look at :ref:`codebase-chapter` where
  23 we've started documenting how GNU MediaGoblin is built and how to add
  24 new things.
  25
  26 Third you'll need to :ref:`get the requirements
  27 <get-requirements-section>`.
  28
  29 Fourth, you'll need to build a development environment.  For this step, there are two options:
  30
  31 1. :ref:`buildout and bootstrap <hacking-with-buildout>` (easier) OR
  32 2. :ref:`virtualenv <hacking-with-virtualenv>` (more flexible, but harder)
  33
  34 Pick one---don't do both!
  35
  36
  37 .. _get-requirements-section:
  38
  39 Getting requirements
  40 ====================
  41
  42 First, you need to have the following installed before you can build
  43 an environment for hacking on GNU MediaGoblin:
  44
  45 * Python 2.6 or 2.7  - http://www.python.org/
  46
  47   You'll need Python as well as the dev files for building modules.
  48
  49 * python-lxml        - http://lxml.de/
  50 * git                - http://git-scm.com/
  51 * MongoDB            - http://www.mongodb.org/
  52
  53 If you're running Debian GNU/Linux or a Debian-derived distribution
  54 such as Mint or Ubuntu, running the following should install these
  55 requirements::
  56
  57     sudo apt-get install mongodb git-core python python-dev \
  58          python-lxml
  59
  60 .. YouCanHelp::
  61
  62    If you have instructions for other GNU/Linux distributions to set
  63    up requirements, let us know!
  64
  65
  66 .. _hacking-with-buildout:
  67
  68 How to set up and maintain an environment for hacking with buildout
  69 ===================================================================
  70
  71 .. Note::
  72
  73    Either follow the instructions in this section OR follow the ones
  74    in :ref:`hacking-with-virtualenv`.  But don't do both!
  75
  76
  77 **Requirements**
  78
  79 No additional requirements.
  80
  81
  82 **Create a development environment**
  83
  84 After installing the requirements, follow these steps:
  85
  86 1. Clone the repository::
  87
  88        git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
  89
  90 2. Bootstrap and run buildout::
  91
  92        cd mediagoblin
  93        python bootstrap.py && ./bin/buildout
  94
  95
  96 That's it!  Using this method, buildout should create a ``user_dev``
  97 directory, in which certain things will be stored (media, beaker
  98 session stuff, etc).  You can change this, but for development
  99 purposes this default should be fine.
 100
 101
 102 **Updating for dependency changes**
 103
 104 While hacking on GNU MediaGoblin over time, you'll eventually have to
 105 update your development environment because the dependencies have
 106 changed.  To do that, run::
 107
 108     ./bin/buildout
 109
 110
 111 **Updating for code changes**
 112
 113 You don't need to do anything---code changes are automatically
 114 available.
 115
 116
 117 **Deleting your buildout**
 118
 119 At some point, you may want to delete your buildout.  Perhaps it's to
 120 start over.  Perhaps it's to test building development environments
 121 with buildout.
 122
 123 To do this, do::
 124
 125     rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev
 126
 127 Usually buildout works pretty great and is super easy, but if you get
 128 problems with python-dateutil conflicts on your system, you may need
 129 to use virtualenv instead.
 130
 131
 132 .. _hacking-with-virtualenv:
 133
 134 How to set up and maintain an environment for hacking with virtualenv
 135 =====================================================================
 136
 137 .. Note::
 138
 139    Either follow the instructions in this section OR follow the ones
 140    in :ref:`hacking-with-buildout`.  But don't do both!
 141
 142
 143 **Requirements**
 144
 145 * virtualenv: http://pypi.python.org/pypi/virtualenv
 146 * virtualenv wrapper:
 147   http://www.doughellmann.com/projects/virtualenvwrapper/ (be sure to
 148   read the `install instructions
 149   <http://www.doughellmann.com/docs/virtualenvwrapper/install.html>`_)
 150
 151
 152 **Create a development environment**
 153
 154 1. Clone the repository::
 155
 156        git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
 157
 158 2. Create a virtual environment::
 159
 160        mkvirtualenv --no-site-packages mediagoblin
 161
 162 3. If that doesn't put you in the virutal environment you just
 163    created, then do::
 164
 165        workon mediagoblin
 166
 167 4. Run::
 168
 169        python setup.py develop
 170
 171 That's it!
 172
 173
 174 **Activating a virtual environment**
 175
 176 When you want to work on GNU MediaGoblin, you need to activate the
 177 virtual environment like this::
 178
 179     workon mediagoblin
 180
 181
 182 **Deactivating a virtual environment**
 183
 184 If you want to deactivate it, you can do this::
 185
 186     deactivate
 187
 188
 189 **Updating a virtual environment with dependency changes**
 190
 191 1. Enter the virtual environment.
 192
 193 2. Run::
 194
 195       python setup.py develop
 196
 197
 198 **Updating a virtual environment with code changes**
 199
 200 You don't need to do anything---code changes are automatically
 201 available.
 202
 203
 204 **Deleting a virtual environment**
 205
 206 At some point you may want to delete your virtual environment.
 207 Perhaps it's to start over.  Perhaps it's so you can test building
 208 development environments with virtualenv.
 209
 210 To do this, do::
 211
 212     rmvirtualenv mediagoblin
 213
 214
 215 Running the server
 216 ==================
 217
 218 If you did virtualenv, run::
 219
 220     paster serve mediagoblin.ini --reload
 221
 222 If you did buildout, run::
 223
 224     ./bin/paster serve mediagoblin.ini --reload
 225
 226
 227 Running celeryd
 228 ===============
 229
 230 You need to do this if you want your media to process and actually
 231 show up.  It's probably a good idea in development to have the web
 232 server (above) running in one terminal and celeryd in another window.
 233
 234 If you did virtualenv, run::
 235
 236     CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery celeryd
 237
 238 If you did buildout, run::
 239
 240     CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery ./bin/celeryd
 241
 242
 243 Running the test suite
 244 ======================
 245
 246 If you did virtualenv, run::
 247
 248     nosetests
 249
 250 If you did buildout, run::
 251
 252     ./bin/nosetests
 253
 254
 255 Troubleshooting
 256 ===============
 257
 258 pymongo.errors.AutoReconnect: could not find master/primary
 259 -----------------------------------------------------------
 260
 261 If you see this::
 262
 263     pymongo.errors.AutoReconnect: could not find master/primary
 264
 265 then make sure mongodb is installed and running.
 266
 267 If it's installed, check the mongodb log.  On my machine, that's ``/var/log/mongodb/mongodb.log``.  If you see something like::
 268
 269     old lock file: /var/lib/mongodb/mongod.lock.  probably means...
 270
 271 Then delete the lock file and relaunch mongodb.
 272
 273
 274 Wiping your user data
 275 =====================
 276
 277 .. Note::
 278
 279    Unless you're doing development and working on and testing creating
 280    a new instance, you will probably never have to do this.  Will
 281    plans to do this work and thus he documented it.
 282
 283 .. YouCanHelp::
 284
 285    If you're familiar with MongoDB, we'd love to get a `script that
 286    removes all the GNU MediaGoblin data from an existing instance
 287    <http://bugs.foocorp.net/issues/296>`_.  Let us know!
 288
 289
 290 Quickstart for Django programmers
 291 =================================
 292
 293 We're not using Django, but the codebase is very Django-like in its
 294 structure.
 295
 296 * ``routing.py`` is like ``urls.py`` in Django
 297 * ``models.py`` has mongokit ORM definitions
 298 * ``views.py`` is where the views go
 299
 300 We're using MongoDB.  Basically, instead of a relational database with
 301 tables, you have a big JSON structure which acts a lot like a Python
 302 dict.
 303
 304
 305 .. YouCanHelp::
 306
 307    If there are other things that you think would help orient someone
 308    new to GNU MediaGoblin but coming from Django, let us know!
 309
 310
 311 Bite-sized bugs to start with
 312 =============================
 313
 314 **May 3rd, 2011**:  We don't have a list of bite-sized bugs, yet, but
 315 this is important to us.  If you're interested in things to work on,
 316 let us know on `the mailing list <http://mediagoblin.org/join/>`_ or
 317 on the `IRC channel <http://mediagoblin.org/join/>`_.
 318
 319
 320 Tips for people new to coding
 321 =============================
 322
 323 Learning Python
 324 ---------------
 325
 326 GNU MediaGoblin is written using a programming language called `Python
 327 <http://python.org/>`_.
 328
 329 There are two different incompatible iterations of Python which I'll
 330 refer to as Python 2 and Python 3.  GNU MediaGoblin is written in
 331 Python 2 and requires Python 2.6 or 2.7.  At some point, we might
 332 switch to Python 3, but that's a future thing.
 333
 334 You can learn how to code in Python 2 from several excellent books
 335 that are freely available on the Internet:
 336
 337 * `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_
 338 * `Dive Into Pyton <http://diveintopython.org/>`_
 339 * `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_
 340 * `A Byte of Python <http://www.swaroopch.com/notes/Python>`_
 341
 342 These are all excellent texts.
 343
 344 .. YouCanHelp::
 345
 346    If you know of other good quality Python tutorials and Python
 347    tutorial videos, let us know!
 348
 349
 350 Learning Libraries GNU MediaGoblin uses
 351 ---------------------------------------
 352
 353 GNU MediaGoblin uses a variety of libraries in order to do what it
 354 does.  These libraries are listed in the :ref:`codebase-chapter`
 355 along with links to the project Web sites and documentation for the
 356 libraries.
 357
 358 There are a variety of Python-related conferences every year that have
 359 sessions covering many aspects of these libraries.  You can find them
 360 at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_.
 361
 362 .. [0] This is a shameless plug.  Will Kahn-Greene runs Python Miro
 363    Community.
 364
 365 If you have questions or need help, find us on the mailing list and on
 366 IRC.
 367
 368
 369 .. _hacking-howto-git:
 370
 371 Learning git
 372 ------------
 373
 374 git is an interesting and very powerful tool.  Like all powerful
 375 tools, it has a learning curve.
 376
 377 If you're new to git, we highly recommend the following resources for
 378 getting the hang of it:
 379
 380 * `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub
 381   intro to git
 382 * `Pro Git <http://progit.org/book/>`_ --- fantastic book
 383 * `Git casts <http://gitcasts.com/>`_ --- screencast covering git
 384   usage
 385 * `Git Reference <http://gitref.org/>`_ --- Git reference that makes
 386   it easier to get the hang of git if you're coming from other version
 387   control systems
 388
 389
 390 Learning other utilities
 391 ------------------------
 392
 393 The `OpenHatch <http://openhatch.org/>`_ site has a series of
 394 `training missions <http://openhatch.org/missions/>`_ which are
 395 designed to help you learn how to use these tools.
 396
 397 If you're new to tar, diff and patch, we highly recommend you sign up
 398 with OpenHatch and do the missions.