docs/hackinghowto.rst

   1 .. _hacking-howto:
   2
   3 ===============
   4  Hacking HOWTO
   5 ===============
   6
   7
   8 So you want to hack on GNU MediaGoblin?
   9 =======================================
  10
  11 First thing to do is check out the `Web site
  12 <http://mediagoblin.org/join/>`_ where we list all the project
  13 infrastructure including:
  14
  15 * the IRC channel
  16 * the mailing list
  17 * the issue tracker
  18
  19 Additionally, we have information on how to get involved, who to talk
  20 to, what needs to be worked on, and other things besides!
  21
  22 Second thing to do is take a look at :ref:`codebase-chapter` where
  23 we've started documenting how GNU MediaGoblin is built and how to add
  24 new things.
  25
  26 Third you'll need to :ref:`get the requirements
  27 <get-requirements-section>`.
  28
  29 Fourth, you'll need to build a development environment.  For this step, there are two options:
  30
  31 1. :ref:`virtualenv <hacking-with-virtualenv>` OR
  32 2. :ref:`buildout and bootstrap <hacking-with-buildout>`
  33
  34 Pick one---don't do both!
  35
  36
  37 .. _get-requirements-section:
  38
  39 Getting requirements
  40 ====================
  41
  42 First, you need to have the following installed before you can build
  43 an environment for hacking on GNU MediaGoblin:
  44
  45 * Python 2.6 or 2.7  - http://www.python.org/
  46
  47   You'll need Python as well as the dev files for building modules.
  48
  49 * python-lxml        - http://lxml.de/
  50 * git                - http://git-scm.com/
  51 * MongoDB            - http://www.mongodb.org/
  52
  53 If you're running Debian GNU/Linux or a Debian-derived distribution
  54 such as Mint or Ubuntu, running the following should install these
  55 requirements::
  56
  57     sudo apt-get install mongodb git-core python python-dev \
  58          python-lxml
  59
  60 .. YouCanHelp::
  61
  62    If you have instructions for other GNU/Linux distributions to set
  63    up requirements, let us know!
  64
  65
  66 .. _hacking-with-virtualenv:
  67
  68 How to set up and maintain an environment for hacking with virtualenv
  69 =====================================================================
  70
  71 .. Note::
  72
  73    Either follow the instructions in this section OR follow the ones
  74    in :ref:`hacking-with-buildout`.  But don't do both!
  75
  76
  77 **Requirements**
  78
  79 * virtualenv: http://pypi.python.org/pypi/virtualenv
  80 * virtualenv wrapper:
  81   http://www.doughellmann.com/projects/virtualenvwrapper/ (be sure to
  82   read the `install instructions
  83   <http://www.doughellmann.com/docs/virtualenvwrapper/install.html>`_)
  84
  85
  86 **Create a development environment**
  87
  88 1. Clone the repository::
  89
  90        git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
  91
  92 2. Create a virtual environment::
  93
  94        mkvirtualenv --no-site-packages mediagoblin
  95
  96 3. If that doesn't put you in the virutal environment you just
  97    created, then do::
  98
  99        workon mediagoblin
 100
 101 4. Run::
 102
 103        python setup.py develop
 104
 105 That's it!
 106
 107
 108 **Activating a virtual environment**
 109
 110 When you want to work on GNU MediaGoblin, you need to activate the
 111 virtual environment like this::
 112
 113     workon mediagoblin
 114
 115
 116 **Deactivating a virtual environment**
 117
 118 If you want to deactivate it, you can do this::
 119
 120     deactivate
 121
 122
 123 **Updating a virtual environment with dependency changes**
 124
 125 1. Enter the virtual environment.
 126
 127 2. Run::
 128
 129       python setup.py develop
 130
 131
 132 **Updating a virtual environment with code changes**
 133
 134 You don't need to do anything---code changes are automatically
 135 available.
 136
 137
 138 **Deleting a virtual environment**
 139
 140 At some point you may want to delete your virtual environment.
 141 Perhaps it's to start over.  Perhaps it's so you can test building
 142 development environments with virtualenv.
 143
 144 To do this, do::
 145
 146     rmvirtualenv mediagoblin
 147
 148
 149 .. _hacking-with-buildout:
 150
 151 How to set up and maintain an environment for hacking with buildout
 152 ===================================================================
 153
 154 .. Note::
 155
 156    Either follow the instructions in this section OR follow the ones
 157    in :ref:`hacking-with-virtualenv`.  But don't do both!
 158
 159
 160 **Requirements**
 161
 162 No additional requirements.
 163
 164
 165 **Create a development environment**
 166
 167 After installing the requirements, follow these steps:
 168
 169 1. Clone the repository::
 170
 171        git clone http://git.gitorious.org/mediagoblin/mediagoblin.git
 172
 173 2. Bootstrap and run buildout::
 174
 175        cd mediagoblin
 176        python bootstrap.py && ./bin/buildout
 177
 178
 179 That's it!  Using this method, buildout should create a ``user_dev``
 180 directory, in which certain things will be stored (media, beaker
 181 session stuff, etc).  You can change this, but for development
 182 purposes this default should be fine.
 183
 184
 185 **Updating for dependency changes**
 186
 187 While hacking on GNU MediaGoblin over time, you'll eventually have to
 188 update your development environment because the dependencies have
 189 changed.  To do that, run::
 190
 191     ./bin/buildout
 192
 193
 194 **Updating for code changes**
 195
 196 You don't need to do anything---code changes are automatically
 197 available.
 198
 199
 200 **Deleting your buildout**
 201
 202 At some point, you may want to delete your buildout.  Perhaps it's to
 203 start over.  Perhaps it's to test building development environments
 204 with buildout.
 205
 206 To do this, do::
 207
 208     rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev
 209
 210
 211 Running the server
 212 ==================
 213
 214 If you did virtualenv, run::
 215
 216     paster serve mediagoblin.ini --reload
 217
 218 If you did buildout, run::
 219
 220     ./bin/paster serve mediagoblin.ini --reload
 221
 222
 223 Running celeryd
 224 ===============
 225
 226 You need to do this if you want your media to process and actually
 227 show up.  It's probably a good idea in development to have the web
 228 server (above) running in one terminal and celeryd in another window.
 229
 230 If you did virtualenv, run::
 231
 232     CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery celeryd
 233
 234 If you did buildout, run::
 235
 236     CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery ./bin/celeryd
 237
 238
 239 Running the test suite
 240 ======================
 241
 242 If you did virtualenv, run::
 243
 244     nosetests
 245
 246 If you did buildout, run::
 247
 248     ./bin/nosetests
 249
 250
 251 Troubleshooting
 252 ===============
 253
 254 pymongo.errors.AutoReconnect: could not find master/primary
 255 -----------------------------------------------------------
 256
 257 If you see this::
 258
 259     pymongo.errors.AutoReconnect: could not find master/primary
 260
 261 then make sure mongodb is installed and running.
 262
 263 If it's installed, check the mongodb log.  On my machine, that's ``/var/log/mongodb/mongodb.log``.  If you see something like::
 264
 265     old lock file: /var/lib/mongodb/mongod.lock.  probably means...
 266
 267 Then delete the lock file and relaunch mongodb.
 268
 269
 270 Wiping your user data
 271 =====================
 272
 273 .. Note::
 274
 275    Unless you're doing development and working on and testing creating
 276    a new instance, you will probably never have to do this.  Will
 277    plans to do this work and thus he documented it.
 278
 279 .. YouCanHelp::
 280
 281    If you're familiar with MongoDB, we'd love to get a `script that
 282    removes all the GNU MediaGoblin data from an existing instance
 283    <http://bugs.foocorp.net/issues/296>`_.  Let us know!
 284
 285
 286 Quickstart for Django programmers
 287 =================================
 288
 289 We're not using Django, but the codebase is very Django-like in its
 290 structure.
 291
 292 * ``routing.py`` is like ``urls.py`` in Django
 293 * ``models.py`` has mongokit ORM definitions
 294 * ``views.py`` is where the views go
 295
 296 We're using MongoDB.  Basically, instead of a relational database with
 297 tables, you have a big JSON structure which acts a lot like a Python
 298 dict.
 299
 300
 301 .. YouCanHelp::
 302
 303    If there are other things that you think would help orient someone
 304    new to GNU MediaGoblin but coming from Django, let us know!
 305
 306
 307 Bite-sized bugs to start with
 308 =============================
 309
 310 **May 3rd, 2011**:  We don't have a list of bite-sized bugs, yet, but
 311 this is important to us.  If you're interested in things to work on,
 312 let us know on `the mailing list <http://mediagoblin.org/join/>`_ or
 313 on the `IRC channel <http://mediagoblin.org/join/>`_.
 314
 315
 316 Tips for people new to coding
 317 =============================
 318
 319 Learning Python
 320 ---------------
 321
 322 GNU MediaGoblin is written using a programming language called `Python
 323 <http://python.org/>`_.
 324
 325 There are two different incompatible iterations of Python which I'll
 326 refer to as Python 2 and Python 3.  GNU MediaGoblin is written in
 327 Python 2 and requires Python 2.6 or 2.7.  At some point, we might
 328 switch to Python 3, but that's a future thing.
 329
 330 You can learn how to code in Python 2 from several excellent books
 331 that are freely available on the Internet:
 332
 333 * `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_
 334 * `Dive Into Pyton <http://diveintopython.org/>`_
 335 * `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_
 336 * `A Byte of Python <http://www.swaroopch.com/notes/Python>`_
 337
 338 These are all excellent texts.
 339
 340 .. YouCanHelp::
 341
 342    If you know of other good quality Python tutorials and Python
 343    tutorial videos, let us know!
 344
 345
 346 Learning Libraries GNU MediaGoblin uses
 347 ---------------------------------------
 348
 349 GNU MediaGoblin uses a variety of libraries in order to do what it
 350 does.  These libraries are listed in the :ref:`codebase-chapter`
 351 along with links to the project Web sites and documentation for the
 352 libraries.
 353
 354 There are a variety of Python-related conferences every year that have
 355 sessions covering many aspects of these libraries.  You can find them
 356 at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_.
 357
 358 .. [0] This is a shameless plug.  Will Kahn-Greene runs Python Miro
 359    Community.
 360
 361 If you have questions or need help, find us on the mailing list and on
 362 IRC.
 363
 364
 365 .. _hacking-howto-git:
 366
 367 Learning git
 368 ------------
 369
 370 git is an interesting and very powerful tool.  Like all powerful
 371 tools, it has a learning curve.
 372
 373 If you're new to git, we highly recommend the following resources for
 374 getting the hang of it:
 375
 376 * `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub
 377   intro to git
 378 * `Pro Git <http://progit.org/book/>`_ --- fantastic book
 379 * `Git casts <http://gitcasts.com/>`_ --- screencast covering git
 380   usage
 381 * `Git Reference <http://gitref.org/>`_ --- Git reference that makes
 382   it easier to get the hang of git if you're coming from other version
 383   control systems
 384
 385
 386 Learning other utilities
 387 ------------------------
 388
 389 The `OpenHatch <http://openhatch.org/>`_ site has a series of
 390 `training missions <http://openhatch.org/missions/>`_ which are
 391 designed to help you learn how to use these tools.
 392
 393 If you're new to tar, diff and patch, we highly recommend you sign up
 394 with OpenHatch and do the missions.