Commit | Line | Data |
---|---|---|
5a40e1ec WKG |
1 | .. _hacking-howto: |
2 | ||
00fdc7bd WKG |
3 | =============== |
4 | Hacking HOWTO | |
5 | =============== | |
6 | ||
5a40e1ec WKG |
7 | |
8 | So you want to hack on GNU MediaGoblin? | |
9 | ======================================= | |
56d507b6 | 10 | |
9d952fdc | 11 | First thing to do is check out the `Web site |
9610848c | 12 | <http://mediagoblin.org/join/>`_ where we list all the project |
9d952fdc | 13 | infrastructure including: |
56d507b6 | 14 | |
56d507b6 | 15 | * the IRC channel |
9610848c WKG |
16 | * the mailing list |
17 | * the issue tracker | |
56d507b6 WKG |
18 | |
19 | Additionally, we have information on how to get involved, who to talk | |
20 | to, what needs to be worked on, and other things besides! | |
21 | ||
9610848c WKG |
22 | Second thing to do is take a look at :ref:`codebase-chapter` where |
23 | we've started documenting how GNU MediaGoblin is built and how to add | |
24 | new things. | |
25 | ||
4564da22 WKG |
26 | Third you'll need to :ref:`get the requirements |
27 | <get-requirements-section>`. | |
56d507b6 | 28 | |
4564da22 | 29 | Fourth, you'll need to build a development environment. For this step, there are two options: |
56d507b6 | 30 | |
4564da22 WKG |
31 | 1. :ref:`virtualenv <hacking-with-virtualenv>` OR |
32 | 2. :ref:`buildout and bootstrap <hacking-with-buildout>` | |
33 | ||
34 | Pick one---don't do both! | |
35 | ||
36 | ||
37 | .. _get-requirements-section: | |
56d507b6 | 38 | |
c66d0f5c | 39 | Getting requirements |
4564da22 | 40 | ==================== |
56d507b6 | 41 | |
c66d0f5c WKG |
42 | First, you need to have the following installed before you can build |
43 | an environment for hacking on GNU MediaGoblin: | |
56d507b6 | 44 | |
c66d0f5c | 45 | * Python 2.6 or 2.7 - http://www.python.org/ |
56d507b6 | 46 | |
c66d0f5c | 47 | You'll need Python as well as the dev files for building modules. |
56d507b6 | 48 | |
c66d0f5c WKG |
49 | * python-lxml - http://lxml.de/ |
50 | * git - http://git-scm.com/ | |
51 | * MongoDB - http://www.mongodb.org/ | |
52 | ||
53 | If you're running Debian GNU/Linux or a Debian-derived distribution | |
54 | such as Mint or Ubuntu, running the following should install these | |
55 | requirements:: | |
56 | ||
5740a0d6 WKG |
57 | sudo apt-get install mongodb git-core python python-dev \ |
58 | python-lxml | |
c66d0f5c | 59 | |
4564da22 | 60 | .. YouCanHelp:: |
52be418a | 61 | |
4564da22 WKG |
62 | If you have instructions for other GNU/Linux distributions to set |
63 | up requirements, let us know! | |
52be418a | 64 | |
52be418a | 65 | |
4564da22 | 66 | .. _hacking-with-virtualenv: |
52be418a | 67 | |
4564da22 WKG |
68 | How to set up and maintain an environment for hacking with virtualenv |
69 | ===================================================================== | |
52be418a WKG |
70 | |
71 | .. Note:: | |
72 | ||
73 | Either follow the instructions in this section OR follow the ones | |
4564da22 | 74 | in :ref:`hacking-with-buildout`. But don't do both! |
c66d0f5c | 75 | |
c66d0f5c | 76 | |
4564da22 WKG |
77 | **Requirements** |
78 | ||
79 | * virtualenv: http://pypi.python.org/pypi/virtualenv | |
2bae877e WKG |
80 | * virtualenv wrapper: |
81 | http://www.doughellmann.com/projects/virtualenvwrapper/ (be sure to | |
82 | read the `install instructions | |
83 | <http://www.doughellmann.com/docs/virtualenvwrapper/install.html>`_) | |
4564da22 WKG |
84 | |
85 | ||
86 | **Create a development environment** | |
c66d0f5c WKG |
87 | |
88 | 1. Clone the repository:: | |
89 | ||
90 | git clone http://git.gitorious.org/mediagoblin/mediagoblin.git | |
56d507b6 | 91 | |
4564da22 | 92 | 2. Create a virtual environment:: |
56d507b6 | 93 | |
4564da22 | 94 | mkvirtualenv --no-site-packages mediagoblin |
c66d0f5c | 95 | |
4564da22 WKG |
96 | 3. If that doesn't put you in the virutal environment you just |
97 | created, then do:: | |
c66d0f5c | 98 | |
4564da22 | 99 | workon mediagoblin |
c66d0f5c | 100 | |
4564da22 | 101 | 4. Run:: |
c66d0f5c | 102 | |
4564da22 | 103 | python setup.py develop |
c66d0f5c | 104 | |
4564da22 | 105 | That's it! |
56d507b6 | 106 | |
56d507b6 | 107 | |
4564da22 | 108 | **Activating a virtual environment** |
56d507b6 | 109 | |
4564da22 WKG |
110 | When you want to work on GNU MediaGoblin, you need to activate the |
111 | virtual environment like this:: | |
112 | ||
113 | workon mediagoblin | |
9610848c | 114 | |
9610848c | 115 | |
4564da22 | 116 | **Deactivating a virtual environment** |
56d507b6 | 117 | |
4564da22 | 118 | If you want to deactivate it, you can do this:: |
52be418a | 119 | |
4564da22 | 120 | deactivate |
52be418a | 121 | |
52be418a | 122 | |
4564da22 | 123 | **Updating a virtual environment with dependency changes** |
52be418a | 124 | |
4564da22 | 125 | 1. Enter the virtual environment. |
52be418a | 126 | |
4564da22 | 127 | 2. Run:: |
52be418a | 128 | |
4564da22 | 129 | python setup.py develop |
52be418a | 130 | |
52be418a | 131 | |
4564da22 | 132 | **Updating a virtual environment with code changes** |
52be418a | 133 | |
4564da22 WKG |
134 | You don't need to do anything---code changes are automatically |
135 | available. | |
52be418a WKG |
136 | |
137 | ||
4564da22 WKG |
138 | **Deleting a virtual environment** |
139 | ||
140 | At some point you may want to delete your virtual environment. | |
141 | Perhaps it's to start over. Perhaps it's so you can test building | |
142 | development environments with virtualenv. | |
143 | ||
144 | To do this, do:: | |
145 | ||
146 | rmvirtualenv mediagoblin | |
147 | ||
148 | ||
149 | .. _hacking-with-buildout: | |
150 | ||
1220e775 WKG |
151 | How to set up and maintain an environment for hacking with buildout |
152 | =================================================================== | |
4564da22 WKG |
153 | |
154 | .. Note:: | |
155 | ||
156 | Either follow the instructions in this section OR follow the ones | |
157 | in :ref:`hacking-with-virtualenv`. But don't do both! | |
158 | ||
159 | ||
160 | **Requirements** | |
161 | ||
162 | No additional requirements. | |
163 | ||
164 | ||
165 | **Create a development environment** | |
166 | ||
167 | After installing the requirements, follow these steps: | |
52be418a WKG |
168 | |
169 | 1. Clone the repository:: | |
170 | ||
171 | git clone http://git.gitorious.org/mediagoblin/mediagoblin.git | |
172 | ||
4564da22 | 173 | 2. Bootstrap and run buildout:: |
52be418a | 174 | |
4564da22 WKG |
175 | cd mediagoblin |
176 | python bootstrap.py && ./bin/buildout | |
52be418a | 177 | |
52be418a | 178 | |
4564da22 WKG |
179 | That's it! Using this method, buildout should create a ``user_dev`` |
180 | directory, in which certain things will be stored (media, beaker | |
181 | session stuff, etc). You can change this, but for development | |
182 | purposes this default should be fine. | |
52be418a | 183 | |
52be418a | 184 | |
4564da22 | 185 | **Updating for dependency changes** |
52be418a | 186 | |
4564da22 WKG |
187 | While hacking on GNU MediaGoblin over time, you'll eventually have to |
188 | update your development environment because the dependencies have | |
189 | changed. To do that, run:: | |
52be418a | 190 | |
4564da22 | 191 | ./bin/buildout |
52be418a | 192 | |
52be418a | 193 | |
4564da22 | 194 | **Updating for code changes** |
52be418a | 195 | |
4564da22 WKG |
196 | You don't need to do anything---code changes are automatically |
197 | available. | |
52be418a | 198 | |
52be418a | 199 | |
4564da22 WKG |
200 | **Deleting your buildout** |
201 | ||
202 | At some point, you may want to delete your buildout. Perhaps it's to | |
203 | start over. Perhaps it's to test building development environments | |
204 | with buildout. | |
205 | ||
206 | To do this, do:: | |
52be418a | 207 | |
4564da22 WKG |
208 | rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev |
209 | ||
210 | ||
11e5b197 CAW |
211 | Running the server |
212 | ================== | |
213 | ||
4564da22 WKG |
214 | If you did virtualenv, run:: |
215 | ||
216 | paster serve mediagoblin.ini --reload | |
217 | ||
218 | If you did buildout, run:: | |
11e5b197 CAW |
219 | |
220 | ./bin/paster serve mediagoblin.ini --reload | |
221 | ||
222 | ||
1c424df5 CAW |
223 | Running celeryd |
224 | =============== | |
225 | ||
226 | You need to do this if you want your media to process and actually | |
227 | show up. It's probably a good idea in development to have the web | |
228 | server (above) running in one terminal and celeryd in another window. | |
229 | ||
4564da22 WKG |
230 | If you did virtualenv, run:: |
231 | ||
232 | CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery celeryd | |
233 | ||
234 | If you did buildout, run:: | |
1c424df5 CAW |
235 | |
236 | CELERY_CONFIG_MODULE=mediagoblin.celery_setup.from_celery ./bin/celeryd | |
237 | ||
238 | ||
11e5b197 CAW |
239 | Running the test suite |
240 | ====================== | |
241 | ||
4564da22 WKG |
242 | If you did virtualenv, run:: |
243 | ||
244 | nosetests | |
245 | ||
246 | If you did buildout, run:: | |
11e5b197 CAW |
247 | |
248 | ./bin/nosetests | |
249 | ||
250 | ||
4564da22 WKG |
251 | Troubleshooting |
252 | =============== | |
5740a0d6 | 253 | |
4564da22 WKG |
254 | pymongo.errors.AutoReconnect: could not find master/primary |
255 | ----------------------------------------------------------- | |
9d952fdc | 256 | |
4564da22 | 257 | If you see this:: |
9d952fdc | 258 | |
4564da22 | 259 | pymongo.errors.AutoReconnect: could not find master/primary |
9d952fdc | 260 | |
4564da22 | 261 | then make sure mongodb is installed and running. |
5740a0d6 | 262 | |
2704f7a6 WKG |
263 | If it's installed, check the mongodb log. On my machine, that's ``/var/log/mongodb/mongodb.log``. If you see something like:: |
264 | ||
265 | old lock file: /var/lib/mongodb/mongod.lock. probably means... | |
266 | ||
267 | Then delete the lock file and relaunch mongodb. | |
268 | ||
5740a0d6 | 269 | |
4564da22 WKG |
270 | Wiping your user data |
271 | ===================== | |
272 | ||
273 | .. Note:: | |
274 | ||
275 | Unless you're doing development and working on and testing creating | |
276 | a new instance, you will probably never have to do this. Will | |
277 | plans to do this work and thus he documented it. | |
9610848c WKG |
278 | |
279 | .. YouCanHelp:: | |
280 | ||
1b734c4d CAW |
281 | If you're familiar with MongoDB, we'd love to get a `script that |
282 | removes all the GNU MediaGoblin data from an existing instance | |
283 | <http://bugs.foocorp.net/issues/296>`_. Let us know! | |
5740a0d6 WKG |
284 | |
285 | ||
9610848c WKG |
286 | Quickstart for Django programmers |
287 | ================================= | |
5740a0d6 | 288 | |
9610848c WKG |
289 | We're not using Django, but the codebase is very Django-like in its |
290 | structure. | |
5740a0d6 | 291 | |
9610848c WKG |
292 | * ``routing.py`` is like ``urls.py`` in Django |
293 | * ``models.py`` has mongokit ORM definitions | |
294 | * ``views.py`` is where the views go | |
5740a0d6 | 295 | |
9610848c WKG |
296 | We're using MongoDB. Basically, instead of a relational database with |
297 | tables, you have a big JSON structure which acts a lot like a Python | |
298 | dict. | |
5740a0d6 WKG |
299 | |
300 | ||
9610848c | 301 | .. YouCanHelp:: |
5a40e1ec | 302 | |
9610848c WKG |
303 | If there are other things that you think would help orient someone |
304 | new to GNU MediaGoblin but coming from Django, let us know! | |
5a40e1ec WKG |
305 | |
306 | ||
307 | Bite-sized bugs to start with | |
308 | ============================= | |
309 | ||
9610848c WKG |
310 | **May 3rd, 2011**: We don't have a list of bite-sized bugs, yet, but |
311 | this is important to us. If you're interested in things to work on, | |
312 | let us know on `the mailing list <http://mediagoblin.org/join/>`_ or | |
313 | on the `IRC channel <http://mediagoblin.org/join/>`_. | |
9d952fdc WKG |
314 | |
315 | ||
316 | Tips for people new to coding | |
317 | ============================= | |
318 | ||
845c3ae2 WKG |
319 | Learning Python |
320 | --------------- | |
9d952fdc WKG |
321 | |
322 | GNU MediaGoblin is written using a programming language called `Python | |
323 | <http://python.org/>`_. | |
324 | ||
325 | There are two different incompatible iterations of Python which I'll | |
326 | refer to as Python 2 and Python 3. GNU MediaGoblin is written in | |
327 | Python 2 and requires Python 2.6 or 2.7. At some point, we might | |
328 | switch to Python 3, but that's a future thing. | |
329 | ||
330 | You can learn how to code in Python 2 from several excellent books | |
331 | that are freely available on the Internet: | |
332 | ||
333 | * `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_ | |
334 | * `Dive Into Pyton <http://diveintopython.org/>`_ | |
335 | * `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_ | |
336 | * `A Byte of Python <http://www.swaroopch.com/notes/Python>`_ | |
337 | ||
338 | These are all excellent texts. | |
339 | ||
9610848c WKG |
340 | .. YouCanHelp:: |
341 | ||
342 | If you know of other good quality Python tutorials and Python | |
343 | tutorial videos, let us know! | |
9d952fdc WKG |
344 | |
345 | ||
845c3ae2 WKG |
346 | Learning Libraries GNU MediaGoblin uses |
347 | --------------------------------------- | |
9d952fdc WKG |
348 | |
349 | GNU MediaGoblin uses a variety of libraries in order to do what it | |
3dca2776 | 350 | does. These libraries are listed in the :ref:`codebase-chapter` |
9d952fdc WKG |
351 | along with links to the project Web sites and documentation for the |
352 | libraries. | |
353 | ||
354 | There are a variety of Python-related conferences every year that have | |
355 | sessions covering many aspects of these libraries. You can find them | |
356 | at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_. | |
357 | ||
358 | .. [0] This is a shameless plug. Will Kahn-Greene runs Python Miro | |
359 | Community. | |
360 | ||
361 | If you have questions or need help, find us on the mailing list and on | |
362 | IRC. | |
845c3ae2 WKG |
363 | |
364 | ||
365 | .. _hacking-howto-git: | |
366 | ||
367 | Learning git | |
368 | ------------ | |
369 | ||
370 | git is an interesting and very powerful tool. Like all powerful | |
371 | tools, it has a learning curve. | |
372 | ||
373 | If you're new to git, we highly recommend the following resources for | |
374 | getting the hang of it: | |
375 | ||
376 | * `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub | |
377 | intro to git | |
378 | * `Pro Git <http://progit.org/book/>`_ --- fantastic book | |
379 | * `Git casts <http://gitcasts.com/>`_ --- screencast covering git | |
380 | usage | |
381 | * `Git Reference <http://gitref.org/>`_ --- Git reference that makes | |
382 | it easier to get the hang of git if you're coming from other version | |
383 | control systems | |
384 | ||
385 | ||
386 | Learning other utilities | |
387 | ------------------------ | |
388 | ||
389 | The `OpenHatch <http://openhatch.org/>`_ site has a series of | |
390 | `training missions <http://openhatch.org/missions/>`_ which are | |
391 | designed to help you learn how to use these tools. | |
392 | ||
393 | If you're new to tar, diff and patch, we highly recommend you sign up | |
394 | with OpenHatch and do the missions. |