Commit | Line | Data |
---|---|---|
5a40e1ec WKG |
1 | .. _hacking-howto: |
2 | ||
00fdc7bd WKG |
3 | =============== |
4 | Hacking HOWTO | |
5 | =============== | |
6 | ||
6729d65a WKG |
7 | .. contents:: Sections |
8 | :local: | |
9 | ||
10 | ||
5a40e1ec WKG |
11 | So you want to hack on GNU MediaGoblin? |
12 | ======================================= | |
56d507b6 | 13 | |
9d952fdc | 14 | First thing to do is check out the `Web site |
9610848c | 15 | <http://mediagoblin.org/join/>`_ where we list all the project |
9d952fdc | 16 | infrastructure including: |
56d507b6 | 17 | |
56d507b6 | 18 | * the IRC channel |
9610848c WKG |
19 | * the mailing list |
20 | * the issue tracker | |
56d507b6 WKG |
21 | |
22 | Additionally, we have information on how to get involved, who to talk | |
23 | to, what needs to be worked on, and other things besides! | |
24 | ||
9610848c WKG |
25 | Second thing to do is take a look at :ref:`codebase-chapter` where |
26 | we've started documenting how GNU MediaGoblin is built and how to add | |
27 | new things. | |
28 | ||
4564da22 WKG |
29 | Third you'll need to :ref:`get the requirements |
30 | <get-requirements-section>`. | |
56d507b6 | 31 | |
2aa3bb11 WKG |
32 | Fourth, you'll need to build a development environment. We use buildout, |
33 | but if you want to use virtualenv, there's a set of mediocre not-very-supported | |
34 | steps in the `wiki <https://gitorious.org/mediagoblin/pages/Home>`_. | |
4564da22 WKG |
35 | |
36 | ||
37 | .. _get-requirements-section: | |
56d507b6 | 38 | |
c66d0f5c | 39 | Getting requirements |
4564da22 | 40 | ==================== |
56d507b6 | 41 | |
c66d0f5c WKG |
42 | First, you need to have the following installed before you can build |
43 | an environment for hacking on GNU MediaGoblin: | |
56d507b6 | 44 | |
c66d0f5c | 45 | * Python 2.6 or 2.7 - http://www.python.org/ |
56d507b6 | 46 | |
c66d0f5c | 47 | You'll need Python as well as the dev files for building modules. |
56d507b6 | 48 | |
c66d0f5c WKG |
49 | * python-lxml - http://lxml.de/ |
50 | * git - http://git-scm.com/ | |
51 | * MongoDB - http://www.mongodb.org/ | |
52 | ||
53 | If you're running Debian GNU/Linux or a Debian-derived distribution | |
54 | such as Mint or Ubuntu, running the following should install these | |
55 | requirements:: | |
56 | ||
5740a0d6 WKG |
57 | sudo apt-get install mongodb git-core python python-dev \ |
58 | python-lxml | |
c66d0f5c | 59 | |
983e11f0 CAW |
60 | On Fedora:: |
61 | ||
62 | yum install mongodb-server python-paste-deploy python-paste-script \ | |
63 | git-core python python-devel | |
64 | ||
4564da22 | 65 | .. YouCanHelp:: |
52be418a | 66 | |
4564da22 WKG |
67 | If you have instructions for other GNU/Linux distributions to set |
68 | up requirements, let us know! | |
52be418a | 69 | |
52be418a | 70 | |
7a5ddb45 CAW |
71 | .. _hacking-with-buildout: |
72 | ||
2aa3bb11 | 73 | |
7a5ddb45 CAW |
74 | How to set up and maintain an environment for hacking with buildout |
75 | =================================================================== | |
76 | ||
7a5ddb45 CAW |
77 | **Requirements** |
78 | ||
79 | No additional requirements. | |
80 | ||
81 | ||
82 | **Create a development environment** | |
83 | ||
84 | After installing the requirements, follow these steps: | |
85 | ||
86 | 1. Clone the repository:: | |
87 | ||
88 | git clone http://git.gitorious.org/mediagoblin/mediagoblin.git | |
89 | ||
90 | 2. Bootstrap and run buildout:: | |
91 | ||
92 | cd mediagoblin | |
93 | python bootstrap.py && ./bin/buildout | |
94 | ||
95 | ||
96 | That's it! Using this method, buildout should create a ``user_dev`` | |
97 | directory, in which certain things will be stored (media, beaker | |
98 | session stuff, etc). You can change this, but for development | |
99 | purposes this default should be fine. | |
100 | ||
101 | ||
102 | **Updating for dependency changes** | |
103 | ||
104 | While hacking on GNU MediaGoblin over time, you'll eventually have to | |
105 | update your development environment because the dependencies have | |
106 | changed. To do that, run:: | |
107 | ||
f4cfb4e9 | 108 | ./bin/buildout && ./bin/gmg migrate |
7a5ddb45 CAW |
109 | |
110 | ||
111 | **Updating for code changes** | |
112 | ||
113 | You don't need to do anything---code changes are automatically | |
114 | available. | |
115 | ||
116 | ||
117 | **Deleting your buildout** | |
118 | ||
119 | At some point, you may want to delete your buildout. Perhaps it's to | |
120 | start over. Perhaps it's to test building development environments | |
121 | with buildout. | |
122 | ||
123 | To do this, do:: | |
124 | ||
125 | rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev | |
126 | ||
4564da22 | 127 | |
11e5b197 CAW |
128 | Running the server |
129 | ================== | |
130 | ||
f1fadb64 CAW |
131 | If you want to get things running quickly and without hassle, just |
132 | run:: | |
133 | ||
134 | ./lazyserver.sh | |
135 | ||
136 | This will start up a python server where you can begin playing with | |
137 | mediagoblin. It will also run celery in "always eager" mode so you | |
138 | don't have to start a separate process for it. | |
139 | ||
140 | This is fine in development, but if you want to actually run celery | |
141 | separately for testing (or deployment purposes), you'll want to run | |
142 | the server independently:: | |
11e5b197 | 143 | |
41f965c5 | 144 | ./bin/paster serve paste.ini --reload |
11e5b197 CAW |
145 | |
146 | ||
1c424df5 CAW |
147 | Running celeryd |
148 | =============== | |
149 | ||
f1fadb64 CAW |
150 | If you aren't using ./lazyserver.sh or otherwise aren't running celery |
151 | in always eager mode, you'll need to do this if you want your media to | |
152 | process and actually show up. It's probably a good idea in | |
153 | development to have the web server (above) running in one terminal and | |
154 | celeryd in another window. | |
1c424df5 | 155 | |
2aa3bb11 | 156 | Run:: |
1c424df5 | 157 | |
073b61fe | 158 | CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd |
1c424df5 CAW |
159 | |
160 | ||
11e5b197 CAW |
161 | Running the test suite |
162 | ====================== | |
163 | ||
2aa3bb11 | 164 | Run:: |
11e5b197 | 165 | |
8b7663a0 | 166 | ./runtests.sh |
11e5b197 CAW |
167 | |
168 | ||
3e4a2f2b CAW |
169 | Running a shell |
170 | =============== | |
171 | ||
172 | If you want a shell with your database pre-setup and an instantiated | |
2aa3bb11 | 173 | application ready and at your fingertips.... |
3e4a2f2b | 174 | |
2aa3bb11 | 175 | Run:: |
3e4a2f2b CAW |
176 | |
177 | ./bin/gmg shell | |
178 | ||
3e4a2f2b | 179 | |
4564da22 WKG |
180 | Troubleshooting |
181 | =============== | |
5740a0d6 | 182 | |
4564da22 WKG |
183 | pymongo.errors.AutoReconnect: could not find master/primary |
184 | ----------------------------------------------------------- | |
9d952fdc | 185 | |
4564da22 | 186 | If you see this:: |
9d952fdc | 187 | |
4564da22 | 188 | pymongo.errors.AutoReconnect: could not find master/primary |
9d952fdc | 189 | |
4564da22 | 190 | then make sure mongodb is installed and running. |
5740a0d6 | 191 | |
2aa3bb11 WKG |
192 | If it's installed, check the mongodb log. On my machine, that's |
193 | ``/var/log/mongodb/mongodb.log``. If you see something like:: | |
2704f7a6 WKG |
194 | |
195 | old lock file: /var/lib/mongodb/mongod.lock. probably means... | |
196 | ||
197 | Then delete the lock file and relaunch mongodb. | |
198 | ||
5740a0d6 | 199 | |
4564da22 WKG |
200 | Wiping your user data |
201 | ===================== | |
202 | ||
203 | .. Note:: | |
204 | ||
205 | Unless you're doing development and working on and testing creating | |
206 | a new instance, you will probably never have to do this. Will | |
207 | plans to do this work and thus he documented it. | |
9610848c WKG |
208 | |
209 | .. YouCanHelp:: | |
210 | ||
1b734c4d CAW |
211 | If you're familiar with MongoDB, we'd love to get a `script that |
212 | removes all the GNU MediaGoblin data from an existing instance | |
213 | <http://bugs.foocorp.net/issues/296>`_. Let us know! | |
5740a0d6 WKG |
214 | |
215 | ||
9610848c WKG |
216 | Quickstart for Django programmers |
217 | ================================= | |
5740a0d6 | 218 | |
9610848c WKG |
219 | We're not using Django, but the codebase is very Django-like in its |
220 | structure. | |
5740a0d6 | 221 | |
9610848c WKG |
222 | * ``routing.py`` is like ``urls.py`` in Django |
223 | * ``models.py`` has mongokit ORM definitions | |
224 | * ``views.py`` is where the views go | |
5740a0d6 | 225 | |
9610848c WKG |
226 | We're using MongoDB. Basically, instead of a relational database with |
227 | tables, you have a big JSON structure which acts a lot like a Python | |
228 | dict. | |
5740a0d6 WKG |
229 | |
230 | ||
9610848c | 231 | .. YouCanHelp:: |
5a40e1ec | 232 | |
9610848c WKG |
233 | If there are other things that you think would help orient someone |
234 | new to GNU MediaGoblin but coming from Django, let us know! | |
5a40e1ec WKG |
235 | |
236 | ||
237 | Bite-sized bugs to start with | |
238 | ============================= | |
239 | ||
9610848c WKG |
240 | **May 3rd, 2011**: We don't have a list of bite-sized bugs, yet, but |
241 | this is important to us. If you're interested in things to work on, | |
242 | let us know on `the mailing list <http://mediagoblin.org/join/>`_ or | |
243 | on the `IRC channel <http://mediagoblin.org/join/>`_. | |
9d952fdc WKG |
244 | |
245 | ||
246 | Tips for people new to coding | |
247 | ============================= | |
248 | ||
845c3ae2 WKG |
249 | Learning Python |
250 | --------------- | |
9d952fdc WKG |
251 | |
252 | GNU MediaGoblin is written using a programming language called `Python | |
253 | <http://python.org/>`_. | |
254 | ||
255 | There are two different incompatible iterations of Python which I'll | |
256 | refer to as Python 2 and Python 3. GNU MediaGoblin is written in | |
257 | Python 2 and requires Python 2.6 or 2.7. At some point, we might | |
258 | switch to Python 3, but that's a future thing. | |
259 | ||
260 | You can learn how to code in Python 2 from several excellent books | |
261 | that are freely available on the Internet: | |
262 | ||
263 | * `Learn Python the Hard Way <http://learnpythonthehardway.org/>`_ | |
264 | * `Dive Into Pyton <http://diveintopython.org/>`_ | |
265 | * `Python for Software Design <http://www.greenteapress.com/thinkpython/>`_ | |
266 | * `A Byte of Python <http://www.swaroopch.com/notes/Python>`_ | |
267 | ||
268 | These are all excellent texts. | |
269 | ||
9610848c WKG |
270 | .. YouCanHelp:: |
271 | ||
272 | If you know of other good quality Python tutorials and Python | |
273 | tutorial videos, let us know! | |
9d952fdc WKG |
274 | |
275 | ||
845c3ae2 WKG |
276 | Learning Libraries GNU MediaGoblin uses |
277 | --------------------------------------- | |
9d952fdc WKG |
278 | |
279 | GNU MediaGoblin uses a variety of libraries in order to do what it | |
3dca2776 | 280 | does. These libraries are listed in the :ref:`codebase-chapter` |
9d952fdc WKG |
281 | along with links to the project Web sites and documentation for the |
282 | libraries. | |
283 | ||
284 | There are a variety of Python-related conferences every year that have | |
285 | sessions covering many aspects of these libraries. You can find them | |
286 | at `Python Miro Community <http://python.mirocommunity.org>`_ [0]_. | |
287 | ||
288 | .. [0] This is a shameless plug. Will Kahn-Greene runs Python Miro | |
289 | Community. | |
290 | ||
291 | If you have questions or need help, find us on the mailing list and on | |
292 | IRC. | |
845c3ae2 WKG |
293 | |
294 | ||
295 | .. _hacking-howto-git: | |
296 | ||
297 | Learning git | |
298 | ------------ | |
299 | ||
300 | git is an interesting and very powerful tool. Like all powerful | |
301 | tools, it has a learning curve. | |
302 | ||
303 | If you're new to git, we highly recommend the following resources for | |
304 | getting the hang of it: | |
305 | ||
306 | * `Learn Git <http://learn.github.com/p/intro.html>`_ --- the GitHub | |
307 | intro to git | |
308 | * `Pro Git <http://progit.org/book/>`_ --- fantastic book | |
309 | * `Git casts <http://gitcasts.com/>`_ --- screencast covering git | |
310 | usage | |
311 | * `Git Reference <http://gitref.org/>`_ --- Git reference that makes | |
312 | it easier to get the hang of git if you're coming from other version | |
313 | control systems | |
314 | ||
2aa3bb11 WKG |
315 | There's also a git mission at `OpenHatch <http://openhatch.org/>`_. |
316 | ||
845c3ae2 WKG |
317 | |
318 | Learning other utilities | |
319 | ------------------------ | |
320 | ||
321 | The `OpenHatch <http://openhatch.org/>`_ site has a series of | |
322 | `training missions <http://openhatch.org/missions/>`_ which are | |
323 | designed to help you learn how to use these tools. | |
324 | ||
2aa3bb11 WKG |
325 | If you're new to tar, diff, patch and git, we highly recommend you sign |
326 | up with OpenHatch and do the missions. |