From 65e7ce634cfecc87ed6f390f9ccf91be513d2eea Mon Sep 17 00:00:00 2001 From: Will Kahn-Greene Date: Sun, 31 Jul 2011 23:07:13 -0400 Subject: [PATCH] Moves hacking howto and design decisions docs to wiki --- docs/source/codebase.rst | 2 +- docs/source/contributinghowto.rst | 6 +- docs/source/deploymenthowto.rst | 4 +- docs/source/designdecisions.rst | 329 ---------------------------- docs/source/git.rst | 2 +- docs/source/hackinghowto.rst | 345 ------------------------------ docs/source/index.rst | 2 - 7 files changed, 7 insertions(+), 683 deletions(-) delete mode 100644 docs/source/designdecisions.rst delete mode 100644 docs/source/hackinghowto.rst diff --git a/docs/source/codebase.rst b/docs/source/codebase.rst index 898eadfe..ba5f1e46 100644 --- a/docs/source/codebase.rst +++ b/docs/source/codebase.rst @@ -21,7 +21,7 @@ various recipes for getting things done. for where we hang out. For more information on how to get started hacking on GNU MediaGoblin, -see :ref:`hacking-howto`. +see `the wiki `_. Software Stack diff --git a/docs/source/contributinghowto.rst b/docs/source/contributinghowto.rst index 06d2814e..a4f5771a 100644 --- a/docs/source/contributinghowto.rst +++ b/docs/source/contributinghowto.rst @@ -46,8 +46,8 @@ Here are some things you can do today: **Write/Fix some code** If you are a coder and you're looking to code, check out the - :ref:`hacking-howto`. We even have tips on *becoming* a coder - and we're willing to help you! + `wiki `_. .. _filing-bugs: diff --git a/docs/source/deploymenthowto.rst b/docs/source/deploymenthowto.rst index f50edfb6..f3093a60 100644 --- a/docs/source/deploymenthowto.rst +++ b/docs/source/deploymenthowto.rst @@ -12,5 +12,5 @@ Step 3: Write the deployment guide and profit! But seriously, this is a stub since we're not quite there (yet) but if you want to see where we are now, you can try to run the latest -development version by following the instructions at -:ref:`hacking-howto`. +development version by following the instructions on +`the wiki `_. diff --git a/docs/source/designdecisions.rst b/docs/source/designdecisions.rst deleted file mode 100644 index afa1e26b..00000000 --- a/docs/source/designdecisions.rst +++ /dev/null @@ -1,329 +0,0 @@ -.. _design-decisions-chapter: - -================== - Design Decisions -================== - -.. contents:: Sections - :local: - - -This chapter talks a bit about design decisions. - - -Why GNU MediaGoblin? -==================== - -Chris and Will on "Why GNU MediaGoblin": - - Chris came up with the name MediaGoblin. The name is pretty fun. - It merges the idea that this is a Media hosting project with - Goblin which sort of sounds like gobbling. Here's a piece of - software that gobbles up your media for all to see. - - `According to Wikipedia `_, a - goblin is: - - a legendary evil or mischievous illiterate creature, described - as grotesquely evil or evil-like phantom - - So are we evil? No. Are we mischievous or illiterate? Not - really. So what kind of goblin are we thinking about? We're - thinking about these goblins: - - .. figure:: goblin.png - :alt: Cute goblin with a beret. - - *Figure 1: Cute goblin with a beret. llustrated by Chris - Webber* - - .. figure:: snugglygoblin.png - :scale: 50% - :alt: Snuggly goblin with a beret. - - *Figure 2: Snuggly goblin. Illustrated by Karen Rustad* - - Those are pretty cute goblins. Those are the kinds of goblins - we're thinking about. - - Chris started doing work on the project after thinking about it - for a year. Then, after talking with Matt and Rob, it became an - official GNU project. Thus we now call it GNU MediaGoblin. - - That's a lot of letters, though, so in the interest of brevity and - facilitating easier casual conversation and balancing that with - what's important to us, we have the following rules: - - 1. "GNU MediaGoblin" is the name we're going to use in all official - capacities: web site, documentation, press releases, ... - - 2. In casual conversation, it's ok to use more casual names. - - 3. If you're writing about the project, we ask that you call it GNU - MediaGoblin. - - 4. If you don't like the name, we kindly ask you to take a deep - breath, think a happy thought about cute little goblins playing - on a playground and taking cute pictures of themselves, and let - it go. (Will added this one.) - - -Why Python -========== - -Chris Webber on "Why Python": - - Because I know Python, love Python, am capable of actually making - this thing happen in Python (I've worked on a lot of large free - software web applications before in Python, including `Miro - Community`_, the `Miro Guide`_, a large portion of `Creative - Commons`_, and a whole bunch of things while working at `Imaginary - Landscape`_). Me starting a project like this makes sense if it's - done in Python. - - You might say that PHP is way more deployable, that Rails has way - more cool developers riding around on fixie bikes---and all of - those things are true. But I know Python, like Python, and think - that Python is pretty great. I do think that deployment in Python - is not as good as with PHP, but I think the days of shared hosting - are (thankfully) coming to an end, and will probably be replaced - by cheap virtual machines spun up on the fly for people who want - that sort of stuff, and Python will be a huge part of that future, - maybe even more than PHP will. The deployment tools are getting - better. Maybe we can use something like Silver Lining. Maybe we - can just distribute as ``.debs`` or ``.rpms``. We'll figure it - out when we get there. - - Regardless, if I'm starting this project, which I am, it's gonna - be in Python. - -.. _Miro Community: http://mirocommunity.org/ -.. _Miro Guide: http://miroguide.org/ -.. _Creative Commons: http://creativecommons.org/ -.. _Imaginary Landscape: http://www.imagescape.com/ - - -Why WSGI Minimalism -=================== - -Chris Webber on "Why WSGI Minimalism": - - If you notice in the technology list I list a lot of components - that are very "django-like", but not actually `Django`_ - components. What can I say, I really like a lot of the ideas in - Django! Which leads to the question: why not just use Django? - - While I really like Django's ideas and a lot of its components, I - also feel that most of the best ideas in Django I want have been - implemented as good or even better outside of Django. I could - just use Django and replace the templating system with Jinja2, and - the form system with wtforms, and the database with MongoDB and - MongoKit, but at that point, how much of Django is really left? - - I also am sometimes saddened and irritated by how coupled all of - Django's components are. Loosely coupled yes, but still coupled. - WSGI has done a good job of providing a base layer for running - applications on and if you know how to do it yourself [1]_, it's - not hard or many lines of code at all to bind them together - without any framework at all (not even say `Pylons`_, `Pyramid`_ - or `Flask`_ which I think are still great projects, especially for - people who want this sort of thing but have no idea how to get - started). And even at this already really early stage of writing - MediaGoblin, that glue work is mostly done. - - Not to say I don't think Django isn't great for a lot of things. - For a lot of stuff, it's still the best, but not for MediaGoblin, - I think. - - One thing that Django does super well though is documentation. It - still has some faults, but even with those considered I can hardly - think of any other project in Python that has as nice of - documentation as Django. It may be worth learning some lessons on - documentation from Django [2]_, on that note. - - I'd really like to have a good, thorough hacking-howto and - deployment-howto, especially in the former making some notes on - how to make it easier for Django hackers to get started. - -.. _Django: http://www.djangoproject.com/ -.. _Pylons: http://pylonshq.com/ -.. _Pyramid: http://docs.pylonsproject.org/projects/pyramid/dev/ -.. _Flask: http://flask.pocoo.org/ - -.. [1] http://pythonpaste.org/webob/do-it-yourself.html -.. [2] http://pycon.blip.tv/file/4881071/ - - -Why MongoDB -=========== - -Chris Webber on "Why MongoDB": - - In case you were wondering, I am not a NOSQL fanboy, I do not go - around telling people that MongoDB is web scale. Actually my - choice for MongoDB isn't scalability, though scaling up really - nicely is a pretty good feature and sets us up well in case large - volume sites eventually do use MediaGoblin. But there's another - side of scalability, and that's scaling down, which is important - for federation, maybe even more important than scaling up in an - ideal universe where everyone ran servers out of their own - housing. As a memory-mapped database, MongoDB is pretty hungry, - so actually I spent a lot of time debating whether the inability - to scale down as nicely as something like SQL has with sqlite - meant that it was out. - - But I decided in the end that I really want MongoDB, not for - scalability, but for flexibility. Schema evolution pains in SQL - are almost enough reason for me to want MongoDB, but not quite. - The real reason is because I want the ability to eventually handle - multiple media types through MediaGoblin, and also allow for - plugins, without the rigidity of tables making that difficult. In - other words, something like:: - - {"title": "Me talking until you are bored", - "description": "blah blah blah", - "media_type": "audio", - "media_data": { - "length": "2:30", - "codec": "OGG Vorbis"}, - "plugin_data": { - "licensing": { - "license": "http://creativecommons.org/licenses/by-sa/3.0/"}}} - - - Being able to just dump media-specific information in a media_data - hashtable is pretty great, and even better is having a plugin - system where you can just let plugins have their own entire - key-value space cleanly inside the document that doesn't interfere - with anyone else's stuff. If we were to let plugins to deposit - their own information inside the database, either we'd let plugins - create their own tables which makes SQL migrations even harder - than they already are, or we'd probably end up creating a table - with a column for key, a column for value, and a column for type - in one huge table called "plugin_data" or something similar. (Yo - dawg, I heard you liked plugins, so I put a database in your - database so you can query while you query.) Gross. - - I also don't want things to be too loose so that we forget or lose - the structure of things, and that's one reason why I want to use - MongoKit, because we can cleanly define a much structure as we - want and verify that documents match that structure generally - without adding too much bloat or overhead (MongoKit is a pretty - lightweight wrapper and doesn't inject extra MongoKit-specific - stuff into the database, which is nice and nicer than many other - ORMs in that way). - - -Why Sphinx for documentation -============================ - -Will Kahn-Greene on "Why Sphinx": - - `Sphinx`_ is a fantastic tool for organizing documentation for a - Python-based project that makes it pretty easy to write docs that - are readable in source form and can be "compiled" into HTML, LaTeX - and other formats. - - There are other doc systems out there, but given that GNU - MediaGoblin is being written in Python and I've done a ton of - documentation using Sphinx, it makes sense to use Sphinx for now. - -.. _Sphinx: http://sphinx.pocoo.org/ - - -Why AGPLv3 and CC0? -=================== - -Chris, Brett, Will, Rob, Matt, et al curated into a story where -everyone is the hero by Will on "Why AGPLv3 and CC0": - - The `AGPL v3`_ preserves the freedoms guaranteed by the GPL v3 in - the context of software as a service. Using this license ensures - that users of the service have the ability to examine the source, - deploy their own instance, and implement their own version. This - is really important to us and a core mission component of this - project. Thus we decided that the software parts should be under - this license. - - However, the project is made up of more than just software: - there's CSS, images, and other output-related things. We wanted - the templates/images/css side of the project all permissive and - permissive in the same absolutely permissive way. We're waiving - our copyrights to non-software things under the CC0 waiver. - - That brings us to the templates where there's some code and some - output. The template engine we're using is called Jinja2. It - mixes HTML markup with Python code to render the output of the - software. We decided the templates are part of the output of the - software and not the software itself. We wanted the output of the - software to be licensed in a hassle-free way so that when someone - deploys their own GNU MediaGoblin instance with their own - templates, they don't have to deal with the copyleft aspects of - the AGPLv3 and we'd be fine with that because the changes they're - making are identity-related. So at first we decided to waive our - copyrights to the templates with a CC0 waiver and then add an - exception to the AGPLv3 for the software such that the templates - can make calls into the software and yet be a separately licensed - work. However, Brett brought up the question of whether this - allows some unscrupulous person to make changes to the software - through the templates in such a way that they're not bound by the - AGPLv3: i.e. a loophole. We thought about this loophole and - between this and the extra legalese involved in the exception to - the AGPLv3, we decided that it's just way simpler if the templates - were also licensed under the AGPLv3. - - Then we have the licensing for the documentation. Given that the - documentation is tied to the software content-wise, we don't feel - like we have to worry about ensuring freedom of the documentation - or worry about attribution concerns. Thus we're waiving our - copyrights to the documentation under CC0 as well. - - Lastly, we have branding. This covers logos and other things that - are distinctive to GNU MediaGoblin that we feel represents this - project. Since we don't currently have any branding, this is an - open issue, but we're thinking we'll go with a CC BY-SA license. - - By licensing in this way, we make sure that users of the software - receive the freedoms that the AGPLv3 ensures regardless of what - fate befalls this project. - - So to summarize: - - * software (Python, JavaScript, HTML templates): licensed - under AGPLv3 - * non-software things (CSS, images, video): copyrights waived - under CC0 because this is output of the software - * documentation: copyrights waived under CC0 because it's not part - of the software - * branding assets: we're kicking this can down the road, but - probably CC BY-SA - - This is all codified in the ``COPYING`` file. - -.. _AGPL v3: http://www.gnu.org/licenses/agpl.html -.. _CC0 v1: http://creativecommons.org/publicdomain/zero/1.0/ - - -Why (non-mandatory) copyright assignment? -========================================= - -Chris Webber on "Why copyright assignment?": - - GNU MediaGoblin is a GNU project with non-mandatory but heavily - encouraged copyright assignment to the FSF. Most, if not all, of - the core contributors to GNU MediaGoblin will have done a - copyright assignment, but unlike some other GNU projects, it isn't - required here. We think this is the best choice for GNU - MediaGoblin: it ensures that the Free Software Foundation may - protect the software by enforcing the AGPL if the FSF sees fit, - but it also means that we can immediately merge in changes from a - new contributor. It also means that some significant non-FSF - contributors might also be able to enforce the AGPL if seen fit. - - Again, assignment is not mandatory, but it is heavily encouraged, - even incentivized: significant contributors who do a copyright - assignment to the FSF are eligible to have a unique goblin drawing - produced for them by the project's main founder, Christopher Allan - Webber. See :ref:`contributing-howto-chapter` for details. - - diff --git a/docs/source/git.rst b/docs/source/git.rst index bd0f9d52..ab3206b6 100644 --- a/docs/source/git.rst +++ b/docs/source/git.rst @@ -221,4 +221,4 @@ because he doesn't need it anymore. How to learn git ================ -Check out :ref:`hacking-howto-git`! +Check out `the wiki `_. diff --git a/docs/source/hackinghowto.rst b/docs/source/hackinghowto.rst deleted file mode 100644 index caafba53..00000000 --- a/docs/source/hackinghowto.rst +++ /dev/null @@ -1,345 +0,0 @@ -.. _hacking-howto: - -=============== - Hacking HOWTO -=============== - -.. contents:: Sections - :local: - - -So you want to hack on GNU MediaGoblin? -======================================= - -First thing to do is check out the `Web site -`_ where we list all the project -infrastructure including: - -* the IRC channel -* the mailing list -* the issue tracker - -Additionally, we have information on how to get involved, who to talk -to, what needs to be worked on, and other things besides! - -Second thing to do is take a look at :ref:`codebase-chapter` where -we've started documenting how GNU MediaGoblin is built and how to add -new things. - -Third you'll need to :ref:`get the requirements -`. - -Fourth, you'll need to build a development environment. We use buildout, -but if you want to use virtualenv, there's a set of mediocre not-very-supported -steps in the `wiki `_. - - -.. _get-requirements-section: - -Getting requirements -==================== - -First, you need to have the following installed before you can build -an environment for hacking on GNU MediaGoblin: - -* Python 2.6 or 2.7 - http://www.python.org/ - - You'll need Python as well as the dev files for building modules. - -* python-lxml - http://lxml.de/ -* git - http://git-scm.com/ -* MongoDB - http://www.mongodb.org/ - -If you're running Debian GNU/Linux or a Debian-derived distribution -such as Mint or Ubuntu, running the following should install these -requirements:: - - sudo apt-get install mongodb git-core python python-dev \ - python-lxml - -On Fedora:: - - yum install mongodb-server python-paste-deploy python-paste-script \ - git-core python python-devel python-lxml - -.. YouCanHelp:: - - If you have instructions for other GNU/Linux distributions to set - up requirements, let us know! - - -.. _hacking-with-buildout: - - -How to set up and maintain an environment for hacking with buildout -=================================================================== - -**Requirements** - -No additional requirements. - - -**Create a development environment** - -After installing the requirements, follow these steps: - -1. Clone the repository:: - - git clone git://gitorious.org/mediagoblin/mediagoblin.git - -2. Bootstrap and run buildout:: - - cd mediagoblin - python bootstrap.py && ./bin/buildout - - -That's it! Using this method, buildout should create a ``user_dev`` -directory, in which certain things will be stored (media, beaker -session stuff, etc). You can change this, but for development -purposes this default should be fine. - - -**Updating for dependency changes** - -While hacking on GNU MediaGoblin over time, you'll eventually have to -update your development environment because the dependencies have -changed. To do that, run:: - - ./bin/buildout && ./bin/gmg migrate - - -**Updating for code changes** - -You don't need to do anything---code changes are automatically -available. - - -**Deleting your buildout** - -At some point, you may want to delete your buildout. Perhaps it's to -start over. Perhaps it's to test building development environments -with buildout. - -To do this, do:: - - rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev - - -Running the server -================== - -If you want to get things running quickly and without hassle, just -run:: - - ./lazyserver.sh - -This will start up a python server where you can begin playing with -mediagoblin. It will also run celery in "always eager" mode so you -don't have to start a separate process for it. - -This is fine in development, but if you want to actually run celery -separately for testing (or deployment purposes), you'll want to run -the server independently:: - - ./bin/paster serve paste.ini --reload - - -Running celeryd -=============== - -If you aren't using ./lazyserver.sh or otherwise aren't running celery -in always eager mode, you'll need to do this if you want your media to -process and actually show up. It's probably a good idea in -development to have the web server (above) running in one terminal and -celeryd in another window. - -Run:: - - CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd - - -Running the test suite -====================== - -Run:: - - ./runtests.sh - - -Running a shell -=============== - -If you want a shell with your database pre-setup and an instantiated -application ready and at your fingertips.... - -Run:: - - ./bin/gmg shell - - -Troubleshooting -=============== - -pymongo.errors.AutoReconnect: could not find master/primary ------------------------------------------------------------ - -If you see this:: - - pymongo.errors.AutoReconnect: could not find master/primary - -then make sure mongodb is installed and running. - -If it's installed, check the mongodb log. On my machine, that's -``/var/log/mongodb/mongodb.log``. If you see something like:: - - old lock file: /var/lib/mongodb/mongod.lock. probably means... - -in that case you might have had an unclean shutdown. Try:: - - sudo mongod --repair - -If that didn't work, just delete the lock file and relaunch mongodb. - -Anyway, then start the mongodb server in whatever way is appropriate -for your distro / OS. - - -pkg_resources.DistributionNotFound: distribute ----------------------------------------------- - -If you get this while running buildout:: - - pkg_resources.DistributionNotFound: distribute - -Try this commmand instead:: - - python bootstrap.py --distribute && ./bin/buildout - - -Wiping your user data -===================== - -.. Note:: - - Unless you're doing development and working on and testing creating - a new instance, you will probably never have to do this. Will - plans to do this work and thus he documented it. - -.. YouCanHelp:: - - If you're familiar with MongoDB, we'd love to get a `script that - removes all the GNU MediaGoblin data from an existing instance - `_. Let us know! - - -Quickstart for Django programmers -================================= - -We're not using Django, but the codebase is very Django-like in its -structure. - -* ``routing.py`` is like ``urls.py`` in Django -* ``models.py`` has mongokit ORM definitions -* ``views.py`` is where the views go - -We're using MongoDB. Basically, instead of a relational database with -tables, you have a big JSON structure which acts a lot like a Python -dict. - - -.. YouCanHelp:: - - If there are other things that you think would help orient someone - new to GNU MediaGoblin but coming from Django, let us know! - - -Bite-sized bugs to start with -============================= - -**May 3rd, 2011**: We don't have a list of bite-sized bugs, yet, but -this is important to us. If you're interested in things to work on, -let us know on `the mailing list `_ or -on the `IRC channel `_. - - -Tips for people new to coding -============================= - -Learning Python ---------------- - -GNU MediaGoblin is written using a programming language called `Python -`_. - -There are two different incompatible iterations of Python which I'll -refer to as Python 2 and Python 3. GNU MediaGoblin is written in -Python 2 and requires Python 2.6 or 2.7. At some point, we might -switch to Python 3, but that's a future thing. - -You can learn how to code in Python 2 from several excellent books -that are freely available on the Internet: - -* `Learn Python the Hard Way `_ -* `Dive Into Pyton `_ -* `Python for Software Design `_ -* `A Byte of Python `_ - -These are all excellent texts. - -.. YouCanHelp:: - - If you know of other good quality Python tutorials and Python - tutorial videos, let us know! - - -Learning Libraries GNU MediaGoblin uses ---------------------------------------- - -GNU MediaGoblin uses a variety of libraries in order to do what it -does. These libraries are listed in the :ref:`codebase-chapter` -along with links to the project Web sites and documentation for the -libraries. - -There are a variety of Python-related conferences every year that have -sessions covering many aspects of these libraries. You can find them -at `Python Miro Community `_ [0]_. - -.. [0] This is a shameless plug. Will Kahn-Greene runs Python Miro - Community. - -If you have questions or need help, find us on the mailing list and on -IRC. - - -.. _hacking-howto-git: - -Learning git ------------- - -git is an interesting and very powerful tool. Like all powerful -tools, it has a learning curve. - -If you're new to git, we highly recommend the following resources for -getting the hang of it: - -* `Learn Git `_ --- the GitHub - intro to git -* `Pro Git `_ --- fantastic book -* `Git casts `_ --- screencast covering git - usage -* `Git Reference `_ --- Git reference that makes - it easier to get the hang of git if you're coming from other version - control systems - -There's also a git mission at `OpenHatch `_. - - -Learning other utilities ------------------------- - -The `OpenHatch `_ site has a series of -`training missions `_ which are -designed to help you learn how to use these tools. - -If you're new to tar, diff, patch and git, we highly recommend you sign -up with OpenHatch and do the missions. diff --git a/docs/source/index.rst b/docs/source/index.rst index 2f84d6a6..8c00869a 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -15,11 +15,9 @@ Table of Contents: mediagoblin contributinghowto deploymenthowto - hackinghowto theminghowto git codebase - designdecisions vision -- 2.25.1