= Genshi Tutorial =

This tutorial is intended to give an introduction on how to use Genshi in your web application, and present common patterns and best practices. It is aimed at developers new to Genshi as well as those who've already used Genshi, but are looking for advice or inspiration on how to improve that usage.

== Introduction ==

In this tutorial we'll create a simple Python web application based on [http://cherrpy.org/ CherryPy 3]. !CherryPy was chosen because it provides a convenient level of abstraction over raw CGI or [http://wsgi.org/wsgi WSGI] development, but is less ambitious than full-stack web frameworks such as [http://pylonshq.com/ Pylons] or [http://www.djangoproject.com/ Django], which tend to come with a preferred templating language, and often show significant bias towards that language.

The application is a stripped-down version of sites such as [http://reddit.com/ reddit] or [http://digg.com/ digg]: it lets users submit links to online articles they find interesting, and then lets other users vote on those stories and post comments. Just for kicks, we'll call that application '''Geddit'''

The project is kept as simple as possible, while still showing many of Genshi features and how to best use them:
 * For persistence, we'll use native Python object serialization (via the `pickle` module), instead of an SQL database and an ORM.
 * There's no authentication of any kind. Anyone can submit links, anyone can comment.

[[PageOutline(2-3, Content, inline)]]

== Prerequisites ==

First, make sure you have !CherryPy 3.0.x installed, as well as recent versions of [http://formencode.org/ FormEncode], Genshi (obviously), and [http://pythonpaste.org/ Paste]. You can download and install those manually, or just use [http://peak.telecommunity.com/DevCenter/EasyInstall easy_install]:

{{{
$ easy_install CherryPy
$ easy_install FormEncode
$ easy_install Genshi
$ easy_install Paste
}}}

== Getting Started ==

Next, set up the basic !CherryPy application. Create a directory that should contain the application, and inside that directory create a Python package named `geddit` (basically a `geddit` directory containing an empty file called `__init__.py`. Inside that package, create a file called `controller.py` with the following content:

{{{
#!python
#!/usr/bin/env python

import os
import pickle
import sys

import cherrypy
from paste.evalexception.middleware import EvalException


class Root(object):

    def __init__(self, data):
        self.data = data

    @cherrypy.expose
    def index(self):
        return 'Geddit'


def main(filename):
    # load data from the pickle file, or initialize it to an empty list
    if os.path.exists(filename):
        fileobj = open(filename, 'rb')
        try:
            data = pickle.load(fileobj)
        finally:
            fileobj.close()
    else:
        data = []

    # save data back to the pickle file when the server is stopped
    def _save_data():
        fileobj = open(filename, 'wb')
        try:
            pickle.dump(data, fileobj)
        finally:
            fileobj.close()
    cherrypy.engine.on_stop_engine_list.append(_save_data)

    # Some global configuration; note that this could be moved into a configuration file
    cherrypy.config.update({
        'request.throw_errors': True,
        'tools.encode.on': True, 'tools.encode.encoding': 'utf-8',
        'tools.decode.on': True,
        'tools.trailing_slash.on': True,
        'tools.staticdir.root': os.path.abspath(os.path.dirname(__file__)),
    })

    # Initialize the application, and add EvalException for more helpful error messages
    app = cherrypy.Application(Root(data))
    app.wsgiapp.pipeline.append(('paste_exc', EvalException))
    cherrypy.quickstart(app, '/', {
        '/media': {
            'tools.staticdir.on': True,
            'tools.staticdir.dir': 'static'
        }
    })

if __name__ == '__main__':
    main(sys.argv[1])
}}}

Enter the tutorial directory in the terminal, and run:

{{{
$ PYTHONPATH=. python geddit/controller.py geddit.db
}}}

You should see a log message pointing you to the URL where the application is being served, which is usually http://localhost:8080/. Visiting that page will respond with just the string “Geddit”, as that's what the `index()` method of the `Root` object returns.

Note that we've configured !CherryPy to serve static files from the `geddit/static` directory. !CherryPy will complain that that directory does not exist, so create it, but leave it empty for now. We'll add static resources later on in the tutorial.

== Basic Template Rendering ==

So far the code doesn't actually use Genshi, or even any kind of templating. Let's change that.

Inside of the `geddit` directory, create a directory called `templates`, and inside that directory create a file called `index.html`, with the following content:

{{{
#!genshi
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/">
  <head>
    <title>$title</title>
  </head>
  <body>
    <h1>$title</h1>
    <p>Welcome!</p>
  </body>
</html>
}}}

This is basically an almost static HTML file with some simple variable substitution.

We now need to change the controller code so that this template is used. First, add the Genshi `TemplateLoader` to the imports at the top of the `genshi/controller.py` file, and instantiate a loader for the `geddit/templates` directory:

{{{
#!python
import cherrypy
from genshi.template import TemplateLoader
from paste.evalexception.middleware import EvalException

loader = TemplateLoader(
    os.path.join(os.path.dirname(__file__), 'templates'),
    auto_reload=True
)
}}}

Next, change the implementation of the `index()` method of the `Root` class to look like this:

{{{
#!python
    @cherrypy.expose
    def index(self):
        tmpl = loader.load('index.html')
        return tmpl.generate(title='Geddit').render('html', doctype='html')
}}}

This asks the template loader for a template named `index.html`, generates the output stream, and finally serializes the output to HTML. You you now reload the page in your browser, you should get back the following HTML:

{{{
#!xml
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
  <head>
    <title>Geddit</title>
  </head>
  <body>
    <h1>Geddit</h1>
    <p>Welcome!</p>
  </body>
</html>
}}}

== Data Model ==

To continue, we'll need to first add some classes to define the data model the application will use. As mentioned above, we're using a simple pickle file for persistence, so all we need to do here is create a couple of very simply Python classes.

Inside the `geddit` directory, create a file named `model.py`, with the following content:

{{{
#!python
from datetime import datetime


class Submission(object):

    def __init__(self, username, url, title):
        self.username = username
        self.url = url
        self.title = title
        self.time = datetime.utcnow()
        self.comments = []

    def __repr__(self):
        return '<%s %r>' % (type(self).__name__, self.title)

    def add_comment(self, username, content):
        comment = Comment(username, content, in_reply_to=self)
        self.comments.append(comment)
        return comment


class Comment(object):

    def __init__(self, username, content, in_reply_to=None):
        self.username = username
        self.content = content
        self.in_reply_to = in_reply_to
        self.time = datetime.utcnow()
        self.replies = []

    def __repr__(self):
        return '<%s>' % (type(self).__name__)

    def add_reply(self, username, content):
        reply = Comment(username, content, in_reply_to=self)
        self.replies.append(reply)
        return reply
}}}

And import those classes in `geddit/controllers.py`, just below the other imports:

{{{
#!python
from geddit.model import Submission, Comment
}}}

Now let's add some initial content to the “database”. You'll need to stop the CherryPy server to do that. Then, in the terminal, from the tutorial directory, launch the interactive Python shell, and execute the following code:

{{{
#!pycon
>>> from geddit.model import *
>>> data = []
>>> submission = Submission(username='joe', url='http://example.org/', title='An example')
>>> comment = submission.add_comment(username='jack', content='Bla bla bla')
>>> comment.add_reply(username='joe', content='Bla blabla bla bla bla')
>>> data.append(submission)
>>> submission = Submission(username='annie', url='http://reddit.com/', title='The real thing')
>>> data.append(submission)
>>> data
[<Submission 'An example'>, <Submission 'The real thing'>]

>>> import pickle
>>> pickle.dump(data, open('geddit.db', 'wb'))
>>> ^D
}}}

You should now have two submissions in the pickle file, with the first submission having a comment, as well as a reply to that comment. Restart the CherryPy server by running:

{{{
$ PYTHONPATH=. python geddit/controller.py geddit.db
}}}

== Extending the Template ==

Now let's change the `Root.index()` method to pass the submissions list to the template:

{{{
#!python
    @cherrypy.expose
    def index(self):
        tmpl = loader.load('index.html')
        stream = tmpl.generate(submissions=self.data)
        return stream.render('html', doctype='html')
}}}

And finally, we'll modify the `index.html` template so that it displays the submissions in a simple ordered list. While we're at it, let's add a link to submit new items:

{{{
#!genshi
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/">
  <head>
    <title>Geddit</title>
  </head>
  <body>
    <h1>Geddit</h1>

    <p><a href="/submit/">Submit new link</a></p>

    <ol py:if="submissions">
      <li py:for="submission in submissions">
        <a href="${submission.url}">${submission.title}</a>
        posted by ${submission.username}
        at ${submission.time.strftime('%M/%d/%Y %H:%m')}
      </li>
    </ol>

  </body>
</html>
}}}

When you reload the page in the browser, you should see a page similar to this:

[[Image(tutorial01.png)]]

== Adding a Submission Form ==

In the previous step, we've already added a link to a submission form to the template, but we haven't implemented the logic to handle requests to that link yet.

To do that, we need to add a method to the `Root` class in `geddit/controller.py`:

{{{
#!python
    @cherrypy.expose
    def submit(self, cancel=False, **data):
        if cherrypy.request.method == 'POST':
            if cancel:
                raise cherrypy.HTTPRedirect('/')
            # TODO: validate the input data!
            submission = Submission(**data)
            self.data.append(submission)
            raise cherrypy.HTTPRedirect('/')

        tmpl = loader.load('submit.html')
        stream = tmpl.generate()
        return stream.render('html', doctype='html')
}}}

And of course we'll need to ad a template to display the submission form. In `geddit/templates`, create a file named `submit.html`, with the following content:

{{{
#!genshi
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/">
  <head>
    <title>Geddit: Submit new link</title>
  </head>
  <body>
    <h1>Geddit</h1>

    <h2>Submit new link</h2>
    <form action="" method="post">
      <table summary=""><tr>
        <th><label for="username">Your name:</label></th>
        <td><input type="text" id="username" name="username" /></td>
      </tr><tr>
        <th><label for="url">Link URL:</label></th>
        <td><input type="text" id="url" name="url" /></td>
      </tr>
      <tr>
        <th><label for="title">Title:</label></th>
        <td><input type="text" name="title" /></td>
      </tr></table>
      <div>
        <input type="submit" value="Submit" />
        <input type="submit" name="cancel" value="Cancel" />
      </div>
    </form>

  </body>
</html>
}}}

Now, if you click on the “Submit new link” link on the start page, you should see the submission form. You can enter values in the form fields and submit the form, which will take you back to the start page, where you'll see that your link has been added to the list. Or you can click on the “Cancel” button, which will take you back to the start page, but not add a link.

Please note though that we're not performing ''any'' kind of validation on the input, and that's of course a bad thing. So let's add validation next.

== Adding Form Validation ==

We'll use [http://formencode.org/ FormEncode] to do the validation, but we'll keep it all fairly basic. Let's declare our form in a separate file, namely `geddit/form.py`, which will have the following content:

{{{
#!python
from formencode import Schema, validators


class SubmissionForm(Schema):
    username = validators.UnicodeString(not_empty=True)
    url = validators.URL(not_empty=True, add_http=True, check_exists=False)
    title = validators.UnicodeString(not_empty=True)
}}}

Now let's use that class in the `Root.submit()` method. First add the `from formencode import Invalid` and `from geddit.form import SubmissionForm` lines to the imports at the top of the file. Then, update the `submit()` method to match the following:

{{{
#!python
    @cherrypy.expose
    def submit(self, cancel=False, **data):
        if cherrypy.request.method == 'POST':
            if cancel:
                raise cherrypy.HTTPRedirect('/')
            form = SubmissionForm()
            try:
                data = form.to_python(data)
                submission = Submission(**data)
                self.data.append(submission)
                raise cherrypy.HTTPRedirect('/')
            except Invalid, e:
                errors = e.unpack_errors()
        else:
            errors = {}

        tmpl = loader.load('submit.html')
        stream = tmpl.generate(errors=errors)
        return stream.render('html', doctype='html')
}}}

As you can tell, we now only add the submitted link to our database when validation is successful: all fields need to be filled out, and the `url` field needs to contain a valid URL. If the submission is valid, we proceed as before. If it is not valid, we render the submission form template again, passing it the dictionary of validation errors. Let's modify the `submit.html` template so that it displays those error messages:

{{{
#!genshi
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/">
  <head>
    <title>Geddit: Submit new link</title>
  </head>
  <body>
    <h1>Geddit</h1>

    <h2>Submit new link</h2>
    <form action="" method="post">
      <table summary=""><tr>
        <th><label for="username">Your name:</label></th>
        <td>
          <input type="text" id="username" name="username" />
          <span py:if="'username' in errors" class="error">${errors.username}</span>
        </td>
      </tr><tr>
        <th><label for="url">Link URL:</label></th>
        <td>
          <input type="text" id="url" name="url" />
          <span py:if="'url' in errors" class="error">${errors.url}</span>
        </td>
      </tr>
      <tr>
        <th><label for="title">Title:</label></th>
        <td>
          <input type="text" name="title" />
          <span py:if="'title' in errors" class="error">${errors.title}</span>
        </td>
      </tr></table>
      <div>
        <input type="submit" value="Submit" />
        <input type="submit" name="cancel" value="Cancel" />
      </div>
    </form>

  </body>
</html>
}}}

So now, if you submit the form without enterering a title, and having entered an invalid URL, you'd see something like the following:

[[Image(tutorial02.png)]]

But there's a problem here: Note how the input values have vanished from the form! We'd have to repopulate the form manually from the data submitted so far. We could do that by adding the required `value=""` attributes to th text fields in the template, but Genshi provides a more elegant way: the `HTMLFormFiller` steam filter. Given a dictionary of values, it can automatically populate HTML forms in the template output stream.

To enable this functionality, first you'll need to add the import `from genshi.filters import HTMLFormFiller` to the `genshi/controller.py` file. Next, update the bottom lines of the `Root.submit()` method implementation so that they look as follows:

{{{
#!python
        tmpl = loader.load('submit.html')
        stream = tmpl.generate(errors=errors) | HTMLFormFiller(data=data)
        return stream.render('html', doctype='html')
}}}

Now, all entered values are preserved when validation errors occur.