Changes between Version 38 and Version 39 of GenshiFaq

Timestamp:

Jun 5, 2007, 11:58:41 PM (17 years ago)

Author:

cmlenz

Comment:

Add Q/A on unicode, inspired by #124

GenshiFaq

@@ -156 +156 @@
  3. you have `<pre>` or `<textarea>` tags that are embedded as opaque `Markup` instances, in which case Genshi never actually sees the tags (see #78)
 
+=== Why does Genshi raise a `UnicodeDecodeError` when I try to render non-ASCII strings? ===
+
+Genshi refuses to work with bytestrings that do not use the system default encoding, which on most systems is ASCII. This behavior is by design. If you want to render strings that contain characters outside of the ASCII range (or don't use the system default encoding), you will have to pass `unicode` objects to Genshi.
+
+This doesn't mean that Genshi has bad support for non-ASCII strings. Quite the contrary -- Genshi works beautifully with unicode. It just requires that you do the decoding of your bytestrings before passing the data to the templates. If this data comes from a database, it's likely that your database connector provides an option to automatically decode any data to unicode. SQLAlchemy, for example, provides a `convert_unicode` flag you can specify when [http://www.sqlalchemy.org/docs/dbengine.html#dbengine_options creating an engine]. Set that flag to `True`, and you get rid of a whole class of potentially nasty problems.
+
+For more information on unicode and using unicode in Python applications, see:
+ * [http://www.joelonsoftware.com/articles/Unicode.html The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)] by Joel Spolsky
+ * [http://www.amk.ca/python/howto/unicode Unicode HOWTO] by A.M. Kuchling
+ * [http://pylonshq.com/docs/0.9.5/internationalization.html#understanding-unicode Internationalization, Localization and Unicode] in the Pylons documentation
+
 === Can I add custom directives? ===
 
 Yes and no. Technically, it is possible to add custom directives by making a subclass of `MarkupTemplate` or `TextTemplate`. However, think twice before doing that. Genshi has been designed to get by with a standard set of generic directives; adding additional ones should not be necessary.
 
-Match templates provide much of the functionality you may want to get from custom directives, and are more convenient to boot. For example, let's assume you want a widget library. Instead of writing your own template directives, add an include file defining a couple match templates. As a simple example, assume the following is stored a file called `widgets.html`:
+Match templates provide much of the functionality you may want to get from custom directives, and are more convenient to boot. For example, let's assume you want a widget library. Instead of writing your own template directives, add an include file defining a couple match templates. As a simple example, assume the following is stored in a file called `widgets.html`:
 
 {{{