KeyError when using [ or ] in the translatable string

[palgarvio]

While using genshi's i18n filter on some "complicated" markup I got a KeyError:

More info with the  traceback and the  html snippet.

[palgarvio]

The cause appears to be the [ and ] on the text.

[palgarvio]

Possible fix:

genshi/filters/i18n.py

@@ -378 +378 @@
             self.values[param] = (kind, data, pos)
         else:
             if kind is START:
-                self.string.append(u'[%d:' % self.order)
+                self.string.append(u'<[%d:' % self.order)
                 self.events.setdefault(self.order, []).append((kind, data, pos))
                 self.stack.append(self.order)
                 self.depth += 1
@@ -387 +387 @@
                 self.depth -= 1
                 if self.depth:
                     self.events[self.stack[-1]].append((kind, data, pos))
-                    self.string.append(u']')
+                    self.string.append(u']>')
                     self.stack.pop()
 
     def format(self):
@@ -421 +421 @@
                         break
 
 
-def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|\]')):
+def parse_msg(string, regex=re.compile(r'(?:<\[(\d+)\:)|\]>')):
     """Parse a translated message using Genshi mixed content message
     formatting.
 

[cmlenz]

Thank you!

Regardless of whether we switch to different tokens (<[…}> instead of just […]) to reduce the potential for conflicts with actual symbols in the message, those tokens will need to be escaped in some form if they do occur in the message itself.

[palgarvio]

Escaping as in \[...\}?

[palgarvio]

Dam, can't get this ticket to the point where I can add ppl to CC :\

I'll see if I can get cboos and asmodai following this ticket too because we need opinions and decisions on which format we'll follow, we need to get advanced-i18n into trunk.

[cboos]

I might be wrong, but if the token is indeed <[ as you originally suggested, then it can't appear in th original text (because the < character itself can't appear in the text extracted from the markup, no?). So in this case, no need for the escaping mentioned in comment:4.

[palgarvio]

Replying to cboos:

I might be wrong, but if the token is indeed <[ as you originally suggested, then it can't appear in th original text (because the < character itself can't appear in the text extracted from the markup, no?). So in this case, no need for the escaping mentioned in comment:4.

Well, at first glance, you're right, entities extracted from the template like < must be an HTML entity, or else Genshi will complain, so in this case no escaping is needed. However, > can appear in the Markup, so, this one will need to be escaped.

So, let's have a vote, which markup should we adopt:

• <[ ... ]>
• <{ ... }>
• <( ... )>
• {( ... )}
• {> ... <}
• (} ... {)
• {[ ... ]}
• ([ ... ])

Got any more suggestions?

[palgarvio]

How about ⌈ ... ⌋ which we could also make genshi output the html entity of it, ie, &lceil; ... &rfloor;?

Or even ⌈⌋ ... ⌈⌋ to avoid potential conflicts...

Or 〉 ... 〈

But, with these, either the user copy/paste's or we have to accept both the char and it's corresponding html entity.

[palgarvio]

Fixed in [1058]. Genshi automatically escapes [ and ] now.