Edgewall Software

Opened 15 years ago

Closed 14 years ago

#388 closed enhancement (invalid)

Allow custom loaders to also plug in custom AstTransformers

Reported by: carsten.klein@… Owned by: cmlenz
Priority: trivial Milestone: 0.7
Component: General Version: 0.6
Keywords: Cc:

Description

The current API does not allow templates to override the default AstTransformer? defined in astutil#_compile.

However, classes derived from Code do allow overriding the default transformer.

It would be nice to have a custom loader also incorporate a custom AstTransformer?.

The main concern here is security, effectfully limiting python API access by a custom AstTransformer?.

As I see it, the markup template's init method would require an additional named parameter that by default would be set to none, e.g. xform=None.

That way, upon instantiation of either Suite or Expression, the specified AstTransformer? could be passed as an alternative the the default transformer.

This would also be backwards compatible, so existing applications would not break.

Change History (3)

comment:1 follow-up: Changed 15 years ago by carsten.klein@…

Here is an untested patch, but it seems quite straight forward and should do the trick:

  • genshi/template/base.py

     
    2020
    2121from genshi.core import Attrs, Stream, StreamEventKind, START, TEXT, _ensure
    2222from genshi.input import ParseError
     23from genshi.template.astutil import ASTTransformer
    2324
    2425__all__ = ['Context', 'DirectiveFactory', 'Template', 'TemplateError',
    2526           'TemplateRuntimeError', 'TemplateSyntaxError', 'BadDirectiveError']
    … …  
    370371    _number_conv = unicode # function used to convert numbers to event data
    371372
    372373    def __init__(self, source, filepath=None, filename=None, loader=None,
    373                  encoding=None, lookup='strict', allow_exec=True):
     374                 encoding=None, lookup='strict', allow_exec=True, xform=None):
    374375        """Initialize a template from either a string, a file-like object, or
    375376        an already parsed markup stream.
    376377       
    … …  
    386387                       default), "lenient", or a custom lookup class
    387388        :param allow_exec: whether Python code blocks in templates should be
    388389                           allowed
    389        
     390        :param xform: the AST transformer class that should be applied to the
     391                      code representing executable code; if `None`, the default
     392                      transformation is chosen depending on that mode.
     393
    390394        :note: Changed in 0.5: Added the `allow_exec` argument
    391395        """
    392396        self.filepath = filepath or filename
    … …  
    394398        self.loader = loader
    395399        self.lookup = lookup
    396400        self.allow_exec = allow_exec
     401
     402        if xform is not None and not ASTTransformer in xform.__mro__:
     403            raise TemplateRuntimeError("xform must inherit from "
     404                                       "ASTTransformer.")
     405        self.xform = xform
    397406        self._init_filters()
    398407        self._prepared = False
    399408

  • genshi/template/markup.py

     
    6161    _number_conv = Markup
    6262
    6363    def __init__(self, source, filepath=None, filename=None, loader=None,
    64                  encoding=None, lookup='strict', allow_exec=True):
     64                 encoding=None, lookup='strict', allow_exec=True, xform=None):
    6565        Template.__init__(self, source, filepath=filepath, filename=filename,
    6666                          loader=loader, encoding=encoding, lookup=lookup,
    67                           allow_exec=allow_exec)
     67                          allow_exec=allow_exec, xform=xform)
    6868        self.add_directives(self.DIRECTIVE_NAMESPACE, self)
    6969
    7070    def _init_filters(self):
    … …  
    9494                    raise TemplateSyntaxError('Python code blocks not allowed',
    9595                                              self.filepath, *pos[1:])
    9696                try:
     97                    effectiveXform = None
     98                    if self.xform is not None:
     99                        effectiveXform = {'exec': self.xform}
    97100                    suite = Suite(data[1], self.filepath, pos[1],
    98                                   lookup=self.lookup)
     101                                  lookup=self.lookup, xform=effectiveXform)
    99102                except SyntaxError, err:
    100103                    raise TemplateSyntaxError(err, self.filepath,
    101104                                              pos[1] + (err.lineno or 1) - 1,

  • genshi/template/text.py

     
    199199                    raise TemplateSyntaxError('Python code blocks not allowed',
    200200                                              self.filepath, lineno)
    201201                try:
     202                    effectiveXform = None
     203                    if self.xform is not None:
     204                        effectiveXform = {'exec': self.xform}
    202205                    suite = Suite(value, self.filepath, lineno,
    203                                   lookup=self.lookup)
     206                                  lookup=self.lookup, xform=effectiveXform)
    204207                except SyntaxError, err:
    205208                    raise TemplateSyntaxError(err, self.filepath,
    206209                                              lineno + (err.lineno or 1) - 1)

comment:2 in reply to: ↑ 1 Changed 15 years ago by Carsten Klein <carsten.klein@…>

Replying to carsten.klein@…:

Here is a revised version of the patch that allows passing in ast transformers for use with both expression transformations and embedded python code transformations. For this, it add the xform parameter to the interpolation routine, so that user defined AST transformers can be used with this as well. It also provides a revised documentation on the additional xform parameter in the template base class:

  • genshi/template/base.py

     
    2020
    2121from genshi.core import Attrs, Stream, StreamEventKind, START, TEXT, _ensure
    2222from genshi.input import ParseError
     23from genshi.template.astutil import ASTTransformer
    2324
    2425__all__ = ['Context', 'DirectiveFactory', 'Template', 'TemplateError',
    2526           'TemplateRuntimeError', 'TemplateSyntaxError', 'BadDirectiveError']
    … …  
    370371    _number_conv = unicode # function used to convert numbers to event data
    371372
    372373    def __init__(self, source, filepath=None, filename=None, loader=None,
    373                  encoding=None, lookup='strict', allow_exec=True):
     374                 encoding=None, lookup='strict', allow_exec=True, xform=None):
    374375        """Initialize a template from either a string, a file-like object, or
    375376        an already parsed markup stream.
    376377       
    … …  
    386387                       default), "lenient", or a custom lookup class
    387388        :param allow_exec: whether Python code blocks in templates should be
    388389                           allowed
     390        :param xform: a dict in the form of
    389391       
     392                      { 'eval' : <class>, 'exec' : <class> }
     393                     
     394                      where <class> is the respective AST transformer class that
     395                      should be applied to the code representing either a python
     396                      expression or executable python code; if `None`, or either
     397                      of the two is omitted, then the default transformation is
     398                      chosen depending on the current execution mode, which is
     399                      either 'eval' or 'exec'.
     400
    390401        :note: Changed in 0.5: Added the `allow_exec` argument
     402        :note: Changed in 0.7: Added the `xform` argument
    391403        """
    392404        self.filepath = filepath or filename
    393405        self.filename = filename
    394406        self.loader = loader
    395407        self.lookup = lookup
    396408        self.allow_exec = allow_exec
     409
     410        if xform is not None :
     411            if 'eval' in xform and not ASTTransformer in xform['eval'].__mro__:
     412                raise TemplateRuntimeError("xform['eval'] must inherit from "
     413                                           "ASTTransformer.")
     414            if 'exec' in xform and not ASTTransformer in xform['exec'].__mro__:
     415                raise TemplateRuntimeError("xform['exec'] must inherit from "
     416                                           "ASTTransformer.")
     417
     418        self.xform = xform
    397419        self._init_filters()
    398420        self._prepared = False
    399421

  • genshi/template/markup.py

     
    6161    _number_conv = Markup
    6262
    6363    def __init__(self, source, filepath=None, filename=None, loader=None,
    64                  encoding=None, lookup='strict', allow_exec=True):
     64                 encoding=None, lookup='strict', allow_exec=True, xform=None):
    6565        Template.__init__(self, source, filepath=filepath, filename=filename,
    6666                          loader=loader, encoding=encoding, lookup=lookup,
    67                           allow_exec=allow_exec)
     67                          allow_exec=allow_exec, xform=xform)
    6868        self.add_directives(self.DIRECTIVE_NAMESPACE, self)
    6969
    7070    def _init_filters(self):
    … …  
    8686
    8787            if kind is TEXT:
    8888                for kind, data, pos in interpolate(data, self.filepath, pos[1],
    89                                                    pos[2], lookup=self.lookup):
     89                                                   pos[2], lookup=self.lookup,
     90                                                   xform=self.xform):
    9091                    stream.append((kind, data, pos))
    9192
    9293            elif kind is PI and data[0] == 'python':
    … …  
    9596                                              self.filepath, *pos[1:])
    9697                try:
    9798                    suite = Suite(data[1], self.filepath, pos[1],
    98                                   lookup=self.lookup)
     99                                  lookup=self.lookup, xform=self.xform)
    99100                except SyntaxError, err:
    100101                    raise TemplateSyntaxError(err, self.filepath,
    101102                                              pos[1] + (err.lineno or 1) - 1,
    … …  
    278279                for name, value in attrs:
    279280                    if value:
    280281                        value = list(interpolate(value, self.filepath, pos[1],
    281                                                  pos[2], lookup=self.lookup))
     282                                                 pos[2], lookup=self.lookup,
     283                                                 xform=self.xform))
    282284                        if len(value) == 1 and value[0][0] is TEXT:
    283285                            value = value[0][1]
    284286                    new_attrs.append((name, value))

  • genshi/template/text.py

     
    179179            if start > offset:
    180180                text = _escape_sub(_escape_repl, source[offset:start])
    181181                for kind, data, pos in interpolate(text, self.filepath, lineno,
    182                                                    lookup=self.lookup):
     182                                                   lookup=self.lookup,
     183                                                   xform=self.xform):
    183184                    stream.append((kind, data, pos))
    184185                lineno += len(text.splitlines())
    185186
    … …  
    189190            if command == 'include':
    190191                pos = (self.filename, lineno, 0)
    191192                value = list(interpolate(value, self.filepath, lineno, 0,
    192                                          lookup=self.lookup))
     193                                         lookup=self.lookup, xform=self.xform))
    193194                if len(value) == 1 and value[0][0] is TEXT:
    194195                    value = value[0][1]
    195196                stream.append((INCLUDE, (value, None, []), pos))
    … …  
    200201                                              self.filepath, lineno)
    201202                try:
    202203                    suite = Suite(value, self.filepath, lineno,
    203                                   lookup=self.lookup)
     204                                  lookup=self.lookup, xform=self.xform)
    204205                except SyntaxError, err:
    205206                    raise TemplateSyntaxError(err, self.filepath,
    206207                                              lineno + (err.lineno or 1) - 1)
    … …  
    228229        if offset < len(source):
    229230            text = _escape_sub(_escape_repl, source[offset:])
    230231            for kind, data, pos in interpolate(text, self.filepath, lineno,
    231                                                lookup=self.lookup):
     232                                               lookup=self.lookup,
     233                                               xform=self.xform):
    232234                stream.append((kind, data, pos))
    233235
    234236        return stream
    … …  
    289291            if start > offset:
    290292                text = source[offset:start]
    291293                for kind, data, pos in interpolate(text, self.filepath, lineno,
    292                                                    lookup=self.lookup):
     294                                                   lookup=self.lookup,
     295                                                   xform=self.xform):
    293296                    stream.append((kind, data, pos))
    294297                lineno += len(text.splitlines())
    295298
    … …  
    324327        if offset < len(source):
    325328            text = source[offset:].replace('\\#', '#')
    326329            for kind, data, pos in interpolate(text, self.filepath, lineno,
    327                                                lookup=self.lookup):
     330                                               lookup=self.lookup,
     331                                               xform=self.xform):
    328332                stream.append((kind, data, pos))
    329333
    330334        return stream

  • genshi/template/interpolation.py

     
    3737))
    3838
    3939
    40 def interpolate(text, filepath=None, lineno=-1, offset=0, lookup='strict'):
     40def interpolate(text, filepath=None, lineno=-1, offset=0, lookup='strict',
     41                xform=None):
    4142    """Parse the given string and extract expressions.
    4243   
    4344    This function is a generator that yields `TEXT` events for literal strings,
    … …  
    5859                   (optional)
    5960    :param lookup: the variable lookup mechanism; either "lenient" (the
    6061                   default), "strict", or a custom lookup class
     62    :param xform:  a dict containing class references to AST transformers being
     63                   applied on evaluation of expressions.
    6164    :return: a list of `TEXT` and `EXPR` events
    6265    :raise TemplateSyntaxError: when a syntax error in an expression is
    6366                                encountered
    … …  
    7578            if chunk:
    7679                try:
    7780                    expr = Expression(chunk.strip(), pos[0], pos[1],
    78                                       lookup=lookup)
     81                                      lookup=lookup, xform=xform)
    7982                    yield EXPR, expr, tuple(pos)
    8083                except SyntaxError, err:
    8184                    raise TemplateSyntaxError(err, filepath, pos[1],

comment:3 Changed 14 years ago by Carsten Klein <carsten.klein@…>

  • Resolution set to invalid
  • Status changed from new to closed

I am closing this an reopening it. Inlining the patches was not a good idea.

Note: See TracTickets for help on using tickets.