Detect recursive includes

[cmlenz]

Recursive includes (foo.html including foo.html, or foo.html including bar.html including foo.html) should be detected and reported as an error. Currently, they result in an infinite loop.

[Carsten Klein <carsten.klein@…>]

Attached you find an initial patch that would solve the problem.

There are still some issues to be resolved, though.

Scenario:

In trac's site.html the user recursively includes layout.html. While this is not your common usecase, it will still have to be detected by the system.

And, with the provided patch it does, however, there are two different stack traces, one in the trac.log which is correct, and the one that you get in the browser, which is incorrect.

Here are the two stack traces, trac.log first

2010-06-07 20:52:46,815 Trac[main] ERROR: Internal Server Error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/web/main.py", line 515, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/web/main.py", line 257, in dispatch
    content_type)
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/web/chrome.py", line 834, in render_template
    stream = template.generate(**data)
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 553, in generate
    stream = self.stream
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 459, in stream
    self._stream = list(self._prepare(self._stream))
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 495, in _prepare
    yield kind, (directives, list(substream)), pos
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 512, in _prepare
    for event in tmpl.stream:
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 459, in stream
    self._stream = list(self._prepare(self._stream))
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 495, in _prepare
    yield kind, (directives, list(substream)), pos
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 512, in _prepare
    for event in tmpl.stream:
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 459, in stream
    self._stream = list(self._prepare(self._stream))
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 495, in _prepare
    yield kind, (directives, list(substream)), pos
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/base.py", line 511, in _prepare
    include_pos=pos)
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/loader.py", line 232, in load
    backtrace[-1][0], *include_pos[1:])
TemplateSyntaxError: Recursive inclusion of "layout.html" detected. Backtrace: "[('/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/wiki/templates/wiki_view.html', (-1, -1)), (u'/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/templates/layout.html', ('wiki_view.html', 10, 35)), (u'/storages/htroots/sites/apache/trac-extensions.local.axn-software.de/templates/site.html', (u'layout.html', 62, 46))]" (/storages/htroots/sites/apache/trac-extensions.local.axn-software.de/templates/site.html, line 6)

and the output in the browser

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/web/api.py", line 436, in send_error
    data, 'text/html')
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/web/chrome.py", line 830, in render_template
    template = self.load_template(filename, method='xml')
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/web/chrome.py", line 799, in load_template
    return self.templates.load(filename, cls=cls)
  File "/storages/workspaces/axn-software.de/products/trac-extensions/genshi-0.6dev/genshi/template/loader.py", line 232, in load
    backtrace[-1][0], *include_pos[1:])
TemplateSyntaxError: Recursive inclusion of "error.html" detected. Backtrace: "[('/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/wiki/templates/wiki_view.html', (-1, -1)), (u'/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/templates/layout.html', ('wiki_view.html', 10, 35)), (u'/storages/htroots/sites/apache/trac-extensions.local.axn-software.de/templates/site.html', (u'layout.html', 62, 46)), ('/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/templates/error.html', (-1, -1))]" (/usr/local/lib/python2.6/dist-packages/Trac-0.12dev_r9566-py2.6.egg/trac/templates/error.html, line -1)

I am still investigating the issue, but until now I have not found a working solution to the problem.

[Carsten Klein <carsten.klein@…>]

initial proposal. TODO text templates, cleanup and remaining issues

[Carsten Klein <carsten.klein@…>]

Other issues with the provided patch:

wiki_view.html includes list_of_attachments.html and attach_file_form.html. list_of... in turn includes attach_file_form.html

This is incorrectly detected as a recursive inclusion right now.

[Carsten Klein <carsten.klein@…>]

See the attached patch which eliminates the problem of attach_file_form.html being reported for having been recursively included.

The problem was that on instantiation of the template the backtrace list was not copied.

[Carsten Klein <carsten.klein@…>]

revised patch. eliminates issue with the backtrace list not being copied.

[Carsten Klein <carsten.klein@…>]

With the latest patch the original error as shown in comment 1 is also eliminated.

Now it is cleanup time. Especially the RecursiveInclusionDetectedError? should be used so that clients of genshi can render these errors in a more appropriate fashion, e.g. provide source reference links and so on.

[Carsten Klein <carsten.klein@…>]

Some cleanup, now uses RecursiveInclusionError? exception, text template is still a TODO

[Carsten Klein <carsten.klein@…>]

Fixes issues with templates that cannot be found on the search path that were introduced during refactoring of the default template loader's load() method

[Carsten Klein <carsten.klein@…>]

Forgot to also change the text templates. However, text templates are still untested, if someone would test this would be most kind.

[Carsten Klein <carsten.klein@…>]

BTW this patch might also provide a solution to the problem described in #328, as from now on, the absolute filepaths will be used for looking up and storing templates in the cache.

[hodgestar]

Moved to milestone 0.9.