Skip to content

Resolve user function modules in CompiledRunner (#145) - #204

Open
RaphaelFakhri wants to merge 1 commit into
cloud-custodian:mainfrom
RaphaelFakhri:fix-compiled-runner-functions
Open

RaphaelFakhri wants to merge 1 commit into
cloud-custodian:mainfrom
RaphaelFakhri:fix-compiled-runner-functions

Conversation

@RaphaelFakhri

Copy link
Copy Markdown

Fixes #145.

Problem

CompiledRunner transpiles a call to a user-supplied function as module.qualname, for example __main__.lower(...). Transpiler.evaluate() runs the code with the globals of celpy.evaluation, which only know the modules that celpy.evaluation imports. A function defined in any other module, such as __main__ or an application package, fails with:

CELEvalError: ('evaluation error', ..., ('undeclared reference to "name \'__main__\' is not defined" ...

The same expression works with InterpretedRunner.

Change

This revives the approach from #146 (closed because of failing checks), with these adjustments:

  • Transpiler.__init__() collects the top-level package of each function's module from sys.modules. It maps _operator to operator the same way Phase1Transpiler.func_name() does.
  • Transpiler.evaluate() executes the code with a copy of the celpy.evaluation globals plus those packages. Existing module globals take precedence, so a user package can't shadow a name that the transpiled code depends on.
  • Type annotations satisfy mypy --strict.

Functions defined with lambda or nested inside another function still can't be referenced by name. That limitation is separate from this fix.

Testing

  • New test test_transpile_function_from_other_module in tests/test_transpilation.py. It registers a function in a module that celpy.evaluation never imports and evaluates device.lower() == 'tablet' with CompiledRunner. The test fails without the change (NameError for the module name) and passes with it.
  • pytest tests: 433 passed, 1 skipped, 100% coverage.
  • pytest --doctest-modules src README.rst: 11 passed.
  • behave --tags=@conformance --tags=~@wip, with both the interpreted and the compiled runner: 1127 scenarios passed, 0 failed.
  • mypy --strict, ruff check src tools and ruff format --diff are clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CompiledRunner Functions Bug

1 participant