Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add builtin functions _module_import, _module_using #57965

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

xal-0
Copy link
Member

@xal-0 xal-0 commented Mar 31, 2025

Overview

Currently, import and using statements are compiled into calls to
jl_toplevel_eval with the Expr as an argument. It can be useful in an
interactive environment to do import/using programmatically, for which
eval(Expr(:import, ...)) or a macro is the only option at the moment (see for
example InteractiveUtils.@activate).

This PR adds two builtins, _module_import and _module_using, with these
signatures, and removes Expr(:import/:using, ...) from the lowered IR:

_module_import(explicit::Bool, to::Module, ctx::Union{Expr, Nothing}, names::Expr...)
_module_using(to::Module, from::Expr{Symbol})

Lowering example

import statements and using X: statements are lowered to _module_import
like so:

import A             => _module_import(true,  Main, nothing,          Expr(:., :A))
import A.b           => _module_import(true,  Main, nothing,          Expr(:., :A, :b))
import A.b as c      => _module_import(true,  Main, nothing,          Expr(:as, Expr(:., :A, :b), :c))
import A.B: C.d, e   => _module_import(true,  Main, Expr(:., :A, :B), Expr(:., :C, :d), Expr(:., :e))
import A.B: C.d as e => _module_import(true,  Main, Expr(:., :A, :B), Expr(:as, Expr(:., :C, :d), :e))
using  A.B: C.d, e   => _module_import(false, Main, Expr(:., :A, :B), Expr(:., :C, :d), Expr(:., :e))

Plain using statements become _module_using:

using A.B            => _module_using(Main, Expr(:., :A, :B))

Multiple comma-separated using or import paths are lowered to multiple calls to the appropriate builtin:

julia> Meta.@lower using A.B, C
:($(Expr(:thunk, CodeInfo(
1 ─       builtin Core._module_using(Main, $(QuoteNode(:($(Expr(:., :A, :B))))))
│         builtin Core._module_using(Main, $(QuoteNode(:($(Expr(:., :C))))))
│       $(Expr(:latestworld))
└──     return nothing
))))

Alternative designs

Some alternatives to these builtins are worth considering.

The lowering pass could do even less work by inserting a call to
_module_import or _module_using that takes a single Expr argument for the
entire statement.

Another option is to do even more in lowering, introducing very low-level
builtins that wrap jl_module_import and jl_module_using directly. The
lowering pass would expand the import path into direct calls to Base.require,
Base.getproperty, etc. For example, import A.B: C.d, e could become
something like:

(block
  (= (ssavalue 1)
     (call (top getproperty)
           (call (top require) (thismodule) 'A)
           'B))
  (call (core module_import)
        (thismodule)
        (call (top getproperty) (ssavalue 1) 'C)
        'd)
  (call (core module_import) (thismodule) (ssavalue 1) 'd)
  (latestworld))

I like the idea of supporting calls to _module_import with a module for
ctx instead of an Expr, but this could be added to the proposed design too.

Removes the lowered IR forms Expr(:using, ...) and Expr(:import, ...).  `import`
and `using` expressions are now lowered to calls to `_module_import` like so:

  import A, B.c

    (block
      (call (core _module_import) (true) (thismodule) (null) (inert (|.| A)))
      (call (core _module_import) (true) (thismodule) (null) (inert (|.| B c)))
      (latestworld))

  import A.B: C.d, e as f

    (block
      (call (core _module_import) (true) (thismodule)
            (inert (|.| A B))
            (inert (|.| C d))
            (inert (as (|.| e) f))))

  using A.B: C.d

    (block
      (call (core _module_import) (false) (thismodule)
            (inert (|.| A B))
            (inert (|.| C d)))
      (latestworld))

Each comma-separated import is lowered to a call to the new _module_import
builtin, which takes an "explicit" flag (`using X:` is lowered to _module_import
with explicit=false), a "context" path (set to `nothing` for `import` without
`:`), and any number of Exprs of the form `(|.| ...)` or `(as (|.| ...) x)`
representing import paths under the context path.

Without `:`, using is lowered to the other new builtin:

    using A.B

    (block
      (call (core _module_using) (thismodule) (inert (|.| A B)))
      (latestworld))
@xal-0 xal-0 added compiler:lowering Syntax lowering (compiler front end, 2nd stage) modules labels Mar 31, 2025
@LilithHafner LilithHafner requested a review from topolarity March 31, 2025 17:32
@Keno
Copy link
Member

Keno commented Apr 1, 2025

Generally positive on the direction.

For this particular implementation, I'll need to think about it in a bit more depth, but my initial impression is that this feels too "syntax oriented" for a builtin. I mean this in two senses. The first is that these builtins are taking Expr, which is a syntax data structure and not really touched anywhere else by the runtime system. The second is that it conflates the resolution of the name to be imported with the establishment of the link between them. I'd have expected something closer to:

_module_import(::Binding, ::Binding, imported::Bool)
_module_using(::Module, ::Module)

From this, there could then be higher level functions or macros that implement the syntax parts. The primary downside I see is that this approach would prevent batching of the world age update. However, I think that is possibly better handled with a generic world agree compression optimization in the runtime rather than trying to design the syntax around it.

@vtjnash
Copy link
Member

vtjnash commented Apr 1, 2025

Agreed as to our eventual plan here. I think there are few phases, one (here) is keeping most of the existing code structure, but introducing a function form for it. Secondly we can move a lot of this C code into julia using some simpler builtins for the primitive bits. I think the tricky part of deciding on that form is dealing with the fact this is very much just interpreting the syntax, so it is hard to see what that julia function's arguments should be other than exactly the AST.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:lowering Syntax lowering (compiler front end, 2nd stage) modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants