Support Erlang/OTP 28 new features

[josevalim]

When Erlang/OTP 28+ is released:

• Deprecate Regex.recompile and related fields

When we require Erlang/OTP 28+ (so please do not send PRs for those):

• Remove code that conditionally compiles regexes
• Track if there is a loader optimization for new regexes
• Replace references to -erl_epmd_port with -kernel erl_epmd_node_listen_port
• Verify if IEx.start could be used to start it interactively
• No longer pass version: 2 to :sets module
• Consider adding end locations to function annotations (Add erl_anno:set_end_location/2 erlang/otp#8966)

[Eiji7]

Looks like due to changes in Regex they cannot be any longer stored within module attributes. This is because a reference is stored inside re_pattern key.

iex> Map.from_struct ~r/elxiir/
%{
  opts: [],
  re_pattern: {:re_pattern, 0, 0, 0, #Reference<0.2812101202.105775114.143208>},
  re_version: {"10.45 2025-02-05", :little},
  source: "elxiir"
}

While it's not used often it's still worth to mention it somewhere as this information is hidden behind a term() in typespec:

iex> t Regex.t
@type t() :: %Regex{
        opts: binary() | [term()],
        re_pattern: term(),
        re_version: term(),
        source: binary()
      }

Edit: I have some problems in ex_unit tests that prevents installing it via asdf on Erlang 28. Somehow :io.get_password() returns sometimes {:error, :enotsup}.

elixir/lib/ex_unit/test/ex_unit/capture_io_test.exs

Lines 184 to 186 in 51e160e

	capture_io(fn ->
	assert :io.get_password(Process.group_leader()) == :eof
	end)

Also one profile test is returning different data, so it needs to be changed as well:

elixir/lib/mix/test/mix/tasks/profile.tprof_test.exs

Lines 84 to 90 in 51e160e

	test "sorts based on memory per call", context do
	in_tmp(context.test, fn ->
	assert capture_io(fn ->
	Tprof.run(["--type", "memory", "--sort", "per_call", "-e", @expr])
	end) =~ ~r/\n:erlang\.integer_to_binary\/1.*\nEnum\.each\/2/s
	end)
	end

Edit: Everyone who uses rebar3 or simply some Erlang dependencies (like telemetry) may have this problem:

Could not find "rebar3", which is needed to build dependency :telemetry
Shall I install rebar3? (if running non-interactively, use "mix local.rebar --force") [Yn] 
* creating /home/eiji/.local/share/mix/elixir/1-18/rebar3
===> Using custom resource rebar_pkg_resource that implements a deprecated api. It should be upgraded to rebar_resource_v2.
=ERROR REPORT==== 21-May-2025::16:32:34.500793 ===
beam/beam_load.c(594): Error loading function rebar_pkg_resource:init/2: op bs_add p x i u x:
  please re-compile this module with an Erlang/OTP 28 compiler or update your Erlang/OTP version


=ERROR REPORT==== 21-May-2025::16:32:34.517241 ===
beam/beam_load.c(594): Error loading function rebar_prv_deps:source_text/1: op bs_append p i u u u x u x:
  please re-compile this module with an Erlang/OTP 28 compiler or update your Erlang/OTP version


===> Failed creating providers. Run with DIAGNOSTIC=1 for stacktrace or consult rebar3.crashdump.

To temporarily fix the issue you must install rebar3 manually following: https://rebar3.org/docs/getting-started/

cp ~/.cache/rebar3/bin/rebar3 $PATH_TO_MIX_XDG/elixir/1-18/

[Eiji7]

I've just got a notification about Elixir 1.18.4 release. 😅

TLDR;

I'm using Erlang 28.0 and tried to install new release using:

$ asdf install elixir ref:v1.18.4

Unfortunately this one does not looks like to be solved:

[Kernel] Recompile regexes when escaped from module attributes for Erlang/OTP 28 compatibility

Since I was not able to install Elixir the quick way I did not tested rebar3 change, but it's tests are failing:

[mix local.rebar] Install Hex per OTP release

More info

I did not get an error for :io.get_password/0 tests, but I have much more errors (related to``git_rebar` dependency).

The first test failing is the profiler test I mentioned before:

  1) test sorts based on memory per call (Mix.Tasks.Profile.TprofTest)
     test/mix/tasks/profile.tprof_test.exs:81
     Assertion with =~ failed
     code:  assert capture_io(fn -> Tprof.run(["--type", "memory", "--sort", "per_call", "-e", @expr]) end) =~
              ~r/\n:erlang\.integer_to_binary\/1.*\nEnum\.each\/2/s
     left:  "Warmup...\n\n\nProfile results of #PID<0.3348.0>\n#                           CALLS      % WORDS PER CALL\nTotal                           6 100.00    18     3.00\nEnum.each/2                     1  16.67     3     3.00\n:erlang.integer_to_binary/1     5  83.33    15     3.00\n\nProfile done over 2 matching functions\n"
     right: ~r/\n:erlang\.integer_to_binary\/1.*\nEnum\.each\/2/s
     stacktrace:
       (elixir 1.18.4) lib/file.ex:1665: File.cd!/2
       test/mix/tasks/profile.tprof_test.exs:82: (test)

The second failing test have the exact same problem with Regex in module attributes, see:

  2) test deps/1 parses Rebar dependencies (Mix.RebarTest)
     test/mix/rebar_test.exs:76
     Assertion with == failed
     code:  assert parse_dep({:git_rebar, ~c"0.1..*", {:git, @git_rebar_charlist, :main}}) ==
              {:git_rebar, ~r"0.1..*", override: true, git: @git_rebar_string, ref: "main"}
     left:  {
              :git_rebar,
              %Regex{
                opts: [],
                re_pattern: {
                  :re_pattern,
                  0,
                  0,
                  0,
                  #Reference<0.856069619.110231572.103394>
                },
                re_version: {"10.45 2025-02-05", :little},
                source: "0.1..*"
              },
              [
                override: true,
                git: "../../test/fixtures/git_rebar",
                ref: "main"
              ]
            }
     right: {
              :git_rebar,
              %Regex{
                opts: [],
                re_pattern: {
                  :re_pattern,
                  0,
                  0,
                  0,
                  #Reference<0.856069619.110231572.103395>
                },
                re_version: {"10.45 2025-02-05", :little},
                source: "0.1..*"
              },
              [
                override: true,
                git: "../../test/fixtures/git_rebar",
                ref: "main"
              ]
            }
     stacktrace:
       test/mix/rebar_test.exs:85: (test)

There are 3 more tests, but they fail because git_rebar3 dependency cannot be compiled (most probably due to previous failing test):

  3) test integration with Mix applies variables from :system_env option on config/compilation (Mix.RebarTest)
     test/mix/rebar_test.exs:222
     ** (Mix.Error) Could not compile dependency :git_rebar, "/home/eiji/.asdf/installs/elixir/ref-v1.18.4/lib/mix/tmp/.mix/elixir/1-18-otp-28/rebar3 bare compile --paths /home/eiji/.asdf/installs/elixir/ref-v1.18.4/lib/mix/tmp/applies variables from system_env/_build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile git_rebar --force", update it with "mix deps.update git_rebar" or clean it with "mix deps.clean git_rebar"
     code: in_tmp("applies variables from system_env", fn ->

  4) test integration with Mix gets and compiles dependencies with MIX_REBAR3 with spaces (Mix.RebarTest)
     test/mix/rebar_test.exs:239
     ** (Mix.Error) Could not compile dependency :git_rebar, "/home/eiji/.asdf/installs/elixir/ref-v1.18.4/lib/mix/tmp/rebar3 env with spaces/rebar3 bare compile --paths /home/eiji/.asdf/installs/elixir/ref-v1.18.4/lib/mix/tmp/rebar3 env with spaces/_build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile git_rebar --force", update it with "mix deps.update git_rebar" or clean it with "mix deps.clean git_rebar"
     code: in_tmp("rebar3 env with spaces", fn ->

  5) test integration with Mix gets and compiles dependencies (Mix.RebarTest)
     test/mix/rebar_test.exs:188
     ** (Mix.Error) Could not compile dependency :git_rebar, "/home/eiji/.asdf/installs/elixir/ref-v1.18.4/lib/mix/tmp/.mix/elixir/1-18-otp-28/rebar3 bare compile --paths /home/eiji/.asdf/installs/elixir/ref-v1.18.4/lib/mix/tmp/get and compile dependencies/_build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile git_rebar --force", update it with "mix deps.update git_rebar" or clean it with "mix deps.clean git_rebar"
     code: in_tmp("get and compile dependencies", fn ->

[josevalim]

Those have all been fixed in main and main's CI passes on Erlang/OTP 28. We have released 1.18.4 with preliminary support for Erlang/OTP 28 (fwiw, it should all work, I have verified that all failures are caused by tests themselves) but full support comes in v1.19.

[Eiji7]

@josevalim Unfortunately I cannot confirm it using asdf:

$ asdf install elixir ref:main
(…)
  1) test I/O protocol with get password (ExUnit.CaptureIOTest)
     test/ex_unit/capture_io_test.exs:183
     Assertion with == failed
     code:  assert :io.get_password() == :eof
     left:  {:error, :enotsup}
     right: :eof
     stacktrace:
       (ex_unit 1.19.0-dev) lib/ex_unit/capture_io.ex:319: ExUnit.CaptureIO.do_capture_gl/2
       (ex_unit 1.19.0-dev) lib/ex_unit/capture_io.ex:277: ExUnit.CaptureIO.do_with_io/3
       (ex_unit 1.19.0-dev) lib/ex_unit/capture_io.ex:146: ExUnit.CaptureIO.capture_io/1
       test/ex_unit/capture_io_test.exs:184: (test)
(…)

So I'm not able to compile both v1.18.4 tag and main branch using asdf and Erlang 28.0. Of course they are just tests, so binary releases should work … What worries me is above :enotsup error. Looks like it fails everytime or at least really often when testing main branch.

[josevalim]

You can see on main that we pass the group leader in the test:

https://github.com/elixir-lang/elixir/blob/main/lib/ex_unit/test/ex_unit/capture_io_test.exs#L185

But that argument is not in your snippet above. Which means your main is not pointing to the latest commit. Please make sure you are running on 51e160e or later.

[Eiji7]

You can see on main that we pass the group leader in the test:

Let me see … if that's so main.tar.xz needs to be really old (as this is how asdf is fetching main):
https://github.com/elixir-lang/elixir/archive/main.tar.gz

Anyway I downloaded it manually, there is of course no git information, but I can verify it by code, see:

$ sed -n 185p lib/ex_unit/test/ex_unit/capture_io_test.exs
        assert :io.get_password(Process.group_leader()) == :eof

so it's not that old, but it's just a start of surprises …

I'm unable to reproduce it no matter how much I tried to run those tests by hand. In my last try I even used a simple go file that calls an sh file that calls make clean test … in a sub-shell and even this works. I don't know … maybe it's some asdf implementation specific issue, but it fails every time I try to call asdf install elixir ref:main.

---

Anyway … I have tried both main archive and the main branch using git clone. As you can see I'm up to date:

$ git log -n 1
commit d636a0a5ccf605e06d9f21d3cf42dbbd70926e1e (HEAD -> main, tag: main-latest, origin/main, origin/HEAD)
Author: Dmitry Kleymenov <DukeFruitarian@gmail.com>
Date:   Wed May 21 17:50:48 2025 +0300

    Add clarity to docs and doctests in Enum (#14511)

but I still see the same issue that I mentioned here for 1.17.3 version …

  1) test sorts based on memory per call (Mix.Tasks.Profile.TprofTest)
     test/mix/tasks/profile.tprof_test.exs:84
     Assertion with =~ failed
     code:  assert capture_io(fn -> Tprof.run(["--type", "memory", "--sort", "per_call", "-e", @expr]) end) =~
              ~r/\n:erlang\.integer_to_binary\/1.*\nEnum\.each\/2/s
     left:  "Warmup...\n\n\nProfile results of #PID<0.3321.0>\n#                           CALLS      % WORDS PER CALL\nTotal                           6 100.00    18     3.00\nEnum.each/2                     1  16.67     3     3.00\n:erlang.integer_to_binary/1     5  83.33    15     3.00\n\nProfile done over 2 matching functions\n"
     right: ~r/\n:erlang\.integer_to_binary\/1.*\nEnum\.each\/2/s
     stacktrace:
       (elixir 1.19.0-dev) lib/file.ex:1689: File.cd!/2
       test/mix/tasks/profile.tprof_test.exs:85: (test)

switching Enum\.each\/2 and :erlang\.integer_to_binary\/1 part of right side fixes the issue.

---

However after manually compiling and installing Elixir to asdf directory I can confirm it and rebar3 works. I have only issues with dialyzer now, but that I have to investigate and report in another issue.

For now I would report back the :io.get_password problem to asdf-elixir project and see if they would have any idea.