Preexisting bug and 1.20 regression in bitstring for generator with reduce/unique

[lukaszsamson]

Existing issue

• I have searched existing issues and could not find a duplicate.

Elixir and Erlang/OTP versions

Erlang/OTP 28 [erts-16.4.0.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]

Elixir 1.20.1, 1.20.0 (compiled with Erlang/OTP 28) - regression
all supported versions - preexisting bug

Operating system

any

Current behavior

For bitstring generator terminates prematurely and looses matching chunks when uniq/reduce option is used

1.20 regression

Repro:

    bin = <<0, 1, 9, 2, 0, 3>>
    IO.inspect for(<<0::8, x::8 <- bin>>, do: x)
    IO.inspect for(<<0::8, x::8 <- bin>>, uniq: true, do: x)
    IO.inspect for(<<0::8, x::8 <- bin>>, into: %{}, do: {x, x})
    r = for(<<0::8, x::8 <- bin>>, reduce: []) do acc -> [x | acc] end
    IO.inspect r
    IO.inspect for(<<"k=", v::8, ";" <- "k=1;k=2;j=9;k=3;">>, uniq: true, do: v - ?0)

Observed on 1.20.1:

    [1, 3]
    [1]
    %{1 => 1}
    [1]
    [1, 2]

vs on 1.19.5:

[1, 3]
[1, 3]
%{1 => 1, 3 => 3}
[3, 1]
[1, 2, 3]

Notice the <<0, 3>> binary chunk is skipped on 1.20 and unique/reduced results are missing 3

Regression introduced in 8fdef5c

Likely reason:
Size is abstract form not an integer and this clause never matches

elixir/lib/elixir/src/elixir_erl_for.erl

Line 323 in 8032f1d

no_var_size(Size) when is_integer(Size) -> Size;

and expressions like x::size(8) throw unbounded_size

Preexisting bug

When size is passed as var the behaviour was broken before 1.20. I reproduced this on 1.16-1.20

k = 8
bin = <<0, 1, 9, 2, 0, 3>>
for(<<0::8, x::size(^k) <- bin>>, uniq: false, do: x)
[1, 3]
for(<<0::8, x::size(^k) <- bin>>, uniq: true, do: x)
[1]
for(<<0::8, x::size(k) <- bin>>, uniq: true, do: x)
[1]
for(<<0::8, x::size(8) <- bin>>, uniq: true, do: x)
[1, 3]

so for variable size it likely never worked

Expected behavior

Valid size expression variants should work in for generator