make p-q illegal in @safe code

[WalterBright]

For 2024 edition.

discussion: https://www.digitalmars.com/d/archives/digitalmars/D/Should_p_-_q_be_disallowed_in_safe_code_380113.html

[dlang-bot]

Thanks for your pull request, @WalterBright!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#22348"

[Herringway]

Why is @safe being redefined? Why and how is this going into an edition two years ago instead of the 2026 edition? None of this makes any sense.

[WalterBright]

Why is @safe being redefined?

More like extended, for the purpose of detecting more bugs.

Why and how is this going into an edition two years ago instead of the 2026 edition?

Great question. The answer is we need to decide what year to make it!

[Herringway]

Why is @safe being redefined?

More like extended, for the purpose of detecting more bugs.

Detecting bugs wasn't the purpose of @safe, and pointer subtraction isn't a bug. If you're going to redefine @safe as something that rejects everything that may have a bug, then you ought to just have it instantly and unconditionally cause a compilation failure. No code is bug free, after all.

Why and how is this going into an edition two years ago instead of the 2026 edition?

Great question. The answer is we need to decide what year to make it!

The current year is 2026. My assumption with the year scheme was that code that last compiled in that year would continue to compile with either the edition of the same year or the one before it. ie, code that compiled in 2025 should compile with either the 2024 or 2025 editions. Retroactively assigning random changes to old years will just mean I have to try random editions until something works, especially if the editions remain undocumented!

[rikkimax]

I've added edition number as the first item on my edition planning list, let's move on.

[WalterBright]

Detecting bugs wasn't the purpose of @safe, and pointer subtraction isn't a bug. If you're going to redefine @safe as something that rejects everything that may have a bug, then you ought to just have it instantly and unconditionally cause a compilation failure. No code is bug free, after all.

As @tgehr pointed out in the n.g., subtracting two pointers (p-q) is "undefined behavior" (UB) when the two pointer point to different memory objects. The trouble with that, is C backends can decide it is UB and then delete the code. UB can mean anything, so yes it can impair memory safety, and so making it an error is a valid behavior for @safe code.

Easy workarounds remain:

1. use arrays and indices, not pointers
2. cast the pointers to size_t
3. rewrite the code as a @trusted lambda

[rikkimax]

Detecting bugs wasn't the purpose of @safe, and pointer subtraction isn't a bug. If you're going to redefine @safe as something that rejects everything that may have a bug, then you ought to just have it instantly and unconditionally cause a compilation failure. No code is bug free, after all.

As @tgehr pointed out in the n.g., subtracting two pointers (p-q) is "undefined behavior" (UB) when the two pointer point to different memory objects. The trouble with that, is C backends can decide it is UB and then delete the code. UB can mean anything, so yes it can impair memory safety, and so making it an error is a valid behavior for @safe code.

Easy workarounds remain:

1. use arrays and indices, not pointers
2. cast the pointers to size_t
3. rewrite the code as a @trusted lambda

Alternatively we could make the compiler cast the pointers to ptrdiff_t, and then it becomes a signed subtraction. Perfectly defined, no UB.

There are other options that Timon was suggesting in defining what it does to remove the UB.

Removing a feature isn't the only option here.

[Herringway]

Detecting bugs wasn't the purpose of @safe, and pointer subtraction isn't a bug. If you're going to redefine @safe as something that rejects everything that may have a bug, then you ought to just have it instantly and unconditionally cause a compilation failure. No code is bug free, after all.

As @tgehr pointed out in the n.g., subtracting two pointers (p-q) is "undefined behavior" (UB) when the two pointer point to different memory objects. The trouble with that, is C backends can decide it is UB and then delete the code. UB can mean anything, so yes it can impair memory safety, and so making it an error is a valid behavior for @safe code.

In order to delete the code, it would have to first prove that the pointers are to different memory objects at compile time. If we can't prove that, there's no problem here. If we CAN prove that, then we should simply issue an error instead of leaving that particular case undefined.

Easy workarounds remain:

1. use arrays and indices, not pointers

2. cast the pointers to `size_t`

3. rewrite the code as a `@trusted` lambda

[rikkimax]

Detecting bugs wasn't the purpose of @safe, and pointer subtraction isn't a bug. If you're going to redefine @safe as something that rejects everything that may have a bug, then you ought to just have it instantly and unconditionally cause a compilation failure. No code is bug free, after all.

As @tgehr pointed out in the n.g., subtracting two pointers (p-q) is "undefined behavior" (UB) when the two pointer point to different memory objects. The trouble with that, is C backends can decide it is UB and then delete the code. UB can mean anything, so yes it can impair memory safety, and so making it an error is a valid behavior for @safe code.

In order to delete the code, it would have to first prove that the pointers are to different memory objects at compile time. If we can't prove that, there's no problem here. If we CAN prove that, then we should simply issue an error instead of leaving that particular case undefined.

We cannot prove anything about aliasing to the same degree that a backend optimizer can.

It won't be consistent with codegen.

[thewilsonator]

is

void* p  = ...;
size_t x = cast(size_t)p;

to be considered unsafe?

The above is isomorphic to the issue in this PR.

[WalterBright]

@thewilsonator I don't see that as unsafe.

[WalterBright]

@rikkimax DFA can neither prove q-p is safe nor prove that it is unsafe in the general case.

[Herringway]

Why not simply lower p - q to (cast(size_t)p - cast(size_t)q) / (*p).sizeof? That way it's magically made @safe.

[WalterBright]

@Herringway Indeed, why not? That might be a great idea!

But one problem remains. If p and q point to different objects, the result will be an unpredictable value. Is that really what we want to allow in @safe code?

[Herringway]

@Herringway Indeed, why not? That might be a great idea!

But one problem remains. If p and q point to different objects, the result will be an unpredictable value. Is that really what we want to allow in @safe code?

Sure. You're pretty much guaranteed a RangeError from using that index.

I mostly just don't want to see this feature go away because it has helped me transition C code to @safe D code in the past. Translating pointer arithmetic to D arrays has been the trickiest part of those projects in the past, so the less that I have to rewrite at a time, the better.

[CyberShadow]

The trouble with that, is C backends can decide it is UB

At what point do D semantics end and C semantics go into effect?

Integer overflow is UB in C. Does this mean integer addition will also be disallowed in @safe ?

If not, where exactly do you draw the line?

[WalterBright]

There's enormous pressure in the industry to get compilers to check for more bugs in code. I've even seen attempts to get borrow checking in C++. If we don't improve our game, we'll get left behind.

That said, @trusted and @System will let you do whatever you want.

@Herringway you raise a good point. ImportC treats all imported C code as @System. I doubt much C code will compile as @safe as it stands now - C uses a lot of pointers, and @safe doesn't allow pointer arithmetic, and so I expect one must pretty much replace all the p-q expressions already.

@CyberShadow yes integer overflow is UB. If the gcc/clang back ends detect it, they delete it. I ran into that recently where x<<64 did nothing (I expected the result to be 0). I don't have a good answer for what to do about this, so I'll just go with what D does now. There is sizable pressure to add overflow checks.

[Herringway]

There's enormous pressure in the industry to get compilers to check for more bugs in code. I've even seen attempts to get borrow checking in C++. If we don't improve our game, we'll get left behind.

Not a lot of people left to leave us behind these days.

That said, @trusted and @System will let you do whatever you want.

Which is why people will simply slap those attributes on and call it a day, like you did with dmd. It's difficult enough to be @safe without it being a moving target. It should be made EASIER, not harder.

@Herringway you raise a good point. ImportC treats all imported C code as @System. I doubt much C code will compile as @safe as it stands now - C uses a lot of pointers, and @safe doesn't allow pointer arithmetic, and so I expect one must pretty much replace all the p-q expressions already.

@safe D is effectively another language when compared to D. The first rule of porting to another language is to keep the changes to a minimum. By making more changes necessary, porting becomes more difficult. There had better be a better reason for doing so than "there might be bugs, sometimes, maybe." if you want people actually doing this porting.

[rikkimax]

I've looked into UB and LLVM, and what ldc does.

In short, pointer subtraction won't trigger the UB.

https://llvm.org/docs/LangRef.html#sub-instruction

https://llvm.org/docs/UndefinedBehavior.html

void main()
{
    void* a, b;
    auto v = a - b;
}

define i32 @_Dmain({ i64, ptr } %unnamed) #0 {
  %a = alloca ptr, align 8                        ; [#uses = 2, size/byte = 8]
  %b = alloca ptr, align 8                        ; [#uses = 2, size/byte = 8]
  %v = alloca i64, align 8                        ; [#uses = 1, size/byte = 8]
  store ptr null, ptr %a, align 8
  store ptr null, ptr %b, align 8
  %1 = load ptr, ptr %a, align 8                  ; [#uses = 1]
  %2 = load ptr, ptr %b, align 8                  ; [#uses = 1]
  %3 = ptrtoint ptr %1 to i64                     ; [#uses = 1]
  %4 = ptrtoint ptr %2 to i64                     ; [#uses = 1]
  %5 = sub i64 %3, %4                             ; [#uses = 1]
  %6 = sdiv i64 %5, 1                             ; [#uses = 1]
  store i64 %6, ptr %v, align 8
  ret i32 0
}

What matters is nuw nsw is not set for the sub or sdiv.

[rikkimax]

I've looked a bit into the left shift, resulting in removal rather than 0.

If the optimizer can see that the shift amount is equal or larger to bitwidth of the integer it'll mark it as poison and it'll optimize out (with time travel support).

This is fixable by doing:

define i1 @fn(i32 %x, i32 %y) {
  %cmp1 = icmp ne i32 %x, 0
  %cmp2 = icmp ugt i32 %x, %y
  %and = select i1 %cmp1, i1 %cmp2, i1 false
  ret i1 %and
}

As long as this is defined ldc/gdc should have no trouble removing the UB for this stuff.

[dkorpel]

What's the impetus of this idea, was there an actual bug related to pointer subtraction that would have been prevented by this?