[Pre-RFC] BTF relocations

ais523:

As such, I'm wondering whether the new attribute should be #[repr(btf)], given that it's mutually exclusive with other repr attributes and has the same purpose of changing where in a struct its data is stored.

I like the idea. I'm not really attached to the naming used in C (I already added the btf prefix to the attributes, since I find the original Clang/GCC naming too generic). The good thing about having a separate repr is that it lets us enforce a C-compatible ABI.

ais523:

The other thought is to wonder about the interaction of this with compile-time introspection. For example, macros like offset_of! normally assume that the offset of a field within a struct is known at compile time, whereas if it's being specified by relocations, that is not the case.

RalfJung:

Currently, in Rust, it is possible to replace every field access by a use of offset with a compile-time constant determined with offset_of! (except for field with a dyn Trait unsized tail). That's no longer true with this proposal. So this proposal is making deep changes to the foundations of the language itself, its operational semantics. This is way more than just a proposal for a bit of new syntax, and it doesn't just expand rustc implementation details like MIR, it expands the core language in a way that's relevant for unsafe code authors. [...]

At the very least, it seems like offset_of! on such fields should fail to compile. But the changes have to go deeper than that; the assumption of a fixed layout is deeply rooted everywhere in the compiler and the language.

jrose:

How does this all work in C? Because C also assumes you can use offsetof on any struct field (and sizeof on any complete struct type, and everything implied by that).

First of all, let's clarify one thing. The offsetof macro is part of stddef.h in libc and therefore cannot be used in eBPF programs. In musl, the definition lowers either to __builtin_offsetof or to a field access:

elixir.bootlin.com

stddef.h - include/stddef.h - Musl source code v1.2.6 - Bootlin Elixir Cross...

Elixir Cross Referencer - source code of Musl v1.2.6: include/stddef.h

What's available on BPF targets is a direct call to __builtin_offsetof. And that indeed does not emit relocations.

struct inner {
  int x;
  int y;
};

struct outer {
  int a;
  struct inner b;
} __attribute__((preserve_access_index));

unsigned long plain_offsetof(void) {
  return __builtin_offsetof(struct outer, b.y);
}

lowers to:

define dso_local i64 @plain_offsetof() #0 !dbg !7 {
  ret i64 8, !dbg !11
}

I do agree that offset_of! on relocatable types/fields should fail to compile, and we shouldn't repeat the mistake of C compilers silently emitting code that is not going to work. The same goes for core::mem::size_of, core::mem::align_of, and probably many other macros and const functions that I haven't thought of yet. I will take some time to go through the entire core API and figure out what else we need to add to that list.

At the same time, I think we could add an intrinsic btf_field_byte_offset that serves as a replacement for offset_of, but emits a relocation for annotated types when building for BPF targets with BTF. On LLVM, it would lower to the @llvm.bpf.preserve.field.info(..., FIELD_BYTE_OFFSET) intrinsic call. On GCC, it would call the __builtin_preserve_field_info(..., FIELD_BYTE_OFFSET) built-in function.

I actually managed to get it working already (again, only for LLVM; I haven't started on the GCC implementation yet), and the change doesn't seem difficult:

github.com/vadorovsky/rust

compiler, library: Add btf_field_byte_offset intrinsic

committed 11:54AM - 10 Apr 26 UTC

      vadorovsky
    

+146 -5

The offset_of intrinsic does not emit BTF relocations even when it is used wit…h a type annotated for BTF access preservation. Add a new intrinsic, btf_field_byte_offset, to represent a relocatable field byte offset query in codegen. Unlike offset_of, which is const-only and folds to a plain layout constant during const-eval, this intrinsic is visible to backend codegen and can lower to BTF CO-RE field-offset relocations on BPF targets. For LLVM, lower btf_field_byte_offset through @llvm.bpf.preserve.field.info(..., FIELD_BYTE_OFFSET) intrinsic by first constructing the corresponding preserve-access chain. On targets or backends without BTF relocation support, fall back to the ordinary layout-computed offset.

If you agree with this approach, I'm happy to update the RFC with it.

RalfJung:

The proposal needs to explain what the new operational semantics are, and how they are supposed to work in a tool like Miri.

Miri does not work for BPF targets for unrelated reasons, and for similar reasons it does not run on embedded targets either: BPF binaries can't be executed on regular operating systems; they can be executed only by virtual machines designed for the BPF ISA, including the one provided by the Linux kernel. We don't really have any solution for testing BPF binaries in Aya right now, apart from extracting testable code into crates that can be used for unit tests with std on host targets. The same approach works for Miri as well.

Apart from the Linux kernel BPF VM, there are some user-space implementations, including rBPF and Solana's SBPF, but we haven't really tried using them to run unit tests. That said, if that ever works, I guess we could look into getting Miri running on one of them as well.

But I don't think that should be a requirement for this proposal.

ais523:

There are existing exceptions where a field offset might not be known at compile time (e.g. a #[repr(C)] struct containing a u8 and a dyn Trait can't determine the offset of its second field because it depends on its alignment, which for dyn Trait can vary at runtime); but all the existing exceptions are for unsized types. Should that imply that BTF types should not be Sized? (Thinking about it, they are logically unSized because the size may change after the compile has already happened, due to the relocations, and thus the size is not known at compile time.)

alice:

I guess this kind of attribute has very similar consequences for the struct as the ARM scalable vectors that the sized hierarchy work is looking at. BTF provides relocations for both field offsets and the struct size, so this means that task_struct is Sized but not const Sized under the terminology introduced here. Its size is constant at runtime, but not known at compile time.

The approach of treating #[repr(btf)] types as Sized and not const Sized sounds great to me. But given that the RFC mentioned above has not been accepted and this distinction is not yet implemented, the only option for now seems to be treating them as not Sized. I will try implementing that and see whether it has any consequences that make it unworkable.