On replacing UnsafeCell

How would this handle safe cells, like Cell and Mutex?

If you write a structure with a field of type Cell, that puts an UnsafeCell inside the structure, but the wrapper Cell around it makes certain guarantees (that it's always initialised, only accessed from the current thread, and only accessed via swaps and copies). The wrapper type seems useful here, because a *mut T or *mut Cell<T> wouldn't be usable from safe code (whereas a Cell can be).

The only way to make it work would be to say "form a reference to the wrapper struct, then have the unsafe cell operations happen inside that struct". But then you could implement UnsafeCell the same way (as a wrapper struct that has one "twilight field"), and avoid the complication of needing a new language feature – it's almost always better to use a standard library type than a language feature in places where the syntax doesn't become too unergonomic.

It's worth noting that there's no significant difference in practice between &UnsafeCell<T> and *mut T: &UnsafeCell has a lifetime, but I think raw pointers should also have a lifetime (that reflects how long their provenance is valid, rather than saying anything about the reference target), because dereferencing a pointer without provenance is always undefined behaviour and the lifetime helps to prevent you doing that by mistake.

The reason that UnsafeCell exists separately from a pointer is that if it isn't behind a shared reference, e.g. if you have &_mut_ UnsafeCell<T>, it can be dereferenced in safe code. UnsafePinned exists to disable that ability, and is a lot less commonly used than cells are (and thus it doesn't really make sense to combine the features).

In terms of the rest of your motivation, I think ManuallyDrop is unrelated (ManuallyDrop<Box<T>> causes problems because you can't move a dropped Box, but in that scenario, you are accessing by value not by reference), and although there may be a connection to !Unpin, it's somewhat complicated and it's unclear how this proposal would simplify things. (Merely containing a cell doesn't make something !Unpin, after all: types are !Unpin if they're address-sensitive or contain self-references, and although I think it can make sense to view a self-reference as a type of cell, most cells aren't used for that purpose and don't prevent types being Unpin. For example, both Cell and UnsafeCell implement Unpin if their contents do.)