Skip to main content

core/ptr/
mod.rs

1//! Manually manage memory through raw pointers.
2//!
3//! *[See also the pointer primitive types](pointer).*
4//!
5//! # Safety
6//!
7//! Many functions in this module take raw pointers as arguments and read from or write to them. For
8//! this to be safe, these pointers must be *valid* for the given access. Whether a pointer is valid
9//! depends on the operation it is used for (read or write), and the extent of the memory that is
10//! accessed (i.e., how many bytes are read/written) -- it makes no sense to ask "is this pointer
11//! valid"; one has to ask "is this pointer valid for a given access". Most functions use `*mut T`
12//! and `*const T` to access only a single value, in which case the documentation omits the size and
13//! implicitly assumes it to be `size_of::<T>()` bytes.
14//!
15//! The precise rules for validity are not determined yet. The guarantees that are
16//! provided at this point are very minimal:
17//!
18//! * A [null] pointer is *never* valid for reads/writes.
19//! * For memory accesses of [size zero][zst], *every* non-null pointer is valid for reads/writes.
20//!   The following points are only concerned with non-zero-sized accesses.
21//! * For a pointer to be valid for reads/writes, it is necessary, but not always sufficient, that
22//!   the pointer be *dereferenceable*. The [provenance] of the pointer is used to determine which
23//!   [allocation] it is derived from; a pointer is dereferenceable if the memory range of the given
24//!   size starting at the pointer is entirely contained within the bounds of that allocation. Note
25//!   that in Rust, every (stack-allocated) variable is considered a separate allocation.
26//! * All accesses performed by functions in this module are *non-atomic* in the sense
27//!   of [atomic operations] used to synchronize between threads. This means it is
28//!   undefined behavior to perform two concurrent accesses to the same location from different
29//!   threads unless both accesses only read from memory.
30//! * The result of casting a reference to a pointer is valid for reads/writes for as long as the
31//!   underlying allocation is live and no reference (just raw pointers) is used to
32//!   access the same memory. That is, reference and pointer accesses cannot be
33//!   interleaved.
34//!
35//! These axioms, along with careful use of [`offset`] for pointer arithmetic,
36//! are enough to correctly implement many useful things in unsafe code. Stronger guarantees
37//! will be provided eventually, as the [aliasing] rules are being determined. For more
38//! information, see the [book] as well as the section in the reference devoted
39//! to [undefined behavior][ub].
40//!
41//! Note that some operations such as [`read`] and [`write`][`write()`] do allow null pointers if
42//! the total size of the access is zero. However, other operations internally convert pointers into
43//! references. Therefore, the general notion of "valid for reads/writes" excludes null pointers,
44//! and the specific operations that permit null pointers mention that as an exception. Furthermore,
45//! [`read_volatile`] and [`write_volatile`] can be used in even more situations; see their
46//! documentation for details.
47//!
48//! We say that a pointer is "dangling" if it is not valid for any non-zero-sized accesses. This
49//! means out-of-bounds pointers, pointers to freed memory, null pointers, and pointers created with
50//! [`NonNull::dangling`] are all dangling.
51//!
52//! ## Alignment
53//!
54//! Valid raw pointers as defined above are not necessarily properly aligned (where
55//! "proper" alignment is defined by the pointee type, i.e., `*const T` must be
56//! aligned to `align_of::<T>()`). However, most functions require their
57//! arguments to be properly aligned, and will explicitly state
58//! this requirement in their documentation. Notable exceptions to this are
59//! [`read_unaligned`] and [`write_unaligned`].
60//!
61//! When a function requires proper alignment, it does so even if the access
62//! has size 0, i.e., even if memory is not actually touched. Consider using
63//! [`NonNull::dangling`] in such cases.
64//!
65//! ## Pointer to reference conversion
66//!
67//! When converting a pointer to a reference (e.g. via `&*ptr` or `&mut *ptr`),
68//! there are several rules that must be followed:
69//!
70//! * The pointer must be properly aligned.
71//!
72//! * It must be non-null.
73//!
74//! * It must be "dereferenceable" in the sense defined above.
75//!
76//! * The pointer must point to a [valid value] of type `T`.
77//!
78//! * You must enforce Rust's aliasing rules. The exact aliasing rules are not decided yet, so we
79//!   only give a rough overview here. The rules also depend on whether a mutable or a shared
80//!   reference is being created.
81//!   * When creating a mutable reference, then while this reference exists, the memory it points to
82//!     must not get accessed (read or written) through any other pointer or reference not derived
83//!     from this reference.
84//!   * When creating a shared reference, then while this reference exists, the memory it points to
85//!     must not get mutated (except inside `UnsafeCell`).
86//!
87//! If a pointer follows all of these rules, it is said to be
88//! *convertible to a (mutable or shared) reference*.
89// ^ we use this term instead of saying that the produced reference must
90// be valid, as the validity of a reference is easily confused for the
91// validity of the thing it refers to, and while the two concepts are
92// closely related, they are not identical.
93//!
94//! These rules apply even if the result is unused!
95//! (The part about being initialized is not yet fully decided, but until
96//! it is, the only safe approach is to ensure that they are indeed initialized.)
97//!
98//! An example of the implications of the above rules is that an expression such
99//! as `unsafe { &*(0 as *const u8) }` is Immediate Undefined Behavior.
100//!
101//! [valid value]: ../../reference/behavior-considered-undefined.html#invalid-values
102//!
103//! ## Allocation
104//!
105//! <a id="allocated-object"></a> <!-- keep old URLs working -->
106//!
107//! An *allocation* is a subset of program memory which is addressable
108//! from Rust, and within which pointer arithmetic is possible. Examples of
109//! allocations include heap allocations, stack-allocated variables,
110//! statics, and consts. The safety preconditions of some Rust operations -
111//! such as `offset` and field projections (`expr.field`) - are defined in
112//! terms of the allocations on which they operate.
113//!
114//! An allocation has a base address, a size, and a set of memory
115//! addresses. It is possible for an allocation to have zero size, but
116//! such an allocation will still have a base address. The base address
117//! of an allocation is not necessarily unique. While it is currently the
118//! case that an allocation always has a set of memory addresses which is
119//! fully contiguous (i.e., has no "holes"), there is no guarantee that this
120//! will not change in the future.
121//!
122//! An allocation can be either mutable (the common case) or *read-only*.
123//! Read-only allocations are implicitly introduced by the compiler for `static` items without
124//! interior mutability and for `const` items. Writing or creating a mutable reference to a
125//! read-only allocation is undefined behavior, and most atomic operations are not supported
126//! for read-only allocations either (see [here][atomic-ro] for exceptions).
127//! Additionally, some target-specific intrinsics are not supported on read-only
128//! allocations even if their memory write is masked off, such as [`_mm_maskmoveu_si128`].
129//!
130//! [atomic-ro]: crate::sync::atomic#atomic-accesses-to-read-only-memory
131//! [`_mm_maskmoveu_si128`]: ../../core/arch/x86/fn._mm_maskmoveu_si128.html
132//!
133//! Allocations must behave like "normal" memory: in particular, reads must not have
134//! side-effects, and writes must become visible to other threads using the usual synchronization
135//! primitives.
136//! Allocations must support all atomic operations that are available for the target (as
137//! determined by the `target_has_atomic*` set of cfg flags).
138//! Read-only allocations only have to support the operations [permitted there][atomic-ro].
139//! The precise instructions used for atomic operations are generally not guaranteed, so portable
140//! software should place all Rust allocations in memory regions that support all atomic
141//! instructions.
142//!
143//! For any allocation with `base` address, `size`, and a set of
144//! `addresses`, the following are guaranteed:
145//! - For all addresses `a` in `addresses`, `a` is in the range `base .. (base +
146//!   size)` (note that this requires `a < base + size`, not `a <= base + size`)
147//! - `base` is not equal to [`null()`] (i.e., the address with the numerical
148//!   value 0)
149//! - `base + size <= usize::MAX`
150//! - `size <= isize::MAX`
151//!
152//! As a consequence of these guarantees, given any address `a` within the set
153//! of addresses of an allocation:
154//! - It is guaranteed that `a - base` does not overflow `isize`
155//! - It is guaranteed that `a - base` is non-negative
156//! - It is guaranteed that, given `o = a - base` (i.e., the offset of `a` within
157//!   the allocation), `base + o` will not wrap around the address space (in
158//!   other words, will not overflow `usize`)
159//!
160//! Allocations typically have a fixed size that cannot change. However, allocations created by
161//! directly invoking page table operations of the operating system, e.g. via `mmap`, are allowed to
162//! grow by adding more pages to them at the end. Adding more pages before the beginning, or
163//! unmapping parts of an allocation (i.e., shrinking it or punching holes into it), is currently
164//! not supported. Allocations created via "compiler-recognized" operations, such as `std::alloc`
165//! methods or `libc::malloc`, can never change their size, even if they use `mmap` under the hood.
166//!
167//! [`null()`]: null
168//!
169//! # Thin and wide pointers
170//!
171//! Some pointers can also carry additional metadata alongside the address of the pointee. These are called
172//! **wide pointers** (or **fat pointers**).
173//! Pointers that only store the address of the pointee are instead called **thin pointers**, every
174//! pointer to a sized type is guaranteed to be thin.
175//!
176//! Currently, wide pointers are twice as big as the size of a thin pointer,
177//! however, this could be changed in the future.
178//!
179//! Wide pointers are commonly used when working with [`DSTs`].
180//!
181//! [`DSTs`]: https://doc.rust-lang.org/reference/dynamically-sized-types.html
182//!
183//! # Provenance
184//!
185//! Pointers are not *simply* an "integer" or "address". For instance, it's uncontroversial
186//! to say that a Use After Free is clearly Undefined Behavior, even if you "get lucky"
187//! and the freed memory gets reallocated before your read/write (in fact this is the
188//! worst-case scenario, UAFs would be much less concerning if this didn't happen!).
189//! As another example, consider that [`wrapping_offset`] is documented to "remember"
190//! the allocation that the original pointer points to, even if it is offset far
191//! outside the memory range occupied by that allocation.
192//! To rationalize claims like this, pointers need to somehow be *more* than just their addresses:
193//! they must have **provenance**.
194//!
195//! A pointer value in Rust semantically contains the following information:
196//!
197//! * The **address** it points to, which can be represented by a `usize`.
198//! * The **provenance** it has, defining the memory it has permission to access. Provenance can be
199//!   absent, in which case the pointer does not have permission to access any memory.
200//!
201//! The exact structure of provenance is not yet specified, but the permission defined by a
202//! pointer's provenance have a *spatial* component, a *temporal* component, and a *mutability*
203//! component:
204//!
205//! * Spatial: The set of memory addresses that the pointer is allowed to access.
206//! * Temporal: The timespan during which the pointer is allowed to access those memory addresses.
207//! * Mutability: Whether the pointer may only access the memory for reads, or also access it for
208//!   writes. Note that this can interact with the other components, e.g. a pointer might permit
209//!   mutation only for a subset of addresses, or only for a subset of its maximal timespan.
210//!
211//! When an [allocation] is created, it has a unique Original Pointer. For alloc
212//! APIs this is literally the pointer the call returns, and for local variables and statics,
213//! this is the name of the variable/static. (This is mildly overloading the term "pointer"
214//! for the sake of brevity/exposition.)
215//!
216//! The Original Pointer for an allocation has provenance that constrains the *spatial*
217//! permissions of this pointer to the memory range of the allocation, and the *temporal*
218//! permissions to the lifetime of the allocation. Provenance is implicitly inherited by all
219//! pointers transitively derived from the Original Pointer through operations like [`offset`],
220//! borrowing, and pointer casts. Some operations may *shrink* the permissions of the derived
221//! provenance, limiting how much memory it can access or how long it's valid for (i.e. borrowing a
222//! subfield and subslicing can shrink the spatial component of provenance, and all borrowing can
223//! shrink the temporal component of provenance). However, no operation can ever *grow* the
224//! permissions of the derived provenance: even if you "know" there is a larger allocation, you
225//! can't derive a pointer with a larger provenance. Similarly, you cannot "recombine" two
226//! contiguous provenances back into one (i.e. with a `fn merge(&[T], &[T]) -> &[T]`).
227//!
228//! A reference to a place always has provenance over at least the memory that place occupies.
229//! A reference to a slice always has provenance over at least the range that slice describes.
230//! Whether and when exactly the provenance of a reference gets "shrunk" to *exactly* fit
231//! the memory it points to is not yet determined.
232//!
233//! A *shared* reference only ever has provenance that permits reading from memory,
234//! and never permits writes, except inside [`UnsafeCell`].
235//!
236//! Provenance can affect whether a program has undefined behavior:
237//!
238//! * It is undefined behavior to access memory through a pointer that does not have provenance over
239//!   that memory. Note that a pointer "at the end" of its provenance is not actually outside its
240//!   provenance, it just has 0 bytes it can load/store. Zero-sized accesses do not require any
241//!   provenance since they access an empty range of memory.
242//!
243//! * It is undefined behavior to [`offset`] a pointer across a memory range that is not contained
244//!   in the allocation it is derived from, or to [`offset_from`] two pointers not derived
245//!   from the same allocation. Provenance is used to say what exactly "derived from" even
246//!   means: the lineage of a pointer is traced back to the Original Pointer it descends from, and
247//!   that identifies the relevant allocation. In particular, it's always UB to offset a
248//!   pointer derived from something that is now deallocated, except if the offset is 0.
249//!
250//! But it *is* still sound to:
251//!
252//! * Create a pointer without provenance from just an address (see [`without_provenance`]). Such a
253//!   pointer cannot be used for memory accesses (except for zero-sized accesses). This can still be
254//!   useful for sentinel values like `null` *or* to represent a tagged pointer that will never be
255//!   dereferenceable. In general, it is always sound for an integer to pretend to be a pointer "for
256//!   fun" as long as you don't use operations on it which require it to be valid (non-zero-sized
257//!   offset, read, write, etc).
258//!
259//! * Forge an allocation of size zero at any sufficiently aligned non-null address.
260//!   i.e. the usual "ZSTs are fake, do what you want" rules apply.
261//!
262//! * [`wrapping_offset`] a pointer outside its provenance. This includes pointers
263//!   which have "no" provenance. In particular, this makes it sound to do pointer tagging tricks.
264//!
265//! * Compare arbitrary pointers by address. Pointer comparison ignores provenance and addresses
266//!   *are* just integers, so there is always a coherent answer, even if the pointers are dangling
267//!   or from different provenances. Note that if you get "lucky" and notice that a pointer at the
268//!   end of one allocation is the "same" address as the start of another allocation,
269//!   anything you do with that fact is *probably* going to be gibberish. The scope of that
270//!   gibberish is kept under control by the fact that the two pointers *still* aren't allowed to
271//!   access the other's allocation (bytes), because they still have different provenance.
272//!
273//! Note that the full definition of provenance in Rust is not decided yet, as this interacts
274//! with the as-yet undecided [aliasing] rules.
275//!
276//! ## Pointers Vs Integers
277//!
278//! From this discussion, it becomes very clear that a `usize` *cannot* accurately represent a pointer,
279//! and converting from a pointer to a `usize` is generally an operation which *only* extracts the
280//! address. Converting this address back into pointer requires somehow answering the question:
281//! which provenance should the resulting pointer have?
282//!
283//! Rust provides two ways of dealing with this situation: *Strict Provenance* and *Exposed Provenance*.
284//!
285//! Note that a pointer *can* represent a `usize` (via [`without_provenance`]), so the right type to
286//! use in situations where a value is "sometimes a pointer and sometimes a bare `usize`" is a
287//! pointer type.
288//!
289//! ## Strict Provenance
290//!
291//! "Strict Provenance" refers to a set of APIs designed to make working with provenance more
292//! explicit. They are intended as substitutes for casting a pointer to an integer and back.
293//!
294//! Entirely avoiding integer-to-pointer casts successfully side-steps the inherent ambiguity of
295//! that operation. This benefits compiler optimizations, and it is pretty much a requirement for
296//! using tools like [Miri] and architectures like [CHERI] that aim to detect and diagnose pointer
297//! misuse.
298//!
299//! The key insight to making programming without integer-to-pointer casts *at all* viable is the
300//! [`with_addr`] method:
301//!
302//! ```text
303//! /// Creates a new pointer with the given address and the provenance  of `self`.
304//! ///
305//! /// This is similar to a `addr as *const T` cast,
306//! /// but copies the provenance of `self` to the new pointer.
307//! /// This avoids the inherent ambiguity of the unary cast.
308//! ///
309//! /// This is equivalent to using `wrapping_offset` to offset `self` to the given address,
310//! /// and therefore has all the same capabilities and restrictions.
311//! pub fn with_addr(self, addr: usize) -> Self;
312//! ```
313//!
314//! So you're still able to drop down to the address representation and do whatever
315//! clever bit tricks you want *as long as* you're able to keep around a pointer
316//! into the allocation you care about that can "reconstitute" the provenance.
317//! Usually this is very easy, because you only are taking a pointer, messing with the address,
318//! and then immediately converting back to a pointer. To make this use case more ergonomic,
319//! we provide the [`map_addr`] method.
320//!
321//! To help make it clear that code is "following" Strict Provenance semantics, we also provide an
322//! [`addr`] method which promises that the returned address is not part of a
323//! pointer-integer-pointer roundtrip. In the future we may provide a lint for pointer<->integer
324//! casts to help you audit if your code conforms to strict provenance.
325//!
326//! ### Using Strict Provenance
327//!
328//! Most code needs no changes to conform to strict provenance, as the only really concerning
329//! operation is casts from `usize` to a pointer. For code which *does* cast a `usize` to a pointer,
330//! the scope of the change depends on exactly what you're doing.
331//!
332//! In general, you just need to make sure that if you want to convert a `usize` address to a
333//! pointer and then use that pointer to read/write memory, you need to keep around a pointer
334//! that has sufficient provenance to perform that read/write itself. In this way all of your
335//! casts from an address to a pointer are essentially just applying offsets/indexing.
336//!
337//! This is generally trivial to do for simple cases like tagged pointers *as long as you
338//! represent the tagged pointer as an actual pointer and not a `usize`*. For instance:
339//!
340//! ```
341//! // A flag we want to pack into our pointer
342//! static HAS_DATA: usize = 0x1;
343//! static FLAG_MASK: usize = !HAS_DATA;
344//!
345//! // Our value, which must have enough alignment to have spare least-significant-bits.
346//! let my_precious_data: u32 = 17;
347//! assert!(align_of::<u32>() > 1);
348//!
349//! // Create a tagged pointer
350//! let ptr = &my_precious_data as *const u32;
351//! let tagged = ptr.map_addr(|addr| addr | HAS_DATA);
352//!
353//! // Check the flag:
354//! if tagged.addr() & HAS_DATA != 0 {
355//!     // Untag and read the pointer
356//!     let data = unsafe { *tagged.map_addr(|addr| addr & FLAG_MASK) };
357//!     assert_eq!(data, 17);
358//! } else {
359//!     unreachable!()
360//! }
361//! ```
362//!
363//! (Yes, if you've been using [`AtomicUsize`] for pointers in concurrent datastructures, you should
364//! be using [`AtomicPtr`] instead. If that messes up the way you atomically manipulate pointers,
365//! we would like to know why, and what needs to be done to fix it.)
366//!
367//! Situations where a valid pointer *must* be created from just an address, such as baremetal code
368//! accessing a memory-mapped interface at a fixed address, cannot currently be handled with strict
369//! provenance APIs and should use [exposed provenance](#exposed-provenance).
370//!
371//! ## Exposed Provenance
372//!
373//! As discussed above, integer-to-pointer casts are not possible with Strict Provenance APIs.
374//! This is by design: the goal of Strict Provenance is to provide a clear specification that we are
375//! confident can be formalized unambiguously and can be subject to precise formal reasoning.
376//! Integer-to-pointer casts do not (currently) have such a clear specification.
377//!
378//! However, there exist situations where integer-to-pointer casts cannot be avoided, or
379//! where avoiding them would require major refactoring. Legacy platform APIs also regularly assume
380//! that `usize` can capture all the information that makes up a pointer.
381//! Bare-metal platforms can also require the synthesis of a pointer "out of thin air" without
382//! anywhere to obtain proper provenance from.
383//!
384//! Rust's model for dealing with integer-to-pointer casts is called *Exposed Provenance*. However,
385//! the semantics of Exposed Provenance are on much less solid footing than Strict Provenance, and
386//! at this point it is not yet clear whether a satisfying unambiguous semantics can be defined for
387//! Exposed Provenance. (If that sounds bad, be reassured that other popular languages that provide
388//! integer-to-pointer casts are not faring any better.) Furthermore, Exposed Provenance will not
389//! work (well) with tools like [Miri] and [CHERI].
390//!
391//! Exposed Provenance is provided by the [`expose_provenance`] and [`with_exposed_provenance`] methods,
392//! which are equivalent to `as` casts between pointers and integers.
393//! - [`expose_provenance`] is a lot like [`addr`], but additionally adds the provenance of the
394//!   pointer to a global list of 'exposed' provenances. (This list is purely conceptual, it exists
395//!   for the purpose of specifying Rust but is not materialized in actual executions, except in
396//!   tools like [Miri].)
397//!   Memory which is outside the control of the Rust abstract machine (MMIO registers, for example)
398//!   is always considered to be exposed, so long as this memory is disjoint from memory that will
399//!   be used by the abstract machine such as the stack, heap, and statics.
400//! - [`with_exposed_provenance`] can be used to construct a pointer with one of these previously
401//!   'exposed' provenances. [`with_exposed_provenance`] takes only `addr: usize` as arguments, so
402//!   unlike in [`with_addr`] there is no indication of what the correct provenance for the returned
403//!   pointer is -- and that is exactly what makes integer-to-pointer casts so tricky to rigorously
404//!   specify! The compiler will do its best to pick the right provenance for you, but currently we
405//!   cannot provide any guarantees about which provenance the resulting pointer will have. Only one
406//!   thing is clear: if there is *no* previously 'exposed' provenance that justifies the way the
407//!   returned pointer will be used, the program has undefined behavior.
408//!
409//! If at all possible, we encourage code to be ported to [Strict Provenance] APIs, thus avoiding
410//! the need for Exposed Provenance. Maximizing the amount of such code is a major win for avoiding
411//! specification complexity and to facilitate adoption of tools like [CHERI] and [Miri] that can be
412//! a big help in increasing the confidence in (unsafe) Rust code. However, we acknowledge that this
413//! is not always possible, and offer Exposed Provenance as a way to explicit "opt out" of the
414//! well-defined semantics of Strict Provenance, and "opt in" to the unclear semantics of
415//! integer-to-pointer casts.
416//!
417//! [aliasing]: ../../nomicon/aliasing.html
418//! [allocation]: #allocation
419//! [provenance]: #provenance
420//! [book]: ../../book/ch19-01-unsafe-rust.html#dereferencing-a-raw-pointer
421//! [ub]: ../../reference/behavior-considered-undefined.html
422//! [zst]: ../../nomicon/exotic-sizes.html#zero-sized-types-zsts
423//! [atomic operations]: crate::sync::atomic
424//! [`offset`]: pointer::offset
425//! [`offset_from`]: pointer::offset_from
426//! [`wrapping_offset`]: pointer::wrapping_offset
427//! [`with_addr`]: pointer::with_addr
428//! [`map_addr`]: pointer::map_addr
429//! [`addr`]: pointer::addr
430//! [`AtomicUsize`]: crate::sync::atomic::AtomicUsize
431//! [`AtomicPtr`]: crate::sync::atomic::AtomicPtr
432//! [`expose_provenance`]: pointer::expose_provenance
433//! [`with_exposed_provenance`]: with_exposed_provenance
434//! [Miri]: https://github.com/rust-lang/miri
435//! [CHERI]: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
436//! [Strict Provenance]: #strict-provenance
437//! [`UnsafeCell`]: core::cell::UnsafeCell
438
439#![stable(feature = "rust1", since = "1.0.0")]
440// There are many unsafe functions taking pointers that don't dereference them.
441#![allow(clippy::not_unsafe_ptr_arg_deref)]
442
443use crate::cmp::Ordering;
444use crate::intrinsics::const_eval_select;
445use crate::marker::{Destruct, PointeeSized};
446use crate::mem::{self, MaybeUninit, SizedTypeProperties};
447use crate::num::NonZero;
448use crate::ops::FnPtr;
449use crate::{fmt, hash, intrinsics, ub_checks};
450
451mod metadata;
452#[unstable(feature = "ptr_metadata", issue = "81513")]
453pub use metadata::{DynMetadata, Pointee, Thin, from_raw_parts, from_raw_parts_mut, metadata};
454
455mod non_null;
456#[stable(feature = "nonnull", since = "1.25.0")]
457pub use non_null::NonNull;
458
459mod const_ptr;
460mod mut_ptr;
461
462// Some functions are defined here because they accidentally got made
463// available in this module on stable. See <https://github.com/rust-lang/rust/issues/15702>.
464// (`transmute` also falls into this category, but it cannot be wrapped due to the
465// check that `T` and `U` have the same size.)
466
467/// Copies `count * size_of::<T>()` bytes from `src` to `dst`. The source
468/// and destination must *not* overlap.
469///
470/// For regions of memory which might overlap, use [`copy`] instead.
471///
472/// `copy_nonoverlapping` is semantically equivalent to C's [`memcpy`], but
473/// with the source and destination arguments swapped,
474/// and `count` counting the number of `T`s instead of bytes.
475///
476/// The copy is "untyped" in the sense that data may be uninitialized or otherwise violate the
477/// requirements of `T`. The initialization state is preserved exactly.
478///
479/// [`memcpy`]: https://en.cppreference.com/w/c/string/byte/memcpy
480///
481/// # Safety
482///
483/// Behavior is undefined if any of the following conditions are violated:
484///
485/// * `src` must be [valid] for reads of `count * size_of::<T>()` bytes or that number must be 0.
486///
487/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes or that number must be 0.
488///
489/// * Both `src` and `dst` must be properly aligned.
490///
491/// * The region of memory beginning at `src` with a size of `count *
492///   size_of::<T>()` bytes must *not* overlap with the region of memory
493///   beginning at `dst` with the same size.
494///
495/// Like [`read`], `copy_nonoverlapping` creates a bitwise copy of `T`, regardless of
496/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using *both* the values
497/// in the region beginning at `*src` and the region beginning at `*dst` can
498/// [violate memory safety][read-ownership].
499///
500/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
501/// `0`, the pointers must be properly aligned.
502///
503/// [`read`]: crate::ptr::read
504/// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value
505/// [valid]: crate::ptr#safety
506///
507/// # Examples
508///
509/// Manually implement [`Vec::append`]:
510///
511/// ```
512/// use std::ptr;
513///
514/// /// Moves all the elements of `src` into `dst`, leaving `src` empty.
515/// fn append<T>(dst: &mut Vec<T>, src: &mut Vec<T>) {
516///     let src_len = src.len();
517///     let dst_len = dst.len();
518///
519///     // Ensure that `dst` has enough capacity to hold all of `src`.
520///     dst.reserve(src_len);
521///
522///     unsafe {
523///         // The call to add is always safe because `Vec` will never
524///         // allocate more than `isize::MAX` bytes.
525///         let dst_ptr = dst.as_mut_ptr().add(dst_len);
526///         let src_ptr = src.as_ptr();
527///
528///         // Truncate `src` without dropping its contents. We do this first,
529///         // to avoid problems in case something further down panics.
530///         src.set_len(0);
531///
532///         // The two regions cannot overlap because mutable references do
533///         // not alias, and two different vectors cannot own the same
534///         // memory.
535///         ptr::copy_nonoverlapping(src_ptr, dst_ptr, src_len);
536///
537///         // Notify `dst` that it now holds the contents of `src`.
538///         dst.set_len(dst_len + src_len);
539///     }
540/// }
541///
542/// let mut a = vec!['r'];
543/// let mut b = vec!['u', 's', 't'];
544///
545/// append(&mut a, &mut b);
546///
547/// assert_eq!(a, &['r', 'u', 's', 't']);
548/// assert!(b.is_empty());
549/// ```
550///
551/// [`Vec::append`]: ../../std/vec/struct.Vec.html#method.append
552#[doc(alias = "memcpy")]
553#[stable(feature = "rust1", since = "1.0.0")]
554#[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")]
555#[inline(always)]
556#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
557#[rustc_diagnostic_item = "ptr_copy_nonoverlapping"]
558pub const unsafe fn copy_nonoverlapping<T>(src: *const T, dst: *mut T, count: usize) {
559    ub_checks::assert_unsafe_precondition!(
560        check_language_ub,
561        "ptr::copy_nonoverlapping requires that both pointer arguments are aligned and non-null \
562        and the specified memory ranges do not overlap",
563        (
564            src: *const () = src as *const (),
565            dst: *mut () = dst as *mut (),
566            size: usize = size_of::<T>(),
567            align: usize = align_of::<T>(),
568            count: usize = count,
569        ) => {
570            let zero_size = count == 0 || size == 0;
571            ub_checks::maybe_is_aligned_and_not_null(src, align, zero_size)
572                && ub_checks::maybe_is_aligned_and_not_null(dst, align, zero_size)
573                && ub_checks::maybe_is_nonoverlapping(src, dst, size, count)
574        }
575    );
576
577    // SAFETY: the safety contract for `copy_nonoverlapping` must be
578    // upheld by the caller.
579    unsafe { crate::intrinsics::copy_nonoverlapping(src, dst, count) }
580}
581
582/// Copies `count * size_of::<T>()` bytes from `src` to `dst`. The source
583/// and destination may overlap.
584///
585/// If the source and destination will *never* overlap,
586/// [`copy_nonoverlapping`] can be used instead.
587///
588/// `copy` is semantically equivalent to C's [`memmove`], but
589/// with the source and destination arguments swapped,
590/// and `count` counting the number of `T`s instead of bytes.
591/// Copying takes place as if the bytes were copied from `src`
592/// to a temporary array and then copied from the array to `dst`.
593///
594/// The copy is "untyped" in the sense that data may be uninitialized or otherwise violate the
595/// requirements of `T`. The initialization state is preserved exactly.
596///
597/// [`memmove`]: https://en.cppreference.com/w/c/string/byte/memmove
598///
599/// # Safety
600///
601/// Behavior is undefined if any of the following conditions are violated:
602///
603/// * `src` must be [valid] for reads of `count * size_of::<T>()` bytes or that number must be 0.
604///
605/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes or that number must be 0,
606///   and `dst` must remain valid even when `src` is read for `count * size_of::<T>()` bytes. (This
607///   means if the memory ranges overlap, the `dst` pointer must not be invalidated by `src` reads.)
608///
609/// * Both `src` and `dst` must be properly aligned.
610///
611/// Like [`read`], `copy` creates a bitwise copy of `T`, regardless of
612/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using both the values
613/// in the region beginning at `*src` and the region beginning at `*dst` can
614/// [violate memory safety][read-ownership].
615///
616/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
617/// `0`, the pointers must be properly aligned.
618///
619/// [`read`]: crate::ptr::read
620/// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value
621/// [valid]: crate::ptr#safety
622///
623/// # Examples
624///
625/// Efficiently create a Rust vector from an unsafe buffer:
626///
627/// ```
628/// use std::ptr;
629///
630/// /// # Safety
631/// ///
632/// /// * `ptr` must be correctly aligned for its type and non-zero.
633/// /// * `ptr` must be valid for reads of `elts` contiguous elements of type `T`.
634/// /// * Those elements must not be used after calling this function unless `T: Copy`.
635/// # #[allow(dead_code)]
636/// unsafe fn from_buf_raw<T>(ptr: *const T, elts: usize) -> Vec<T> {
637///     let mut dst = Vec::with_capacity(elts);
638///
639///     // SAFETY: Our precondition ensures the source is aligned and valid,
640///     // and `Vec::with_capacity` ensures that we have usable space to write them.
641///     unsafe { ptr::copy(ptr, dst.as_mut_ptr(), elts); }
642///
643///     // SAFETY: We created it with this much capacity earlier,
644///     // and the previous `copy` has initialized these elements.
645///     unsafe { dst.set_len(elts); }
646///     dst
647/// }
648/// ```
649#[doc(alias = "memmove")]
650#[stable(feature = "rust1", since = "1.0.0")]
651#[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")]
652#[inline(always)]
653#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
654#[rustc_diagnostic_item = "ptr_copy"]
655pub const unsafe fn copy<T>(src: *const T, dst: *mut T, count: usize) {
656    // SAFETY: the safety contract for `copy` must be upheld by the caller.
657    unsafe {
658        ub_checks::assert_unsafe_precondition!(
659            check_language_ub,
660            "ptr::copy requires that both pointer arguments are aligned and non-null",
661            (
662                src: *const () = src as *const (),
663                dst: *mut () = dst as *mut (),
664                align: usize = align_of::<T>(),
665                zero_size: bool = T::IS_ZST || count == 0,
666            ) =>
667            ub_checks::maybe_is_aligned_and_not_null(src, align, zero_size)
668                && ub_checks::maybe_is_aligned_and_not_null(dst, align, zero_size)
669        );
670        crate::intrinsics::copy(src, dst, count)
671    }
672}
673
674/// Sets `count * size_of::<T>()` bytes of memory starting at `dst` to
675/// `val`.
676///
677/// `write_bytes` is similar to C's [`memset`], but sets `count *
678/// size_of::<T>()` bytes to `val`.
679///
680/// [`memset`]: https://en.cppreference.com/w/c/string/byte/memset
681///
682/// # Safety
683///
684/// Behavior is undefined if any of the following conditions are violated:
685///
686/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes.
687///
688/// * `dst` must be properly aligned.
689///
690/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
691/// `0`, the pointer must be properly aligned.
692///
693/// Additionally, note that changing `*dst` in this way can easily lead to undefined behavior (UB)
694/// later if the written bytes are not a valid representation of some `T`. For instance, the
695/// following is an **incorrect** use of this function:
696///
697/// ```rust,no_run
698/// unsafe {
699///     let mut value: u8 = 0;
700///     let ptr: *mut bool = &mut value as *mut u8 as *mut bool;
701///     let _bool = ptr.read(); // This is fine, `ptr` points to a valid `bool`.
702///     ptr.write_bytes(42u8, 1); // This function itself does not cause UB...
703///     let _bool = ptr.read(); // ...but it makes this operation UB! ⚠️
704/// }
705/// ```
706///
707/// [valid]: crate::ptr#safety
708///
709/// # Examples
710///
711/// Basic usage:
712///
713/// ```
714/// use std::ptr;
715///
716/// let mut vec = vec![0u32; 4];
717/// unsafe {
718///     let vec_ptr = vec.as_mut_ptr();
719///     ptr::write_bytes(vec_ptr, 0xfe, 2);
720/// }
721/// assert_eq!(vec, [0xfefefefe, 0xfefefefe, 0, 0]);
722/// ```
723#[doc(alias = "memset")]
724#[stable(feature = "rust1", since = "1.0.0")]
725#[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")]
726#[inline(always)]
727#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
728#[rustc_diagnostic_item = "ptr_write_bytes"]
729pub const unsafe fn write_bytes<T>(dst: *mut T, val: u8, count: usize) {
730    // SAFETY: the safety contract for `write_bytes` must be upheld by the caller.
731    unsafe {
732        ub_checks::assert_unsafe_precondition!(
733            check_language_ub,
734            "ptr::write_bytes requires that the destination pointer is aligned and non-null",
735            (
736                addr: *const () = dst as *const (),
737                align: usize = align_of::<T>(),
738                zero_size: bool = T::IS_ZST || count == 0,
739            ) => ub_checks::maybe_is_aligned_and_not_null(addr, align, zero_size)
740        );
741        crate::intrinsics::write_bytes(dst, val, count)
742    }
743}
744
745/// Executes the destructor (if any) of the pointed-to value.
746///
747/// This is almost the same as calling [`ptr::read`] and discarding
748/// the result, but has the following advantages:
749// FIXME: say something more useful than "almost the same"?
750// There are open questions here: `read` requires the value to be fully valid, e.g. if `T` is a
751// `bool` it must be 0 or 1, if it is a reference then it must be dereferenceable. `drop_in_place`
752// only requires that `*to_drop` be "valid for dropping" and we have not defined what that means. In
753// Miri it currently (May 2024) requires nothing at all for types without drop glue.
754///
755/// * It is *required* to use `drop_in_place` to drop unsized types like
756///   trait objects, because they can't be read out onto the stack and
757///   dropped normally.
758///
759/// * It is friendlier to the optimizer to do this over [`ptr::read`] when
760///   dropping manually allocated memory (e.g., in the implementations of
761///   `Box`/`Rc`/`Vec`), as the compiler doesn't need to prove that it's
762///   sound to elide the copy.
763///
764/// * It can be used to drop [pinned] data when `T` is not `repr(packed)`
765///   (pinned data must not be moved before it is dropped).
766///
767/// Unaligned values cannot be dropped in place, they must be copied to an aligned
768/// location first using [`ptr::read_unaligned`]. For packed structs, this move is
769/// done automatically by the compiler. This means the fields of packed structs
770/// are not dropped in-place.
771///
772/// [`ptr::read`]: self::read
773/// [`ptr::read_unaligned`]: self::read_unaligned
774/// [pinned]: crate::pin
775///
776/// # Safety
777///
778/// Behavior is undefined if any of the following conditions are violated:
779///
780/// * `to_drop` must be [valid] for both reads and writes.
781///
782/// * `to_drop` must be properly aligned, even if `T` has size 0.
783///
784/// * `to_drop` must be nonnull, even if `T` has size 0.
785///
786/// * The value `to_drop` points to must be valid for dropping, which may mean
787///   it must uphold additional invariants. These invariants depend on the type
788///   of the value being dropped. For instance, when dropping a Box, the box's
789///   pointer to the heap must be valid.
790///
791/// * While `drop_in_place` is executing, the only way to access parts of
792///   `to_drop` is through the `&mut self` references supplied to the
793///   `Drop::drop` methods that `drop_in_place` invokes.
794///
795/// Additionally, if `T` is not [`Copy`], using the pointed-to value after
796/// calling `drop_in_place` can cause undefined behavior. Note that `*to_drop =
797/// foo` counts as a use because it will cause the value to be dropped
798/// again. [`write()`] can be used to overwrite data without causing it to be
799/// dropped.
800///
801/// [valid]: self#safety
802///
803/// # Examples
804///
805/// Manually remove the last item from a vector:
806///
807/// ```
808/// use std::ptr;
809/// use std::rc::Rc;
810///
811/// let last = Rc::new(1);
812/// let weak = Rc::downgrade(&last);
813///
814/// let mut v = vec![Rc::new(0), last];
815///
816/// unsafe {
817///     // Get a raw pointer to the last element in `v`.
818///     let ptr = &mut v[1] as *mut _;
819///     // Shorten `v` to prevent the last item from being dropped. We do that first,
820///     // to prevent issues if the `drop_in_place` below panics.
821///     v.set_len(1);
822///     // Without a call `drop_in_place`, the last item would never be dropped,
823///     // and the memory it manages would be leaked.
824///     ptr::drop_in_place(ptr);
825/// }
826///
827/// assert_eq!(v, &[0.into()]);
828///
829/// // Ensure that the last item was dropped.
830/// assert!(weak.upgrade().is_none());
831/// ```
832#[inline(always)]
833#[stable(feature = "drop_in_place", since = "1.8.0")]
834#[rustc_diagnostic_item = "ptr_drop_in_place"]
835#[rustc_const_unstable(feature = "const_drop_in_place", issue = "109342")]
836pub const unsafe fn drop_in_place<T: PointeeSized>(to_drop: *mut T)
837where
838    T: [const] Destruct,
839{
840    // Due to historic reasons, `drop_in_place` takes a pointer rather than a reference,
841    // which results in worse codegen since we don't apply noalias/dereferenceable llvm
842    // attributes to pointer arguments. To workaround this without breaking public
843    // interface, `drop_in_place` calls the lang item, rather than being one directly.
844
845    // SAFETY:
846    // - compiler glue has the same safety requirements as this function
847    // - the pointer must be valid as per the safety requirement of this function
848    unsafe { drop_glue(&mut *to_drop) }
849}
850
851/// Helper function for `drop_in_place`. The compiler replaces this by the actual drop glue.
852#[lang = "drop_glue"]
853pub(crate) const unsafe fn drop_glue<T: PointeeSized>(_: &mut T)
854where
855    T: [const] Destruct,
856{
857    // Code here does not matter - this is replaced by the
858    // real drop glue by the compiler.
859}
860
861/// Creates a null raw pointer.
862///
863/// This function is equivalent to zero-initializing the pointer:
864/// `MaybeUninit::<*const T>::zeroed().assume_init()`.
865/// The resulting pointer has the address 0.
866///
867/// # Examples
868///
869/// ```
870/// use std::ptr;
871///
872/// let p: *const i32 = ptr::null();
873/// assert!(p.is_null());
874/// assert_eq!(p as usize, 0); // this pointer has the address 0
875/// ```
876#[inline(always)]
877#[must_use]
878#[stable(feature = "rust1", since = "1.0.0")]
879#[rustc_promotable]
880#[rustc_const_stable(feature = "const_ptr_null", since = "1.24.0")]
881#[rustc_diagnostic_item = "ptr_null"]
882pub const fn null<T: PointeeSized + Thin>() -> *const T {
883    from_raw_parts(without_provenance::<()>(0), ())
884}
885
886/// Creates a null mutable raw pointer.
887///
888/// This function is equivalent to zero-initializing the pointer:
889/// `MaybeUninit::<*mut T>::zeroed().assume_init()`.
890/// The resulting pointer has the address 0.
891///
892/// # Examples
893///
894/// ```
895/// use std::ptr;
896///
897/// let p: *mut i32 = ptr::null_mut();
898/// assert!(p.is_null());
899/// assert_eq!(p as usize, 0); // this pointer has the address 0
900/// ```
901#[inline(always)]
902#[must_use]
903#[stable(feature = "rust1", since = "1.0.0")]
904#[rustc_promotable]
905#[rustc_const_stable(feature = "const_ptr_null", since = "1.24.0")]
906#[rustc_diagnostic_item = "ptr_null_mut"]
907pub const fn null_mut<T: PointeeSized + Thin>() -> *mut T {
908    from_raw_parts_mut(without_provenance_mut::<()>(0), ())
909}
910
911/// Creates a pointer with the given address and no [provenance][crate::ptr#provenance].
912///
913/// This is equivalent to `ptr::null().with_addr(addr)`.
914///
915/// Without provenance, this pointer is not associated with any actual allocation. Such a
916/// no-provenance pointer may be used for zero-sized memory accesses (if suitably aligned), but
917/// non-zero-sized memory accesses with a no-provenance pointer are UB. No-provenance pointers are
918/// little more than a `usize` address in disguise.
919///
920/// This is different from `addr as *const T`, which creates a pointer that picks up a previously
921/// exposed provenance. See [`with_exposed_provenance`] for more details on that operation.
922///
923/// This is a [Strict Provenance][crate::ptr#strict-provenance] API.
924#[inline(always)]
925#[must_use]
926#[stable(feature = "strict_provenance", since = "1.84.0")]
927#[rustc_const_stable(feature = "strict_provenance", since = "1.84.0")]
928#[rustc_diagnostic_item = "ptr_without_provenance"]
929pub const fn without_provenance<T>(addr: usize) -> *const T {
930    without_provenance_mut(addr)
931}
932
933/// Creates a new pointer that is dangling, but non-null and well-aligned.
934///
935/// This is useful for initializing types which lazily allocate, like
936/// `Vec::new` does.
937///
938/// Note that the address of the returned pointer may potentially
939/// be that of a valid pointer, which means this must not be used
940/// as a "not yet initialized" sentinel value.
941/// Types that lazily allocate must track initialization by some other means.
942#[inline(always)]
943#[must_use]
944#[stable(feature = "strict_provenance", since = "1.84.0")]
945#[rustc_const_stable(feature = "strict_provenance", since = "1.84.0")]
946pub const fn dangling<T>() -> *const T {
947    dangling_mut()
948}
949
950/// Creates a pointer with the given address and no [provenance][crate::ptr#provenance].
951///
952/// This is equivalent to `ptr::null_mut().with_addr(addr)`.
953///
954/// Without provenance, this pointer is not associated with any actual allocation. Such a
955/// no-provenance pointer may be used for zero-sized memory accesses (if suitably aligned), but
956/// non-zero-sized memory accesses with a no-provenance pointer are UB. No-provenance pointers are
957/// little more than a `usize` address in disguise.
958///
959/// This is different from `addr as *mut T`, which creates a pointer that picks up a previously
960/// exposed provenance. See [`with_exposed_provenance_mut`] for more details on that operation.
961///
962/// This is a [Strict Provenance][crate::ptr#strict-provenance] API.
963#[inline(always)]
964#[must_use]
965#[stable(feature = "strict_provenance", since = "1.84.0")]
966#[rustc_const_stable(feature = "strict_provenance", since = "1.84.0")]
967#[rustc_diagnostic_item = "ptr_without_provenance_mut"]
968#[allow(integer_to_ptr_transmutes)] // Expected semantics here.
969pub const fn without_provenance_mut<T>(addr: usize) -> *mut T {
970    // An int-to-pointer transmute currently has exactly the intended semantics: it creates a
971    // pointer without provenance. Note that this is *not* a stable guarantee about transmute
972    // semantics, it relies on sysroot crates having special status.
973    // SAFETY: every valid integer is also a valid pointer (as long as you don't dereference that
974    // pointer).
975    unsafe { mem::transmute(addr) }
976}
977
978/// Creates a new pointer that is dangling, but non-null and well-aligned.
979///
980/// This is useful for initializing types which lazily allocate, like
981/// `Vec::new` does.
982///
983/// Note that the address of the returned pointer may potentially
984/// be that of a valid pointer, which means this must not be used
985/// as a "not yet initialized" sentinel value.
986/// Types that lazily allocate must track initialization by some other means.
987#[inline(always)]
988#[must_use]
989#[stable(feature = "strict_provenance", since = "1.84.0")]
990#[rustc_const_stable(feature = "strict_provenance", since = "1.84.0")]
991pub const fn dangling_mut<T>() -> *mut T {
992    NonNull::dangling().as_ptr()
993}
994
995/// Converts an address back to a pointer, picking up some previously 'exposed'
996/// [provenance][crate::ptr#provenance].
997///
998/// This is fully equivalent to `addr as *const T`. The provenance of the returned pointer is that
999/// of *some* pointer that was previously exposed by passing it to
1000/// [`expose_provenance`][pointer::expose_provenance], or a `ptr as usize` cast. In addition, memory
1001/// which is outside the control of the Rust abstract machine (MMIO registers, for example) is
1002/// always considered to be accessible with an exposed provenance, so long as this memory is disjoint
1003/// from memory that will be used by the abstract machine such as the stack, heap, and statics.
1004///
1005/// The exact provenance that gets picked is not specified. The compiler will do its best to pick
1006/// the "right" provenance for you (whatever that may be), but currently we cannot provide any
1007/// guarantees about which provenance the resulting pointer will have -- and therefore there
1008/// is no definite specification for which memory the resulting pointer may access.
1009///
1010/// If there is *no* previously 'exposed' provenance that justifies the way the returned pointer
1011/// will be used, the program has undefined behavior. In particular, the aliasing rules still apply:
1012/// pointers and references that have been invalidated due to aliasing accesses cannot be used
1013/// anymore, even if they have been exposed!
1014///
1015/// Due to its inherent ambiguity, this operation may not be supported by tools that help you to
1016/// stay conformant with the Rust memory model. It is recommended to use [Strict
1017/// Provenance][self#strict-provenance] APIs such as [`with_addr`][pointer::with_addr] wherever
1018/// possible.
1019///
1020/// On most platforms this will produce a value with the same bytes as the address. Platforms
1021/// which need to store additional information in a pointer may not support this operation,
1022/// since it is generally not possible to actually *compute* which provenance the returned
1023/// pointer has to pick up.
1024///
1025/// This is an [Exposed Provenance][crate::ptr#exposed-provenance] API.
1026#[must_use]
1027#[inline(always)]
1028#[stable(feature = "exposed_provenance", since = "1.84.0")]
1029#[rustc_const_stable(feature = "const_exposed_provenance", since = "1.91.0")]
1030#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1031#[allow(implicit_provenance_casts)] // this *is* the explicit provenance API one should use instead
1032pub const fn with_exposed_provenance<T>(addr: usize) -> *const T {
1033    addr as *const T
1034}
1035
1036/// Converts an address back to a mutable pointer, picking up some previously 'exposed'
1037/// [provenance][crate::ptr#provenance].
1038///
1039/// This is fully equivalent to `addr as *mut T`. The provenance of the returned pointer is that
1040/// of *some* pointer that was previously exposed by passing it to
1041/// [`expose_provenance`][pointer::expose_provenance], or a `ptr as usize` cast. In addition, memory
1042/// which is outside the control of the Rust abstract machine (MMIO registers, for example) is
1043/// always considered to be accessible with an exposed provenance, so long as this memory is disjoint
1044/// from memory that will be used by the abstract machine such as the stack, heap, and statics.
1045///
1046/// The exact provenance that gets picked is not specified. The compiler will do its best to pick
1047/// the "right" provenance for you (whatever that may be), but currently we cannot provide any
1048/// guarantees about which provenance the resulting pointer will have -- and therefore there
1049/// is no definite specification for which memory the resulting pointer may access.
1050///
1051/// If there is *no* previously 'exposed' provenance that justifies the way the returned pointer
1052/// will be used, the program has undefined behavior. In particular, the aliasing rules still apply:
1053/// pointers and references that have been invalidated due to aliasing accesses cannot be used
1054/// anymore, even if they have been exposed!
1055///
1056/// Due to its inherent ambiguity, this operation may not be supported by tools that help you to
1057/// stay conformant with the Rust memory model. It is recommended to use [Strict
1058/// Provenance][self#strict-provenance] APIs such as [`with_addr`][pointer::with_addr] wherever
1059/// possible.
1060///
1061/// On most platforms this will produce a value with the same bytes as the address. Platforms
1062/// which need to store additional information in a pointer may not support this operation,
1063/// since it is generally not possible to actually *compute* which provenance the returned
1064/// pointer has to pick up.
1065///
1066/// This is an [Exposed Provenance][crate::ptr#exposed-provenance] API.
1067#[must_use]
1068#[inline(always)]
1069#[stable(feature = "exposed_provenance", since = "1.84.0")]
1070#[rustc_const_stable(feature = "const_exposed_provenance", since = "1.91.0")]
1071#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1072#[allow(implicit_provenance_casts)] // this *is* the explicit provenance API one should use instead
1073pub const fn with_exposed_provenance_mut<T>(addr: usize) -> *mut T {
1074    addr as *mut T
1075}
1076
1077/// Converts a reference to a raw pointer.
1078///
1079/// For `r: &T`, `from_ref(r)` is equivalent to `r as *const T` (except for the caveat noted below),
1080/// but is a bit safer since it will never silently change type or mutability, in particular if the
1081/// code is refactored.
1082///
1083/// The caller must ensure that the pointee outlives the pointer this function returns, or else it
1084/// will end up dangling.
1085///
1086/// The caller must also ensure that the memory the pointer (non-transitively) points to is never
1087/// written to (except inside an `UnsafeCell`) using this pointer or any pointer derived from it. If
1088/// you need to mutate the pointee, use [`from_mut`]. Specifically, to turn a mutable reference `m:
1089/// &mut T` into `*const T`, prefer `from_mut(m).cast_const()` to obtain a pointer that can later be
1090/// used for mutation.
1091///
1092/// ## Interaction with lifetime extension
1093///
1094/// Note that this has subtle interactions with the rules for lifetime extension of temporaries in
1095/// tail expressions. This code is valid, albeit in a non-obvious way:
1096/// ```rust
1097/// # type T = i32;
1098/// # fn foo() -> T { 42 }
1099/// // The temporary holding the return value of `foo` has its lifetime extended,
1100/// // because the surrounding expression involves no function call.
1101/// let p = &foo() as *const T;
1102/// unsafe { p.read() };
1103/// ```
1104/// Naively replacing the cast with `from_ref` is not valid:
1105/// ```rust,no_run
1106/// # use std::ptr;
1107/// # type T = i32;
1108/// # fn foo() -> T { 42 }
1109/// // The temporary holding the return value of `foo` does *not* have its lifetime extended,
1110/// // because the surrounding expression involves a function call.
1111/// let p = ptr::from_ref(&foo());
1112/// unsafe { p.read() }; // UB! Reading from a dangling pointer ⚠️
1113/// ```
1114/// The recommended way to write this code is to avoid relying on lifetime extension
1115/// when raw pointers are involved:
1116/// ```rust
1117/// # use std::ptr;
1118/// # type T = i32;
1119/// # fn foo() -> T { 42 }
1120/// let x = foo();
1121/// let p = ptr::from_ref(&x);
1122/// unsafe { p.read() };
1123/// ```
1124#[inline(always)]
1125#[must_use]
1126#[stable(feature = "ptr_from_ref", since = "1.76.0")]
1127#[rustc_const_stable(feature = "ptr_from_ref", since = "1.76.0")]
1128#[lint::never_returns_null_ptr]
1129#[rustc_diagnostic_item = "ptr_from_ref"]
1130pub const fn from_ref<T: PointeeSized>(r: &T) -> *const T {
1131    r
1132}
1133
1134/// Converts a mutable reference to a raw pointer.
1135///
1136/// For `r: &mut T`, `from_mut(r)` is equivalent to `r as *mut T` (except for the caveat noted
1137/// below), but is a bit safer since it will never silently change type or mutability, in particular
1138/// if the code is refactored.
1139///
1140/// The caller must ensure that the pointee outlives the pointer this function returns, or else it
1141/// will end up dangling.
1142///
1143/// ## Interaction with lifetime extension
1144///
1145/// Note that this has subtle interactions with the rules for lifetime extension of temporaries in
1146/// tail expressions. This code is valid, albeit in a non-obvious way:
1147/// ```rust
1148/// # type T = i32;
1149/// # fn foo() -> T { 42 }
1150/// // The temporary holding the return value of `foo` has its lifetime extended,
1151/// // because the surrounding expression involves no function call.
1152/// let p = &mut foo() as *mut T;
1153/// unsafe { p.write(T::default()) };
1154/// ```
1155/// Naively replacing the cast with `from_mut` is not valid:
1156/// ```rust,no_run
1157/// # use std::ptr;
1158/// # type T = i32;
1159/// # fn foo() -> T { 42 }
1160/// // The temporary holding the return value of `foo` does *not* have its lifetime extended,
1161/// // because the surrounding expression involves a function call.
1162/// let p = ptr::from_mut(&mut foo());
1163/// unsafe { p.write(T::default()) }; // UB! Writing to a dangling pointer ⚠️
1164/// ```
1165/// The recommended way to write this code is to avoid relying on lifetime extension
1166/// when raw pointers are involved:
1167/// ```rust
1168/// # use std::ptr;
1169/// # type T = i32;
1170/// # fn foo() -> T { 42 }
1171/// let mut x = foo();
1172/// let p = ptr::from_mut(&mut x);
1173/// unsafe { p.write(T::default()) };
1174/// ```
1175#[inline(always)]
1176#[must_use]
1177#[stable(feature = "ptr_from_ref", since = "1.76.0")]
1178#[rustc_const_stable(feature = "ptr_from_ref", since = "1.76.0")]
1179#[lint::never_returns_null_ptr]
1180pub const fn from_mut<T: PointeeSized>(r: &mut T) -> *mut T {
1181    r
1182}
1183
1184/// Forms a raw slice from a pointer and a length.
1185///
1186/// The `len` argument is the number of **elements**, not the number of bytes.
1187///
1188/// This function is safe, but actually using the return value is unsafe.
1189/// See the documentation of [`slice::from_raw_parts`] for slice safety requirements.
1190///
1191/// [`slice::from_raw_parts`]: crate::slice::from_raw_parts
1192///
1193/// # Examples
1194///
1195/// ```rust
1196/// use std::ptr;
1197///
1198/// // create a slice pointer when starting out with a pointer to the first element
1199/// let x = [5, 6, 7];
1200/// let raw_pointer = x.as_ptr();
1201/// let slice = ptr::slice_from_raw_parts(raw_pointer, 3);
1202/// assert_eq!(unsafe { &*slice }[2], 7);
1203/// ```
1204///
1205/// You must ensure that the pointer is valid and not null before dereferencing
1206/// the raw slice. A slice reference must never have a null pointer, even if it's empty.
1207///
1208/// ```rust,should_panic
1209/// use std::ptr;
1210/// let danger: *const [u8] = ptr::slice_from_raw_parts(ptr::null(), 0);
1211/// unsafe {
1212///     danger.as_ref().expect("references must not be null");
1213/// }
1214/// ```
1215#[inline]
1216#[stable(feature = "slice_from_raw_parts", since = "1.42.0")]
1217#[rustc_const_stable(feature = "const_slice_from_raw_parts", since = "1.64.0")]
1218#[rustc_diagnostic_item = "ptr_slice_from_raw_parts"]
1219pub const fn slice_from_raw_parts<T>(data: *const T, len: usize) -> *const [T] {
1220    from_raw_parts(data, len)
1221}
1222
1223/// Forms a raw mutable slice from a pointer and a length.
1224///
1225/// The `len` argument is the number of **elements**, not the number of bytes.
1226///
1227/// Performs the same functionality as [`slice_from_raw_parts`], except that a
1228/// raw mutable slice is returned, as opposed to a raw immutable slice.
1229///
1230/// This function is safe, but actually using the return value is unsafe.
1231/// See the documentation of [`slice::from_raw_parts_mut`] for slice safety requirements.
1232///
1233/// [`slice::from_raw_parts_mut`]: crate::slice::from_raw_parts_mut
1234///
1235/// # Examples
1236///
1237/// ```rust
1238/// use std::ptr;
1239///
1240/// let x = &mut [5, 6, 7];
1241/// let raw_pointer = x.as_mut_ptr();
1242/// let slice = ptr::slice_from_raw_parts_mut(raw_pointer, 3);
1243///
1244/// unsafe {
1245///     (*slice)[2] = 99; // assign a value at an index in the slice
1246/// };
1247///
1248/// assert_eq!(unsafe { &*slice }[2], 99);
1249/// ```
1250///
1251/// You must ensure that the pointer is valid and not null before dereferencing
1252/// the raw slice. A slice reference must never have a null pointer, even if it's empty.
1253///
1254/// ```rust,should_panic
1255/// use std::ptr;
1256/// let danger: *mut [u8] = ptr::slice_from_raw_parts_mut(ptr::null_mut(), 0);
1257/// unsafe {
1258///     danger.as_mut().expect("references must not be null");
1259/// }
1260/// ```
1261#[inline]
1262#[stable(feature = "slice_from_raw_parts", since = "1.42.0")]
1263#[rustc_const_stable(feature = "const_slice_from_raw_parts_mut", since = "1.83.0")]
1264#[rustc_diagnostic_item = "ptr_slice_from_raw_parts_mut"]
1265pub const fn slice_from_raw_parts_mut<T>(data: *mut T, len: usize) -> *mut [T] {
1266    from_raw_parts_mut(data, len)
1267}
1268
1269/// Swaps the values at two mutable locations of the same type, without
1270/// deinitializing either.
1271///
1272/// But for the following exceptions, this function is semantically
1273/// equivalent to [`mem::swap`]:
1274///
1275/// * It operates on raw pointers instead of references. When references are
1276///   available, [`mem::swap`] should be preferred.
1277///
1278/// * The two pointed-to values may overlap. If the values do overlap, then the
1279///   overlapping region of memory from `x` will be used. This is demonstrated
1280///   in the second example below.
1281///
1282/// * The operation is "untyped" in the sense that data may be uninitialized or otherwise violate
1283///   the requirements of `T`. The initialization state is preserved exactly.
1284///
1285/// # Safety
1286///
1287/// Behavior is undefined if any of the following conditions are violated:
1288///
1289/// * Both `x` and `y` must be [valid] for both reads and writes. They must remain valid even when the
1290///   other pointer is written. (This means if the memory ranges overlap, the two pointers must not
1291///   be subject to aliasing restrictions relative to each other.)
1292///
1293/// * Both `x` and `y` must be properly aligned.
1294///
1295/// Note that even if `T` has size `0`, the pointers must be properly aligned.
1296///
1297/// [valid]: self#safety
1298///
1299/// # Examples
1300///
1301/// Swapping two non-overlapping regions:
1302///
1303/// ```
1304/// use std::ptr;
1305///
1306/// let mut array = [0, 1, 2, 3];
1307///
1308/// let (x, y) = array.split_at_mut(2);
1309/// let x = x.as_mut_ptr().cast::<[u32; 2]>(); // this is `array[0..2]`
1310/// let y = y.as_mut_ptr().cast::<[u32; 2]>(); // this is `array[2..4]`
1311///
1312/// unsafe {
1313///     ptr::swap(x, y);
1314///     assert_eq!([2, 3, 0, 1], array);
1315/// }
1316/// ```
1317///
1318/// Swapping two overlapping regions:
1319///
1320/// ```
1321/// use std::ptr;
1322///
1323/// let mut array: [i32; 4] = [0, 1, 2, 3];
1324///
1325/// let array_ptr: *mut i32 = array.as_mut_ptr();
1326///
1327/// let x = array_ptr as *mut [i32; 3]; // this is `array[0..3]`
1328/// let y = unsafe { array_ptr.add(1) } as *mut [i32; 3]; // this is `array[1..4]`
1329///
1330/// unsafe {
1331///     ptr::swap(x, y);
1332///     // The indices `1..3` of the slice overlap between `x` and `y`.
1333///     // Reasonable results would be for to them be `[2, 3]`, so that indices `0..3` are
1334///     // `[1, 2, 3]` (matching `y` before the `swap`); or for them to be `[0, 1]`
1335///     // so that indices `1..4` are `[0, 1, 2]` (matching `x` before the `swap`).
1336///     // This implementation is defined to make the latter choice.
1337///     assert_eq!([1, 0, 1, 2], array);
1338/// }
1339/// ```
1340#[inline(always)]
1341#[stable(feature = "rust1", since = "1.0.0")]
1342#[rustc_const_stable(feature = "const_swap", since = "1.85.0")]
1343#[rustc_diagnostic_item = "ptr_swap"]
1344pub const unsafe fn swap<T>(x: *mut T, y: *mut T) {
1345    // Give ourselves some scratch space to work with.
1346    // We do not have to worry about drops: `MaybeUninit` does nothing when dropped.
1347    let mut tmp = MaybeUninit::<T>::uninit();
1348
1349    // Perform the swap
1350    // SAFETY: the caller must guarantee that `x` and `y` are
1351    // valid for writes and properly aligned. `tmp` cannot be
1352    // overlapping either `x` or `y` because `tmp` was just allocated
1353    // on the stack as a separate allocation.
1354    unsafe {
1355        copy_nonoverlapping(x, tmp.as_mut_ptr(), 1);
1356        copy(y, x, 1); // `x` and `y` may overlap
1357        copy_nonoverlapping(tmp.as_ptr(), y, 1);
1358    }
1359}
1360
1361/// Swaps `count * size_of::<T>()` bytes between the two regions of memory
1362/// beginning at `x` and `y`. The two regions must *not* overlap.
1363///
1364/// The operation is "untyped" in the sense that data may be uninitialized or otherwise violate the
1365/// requirements of `T`. The initialization state is preserved exactly.
1366///
1367/// # Safety
1368///
1369/// Behavior is undefined if any of the following conditions are violated:
1370///
1371/// * Both `x` and `y` must be [valid] for both reads and writes of `count *
1372///   size_of::<T>()` bytes.
1373///
1374/// * Both `x` and `y` must be properly aligned.
1375///
1376/// * The region of memory beginning at `x` with a size of `count *
1377///   size_of::<T>()` bytes must *not* overlap with the region of memory
1378///   beginning at `y` with the same size.
1379///
1380/// Note that even if the effectively copied size (`count * size_of::<T>()`) is `0`,
1381/// the pointers must be properly aligned.
1382///
1383/// [valid]: self#safety
1384///
1385/// # Examples
1386///
1387/// Basic usage:
1388///
1389/// ```
1390/// use std::ptr;
1391///
1392/// let mut x = [1, 2, 3, 4];
1393/// let mut y = [7, 8, 9];
1394///
1395/// unsafe {
1396///     ptr::swap_nonoverlapping(x.as_mut_ptr(), y.as_mut_ptr(), 2);
1397/// }
1398///
1399/// assert_eq!(x, [7, 8, 3, 4]);
1400/// assert_eq!(y, [1, 2, 9]);
1401/// ```
1402#[inline]
1403#[stable(feature = "swap_nonoverlapping", since = "1.27.0")]
1404#[rustc_const_stable(feature = "const_swap_nonoverlapping", since = "1.88.0")]
1405#[rustc_diagnostic_item = "ptr_swap_nonoverlapping"]
1406#[rustc_allow_const_fn_unstable(const_eval_select)] // both implementations behave the same
1407#[track_caller]
1408pub const unsafe fn swap_nonoverlapping<T>(x: *mut T, y: *mut T, count: usize) {
1409    ub_checks::assert_unsafe_precondition!(
1410        check_library_ub,
1411        "ptr::swap_nonoverlapping requires that both pointer arguments are aligned and non-null \
1412        and the specified memory ranges do not overlap",
1413        (
1414            x: *mut () = x as *mut (),
1415            y: *mut () = y as *mut (),
1416            size: usize = size_of::<T>(),
1417            align: usize = align_of::<T>(),
1418            count: usize = count,
1419        ) => {
1420            let zero_size = size == 0 || count == 0;
1421            ub_checks::maybe_is_aligned_and_not_null(x, align, zero_size)
1422                && ub_checks::maybe_is_aligned_and_not_null(y, align, zero_size)
1423                && ub_checks::maybe_is_nonoverlapping(x, y, size, count)
1424        }
1425    );
1426
1427    const_eval_select!(
1428        @capture[T] { x: *mut T, y: *mut T, count: usize }:
1429        if const {
1430            // At compile-time we don't need all the special code below.
1431            // SAFETY: Same preconditions as this function
1432            unsafe { swap_nonoverlapping_const(x, y, count) }
1433        } else {
1434            // Going though a slice here helps codegen know the size fits in `isize`
1435            let slice = slice_from_raw_parts_mut(x, count);
1436            // SAFETY: This is all readable from the pointer, meaning it's one
1437            // allocation, and thus cannot be more than isize::MAX bytes.
1438            let bytes = unsafe { mem::size_of_val_raw::<[T]>(slice) };
1439            if let Some(bytes) = NonZero::new(bytes) {
1440                // SAFETY: These are the same ranges, just expressed in a different
1441                // type, so they're still non-overlapping.
1442                unsafe { swap_nonoverlapping_bytes(x.cast(), y.cast(), bytes) };
1443            }
1444        }
1445    )
1446}
1447
1448/// Same behavior and safety conditions as [`swap_nonoverlapping`]
1449#[inline]
1450const unsafe fn swap_nonoverlapping_const<T>(x: *mut T, y: *mut T, count: usize) {
1451    let mut i = 0;
1452    while i < count {
1453        // SAFETY: By precondition, `i` is in-bounds because it's below `n`
1454        let x = unsafe { x.add(i) };
1455        // SAFETY: By precondition, `i` is in-bounds because it's below `n`
1456        // and it's distinct from `x` since the ranges are non-overlapping
1457        let y = unsafe { y.add(i) };
1458
1459        // SAFETY: we're only ever given pointers that are valid to read/write,
1460        // including being aligned, and nothing here panics so it's drop-safe.
1461        unsafe {
1462            // Note that it's critical that these use `copy_nonoverlapping`,
1463            // rather than `read`/`write`, to avoid #134713 if T has padding.
1464            let mut temp = MaybeUninit::<T>::uninit();
1465            copy_nonoverlapping(x, temp.as_mut_ptr(), 1);
1466            copy_nonoverlapping(y, x, 1);
1467            copy_nonoverlapping(temp.as_ptr(), y, 1);
1468        }
1469
1470        i += 1;
1471    }
1472}
1473
1474// Don't let MIR inline this, because we really want it to keep its noalias metadata
1475#[rustc_no_mir_inline]
1476#[inline]
1477fn swap_chunk<const N: usize>(x: &mut MaybeUninit<[u8; N]>, y: &mut MaybeUninit<[u8; N]>) {
1478    let a = *x;
1479    let b = *y;
1480    *x = b;
1481    *y = a;
1482}
1483
1484#[inline]
1485unsafe fn swap_nonoverlapping_bytes(x: *mut u8, y: *mut u8, bytes: NonZero<usize>) {
1486    // Same as `swap_nonoverlapping::<[u8; N]>`.
1487    unsafe fn swap_nonoverlapping_chunks<const N: usize>(
1488        x: *mut MaybeUninit<[u8; N]>,
1489        y: *mut MaybeUninit<[u8; N]>,
1490        chunks: NonZero<usize>,
1491    ) {
1492        let chunks = chunks.get();
1493        for i in 0..chunks {
1494            // SAFETY: i is in [0, chunks) so the adds and dereferences are in-bounds.
1495            unsafe { swap_chunk(&mut *x.add(i), &mut *y.add(i)) };
1496        }
1497    }
1498
1499    // Same as `swap_nonoverlapping_bytes`, but accepts at most 1+2+4=7 bytes
1500    #[inline]
1501    unsafe fn swap_nonoverlapping_short(x: *mut u8, y: *mut u8, bytes: NonZero<usize>) {
1502        // Tail handling for auto-vectorized code sometimes has element-at-a-time behaviour,
1503        // see <https://github.com/rust-lang/rust/issues/134946>.
1504        // By swapping as different sizes, rather than as a loop over bytes,
1505        // we make sure not to end up with, say, seven byte-at-a-time copies.
1506
1507        let bytes = bytes.get();
1508        let mut i = 0;
1509        macro_rules! swap_prefix {
1510            ($($n:literal)+) => {$(
1511                if (bytes & $n) != 0 {
1512                    // SAFETY: `i` can only have the same bits set as those in bytes,
1513                    // so these `add`s are in-bounds of `bytes`.  But the bit for
1514                    // `$n` hasn't been set yet, so the `$n` bytes that `swap_chunk`
1515                    // will read and write are within the usable range.
1516                    unsafe { swap_chunk::<$n>(&mut*x.add(i).cast(), &mut*y.add(i).cast()) };
1517                    i |= $n;
1518                }
1519            )+};
1520        }
1521        swap_prefix!(4 2 1);
1522        debug_assert_eq!(i, bytes);
1523    }
1524
1525    const CHUNK_SIZE: usize = size_of::<*const ()>();
1526    let bytes = bytes.get();
1527
1528    let chunks = bytes / CHUNK_SIZE;
1529    let tail = bytes % CHUNK_SIZE;
1530    if let Some(chunks) = NonZero::new(chunks) {
1531        // SAFETY: this is bytes/CHUNK_SIZE*CHUNK_SIZE bytes, which is <= bytes,
1532        // so it's within the range of our non-overlapping bytes.
1533        unsafe { swap_nonoverlapping_chunks::<CHUNK_SIZE>(x.cast(), y.cast(), chunks) };
1534    }
1535    if let Some(tail) = NonZero::new(tail) {
1536        const { assert!(CHUNK_SIZE <= 8) };
1537        let delta = chunks * CHUNK_SIZE;
1538        // SAFETY: the tail length is below CHUNK SIZE because of the remainder,
1539        // and CHUNK_SIZE is at most 8 by the const assert, so tail <= 7
1540        unsafe { swap_nonoverlapping_short(x.add(delta), y.add(delta), tail) };
1541    }
1542}
1543
1544/// Moves `src` into the pointed `dst`, returning the previous `dst` value.
1545///
1546/// Neither value is dropped.
1547///
1548/// This function is semantically equivalent to [`mem::replace`] except that it
1549/// operates on raw pointers instead of references. When references are
1550/// available, [`mem::replace`] should be preferred.
1551///
1552/// # Safety
1553///
1554/// Behavior is undefined if any of the following conditions are violated:
1555///
1556/// * `dst` must be [valid] for both reads and writes or `T` must be a ZST.
1557///
1558/// * `dst` must be properly aligned.
1559///
1560/// * `dst` must point to a properly initialized value of type `T`.
1561///
1562/// Note that even if `T` has size `0`, the pointer must be properly aligned.
1563///
1564/// [valid]: self#safety
1565///
1566/// # Examples
1567///
1568/// ```
1569/// use std::ptr;
1570///
1571/// let mut rust = vec!['b', 'u', 's', 't'];
1572///
1573/// // `mem::replace` would have the same effect without requiring the unsafe
1574/// // block.
1575/// let b = unsafe {
1576///     ptr::replace(&mut rust[0], 'r')
1577/// };
1578///
1579/// assert_eq!(b, 'b');
1580/// assert_eq!(rust, &['r', 'u', 's', 't']);
1581/// ```
1582#[inline(always)]
1583#[stable(feature = "rust1", since = "1.0.0")]
1584#[rustc_const_stable(feature = "const_replace", since = "1.83.0")]
1585#[rustc_diagnostic_item = "ptr_replace"]
1586#[track_caller]
1587pub const unsafe fn replace<T>(dst: *mut T, src: T) -> T {
1588    // SAFETY: the caller must guarantee that `dst` is valid to be
1589    // cast to a mutable reference (valid for writes, aligned, initialized),
1590    // and cannot overlap `src` since `dst` must point to a distinct
1591    // allocation. We are excluding null (with a ZST check) before creating a reference.
1592    unsafe {
1593        ub_checks::assert_unsafe_precondition!(
1594            check_language_ub,
1595            "ptr::replace requires that the pointer argument is aligned and non-null",
1596            (
1597                addr: *const () = dst as *const (),
1598                align: usize = align_of::<T>(),
1599                is_zst: bool = T::IS_ZST,
1600            ) => ub_checks::maybe_is_aligned_and_not_null(addr, align, is_zst)
1601        );
1602        if T::IS_ZST {
1603            // If `T` is a ZST, `dst` is allowed to be null. However, we also don't have to actually
1604            // do anything since there isn't actually any data to be copied anyway. All values of
1605            // type `T` are bit-identical, so we can just return `src` here.
1606            return src;
1607        }
1608        mem::replace(&mut *dst, src)
1609    }
1610}
1611
1612/// Reads the value from `src` without moving it. This leaves the
1613/// memory in `src` unchanged.
1614///
1615/// # Safety
1616///
1617/// Behavior is undefined if any of the following conditions are violated:
1618///
1619/// * `src` must be [valid] for reads or `T` must be a ZST.
1620///
1621/// * `src` must be properly aligned. Use [`read_unaligned`] if this is not the
1622///   case.
1623///
1624/// * `src` must point to a properly initialized value of type `T`.
1625///
1626/// Note that even if `T` has size `0`, the pointer must be properly aligned.
1627///
1628/// Care must be taken around [ownership of the returned value](#ownership-of-the-returned-value).
1629///
1630/// # Examples
1631///
1632/// Basic usage:
1633///
1634/// ```
1635/// let x = 12;
1636/// let y = &x as *const i32;
1637///
1638/// unsafe {
1639///     assert_eq!(std::ptr::read(y), 12);
1640/// }
1641/// ```
1642///
1643/// Manually implement [`mem::swap`]:
1644///
1645/// ```
1646/// use std::ptr;
1647///
1648/// fn swap<T>(a: &mut T, b: &mut T) {
1649///     unsafe {
1650///         // Create a bitwise copy of the value at `a` in `tmp`.
1651///         let tmp = ptr::read(a);
1652///
1653///         // Exiting at this point (either by explicitly returning or by
1654///         // calling a function which panics) would cause the value in `tmp` to
1655///         // be dropped while the same value is still referenced by `a`. This
1656///         // could trigger undefined behavior if `T` is not `Copy`.
1657///
1658///         // Create a bitwise copy of the value at `b` in `a`.
1659///         // This is safe because mutable references cannot alias.
1660///         ptr::copy_nonoverlapping(b, a, 1);
1661///
1662///         // As above, exiting here could trigger undefined behavior because
1663///         // the same value is referenced by `a` and `b`.
1664///
1665///         // Move `tmp` into `b`.
1666///         ptr::write(b, tmp);
1667///
1668///         // `tmp` has been moved (`write` takes ownership of its second argument),
1669///         // so nothing is dropped implicitly here.
1670///     }
1671/// }
1672///
1673/// let mut foo = "foo".to_owned();
1674/// let mut bar = "bar".to_owned();
1675///
1676/// swap(&mut foo, &mut bar);
1677///
1678/// assert_eq!(foo, "bar");
1679/// assert_eq!(bar, "foo");
1680/// ```
1681///
1682/// ## Ownership of the Returned Value
1683///
1684/// `read` creates a bitwise copy of `T`, regardless of whether `T` is [`Copy`].
1685/// If `T` is not [`Copy`], using both the returned value and the value at
1686/// `*src` can violate memory safety. Note that assigning to `*src` counts as a
1687/// use because it will attempt to drop the value at `*src`.
1688///
1689/// [`write()`] can be used to overwrite data without causing it to be dropped.
1690///
1691/// ```
1692/// use std::ptr;
1693///
1694/// let mut s = String::from("foo");
1695/// unsafe {
1696///     // `s2` now points to the same underlying memory as `s`.
1697///     let mut s2: String = ptr::read(&s);
1698///
1699///     assert_eq!(s2, "foo");
1700///
1701///     // Assigning to `s2` causes its original value to be dropped. Beyond
1702///     // this point, `s` must no longer be used, as the underlying memory has
1703///     // been freed.
1704///     s2 = String::default();
1705///     assert_eq!(s2, "");
1706///
1707///     // Assigning to `s` would cause the old value to be dropped again,
1708///     // resulting in undefined behavior.
1709///     // s = String::from("bar"); // ERROR
1710///
1711///     // `ptr::write` can be used to overwrite a value without dropping it.
1712///     ptr::write(&mut s, String::from("bar"));
1713/// }
1714///
1715/// assert_eq!(s, "bar");
1716/// ```
1717///
1718/// [valid]: self#safety
1719#[inline(always)]
1720#[stable(feature = "rust1", since = "1.0.0")]
1721#[rustc_const_stable(feature = "const_ptr_read", since = "1.71.0")]
1722#[track_caller]
1723#[rustc_diagnostic_item = "ptr_read"]
1724pub const unsafe fn read<T>(src: *const T) -> T {
1725    // It would be semantically correct to implement this via `copy_nonoverlapping`
1726    // and `MaybeUninit`, as was done before PR #109035. Calling `assume_init`
1727    // provides enough information to know that this is a typed operation.
1728
1729    // However, as of March 2023 the compiler was not capable of taking advantage
1730    // of that information. Thus, the implementation here switched to an intrinsic,
1731    // which lowers to `_0 = *src` in MIR, to address a few issues:
1732    //
1733    // - Using `MaybeUninit::assume_init` after a `copy_nonoverlapping` was not
1734    //   turning the untyped copy into a typed load. As such, the generated
1735    //   `load` in LLVM didn't get various metadata, such as `!range` (#73258),
1736    //   `!nonnull`, and `!noundef`, resulting in poorer optimization.
1737    // - Going through the extra local resulted in multiple extra copies, even
1738    //   in optimized MIR.  (Ignoring StorageLive/Dead, the intrinsic is one
1739    //   MIR statement, while the previous implementation was eight.)  LLVM
1740    //   could sometimes optimize them away, but because `read` is at the core
1741    //   of so many things, not having them in the first place improves what we
1742    //   hand off to the backend.  For example, `mem::replace::<Big>` previously
1743    //   emitted 4 `alloca` and 6 `memcpy`s, but is now 1 `alloc` and 3 `memcpy`s.
1744    // - In general, this approach keeps us from getting any more bugs (like
1745    //   #106369) that boil down to "`read(p)` is worse than `*p`", as this
1746    //   makes them look identical to the backend (or other MIR consumers).
1747    //
1748    // Future enhancements to MIR optimizations might well allow this to return
1749    // to the previous implementation, rather than using an intrinsic.
1750
1751    // SAFETY: the caller must guarantee that `src` is valid for reads.
1752    unsafe {
1753        #[cfg(debug_assertions)] // Too expensive to always enable (for now?)
1754        ub_checks::assert_unsafe_precondition!(
1755            check_language_ub,
1756            "ptr::read requires that the pointer argument is aligned and non-null",
1757            (
1758                addr: *const () = src as *const (),
1759                align: usize = align_of::<T>(),
1760                is_zst: bool = T::IS_ZST,
1761            ) => ub_checks::maybe_is_aligned_and_not_null(addr, align, is_zst)
1762        );
1763        crate::intrinsics::read_via_copy(src)
1764    }
1765}
1766
1767/// Reads the value from `src` without moving it. This leaves the
1768/// memory in `src` unchanged.
1769///
1770/// Unlike [`read`], `read_unaligned` works with unaligned pointers.
1771///
1772/// # Safety
1773///
1774/// Behavior is undefined if any of the following conditions are violated:
1775///
1776/// * `src` must be [valid] for reads.
1777///
1778/// * `src` must point to a properly initialized value of type `T`.
1779///
1780/// Like [`read`], `read_unaligned` creates a bitwise copy of `T`, regardless of
1781/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using both the returned
1782/// value and the value at `*src` can [violate memory safety][read-ownership].
1783///
1784/// [read-ownership]: read#ownership-of-the-returned-value
1785/// [valid]: self#safety
1786///
1787/// ## On `packed` structs
1788///
1789/// Attempting to create a raw pointer to an `unaligned` struct field with
1790/// an expression such as `&packed.unaligned as *const FieldType` creates an
1791/// intermediate unaligned reference before converting that to a raw pointer.
1792/// That this reference is temporary and immediately cast is inconsequential
1793/// as the compiler always expects references to be properly aligned.
1794/// As a result, using `&packed.unaligned as *const FieldType` causes immediate
1795/// *undefined behavior* in your program.
1796///
1797/// Instead you must use the `&raw const` syntax to create the pointer.
1798/// You may use that constructed pointer together with this function.
1799///
1800/// An example of what not to do and how this relates to `read_unaligned` is:
1801///
1802/// ```
1803/// #[repr(packed, C)]
1804/// struct Packed {
1805///     _padding: u8,
1806///     unaligned: u32,
1807/// }
1808///
1809/// let packed = Packed {
1810///     _padding: 0x00,
1811///     unaligned: 0x01020304,
1812/// };
1813///
1814/// // Take the address of a 32-bit integer which is not aligned.
1815/// // In contrast to `&packed.unaligned as *const _`, this has no undefined behavior.
1816/// let unaligned = &raw const packed.unaligned;
1817///
1818/// let v = unsafe { std::ptr::read_unaligned(unaligned) };
1819/// assert_eq!(v, 0x01020304);
1820/// ```
1821///
1822/// Accessing unaligned fields directly with e.g. `packed.unaligned` is safe however.
1823///
1824/// # Examples
1825///
1826/// Read a `usize` value from a byte buffer:
1827///
1828/// ```
1829/// fn read_usize(x: &[u8]) -> usize {
1830///     assert!(x.len() >= size_of::<usize>());
1831///
1832///     let ptr = x.as_ptr() as *const usize;
1833///
1834///     unsafe { ptr.read_unaligned() }
1835/// }
1836/// ```
1837#[inline(always)]
1838#[stable(feature = "ptr_unaligned", since = "1.17.0")]
1839#[rustc_const_stable(feature = "const_ptr_read", since = "1.71.0")]
1840#[track_caller]
1841#[rustc_diagnostic_item = "ptr_read_unaligned"]
1842pub const unsafe fn read_unaligned<T>(src: *const T) -> T {
1843    // Always true thanks to the repr, but to demonstrate
1844    const {
1845        assert!(mem::offset_of!(Unaligned::<T>, 0) == 0);
1846        assert!(size_of::<T>() == size_of::<Unaligned<T>>());
1847    }
1848
1849    let src = src.cast::<Unaligned<T>>();
1850    // SAFETY: the caller must guarantee that `src` is valid for reads.
1851    // Reading it as `Unaligned<T>` instead of `T` reads those same bytes because
1852    // it's the same size (thus zero offset), but with alignment 1 instead.
1853    //
1854    // Similarly, because it's the same bytes it's sound to transmute from the
1855    // `Unaligned<T>` to `T`.  Transmute is a value-based (not a place-based)
1856    // operation that doesn't care about alignment.
1857    unsafe {
1858        let unaligned = read(src);
1859        // Can't just destructure because that's not allowed in const fn
1860        mem::transmute_neo(unaligned)
1861    }
1862}
1863
1864/// Overwrites a memory location with the given value without reading or
1865/// dropping the old value.
1866///
1867/// `write` does not drop the contents of `dst`. This is safe, but it could leak
1868/// allocations or resources, so care should be taken not to overwrite an object
1869/// that should be dropped.
1870///
1871/// Additionally, it does not drop `src`. Semantically, `src` is moved into the
1872/// location pointed to by `dst`.
1873///
1874/// This is appropriate for initializing uninitialized memory, or overwriting
1875/// memory that has previously been [`read`] from.
1876///
1877/// # Safety
1878///
1879/// Behavior is undefined if any of the following conditions are violated:
1880///
1881/// * `dst` must be [valid] for writes or `T` must be a ZST.
1882///
1883/// * `dst` must be properly aligned. Use [`write_unaligned`] if this is not the
1884///   case.
1885///
1886/// Note that even if `T` has size `0`, the pointer must be properly aligned.
1887///
1888/// [valid]: self#safety
1889///
1890/// # Examples
1891///
1892/// Basic usage:
1893///
1894/// ```
1895/// let mut x = 0;
1896/// let y = &mut x as *mut i32;
1897/// let z = 12;
1898///
1899/// unsafe {
1900///     std::ptr::write(y, z);
1901///     assert_eq!(std::ptr::read(y), 12);
1902/// }
1903/// ```
1904///
1905/// Manually implement [`mem::swap`]:
1906///
1907/// ```
1908/// use std::ptr;
1909///
1910/// fn swap<T>(a: &mut T, b: &mut T) {
1911///     unsafe {
1912///         // Create a bitwise copy of the value at `a` in `tmp`.
1913///         let tmp = ptr::read(a);
1914///
1915///         // Exiting at this point (either by explicitly returning or by
1916///         // calling a function which panics) would cause the value in `tmp` to
1917///         // be dropped while the same value is still referenced by `a`. This
1918///         // could trigger undefined behavior if `T` is not `Copy`.
1919///
1920///         // Create a bitwise copy of the value at `b` in `a`.
1921///         // This is safe because mutable references cannot alias.
1922///         ptr::copy_nonoverlapping(b, a, 1);
1923///
1924///         // As above, exiting here could trigger undefined behavior because
1925///         // the same value is referenced by `a` and `b`.
1926///
1927///         // Move `tmp` into `b`.
1928///         ptr::write(b, tmp);
1929///
1930///         // `tmp` has been moved (`write` takes ownership of its second argument),
1931///         // so nothing is dropped implicitly here.
1932///     }
1933/// }
1934///
1935/// let mut foo = "foo".to_owned();
1936/// let mut bar = "bar".to_owned();
1937///
1938/// swap(&mut foo, &mut bar);
1939///
1940/// assert_eq!(foo, "bar");
1941/// assert_eq!(bar, "foo");
1942/// ```
1943#[inline(always)]
1944#[stable(feature = "rust1", since = "1.0.0")]
1945#[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")]
1946#[rustc_diagnostic_item = "ptr_write"]
1947#[track_caller]
1948pub const unsafe fn write<T>(dst: *mut T, src: T) {
1949    // Semantically, it would be fine for this to be implemented as a
1950    // `copy_nonoverlapping` and appropriate drop suppression of `src`.
1951
1952    // However, implementing via that currently produces more MIR than is ideal.
1953    // Using an intrinsic keeps it down to just the simple `*dst = move src` in
1954    // MIR (11 statements shorter, at the time of writing), and also allows
1955    // `src` to stay an SSA value in codegen_ssa, rather than a memory one.
1956
1957    // SAFETY: the caller must guarantee that `dst` is valid for writes.
1958    // `dst` cannot overlap `src` because the caller has mutable access
1959    // to `dst` while `src` is owned by this function.
1960    unsafe {
1961        #[cfg(debug_assertions)] // Too expensive to always enable (for now?)
1962        ub_checks::assert_unsafe_precondition!(
1963            check_language_ub,
1964            "ptr::write requires that the pointer argument is aligned and non-null",
1965            (
1966                addr: *mut () = dst as *mut (),
1967                align: usize = align_of::<T>(),
1968                is_zst: bool = T::IS_ZST,
1969            ) => ub_checks::maybe_is_aligned_and_not_null(addr, align, is_zst)
1970        );
1971        intrinsics::write_via_move(dst, src)
1972    }
1973}
1974
1975/// Overwrites a memory location with the given value without reading or
1976/// dropping the old value.
1977///
1978/// Unlike [`write()`], the pointer may be unaligned.
1979///
1980/// `write_unaligned` does not drop the contents of `dst`. This is safe, but it
1981/// could leak allocations or resources, so care should be taken not to overwrite
1982/// an object that should be dropped.
1983///
1984/// Additionally, it does not drop `src`. Semantically, `src` is moved into the
1985/// location pointed to by `dst`.
1986///
1987/// This is appropriate for initializing uninitialized memory, or overwriting
1988/// memory that has previously been read with [`read_unaligned`].
1989///
1990/// # Safety
1991///
1992/// Behavior is undefined if any of the following conditions are violated:
1993///
1994/// * `dst` must be [valid] for writes.
1995///
1996/// [valid]: self#safety
1997///
1998/// ## On `packed` structs
1999///
2000/// Attempting to create a raw pointer to an `unaligned` struct field with
2001/// an expression such as `&packed.unaligned as *const FieldType` creates an
2002/// intermediate unaligned reference before converting that to a raw pointer.
2003/// That this reference is temporary and immediately cast is inconsequential
2004/// as the compiler always expects references to be properly aligned.
2005/// As a result, using `&packed.unaligned as *const FieldType` causes immediate
2006/// *undefined behavior* in your program.
2007///
2008/// Instead, you must use the `&raw mut` syntax to create the pointer.
2009/// You may use that constructed pointer together with this function.
2010///
2011/// An example of how to do it and how this relates to `write_unaligned` is:
2012///
2013/// ```
2014/// #[repr(packed, C)]
2015/// struct Packed {
2016///     _padding: u8,
2017///     unaligned: u32,
2018/// }
2019///
2020/// let mut packed: Packed = unsafe { std::mem::zeroed() };
2021///
2022/// // Take the address of a 32-bit integer which is not aligned.
2023/// // In contrast to `&packed.unaligned as *mut _`, this has no undefined behavior.
2024/// let unaligned = &raw mut packed.unaligned;
2025///
2026/// unsafe { std::ptr::write_unaligned(unaligned, 42) };
2027///
2028/// assert_eq!({packed.unaligned}, 42); // `{...}` forces copying the field instead of creating a reference.
2029/// ```
2030///
2031/// Accessing unaligned fields directly with e.g. `packed.unaligned` is safe however
2032/// (as can be seen in the `assert_eq!` above).
2033///
2034/// # Examples
2035///
2036/// Write a `usize` value to a byte buffer:
2037///
2038/// ```
2039/// fn write_usize(x: &mut [u8], val: usize) {
2040///     assert!(x.len() >= size_of::<usize>());
2041///
2042///     let ptr = x.as_mut_ptr() as *mut usize;
2043///
2044///     unsafe { ptr.write_unaligned(val) }
2045/// }
2046/// ```
2047#[inline(always)]
2048#[stable(feature = "ptr_unaligned", since = "1.17.0")]
2049#[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")]
2050#[rustc_diagnostic_item = "ptr_write_unaligned"]
2051#[track_caller]
2052pub const unsafe fn write_unaligned<T>(dst: *mut T, src: T) {
2053    // Always true thanks to the repr, but to demonstrate
2054    const {
2055        assert!(mem::offset_of!(Unaligned::<T>, 0) == 0);
2056        assert!(size_of::<T>() == size_of::<Unaligned<T>>());
2057    }
2058
2059    let dst = dst.cast::<Unaligned<T>>();
2060    let src = Unaligned(src);
2061    // SAFETY: the caller must guarantee that `dst` is valid for writes.
2062    // Writing it as `Unaligned<T>` instead of `T` writes those same bytes because
2063    // it's the same size (thus zero offset), but with alignment 1 instead.
2064    unsafe { write(dst, src) }
2065}
2066
2067/// Performs a volatile read of the value from `src` without moving it.
2068///
2069/// Volatile operations are intended to act on I/O memory. As such, they are considered externally
2070/// observable events (just like syscalls, but less opaque), and are guaranteed to not be elided or
2071/// reordered by the compiler across other externally observable events. With this in mind, there
2072/// are two cases of usage that need to be distinguished:
2073///
2074/// - When a volatile operation is used for memory inside an [allocation], it behaves exactly like
2075///   [`read`], except for the additional guarantee that it won't be elided or reordered across
2076///   other externally observable events (see above). This implies that the operation will actually
2077///   access memory and not e.g. be lowered to reusing data from a previous read. Other than that,
2078///   all the usual rules for memory accesses apply (including provenance). In particular, just
2079///   like in C, whether an operation is volatile has no bearing whatsoever on questions involving
2080///   concurrent accesses from multiple threads. Volatile accesses behave exactly like non-atomic
2081///   accesses in that regard.
2082///
2083/// - Volatile operations, however, may also be used to access memory that is _outside_ of any Rust
2084///   allocation. In this use-case, the pointer does *not* have to be [valid] for reads. This is
2085///   typically used for CPU and peripheral registers that must be accessed via an I/O memory
2086///   mapping, most commonly at fixed addresses reserved by the hardware. These often have special
2087///   semantics associated to their manipulation, and cannot be used as general purpose memory.
2088///   Here, any address value is possible, including 0 and [`usize::MAX`], so long as the semantics
2089///   of such a read are well-defined by the target hardware. The provenance of the pointer is
2090///   irrelevant, and it can be created with [`without_provenance`]. The access is allowed to trap,
2091///   which must immediately abort the process. It can also cause other side-effects, but those
2092///   must not affect Rust-allocated memory in any way. This access is still not considered
2093///   [atomic], and as such it cannot be used for inter-thread synchronization.
2094///
2095/// Note that volatile memory operations where T is a zero-sized type are noops and may be ignored.
2096///
2097/// When invoked during const evaluation, this behaves like a regular read. In particular, such
2098/// reads must always follow the first of the two cases above.
2099///
2100/// [allocation]: crate::ptr#allocated-object
2101/// [atomic]: crate::sync::atomic#memory-model-for-atomic-accesses
2102///
2103/// # Load splitting
2104///
2105/// Exactly which hardware loads are performed by this function is, in general, highly target-dependent.
2106///
2107/// For a simple scalar, such as when `T` is a thin pointer, this will typically be one load assuming
2108/// your target supports a load of exactly that size and alignment.
2109///
2110/// For anything else, it will be split into multiple loads in some unspecified way.
2111/// This can happen even for scalars: notably, on many targets loading a `u128` will still need to be split,
2112/// despite being "one" scalar.  On many targets loading anything larger than a pointer will need to be split.
2113/// On all current targets a load larger than 64 bytes will need to be split.
2114/// Any load whose size is not a power of two will also almost certainly need to be split.
2115///
2116/// There is no stability guarantee on how that splitting happens.  It may change at any point.
2117///
2118/// # Safety
2119///
2120/// Like [`read`], `read_volatile` creates a bitwise copy of `T`, regardless of whether `T` is
2121/// [`Copy`]. If `T` is not [`Copy`], using both the returned value and the value at `*src` can
2122/// [violate memory safety][read-ownership]. However, storing non-[`Copy`] types in volatile memory
2123/// is almost certainly incorrect.
2124///
2125/// Behavior is undefined if any of the following conditions are violated:
2126///
2127/// * `src` must be either [valid] for reads, or `T` must be a ZST, or `src` must point to memory
2128///   outside of all Rust allocations and reading from that memory must not cause any memory inside
2129///   a Rust allocation to be modified.
2130///
2131/// * `src` must be properly aligned.
2132///
2133/// * Reading from `src` must produce a properly initialized value of type `T`.
2134///
2135/// Note that even if `T` has size `0`, the pointer must be properly aligned.
2136///
2137/// [valid]: self#safety
2138/// [read-ownership]: read#ownership-of-the-returned-value
2139///
2140/// # Examples
2141///
2142/// Basic usage:
2143///
2144/// ```
2145/// let x = 12;
2146/// let y = &x as *const i32;
2147///
2148/// unsafe {
2149///     assert_eq!(std::ptr::read_volatile(y), 12);
2150/// }
2151/// ```
2152#[inline(always)]
2153#[stable(feature = "volatile", since = "1.9.0")]
2154#[rustc_const_unstable(feature = "const_volatile", issue = "159094")]
2155#[track_caller]
2156#[rustc_diagnostic_item = "ptr_read_volatile"]
2157pub const unsafe fn read_volatile<T>(src: *const T) -> T {
2158    // SAFETY: the caller must uphold the safety contract for `volatile_load`.
2159    unsafe {
2160        ub_checks::assert_unsafe_precondition!(
2161            check_language_ub,
2162            "ptr::read_volatile requires that the pointer argument is aligned",
2163            (
2164                addr: *const () = src as *const (),
2165                align: usize = align_of::<T>(),
2166            ) => ub_checks::maybe_is_aligned(addr, align)
2167        );
2168        intrinsics::volatile_load(src)
2169    }
2170}
2171
2172/// Performs a volatile write of a memory location with the given value without reading or dropping
2173/// the old value.
2174///
2175/// Volatile operations are intended to act on I/O memory. As such, they are considered externally
2176/// observable events (just like syscalls), and are guaranteed to not be elided or reordered by the
2177/// compiler across other externally observable events. With this in mind, there are two cases of
2178/// usage that need to be distinguished:
2179///
2180/// - When a volatile operation is used for memory inside an [allocation], it behaves exactly like
2181///   [`write`][write()], except for the additional guarantee that it won't be elided or reordered
2182///   across other externally observable events (see above). This implies that the operation will
2183///   actually access memory and not e.g. be lowered to a register access. Other than that, all the
2184///   usual rules for memory accesses apply (including provenance). In particular, just like in C,
2185///   whether an operation is volatile has no bearing whatsoever on questions involving concurrent
2186///   access from multiple threads. Volatile accesses behave exactly like non-atomic accesses in
2187///   that regard.
2188///
2189/// - Volatile operations, however, may also be used to access memory that is _outside_ of any Rust
2190///   allocation. In this use-case, the pointer does *not* have to be [valid] for writes. This is
2191///   typically used for CPU and peripheral registers that must be accessed via an I/O memory
2192///   mapping, most commonly at fixed addresses reserved by the hardware. These often have special
2193///   semantics associated to their manipulation, and cannot be used as general purpose memory.
2194///   Here, any address value is possible, including 0 and [`usize::MAX`], so long as the semantics
2195///   of such a write are well-defined by the target hardware. The provenance of the pointer is
2196///   irrelevant, and it can be created with [`without_provenance_mut`]. The access is allowed to
2197///   trap, which must immediately abort the process. It can also cause other side-effects, but
2198///   those must not affect Rust-allocated memory in any way. This access is still not considered
2199///   [atomic], and as such it cannot be used for inter-thread synchronization.
2200///
2201/// Note that volatile memory operations on zero-sized types (e.g., if a zero-sized type is passed
2202/// to `write_volatile`) are noops and may be ignored.
2203///
2204/// `write_volatile` does not drop the contents of `dst`. This is safe, but it could leak
2205/// allocations or resources, so care should be taken not to overwrite an object that should be
2206/// dropped when operating on Rust memory. Additionally, it does not drop `src`. Semantically, `src`
2207/// is moved into the location pointed to by `dst`.
2208///
2209/// When invoked during const evaluation, this behaves like a regular write. In particular, such
2210/// reads must always follow the first of the two cases above.
2211///
2212/// [allocation]: crate::ptr#allocated-object
2213/// [atomic]: crate::sync::atomic#memory-model-for-atomic-accesses
2214///
2215/// # Store splitting
2216///
2217/// Exactly which hardware stores are performed by this function is, in general, highly target-dependent.
2218///
2219/// For a simple scalar, such as when `T` is a thin pointer, this will typically be one store assuming
2220/// your target supports a store of exactly that size and alignment.
2221///
2222/// For anything else, it will be split into multiple stores in some unspecified way.
2223/// This can happen even for scalars: notably, on many targets storing a `u128` will still need to be split,
2224/// despite being "one" scalar.  On many targets storing anything larger than a pointer will need to be split.
2225/// On all current targets a store larger than 64 bytes will need to be split.
2226/// Any store whose size is not a power of two will also almost certainly need to be split.
2227///
2228/// There is no stability guarantee on how that splitting happens.  It may change at any point.
2229///
2230/// # Safety
2231///
2232/// Behavior is undefined if any of the following conditions are violated:
2233///
2234/// * `dst` must be either [valid] for writes, or `T` must be a ZST, or `dst` must point to memory
2235///   outside of all Rust allocations and writing to that memory must not cause any memory inside a
2236///   Rust allocation to be modified.
2237///
2238/// * `dst` must be properly aligned.
2239///
2240/// Note that even if `T` has size `0`, the pointer must be properly aligned.
2241///
2242/// [valid]: self#safety
2243///
2244/// # Examples
2245///
2246/// Basic usage:
2247///
2248/// ```
2249/// let mut x = 0;
2250/// let y = &mut x as *mut i32;
2251/// let z = 12;
2252///
2253/// unsafe {
2254///     std::ptr::write_volatile(y, z);
2255///     assert_eq!(std::ptr::read_volatile(y), 12);
2256/// }
2257/// ```
2258#[inline(always)]
2259#[stable(feature = "volatile", since = "1.9.0")]
2260#[rustc_const_unstable(feature = "const_volatile", issue = "159094")]
2261#[rustc_diagnostic_item = "ptr_write_volatile"]
2262#[track_caller]
2263pub const unsafe fn write_volatile<T>(dst: *mut T, src: T) {
2264    // SAFETY: the caller must uphold the safety contract for `volatile_store`.
2265    unsafe {
2266        ub_checks::assert_unsafe_precondition!(
2267            check_language_ub,
2268            "ptr::write_volatile requires that the pointer argument is aligned",
2269            (
2270                addr: *mut () = dst as *mut (),
2271                align: usize = align_of::<T>(),
2272            ) => ub_checks::maybe_is_aligned(addr, align)
2273        );
2274        intrinsics::volatile_store(dst, src);
2275    }
2276}
2277
2278/// Calculate an element-offset that increases a pointer's alignment.
2279///
2280/// Calculate an element-offset (not byte-offset) that when added to a given pointer `p`, increases `p`'s alignment to at least the given alignment `a`.
2281///
2282/// # Safety
2283/// `a` must be a power of two.
2284///
2285/// # Notes
2286/// This implementation has been carefully tailored to not panic. It is UB for this to panic.
2287/// The only real change that can be made here is change of `INV_TABLE_MOD_16` and associated
2288/// constants.
2289///
2290/// If we ever decide to make it possible to call the intrinsic with `a` that is not a
2291/// power-of-two, it will probably be more prudent to just change to a naive implementation rather
2292/// than trying to adapt this to accommodate that change.
2293///
2294/// Any questions go to @nagisa.
2295#[allow(ptr_to_integer_transmute_in_consts)]
2296pub(crate) unsafe fn align_offset<T: Sized>(p: *const T, a: usize) -> usize {
2297    // FIXME(#75598): Direct use of these intrinsics improves codegen significantly at opt-level <=
2298    // 1, where the method versions of these operations are not inlined.
2299    use intrinsics::{
2300        assume, cttz_nonzero, exact_div, mul_with_overflow, unchecked_rem, unchecked_shl,
2301        unchecked_shr, unchecked_sub, wrapping_add, wrapping_mul, wrapping_sub,
2302    };
2303
2304    /// Calculate multiplicative modular inverse of `x` modulo `m`.
2305    ///
2306    /// This implementation is tailored for `align_offset` and has following preconditions:
2307    ///
2308    /// * `m` is a power-of-two;
2309    /// * `x < m`; (if `x ≥ m`, pass in `x % m` instead)
2310    ///
2311    /// Implementation of this function shall not panic. Ever.
2312    #[inline]
2313    const unsafe fn mod_inv(x: usize, m: usize) -> usize {
2314        /// Multiplicative modular inverse table modulo 2⁴ = 16.
2315        ///
2316        /// Note, that this table does not contain values where inverse does not exist (i.e., for
2317        /// `0⁻¹ mod 16`, `2⁻¹ mod 16`, etc.)
2318        const INV_TABLE_MOD_16: [u8; 8] = [1, 11, 13, 7, 9, 3, 5, 15];
2319        /// Modulo for which the `INV_TABLE_MOD_16` is intended.
2320        const INV_TABLE_MOD: usize = 16;
2321
2322        // SAFETY: `m` is required to be a power-of-two, hence non-zero.
2323        let m_minus_one = unsafe { unchecked_sub(m, 1) };
2324        let mut inverse = INV_TABLE_MOD_16[(x & (INV_TABLE_MOD - 1)) >> 1] as usize;
2325        let mut mod_gate = INV_TABLE_MOD;
2326        // We iterate "up" using the following formula:
2327        //
2328        // $$ xy ≡ 1 (mod 2ⁿ) → xy (2 - xy) ≡ 1 (mod 2²ⁿ) $$
2329        //
2330        // This application needs to be applied at least until `2²ⁿ ≥ m`, at which point we can
2331        // finally reduce the computation to our desired `m` by taking `inverse mod m`.
2332        //
2333        // This computation is `O(log log m)`, which is to say, that on 64-bit machines this loop
2334        // will always finish in at most 4 iterations.
2335        loop {
2336            // y = y * (2 - xy) mod n
2337            //
2338            // Note, that we use wrapping operations here intentionally – the original formula
2339            // uses e.g., subtraction `mod n`. It is entirely fine to do them `mod
2340            // usize::MAX` instead, because we take the result `mod n` at the end
2341            // anyway.
2342            if mod_gate >= m {
2343                break;
2344            }
2345            inverse = wrapping_mul(inverse, wrapping_sub(2usize, wrapping_mul(x, inverse)));
2346            let (new_gate, overflow) = mul_with_overflow(mod_gate, mod_gate);
2347            if overflow {
2348                break;
2349            }
2350            mod_gate = new_gate;
2351        }
2352        inverse & m_minus_one
2353    }
2354
2355    let stride = size_of::<T>();
2356
2357    let addr: usize = p.addr();
2358
2359    // SAFETY: `a` is a power-of-two, therefore non-zero.
2360    let a_minus_one = unsafe { unchecked_sub(a, 1) };
2361
2362    if stride == 0 {
2363        // SPECIAL_CASE: handle 0-sized types. No matter how many times we step, the address will
2364        // stay the same, so no offset will be able to align the pointer unless it is already
2365        // aligned. This branch _will_ be optimized out as `stride` is known at compile-time.
2366        let p_mod_a = addr & a_minus_one;
2367        return if p_mod_a == 0 { 0 } else { usize::MAX };
2368    }
2369
2370    // SAFETY: `stride == 0` case has been handled by the special case above.
2371    let a_mod_stride = unsafe { unchecked_rem(a, stride) };
2372    if a_mod_stride == 0 {
2373        // SPECIAL_CASE: In cases where the `a` is divisible by `stride`, byte offset to align a
2374        // pointer can be computed more simply through `-p (mod a)`. In the off-chance the byte
2375        // offset is not a multiple of `stride`, the input pointer was misaligned and no pointer
2376        // offset will be able to produce a `p` aligned to the specified `a`.
2377        //
2378        // The naive `-p (mod a)` equation inhibits LLVM's ability to select instructions
2379        // like `lea`. We compute `(round_up_to_next_alignment(p, a) - p)` instead. This
2380        // redistributes operations around the load-bearing, but pessimizing `and` instruction
2381        // sufficiently for LLVM to be able to utilize the various optimizations it knows about.
2382        //
2383        // LLVM handles the branch here particularly nicely. If this branch needs to be evaluated
2384        // at runtime, it will produce a mask `if addr_mod_stride == 0 { 0 } else { usize::MAX }`
2385        // in a branch-free way and then bitwise-OR it with whatever result the `-p mod a`
2386        // computation produces.
2387
2388        let aligned_address = wrapping_add(addr, a_minus_one) & wrapping_sub(0, a);
2389        let byte_offset = wrapping_sub(aligned_address, addr);
2390        // FIXME: Remove the assume after <https://github.com/llvm/llvm-project/issues/62502>
2391        // SAFETY: Masking by `-a` can only affect the low bits, and thus cannot have reduced
2392        // the value by more than `a-1`, so even though the intermediate values might have
2393        // wrapped, the byte_offset is always in `[0, a)`.
2394        unsafe { assume(byte_offset < a) };
2395
2396        // SAFETY: `stride == 0` case has been handled by the special case above.
2397        let addr_mod_stride = unsafe { unchecked_rem(addr, stride) };
2398
2399        return if addr_mod_stride == 0 {
2400            // SAFETY: `stride` is non-zero. This is guaranteed to divide exactly as well, because
2401            // addr has been verified to be aligned to the original type’s alignment requirements.
2402            unsafe { exact_div(byte_offset, stride) }
2403        } else {
2404            usize::MAX
2405        };
2406    }
2407
2408    // GENERAL_CASE: From here on we’re handling the very general case where `addr` may be
2409    // misaligned, there isn’t an obvious relationship between `stride` and `a` that we can take an
2410    // advantage of, etc. This case produces machine code that isn’t particularly high quality,
2411    // compared to the special cases above. The code produced here is still within the realm of
2412    // miracles, given the situations this case has to deal with.
2413
2414    // SAFETY: a is power-of-two hence non-zero. stride == 0 case is handled above.
2415    // FIXME(const-hack) replace with min
2416    let gcdpow = unsafe {
2417        let x = cttz_nonzero(stride);
2418        let y = cttz_nonzero(a);
2419        if x < y { x } else { y }
2420    };
2421    // SAFETY: gcdpow has an upper-bound that’s at most the number of bits in a `usize`.
2422    let gcd = unsafe { unchecked_shl(1usize, gcdpow) };
2423    // SAFETY: gcd is always greater or equal to 1.
2424    if addr & unsafe { unchecked_sub(gcd, 1) } == 0 {
2425        // This branch solves for the following linear congruence equation:
2426        //
2427        // ` p + so = 0 mod a `
2428        //
2429        // `p` here is the pointer value, `s` - stride of `T`, `o` offset in `T`s, and `a` - the
2430        // requested alignment.
2431        //
2432        // With `g = gcd(a, s)`, and the above condition asserting that `p` is also divisible by
2433        // `g`, we can denote `a' = a/g`, `s' = s/g`, `p' = p/g`, then this becomes equivalent to:
2434        //
2435        // ` p' + s'o = 0 mod a' `
2436        // ` o = (a' - (p' mod a')) * (s'^-1 mod a') `
2437        //
2438        // The first term is "the relative alignment of `p` to `a`" (divided by the `g`), the
2439        // second term is "how does incrementing `p` by `s` bytes change the relative alignment of
2440        // `p`" (again divided by `g`). Division by `g` is necessary to make the inverse well
2441        // formed if `a` and `s` are not co-prime.
2442        //
2443        // Furthermore, the result produced by this solution is not "minimal", so it is necessary
2444        // to take the result `o mod lcm(s, a)`. This `lcm(s, a)` is the same as `a'`.
2445
2446        // SAFETY: `gcdpow` has an upper-bound not greater than the number of trailing 0-bits in
2447        // `a`.
2448        let a2 = unsafe { unchecked_shr(a, gcdpow) };
2449        // SAFETY: `a2` is non-zero. Shifting `a` by `gcdpow` cannot shift out any of the set bits
2450        // in `a` (of which it has exactly one).
2451        let a2minus1 = unsafe { unchecked_sub(a2, 1) };
2452        // SAFETY: `gcdpow` has an upper-bound not greater than the number of trailing 0-bits in
2453        // `a`.
2454        let s2 = unsafe { unchecked_shr(stride & a_minus_one, gcdpow) };
2455        // SAFETY: `gcdpow` has an upper-bound not greater than the number of trailing 0-bits in
2456        // `a`. Furthermore, the subtraction cannot overflow, because `a2 = a >> gcdpow` will
2457        // always be strictly greater than `(p % a) >> gcdpow`.
2458        let minusp2 = unsafe { unchecked_sub(a2, unchecked_shr(addr & a_minus_one, gcdpow)) };
2459        // SAFETY: `a2` is a power-of-two, as proven above. `s2` is strictly less than `a2`
2460        // because `(s % a) >> gcdpow` is strictly less than `a >> gcdpow`.
2461        return wrapping_mul(minusp2, unsafe { mod_inv(s2, a2) }) & a2minus1;
2462    }
2463
2464    // Cannot be aligned at all.
2465    usize::MAX
2466}
2467
2468/// Compares raw pointers for equality.
2469///
2470/// This is the same as using the `==` operator, but less generic:
2471/// the arguments have to be `*const T` raw pointers,
2472/// not anything that implements `PartialEq`.
2473///
2474/// This can be used to compare `&T` references (which coerce to `*const T` implicitly)
2475/// by their address rather than comparing the values they point to
2476/// (which is what the `PartialEq for &T` implementation does).
2477///
2478/// When comparing wide pointers, both the address and the metadata are tested for equality.
2479/// However, note that comparing trait object pointers (`*const dyn Trait`) is unreliable: pointers
2480/// to values of the same underlying type can compare inequal (because vtables are duplicated in
2481/// multiple codegen units), and pointers to values of *different* underlying type can compare equal
2482/// (since identical vtables can be deduplicated within a codegen unit).
2483///
2484/// # Examples
2485///
2486/// ```
2487/// use std::ptr;
2488///
2489/// let five = 5;
2490/// let other_five = 5;
2491/// let five_ref = &five;
2492/// let same_five_ref = &five;
2493/// let other_five_ref = &other_five;
2494///
2495/// assert!(five_ref == same_five_ref);
2496/// assert!(ptr::eq(five_ref, same_five_ref));
2497///
2498/// assert!(five_ref == other_five_ref);
2499/// assert!(!ptr::eq(five_ref, other_five_ref));
2500/// ```
2501///
2502/// Slices are also compared by their length (fat pointers):
2503///
2504/// ```
2505/// let a = [1, 2, 3];
2506/// assert!(std::ptr::eq(&a[..3], &a[..3]));
2507/// assert!(!std::ptr::eq(&a[..2], &a[..3]));
2508/// assert!(!std::ptr::eq(&a[0..2], &a[1..3]));
2509/// ```
2510#[stable(feature = "ptr_eq", since = "1.17.0")]
2511#[inline(always)]
2512#[must_use = "pointer comparison produces a value"]
2513#[rustc_diagnostic_item = "ptr_eq"]
2514#[allow(ambiguous_wide_pointer_comparisons)] // it's actually clear here
2515pub fn eq<T: PointeeSized>(a: *const T, b: *const T) -> bool {
2516    a == b
2517}
2518
2519/// Compares the *addresses* of the two pointers for equality,
2520/// ignoring any metadata in [wide](core::ptr#thin-and-wide-pointers) pointers.
2521///
2522/// If the arguments are thin pointers of the same type,
2523/// then this is the same as [`eq`].
2524///
2525/// # Examples
2526///
2527/// ```
2528/// use std::ptr;
2529///
2530/// let whole: &[i32; 3] = &[1, 2, 3];
2531/// let first: &i32 = &whole[0];
2532///
2533/// assert!(ptr::addr_eq(whole, first));
2534/// assert!(!ptr::eq::<dyn std::fmt::Debug>(whole, first));
2535/// ```
2536#[stable(feature = "ptr_addr_eq", since = "1.76.0")]
2537#[inline(always)]
2538#[must_use = "pointer comparison produces a value"]
2539pub fn addr_eq<T: PointeeSized, U: PointeeSized>(p: *const T, q: *const U) -> bool {
2540    (p as *const ()) == (q as *const ())
2541}
2542
2543/// Compares the *addresses* of the two function pointers for equality.
2544///
2545/// This is the same as `f == g`, but using this function makes clear that the potentially
2546/// surprising semantics of function pointer comparison are involved.
2547///
2548/// There are **basically no guarantees** about how functions are compiled as they have no intrinsic
2549/// “identity”; in particular, this comparison:
2550///
2551/// * May return `true` unexpectedly, even in cases where functions are not logically equivalent,
2552///   as long as they compile to the same machine code.
2553///
2554///   For example, the following program is likely (but not guaranteed) to print `(true, true)`
2555///   when compiled with optimization:
2556///
2557///   ```
2558///   use std::hint::assert_unchecked;
2559///
2560///   let f: fn(i32) -> i32 = |x| x;
2561///   let g: fn(u32) -> u32 = |x| x + 0;  // different type, different body
2562///   let h: unsafe fn(i32) -> i32 = |x| unsafe { assert_unchecked(x != 0); x };  // different unsafe preconditions
2563///   // These three functions are allowed but not guaranteed to compare equal.
2564///   dbg!(std::ptr::fn_addr_eq(f, g), std::ptr::fn_addr_eq(f, h));
2565///   ```
2566///
2567///   In other words, functions that compare equal can have different behavior! Even if
2568///   `fn_addr_eq(f, h)` is true, it is still undefined behavior to call `h(0)`, while `f(0)` is
2569///   safe to call.
2570///
2571///   The explanation for this is that function pointers have [provenance][crate::ptr#provenance].
2572///   If functions have the same assembly and get merged, their provenance decides which variant of
2573///   the function will be called. `f` and `h` therefore end up with the same address but different
2574///   provenance, and `fn_addr_eq` only compares the address.
2575///
2576/// * May return `false` in any case.
2577///
2578///   This is particularly likely with generic functions but may happen with any function.
2579///   (From an implementation perspective, this is possible because functions may sometimes be
2580///   processed more than once by the compiler, resulting in duplicate machine code.)
2581///
2582/// These limitations imply that comparing function pointers (via this function or via `==`) is only
2583/// useful in extremely niche circumstances. For instance, if a library maintains a set of
2584/// callbacks, and doesn't guarantee which of them are actually called (i.e., callbacks can be
2585/// spuriously skipped), then it can be legitimate to deduplicate callbacks based on `fn_addr_eq`.
2586/// This can never introduce undefined behavior because it only skips executing some of the
2587/// registered callbacks.
2588#[stable(feature = "ptr_fn_addr_eq", since = "1.85.0")]
2589#[inline(always)]
2590#[must_use = "function pointer comparison produces a value"]
2591pub fn fn_addr_eq<T: FnPtr, U: FnPtr>(f: T, g: U) -> bool {
2592    f.addr() == g.addr()
2593}
2594
2595/// Hash a raw pointer.
2596///
2597/// This can be used to hash a `&T` reference (which coerces to `*const T` implicitly)
2598/// by its address rather than the value it points to
2599/// (which is what the `Hash for &T` implementation does).
2600///
2601/// # Examples
2602///
2603/// ```
2604/// use std::hash::{DefaultHasher, Hash, Hasher};
2605/// use std::ptr;
2606///
2607/// let five = 5;
2608/// let five_ref = &five;
2609///
2610/// let mut hasher = DefaultHasher::new();
2611/// ptr::hash(five_ref, &mut hasher);
2612/// let actual = hasher.finish();
2613///
2614/// let mut hasher = DefaultHasher::new();
2615/// (five_ref as *const i32).hash(&mut hasher);
2616/// let expected = hasher.finish();
2617///
2618/// assert_eq!(actual, expected);
2619/// ```
2620#[stable(feature = "ptr_hash", since = "1.35.0")]
2621pub fn hash<T: PointeeSized, S: hash::Hasher>(hashee: *const T, into: &mut S) {
2622    use crate::hash::Hash;
2623    hashee.hash(into);
2624}
2625
2626#[stable(feature = "fnptr_impls", since = "1.4.0")]
2627#[diagnostic::on_const(
2628    message = "pointers cannot be reliably compared during const eval",
2629    note = "see issue #53020 <https://github.com/rust-lang/rust/issues/53020> for more information"
2630)]
2631impl<F: FnPtr> PartialEq for F {
2632    #[inline]
2633    fn eq(&self, other: &Self) -> bool {
2634        self.addr() == other.addr()
2635    }
2636}
2637#[stable(feature = "fnptr_impls", since = "1.4.0")]
2638#[diagnostic::on_const(
2639    message = "pointers cannot be reliably compared during const eval",
2640    note = "see issue #53020 <https://github.com/rust-lang/rust/issues/53020> for more information"
2641)]
2642impl<F: FnPtr> Eq for F {}
2643
2644#[stable(feature = "fnptr_impls", since = "1.4.0")]
2645#[diagnostic::on_const(
2646    message = "pointers cannot be reliably compared during const eval",
2647    note = "see issue #53020 <https://github.com/rust-lang/rust/issues/53020> for more information"
2648)]
2649impl<F: FnPtr> PartialOrd for F {
2650    #[inline]
2651    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
2652        self.addr().partial_cmp(&other.addr())
2653    }
2654}
2655#[stable(feature = "fnptr_impls", since = "1.4.0")]
2656#[diagnostic::on_const(
2657    message = "pointers cannot be reliably compared during const eval",
2658    note = "see issue #53020 <https://github.com/rust-lang/rust/issues/53020> for more information"
2659)]
2660impl<F: FnPtr> Ord for F {
2661    #[inline]
2662    fn cmp(&self, other: &Self) -> Ordering {
2663        self.addr().cmp(&other.addr())
2664    }
2665}
2666
2667#[stable(feature = "fnptr_impls", since = "1.4.0")]
2668impl<F: FnPtr> hash::Hash for F {
2669    fn hash<HH: hash::Hasher>(&self, state: &mut HH) {
2670        state.write_usize(self.addr())
2671    }
2672}
2673
2674#[stable(feature = "fnptr_impls", since = "1.4.0")]
2675impl<F: FnPtr> fmt::Pointer for F {
2676    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2677        fmt::pointer_fmt_inner(self.addr(), f)
2678    }
2679}
2680
2681#[stable(feature = "fnptr_impls", since = "1.4.0")]
2682impl<F: FnPtr> fmt::Debug for F {
2683    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2684        fmt::pointer_fmt_inner(self.addr(), f)
2685    }
2686}
2687
2688/// Creates a `const` raw pointer to a place, without creating an intermediate reference.
2689///
2690/// `addr_of!(expr)` is equivalent to `&raw const expr`. The macro is *soft-deprecated*;
2691/// use `&raw const` instead.
2692///
2693/// It is still an open question under which conditions writing through an `addr_of!`-created
2694/// pointer is permitted. If the place `expr` evaluates to is based on a raw pointer, then the
2695/// result of `addr_of!` inherits all permissions from that raw pointer. However, if the place is
2696/// based on a reference, local variable, or `static`, then until all details are decided, the same
2697/// rules as for shared references apply: it is UB to write through a pointer created with this
2698/// operation, except for bytes located inside an `UnsafeCell`. Use `&raw mut` (or [`addr_of_mut`])
2699/// to create a raw pointer that definitely permits mutation.
2700///
2701/// Creating a reference with `&`/`&mut` is only allowed if the pointer is properly aligned
2702/// and points to initialized data. For cases where those requirements do not hold,
2703/// raw pointers should be used instead. However, `&expr as *const _` creates a reference
2704/// before casting it to a raw pointer, and that reference is subject to the same rules
2705/// as all other references. This macro can create a raw pointer *without* creating
2706/// a reference first.
2707///
2708/// See [`addr_of_mut`] for how to create a pointer to uninitialized data.
2709/// Doing that with `addr_of` would not make much sense since one could only
2710/// read the data, and that would be Undefined Behavior.
2711///
2712/// # Safety
2713///
2714/// The `expr` in `addr_of!(expr)` is evaluated as a place expression, but never loads from the
2715/// place or requires the place to be dereferenceable. This means that `addr_of!((*ptr).field)`
2716/// still requires the projection to `field` to be in-bounds, using the same rules as [`offset`].
2717/// However, `addr_of!(*ptr)` is defined behavior even if `ptr` is null, dangling, or misaligned.
2718///
2719/// Note that `Deref`/`Index` coercions (and their mutable counterparts) are applied inside
2720/// `addr_of!` like everywhere else, in which case a reference is created to call `Deref::deref` or
2721/// `Index::index`, respectively. The statements above only apply when no such coercions are
2722/// applied.
2723///
2724/// [`offset`]: pointer::offset
2725///
2726/// # Example
2727///
2728/// **Correct usage: Creating a pointer to unaligned data**
2729///
2730/// ```
2731/// use std::ptr;
2732///
2733/// #[repr(packed)]
2734/// struct Packed {
2735///     f1: u8,
2736///     f2: u16,
2737/// }
2738///
2739/// let packed = Packed { f1: 1, f2: 2 };
2740/// // `&packed.f2` would create an unaligned reference, and thus be Undefined Behavior!
2741/// let raw_f2 = ptr::addr_of!(packed.f2);
2742/// assert_eq!(unsafe { raw_f2.read_unaligned() }, 2);
2743/// ```
2744///
2745/// **Incorrect usage: Out-of-bounds fields projection**
2746///
2747/// ```rust,no_run
2748/// use std::ptr;
2749///
2750/// #[repr(C)]
2751/// struct MyStruct {
2752///     field1: i32,
2753///     field2: i32,
2754/// }
2755///
2756/// let ptr: *const MyStruct = ptr::null();
2757/// let fieldptr = unsafe { ptr::addr_of!((*ptr).field2) }; // Undefined Behavior ⚠️
2758/// ```
2759///
2760/// The field projection `.field2` would offset the pointer by 4 bytes,
2761/// but the pointer is not in-bounds of an allocation for 4 bytes,
2762/// so this offset is Undefined Behavior.
2763/// See the [`offset`] docs for a full list of requirements for inbounds pointer arithmetic; the
2764/// same requirements apply to field projections, even inside `addr_of!`. (In particular, it makes
2765/// no difference whether the pointer is null or dangling.)
2766#[stable(feature = "raw_ref_macros", since = "1.51.0")]
2767#[rustc_macro_transparency = "semiopaque"]
2768pub macro addr_of($place:expr) {
2769    &raw const $place
2770}
2771
2772/// Creates a `mut` raw pointer to a place, without creating an intermediate reference.
2773///
2774/// `addr_of_mut!(expr)` is equivalent to `&raw mut expr`. The macro is *soft-deprecated*;
2775/// use `&raw mut` instead.
2776///
2777/// Creating a reference with `&`/`&mut` is only allowed if the pointer is properly aligned
2778/// and points to initialized data. For cases where those requirements do not hold,
2779/// raw pointers should be used instead. However, `&mut expr as *mut _` creates a reference
2780/// before casting it to a raw pointer, and that reference is subject to the same rules
2781/// as all other references. This macro can create a raw pointer *without* creating
2782/// a reference first.
2783///
2784/// # Safety
2785///
2786/// The `expr` in `addr_of_mut!(expr)` is evaluated as a place expression, but never loads from the
2787/// place or requires the place to be dereferenceable. This means that `addr_of_mut!((*ptr).field)`
2788/// still requires the projection to `field` to be in-bounds, using the same rules as [`offset`].
2789/// However, `addr_of_mut!(*ptr)` is defined behavior even if `ptr` is null, dangling, or misaligned.
2790///
2791/// Note that `Deref`/`Index` coercions (and their mutable counterparts) are applied inside
2792/// `addr_of_mut!` like everywhere else, in which case a reference is created to call `Deref::deref`
2793/// or `Index::index`, respectively. The statements above only apply when no such coercions are
2794/// applied.
2795///
2796/// [`offset`]: pointer::offset
2797///
2798/// # Examples
2799///
2800/// **Correct usage: Creating a pointer to unaligned data**
2801///
2802/// ```
2803/// use std::ptr;
2804///
2805/// #[repr(packed)]
2806/// struct Packed {
2807///     f1: u8,
2808///     f2: u16,
2809/// }
2810///
2811/// let mut packed = Packed { f1: 1, f2: 2 };
2812/// // `&mut packed.f2` would create an unaligned reference, and thus be Undefined Behavior!
2813/// let raw_f2 = ptr::addr_of_mut!(packed.f2);
2814/// unsafe { raw_f2.write_unaligned(42); }
2815/// assert_eq!({packed.f2}, 42); // `{...}` forces copying the field instead of creating a reference.
2816/// ```
2817///
2818/// **Correct usage: Creating a pointer to uninitialized data**
2819///
2820/// ```rust
2821/// use std::{ptr, mem::MaybeUninit};
2822///
2823/// struct Demo {
2824///     field: bool,
2825/// }
2826///
2827/// let mut uninit = MaybeUninit::<Demo>::uninit();
2828/// // `&uninit.as_mut().field` would create a reference to an uninitialized `bool`,
2829/// // and thus be Undefined Behavior!
2830/// let f1_ptr = unsafe { ptr::addr_of_mut!((*uninit.as_mut_ptr()).field) };
2831/// unsafe { f1_ptr.write(true); }
2832/// let init = unsafe { uninit.assume_init() };
2833/// ```
2834///
2835/// **Incorrect usage: Out-of-bounds fields projection**
2836///
2837/// ```rust,no_run
2838/// use std::ptr;
2839///
2840/// #[repr(C)]
2841/// struct MyStruct {
2842///     field1: i32,
2843///     field2: i32,
2844/// }
2845///
2846/// let ptr: *mut MyStruct = ptr::null_mut();
2847/// let fieldptr = unsafe { ptr::addr_of_mut!((*ptr).field2) }; // Undefined Behavior ⚠️
2848/// ```
2849///
2850/// The field projection `.field2` would offset the pointer by 4 bytes,
2851/// but the pointer is not in-bounds of an allocation for 4 bytes,
2852/// so this offset is Undefined Behavior.
2853/// See the [`offset`] docs for a full list of requirements for inbounds pointer arithmetic; the
2854/// same requirements apply to field projections, even inside `addr_of_mut!`. (In particular, it
2855/// makes no difference whether the pointer is null or dangling.)
2856#[stable(feature = "raw_ref_macros", since = "1.51.0")]
2857#[rustc_macro_transparency = "semiopaque"]
2858pub macro addr_of_mut($place:expr) {
2859    &raw mut $place
2860}
2861
2862/// Used in [`read_unaligned`] and [`write_unaligned`] to load and store `T`
2863/// with alignment 1 rather than its usual `align_of::<T>()` alignment.
2864#[repr(Rust, packed)]
2865struct Unaligned<T>(T);