mirror of
https://github.com/torvalds/linux.git
synced 2026-03-08 03:24:45 +01:00
Documentation/driver-api/cxl: device hotplug section
Describe cxl memory device hotplug implications, in particular how the platform CEDT CFMWS must be described to support successful hot-add of memory devices. Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Signed-off-by: Gregory Price <gourry@gourry.net> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Alejandro Lucero Palau <alucerop@amd.com> Link: https://patch.msgid.link/20251219170538.1675743-3-gourry@gourry.net Signed-off-by: Dave Jiang <dave.jiang@intel.com>
This commit is contained in:
Gregory Price
Dave Jiang
parent
2489d83c22
commit
7362facf6e
3 changed files with 134 additions and 0 deletions
|
|
@ -30,6 +30,7 @@ that have impacts on each other. The docs here break up configurations steps.
|
|||
platform/acpi
|
||||
platform/cdat
|
||||
platform/example-configs
|
||||
platform/device-hotplug
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
|
|
|||
|
|
@ -49,6 +49,9 @@ up without requiring CXL driver support. These platform vendors should
|
|||
test their configurations with the existing CXL driver and provide driver
|
||||
support for their auto-configurations if features like RAS are required.
|
||||
|
||||
Platforms requiring boot-time programming and/or locking of CXL fabric
|
||||
components may prevent features, such as device hot-plug, from working.
|
||||
|
||||
UEFI Settings
|
||||
=============
|
||||
If your platform supports it, the :code:`uefisettings` command can be used to
|
||||
|
|
|
|||
130
Documentation/driver-api/cxl/platform/device-hotplug.rst
Normal file
130
Documentation/driver-api/cxl/platform/device-hotplug.rst
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
.. SPDX-License-Identifier: GPL-2.0
|
||||
|
||||
==================
|
||||
CXL Device Hotplug
|
||||
==================
|
||||
|
||||
Device hotplug refers to *physical* hotplug of a device (addition or removal
|
||||
of a physical device from the machine).
|
||||
|
||||
BIOS/EFI software is expected to configure sufficient resources **at boot
|
||||
time** to allow hotplugged devices to be configured by software (such as
|
||||
proximity domains, HPA regions, and host-bridge configurations).
|
||||
|
||||
BIOS/EFI is not expected (**nor suggested**) to configure hotplugged
|
||||
devices at hotplug time (i.e. HDM decoders should be left unprogrammed).
|
||||
|
||||
This document covers some examples of those resources, but should not
|
||||
be considered exhaustive.
|
||||
|
||||
Hot-Remove
|
||||
==========
|
||||
Hot removal of a device typically requires careful removal of software
|
||||
constructs (memory regions, associated drivers) which manage these devices.
|
||||
|
||||
Hard-removing a CXL.mem device without carefully tearing down driver stacks
|
||||
is likely to cause the system to machine-check (or at least SIGBUS if memory
|
||||
access is limited to user space).
|
||||
|
||||
Memory Device Hot-Add
|
||||
=====================
|
||||
A device present at boot may be associated with a CXL Fixed Memory Window
|
||||
reported in :doc:`CEDT<acpi/cedt>`. That CFMWS may match the size of the
|
||||
device, but the construction of the CEDT CFMWS is platform-defined.
|
||||
|
||||
Hot-adding a memory device requires this pre-defined, **static** CFMWS to
|
||||
have sufficient HPA space to describe that device.
|
||||
|
||||
There are a few common scenarios to consider.
|
||||
|
||||
Single-Endpoint Memory Device Present at Boot
|
||||
---------------------------------------------
|
||||
A device present at boot likely had its capacity reported in the
|
||||
:doc:`CEDT<acpi/cedt>`. If a device is removed and a new device hotplugged,
|
||||
the capacity of the new device will be limited to the original CFMWS capacity.
|
||||
|
||||
Adding capacity larger than the original device will cause memory region
|
||||
creation to fail if the region size is greater than the CFMWS size.
|
||||
|
||||
The CFMWS is **static** and cannot be adjusted. Platforms which may expect
|
||||
different sized devices to be hotplugged must allocate sufficient CFMWS space
|
||||
**at boot time** to cover all future expected devices.
|
||||
|
||||
Multi-Endpoint Memory Device Present at Boot
|
||||
--------------------------------------------
|
||||
Non-switch-based Multi-Endpoint devices are outside the scope of what the
|
||||
CXL specification describes, but they are technically possible. We describe
|
||||
them here for instructive reasons only - this does not imply Linux support.
|
||||
|
||||
A hot-plug capable CXL memory device, such as one which presents multiple
|
||||
expanders as a single large-capacity device, should report the **maximum
|
||||
possible capacity** for the device at boot. ::
|
||||
|
||||
HB0
|
||||
RP0
|
||||
|
|
||||
[Multi-Endpoint Memory Device]
|
||||
_____|_____
|
||||
| |
|
||||
[Endpoint0] [Empty]
|
||||
|
||||
|
||||
Limiting the size to the capacity preset at boot will limit hot-add support
|
||||
to replacing capacity that was present at boot.
|
||||
|
||||
No CXL Device Present at Boot
|
||||
-----------------------------
|
||||
When no CXL memory device is present on boot, some platforms omit the CFMWS
|
||||
in the :doc:`CEDT<acpi/cedt>`. When this occurs, hot-add is not possible.
|
||||
|
||||
This describes the base case for any given device not being present at boot.
|
||||
If a future possible device is not described in the CEDT at boot, hot-add
|
||||
of that device is either limited or not possible.
|
||||
|
||||
For a platform to support hot-add of a full memory device, it must allocate
|
||||
a CEDT CFMWS region with sufficient memory capacity to cover all future
|
||||
potentially added capacity (along with any relevant CEDT CHBS entry).
|
||||
|
||||
To support memory hotplug directly on the host bridge/root port, or on a switch
|
||||
downstream of the host bridge, a platform must construct a CEDT CFMWS at boot
|
||||
with sufficient resources to support the max possible (or expected) hotplug
|
||||
memory capacity. ::
|
||||
|
||||
HB0 HB1
|
||||
RP0 RP1 RP2
|
||||
| | |
|
||||
Empty Empty USP
|
||||
________|________
|
||||
| | | |
|
||||
DSP DSP DSP DSP
|
||||
| | | |
|
||||
All Empty
|
||||
|
||||
For example, a BIOS/EFI may expose an option to configure a CEDT CFMWS with
|
||||
a pre-configured amount of memory capacity (per host bridge, or host bridge
|
||||
interleave set), even if no device is attached to Root Ports or Downstream
|
||||
Ports at boot (as depicted in the figure above).
|
||||
|
||||
|
||||
Interleave Sets
|
||||
===============
|
||||
|
||||
Host Bridge Interleave
|
||||
----------------------
|
||||
Host-bridge interleaved memory regions are defined **statically** in the
|
||||
:doc:`CEDT<acpi/cedt>`. To apply cross-host-bridge interleave, a CFMWS entry
|
||||
describing that interleave must have been provided **at boot**. Hotplugged
|
||||
devices cannot add host-bridge interleave capabilities at hotplug time.
|
||||
|
||||
See the :doc:`Flexible CEDT Configuration<example-configurations/flexible>`
|
||||
example to see how a platform can provide this kind of flexibility regarding
|
||||
hotplugged memory devices. BIOS/EFI software should consider options to
|
||||
present flexible CEDT configurations with hotplug support.
|
||||
|
||||
HDM Interleave
|
||||
--------------
|
||||
Decoder-applied interleave can flexibly handle hotplugged devices, as decoders
|
||||
can be re-programmed after hotplug.
|
||||
|
||||
To add or remove a device to/from an existing HDM-applied interleaved region,
|
||||
that region must be torn down an re-created.
|
||||
Loading…
Add table
Add a link
Reference in a new issue