Skip to content

Commit b168ed4

Browse files
Maarten Lankhorstmripard
Maarten Lankhorst
authored and
mripard
committed
kernel/cgroup: Add "dmem" memory accounting cgroup
This code is based on the RDMA and misc cgroup initially, but now uses page_counter. It uses the same min/low/max semantics as the memory cgroup as a result. There's a small mismatch as TTM uses u64, and page_counter long pages. In practice it's not a problem. 32-bits systems don't really come with >=4GB cards and as long as we're consistently wrong with units, it's fine. The device page size may not be in the same units as kernel page size, and each region might also have a different page size (VRAM vs GART for example). The interface is simple: - Call dmem_cgroup_register_region() - Use dmem_cgroup_try_charge to check if you can allocate a chunk of memory, use dmem_cgroup__uncharge when freeing it. This may return an error code, or -EAGAIN when the cgroup limit is reached. In that case a reference to the limiting pool is returned. - The limiting cs can be used as compare function for dmem_cgroup_state_evict_valuable. - After having evicted enough, drop reference to limiting cs with dmem_cgroup_pool_state_put. This API allows you to limit device resources with cgroups. You can see the supported cards in /sys/fs/cgroup/dmem.capacity You need to echo +dmem to cgroup.subtree_control, and then you can partition device memory. Co-developed-by: Friedrich Vock <[email protected]> Signed-off-by: Friedrich Vock <[email protected]> Co-developed-by: Maxime Ripard <[email protected]> Signed-off-by: Maarten Lankhorst <[email protected]> Acked-by: Tejun Heo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Maxime Ripard <[email protected]>
1 parent 9d89551 commit b168ed4

File tree

11 files changed

+1060
-10
lines changed

11 files changed

+1060
-10
lines changed

‎Documentation/admin-guide/cgroup-v2.rst

Lines changed: 51 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -64,13 +64,14 @@ v1 is available under :ref:`Documentation/admin-guide/cgroup-v1/index.rst <cgrou
6464
5-6. Device
6565
5-7. RDMA
6666
5-7-1. RDMA Interface Files
67-
5-8. HugeTLB
68-
5.8-1. HugeTLB Interface Files
69-
5-9. Misc
70-
5.9-1 Miscellaneous cgroup Interface Files
71-
5.9-2 Migration and Ownership
72-
5-10. Others
73-
5-10-1. perf_event
67+
5-8. DMEM
68+
5-9. HugeTLB
69+
5.9-1. HugeTLB Interface Files
70+
5-10. Misc
71+
5.10-1 Miscellaneous cgroup Interface Files
72+
5.10-2 Migration and Ownership
73+
5-11. Others
74+
5-11-1. perf_event
7475
5-N. Non-normative information
7576
5-N-1. CPU controller root cgroup process behaviour
7677
5-N-2. IO controller root cgroup process behaviour
@@ -2626,6 +2627,49 @@ RDMA Interface Files
26262627
mlx4_0 hca_handle=1 hca_object=20
26272628
ocrdma1 hca_handle=1 hca_object=23
26282629

2630+
DMEM
2631+
----
2632+
2633+
The "dmem" controller regulates the distribution and accounting of
2634+
device memory regions. Because each memory region may have its own page size,
2635+
which does not have to be equal to the system page size, the units are always bytes.
2636+
2637+
DMEM Interface Files
2638+
~~~~~~~~~~~~~~~~~~~~
2639+
2640+
dmem.max, dmem.min, dmem.low
2641+
A readwrite nested-keyed file that exists for all the cgroups
2642+
except root that describes current configured resource limit
2643+
for a region.
2644+
2645+
An example for xe follows::
2646+
2647+
drm/0000:03:00.0/vram0 1073741824
2648+
drm/0000:03:00.0/stolen max
2649+
2650+
The semantics are the same as for the memory cgroup controller, and are
2651+
calculated in the same way.
2652+
2653+
dmem.capacity
2654+
A read-only file that describes maximum region capacity.
2655+
It only exists on the root cgroup. Not all memory can be
2656+
allocated by cgroups, as the kernel reserves some for
2657+
internal use.
2658+
2659+
An example for xe follows::
2660+
2661+
drm/0000:03:00.0/vram0 8514437120
2662+
drm/0000:03:00.0/stolen 67108864
2663+
2664+
dmem.current
2665+
A read-only file that describes current resource usage.
2666+
It exists for all the cgroup except root.
2667+
2668+
An example for xe follows::
2669+
2670+
drm/0000:03:00.0/vram0 12550144
2671+
drm/0000:03:00.0/stolen 8650752
2672+
26292673
HugeTLB
26302674
-------
26312675

‎Documentation/core-api/cgroup.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
==================
2+
Cgroup Kernel APIs
3+
==================
4+
5+
Device Memory Cgroup API (dmemcg)
6+
=========================
7+
.. kernel-doc:: kernel/cgroup/dmem.c
8+
:export:
9+

‎Documentation/core-api/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,7 @@ more memory-management documentation in Documentation/mm/index.rst.
109109
dma-isa-lpc
110110
swiotlb
111111
mm-api
112+
cgroup
112113
genalloc
113114
pin_user_pages
114115
boot-time-mm

‎Documentation/gpu/drm-compute.rst

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
==================================
2+
Long running workloads and compute
3+
==================================
4+
5+
Long running workloads (compute) are workloads that will not complete in 10
6+
seconds. (The time let the user wait before he reaches for the power button).
7+
This means that other techniques need to be used to manage those workloads,
8+
that cannot use fences.
9+
10+
Some hardware may schedule compute jobs, and have no way to pre-empt them, or
11+
have their memory swapped out from them. Or they simply want their workload
12+
not to be preempted or swapped out at all.
13+
14+
This means that it differs from what is described in driver-api/dma-buf.rst.
15+
16+
As with normal compute jobs, dma-fence may not be used at all. In this case,
17+
not even to force preemption. The driver with is simply forced to unmap a BO
18+
from the long compute job's address space on unbind immediately, not even
19+
waiting for the workload to complete. Effectively this terminates the workload
20+
when there is no hardware support to recover.
21+
22+
Since this is undesirable, there need to be mitigations to prevent a workload
23+
from being terminated. There are several possible approach, all with their
24+
advantages and drawbacks.
25+
26+
The first approach you will likely try is to pin all buffers used by compute.
27+
This guarantees that the job will run uninterrupted, but also allows a very
28+
denial of service attack by pinning as much memory as possible, hogging the
29+
all GPU memory, and possibly a huge chunk of CPU memory.
30+
31+
A second approach that will work slightly better on its own is adding an option
32+
not to evict when creating a new job (any kind). If all of userspace opts in
33+
to this flag, it would prevent cooperating userspace from forced terminating
34+
older compute jobs to start a new one.
35+
36+
If job preemption and recoverable pagefaults are not available, those are the
37+
only approaches possible. So even with those, you want a separate way of
38+
controlling resources. The standard kernel way of doing so is cgroups.
39+
40+
This creates a third option, using cgroups to prevent eviction. Both GPU and
41+
driver-allocated CPU memory would be accounted to the correct cgroup, and
42+
eviction would be made cgroup aware. This allows the GPU to be partitioned
43+
into cgroups, that will allow jobs to run next to each other without
44+
interference.
45+
46+
The interface to the cgroup would be similar to the current CPU memory
47+
interface, with similar semantics for min/low/high/max, if eviction can
48+
be made cgroup aware.
49+
50+
What should be noted is that each memory region (tiled memory for example)
51+
should have its own accounting.
52+
53+
The key is set to the regionid set by the driver, for example "tile0".
54+
For the value of $card, we use drmGetUnique().

‎include/linux/cgroup_dmem.h

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
/* SPDX-License-Identifier: MIT */
2+
/*
3+
* Copyright © 2023-2024 Intel Corporation
4+
*/
5+
6+
#ifndef _CGROUP_DMEM_H
7+
#define _CGROUP_DMEM_H
8+
9+
#include <linux/types.h>
10+
#include <linux/llist.h>
11+
12+
struct dmem_cgroup_pool_state;
13+
14+
/* Opaque definition of a cgroup region, used internally */
15+
struct dmem_cgroup_region;
16+
17+
#if IS_ENABLED(CONFIG_CGROUP_DMEM)
18+
struct dmem_cgroup_region *dmem_cgroup_register_region(u64 size, const char *name_fmt, ...) __printf(2,3);
19+
void dmem_cgroup_unregister_region(struct dmem_cgroup_region *region);
20+
int dmem_cgroup_try_charge(struct dmem_cgroup_region *region, u64 size,
21+
struct dmem_cgroup_pool_state **ret_pool,
22+
struct dmem_cgroup_pool_state **ret_limit_pool);
23+
void dmem_cgroup_uncharge(struct dmem_cgroup_pool_state *pool, u64 size);
24+
bool dmem_cgroup_state_evict_valuable(struct dmem_cgroup_pool_state *limit_pool,
25+
struct dmem_cgroup_pool_state *test_pool,
26+
bool ignore_low, bool *ret_hit_low);
27+
28+
void dmem_cgroup_pool_state_put(struct dmem_cgroup_pool_state *pool);
29+
#else
30+
static inline __printf(2,3) struct dmem_cgroup_region *
31+
dmem_cgroup_register_region(u64 size, const char *name_fmt, ...)
32+
{
33+
return NULL;
34+
}
35+
36+
static inline void dmem_cgroup_unregister_region(struct dmem_cgroup_region *region)
37+
{ }
38+
39+
static inline int dmem_cgroup_try_charge(struct dmem_cgroup_region *region, u64 size,
40+
struct dmem_cgroup_pool_state **ret_pool,
41+
struct dmem_cgroup_pool_state **ret_limit_pool)
42+
{
43+
*ret_pool = NULL;
44+
45+
if (ret_limit_pool)
46+
*ret_limit_pool = NULL;
47+
48+
return 0;
49+
}
50+
51+
static inline void dmem_cgroup_uncharge(struct dmem_cgroup_pool_state *pool, u64 size)
52+
{ }
53+
54+
static inline
55+
bool dmem_cgroup_state_evict_valuable(struct dmem_cgroup_pool_state *limit_pool,
56+
struct dmem_cgroup_pool_state *test_pool,
57+
bool ignore_low, bool *ret_hit_low)
58+
{
59+
return true;
60+
}
61+
62+
static inline void dmem_cgroup_pool_state_put(struct dmem_cgroup_pool_state *pool)
63+
{ }
64+
65+
#endif
66+
#endif /* _CGROUP_DMEM_H */

‎include/linux/cgroup_subsys.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,10 @@ SUBSYS(rdma)
6565
SUBSYS(misc)
6666
#endif
6767

68+
#if IS_ENABLED(CONFIG_CGROUP_DMEM)
69+
SUBSYS(dmem)
70+
#endif
71+
6872
/*
6973
* The following subsystems are not supported on the default hierarchy.
7074
*/

‎include/linux/page_counter.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ static inline void page_counter_reset_watermark(struct page_counter *counter)
9696
counter->watermark = usage;
9797
}
9898

99-
#ifdef CONFIG_MEMCG
99+
#if IS_ENABLED(CONFIG_MEMCG) || IS_ENABLED(CONFIG_CGROUP_DMEM)
100100
void page_counter_calculate_protection(struct page_counter *root,
101101
struct page_counter *counter,
102102
bool recursive_protection);

‎init/Kconfig

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1128,6 +1128,7 @@ config CGROUP_PIDS
11281128

11291129
config CGROUP_RDMA
11301130
bool "RDMA controller"
1131+
select PAGE_COUNTER
11311132
help
11321133
Provides enforcement of RDMA resources defined by IB stack.
11331134
It is fairly easy for consumers to exhaust RDMA resources, which
@@ -1136,6 +1137,15 @@ config CGROUP_RDMA
11361137
Attaching processes with active RDMA resources to the cgroup
11371138
hierarchy is allowed even if can cross the hierarchy's limit.
11381139

1140+
config CGROUP_DMEM
1141+
bool "Device memory controller (DMEM)"
1142+
help
1143+
The DMEM controller allows compatible devices to restrict device
1144+
memory usage based on the cgroup hierarchy.
1145+
1146+
As an example, it allows you to restrict VRAM usage for applications
1147+
in the DRM subsystem.
1148+
11391149
config CGROUP_FREEZER
11401150
bool "Freezer controller"
11411151
help

‎kernel/cgroup/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,5 @@ obj-$(CONFIG_CGROUP_RDMA) += rdma.o
77
obj-$(CONFIG_CPUSETS) += cpuset.o
88
obj-$(CONFIG_CPUSETS_V1) += cpuset-v1.o
99
obj-$(CONFIG_CGROUP_MISC) += misc.o
10+
obj-$(CONFIG_CGROUP_DMEM) += dmem.o
1011
obj-$(CONFIG_CGROUP_DEBUG) += debug.o

0 commit comments

Comments
 (0)