[v7,20/24] x86/resctrl: Add CPU online callback for resctrl work
Commit Message
The resctrl architecture specific code may need to create a domain when
a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register.
The resctrl filesystem code needs to update the rdtgroup_default CPU
mask when CPUs are brought online.
Currently this is all done in one function, resctrl_online_cpu().
This will need to be split into architecture and filesystem parts
before resctrl can be moved to /fs/.
Pull the rdtgroup_default update work out as a filesystem specific
cpu_online helper. resctrl_online_cpu() is the obvious name for this,
which means the version in core.c needs renaming.
resctrl_online_cpu() is called by the arch code once it has done the
work to add the new CPU to any domains.
In future patches, resctrl_online_cpu() will take the rdtgroup_mutex
itself.
Tested-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
Tested-by: Peter Newman <peternewman@google.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: James Morse <james.morse@arm.com>
---
Changes since v3:
* Renamed err to ret
Changes since v4:
* Changes in capitalisation.
Changes since v5:
* More changes in capitalisation.
* Made resctrl_online_cpu() return void.
No changes since v6
---
arch/x86/kernel/cpu/resctrl/core.c | 8 ++++----
arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++++++
include/linux/resctrl.h | 1 +
3 files changed, 13 insertions(+), 4 deletions(-)
Comments
On 10/25/23 13:03, James Morse wrote:
> The resctrl architecture specific code may need to create a domain when
> a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register.
> The resctrl filesystem code needs to update the rdtgroup_default CPU
> mask when CPUs are brought online.
>
> Currently this is all done in one function, resctrl_online_cpu().
> This will need to be split into architecture and filesystem parts
> before resctrl can be moved to /fs/.
>
> Pull the rdtgroup_default update work out as a filesystem specific
> cpu_online helper. resctrl_online_cpu() is the obvious name for this,
> which means the version in core.c needs renaming.
>
> resctrl_online_cpu() is called by the arch code once it has done the
> work to add the new CPU to any domains.
>
> In future patches, resctrl_online_cpu() will take the rdtgroup_mutex
> itself.
>
> Tested-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
> Tested-by: Peter Newman <peternewman@google.com>
> Reviewed-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
> Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
> Signed-off-by: James Morse <james.morse@arm.com>
Reviewed-by: Babu Moger <babu.moger@amd.com>
> ---
> Changes since v3:
> * Renamed err to ret
>
> Changes since v4:
> * Changes in capitalisation.
>
> Changes since v5:
> * More changes in capitalisation.
> * Made resctrl_online_cpu() return void.
>
> No changes since v6
> ---
> arch/x86/kernel/cpu/resctrl/core.c | 8 ++++----
> arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++++++
> include/linux/resctrl.h | 1 +
> 3 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index fc70a2650729..1a74e9c47416 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -608,16 +608,16 @@ static void clear_closid_rmid(int cpu)
> RESCTRL_RESERVED_CLOSID);
> }
>
> -static int resctrl_online_cpu(unsigned int cpu)
> +static int resctrl_arch_online_cpu(unsigned int cpu)
> {
> struct rdt_resource *r;
>
> mutex_lock(&rdtgroup_mutex);
> for_each_capable_rdt_resource(r)
> domain_add_cpu(cpu, r);
> - /* The cpu is set in default rdtgroup after online. */
> - cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
> clear_closid_rmid(cpu);
> +
> + resctrl_online_cpu(cpu);
> mutex_unlock(&rdtgroup_mutex);
>
> return 0;
> @@ -969,7 +969,7 @@ static int __init resctrl_late_init(void)
>
> state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> "x86/resctrl/cat:online:",
> - resctrl_online_cpu, resctrl_offline_cpu);
> + resctrl_arch_online_cpu, resctrl_offline_cpu);
> if (state < 0)
> return state;
>
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index bd4d1c8a9b4f..ab9db7ce706f 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -4012,6 +4012,14 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
> return 0;
> }
>
> +void resctrl_online_cpu(unsigned int cpu)
> +{
> + lockdep_assert_held(&rdtgroup_mutex);
> +
> + /* The CPU is set in default rdtgroup after online. */
> + cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
> +}
> +
> /*
> * rdtgroup_init - rdtgroup initialization
> *
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index bf460c912bf5..4c4bad3c34e4 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -223,6 +223,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
> u32 closid, enum resctrl_conf_type type);
> int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d);
> void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);
> +void resctrl_online_cpu(unsigned int cpu);
>
> /**
> * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Hi Babu,
On 09/11/2023 20:51, Moger, Babu wrote:
> On 10/25/23 13:03, James Morse wrote:
>> The resctrl architecture specific code may need to create a domain when
>> a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register.
>> The resctrl filesystem code needs to update the rdtgroup_default CPU
>> mask when CPUs are brought online.
>>
>> Currently this is all done in one function, resctrl_online_cpu().
>> This will need to be split into architecture and filesystem parts
>> before resctrl can be moved to /fs/.
>>
>> Pull the rdtgroup_default update work out as a filesystem specific
>> cpu_online helper. resctrl_online_cpu() is the obvious name for this,
>> which means the version in core.c needs renaming.
>>
>> resctrl_online_cpu() is called by the arch code once it has done the
>> work to add the new CPU to any domains.
>>
>> In future patches, resctrl_online_cpu() will take the rdtgroup_mutex
>> itself.
> Reviewed-by: Babu Moger <babu.moger@amd.com>
Thanks!
James
@@ -608,16 +608,16 @@ static void clear_closid_rmid(int cpu)
RESCTRL_RESERVED_CLOSID);
}
-static int resctrl_online_cpu(unsigned int cpu)
+static int resctrl_arch_online_cpu(unsigned int cpu)
{
struct rdt_resource *r;
mutex_lock(&rdtgroup_mutex);
for_each_capable_rdt_resource(r)
domain_add_cpu(cpu, r);
- /* The cpu is set in default rdtgroup after online. */
- cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
clear_closid_rmid(cpu);
+
+ resctrl_online_cpu(cpu);
mutex_unlock(&rdtgroup_mutex);
return 0;
@@ -969,7 +969,7 @@ static int __init resctrl_late_init(void)
state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
"x86/resctrl/cat:online:",
- resctrl_online_cpu, resctrl_offline_cpu);
+ resctrl_arch_online_cpu, resctrl_offline_cpu);
if (state < 0)
return state;
@@ -4012,6 +4012,14 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
return 0;
}
+void resctrl_online_cpu(unsigned int cpu)
+{
+ lockdep_assert_held(&rdtgroup_mutex);
+
+ /* The CPU is set in default rdtgroup after online. */
+ cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
+}
+
/*
* rdtgroup_init - rdtgroup initialization
*
@@ -223,6 +223,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
u32 closid, enum resctrl_conf_type type);
int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d);
void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);
+void resctrl_online_cpu(unsigned int cpu);
/**
* resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid