[v11,1/3] ACPI: APEI: send SIGBUS to current task if synchronous memory error not recovered
Message ID | 20240204080144.7977-2-xueshuai@linux.alibaba.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-51509-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp244144dyb; Sun, 4 Feb 2024 00:02:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IFyTsZjaVnZ/gnVHAft/Y6zcQxMHbKFZ1kSU/bboUnjYpxx35ZI1g7+y/1B68491/TH+/yf X-Received: by 2002:a05:6122:182a:b0:4c0:e71:7807 with SMTP id ay42-20020a056122182a00b004c00e717807mr3681291vkb.15.1707033761832; Sun, 04 Feb 2024 00:02:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707033761; cv=pass; d=google.com; s=arc-20160816; b=MSEfkg7o3LS8XnZuuGTplADzaxwglLgeaDmt90Lz4oknTsfWKhGWkf6K3ovSyAyAOp zcM/XSkKaM0E1AknxehzVNzVDrrcovLv5bKBXX65RNG06ZPDxQXySMrTJSfXfhXtmwvn zcRfgtqe7IBF7vC64VSl8r04rVGGcMCYnhw7CF3b6wsrGWetd7/wQMjkwc/rreS1G0Ea E0SgUIXaTeobsHP7TXWkUl1xTicpPMd+e/ZHl80JDmT9Lz6CN2D4OuY67o0TVMTFvODs z1D6W75PHZfDJgOrD8ZKA6+HI5mHNQIaf/X1ARrZ13anl91EJT96lIY7Hd3A3L2GxNg6 Fv1g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=da8Wgzxv5myuYH4WAVzsx3YJvaNBmr54eTJ+3qvFUD4=; fh=OpjFtsGJYOiHzq2IrSSWkjXTvIbO3MOgp41imPvDPvI=; b=GQf+pPUQK0KOU9VwnSaxPfaqfkoNMddzyogFWRpvdZEbqyvNCKtD7gp6OKZphZL/5P lm1idJCQ6iTYrjqY7T0VlUVTKggWcGk7QknTn7Aa2z4slV9JNbQ6B01HH+NA100ZMIfg iEzPMejEzIWSEo+ZUOfyVIgJTPBa1V6dvStN1WGIyHHsW68EiWs7Shj+ba1efmVydRp3 ZNPUgyNKupaR9XiGWPm5iigm7MlYGTNS1c5D6SrVs4u6G+oTioq4d057SQxg7XjkqEGO bOQbYSa12fwAEgzvHFVezRXD38Qrk4oDufMFk3ycWxYisQ/vfRlZN50U3ly0DL7j/UsV tILw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=TY3+MeP3; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-51509-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-51509-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com X-Forwarded-Encrypted: i=1; AJvYcCUlFKaSv7dA5wHQ04tEHX3hEv2pP/MsTX7YQcEkjZf6qltnpq2O2CWX9cAIhnk5su5o7O8lqiCKxqnrjjTojtqVcUZLIQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id a16-20020a0ce910000000b0068c8771623csi5161176qvo.289.2024.02.04.00.02.41 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Feb 2024 00:02:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-51509-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=TY3+MeP3; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-51509-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-51509-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9E6D81C210E6 for <ouuuleilei@gmail.com>; Sun, 4 Feb 2024 08:02:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 35A271119C; Sun, 4 Feb 2024 08:02:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="TY3+MeP3" Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1914C139; Sun, 4 Feb 2024 08:02:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707033733; cv=none; b=KB6q7iHsTjoY3dKvTXrDiA8xUOBzApIJCtbhhM0NhyaHunRVgtS+eMasHhiVqjv8NkwSzKbHS1xc3Bdzr72d9Ju8FUJDp/q7dBnLv/J9drKjumVtjZh6qPEvk4OzvPGI6blri3AJs3BcxkJB/fLmODV+jrbpM6m/z07+eUbJ9gI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707033733; c=relaxed/simple; bh=91jMp1zqiCFeYpSsr2SaCC6w78t1qGrqnT5tMqIAdfU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HQLjuOAk3tDxYOWB4yxsT2n7K6B7dpBzcjJSrC+udCAi+/RnV3gB7rPzKNj+kaNGyWFWODI8BiNgxNl4ft9vKK6RvXYCOLP+v3xD/yeeSOa3rSIZAdivntJYndDKD+7jvl8P3fQPr9rGHjwjCOzdEeypJT8VKg3zVbqd3HdAhSQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=TY3+MeP3; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1707033723; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=da8Wgzxv5myuYH4WAVzsx3YJvaNBmr54eTJ+3qvFUD4=; b=TY3+MeP3MzxAkrQF+/624RPSmBF9HoLlB9NdCSG6f4pMA9GzQzVGiWiq2LAqC6ETuSEpyDUMQ55jb1RRVk/9fR+CA4CASqqAlCZb4Jti0jTsb+4nR2yLwo2Kour20dYV1vk3Ms4rb3IfgPU/Z/S/9QoGOXghyNIn5ECurur52dU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R471e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xueshuai@linux.alibaba.com;NM=1;PH=DS;RN=33;SR=0;TI=SMTPD_---0W0003JZ_1707033719; Received: from localhost.localdomain(mailfrom:xueshuai@linux.alibaba.com fp:SMTPD_---0W0003JZ_1707033719) by smtp.aliyun-inc.com; Sun, 04 Feb 2024 16:02:01 +0800 From: Shuai Xue <xueshuai@linux.alibaba.com> To: bp@alien8.de, rafael@kernel.org, wangkefeng.wang@huawei.com, tanxiaofei@huawei.com, mawupeng1@huawei.com, tony.luck@intel.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, james.morse@arm.com, gregkh@linuxfoundation.org, will@kernel.org, jarkko@kernel.org Cc: linux-acpi@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-edac@vger.kernel.org, x86@kernel.org, xueshuai@linux.alibaba.com, justin.he@arm.com, ardb@kernel.org, ying.huang@intel.com, ashish.kalra@amd.com, baolin.wang@linux.alibaba.com, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, lenb@kernel.org, hpa@zytor.com, robert.moore@intel.com, lvying6@huawei.com, xiexiuqi@huawei.com, zhuo.song@linux.alibaba.com Subject: [PATCH v11 1/3] ACPI: APEI: send SIGBUS to current task if synchronous memory error not recovered Date: Sun, 4 Feb 2024 16:01:42 +0800 Message-Id: <20240204080144.7977-2-xueshuai@linux.alibaba.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221027042445.60108-1-xueshuai@linux.alibaba.com> References: <20221027042445.60108-1-xueshuai@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785601506577001452 X-GMAIL-MSGID: 1789954633822384189 |
Series |
[v11,1/3] ACPI: APEI: send SIGBUS to current task if synchronous memory error not recovered
|
|
Commit Message
Shuai Xue
Feb. 4, 2024, 8:01 a.m. UTC
Synchronous error was detected as a result of user-space process accessing
a 2-bit uncorrected error. The CPU will take a synchronous error exception
such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a
memory_failure() work which poisons the related page, unmaps the page, and
then sends a SIGBUS to the process, so that a system wide panic can be
avoided.
However, no memory_failure() work will be queued when abnormal synchronous
errors occur. These errors can include situations such as invalid PA,
unexpected severity, no memory failure config support, invalid GUID
section, etc. In such case, the user-space process will trigger SEA again.
This loop can potentially exceed the platform firmware threshold or even
trigger a kernel hard lockup, leading to a system reboot.
Fix it by performing a force kill if no memory_failure() work is queued
for synchronous errors.
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
---
drivers/acpi/apei/ghes.c | 9 +++++++++
1 file changed, 9 insertions(+)
Comments
On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: > Synchronous error was detected as a result of user-space process accessing > a 2-bit uncorrected error. The CPU will take a synchronous error exception > such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a > memory_failure() work which poisons the related page, unmaps the page, and > then sends a SIGBUS to the process, so that a system wide panic can be > avoided. > > However, no memory_failure() work will be queued when abnormal synchronous > errors occur. These errors can include situations such as invalid PA, > unexpected severity, no memory failure config support, invalid GUID > section, etc. In such case, the user-space process will trigger SEA again. > This loop can potentially exceed the platform firmware threshold or even > trigger a kernel hard lockup, leading to a system reboot. > > Fix it by performing a force kill if no memory_failure() work is queued > for synchronous errors. > > Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> > --- > drivers/acpi/apei/ghes.c | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c > index 7b7c605166e0..0892550732d4 100644 > --- a/drivers/acpi/apei/ghes.c > +++ b/drivers/acpi/apei/ghes.c > @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, > } > } > > + /* > + * If no memory failure work is queued for abnormal synchronous > + * errors, do a force kill. > + */ > + if (sync && !queued) { > + pr_err("Sending SIGBUS to current task due to memory error not recovered"); > + force_sig(SIGBUS); > + } Except that there are a bunch of CXL GUIDs being handled there too and this will sigbus those processes now automatically. Lemme add the whole bunch from 671a794c33c6 ("acpi/ghes: Process CXL Component Events") for comment to Cc.
On 2024/2/19 17:25, Borislav Petkov wrote: > On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: >> Synchronous error was detected as a result of user-space process accessing >> a 2-bit uncorrected error. The CPU will take a synchronous error exception >> such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a >> memory_failure() work which poisons the related page, unmaps the page, and >> then sends a SIGBUS to the process, so that a system wide panic can be >> avoided. >> >> However, no memory_failure() work will be queued when abnormal synchronous >> errors occur. These errors can include situations such as invalid PA, >> unexpected severity, no memory failure config support, invalid GUID >> section, etc. In such case, the user-space process will trigger SEA again. >> This loop can potentially exceed the platform firmware threshold or even >> trigger a kernel hard lockup, leading to a system reboot. >> >> Fix it by performing a force kill if no memory_failure() work is queued >> for synchronous errors. >> >> Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> >> --- >> drivers/acpi/apei/ghes.c | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> >> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c >> index 7b7c605166e0..0892550732d4 100644 >> --- a/drivers/acpi/apei/ghes.c >> +++ b/drivers/acpi/apei/ghes.c >> @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, >> } >> } >> >> + /* >> + * If no memory failure work is queued for abnormal synchronous >> + * errors, do a force kill. >> + */ >> + if (sync && !queued) { >> + pr_err("Sending SIGBUS to current task due to memory error not recovered"); >> + force_sig(SIGBUS); >> + } > > Except that there are a bunch of CXL GUIDs being handled there too and > this will sigbus those processes now automatically. Before the CXL GUIDs added, @Tony confirmed that the HEST notifications are always asynchronous on x86 platform, so only Synchronous External Abort (SEA) on ARM is delivered as a synchronous notification. Will the CXL component trigger synchronous events for which we need to terminate the current process by sending sigbus to process? > > Lemme add the whole bunch from > > 671a794c33c6 ("acpi/ghes: Process CXL Component Events") > > for comment to Cc. > Thank you. Best Regards, Shuai
Shuai Xue wrote: > > > On 2024/2/19 17:25, Borislav Petkov wrote: > > On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: > >> Synchronous error was detected as a result of user-space process accessing > >> a 2-bit uncorrected error. The CPU will take a synchronous error exception > >> such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a > >> memory_failure() work which poisons the related page, unmaps the page, and > >> then sends a SIGBUS to the process, so that a system wide panic can be > >> avoided. > >> > >> However, no memory_failure() work will be queued when abnormal synchronous > >> errors occur. These errors can include situations such as invalid PA, > >> unexpected severity, no memory failure config support, invalid GUID > >> section, etc. In such case, the user-space process will trigger SEA again. > >> This loop can potentially exceed the platform firmware threshold or even > >> trigger a kernel hard lockup, leading to a system reboot. > >> > >> Fix it by performing a force kill if no memory_failure() work is queued > >> for synchronous errors. > >> > >> Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> > >> --- > >> drivers/acpi/apei/ghes.c | 9 +++++++++ > >> 1 file changed, 9 insertions(+) > >> > >> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c > >> index 7b7c605166e0..0892550732d4 100644 > >> --- a/drivers/acpi/apei/ghes.c > >> +++ b/drivers/acpi/apei/ghes.c > >> @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, > >> } > >> } > >> > >> + /* > >> + * If no memory failure work is queued for abnormal synchronous > >> + * errors, do a force kill. > >> + */ > >> + if (sync && !queued) { > >> + pr_err("Sending SIGBUS to current task due to memory error not recovered"); > >> + force_sig(SIGBUS); > >> + } > > > > Except that there are a bunch of CXL GUIDs being handled there too and > > this will sigbus those processes now automatically. > > Before the CXL GUIDs added, @Tony confirmed that the HEST notifications are always > asynchronous on x86 platform, so only Synchronous External Abort (SEA) on ARM is > delivered as a synchronous notification. > > Will the CXL component trigger synchronous events for which we need to terminate the > current process by sending sigbus to process? None of the CXL component errors should be handled as synchronous events. They are either asynchronous protocol errors, or effectively equivalent to CPER_SEC_PLATFORM_MEM notifications.
On Thu, 22 Feb 2024 21:26:43 -0800 Dan Williams <dan.j.williams@intel.com> wrote: > Shuai Xue wrote: > > > > > > On 2024/2/19 17:25, Borislav Petkov wrote: > > > On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: > > >> Synchronous error was detected as a result of user-space process accessing > > >> a 2-bit uncorrected error. The CPU will take a synchronous error exception > > >> such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a > > >> memory_failure() work which poisons the related page, unmaps the page, and > > >> then sends a SIGBUS to the process, so that a system wide panic can be > > >> avoided. > > >> > > >> However, no memory_failure() work will be queued when abnormal synchronous > > >> errors occur. These errors can include situations such as invalid PA, > > >> unexpected severity, no memory failure config support, invalid GUID > > >> section, etc. In such case, the user-space process will trigger SEA again. > > >> This loop can potentially exceed the platform firmware threshold or even > > >> trigger a kernel hard lockup, leading to a system reboot. > > >> > > >> Fix it by performing a force kill if no memory_failure() work is queued > > >> for synchronous errors. > > >> > > >> Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> > > >> --- > > >> drivers/acpi/apei/ghes.c | 9 +++++++++ > > >> 1 file changed, 9 insertions(+) > > >> > > >> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c > > >> index 7b7c605166e0..0892550732d4 100644 > > >> --- a/drivers/acpi/apei/ghes.c > > >> +++ b/drivers/acpi/apei/ghes.c > > >> @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, > > >> } > > >> } > > >> > > >> + /* > > >> + * If no memory failure work is queued for abnormal synchronous > > >> + * errors, do a force kill. > > >> + */ > > >> + if (sync && !queued) { > > >> + pr_err("Sending SIGBUS to current task due to memory error not recovered"); > > >> + force_sig(SIGBUS); > > >> + } > > > > > > Except that there are a bunch of CXL GUIDs being handled there too and > > > this will sigbus those processes now automatically. > > > > Before the CXL GUIDs added, @Tony confirmed that the HEST notifications are always > > asynchronous on x86 platform, so only Synchronous External Abort (SEA) on ARM is > > delivered as a synchronous notification. > > > > Will the CXL component trigger synchronous events for which we need to terminate the > > current process by sending sigbus to process? > > None of the CXL component errors should be handled as synchronous > events. They are either asynchronous protocol errors, or effectively > equivalent to CPER_SEC_PLATFORM_MEM notifications. Not a good example, CPER_SEC_PLATFORM_MEM is sometimes signaled via SEA.
On Fri, 23 Feb 2024 12:08:13 +0000 Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote: > On Thu, 22 Feb 2024 21:26:43 -0800 > Dan Williams <dan.j.williams@intel.com> wrote: > > > Shuai Xue wrote: > > > > > > > > > On 2024/2/19 17:25, Borislav Petkov wrote: > > > > On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: > > > >> Synchronous error was detected as a result of user-space process accessing > > > >> a 2-bit uncorrected error. The CPU will take a synchronous error exception > > > >> such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a > > > >> memory_failure() work which poisons the related page, unmaps the page, and > > > >> then sends a SIGBUS to the process, so that a system wide panic can be > > > >> avoided. > > > >> > > > >> However, no memory_failure() work will be queued when abnormal synchronous > > > >> errors occur. These errors can include situations such as invalid PA, > > > >> unexpected severity, no memory failure config support, invalid GUID > > > >> section, etc. In such case, the user-space process will trigger SEA again. > > > >> This loop can potentially exceed the platform firmware threshold or even > > > >> trigger a kernel hard lockup, leading to a system reboot. > > > >> > > > >> Fix it by performing a force kill if no memory_failure() work is queued > > > >> for synchronous errors. > > > >> > > > >> Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> > > > >> --- > > > >> drivers/acpi/apei/ghes.c | 9 +++++++++ > > > >> 1 file changed, 9 insertions(+) > > > >> > > > >> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c > > > >> index 7b7c605166e0..0892550732d4 100644 > > > >> --- a/drivers/acpi/apei/ghes.c > > > >> +++ b/drivers/acpi/apei/ghes.c > > > >> @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, > > > >> } > > > >> } > > > >> > > > >> + /* > > > >> + * If no memory failure work is queued for abnormal synchronous > > > >> + * errors, do a force kill. > > > >> + */ > > > >> + if (sync && !queued) { > > > >> + pr_err("Sending SIGBUS to current task due to memory error not recovered"); > > > >> + force_sig(SIGBUS); > > > >> + } > > > > > > > > Except that there are a bunch of CXL GUIDs being handled there too and > > > > this will sigbus those processes now automatically. > > > > > > Before the CXL GUIDs added, @Tony confirmed that the HEST notifications are always > > > asynchronous on x86 platform, so only Synchronous External Abort (SEA) on ARM is > > > delivered as a synchronous notification. > > > > > > Will the CXL component trigger synchronous events for which we need to terminate the > > > current process by sending sigbus to process? > > > > None of the CXL component errors should be handled as synchronous > > events. They are either asynchronous protocol errors, or effectively > > equivalent to CPER_SEC_PLATFORM_MEM notifications. > > Not a good example, CPER_SEC_PLATFORM_MEM is sometimes signaled via SEA. > Premature send.:( One example I can point at is how we do signaling of memory errors detected by the host into a VM on arm64. https://elixir.bootlin.com/qemu/latest/source/hw/acpi/ghes.c#L391 CPER_SEC_PLATFORM_MEM via ARM Synchronous External Abort (SEA). Right now we've only used async in QEMU for proposed CXL error CPER records signalling but your reference to them being similar to CPER_SEC_PLATFORM_MEM is valid so 'maybe' they will be synchronous in some physical systems as it's one viable way to provide rich information for synchronous reception of poison. For the VM case my assumption today is we don't care about providing the VM with rich data, so CPER_SEC_PLATFORM_MEM is fine as a path for errors whether from CXL CPER records or not. Jonathan
On 2024/2/23 20:17, Jonathan Cameron wrote: > On Fri, 23 Feb 2024 12:08:13 +0000 > Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote: > >> On Thu, 22 Feb 2024 21:26:43 -0800 >> Dan Williams <dan.j.williams@intel.com> wrote: >> >>> Shuai Xue wrote: >>>> >>>> >>>> On 2024/2/19 17:25, Borislav Petkov wrote: >>>>> On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: >>>>>> Synchronous error was detected as a result of user-space process accessing >>>>>> a 2-bit uncorrected error. The CPU will take a synchronous error exception >>>>>> such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a >>>>>> memory_failure() work which poisons the related page, unmaps the page, and >>>>>> then sends a SIGBUS to the process, so that a system wide panic can be >>>>>> avoided. >>>>>> >>>>>> However, no memory_failure() work will be queued when abnormal synchronous >>>>>> errors occur. These errors can include situations such as invalid PA, >>>>>> unexpected severity, no memory failure config support, invalid GUID >>>>>> section, etc. In such case, the user-space process will trigger SEA again. >>>>>> This loop can potentially exceed the platform firmware threshold or even >>>>>> trigger a kernel hard lockup, leading to a system reboot. >>>>>> >>>>>> Fix it by performing a force kill if no memory_failure() work is queued >>>>>> for synchronous errors. >>>>>> >>>>>> Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> >>>>>> --- >>>>>> drivers/acpi/apei/ghes.c | 9 +++++++++ >>>>>> 1 file changed, 9 insertions(+) >>>>>> >>>>>> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c >>>>>> index 7b7c605166e0..0892550732d4 100644 >>>>>> --- a/drivers/acpi/apei/ghes.c >>>>>> +++ b/drivers/acpi/apei/ghes.c >>>>>> @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, >>>>>> } >>>>>> } >>>>>> >>>>>> + /* >>>>>> + * If no memory failure work is queued for abnormal synchronous >>>>>> + * errors, do a force kill. >>>>>> + */ >>>>>> + if (sync && !queued) { >>>>>> + pr_err("Sending SIGBUS to current task due to memory error not recovered"); >>>>>> + force_sig(SIGBUS); >>>>>> + } >>>>> >>>>> Except that there are a bunch of CXL GUIDs being handled there too and >>>>> this will sigbus those processes now automatically. >>>> >>>> Before the CXL GUIDs added, @Tony confirmed that the HEST notifications are always >>>> asynchronous on x86 platform, so only Synchronous External Abort (SEA) on ARM is >>>> delivered as a synchronous notification. >>>> >>>> Will the CXL component trigger synchronous events for which we need to terminate the >>>> current process by sending sigbus to process? >>> >>> None of the CXL component errors should be handled as synchronous >>> events. They are either asynchronous protocol errors, or effectively >>> equivalent to CPER_SEC_PLATFORM_MEM notifications. >> >> Not a good example, CPER_SEC_PLATFORM_MEM is sometimes signaled via SEA. >> > > Premature send.:( > > One example I can point at is how we do signaling of memory > errors detected by the host into a VM on arm64. > https://elixir.bootlin.com/qemu/latest/source/hw/acpi/ghes.c#L391 > CPER_SEC_PLATFORM_MEM via ARM Synchronous External Abort (SEA). > > Right now we've only used async in QEMU for proposed CXL error > CPER records signalling but your reference to them being similar > to CPER_SEC_PLATFORM_MEM is valid so 'maybe' they will be > synchronous in some physical systems as it's one viable way to > provide rich information for synchronous reception of poison. > For the VM case my assumption today is we don't care about providing the > VM with rich data, so CPER_SEC_PLATFORM_MEM is fine as a path for > errors whether from CXL CPER records or not. > > Jonathan Thank you for your confirmation and explanation. So I think the condition: - `sync` for synchronous event, - `!queued` for CPER_SEC_PLATFORM_MEM notifications which do not handle memory failures. is fine. @Borislav, do you have any other concerns? Best Regards, Shuai
Borislav Petkov wrote: > On Sun, Feb 04, 2024 at 04:01:42PM +0800, Shuai Xue wrote: > > Synchronous error was detected as a result of user-space process accessing > > a 2-bit uncorrected error. The CPU will take a synchronous error exception > > such as Synchronous External Abort (SEA) on Arm64. The kernel will queue a > > memory_failure() work which poisons the related page, unmaps the page, and > > then sends a SIGBUS to the process, so that a system wide panic can be > > avoided. > > > > However, no memory_failure() work will be queued when abnormal synchronous > > errors occur. These errors can include situations such as invalid PA, > > unexpected severity, no memory failure config support, invalid GUID > > section, etc. In such case, the user-space process will trigger SEA again. > > This loop can potentially exceed the platform firmware threshold or even > > trigger a kernel hard lockup, leading to a system reboot. > > > > Fix it by performing a force kill if no memory_failure() work is queued > > for synchronous errors. > > > > Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com> > > --- > > drivers/acpi/apei/ghes.c | 9 +++++++++ > > 1 file changed, 9 insertions(+) > > > > diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c > > index 7b7c605166e0..0892550732d4 100644 > > --- a/drivers/acpi/apei/ghes.c > > +++ b/drivers/acpi/apei/ghes.c > > @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, > > } > > } > > > > + /* > > + * If no memory failure work is queued for abnormal synchronous > > + * errors, do a force kill. > > + */ > > + if (sync && !queued) { > > + pr_err("Sending SIGBUS to current task due to memory error not recovered"); > > + force_sig(SIGBUS); > > + } > > Except that there are a bunch of CXL GUIDs being handled there too and > this will sigbus those processes now automatically. > > Lemme add the whole bunch from > > 671a794c33c6 ("acpi/ghes: Process CXL Component Events") > > for comment to Cc. BTW, I am about to revert all the CXL GUIDs for v6.8 to try again for v6.9: http://lore.kernel.org/r/170820177849.631006.8893584762602010898.stgit@dwillia2-xfh.jf.intel.com
Jonathan Cameron wrote: [..] > > > None of the CXL component errors should be handled as synchronous > > > events. They are either asynchronous protocol errors, or effectively > > > equivalent to CPER_SEC_PLATFORM_MEM notifications. > > > > Not a good example, CPER_SEC_PLATFORM_MEM is sometimes signaled via SEA. > > > > Premature send.:( > > One example I can point at is how we do signaling of memory > errors detected by the host into a VM on arm64. > https://elixir.bootlin.com/qemu/latest/source/hw/acpi/ghes.c#L391 > CPER_SEC_PLATFORM_MEM via ARM Synchronous External Abort (SEA). > > Right now we've only used async in QEMU for proposed CXL error > CPER records signalling but your reference to them being similar > to CPER_SEC_PLATFORM_MEM is valid so 'maybe' they will be > synchronous in some physical systems as it's one viable way to > provide rich information for synchronous reception of poison. > For the VM case my assumption today is we don't care about providing the > VM with rich data, so CPER_SEC_PLATFORM_MEM is fine as a path for > errors whether from CXL CPER records or not. Makes sense... and I was not precise when I mentioned the equivalency, I was only considering x86.
On Sat, Feb 24, 2024 at 02:08:42PM +0800, Shuai Xue wrote:
> @Borislav, do you have any other concerns?
Yes, this change needs to be further reviewed by an ARM person: I have
no clue what those "abnormal synchronous errors" on ARM are and how
they're supposed to be handled properly there:
- what happens if you get such an error when ghes is disabled there?
- is that even the right place to handle them?
James?
On 2024/2/26 18:29, Borislav Petkov wrote: > On Sat, Feb 24, 2024 at 02:08:42PM +0800, Shuai Xue wrote: >> @Borislav, do you have any other concerns? > > Yes, this change needs to be further reviewed by an ARM person: I have > no clue what those "abnormal synchronous errors" on ARM are Hi, Borislav, May the `abnormal` is not inaccurate and misled you. I mean the preconditions check before memory_failure_queue(): - `if (!(mem_err->validation_bits & CPER_MEM_VALID_PA))` in ghes_handle_memory_failure() - `if (flags == -1)` in ghes_handle_memory_failure() - `if (!IS_ENABLED(CONFIG_ACPI_APEI_MEMORY_FAILURE))` in ghes_do_memory_failure() - `if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) ` in ghes_do_memory_failure() If the preconditions are not passed, the user-space process will trigger SEA again. This loop can potentially exceed the platform firmware threshold or even trigger a kernel hard lockup, leading to a system reboot. > and how > they're supposed to be handled properly there: > > - what happens if you get such an error when ghes is disabled there? If ghes_disable is set, the GHES driver will not be inited by acpi_ghes_init(), so none of error notifications will be handled. IMHO, it is expected. > > - is that even the right place to handle them? > > James? > Leave this to @James. Thank you. Best Regards, Shuai
diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index 7b7c605166e0..0892550732d4 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -806,6 +806,15 @@ static bool ghes_do_proc(struct ghes *ghes, } } + /* + * If no memory failure work is queued for abnormal synchronous + * errors, do a force kill. + */ + if (sync && !queued) { + pr_err("Sending SIGBUS to current task due to memory error not recovered"); + force_sig(SIGBUS); + } + return queued; }