From patchwork Wed Dec 21 14:23:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 35389 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp3552865wrn; Wed, 21 Dec 2022 06:26:30 -0800 (PST) X-Google-Smtp-Source: AMrXdXuXeC8DS/Fqlp4PoJl+R4Kez7fjQ9gWpHHsxUQsbWddieZp72ocpq0B5UO7vJm5GWYrr4TI X-Received: by 2002:a17:902:e392:b0:192:4d6a:2adc with SMTP id g18-20020a170902e39200b001924d6a2adcmr326606ple.52.1671632790389; Wed, 21 Dec 2022 06:26:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671632790; cv=none; d=google.com; s=arc-20160816; b=LroxwIP5jV8eAybBrISv2j94FS5IEi51XY95D5jw12sLta+lfywV96kXP1R7UMh+FJ j0ld4zJK/dgpenp8RToD4LmPS8KZjKd7g/pyt5ztLVAsqlTjtwnWG0ARNTOEVMhq/zsS Mve/1pcuXXXV9DdQTqLxhhh4MSiqIK1r2qjxvTS6TOcrm+wC+XrVdSmzsTaQ5FnzLsO0 5Bv7f/O9pr7QZZsO8ysVIPF636MAtl0CKfEFHviTGo9aCi8uENvbM25+V0zr+kuwhcS+ nwSw17omngUQtxBJNlknCirHBp26/rasXprvN0P7W0DZe6BXzgTNJ/aHPGLh0+bS1iJd FBNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UiIAUwifV2YyWZQAy2FXEyRLZlyxjnNXA+vsOwxLvB8=; b=aWoqcU3zue+0nJX6+h4XoYESlyCtlND9ecB9JxLJSj0cI+BTyzg02Qh4X5XccDG7p+ D/hhQnVsgMjHzh/PjBO6izowEYOYtnlvAXdb939FvHwoM0yhGU0aYdERLzGbtwnbaYBZ 8TdsGeliL78KVAlnU8cut4Ce/K/ZF0ejpEzVOkeLjCETGuw5+nsCzwIR6jG9edbrer2R qoNDW/GTyYTXYQZtwzjpA1Sg8p6Nf9JBxG5pWTEKcdfVLStZDolyAo92kteGy9drEcNw Oq3FG8wm8qs0FoLLlPHZOU71QWs1yhYcupLxJlo9Tk0S18Pr9tRmu+qGqFr/vzksqhSs Z+Jg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@zx2c4.com header.s=20210105 header.b=Ao0tnENC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=zx2c4.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b11-20020a170903228b00b00188bee6e9f2si321681plh.239.2022.12.21.06.26.11; Wed, 21 Dec 2022 06:26:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@zx2c4.com header.s=20210105 header.b=Ao0tnENC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=zx2c4.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229791AbiLUOYT (ORCPT + 99 others); Wed, 21 Dec 2022 09:24:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233674AbiLUOYA (ORCPT ); Wed, 21 Dec 2022 09:24:00 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E58032339D; Wed, 21 Dec 2022 06:23:48 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 9AF7CB81B90; Wed, 21 Dec 2022 14:23:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D176DC433F0; Wed, 21 Dec 2022 14:23:44 +0000 (UTC) Authentication-Results: smtp.kernel.org; dkim=pass (1024-bit key) header.d=zx2c4.com header.i=@zx2c4.com header.b="Ao0tnENC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1671632624; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UiIAUwifV2YyWZQAy2FXEyRLZlyxjnNXA+vsOwxLvB8=; b=Ao0tnENCjdlEmgrrLimO/1H4vX88RsqTMlKIUi2l1rMCPgr9NYG5elruck2BP1UO9bdMzV BGA7V1OsByS0G3q0cvPyX8oRJLctZdHRiP6emiwWXNtpYOZimRM20eEmRRNLu3W81i9RHD Lnf394xVEEdaSlqZvaVlfJZiEzIE2f8= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id 95b72b9d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 21 Dec 2022 14:23:44 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org, patches@lists.linux.dev, tglx@linutronix.de Cc: "Jason A. Donenfeld" , linux-crypto@vger.kernel.org, linux-api@vger.kernel.org, x86@kernel.org, Greg Kroah-Hartman , Adhemerval Zanella Netto , Carlos O'Donell , Florian Weimer , Arnd Bergmann , Jann Horn , Christian Brauner , linux-mm@kvack.org Subject: [PATCH v13 3/7] x86: mm: Skip faulting instruction for VM_DROPPABLE faults Date: Wed, 21 Dec 2022 15:23:23 +0100 Message-Id: <20221221142327.126451-4-Jason@zx2c4.com> In-Reply-To: <20221221142327.126451-1-Jason@zx2c4.com> References: <20221221142327.126451-1-Jason@zx2c4.com> MIME-Version: 1.0 X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752834024735663702?= X-GMAIL-MSGID: =?utf-8?q?1752834024735663702?= The prior commit introduced VM_DROPPABLE, but in a limited form where the faulting instruction was retried instead of skipped. Finish that up with the platform-specific aspect of skipping the actual instruction. This works by copying userspace's %rip to a stack buffer of size MAX_INSN_SIZE, decoding it, and then adding the length of the decoded instruction to userspace's %rip. In the event any of these fail, just fallback to not advancing %rip and trying again. Cc: linux-mm@kvack.org Signed-off-by: Jason A. Donenfeld --- arch/x86/mm/fault.c | 19 +++++++++++++++++++ include/linux/mm_types.h | 5 ++++- mm/memory.c | 4 +++- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7b0d4ab894c8..76ca99ab6eb7 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -33,6 +33,8 @@ #include /* kvm_handle_async_pf */ #include /* fixup_vdso_exception() */ #include +#include /* struct insn */ +#include /* insn_fetch_from_user(), ... */ #define CREATE_TRACE_POINTS #include @@ -1454,6 +1456,23 @@ void do_user_addr_fault(struct pt_regs *regs, } mmap_read_unlock(mm); + + if (fault & VM_FAULT_SKIP_INSN) { + u8 buf[MAX_INSN_SIZE]; + struct insn insn; + int nr_copied; + + nr_copied = insn_fetch_from_user(regs, buf); + if (nr_copied <= 0) + return; + + if (!insn_decode_from_regs(&insn, regs, buf, nr_copied)) + return; + + regs->ip += insn.length; + return; + } + if (likely(!(fault & VM_FAULT_ERROR))) return; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 3b8475007734..e76ab9ad555c 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -945,6 +945,7 @@ typedef __bitwise unsigned int vm_fault_t; * fsync() to complete (for synchronous page faults * in DAX) * @VM_FAULT_COMPLETED: ->fault completed, meanwhile mmap lock released + * @VM_FAULT_SKIP_INSN: ->handle the fault by skipping faulting instruction * @VM_FAULT_HINDEX_MASK: mask HINDEX value * */ @@ -962,6 +963,7 @@ enum vm_fault_reason { VM_FAULT_DONE_COW = (__force vm_fault_t)0x001000, VM_FAULT_NEEDDSYNC = (__force vm_fault_t)0x002000, VM_FAULT_COMPLETED = (__force vm_fault_t)0x004000, + VM_FAULT_SKIP_INSN = (__force vm_fault_t)0x008000, VM_FAULT_HINDEX_MASK = (__force vm_fault_t)0x0f0000, }; @@ -985,7 +987,8 @@ enum vm_fault_reason { { VM_FAULT_RETRY, "RETRY" }, \ { VM_FAULT_FALLBACK, "FALLBACK" }, \ { VM_FAULT_DONE_COW, "DONE_COW" }, \ - { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" } + { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \ + { VM_FAULT_SKIP_INSN, "SKIP_INSN" } struct vm_special_mapping { const char *name; /* The name, e.g. "[vdso]". */ diff --git a/mm/memory.c b/mm/memory.c index 1ade407ccbf9..62ba9b7b713e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5221,8 +5221,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, lru_gen_exit_fault(); /* If the mapping is droppable, then errors due to OOM aren't fatal. */ - if (vma->vm_flags & VM_DROPPABLE) + if ((ret & VM_FAULT_OOM) && (vma->vm_flags & VM_DROPPABLE)) { ret &= ~VM_FAULT_OOM; + ret |= VM_FAULT_SKIP_INSN; + } if (flags & FAULT_FLAG_USER) { mem_cgroup_exit_user_fault();