From patchwork Fri Nov 25 06:36:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Sun X-Patchwork-Id: 25845 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3806800wrr; Thu, 24 Nov 2022 22:40:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf45NPKDEtHnZXwaRnmoELTNTHbMej8Pm5E5sVuVsi7DvPLxM27J6c5A4cXlrRTdWqLZ8I2H X-Received: by 2002:a17:906:a58:b0:7ad:b45c:dbca with SMTP id x24-20020a1709060a5800b007adb45cdbcamr19085245ejf.388.1669358400600; Thu, 24 Nov 2022 22:40:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669358400; cv=none; d=google.com; s=arc-20160816; b=tsyWx36seIv/qJJ4DxqeMfeUFJX0I4vklkzIxEp/9jmUJVoYsTA351DVniKGbhu6e2 p9tQAeR+J+UGRgt7wTN5sU2sFWOSC/9MsTTZ75eDpTS3hYAz4noct5DgbDsIwEihAPS4 AcQ/fuo2cgvj5B44Mynbw1DwNp7diGD38AmvhPyKHHFWYLBr0IhJHnOhoEPtsklG/Ipo WykAY+sgVopv5k8+tyfzijCI7ckS9O0Huik4BfZq/cXC1GA8nJAb9VJaXQtE6k7nl2fh gB/cblK+f9uWxuwXS1ucaI6RBnfZNz6y3osoHg+yx1kYZoy/cFjXkKCCSuIX50fksbSC i0Bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D3IneC1I9Oj2a9ZQYDCBeO854k07TNWMYUvM3rNqiuM=; b=fnSBSzSFVyaSDZXZvZW4tfMyua0lDrQxuXErMxI2uL+frIIOStejcSqPhf30ybtdn1 tU1WaKRQRN8iTESpl2ukDjj8gGLW02pHQ53XpiaasEyDfIxQaiM+Wcg6nvJwXeKkwReP LPTUWKjSkz4szloqbkSoPSKjW40aoUuB83p+EDF6p+gpIiAxat/4fP0VZIQec7W207Xa t9B40RhNwLe8Z13T0At/zZFFmGwugv6r+At/yDBLeiIROcyaGKZN8lErkhEtMkGf474b Q0dUWXaO8OArMvgoxqz/PHyplyJ1d0Eso0/IMKP+HtY3cu+dfJsdYKSVbBfb4PwkqXAe +5Fw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=VK14oM4+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z3-20020a05640235c300b0046107f9520csi3204185edc.210.2022.11.24.22.39.36; Thu, 24 Nov 2022 22:40:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=VK14oM4+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229704AbiKYGgr (ORCPT + 99 others); Fri, 25 Nov 2022 01:36:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229657AbiKYGgp (ORCPT ); Fri, 25 Nov 2022 01:36:45 -0500 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 589972936B; Thu, 24 Nov 2022 22:36:44 -0800 (PST) Received: by mail-pl1-x629.google.com with SMTP id io19so3163831plb.8; Thu, 24 Nov 2022 22:36:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D3IneC1I9Oj2a9ZQYDCBeO854k07TNWMYUvM3rNqiuM=; b=VK14oM4+0HxyoWy1YXQJ5JbPd1Ap7jYTcgoSfi5NBwTLnheCTgWFLCVzekM6Aj9jh2 89dasAFT0kPib8zurZSr9U2N1h8YmcKUokQgPsz44zMvJAc+CISwgVCH1/5ByxwLh/SB rEMSytQik2Vd4mOwBoCKXUB2MCPPRsY/QIkwtSdW/BCVnYTAl6WRDhZ/GP+O4ZAii/MR QozxJPMwizuEmHXabY5yfe89FgVLjzxKt6Abd7FaWQlE1zawpvA92LUeyl3tVI6da9Qw hMXeSif3QTSXZBksKAKehEaAXxxfA9hM9ERBmEDI1EJMglfiGkphoAB7oJnQn1tPls0T YD1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D3IneC1I9Oj2a9ZQYDCBeO854k07TNWMYUvM3rNqiuM=; b=3JisJIMTnj2ePzRjt5CAO1+YMxN31oZQWplQuDiL+F7yGdDvDX1pkqzHVcVa04GsgO w/4/JC3QhNzctF2PjbAvbt/O2TIxRO+DzYw5mcUwmWf/nShF4i7hqV/Cyunh+HQ5pGFI Am4J62y4GWpRRR/aFk6+tFFkDkV9/OEnmbKtTRtL3Yu7BXXD5+fA99+hBOTPu7xsiPyn nVx7N+xUJIaCvCBKTv6+u8RRX5k+VIbD40QQEtwEGSR1+L/Xmsnv887A1mj3GwEHNBvu pQcyAUnXJ/XV8SO0A/dwW+PtvpeAOK5BMMu1Vw2J4rV/3Tw6J2CwV4WKXxSUQXGdDCdt CCGw== X-Gm-Message-State: ANoB5pnXHBGN+fvaz/a5s59ifUpdF5qeobuUBmsNGJJHc3bTnpQ8qv4T LRzstCQ3dwoV1aQ7/maDCWbK14dahYQ3 X-Received: by 2002:a17:903:2ce:b0:186:e852:b271 with SMTP id s14-20020a17090302ce00b00186e852b271mr17344706plk.117.1669358203461; Thu, 24 Nov 2022 22:36:43 -0800 (PST) Received: from pc.localdomain ([166.111.83.15]) by smtp.gmail.com with ESMTPSA id nm18-20020a17090b19d200b001fd6066284dsm2214891pjb.6.2022.11.24.22.36.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 22:36:43 -0800 (PST) From: Hao Sun To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, davem@davemloft.net, linux-kernel@vger.kernel.org, Hao Sun Subject: [PATCH bpf-next v2 1/3] bpf: Sanitize STX/ST in jited BPF progs with KASAN Date: Fri, 25 Nov 2022 14:36:28 +0800 Message-Id: <20221125063630.536657-2-sunhao.th@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221125063630.536657-1-sunhao.th@gmail.com> References: <20221125063630.536657-1-sunhao.th@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750449154484479699?= X-GMAIL-MSGID: =?utf-8?q?1750449154484479699?= Make the verifier sanitize STX/ST insns in jited BPF programs by dispatching addr to kernel functions that are instrumented by KASAN. Only STX/ST insns that aren't in patches added by other passes using REG_AX or dst_reg isn't R10 are sanitized. The former confilicts with us, the latter are trivial for the verifier to check, skip them to reduce the footprint. The instrumentation is conducted in bpf_misc_fixup(). During it, R0 and R1 are backed up or exchanged with dst_reg, and the addr to check is stored into R1. We extend stack size to backup all the scatch regs, because we don't rely on verifier's knowledge about the calculated stack size and liveness of each regs. And the corresponding bpf_asan_storeN() is inserted before store. The sanitize functions are instrumented with KASAN and they simply write to the target addr for certain bytes, KASAN conducts the actual checking. An extra Kconfig is used to enable this, so normal use case won't be impacted at all. Signed-off-by: Hao Sun --- kernel/bpf/Kconfig | 13 ++++ kernel/bpf/verifier.c | 134 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/kernel/bpf/Kconfig b/kernel/bpf/Kconfig index 2dfe1079f772..d41e1d2d26f1 100644 --- a/kernel/bpf/Kconfig +++ b/kernel/bpf/Kconfig @@ -99,4 +99,17 @@ config BPF_LSM If you are unsure how to answer this question, answer N. +config BPF_PROG_KASAN + bool "Enable BPF Program Address Sanitize" + depends on BPF_JIT_ALWAYS_ON + depends on KASAN + help + Enables instrumentation on LDX/STX/ST insn to capture memory + access errors in BPF programs missed by the verifier. + + The actual check is conducted by KASAN, this feature presents + certain overhead, and should be used mainly by testing purpose. + + If you are unsure how to answer this question, answer N. + endmenu # "BPF subsystem" diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 6599d25dae38..5519c24c5bd4 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15327,6 +15327,25 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, return 0; } +#ifdef CONFIG_BPF_PROG_KASAN + +/* Those are functions instrumented with KASAN for actual sanitizing. */ + +#define BPF_ASAN_STORE(n) \ + notrace u64 bpf_asan_store##n(u##n *addr) \ + { \ + u##n ret = *addr; \ + *addr = ret; \ + return ret; \ + } + +BPF_ASAN_STORE(8); +BPF_ASAN_STORE(16); +BPF_ASAN_STORE(32); +BPF_ASAN_STORE(64); + +#endif + /* Do various post-verification rewrites in a single program pass. * These rewrites simplify JIT and interpreter implementations. */ @@ -15340,7 +15359,12 @@ static int do_misc_fixups(struct bpf_verifier_env *env) const int insn_cnt = prog->len; const struct bpf_map_ops *ops; struct bpf_insn_aux_data *aux; +#ifndef CONFIG_BPF_PROG_KASAN struct bpf_insn insn_buf[16]; +#else + struct bpf_insn insn_buf[32]; + bool in_patch_use_ax = false; +#endif struct bpf_prog *new_prog; struct bpf_map *map_ptr; int i, ret, cnt, delta = 0; @@ -15460,6 +15484,112 @@ static int do_misc_fixups(struct bpf_verifier_env *env) continue; } +#ifdef CONFIG_BPF_PROG_KASAN + +/* With CONFIG_BPF_PROG_KASAN, we extend prog stack to MAX_BPF_STACK + 64 + * to backup scratch regs before calling the sanitize functions, because + * we don't rely on verifier's knowledge about calculated stack size or + * liveness of each reg. + */ +#define __BACKUP_REG(n) \ + *patch++ = BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_##n, -(MAX_BPF_STACK + 8 * n)) +#define BACKUP_SCRATCH_REGS \ + __BACKUP_REG(1); \ + __BACKUP_REG(2); \ + __BACKUP_REG(3); \ + __BACKUP_REG(4); \ + __BACKUP_REG(5) + +#define __RESTORE_REG(n) \ + *patch++ = BPF_LDX_MEM(BPF_DW, BPF_REG_##n, BPF_REG_10, -(MAX_BPF_STACK + 8 * n)) +#define RESTORE_SCRATCH_REGS \ + __RESTORE_REG(1); \ + __RESTORE_REG(2); \ + __RESTORE_REG(3); \ + __RESTORE_REG(4); \ + __RESTORE_REG(5) + + /* Patches that use REG_AX confilict with us, skip it. + * This starts with first use of REG_AX, stops only when + * we see next ldx/stx/st insn with valid aux information. + */ + aux = &env->insn_aux_data[i + delta]; + if (in_patch_use_ax && (int)aux->ptr_type != 0) + in_patch_use_ax = false; + if (insn->dst_reg == BPF_REG_AX || insn->src_reg == BPF_REG_AX) + in_patch_use_ax = true; + + /* Sanitize ST/STX operation. */ + if (BPF_CLASS(insn->code) == BPF_ST || + BPF_CLASS(insn->code) == BPF_STX) { + struct bpf_insn sanitize_fn; + struct bpf_insn *patch = &insn_buf[0]; + + /* Skip st/stx to R10, they're trivial to check. */ + if (in_patch_use_ax || insn->dst_reg == BPF_REG_10 || + BPF_MODE(insn->code) == BPF_NOSPEC) + continue; + + switch (BPF_SIZE(insn->code)) { + case BPF_B: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_store8); + break; + case BPF_H: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_store16); + break; + case BPF_W: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_store32); + break; + case BPF_DW: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_store64); + break; + } + + /* Backup R0 and R1, store `dst + off` to R1, invoke the + * sanitize fn, and then restore each reg. + */ + if (insn->dst_reg == BPF_REG_1) { + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_0); + } else if (insn->dst_reg == BPF_REG_0) { + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_0); + } else { + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, insn->dst_reg); + *patch++ = BPF_MOV64_REG(insn->dst_reg, BPF_REG_0); + } + if (insn->off != 0) + *patch++ = BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, insn->off); + BACKUP_SCRATCH_REGS; + /* Call sanitize fn, R1~R5 are saved to stack during jit. */ + *patch++ = sanitize_fn; + RESTORE_SCRATCH_REGS; + if (insn->off != 0) + *patch++ = BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -insn->off); + if (insn->dst_reg == BPF_REG_1) { + *patch++ = BPF_MOV64_REG(BPF_REG_0, BPF_REG_AX); + } else if (insn->dst_reg == BPF_REG_0) { + *patch++ = BPF_MOV64_REG(BPF_REG_0, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_AX); + } else { + *patch++ = BPF_MOV64_REG(BPF_REG_0, insn->dst_reg); + *patch++ = BPF_MOV64_REG(insn->dst_reg, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_AX); + } + *patch++ = *insn; + cnt = patch - insn_buf; + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta += cnt - 1; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + continue; + } +#endif + if (insn->code != (BPF_JMP | BPF_CALL)) continue; if (insn->src_reg == BPF_PSEUDO_CALL) @@ -15852,6 +15982,10 @@ static int do_misc_fixups(struct bpf_verifier_env *env) } } +#ifdef CONFIG_BPF_PROG_KASAN + prog->aux->stack_depth = MAX_BPF_STACK + 64; +#endif + sort_kfunc_descs_by_imm(env->prog); return 0; From patchwork Fri Nov 25 06:36:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Sun X-Patchwork-Id: 25846 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3806878wrr; Thu, 24 Nov 2022 22:40:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf577Wm6jra/d3TcSM60Wi4ovGh5ivMxY6ZUXRcZq0ZNRuqRv51LiM+t6d9VQqJSzc1oKZIF X-Received: by 2002:a17:906:160b:b0:78d:dddb:3974 with SMTP id m11-20020a170906160b00b0078ddddb3974mr30090590ejd.411.1669358416175; Thu, 24 Nov 2022 22:40:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669358416; cv=none; d=google.com; s=arc-20160816; b=CJ+nNPUVVqlWTrw0zIqX8USHZYfHTb8SYdVtq+6mrtBUI5jMzkV/ZxgaM3TBgKERtR THKaFvvSfRIk21toNJYnX5EyxgY1zQU7COVe7DQlkC9kn1asuhYBX89f2fZ4itFwT4JL cc/V0Hm37J+LjTzAGSiH3aeoZVtQG0yBOph2fI7uPkiznl9G74yFHp8C20vG/yPjcZIA IPMRBpPS7geh6qN3A3oQoexutPOWVUtLdCEeCZKYP1RAwmHyXK8ztrJxkKPk3l5A4F83 trBBl9sEBg7p4QRKl5QT3QiooSi9qEg++ETqaS8s143yP26O+3yoCvClqUSZatRBI4fD EQGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vBJjxQmOrEZrnpEuQrvFjGMPHF9guw+LWjz0eneC8SE=; b=eGjt5ayWsKj+Us4XpYLQhXxGs7omOwRfBwK+ugjcspKwrIuAAhK2OtRjQVOyaWIe3P wrmQ1nhhCBj/pq2Cm0CqX4s8Z5cqFVSZlm0XJlBL7B+XwmBjo2JgoRJA2L26BNVmovYd hc86E+UuwU99l8lDbyk8Lhks7DS2Hz/LIUOjqZL5b/V9SYSbyHOLIgj55AfEJ3ADycVP VtI7ZMHOUpkO3+hFSLe3PeIemEd7+WRAPWW6/z4ZwbcvRd7UOxYXWDLThYWHcGLmVLpQ kn3XEAj969joTEsx1lrTaC7OHMQtgBz5w62rirpLLt5pqAw9VOaH+xOvy7rIb38399rx xYqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=BiH1Nypr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ji11-20020a170907980b00b0078dd22dd569si1813305ejc.121.2022.11.24.22.39.52; Thu, 24 Nov 2022 22:40:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=BiH1Nypr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229602AbiKYGg5 (ORCPT + 99 others); Fri, 25 Nov 2022 01:36:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229712AbiKYGgz (ORCPT ); Fri, 25 Nov 2022 01:36:55 -0500 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C11C329838; Thu, 24 Nov 2022 22:36:48 -0800 (PST) Received: by mail-pg1-x529.google.com with SMTP id b62so3221929pgc.0; Thu, 24 Nov 2022 22:36:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vBJjxQmOrEZrnpEuQrvFjGMPHF9guw+LWjz0eneC8SE=; b=BiH1Nypr9XqG9qII974B13vHBqKSfJOB+OrFzA0+L2WUCatoReNQS/f4d49Ls/4j8t K9Wi7Uu71i6dZawT2IhD2GeJGojOn2UQjAksAJPL88Har5uTiFCAZlmckU7tCR7mPFD+ uTcNFKi/aAu6X7nQc5ooqdDOzKhGdo0vO9xAELTV1pBSPmOfkrCBZTxZbPim5k0kUyuK PrAhc0ZtVjBwwj07CpN12+5kK4lIYcmTgba/RXuwIX3zizYFrTGiIjVt6PtqxY0u8mz5 qPO7qzffSPnfclXmiWO9jAbJh/aDGVMt2UPLXW+InxhvqZZyPMYrOgzs5spH9Jv1q7zP SUrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vBJjxQmOrEZrnpEuQrvFjGMPHF9guw+LWjz0eneC8SE=; b=Dn4FhZdcSRCWPWsI1gYYQ+ctmA4bPtTYaQp0wVnGQyBSHsLOxn49DUc8Fo2DBhePZR 55p9TIL0aoIVeYuxS4+7xbf1IvBWvJQFIOrgIF89ZhvdcenY5djBmHRi1hZx6F3ujYYq TzJ/LXqna/snQLK2tW6+i3ayXI/fZBXssVoZvRAZkXmzTirsg2F9tBLseLZaBFtBmajy frRspPZ0SIJV/LSAHbOt+KBEv7JjRas7sAu6sRLmak8BDR7/kcHH05fTfCVXQo9bzxul 6o5i1OFrNn9Qqi6WisVNtrdg6LH15s3nOnc5BBN8FEVLNVvIpZXw7BuvM8BFsQKXTNBW Aq+Q== X-Gm-Message-State: ANoB5plp/w8R937ABmVaLw4S7X6HGYZ9So5Pm+6f3md68NSE+ZWF1wyS pVqro4vctwwb+giNp9RdhWBqPjHFtcZV X-Received: by 2002:a63:1e62:0:b0:46b:3acb:77b2 with SMTP id p34-20020a631e62000000b0046b3acb77b2mr13934331pgm.560.1669358207936; Thu, 24 Nov 2022 22:36:47 -0800 (PST) Received: from pc.localdomain ([166.111.83.15]) by smtp.gmail.com with ESMTPSA id nm18-20020a17090b19d200b001fd6066284dsm2214891pjb.6.2022.11.24.22.36.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 22:36:47 -0800 (PST) From: Hao Sun To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, davem@davemloft.net, linux-kernel@vger.kernel.org, Hao Sun Subject: [PATCH bpf-next v2 2/3] bpf: Sanitize LDX in jited BPF progs with KASAN Date: Fri, 25 Nov 2022 14:36:29 +0800 Message-Id: <20221125063630.536657-3-sunhao.th@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221125063630.536657-1-sunhao.th@gmail.com> References: <20221125063630.536657-1-sunhao.th@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750449170489700826?= X-GMAIL-MSGID: =?utf-8?q?1750449170489700826?= Make the verifier sanitize LDX insns in jited BPF programs. The dst_reg and AX are free here, different insns that backup R0&R1 are inserted based on their relationships with dst_reg and src, all the scratch regs are backed up to extended stack space before calling the checking functions. Finally, the checking funcs are inserted, and regs are restored. Signed-off-by: Hao Sun --- kernel/bpf/verifier.c | 90 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5519c24c5bd4..4e253fc20bf2 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15344,6 +15344,17 @@ BPF_ASAN_STORE(16); BPF_ASAN_STORE(32); BPF_ASAN_STORE(64); +#define BPF_ASAN_LOAD(n) \ + notrace u64 bpf_asan_load##n(u##n *addr) \ + { \ + return *addr; \ + } + +BPF_ASAN_LOAD(8); +BPF_ASAN_LOAD(16); +BPF_ASAN_LOAD(32); +BPF_ASAN_LOAD(64); + #endif /* Do various post-verification rewrites in a single program pass. @@ -15588,6 +15599,85 @@ static int do_misc_fixups(struct bpf_verifier_env *env) insn = new_prog->insnsi + i + delta; continue; } + + /* Sanitize LDX operation*/ + if (BPF_CLASS(insn->code) == BPF_LDX) { + struct bpf_insn sanitize_fn; + struct bpf_insn *patch = &insn_buf[0]; + bool dst_is_r0 = insn->dst_reg == BPF_REG_0; + bool dst_is_r1 = insn->dst_reg == BPF_REG_1; + + if (in_patch_use_ax || insn->src_reg == BPF_REG_10) + continue; + + switch (BPF_SIZE(insn->code)) { + case BPF_B: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_load8); + break; + case BPF_H: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_load16); + break; + case BPF_W: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_load32); + break; + case BPF_DW: + sanitize_fn = BPF_EMIT_CALL(bpf_asan_load64); + break; + } + + /* Backup R0 and R1, REG_AX and dst_reg are free. */ + if (insn->src_reg == BPF_REG_1) { + if (!dst_is_r0) + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_0); + } else if (insn->src_reg == BPF_REG_0) { + if (!dst_is_r1) + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_0); + } else if (!dst_is_r1) { + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, insn->src_reg); + if (!dst_is_r0) + *patch++ = BPF_MOV64_REG(insn->dst_reg, BPF_REG_0); + } else { + *patch++ = BPF_MOV64_REG(BPF_REG_1, insn->src_reg); + *patch++ = BPF_MOV64_REG(BPF_REG_AX, BPF_REG_0); + } + if (insn->off != 0) + *patch++ = BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, insn->off); + BACKUP_SCRATCH_REGS; + /* Invoke sanitize fn, R1~R5 are stored to stack during jit. */ + *patch++ = sanitize_fn; + RESTORE_SCRATCH_REGS; + if (insn->off != 0) + *patch++ = BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -insn->off); + if (insn->src_reg == BPF_REG_1) { + if (!dst_is_r0) + *patch++ = BPF_MOV64_REG(BPF_REG_0, BPF_REG_AX); + } else if (insn->src_reg == BPF_REG_0) { + *patch++ = BPF_MOV64_REG(BPF_REG_0, BPF_REG_1); + if (!dst_is_r1) + *patch++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_AX); + } else if (!dst_is_r1) { + if (!dst_is_r0) + *patch++ = BPF_MOV64_REG(BPF_REG_0, insn->dst_reg); + if (insn->src_reg == insn->dst_reg) + *patch++ = BPF_MOV64_REG(insn->src_reg, BPF_REG_1); + *patch++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_AX); + } else { + *patch++ = BPF_MOV64_REG(BPF_REG_0, BPF_REG_AX); + } + *patch++ = *insn; + cnt = patch - insn_buf; + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta += cnt - 1; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + continue; + } #endif if (insn->code != (BPF_JMP | BPF_CALL)) From patchwork Fri Nov 25 06:36:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Sun X-Patchwork-Id: 25848 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3807182wrr; Thu, 24 Nov 2022 22:41:01 -0800 (PST) X-Google-Smtp-Source: AA0mqf4WxT7hjTkdc0r91mH0EnUGB7DS4b0klEpvCJ5jwA0Qr7JGSC/SIm+1pCxA/K0dlLaqX6tN X-Received: by 2002:a05:6402:707:b0:467:6035:285c with SMTP id w7-20020a056402070700b004676035285cmr33302559edx.386.1669358461749; Thu, 24 Nov 2022 22:41:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669358461; cv=none; d=google.com; s=arc-20160816; b=1DtWAztOpBj4OO8VSSrCkv5JmZ8pNdqV9uIOtkiUbhuuQywmrJZC+Lz18qsBD6uZw/ FYrQzNNWG7DmaTIxmP3b1imktffQrUTEW0GMsn1GlJWJx9VD0zTM/J/acuTf9rLAw6gP gm5Od0uQ2+8alrbrHZSSFckXbZjzuDY4DydsZ/9wBNLNOaIgyHo+wo7jVN/UjXLPdC7t 5S70BM+eY2O/oS/RHCDOGvFeiLAu8Xw8ibpwuVwhAUWZU72vsF/1geK6UUspXK3pht65 HAi6e+o6zgILIvopmV1iLPMA3bdpSBaG3Qqy4e4zKKVpNDKN0VoOO1m79j38+vq0YAEP Ud2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BgKC00kdEHyZ2BMxISGOkS8d2W0npo/wh8nj7cf1w34=; b=whul9KCVchfpLwXa1CB5TOEHs5z2imbzTDY0Kq3AEqPpZjQBNkiEBTfKDLP+TbELeh 7B+pUSe+58u3RbXGsX+FzWXYi09pH/kBLESoGAUH7JJIDSpiYdFhlqNMkF+jSMIStbSK /WNU4v36Jnx5KX+yvPDqB1DMugwnLa/AZWpG+lHauurTdXY6MTFidxl7xn+S9g7OJoTu 2bjEdngL1KyWpLfS5nrOfjNg7Sm2Z2xnwtALawjXvGIOCsJJXjxj43wSncp3MQpkBvtr ceO0/EWtUVNxvWlOTs6qJBhFkgwd5oGzGZivf2G6uCdH7GQLvz4y5yNjQaUPHtelQzCe T6cA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=a12nDl9D; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qb21-20020a1709077e9500b007919dbd3770si1596521ejc.485.2022.11.24.22.40.38; Thu, 24 Nov 2022 22:41:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=a12nDl9D; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229476AbiKYGhI (ORCPT + 99 others); Fri, 25 Nov 2022 01:37:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229787AbiKYGg6 (ORCPT ); Fri, 25 Nov 2022 01:36:58 -0500 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8362D2A94D; Thu, 24 Nov 2022 22:36:54 -0800 (PST) Received: by mail-pg1-x52c.google.com with SMTP id q1so3167316pgl.11; Thu, 24 Nov 2022 22:36:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BgKC00kdEHyZ2BMxISGOkS8d2W0npo/wh8nj7cf1w34=; b=a12nDl9D1CbPueV4YzaLMBHFl44tEKfpy19bep699O5HacSnAmWdbesFzgbz5gCiIq KlreKoUI/BMhrGlKscdidLDnfAh75owtsIWG5OeZCiJzJWJMiwQJBZBt9Ef1uxjvGWSm xhkmfJJZ9cH0XF1xbY9PPO89MtY+Uaj8NwaUAhbYU5g5BcAy1oySr1Syj7lGefy8jnmj 5wzbkYpDO16NyKbydsuw84VaQKCfl+DnTJTI6arNvhPVc6ZVHB86ecqAtD6EBdud61iw i8Unw39nlRcvhwrUfSZiW2zomgs7DF+C//JqdKhTDbYslAnLrq/4Iy61qXpKUU8tgAKh nDZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BgKC00kdEHyZ2BMxISGOkS8d2W0npo/wh8nj7cf1w34=; b=uAprJR9/x2no7FYQL6iYPtNQKLVJxZtdUPPeMg+kBWGXUHTl/95PW599hcILbqvRuH fiVYG/FEe23U4nT8H8Fa7WTuVjg6bD6tFXgC3kLefUuDhz5O4XtTXAXxQ0TOG5xsRN80 bw5ViY170Me9nXIQS20OhsERjig89PXmwmXUwFm889r5JYo6TQBsLMzZyg9dW2q8uh+x QoMMPSTn2sLA8g7PxShp4DUGapI/j3jMPTMhMfl+1ZXMs5/CBoxf0kHY4othbk8GaNnb UYlqan8jilZCk1oqh/XV3rTtvSesGnTfUpWhoOW2sKWwPWTcTZyEs109UpD/COit3EZx 0IFA== X-Gm-Message-State: ANoB5plm8iZYRhfNc3JtO17P/ljyEgxQrKJeuLRGJQ3HZZOd5ibUf+eG GtBi/wrvMC6m5o0KWk9UrFRDUJ9sOjtp X-Received: by 2002:a63:711e:0:b0:477:5654:c37e with SMTP id m30-20020a63711e000000b004775654c37emr23222471pgc.206.1669358213619; Thu, 24 Nov 2022 22:36:53 -0800 (PST) Received: from pc.localdomain ([166.111.83.15]) by smtp.gmail.com with ESMTPSA id nm18-20020a17090b19d200b001fd6066284dsm2214891pjb.6.2022.11.24.22.36.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 22:36:52 -0800 (PST) From: Hao Sun To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, davem@davemloft.net, linux-kernel@vger.kernel.org, Hao Sun Subject: [PATCH bpf-next v2 3/3] selftests/bpf: Add tests for LDX/STX/ST sanitize Date: Fri, 25 Nov 2022 14:36:30 +0800 Message-Id: <20221125063630.536657-4-sunhao.th@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221125063630.536657-1-sunhao.th@gmail.com> References: <20221125063630.536657-1-sunhao.th@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,UPPERCASE_50_75 autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750449218545814878?= X-GMAIL-MSGID: =?utf-8?q?1750449218545814878?= Add tests for LDX/STX/ST instrumentation in each possible case. Four cases for STX/ST, include dst_reg equals to R0, R1, R10, other regs, respectively, ten cases for LDX. All new/existing selftests can pass. A slab-out-of-bounds read report is also availble, which is achieved by exploiting CVE-2022-23222 and can be reproduced in Linux v5.10: https://pastebin.com/raw/Ee1Cw492. Signed-off-by: Hao Sun --- .../selftests/bpf/verifier/sanitize_st_ldx.c | 362 ++++++++++++++++++ 1 file changed, 362 insertions(+) create mode 100644 tools/testing/selftests/bpf/verifier/sanitize_st_ldx.c diff --git a/tools/testing/selftests/bpf/verifier/sanitize_st_ldx.c b/tools/testing/selftests/bpf/verifier/sanitize_st_ldx.c new file mode 100644 index 000000000000..1db0d1794f29 --- /dev/null +++ b/tools/testing/selftests/bpf/verifier/sanitize_st_ldx.c @@ -0,0 +1,362 @@ +#ifdef CONFIG_BPF_PROG_KASAN + +#define __BACKUP_REG(n) \ + BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_##n, INSN_OFF_MASK) +#define BACKUP_SCRATCH_REGS \ + __BACKUP_REG(1), __BACKUP_REG(2), __BACKUP_REG(3), __BACKUP_REG(4), \ + __BACKUP_REG(5) + +#define __RESTORE_REG(n) \ + BPF_LDX_MEM(BPF_DW, BPF_REG_##n, BPF_REG_10, INSN_OFF_MASK) +#define RESTORE_SCRATCH_REGS \ + __RESTORE_REG(1), __RESTORE_REG(2), __RESTORE_REG(3), \ + __RESTORE_REG(4), __RESTORE_REG(5) + +{ + "sanitize stx: dst is R1", + .insns = { + BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_1, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 1, 1), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, MAX_BPF_REG), + BPF_ST_MEM(BPF_DW, BPF_REG_1, -8, 1), + }, +}, +{ + "sanitize stx: dst is R0", + .insns = { + BPF_MOV64_REG(BPF_REG_0, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_0, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_ST_MEM(BPF_DW, BPF_REG_0, -8, 1), + }, +}, +{ + "sanitize stx: dst is R10", + .insns = { + BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 1, 1), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .unexpected_insns = { + BPF_EMIT_CALL(INSN_IMM_MASK), + }, +}, +{ + "sanitize stx: dst is other regs", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_2, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 1, 1), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_2), + BPF_MOV64_REG(BPF_REG_2, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_2), + BPF_MOV64_REG(BPF_REG_2, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_ST_MEM(BPF_DW, BPF_REG_2, -8, 1), + }, +}, +{ + "sanitize ldx: src is R1, dst is R0", + .insns = { + BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_1, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 1, 1), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8), + }, +}, +{ + "sanitize ldx: src is R1, dst is R1", + .insns = { + BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_1, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, -8), + }, +}, +{ + "sanitize ldx: src is R1, dst is other regs", + .insns = { + BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_1, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -8), + }, +}, +{ + "sanitize ldx: src is R0, dst is R1", + .insns = { + BPF_MOV64_REG(BPF_REG_0, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_0, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, -8), + }, +}, +{ + "sanitize ldx: src is R0, dst is R0", + .insns = { + BPF_MOV64_REG(BPF_REG_0, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_0, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 1, 1), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, -8), + }, +}, +{ + "sanitize ldx: src is R0, dst is other regs", + .insns = { + BPF_MOV64_REG(BPF_REG_0, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_0, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_0, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_0, -8), + }, +}, +{ + "sanitize ldx: src is other regs, dst is R0", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_2, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 1, 1), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_2), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, -8), + }, +}, +{ + "sanitize ldx: src is other regs, dst is R1", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_2, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_1), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(BPF_REG_1, BPF_REG_2), + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -8), + }, +}, +{ + "sanitize ldx: src is other regs, dst is self", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_2, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_2, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_2), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_2), + BPF_MOV64_REG(BPF_REG_2, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_2), + BPF_MOV64_REG(BPF_REG_2, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_2, -8), + }, +}, +{ + "sanitize ldx: src is other regs, dst is other regs", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_ST_MEM(BPF_DW, BPF_REG_2, -8, 1), + BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_2, -8), + BPF_JMP_IMM(BPF_JEQ, BPF_REG_3, 1, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_3), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .retval = 1, + .expected_insns = { + BPF_MOV64_REG(MAX_BPF_REG, BPF_REG_1), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_2), + BPF_MOV64_REG(BPF_REG_3, BPF_REG_0), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BACKUP_SCRATCH_REGS, + BPF_EMIT_CALL(INSN_IMM_MASK), + RESTORE_SCRATCH_REGS, + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8), + BPF_MOV64_REG(BPF_REG_0, BPF_REG_3), + BPF_MOV64_REG(BPF_REG_1, MAX_BPF_REG), + BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_2, -8), + }, +}, +#endif /* CONFIG_BPF_PROG_KASAN */