From patchwork Fri Jun 9 09:56:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Menglong Dong X-Patchwork-Id: 105522 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp848636vqr; Fri, 9 Jun 2023 03:35:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4HGcy2yDD/q5PIgvefd9gGJmfOBFyqbr9OBi1anvnQy2cPI3VleeUOjzb0XADI1zn2Crv8 X-Received: by 2002:a05:6a21:78a7:b0:10b:764b:a942 with SMTP id bf39-20020a056a2178a700b0010b764ba942mr5335099pzc.11.1686306955269; Fri, 09 Jun 2023 03:35:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686306955; cv=none; d=google.com; s=arc-20160816; b=VU4H74Cooup5M+q3G4kRM32EiXagjX/BhaRcxsAAjzWg34h8LmdYC9D8oKDTi54D0c TW9ilQXtb2ivu7b5X81RIUUNRHbkCsJyeQKdRYCN2MRqJer1oXVUJLDhLVivpEzUeBRC aAQsHC2QMG3FNuSO8TZzyZ3x4fpRrC6yzcYw98773SMk30Tq9YJ2YnSQ1+s0AmLcc7sb wq1Ghn/hY5i3Pg2x/Q0dG63PQOv/w/jaFmFN3Eb/MLW04MRbytV0t3KGORFwCJhq5X3/ NcpMLHF/FljxQNokxQiCKG+w6jEvGjWQVpUqeLp9QFC8WQjqE5C8DoB3sRgpSrFMliWx 8GjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Crp08FOLgFvZnpAjUWNQfC7YZ3uDv9qVkOafGIbXOmc=; b=rgKj0nd6dvHYCJ3C58rA6bd00qPjfO1EoIla/7VlMStDMCqBPlI5Q/sUIWdyvy85V7 V/EOV2STIE1Siz9CG2LozobRzd9wgQpgpKJl7oQxsqYK1vZEL4nsqbMY+jWjOACkOLd4 ud3sf9OTVQ5VTvL6PFpaxHrvRHNJjYn8dZ9aqITmdkBZa1oA7ZWD5D+NGtT32qgjT2gq /2DHrul4ad6GgscNWABmLTwySSUpQO4v6nlRVdnnG7IPWkLymIGdtUpEfNaReMFcHuXz 6azG+J9YFr0flsroelbz2QhL4KLSQGBFL/mtKD+DOUNGH9VkXfbMi+vCDxHYqPu/TSwu hYrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=Smb8m1uI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u69-20020a638548000000b00543f2f69f14si2449110pgd.160.2023.06.09.03.35.40; Fri, 09 Jun 2023 03:35:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=Smb8m1uI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241705AbjFIKHQ (ORCPT + 99 others); Fri, 9 Jun 2023 06:07:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240768AbjFIKGe (ORCPT ); Fri, 9 Jun 2023 06:06:34 -0400 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECEE17AB5; Fri, 9 Jun 2023 02:57:06 -0700 (PDT) Received: by mail-pf1-x443.google.com with SMTP id d2e1a72fcca58-653bed78635so1172530b3a.0; Fri, 09 Jun 2023 02:57:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304626; x=1688896626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Crp08FOLgFvZnpAjUWNQfC7YZ3uDv9qVkOafGIbXOmc=; b=Smb8m1uIPGvu+9/rI1RV7KsRjkbz3obLfvopVGmch4HO34+2L5cVmeqUvwT14H9XJ0 ppMoPMoGIgUMmErqqQVf/X4hV3gXp7w5ZazODxV3j/71+LI1JhIRM6VQtqDNAB9HQocr mK6woCwCJqUykOdrHqVgTg0X31rCzfZjQ0IOxOWarwNpReZZarn7sG3E3yTe+TntYSLu K2Lkk1QisQh9bTbma2hc4QzLi5Wf9+AgOTtiuml1GXpzsrmpsimuF00QSs45YNtJRpWp SvSsXFxVAIyiCyDEzZK5ZGyL4YZPuXyBBDhE+8zti3grnsImeIhyt8wKJf+1mj+c5SEk yibQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304626; x=1688896626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Crp08FOLgFvZnpAjUWNQfC7YZ3uDv9qVkOafGIbXOmc=; b=I75IBPRVJrxP9c2RK4V68VQwj9Tvm1iLSUWww9AllrvGIvykR2tMe+tUSYnJKNBIna OpDhpyzpnsiaCw87L8sMiU7p1vn8pLL9B2NvQeb15hmJfTQr7gZibDYUEXj4H4tUQ0p6 eyiczDODkpNJJCMTS6zkX465i7Yw2Fv8TDMxMLv8f91/0c7BbWUOV4Ndke7j1xLBENPg MpxtBGARAgXwF37yrnG+3A7lw7rf2vlQx5iFVlMOM7cNXPGl/iBfRAToNZz6RwlmfBb7 gmqkR119ByiEsypqPaX+m1mVd6X1EEZt2gH5AvDaSrgNLbUYVOOr+nVWnuDOZhtpN3MN r87w== X-Gm-Message-State: AC+VfDwYvBzVEU+pG8kSV6N+fo4sl2ZbegPZh51iimIWvSJxh/xZSRra Mzcw1KyEaorE8yMOLiKJeao= X-Received: by 2002:a05:6a21:99a0:b0:117:4d57:d3ad with SMTP id ve32-20020a056a2199a000b001174d57d3admr5350154pzb.3.1686304626199; Fri, 09 Jun 2023 02:57:06 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.114]) by smtp.gmail.com with ESMTPSA id 26-20020a63175a000000b0053f3797fc4asm2603369pgx.0.2023.06.09.02.57.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 02:57:05 -0700 (PDT) From: menglong8.dong@gmail.com X-Google-Original-From: imagedong@tencent.com To: andrii.nakryiko@gmail.com, alan.maguire@oracle.com Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Menglong Dong Subject: [PATCH bpf-next v4 1/3] bpf, x86: allow function arguments up to 12 for TRACING Date: Fri, 9 Jun 2023 17:56:51 +0800 Message-Id: <20230609095653.1406173-2-imagedong@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230609095653.1406173-1-imagedong@tencent.com> References: <20230609095653.1406173-1-imagedong@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768220710852289931?= X-GMAIL-MSGID: =?utf-8?q?1768221001665704085?= From: Menglong Dong For now, the BPF program of type BPF_PROG_TYPE_TRACING can only be used on the kernel functions whose arguments count less than 6. This is not friendly at all, as too many functions have arguments count more than 6. According to the current kernel version, below is a statistics of the function arguments count: argument count | function count 7 | 704 8 | 270 9 | 84 10 | 47 11 | 47 12 | 27 13 | 22 14 | 5 15 | 0 16 | 1 Therefore, let's enhance it by increasing the function arguments count allowed in arch_prepare_bpf_trampoline(), for now, only x86_64. For the case that we don't need to call origin function, which means without BPF_TRAMP_F_CALL_ORIG, we need only copy the function arguments that stored in the frame of the caller to current frame. The arguments of arg6-argN are stored in "$rbp + 0x18", we need copy them to "$rbp - regs_off + (6 * 8)". For the case with BPF_TRAMP_F_CALL_ORIG, we need prepare the arguments in stack before call origin function, which means we need alloc extra "8 * (arg_count - 6)" memory in the top of the stack. Note, there should not be any data be pushed to the stack before call the origin function. Then, we have to store rbx with 'mov' instead of 'push'. We use EMIT3_off32() or EMIT4() for "lea" and "sub". The range of the imm in "lea" and "sub" is [-128, 127] if EMIT4() is used. Therefore, we use EMIT3_off32() instead if the imm out of the range. It works well for the FENTRY/FEXIT/MODIFY_RETURN, I'm not sure if there are other complicated cases. Signed-off-by: Menglong Dong --- v4: - make the stack 16-byte aligned if passing args on-stack is needed - add the function arguments statistics to the commit log v3: - use EMIT3_off32() for "lea" and "sub" only on necessary - make 12 as the maximum arguments count v2: - instead EMIT4 with EMIT3_off32 for "lea" to prevent overflow - make MAX_BPF_FUNC_ARGS as the maximum argument count --- arch/x86/net/bpf_jit_comp.c | 125 ++++++++++++++++++++++++++++++++---- 1 file changed, 111 insertions(+), 14 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 1056bbf55b17..a767e13c8c85 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1868,7 +1868,7 @@ static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, * mov QWORD PTR [rbp-0x10],rdi * mov QWORD PTR [rbp-0x8],rsi */ - for (i = 0, j = 0; i < min(nr_regs, 6); i++) { + for (i = 0, j = 0; i < min(nr_regs, MAX_BPF_FUNC_ARGS); i++) { /* The arg_size is at most 16 bytes, enforced by the verifier. */ arg_size = m->arg_size[j]; if (arg_size > 8) { @@ -1876,10 +1876,31 @@ static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, next_same_struct = !next_same_struct; } - emit_stx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_FP, - i == 5 ? X86_REG_R9 : BPF_REG_1 + i, - -(stack_size - i * 8)); + if (i <= 5) { + /* copy function arguments from regs into stack */ + emit_stx(prog, bytes_to_bpf_size(arg_size), + BPF_REG_FP, + i == 5 ? X86_REG_R9 : BPF_REG_1 + i, + -(stack_size - i * 8)); + } else { + /* copy function arguments from origin stack frame + * into current stack frame. + * + * The starting address of the arguments on-stack + * is: + * rbp + 8(push rbp) + + * 8(return addr of origin call) + + * 8(return addr of the caller) + * which means: rbp + 24 + */ + emit_ldx(prog, bytes_to_bpf_size(arg_size), + BPF_REG_0, BPF_REG_FP, + (i - 6) * 8 + 0x18); + emit_stx(prog, bytes_to_bpf_size(arg_size), + BPF_REG_FP, + BPF_REG_0, + -(stack_size - i * 8)); + } j = next_same_struct ? j : j + 1; } @@ -1913,6 +1934,41 @@ static void restore_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, } } +static void prepare_origin_stack(const struct btf_func_model *m, u8 **prog, + int nr_regs, int stack_size) +{ + int i, j, arg_size; + bool next_same_struct = false; + + if (nr_regs <= 6) + return; + + /* Prepare the function arguments in stack before call origin + * function. These arguments must be stored in the top of the + * stack. + */ + for (i = 0, j = 0; i < min(nr_regs, MAX_BPF_FUNC_ARGS); i++) { + /* The arg_size is at most 16 bytes, enforced by the verifier. */ + arg_size = m->arg_size[j]; + if (arg_size > 8) { + arg_size = 8; + next_same_struct = !next_same_struct; + } + + if (i > 5) { + emit_ldx(prog, bytes_to_bpf_size(arg_size), + BPF_REG_0, BPF_REG_FP, + (i - 6) * 8 + 0x18); + emit_stx(prog, bytes_to_bpf_size(arg_size), + BPF_REG_FP, + BPF_REG_0, + -(stack_size - (i - 6) * 8)); + } + + j = next_same_struct ? j : j + 1; + } +} + static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, struct bpf_tramp_link *l, int stack_size, int run_ctx_off, bool save_ret) @@ -1938,7 +1994,10 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, /* arg1: mov rdi, progs[i] */ emit_mov_imm64(&prog, BPF_REG_1, (long) p >> 32, (u32) (long) p); /* arg2: lea rsi, [rbp - ctx_cookie_off] */ - EMIT4(0x48, 0x8D, 0x75, -run_ctx_off); + if (!is_imm8(-run_ctx_off)) + EMIT3_off32(0x48, 0x8D, 0xB5, -run_ctx_off); + else + EMIT4(0x48, 0x8D, 0x75, -run_ctx_off); if (emit_rsb_call(&prog, bpf_trampoline_enter(p), prog)) return -EINVAL; @@ -1954,7 +2013,10 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, emit_nops(&prog, 2); /* arg1: lea rdi, [rbp - stack_size] */ - EMIT4(0x48, 0x8D, 0x7D, -stack_size); + if (!is_imm8(-stack_size)) + EMIT3_off32(0x48, 0x8D, 0xBD, -stack_size); + else + EMIT4(0x48, 0x8D, 0x7D, -stack_size); /* arg2: progs[i]->insnsi for interpreter */ if (!p->jited) emit_mov_imm64(&prog, BPF_REG_2, @@ -1984,7 +2046,10 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, /* arg2: mov rsi, rbx <- start time in nsec */ emit_mov_reg(&prog, true, BPF_REG_2, BPF_REG_6); /* arg3: lea rdx, [rbp - run_ctx_off] */ - EMIT4(0x48, 0x8D, 0x55, -run_ctx_off); + if (!is_imm8(-run_ctx_off)) + EMIT3_off32(0x48, 0x8D, 0x95, -run_ctx_off); + else + EMIT4(0x48, 0x8D, 0x55, -run_ctx_off); if (emit_rsb_call(&prog, bpf_trampoline_exit(p), prog)) return -EINVAL; @@ -2136,7 +2201,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i void *func_addr) { int i, ret, nr_regs = m->nr_args, stack_size = 0; - int regs_off, nregs_off, ip_off, run_ctx_off; + int regs_off, nregs_off, ip_off, run_ctx_off, arg_stack_off, rbx_off; struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; @@ -2150,8 +2215,10 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (m->arg_flags[i] & BTF_FMODEL_STRUCT_ARG) nr_regs += (m->arg_size[i] + 7) / 8 - 1; - /* x86-64 supports up to 6 arguments. 7+ can be added in the future */ - if (nr_regs > 6) + /* x86-64 supports up to MAX_BPF_FUNC_ARGS arguments. 1-6 + * are passed through regs, the remains are through stack. + */ + if (nr_regs > MAX_BPF_FUNC_ARGS) return -ENOTSUPP; /* Generated trampoline stack layout: @@ -2170,7 +2237,14 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i * * RBP - ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag * + * RBP - rbx_off [ rbx value ] always + * * RBP - run_ctx_off [ bpf_tramp_run_ctx ] + * + * [ stack_argN ] BPF_TRAMP_F_CALL_ORIG + * [ ... ] + * [ stack_arg2 ] + * RBP - arg_stack_off [ stack_arg1 ] */ /* room for return value of orig_call or fentry prog */ @@ -2190,9 +2264,25 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i ip_off = stack_size; + stack_size += 8; + rbx_off = stack_size; + stack_size += (sizeof(struct bpf_tramp_run_ctx) + 7) & ~0x7; run_ctx_off = stack_size; + if (nr_regs > 6 && (flags & BPF_TRAMP_F_CALL_ORIG)) { + stack_size += (nr_regs - 6) * 8; + /* make sure the stack pointer is 16-byte aligned if we + * need pass arguments on stack, which means + * [stack_size + 8(rbp) + 8(rip) + 8(origin rip)] + * should be 16-byte aligned. Following code depend on + * that stack_size is already 8-byte aligned. + */ + stack_size += (stack_size % 16) ? 0 : 8; + } + + arg_stack_off = stack_size; + if (flags & BPF_TRAMP_F_SKIP_FRAME) { /* skip patched call instruction and point orig_call to actual * body of the kernel function. @@ -2212,8 +2302,14 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i x86_call_depth_emit_accounting(&prog, NULL); EMIT1(0x55); /* push rbp */ EMIT3(0x48, 0x89, 0xE5); /* mov rbp, rsp */ - EMIT4(0x48, 0x83, 0xEC, stack_size); /* sub rsp, stack_size */ - EMIT1(0x53); /* push rbx */ + if (!is_imm8(stack_size)) + /* sub rsp, stack_size */ + EMIT3_off32(0x48, 0x81, 0xEC, stack_size); + else + /* sub rsp, stack_size */ + EMIT4(0x48, 0x83, 0xEC, stack_size); + /* mov QWORD PTR [rbp - rbx_off], rbx */ + emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_6, -rbx_off); /* Store number of argument registers of the traced function: * mov rax, nr_regs @@ -2262,6 +2358,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (flags & BPF_TRAMP_F_CALL_ORIG) { restore_regs(m, &prog, nr_regs, regs_off); + prepare_origin_stack(m, &prog, nr_regs, arg_stack_off); if (flags & BPF_TRAMP_F_ORIG_STACK) { emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); @@ -2321,7 +2418,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (save_ret) emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, -8); - EMIT1(0x5B); /* pop rbx */ + emit_ldx(&prog, BPF_DW, BPF_REG_6, BPF_REG_FP, -rbx_off); EMIT1(0xC9); /* leave */ if (flags & BPF_TRAMP_F_SKIP_FRAME) /* skip our return address and return to parent */ From patchwork Fri Jun 9 09:56:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Menglong Dong X-Patchwork-Id: 105535 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp851204vqr; Fri, 9 Jun 2023 03:41:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6fB5pQpKRGBA003qXEswv5tlRKm8kMe1moo0Up7/cLOK6Q30lgBZF7/UsdWcdDJFIu6dSh X-Received: by 2002:a17:902:e5cc:b0:1ab:7fb:aac1 with SMTP id u12-20020a170902e5cc00b001ab07fbaac1mr1141159plf.24.1686307307375; Fri, 09 Jun 2023 03:41:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686307307; cv=none; d=google.com; s=arc-20160816; b=bj3tQXQEayu/sMtqypWe7XLD/fly8xTvUfU6WO3k7QYQjnWQ55VyaEU14SE4hq/EvO RlfCtGeZHsK7baPqG9GR4lKqDoiDj5jzXAXFxT4Pw1S3E3HMIrttAfsSfcy9HXFBL1Vr 77fHvTjXVNX7m1B3qudzgChavstHcKfdYRlxQI0JRDMeSTV+2NCxdMBwAeHjZ6IroJB7 BgKe6lUvY7lKqJoYjdBDa/QfTA5L8U+6pnAF92+SsVOb5J0NW44YH/px8oHPKY6fu+Hz 3WRIEDbQXUGbvhQ9bc57Kc/g/uzHiASbuVQNoeKTBos9IN5DZ2BNFNVjNPGyoN0Xi2Gi nRvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=odT6+rgQqp77JJ8LV+JNNotDPvzRm1aU2ce2tTNslKg=; b=wAykY3bLUscpJk1qPxm8jyiJoBbJC1l+Txa3/Tl3dHjOIJbTSSn7gCW85mbCAdIUTa C3dB+OYeHJAza7WOsWS9j156sitBjgZ8n5LAIudgydizYHPOZYXjHq7VVdaApNbgwW1+ jXVcwu7HMi3rvsj3qfASqv1+AY1aHgyNB/htBIv7N7q7p7AAg+flurNrXOi0sMCnA9rq 6Z39Qk8XRvV2ViXfw9shn4i0jBwJPxAUWNMZEc2SIeCFe0LMr8KkFjel6PxRv/m1Gl1P f/4P2BEu8A5QmzppZ9ryvZcJUJ8DDi7eReIAAgR+jC1Ab8e/m/+9Lo0ecOUJEphy2Ms/ SYRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=nGtuEwHE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z8-20020a170903018800b001b1be1317a1si2767505plg.219.2023.06.09.03.41.34; Fri, 09 Jun 2023 03:41:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=nGtuEwHE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239104AbjFIKHX (ORCPT + 99 others); Fri, 9 Jun 2023 06:07:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241979AbjFIKGg (ORCPT ); Fri, 9 Jun 2023 06:06:36 -0400 Received: from mail-oo1-xc44.google.com (mail-oo1-xc44.google.com [IPv6:2607:f8b0:4864:20::c44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D2CB7AB6; Fri, 9 Jun 2023 02:57:11 -0700 (PDT) Received: by mail-oo1-xc44.google.com with SMTP id 006d021491bc7-559409cc490so1019905eaf.3; Fri, 09 Jun 2023 02:57:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304630; x=1688896630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=odT6+rgQqp77JJ8LV+JNNotDPvzRm1aU2ce2tTNslKg=; b=nGtuEwHEkJJ/K9Cxr9t+VLarD+KhlwiVqpwzlWVzTAtiotn2+8vMZSaYKUGQ3Bwyx7 Y9HXa/bGqMSFbWAznCM18RPQNpuYASmEx+jUjWwQOLRDY2SiUMQnWMhTHSPghgjzO2s/ ftrAIH2TEv84ZfMI+/6QdFdbyT4X5USxzOtilJnJxQ3AOVGLjqLSddP4Gf2PY71jR6Br FDc8jCjbW97PHyUWGX2WO4TDf8H9QK0yiT5a8sshAjMn7X6rn8ek0wH/huLwDCfGi7Aj SSk8iFI/YMrWibMVjD/0ke9RRhkt/V9oVk1PwOtARSNR85znARMLxESZtYJCXt/JUEuX slRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304630; x=1688896630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=odT6+rgQqp77JJ8LV+JNNotDPvzRm1aU2ce2tTNslKg=; b=PdLK/2asvBp30kivTrnNsr7qJa84uL9mUEg9W5yV6kIGObZAVC88eHOFXTRJy/bNTY QyI/WYPra+56NIcy6aG1MRpj0aUxZwLet5ig9VgdaJ7kO2yv8vxFgyS4PD/jMBK/xfSN nv2bYnWtV/9C1ch4zJVqp9TylS4N3DIlS9gZTfXM7oBhCN4MDLZs54WFdMMXgdAxi7bg ua8x3PgFHWSMBxszLGULysjneOWF/xAyT5pbJNnDEpbYv12l3PyydfJGgZBUvv99YAto bNfh0UOG6zLZG8Pk9j1w2IEVhz4LA8QNOrbljmpl81xd1ScZDr4VBvheKIWJ0tsf78od pFNg== X-Gm-Message-State: AC+VfDyV+E6KqrCfiR3lWbJVe0WXzMJ1HDpIieOWNcx1W0B1oHYg90hz CGoA3XVkNGhGg8/azO7wf/k= X-Received: by 2002:a05:6358:4f82:b0:129:f7ad:20f1 with SMTP id cg2-20020a0563584f8200b00129f7ad20f1mr794812rwb.23.1686304630163; Fri, 09 Jun 2023 02:57:10 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.114]) by smtp.gmail.com with ESMTPSA id 26-20020a63175a000000b0053f3797fc4asm2603369pgx.0.2023.06.09.02.57.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 02:57:09 -0700 (PDT) From: menglong8.dong@gmail.com X-Google-Original-From: imagedong@tencent.com To: andrii.nakryiko@gmail.com, alan.maguire@oracle.com Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Menglong Dong Subject: [PATCH bpf-next v4 2/3] bpf, x86: clean garbage value in the stack of trampoline Date: Fri, 9 Jun 2023 17:56:52 +0800 Message-Id: <20230609095653.1406173-3-imagedong@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230609095653.1406173-1-imagedong@tencent.com> References: <20230609095653.1406173-1-imagedong@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768221371106256924?= X-GMAIL-MSGID: =?utf-8?q?1768221371106256924?= From: Menglong Dong There are garbage values in upper bytes when we store the arguments into stack in save_regs() if the size of the argument less then 8. As we already reserve 8 byte for the arguments in regs and stack, it is ok to store/restore the regs in BPF_DW size. Then, the garbage values in upper bytes will be cleaned. Signed-off-by: Menglong Dong --- v4: - clean grabage value when argument count is 7 --- arch/x86/net/bpf_jit_comp.c | 45 ++++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 13 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index a767e13c8c85..f6f51a5d14db 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1857,6 +1857,28 @@ st: if (is_imm8(insn->off)) return proglen; } +static inline void clean_garbage(u8 **pprog, int nr_regs, int stack_size, + int arg_size) +{ + u8 *prog; + + /* clean potential garbage values in upper 32-bit. 'stack_size' + * here is the offset of the 7th argument on-stack. + */ + if (nr_regs == 7 && arg_size <= 4) { + int off = -(stack_size - 4); + + prog = *pprog; + /* mov DWORD PTR [rbp + off], 0 */ + if (!is_imm8(off)) + EMIT2_off32(0xC7, 0x85, off); + else + EMIT3(0xC7, 0x45, off); + EMIT(0, 4); + *pprog = prog; + } +} + static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, int stack_size) { @@ -1878,8 +1900,7 @@ static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, if (i <= 5) { /* copy function arguments from regs into stack */ - emit_stx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_FP, + emit_stx(prog, BPF_DW, BPF_REG_FP, i == 5 ? X86_REG_R9 : BPF_REG_1 + i, -(stack_size - i * 8)); } else { @@ -1893,17 +1914,16 @@ static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, * 8(return addr of the caller) * which means: rbp + 24 */ - emit_ldx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_0, BPF_REG_FP, + emit_ldx(prog, BPF_DW, BPF_REG_0, BPF_REG_FP, (i - 6) * 8 + 0x18); - emit_stx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_FP, - BPF_REG_0, + emit_stx(prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -(stack_size - i * 8)); } j = next_same_struct ? j : j + 1; } + + clean_garbage(prog, nr_regs, stack_size - 6 * 8, arg_size); } static void restore_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, @@ -1925,7 +1945,7 @@ static void restore_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, next_same_struct = !next_same_struct; } - emit_ldx(prog, bytes_to_bpf_size(arg_size), + emit_ldx(prog, BPF_DW, i == 5 ? X86_REG_R9 : BPF_REG_1 + i, BPF_REG_FP, -(stack_size - i * 8)); @@ -1956,17 +1976,16 @@ static void prepare_origin_stack(const struct btf_func_model *m, u8 **prog, } if (i > 5) { - emit_ldx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_0, BPF_REG_FP, + emit_ldx(prog, BPF_DW, BPF_REG_0, BPF_REG_FP, (i - 6) * 8 + 0x18); - emit_stx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_FP, - BPF_REG_0, + emit_stx(prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -(stack_size - (i - 6) * 8)); } j = next_same_struct ? j : j + 1; } + + clean_garbage(prog, nr_regs, stack_size, arg_size); } static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, From patchwork Fri Jun 9 09:56:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Menglong Dong X-Patchwork-Id: 105528 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp850212vqr; Fri, 9 Jun 2023 03:39:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ759oFmu8QJU35IJyJBvNB1aegNqRursHcucOQr9bQA44xuOYxQ2d4owHsq+tZwRUPZc1GV X-Received: by 2002:a05:6358:c5a4:b0:121:ab21:1632 with SMTP id fc36-20020a056358c5a400b00121ab211632mr874800rwb.26.1686307155153; Fri, 09 Jun 2023 03:39:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686307155; cv=none; d=google.com; s=arc-20160816; b=Db9HjFUXDdYSDJV3OMCeNMAGpIIL/e8fNJawgTdfJEkTL/EurPTCNQgrMmoTlafqWi D3/BmH03TkDwgtmfJ4QSqQKvxIpJTtSADz+SCZb7f4EcSBiHtE4ylhdFrd+ZIU+fjztB 0U8fuJU583FVHShtNfZnOKt3sBgMa0GIH8ZlWkK5kKBzp6N6EJhYk730MUQhMUuU1UGr ernm8bY+pxRv3mmBPD2QAipSqAPc93ByUZSB9qR0umFXw3YyoqXCQiblPj8UaiF1TnRv SuUgua/LiSfxHXnxdlR+uJVCDUTiTU7VKnd6ACaqFK/8wzXfmZbwhXmBqBZwco7Gq/ob mc0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JxLT1h6F8xwdKl2Puy3yKVy/htvueJ5tWaXe00nH7Yg=; b=zrTk4zAS7wdR3eXOQjRKCQrp1XF3bpCgdX/2b8FVoPgeKwArVvc6CeqeFjVVJj3Ag/ lWl8riZT8N33utdDalvhEKmaSW/OHiJRFHHwWNIdA5UNfPmIdLseXxc0nylCGvJs/628 ntoj/qAdSEXU2os0SLuwaxKbHPfvv8pbqKLS/OYaL348cgpwtgQ8HLES9sp8oLmoRoy1 v/asrxYQGShLCgjf8pWdmNeu9LdSP3YRQYoTukhMeIqpfVVijOWOucVkohIuAKnRjuXi fD7JsPFfnTE8E5BDK+mBTnUSxM/t7bdqd8gIjcwYqeSk2r6fByNBFj5uA1JbuE/wbgHU TheQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=sp36f1+U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k137-20020a636f8f000000b00543a6b76431si2386252pgc.326.2023.06.09.03.39.00; Fri, 09 Jun 2023 03:39:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=sp36f1+U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238011AbjFIKHp (ORCPT + 99 others); Fri, 9 Jun 2023 06:07:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229923AbjFIKGp (ORCPT ); Fri, 9 Jun 2023 06:06:45 -0400 Received: from mail-ot1-x344.google.com (mail-ot1-x344.google.com [IPv6:2607:f8b0:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35A046A41; Fri, 9 Jun 2023 02:57:15 -0700 (PDT) Received: by mail-ot1-x344.google.com with SMTP id 46e09a7af769-6af74ca9f4aso512144a34.3; Fri, 09 Jun 2023 02:57:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304634; x=1688896634; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JxLT1h6F8xwdKl2Puy3yKVy/htvueJ5tWaXe00nH7Yg=; b=sp36f1+U7UIE/GzY+Yor6cPco4VrwtLSsueBy42O7/RbqdM7sh1E7DAJoVGRB4D08C AVRyAps37X77X+p7geVsUgTcJPOLERyGPG0wfVRqDCKbOQgaE+O4sX3bu715jsMnlfhJ Hms9ZrQfwjIrV4eTIPIHzePPlVGLCb3yure6JJUIger0zzxnrOPh3fwJAzMmXIBGZ1Im yLwSWFK2oDysLS/u4I+YEviZELWHuYYiF2j+4MmpD5ry9+5LTr9bH9Ho31qkciaUFuRr fa8/fl3uRk+r57KbVT+0Y/2vt5kqncvRj9S+TPpKXYZAADCsVsroW3diglG7UHMYr/lH yNRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304634; x=1688896634; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JxLT1h6F8xwdKl2Puy3yKVy/htvueJ5tWaXe00nH7Yg=; b=a05Inib7kikKzMjIfu/LTjuiTBW3L1Lt5PRQ6FpBYc9TlDmauOalAR23fYSPfEE0od eiqMSqzCFRxe6sHnMMKnVFzL9bkAjx1nnuPwfsIJ0jGJw58rjSsDMuT5adUBZG6Cz6ZS xONLWoCBHtjn2GlR4JZ3qDsXhvTADAbpJ5WLafGRMcB0l9l8LWHfFP2Jaj07qbSxgQn9 A1X0uq6d/lYa1nbE6nngVnIU5ofiYaRYmR90UkOr4WTGBbvxhwAKRPFMJUNEGaccljtb jxpELTKm5cuAe6TItybygiOS1LyC731moqKk3+tkhyIy096a+FXcZqd2rPK0ksxmgA2d D5Aw== X-Gm-Message-State: AC+VfDwHYQJg6BEMg58CLGjdpgvkoHnNeiKbE0vvOkWRL6DDXeLip2ZB zJJYE95XCxU7/RzEtqH1DmM= X-Received: by 2002:a05:6359:3a8:b0:127:c478:641c with SMTP id eg40-20020a05635903a800b00127c478641cmr783805rwb.28.1686304634260; Fri, 09 Jun 2023 02:57:14 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.114]) by smtp.gmail.com with ESMTPSA id 26-20020a63175a000000b0053f3797fc4asm2603369pgx.0.2023.06.09.02.57.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 02:57:13 -0700 (PDT) From: menglong8.dong@gmail.com X-Google-Original-From: imagedong@tencent.com To: andrii.nakryiko@gmail.com, alan.maguire@oracle.com Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Menglong Dong Subject: [PATCH bpf-next v4 3/3] selftests/bpf: add testcase for FENTRY/FEXIT with 6+ arguments Date: Fri, 9 Jun 2023 17:56:53 +0800 Message-Id: <20230609095653.1406173-4-imagedong@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230609095653.1406173-1-imagedong@tencent.com> References: <20230609095653.1406173-1-imagedong@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768221211758690988?= X-GMAIL-MSGID: =?utf-8?q?1768221211758690988?= From: Menglong Dong Add test9/test10 in fexit_test.c and fentry_test.c to test the fentry and fexit whose target function have 7/12 arguments. Correspondingly, add bpf_testmod_fentry_test7() and bpf_testmod_fentry_test12() to bpf_testmod.c And the testcases passed: ./test_progs -t fexit Summary: 5/12 PASSED, 0 SKIPPED, 0 FAILED ./test_progs -t fentry Summary: 3/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Menglong Dong --- v4: - use different type for args in bpf_testmod_fentry_test{7,12} - add testcase for grabage values in ctx v3: - move bpf_fentry_test{7,12} to bpf_testmod.c and rename them to bpf_testmod_fentry_test{7,12} meanwhile - get return value by bpf_get_func_ret() in "fexit/bpf_testmod_fentry_test12", as we don't change ___bpf_ctx_cast() in this version --- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 19 ++++++- .../selftests/bpf/prog_tests/fentry_fexit.c | 4 +- .../selftests/bpf/prog_tests/fentry_test.c | 2 + .../selftests/bpf/prog_tests/fexit_test.c | 2 + .../testing/selftests/bpf/progs/fentry_test.c | 33 +++++++++++ .../testing/selftests/bpf/progs/fexit_test.c | 57 +++++++++++++++++++ 6 files changed, 115 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index cf216041876c..66615fdbe3df 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -191,6 +191,19 @@ noinline int bpf_testmod_fentry_test3(char a, int b, u64 c) return a + b + c; } +noinline int bpf_testmod_fentry_test7(u64 a, void *b, short c, int d, + void *e, u64 f, u64 g) +{ + return a + (long)b + c + d + (long)e + f + g; +} + +noinline int bpf_testmod_fentry_test12(u64 a, void *b, short c, int d, + void *e, u64 f, u64 g, u64 h, + u64 i, u64 j, u64 k, u64 l) +{ + return a + (long)b + c + d + (long)e + f + g + h + i + j + k + l; +} + __diag_pop(); int bpf_testmod_fentry_ok; @@ -245,7 +258,11 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, if (bpf_testmod_fentry_test1(1) != 2 || bpf_testmod_fentry_test2(2, 3) != 5 || - bpf_testmod_fentry_test3(4, 5, 6) != 15) + bpf_testmod_fentry_test3(4, 5, 6) != 15 || + bpf_testmod_fentry_test7(16, (void *)17, 18, 19, (void *)20, + 21, 22) != 133 || + bpf_testmod_fentry_test12(16, (void *)17, 18, 19, (void *)20, + 21, 22, 23, 24, 25, 26, 27) != 258) goto out; bpf_testmod_fentry_ok = 1; diff --git a/tools/testing/selftests/bpf/prog_tests/fentry_fexit.c b/tools/testing/selftests/bpf/prog_tests/fentry_fexit.c index 130f5b82d2e6..0078acee0ede 100644 --- a/tools/testing/selftests/bpf/prog_tests/fentry_fexit.c +++ b/tools/testing/selftests/bpf/prog_tests/fentry_fexit.c @@ -31,10 +31,12 @@ void test_fentry_fexit(void) ASSERT_OK(err, "ipv6 test_run"); ASSERT_OK(topts.retval, "ipv6 test retval"); + ASSERT_OK(trigger_module_test_read(1), "trigger_read"); + fentry_res = (__u64 *)fentry_skel->bss; fexit_res = (__u64 *)fexit_skel->bss; printf("%lld\n", fentry_skel->bss->test1_result); - for (i = 0; i < 8; i++) { + for (i = 0; i < 11; i++) { ASSERT_EQ(fentry_res[i], 1, "fentry result"); ASSERT_EQ(fexit_res[i], 1, "fexit result"); } diff --git a/tools/testing/selftests/bpf/prog_tests/fentry_test.c b/tools/testing/selftests/bpf/prog_tests/fentry_test.c index c0d1d61d5f66..e1c0ce40febf 100644 --- a/tools/testing/selftests/bpf/prog_tests/fentry_test.c +++ b/tools/testing/selftests/bpf/prog_tests/fentry_test.c @@ -24,6 +24,8 @@ static int fentry_test(struct fentry_test_lskel *fentry_skel) ASSERT_OK(err, "test_run"); ASSERT_EQ(topts.retval, 0, "test_run"); + ASSERT_OK(trigger_module_test_read(1), "trigger_read"); + result = (__u64 *)fentry_skel->bss; for (i = 0; i < sizeof(*fentry_skel->bss) / sizeof(__u64); i++) { if (!ASSERT_EQ(result[i], 1, "fentry_result")) diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_test.c b/tools/testing/selftests/bpf/prog_tests/fexit_test.c index 101b7343036b..ea81fa913ec6 100644 --- a/tools/testing/selftests/bpf/prog_tests/fexit_test.c +++ b/tools/testing/selftests/bpf/prog_tests/fexit_test.c @@ -24,6 +24,8 @@ static int fexit_test(struct fexit_test_lskel *fexit_skel) ASSERT_OK(err, "test_run"); ASSERT_EQ(topts.retval, 0, "test_run"); + ASSERT_OK(trigger_module_test_read(1), "trigger_read"); + result = (__u64 *)fexit_skel->bss; for (i = 0; i < sizeof(*fexit_skel->bss) / sizeof(__u64); i++) { if (!ASSERT_EQ(result[i], 1, "fexit_result")) diff --git a/tools/testing/selftests/bpf/progs/fentry_test.c b/tools/testing/selftests/bpf/progs/fentry_test.c index 52a550d281d9..91dbf63b3ba1 100644 --- a/tools/testing/selftests/bpf/progs/fentry_test.c +++ b/tools/testing/selftests/bpf/progs/fentry_test.c @@ -77,3 +77,36 @@ int BPF_PROG(test8, struct bpf_fentry_test_t *arg) test8_result = 1; return 0; } + +__u64 test9_result = 0; +SEC("fentry/bpf_testmod_fentry_test7") +int BPF_PROG(test9, __u64 a, void *b, short c, int d, void *e, char f, + int g) +{ + test9_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22; + return 0; +} + +__u64 test10_result = 0; +SEC("fentry/bpf_testmod_fentry_test12") +int BPF_PROG(test10, __u64 a, void *b, short c, int d, void *e, char f, + int g, unsigned int h, long i, __u64 j, unsigned long k, + unsigned char l) +{ + test10_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26 && l == 27; + return 0; +} + +__u64 test11_result = 0; +SEC("fentry/bpf_testmod_fentry_test12") +int BPF_PROG(test11, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, + __u64 g, __u64 h, __u64 i, __u64 j, __u64 k, __u64 l) +{ + test11_result = a == 16 && b == 17 && c == 18 && d == 19 && + e == 20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26 && l == 27; + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/fexit_test.c b/tools/testing/selftests/bpf/progs/fexit_test.c index 8f1ccb7302e1..a6d8e03ff5b7 100644 --- a/tools/testing/selftests/bpf/progs/fexit_test.c +++ b/tools/testing/selftests/bpf/progs/fexit_test.c @@ -78,3 +78,60 @@ int BPF_PROG(test8, struct bpf_fentry_test_t *arg) test8_result = 1; return 0; } + +__u64 test9_result = 0; +SEC("fexit/bpf_testmod_fentry_test7") +int BPF_PROG(test9, __u64 a, void *b, short c, int d, void *e, char f, + int g, int ret) +{ + test9_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22 && ret == 133; + return 0; +} + +__u64 test10_result = 0; +SEC("fexit/bpf_testmod_fentry_test12") +int BPF_PROG(test10, __u64 a, void *b, short c, int d, void *e, char f, + int g, unsigned int h, long i, __u64 j, unsigned long k, + unsigned char l) +{ + __u64 ret; + int err; + + /* BPF_PROG() don't support 14 arguments, and ctx[12] can't be + * accessed yet. So we get the return value by bpf_get_func_ret() + * for now. + */ + err = bpf_get_func_ret(ctx, &ret); + if (err) + return 0; + + test10_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26 && l == 27 && + (int)ret == 258; + return 0; +} + +__u64 test11_result = 0; +SEC("fexit/bpf_testmod_fentry_test12") +int BPF_PROG(test11, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, + __u64 g, __u64 h, __u64 i, __u64 j, __u64 k, __u64 l) +{ + __u64 ret; + int err; + + /* BPF_PROG() don't support 14 arguments, and ctx[12] can't be + * accessed yet. So we get the return value by bpf_get_func_ret() + * for now. + */ + err = bpf_get_func_ret(ctx, &ret); + if (err) + return 0; + + test11_result = a == 16 && b == 17 && c == 18 && d == 19 && + e == 20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26 && l == 27 && + ret == 258; + return 0; +}