From patchwork Tue Jun 27 11:53:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Menglong Dong X-Patchwork-Id: 113399 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8150942vqr; Tue, 27 Jun 2023 05:19:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4F3X/wjlVBiUTyyv3DA4fK4HgARTgCSYyZwSvMZoaQZYE+2pqc6UqkfNtiP7fbX1zPAodl X-Received: by 2002:a92:d305:0:b0:345:af1c:29c4 with SMTP id x5-20020a92d305000000b00345af1c29c4mr3849798ila.25.1687868365557; Tue, 27 Jun 2023 05:19:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687868365; cv=none; d=google.com; s=arc-20160816; b=TI+2TogffeU5T69432AjqHg4FVaQyNWKXVxaTSCCItEr69RR6gJIGAJXfBHvroMMug LrH5iRTag7b1CJIx6X1BrunX6fgzncL/GsZZIjdj8Iy22Hqik6yyx2VMx+cBYfOTZ4N5 sHlNiBRzlNoCrFm9oijDqvioMtNA0cP4Pfz6+p4wWLK4crqNSK8hmq83gQ0iAQaHbNj+ M6zPcORCTHOxsptWWolmb+hcaAIQSGFq93cKSWRcxtI2XZr7XXsMdsj5L4CLOp5TLHUA myfBQKRTl0MQlUFZ9BzUgK93cooc9FNal6Agagos0pKGsP8fwdjoQCFiL25gt+1RIq+B FJHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9UQHgCYMSUpJKwkxCT8Gz4j2ViWlUBZ9rAr/225IOUc=; fh=oOyr5eKAhccn18uZj16GqOzyJa7ePeXikTQWMgPs4Qg=; b=s4E4Unfs8upMq2C4ltLc43B6L/+Och7cdhKb3JYQdWdly7ASv4805TVwNBM211ydSV X5z9NtAdw+Z0CztgTpcLfbYLNd8AM+LZxXZvY6T3jmKrnMU4v++cAQblN0NGt7PxlgHr wIhamJD1s4K8EelAwfWGg1WRfztv74gUfkpuGm1DXu/f0vWQeZI2YnoS9l41lkCom+X+ vH/2IG9R+yQO0JRdi5c1NJwTQuJPB7U293rUBha63IO7IZYfy0KziFvt7UmY4FZSH69A PAVuVkvkQek/ikr9IK35zvLHxf7Mclw9oEKOu1YgFtCB+xDHTb57cCetKLk3ygQUQLms 8NIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=l79nZCW6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l186-20020a6391c3000000b005573bd5add3si7215580pge.214.2023.06.27.05.19.12; Tue, 27 Jun 2023 05:19:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=l79nZCW6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230224AbjF0LyF (ORCPT + 99 others); Tue, 27 Jun 2023 07:54:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229593AbjF0LyD (ORCPT ); Tue, 27 Jun 2023 07:54:03 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E39E4E5B; Tue, 27 Jun 2023 04:54:02 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id 41be03b00d2f7-51b4ef5378bso3549144a12.1; Tue, 27 Jun 2023 04:54:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687866842; x=1690458842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9UQHgCYMSUpJKwkxCT8Gz4j2ViWlUBZ9rAr/225IOUc=; b=l79nZCW6i+H8KEntEfZJU3WoFNVHt3GJ2ez0IKkzMmhANEfia2mXwZIJmhTOkUUAsA j0n5ehct3U4l+REY4Cq25vGRWuYSR99M/iVT3GYGXUh6/KjDmFjF2zwxvVz379EKWjO0 brZ59B4YjkIsFSwsMNvvEZoKsywq4N9QMACvI32ycB1Rzvo6Lc+B/7f7bZAFL1WwE01H XwcihlaXNzCDCfAVkZH4oxIGFZQvfpPkTYyPfltEYdsKIyuIz0EpikaUA1X2VFPqrN5D prDSeNoAnUjJKLBKzzg7d9wgR3HVKkBCwaAwNqA4quVBaO8EzVSsgioAXlRjsW3DdEK5 weTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687866842; x=1690458842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9UQHgCYMSUpJKwkxCT8Gz4j2ViWlUBZ9rAr/225IOUc=; b=HPQ1hLUp3I3hGoQAUkVdGAsEhMlIgyF2NTBTnqwV7nDz/FP9BWfmmHy+JYGo1VAJF7 IPpNcTc16zcRnsblxfw+U1g5S5jW5Jy64UXcsqSan09HaZXsNmr4LsaDLxGh6BLAmlt9 yUo2Z3Eiupojsi/jvfxFRZ0zgdv+r5aahtlykC6WLXHcD6JTZ1rqofj32Hnt/fFejKUq dVNjaGPWzzfSM/nelFl/Fo82VzQmVbuXL5Qq+9y1ORWpsruIUn/guyTlz4u5eScvpGAJ AuvA3+xVprnql3kELPtrFp1tjk3cHki6wNjGl7SK5z4OkSTDswUi2AguX6SCtzVNeEcI CJGg== X-Gm-Message-State: AC+VfDwoXH95sFDYDz2Rm5JVqGYcz1e1ZTSJmzjZVHx3cq3TRFBrVkm9 N+zuXN8mVxw4AOx5PRa1KPE= X-Received: by 2002:a17:902:f552:b0:1b5:1e24:8a76 with SMTP id h18-20020a170902f55200b001b51e248a76mr11609716plf.65.1687866842319; Tue, 27 Jun 2023 04:54:02 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.112]) by smtp.gmail.com with ESMTPSA id jg3-20020a17090326c300b001a2104d706fsm4961236plb.225.2023.06.27.04.53.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Jun 2023 04:54:01 -0700 (PDT) From: menglong8.dong@gmail.com X-Google-Original-From: imagedong@tencent.com To: daniel@iogearbox.net, yhs@meta.com, alexei.starovoitov@gmail.com Cc: ast@kernel.org, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Menglong Dong Subject: [PATCH bpf-next v8 1/3] bpf, x86: save/restore regs with BPF_DW size Date: Tue, 27 Jun 2023 19:53:17 +0800 Message-Id: <20230627115319.13128-2-imagedong@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230627115319.13128-1-imagedong@tencent.com> References: <20230627115319.13128-1-imagedong@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769858259428832757?= X-GMAIL-MSGID: =?utf-8?q?1769858259428832757?= From: Menglong Dong As we already reserve 8 byte in the stack for each reg, it is ok to store/restore the regs in BPF_DW size. This will make the code in save_regs()/restore_regs() simpler. Signed-off-by: Menglong Dong Acked-by: Yonghong Song --- v6: - adjust the commit log --- arch/x86/net/bpf_jit_comp.c | 35 ++++++----------------------------- 1 file changed, 6 insertions(+), 29 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 1056bbf55b17..a407fbbffecd 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1860,57 +1860,34 @@ st: if (is_imm8(insn->off)) static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, int stack_size) { - int i, j, arg_size; - bool next_same_struct = false; + int i; /* Store function arguments to stack. * For a function that accepts two pointers the sequence will be: * mov QWORD PTR [rbp-0x10],rdi * mov QWORD PTR [rbp-0x8],rsi */ - for (i = 0, j = 0; i < min(nr_regs, 6); i++) { - /* The arg_size is at most 16 bytes, enforced by the verifier. */ - arg_size = m->arg_size[j]; - if (arg_size > 8) { - arg_size = 8; - next_same_struct = !next_same_struct; - } - - emit_stx(prog, bytes_to_bpf_size(arg_size), - BPF_REG_FP, + for (i = 0; i < min(nr_regs, 6); i++) + emit_stx(prog, BPF_DW, BPF_REG_FP, i == 5 ? X86_REG_R9 : BPF_REG_1 + i, -(stack_size - i * 8)); - - j = next_same_struct ? j : j + 1; - } } static void restore_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, int stack_size) { - int i, j, arg_size; - bool next_same_struct = false; + int i; /* Restore function arguments from stack. * For a function that accepts two pointers the sequence will be: * EMIT4(0x48, 0x8B, 0x7D, 0xF0); mov rdi,QWORD PTR [rbp-0x10] * EMIT4(0x48, 0x8B, 0x75, 0xF8); mov rsi,QWORD PTR [rbp-0x8] */ - for (i = 0, j = 0; i < min(nr_regs, 6); i++) { - /* The arg_size is at most 16 bytes, enforced by the verifier. */ - arg_size = m->arg_size[j]; - if (arg_size > 8) { - arg_size = 8; - next_same_struct = !next_same_struct; - } - - emit_ldx(prog, bytes_to_bpf_size(arg_size), + for (i = 0; i < min(nr_regs, 6); i++) + emit_ldx(prog, BPF_DW, i == 5 ? X86_REG_R9 : BPF_REG_1 + i, BPF_REG_FP, -(stack_size - i * 8)); - - j = next_same_struct ? j : j + 1; - } } static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, From patchwork Tue Jun 27 11:53:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Menglong Dong X-Patchwork-Id: 113413 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8164931vqr; Tue, 27 Jun 2023 05:43:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4ia1yU+STGFrYNZ9XFhTILtA4A+eNydfC7cK6mkG4L+u7HULCrON9AJrDajhnz1fTxMkA6 X-Received: by 2002:a17:907:1ca8:b0:973:d076:67ab with SMTP id nb40-20020a1709071ca800b00973d07667abmr29298383ejc.42.1687869785114; Tue, 27 Jun 2023 05:43:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687869785; cv=none; d=google.com; s=arc-20160816; b=llNuujWawXTTNEynSwRucxzArM37ALmwVQl+NHoDUZ++6/gJaQaM+qhNpZD49tHdBn Ck/BfweREVse0hpNPBnW7JLXfgtH8mi047ZfuhX+dysYp3GUq2emaVZlfQm3HXLSDRfW NJqHnoobKwXMa9sQckosRJcwSTdeASgve7JKMSmGUrPAAbTaHjdQGRMio4RHUqSJkydd VBZ2HIOaM7eRZAWI4FnohBIMayzzToCWtt4GgCDJBdFDLai/6JwcrvtMeiGHneeRMD7k vSI3BrziE/TzdRZqNyRzVdQhYJWAesDr6AHwp9p4xZKuajk76qJkp0cLH481nDwBmSz5 dZ0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=oomXNIYbXuRlYHkS3Ip9w1mB8xpX1TPCSZ+2PzWA9QM=; fh=oOyr5eKAhccn18uZj16GqOzyJa7ePeXikTQWMgPs4Qg=; b=DS5N1bnYshIv3Sh6Vnoe37nmKow//NYLzdWyH/TaNmvY8ADayXaQJw43S10/fcxyP+ kCpVqDxREKT/116WnqQ/vx69xsDYVJq6LL/kWig5EcDaD8aQs7m0Nyh37fXNGOTKkp5j G0z6wRyRB/O/AdiaQnlUlH5OJKHbPVkW2NG9knW7wpdymiEwYfrAj9T3sOcnAGrKH1NK yorGllnYuMMtpioaqacf3hbZ4tGZDHPNyxYKoSsVGPhtr461x8vihbPdejzeKfM01zi4 dn6rianADmBaaAjURDvUyOcHEM61cLdn9H9o2hK+po0u9+ShorqTPSyiyw/CguJFWKb7 pZ/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=P1l55yXV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jp9-20020a170906f74900b00983c3a0a4adsi4340508ejb.874.2023.06.27.05.42.39; Tue, 27 Jun 2023 05:43:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=P1l55yXV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231497AbjF0LyO (ORCPT + 99 others); Tue, 27 Jun 2023 07:54:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231165AbjF0LyJ (ORCPT ); Tue, 27 Jun 2023 07:54:09 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BB5DDA; Tue, 27 Jun 2023 04:54:07 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id d9443c01a7336-1b7f9d66812so21341595ad.0; Tue, 27 Jun 2023 04:54:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687866847; x=1690458847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oomXNIYbXuRlYHkS3Ip9w1mB8xpX1TPCSZ+2PzWA9QM=; b=P1l55yXVaMkyYj3/pRmhqbddCy8Uwdjn1N+xb1wRA9yhMHiklLzQyUNY/90tBkAtwj VwVwNwoSRhJCAtH/fociko6OjPB+66VsFVoxSeGXAnGYZ2iFN4k/yNQKXSPcTpLbhwGx vDUFe0Xm4IRRCGTvsacBLxWHA38THYxEp5//o4gZ291Yof3uP2nJ0WCPyYwtJmxEsCje wxIvF6PiR9edVtguXAQZJsRbvvP3kGf34bnvaWxEEJxCSQ2QDhRkJNVXh53n2F9Q3HnW CfXgR0S1Qyyo2BtSChiiXYdiGHX3INoy8ZjwHUjsIc+JV4KSk4bhvAZ43uyHzrXF64Tc ypJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687866847; x=1690458847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oomXNIYbXuRlYHkS3Ip9w1mB8xpX1TPCSZ+2PzWA9QM=; b=ZjjaMHNZm40mcVmH8RdWcYXk5TSxVY55KaS3gMvnT7cyUekc03owXbXvw6xNPyF4Ry 7W0lVZ2M5ek3VLW07ImYb0Et9Iw9mgd/H+zFaOkk1J1JmAH4w7POx8oaPk1Ma74tIUoe V77CKXB/+G5QYs9aS7b6yHlysggU1+AztOSerCrOxG5Y7Xo2GgbqSqeRHqH+Aklnw/if fE1zvxAosKaklmjt//AQNTKFTvEE8AB7iYhqwSQLQFRC3ec01tgasD32ymra2xOYdshf NQNqDMDt5W7c8iRE9Wos4b+iNbZUaL4AYVIfHkt2Dk17TI4Pr2hLkmGgTOp/2TUcO1/W 1OzQ== X-Gm-Message-State: AC+VfDyqd/2pVHipnjQBlrB4UrKEfgwTFzDTOFRzJwdC4a8wBDooi2vn QJoj1OrTHT94wDZntmksZro= X-Received: by 2002:a17:902:e84a:b0:1b8:15e3:d9b1 with SMTP id t10-20020a170902e84a00b001b815e3d9b1mr3233158plg.22.1687866846825; Tue, 27 Jun 2023 04:54:06 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.112]) by smtp.gmail.com with ESMTPSA id jg3-20020a17090326c300b001a2104d706fsm4961236plb.225.2023.06.27.04.54.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Jun 2023 04:54:06 -0700 (PDT) From: menglong8.dong@gmail.com X-Google-Original-From: imagedong@tencent.com To: daniel@iogearbox.net, yhs@meta.com, alexei.starovoitov@gmail.com Cc: ast@kernel.org, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Menglong Dong Subject: [PATCH bpf-next v8 2/3] bpf, x86: allow function arguments up to 12 for TRACING Date: Tue, 27 Jun 2023 19:53:18 +0800 Message-Id: <20230627115319.13128-3-imagedong@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230627115319.13128-1-imagedong@tencent.com> References: <20230627115319.13128-1-imagedong@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769858376499280738?= X-GMAIL-MSGID: =?utf-8?q?1769859747994205574?= From: Menglong Dong For now, the BPF program of type BPF_PROG_TYPE_TRACING can only be used on the kernel functions whose arguments count less than or equal to 6, if not considering '> 8 bytes' struct argument. This is not friendly at all, as too many functions have arguments count more than 6. According to the current kernel version, below is a statistics of the function arguments count: argument count | function count 7 | 704 8 | 270 9 | 84 10 | 47 11 | 47 12 | 27 13 | 22 14 | 5 15 | 0 16 | 1 Therefore, let's enhance it by increasing the function arguments count allowed in arch_prepare_bpf_trampoline(), for now, only x86_64. For the case that we don't need to call origin function, which means without BPF_TRAMP_F_CALL_ORIG, we need only copy the function arguments that stored in the frame of the caller to current frame. The 7th and later arguments are stored in "$rbp + 0x18", and they will be copied to the stack area following where register values are saved. For the case with BPF_TRAMP_F_CALL_ORIG, we need prepare the arguments in stack before call origin function, which means we need alloc extra "8 * (arg_count - 6)" memory in the top of the stack. Note, there should not be any data be pushed to the stack before calling the origin function. So 'rbx' value will be stored on a stack position higher than where stack arguments are stored for BPF_TRAMP_F_CALL_ORIG. According to the research of Yonghong, struct members should be all in register or all on the stack. Meanwhile, the compiler will pass the argument on regs if the remaining regs can hold the argument. Therefore, we need save the arguments in order. Otherwise, disorder of the args can happen. For example: struct foo_struct { long a; int b; }; int foo(char, char, char, char, char, struct foo_struct, char); the arg1-5,arg7 will be passed by regs, and arg6 will by stack. Therefore, we should save/restore the arguments in the same order with the declaration of foo(). And the args used as ctx in stack will be like this: reg_arg6 -- copy from regs stack_arg2 -- copy from stack stack_arg1 reg_arg5 -- copy from regs reg_arg4 reg_arg3 reg_arg2 reg_arg1 We use EMIT3_off32() or EMIT4() for "lea" and "sub". The range of the imm in "lea" and "sub" is [-128, 127] if EMIT4() is used. Therefore, we use EMIT3_off32() instead if the imm out of the range. It works well for the FENTRY/FEXIT/MODIFY_RETURN. Signed-off-by: Menglong Dong Acked-by: Yonghong Song --- v7: - somit nits from commit message and comment - remove the inline in get_nr_regs() - rename some function and various v6: - introduce get_nr_regs() to get the space that used to pass args on stack correct - rename some args and fix some spelling mistake v5: - consider the case of the struct in arguments can't be hold by regs v4: - make the stack 16-byte aligned if passing args on-stack is needed - add the function arguments statistics to the commit log v3: - use EMIT3_off32() for "lea" and "sub" only on necessary - make 12 as the maximum arguments count v2: - instead EMIT4 with EMIT3_off32 for "lea" to prevent overflow - make MAX_BPF_FUNC_ARGS as the maximum argument count --- arch/x86/net/bpf_jit_comp.c | 235 ++++++++++++++++++++++++++++++++---- 1 file changed, 209 insertions(+), 26 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index a407fbbffecd..08608284041a 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1857,37 +1857,178 @@ st: if (is_imm8(insn->off)) return proglen; } -static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, - int stack_size) +static void clean_stack_garbage(const struct btf_func_model *m, + u8 **pprog, int nr_stack_slots, + int stack_size) { - int i; + int arg_size, off; + u8 *prog; + + /* Generally speaking, the compiler will pass the arguments + * on-stack with "push" instruction, which will take 8-byte + * on the stack. In this case, there won't be garbage values + * while we copy the arguments from origin stack frame to current + * in BPF_DW. + * + * However, sometimes the compiler will only allocate 4-byte on + * the stack for the arguments. For now, this case will only + * happen if there is only one argument on-stack and its size + * not more than 4 byte. In this case, there will be garbage + * values on the upper 4-byte where we store the argument on + * current stack frame. + * + * arguments on origin stack: + * + * stack_arg_1(4-byte) xxx(4-byte) + * + * what we copy: + * + * stack_arg_1(8-byte): stack_arg_1(origin) xxx + * + * and the xxx is the garbage values which we should clean here. + */ + if (nr_stack_slots != 1) + return; + + /* the size of the last argument */ + arg_size = m->arg_size[m->nr_args - 1]; + if (arg_size <= 4) { + off = -(stack_size - 4); + prog = *pprog; + /* mov DWORD PTR [rbp + off], 0 */ + if (!is_imm8(off)) + EMIT2_off32(0xC7, 0x85, off); + else + EMIT3(0xC7, 0x45, off); + EMIT(0, 4); + *pprog = prog; + } +} + +/* get the count of the regs that are used to pass arguments */ +static int get_nr_used_regs(const struct btf_func_model *m) +{ + int i, arg_regs, nr_used_regs = 0; + + for (i = 0; i < min_t(int, m->nr_args, MAX_BPF_FUNC_ARGS); i++) { + arg_regs = (m->arg_size[i] + 7) / 8; + if (nr_used_regs + arg_regs <= 6) + nr_used_regs += arg_regs; + + if (nr_used_regs >= 6) + break; + } + + return nr_used_regs; +} + +static void save_args(const struct btf_func_model *m, u8 **prog, + int stack_size, bool for_call_origin) +{ + int arg_regs, first_off, nr_regs = 0, nr_stack_slots = 0; + int i, j; /* Store function arguments to stack. * For a function that accepts two pointers the sequence will be: * mov QWORD PTR [rbp-0x10],rdi * mov QWORD PTR [rbp-0x8],rsi */ - for (i = 0; i < min(nr_regs, 6); i++) - emit_stx(prog, BPF_DW, BPF_REG_FP, - i == 5 ? X86_REG_R9 : BPF_REG_1 + i, - -(stack_size - i * 8)); + for (i = 0; i < min_t(int, m->nr_args, MAX_BPF_FUNC_ARGS); i++) { + arg_regs = (m->arg_size[i] + 7) / 8; + + /* According to the research of Yonghong, struct members + * should be all in register or all on the stack. + * Meanwhile, the compiler will pass the argument on regs + * if the remaining regs can hold the argument. + * + * Disorder of the args can happen. For example: + * + * struct foo_struct { + * long a; + * int b; + * }; + * int foo(char, char, char, char, char, struct foo_struct, + * char); + * + * the arg1-5,arg7 will be passed by regs, and arg6 will + * by stack. + */ + if (nr_regs + arg_regs > 6) { + /* copy function arguments from origin stack frame + * into current stack frame. + * + * The starting address of the arguments on-stack + * is: + * rbp + 8(push rbp) + + * 8(return addr of origin call) + + * 8(return addr of the caller) + * which means: rbp + 24 + */ + for (j = 0; j < arg_regs; j++) { + emit_ldx(prog, BPF_DW, BPF_REG_0, BPF_REG_FP, + nr_stack_slots * 8 + 0x18); + emit_stx(prog, BPF_DW, BPF_REG_FP, BPF_REG_0, + -stack_size); + + if (!nr_stack_slots) + first_off = stack_size; + stack_size -= 8; + nr_stack_slots++; + } + } else { + /* Only copy the arguments on-stack to current + * 'stack_size' and ignore the regs, used to + * prepare the arguments on-stack for orign call. + */ + if (for_call_origin) { + nr_regs += arg_regs; + continue; + } + + /* copy the arguments from regs into stack */ + for (j = 0; j < arg_regs; j++) { + emit_stx(prog, BPF_DW, BPF_REG_FP, + nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs, + -stack_size); + stack_size -= 8; + nr_regs++; + } + } + } + + clean_stack_garbage(m, prog, nr_stack_slots, first_off); } -static void restore_regs(const struct btf_func_model *m, u8 **prog, int nr_regs, +static void restore_regs(const struct btf_func_model *m, u8 **prog, int stack_size) { - int i; + int i, j, arg_regs, nr_regs = 0; /* Restore function arguments from stack. * For a function that accepts two pointers the sequence will be: * EMIT4(0x48, 0x8B, 0x7D, 0xF0); mov rdi,QWORD PTR [rbp-0x10] * EMIT4(0x48, 0x8B, 0x75, 0xF8); mov rsi,QWORD PTR [rbp-0x8] + * + * The logic here is similar to what we do in save_args() */ - for (i = 0; i < min(nr_regs, 6); i++) - emit_ldx(prog, BPF_DW, - i == 5 ? X86_REG_R9 : BPF_REG_1 + i, - BPF_REG_FP, - -(stack_size - i * 8)); + for (i = 0; i < min_t(int, m->nr_args, MAX_BPF_FUNC_ARGS); i++) { + arg_regs = (m->arg_size[i] + 7) / 8; + if (nr_regs + arg_regs <= 6) { + for (j = 0; j < arg_regs; j++) { + emit_ldx(prog, BPF_DW, + nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs, + BPF_REG_FP, + -stack_size); + stack_size -= 8; + nr_regs++; + } + } else { + stack_size -= 8 * arg_regs; + } + + if (nr_regs >= 6) + break; + } } static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, @@ -1915,7 +2056,10 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, /* arg1: mov rdi, progs[i] */ emit_mov_imm64(&prog, BPF_REG_1, (long) p >> 32, (u32) (long) p); /* arg2: lea rsi, [rbp - ctx_cookie_off] */ - EMIT4(0x48, 0x8D, 0x75, -run_ctx_off); + if (!is_imm8(-run_ctx_off)) + EMIT3_off32(0x48, 0x8D, 0xB5, -run_ctx_off); + else + EMIT4(0x48, 0x8D, 0x75, -run_ctx_off); if (emit_rsb_call(&prog, bpf_trampoline_enter(p), prog)) return -EINVAL; @@ -1931,7 +2075,10 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, emit_nops(&prog, 2); /* arg1: lea rdi, [rbp - stack_size] */ - EMIT4(0x48, 0x8D, 0x7D, -stack_size); + if (!is_imm8(-stack_size)) + EMIT3_off32(0x48, 0x8D, 0xBD, -stack_size); + else + EMIT4(0x48, 0x8D, 0x7D, -stack_size); /* arg2: progs[i]->insnsi for interpreter */ if (!p->jited) emit_mov_imm64(&prog, BPF_REG_2, @@ -1961,7 +2108,10 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, /* arg2: mov rsi, rbx <- start time in nsec */ emit_mov_reg(&prog, true, BPF_REG_2, BPF_REG_6); /* arg3: lea rdx, [rbp - run_ctx_off] */ - EMIT4(0x48, 0x8D, 0x55, -run_ctx_off); + if (!is_imm8(-run_ctx_off)) + EMIT3_off32(0x48, 0x8D, 0x95, -run_ctx_off); + else + EMIT4(0x48, 0x8D, 0x55, -run_ctx_off); if (emit_rsb_call(&prog, bpf_trampoline_exit(p), prog)) return -EINVAL; @@ -2113,7 +2263,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i void *func_addr) { int i, ret, nr_regs = m->nr_args, stack_size = 0; - int regs_off, nregs_off, ip_off, run_ctx_off; + int regs_off, nregs_off, ip_off, run_ctx_off, arg_stack_off, rbx_off; struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; @@ -2127,8 +2277,10 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (m->arg_flags[i] & BTF_FMODEL_STRUCT_ARG) nr_regs += (m->arg_size[i] + 7) / 8 - 1; - /* x86-64 supports up to 6 arguments. 7+ can be added in the future */ - if (nr_regs > 6) + /* x86-64 supports up to MAX_BPF_FUNC_ARGS arguments. 1-6 + * are passed through regs, the remains are through stack. + */ + if (nr_regs > MAX_BPF_FUNC_ARGS) return -ENOTSUPP; /* Generated trampoline stack layout: @@ -2147,7 +2299,14 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i * * RBP - ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag * + * RBP - rbx_off [ rbx value ] always + * * RBP - run_ctx_off [ bpf_tramp_run_ctx ] + * + * [ stack_argN ] BPF_TRAMP_F_CALL_ORIG + * [ ... ] + * [ stack_arg2 ] + * RBP - arg_stack_off [ stack_arg1 ] */ /* room for return value of orig_call or fentry prog */ @@ -2167,9 +2326,26 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i ip_off = stack_size; + stack_size += 8; + rbx_off = stack_size; + stack_size += (sizeof(struct bpf_tramp_run_ctx) + 7) & ~0x7; run_ctx_off = stack_size; + if (nr_regs > 6 && (flags & BPF_TRAMP_F_CALL_ORIG)) { + /* the space that used to pass arguments on-stack */ + stack_size += (nr_regs - get_nr_used_regs(m)) * 8; + /* make sure the stack pointer is 16-byte aligned if we + * need pass arguments on stack, which means + * [stack_size + 8(rbp) + 8(rip) + 8(origin rip)] + * should be 16-byte aligned. Following code depend on + * that stack_size is already 8-byte aligned. + */ + stack_size += (stack_size % 16) ? 0 : 8; + } + + arg_stack_off = stack_size; + if (flags & BPF_TRAMP_F_SKIP_FRAME) { /* skip patched call instruction and point orig_call to actual * body of the kernel function. @@ -2189,8 +2365,14 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i x86_call_depth_emit_accounting(&prog, NULL); EMIT1(0x55); /* push rbp */ EMIT3(0x48, 0x89, 0xE5); /* mov rbp, rsp */ - EMIT4(0x48, 0x83, 0xEC, stack_size); /* sub rsp, stack_size */ - EMIT1(0x53); /* push rbx */ + if (!is_imm8(stack_size)) + /* sub rsp, stack_size */ + EMIT3_off32(0x48, 0x81, 0xEC, stack_size); + else + /* sub rsp, stack_size */ + EMIT4(0x48, 0x83, 0xEC, stack_size); + /* mov QWORD PTR [rbp - rbx_off], rbx */ + emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_6, -rbx_off); /* Store number of argument registers of the traced function: * mov rax, nr_regs @@ -2208,7 +2390,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -ip_off); } - save_regs(m, &prog, nr_regs, regs_off); + save_args(m, &prog, regs_off, false); if (flags & BPF_TRAMP_F_CALL_ORIG) { /* arg1: mov rdi, im */ @@ -2238,7 +2420,8 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i } if (flags & BPF_TRAMP_F_CALL_ORIG) { - restore_regs(m, &prog, nr_regs, regs_off); + restore_regs(m, &prog, regs_off); + save_args(m, &prog, arg_stack_off, true); if (flags & BPF_TRAMP_F_ORIG_STACK) { emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); @@ -2279,7 +2462,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i } if (flags & BPF_TRAMP_F_RESTORE_REGS) - restore_regs(m, &prog, nr_regs, regs_off); + restore_regs(m, &prog, regs_off); /* This needs to be done regardless. If there were fmod_ret programs, * the return value is only updated on the stack and still needs to be @@ -2298,7 +2481,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (save_ret) emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, -8); - EMIT1(0x5B); /* pop rbx */ + emit_ldx(&prog, BPF_DW, BPF_REG_6, BPF_REG_FP, -rbx_off); EMIT1(0xC9); /* leave */ if (flags & BPF_TRAMP_F_SKIP_FRAME) /* skip our return address and return to parent */ From patchwork Tue Jun 27 11:53:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Menglong Dong X-Patchwork-Id: 113412 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8164773vqr; Tue, 27 Jun 2023 05:42:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4ICkzwbsknk9jwVdf51YiybEFl6woFbt2/9JJdmD08uIz4YjazUcVD7XSdorc1aU5Yb7QO X-Received: by 2002:a17:907:762f:b0:991:cd1f:e67a with SMTP id jy15-20020a170907762f00b00991cd1fe67amr3769859ejc.29.1687869770772; Tue, 27 Jun 2023 05:42:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687869770; cv=none; d=google.com; s=arc-20160816; b=HVSeHNxpULI64AMbzPmDxXAY9gs1fTq6TZjIVW7sP7D7YX9u7pC1LQwBgjAixRcIy5 GHfXbxsP2w8YEbC1CQaE9Ux7EBrocG1yHWLBevsnGvnVs+xiljiNosrX+NPMH/It3XlC iApFyJkojU2yrLoFO7lUpiITiZRqcwnjK1QozLLI4sur6D786Yf0vPgRW6z2HaEfP+Jg v+knR8xLCBBD+Pr6Pus1Ub3YrAw7NeXR3dQGr3UV840TiF44xdkEAxF9j/T/q0kkCRJY FP1nHZNKaAvdRaRsPw+toqEdbVccQVWf19WqSVvsYEQRCckoHWQeiwXngwrntNwPtrK4 5HDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=sc9v4qgoRPFxfTeKcedhkCHwNxQn43aij2BhsBxVmxo=; fh=oOyr5eKAhccn18uZj16GqOzyJa7ePeXikTQWMgPs4Qg=; b=HaU9VZnzZbSjSgec03BcpHE8eNC8t6kOReSRef/MrzlzblQaaB+vmGSFKPjbFnFc3b ANOZK4xD6uiEDWODgjoc1EHaen+zcrs3LTUWKNmcjvS+so2aJ5n+lF0eLqwcXgRAv0mt 5V3yodNQmCvQ8sw1deGpmUMvoyIArFGfMTxuudERpeTEQmBju1fw7RO7uedeiDyPBrwc 7x5F3B9XCThhZ9yPSMsXJezDUrt7kR3ZytIXZFZaQyfQ8y8xIwhwFWYX0KTKSJ471WU7 Ffez1GQM9qakTnVgQBON9+04KdP5jB1WSK/ch0g0TBK6GkkIh4aKrkxSedCC1jyhXuq4 Nsxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=GPkEj6wM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lw20-20020a170906bcd400b0098e4aef0789si3007188ejb.613.2023.06.27.05.42.22; Tue, 27 Jun 2023 05:42:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=GPkEj6wM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231372AbjF0LyZ (ORCPT + 99 others); Tue, 27 Jun 2023 07:54:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231127AbjF0LyR (ORCPT ); Tue, 27 Jun 2023 07:54:17 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 254B8E5B; Tue, 27 Jun 2023 04:54:12 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id d9443c01a7336-1b7e1875cc1so14162055ad.1; Tue, 27 Jun 2023 04:54:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687866851; x=1690458851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sc9v4qgoRPFxfTeKcedhkCHwNxQn43aij2BhsBxVmxo=; b=GPkEj6wMwAgjmZ4lw0VvB1kB3j4BfEq+Ge9Ch/eyDza5X1QcgbPpTENGDIj8NAqYiT I7BNXQin+dzM3zmuXAVWOOq5xeSDemG7yvQTcz8DO5kGBa1ghx5135sZbzvwlskWegNU O0CyqKYZxAkNFlqlsWzFc/R1rzPNhLbDskNf9YIxkjPfqwO2v1WRzJI6CIfnHSI2IRuy rAKUZ0hKxDaPOuwPc0GiJ6MxMvZ4k44sa9IXXJpw7fcD2Oy4y/oSUSR+ncjANC3MrTrh PTq+gKLB9mCa6U9Pzab7m9wkLhHof8rbmO3Bz+Zf6FkxxRG3aGHCmw1cs3QanTadatBq 0fFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687866851; x=1690458851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sc9v4qgoRPFxfTeKcedhkCHwNxQn43aij2BhsBxVmxo=; b=Z4thdIC0MDWS8O5LQqalWo6ciLlagSfIwEkSetLJ54Us4hU0tiZAIkhGfDidfPgiId 4Vyj6xJnu43DYvkrKq2tpp5QDqXHbaLSmkFjO0pnAv21kneBCkqh8rnWb76hI3SGcJQb wxznyMXfrYzNrN7+a4OH6nNSbJJxTMZMPgMgFp8YH4oUFeUtRYKOYNeursxW3AVqBdkh UGt9lAKAqBPBwWim/8BxpFZybDDt6uNoLCNCkz6S/dSyv+N210jVWB4OqLYCtTE4ASLu l7uIv7XCai3oIz8LvCZuQ/YHgiC/wUpIDfUdRAOvxIBkgGUyQ7Op8JrmYLHat+wQRnzD hYAQ== X-Gm-Message-State: AC+VfDxVXzWqfMIaR5g0uZ7S0TPscONEw/EupqIOErKOq6rQVXhcbYCx kzvVCUEZa/HQVoyXe3exxSU= X-Received: by 2002:a17:902:b289:b0:1b7:fb17:2edb with SMTP id u9-20020a170902b28900b001b7fb172edbmr4224318plr.18.1687866851242; Tue, 27 Jun 2023 04:54:11 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.112]) by smtp.gmail.com with ESMTPSA id jg3-20020a17090326c300b001a2104d706fsm4961236plb.225.2023.06.27.04.54.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Jun 2023 04:54:10 -0700 (PDT) From: menglong8.dong@gmail.com X-Google-Original-From: imagedong@tencent.com To: daniel@iogearbox.net, yhs@meta.com, alexei.starovoitov@gmail.com Cc: ast@kernel.org, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Menglong Dong Subject: [PATCH bpf-next v8 3/3] selftests/bpf: add testcase for TRACING with 6+ arguments Date: Tue, 27 Jun 2023 19:53:19 +0800 Message-Id: <20230627115319.13128-4-imagedong@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230627115319.13128-1-imagedong@tencent.com> References: <20230627115319.13128-1-imagedong@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769859733261426624?= X-GMAIL-MSGID: =?utf-8?q?1769859733261426624?= From: Menglong Dong Add fentry_many_args.c and fexit_many_args.c to test the fentry/fexit with 7/11 arguments. As this feature is not supported by arm64 yet, we disable these testcases for arm64 in DENYLIST.aarch64. We can combine them with fentry_test.c/fexit_test.c when arm64 is supported too. Correspondingly, add bpf_testmod_fentry_test7() and bpf_testmod_fentry_test11() to bpf_testmod.c Meanwhile, add bpf_modify_return_test2() to test_run.c to test the MODIFY_RETURN with 7 arguments. Add bpf_testmod_test_struct_arg_7/bpf_testmod_test_struct_arg_7 in bpf_testmod.c to test the struct in the arguments. And the testcases passed on x86_64: ./test_progs -t fexit Summary: 5/14 PASSED, 0 SKIPPED, 0 FAILED ./test_progs -t fentry Summary: 3/2 PASSED, 0 SKIPPED, 0 FAILED ./test_progs -t modify_return Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED ./test_progs -t tracing_struct Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Menglong Dong Acked-by: Yonghong Song --- v7: - split the testcases, and add fentry_many_args/fexit_many_args to DENYLIST.aarch64 v6: - add testcases to tracing_struct.c instead of fentry_test.c and fexit_test.c v5: - add testcases for MODIFY_RETURN v4: - use different type for args in bpf_testmod_fentry_test{7,12} - add testcase for grabage values in ctx v3: - move bpf_fentry_test{7,12} to bpf_testmod.c and rename them to bpf_testmod_fentry_test{7,12} meanwhile - get return value by bpf_get_func_ret() in "fexit/bpf_testmod_fentry_test12", as we don't change ___bpf_ctx_cast() in this version --- net/bpf/test_run.c | 23 ++++++-- tools/testing/selftests/bpf/DENYLIST.aarch64 | 2 + .../selftests/bpf/bpf_testmod/bpf_testmod.c | 49 ++++++++++++++++- .../selftests/bpf/prog_tests/fentry_test.c | 43 +++++++++++++-- .../selftests/bpf/prog_tests/fexit_test.c | 43 +++++++++++++-- .../selftests/bpf/prog_tests/modify_return.c | 20 ++++++- .../selftests/bpf/prog_tests/tracing_struct.c | 19 +++++++ .../selftests/bpf/progs/fentry_many_args.c | 39 ++++++++++++++ .../selftests/bpf/progs/fexit_many_args.c | 40 ++++++++++++++ .../selftests/bpf/progs/modify_return.c | 40 ++++++++++++++ .../selftests/bpf/progs/tracing_struct.c | 54 +++++++++++++++++++ 11 files changed, 358 insertions(+), 14 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/fentry_many_args.c create mode 100644 tools/testing/selftests/bpf/progs/fexit_many_args.c diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index 2321bd2f9964..df58e8bf5e07 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -561,6 +561,13 @@ __bpf_kfunc int bpf_modify_return_test(int a, int *b) return a + *b; } +__bpf_kfunc int bpf_modify_return_test2(int a, int *b, short c, int d, + void *e, char f, int g) +{ + *b += 1; + return a + *b + c + d + (long)e + f + g; +} + int noinline bpf_fentry_shadow_test(int a) { return a + 1; @@ -596,9 +603,13 @@ __diag_pop(); BTF_SET8_START(bpf_test_modify_return_ids) BTF_ID_FLAGS(func, bpf_modify_return_test) +BTF_ID_FLAGS(func, bpf_modify_return_test2) BTF_ID_FLAGS(func, bpf_fentry_test1, KF_SLEEPABLE) BTF_SET8_END(bpf_test_modify_return_ids) +BTF_ID_LIST(bpf_modify_return_test_id) +BTF_ID(func, bpf_modify_return_test) + static const struct btf_kfunc_id_set bpf_test_modify_return_set = { .owner = THIS_MODULE, .set = &bpf_test_modify_return_ids, @@ -661,9 +672,15 @@ int bpf_prog_test_run_tracing(struct bpf_prog *prog, goto out; break; case BPF_MODIFY_RETURN: - ret = bpf_modify_return_test(1, &b); - if (b != 2) - side_effect = 1; + if (prog->aux->attach_btf_id == *bpf_modify_return_test_id) { + ret = bpf_modify_return_test(1, &b); + if (b != 2) + side_effect = 1; + } else { + ret = bpf_modify_return_test2(1, &b, 3, 4, (void *)5, 6, 7); + if (b != 2) + side_effect = 1; + } break; default: goto out; diff --git a/tools/testing/selftests/bpf/DENYLIST.aarch64 b/tools/testing/selftests/bpf/DENYLIST.aarch64 index 08adc805878b..3b61e8b35d62 100644 --- a/tools/testing/selftests/bpf/DENYLIST.aarch64 +++ b/tools/testing/selftests/bpf/DENYLIST.aarch64 @@ -10,3 +10,5 @@ kprobe_multi_test/link_api_addrs # link_fd unexpected link_fd: a kprobe_multi_test/link_api_syms # link_fd unexpected link_fd: actual -95 < expected 0 kprobe_multi_test/skel_api # libbpf: failed to load BPF skeleton 'kprobe_multi': -3 module_attach # prog 'kprobe_multi': failed to auto-attach: -95 +fentry_test/fentry_many_args # fentry_many_args:FAIL:fentry_many_args_attach unexpected error: -524 +fexit_test/fexit_many_args # fexit_many_args:FAIL:fexit_many_args_attach unexpected error: -524 diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index aaf6ef1201c7..a6f991b56345 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -34,6 +34,11 @@ struct bpf_testmod_struct_arg_3 { int b[]; }; +struct bpf_testmod_struct_arg_4 { + u64 a; + int b; +}; + __diag_push(); __diag_ignore_all("-Wmissing-prototypes", "Global functions as their definitions will be in bpf_testmod.ko BTF"); @@ -75,6 +80,24 @@ bpf_testmod_test_struct_arg_6(struct bpf_testmod_struct_arg_3 *a) { return bpf_testmod_test_struct_arg_result; } +noinline int +bpf_testmod_test_struct_arg_7(u64 a, void *b, short c, int d, void *e, + struct bpf_testmod_struct_arg_4 f) +{ + bpf_testmod_test_struct_arg_result = a + (long)b + c + d + + (long)e + f.a + f.b; + return bpf_testmod_test_struct_arg_result; +} + +noinline int +bpf_testmod_test_struct_arg_8(u64 a, void *b, short c, int d, void *e, + struct bpf_testmod_struct_arg_4 f, int g) +{ + bpf_testmod_test_struct_arg_result = a + (long)b + c + d + + (long)e + f.a + f.b + g; + return bpf_testmod_test_struct_arg_result; +} + __bpf_kfunc void bpf_testmod_test_mod_kfunc(int i) { @@ -191,6 +214,20 @@ noinline int bpf_testmod_fentry_test3(char a, int b, u64 c) return a + b + c; } +noinline int bpf_testmod_fentry_test7(u64 a, void *b, short c, int d, + void *e, char f, int g) +{ + return a + (long)b + c + d + (long)e + f + g; +} + +noinline int bpf_testmod_fentry_test11(u64 a, void *b, short c, int d, + void *e, char f, int g, + unsigned int h, long i, __u64 j, + unsigned long k) +{ + return a + (long)b + c + d + (long)e + f + g + h + i + j + k; +} + int bpf_testmod_fentry_ok; noinline ssize_t @@ -206,6 +243,7 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, struct bpf_testmod_struct_arg_1 struct_arg1 = {10}; struct bpf_testmod_struct_arg_2 struct_arg2 = {2, 3}; struct bpf_testmod_struct_arg_3 *struct_arg3; + struct bpf_testmod_struct_arg_4 struct_arg4 = {21, 22}; int i = 1; while (bpf_testmod_return_ptr(i)) @@ -216,6 +254,11 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, (void)bpf_testmod_test_struct_arg_3(1, 4, struct_arg2); (void)bpf_testmod_test_struct_arg_4(struct_arg1, 1, 2, 3, struct_arg2); (void)bpf_testmod_test_struct_arg_5(); + (void)bpf_testmod_test_struct_arg_7(16, (void *)17, 18, 19, + (void *)20, struct_arg4); + (void)bpf_testmod_test_struct_arg_8(16, (void *)17, 18, 19, + (void *)20, struct_arg4, 23); + struct_arg3 = kmalloc((sizeof(struct bpf_testmod_struct_arg_3) + sizeof(int)), GFP_KERNEL); @@ -243,7 +286,11 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, if (bpf_testmod_fentry_test1(1) != 2 || bpf_testmod_fentry_test2(2, 3) != 5 || - bpf_testmod_fentry_test3(4, 5, 6) != 15) + bpf_testmod_fentry_test3(4, 5, 6) != 15 || + bpf_testmod_fentry_test7(16, (void *)17, 18, 19, (void *)20, + 21, 22) != 133 || + bpf_testmod_fentry_test11(16, (void *)17, 18, 19, (void *)20, + 21, 22, 23, 24, 25, 26) != 231) goto out; bpf_testmod_fentry_ok = 1; diff --git a/tools/testing/selftests/bpf/prog_tests/fentry_test.c b/tools/testing/selftests/bpf/prog_tests/fentry_test.c index c0d1d61d5f66..aee1bc77a17f 100644 --- a/tools/testing/selftests/bpf/prog_tests/fentry_test.c +++ b/tools/testing/selftests/bpf/prog_tests/fentry_test.c @@ -2,8 +2,9 @@ /* Copyright (c) 2019 Facebook */ #include #include "fentry_test.lskel.h" +#include "fentry_many_args.skel.h" -static int fentry_test(struct fentry_test_lskel *fentry_skel) +static int fentry_test_common(struct fentry_test_lskel *fentry_skel) { int err, prog_fd, i; int link_fd; @@ -37,7 +38,7 @@ static int fentry_test(struct fentry_test_lskel *fentry_skel) return 0; } -void test_fentry_test(void) +static void fentry_test(void) { struct fentry_test_lskel *fentry_skel = NULL; int err; @@ -46,13 +47,47 @@ void test_fentry_test(void) if (!ASSERT_OK_PTR(fentry_skel, "fentry_skel_load")) goto cleanup; - err = fentry_test(fentry_skel); + err = fentry_test_common(fentry_skel); if (!ASSERT_OK(err, "fentry_first_attach")) goto cleanup; - err = fentry_test(fentry_skel); + err = fentry_test_common(fentry_skel); ASSERT_OK(err, "fentry_second_attach"); cleanup: fentry_test_lskel__destroy(fentry_skel); } + +static void fentry_many_args(void) +{ + struct fentry_many_args *fentry_skel = NULL; + int err; + + fentry_skel = fentry_many_args__open_and_load(); + if (!ASSERT_OK_PTR(fentry_skel, "fentry_many_args_skel_load")) + goto cleanup; + + err = fentry_many_args__attach(fentry_skel); + if (!ASSERT_OK(err, "fentry_many_args_attach")) + goto cleanup; + + ASSERT_OK(trigger_module_test_read(1), "trigger_read"); + + ASSERT_EQ(fentry_skel->bss->test1_result, 1, + "fentry_many_args_result1"); + ASSERT_EQ(fentry_skel->bss->test2_result, 1, + "fentry_many_args_result2"); + ASSERT_EQ(fentry_skel->bss->test3_result, 1, + "fentry_many_args_result3"); + +cleanup: + fentry_many_args__destroy(fentry_skel); +} + +void test_fentry_test(void) +{ + if (test__start_subtest("fentry")) + fentry_test(); + if (test__start_subtest("fentry_many_args")) + fentry_many_args(); +} diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_test.c b/tools/testing/selftests/bpf/prog_tests/fexit_test.c index 101b7343036b..1c13007e37dd 100644 --- a/tools/testing/selftests/bpf/prog_tests/fexit_test.c +++ b/tools/testing/selftests/bpf/prog_tests/fexit_test.c @@ -2,8 +2,9 @@ /* Copyright (c) 2019 Facebook */ #include #include "fexit_test.lskel.h" +#include "fexit_many_args.skel.h" -static int fexit_test(struct fexit_test_lskel *fexit_skel) +static int fexit_test_common(struct fexit_test_lskel *fexit_skel) { int err, prog_fd, i; int link_fd; @@ -37,7 +38,7 @@ static int fexit_test(struct fexit_test_lskel *fexit_skel) return 0; } -void test_fexit_test(void) +static void fexit_test(void) { struct fexit_test_lskel *fexit_skel = NULL; int err; @@ -46,13 +47,47 @@ void test_fexit_test(void) if (!ASSERT_OK_PTR(fexit_skel, "fexit_skel_load")) goto cleanup; - err = fexit_test(fexit_skel); + err = fexit_test_common(fexit_skel); if (!ASSERT_OK(err, "fexit_first_attach")) goto cleanup; - err = fexit_test(fexit_skel); + err = fexit_test_common(fexit_skel); ASSERT_OK(err, "fexit_second_attach"); cleanup: fexit_test_lskel__destroy(fexit_skel); } + +static void fexit_many_args(void) +{ + struct fexit_many_args *fexit_skel = NULL; + int err; + + fexit_skel = fexit_many_args__open_and_load(); + if (!ASSERT_OK_PTR(fexit_skel, "fexit_many_args_skel_load")) + goto cleanup; + + err = fexit_many_args__attach(fexit_skel); + if (!ASSERT_OK(err, "fexit_many_args_attach")) + goto cleanup; + + ASSERT_OK(trigger_module_test_read(1), "trigger_read"); + + ASSERT_EQ(fexit_skel->bss->test1_result, 1, + "fexit_many_args_result1"); + ASSERT_EQ(fexit_skel->bss->test2_result, 1, + "fexit_many_args_result2"); + ASSERT_EQ(fexit_skel->bss->test3_result, 1, + "fexit_many_args_result3"); + +cleanup: + fexit_many_args__destroy(fexit_skel); +} + +void test_fexit_test(void) +{ + if (test__start_subtest("fexit")) + fexit_test(); + if (test__start_subtest("fexit_many_args")) + fexit_many_args(); +} diff --git a/tools/testing/selftests/bpf/prog_tests/modify_return.c b/tools/testing/selftests/bpf/prog_tests/modify_return.c index 5d9955af6247..93febb6d81ef 100644 --- a/tools/testing/selftests/bpf/prog_tests/modify_return.c +++ b/tools/testing/selftests/bpf/prog_tests/modify_return.c @@ -11,7 +11,8 @@ #define UPPER(x) ((x) >> 16) -static void run_test(__u32 input_retval, __u16 want_side_effect, __s16 want_ret) +static void run_test(__u32 input_retval, __u16 want_side_effect, + __s16 want_ret, __s16 want_ret2) { struct modify_return *skel = NULL; int err, prog_fd; @@ -41,6 +42,19 @@ static void run_test(__u32 input_retval, __u16 want_side_effect, __s16 want_ret) ASSERT_EQ(skel->bss->fexit_result, 1, "modify_return fexit_result"); ASSERT_EQ(skel->bss->fmod_ret_result, 1, "modify_return fmod_ret_result"); + prog_fd = bpf_program__fd(skel->progs.fmod_ret_test2); + err = bpf_prog_test_run_opts(prog_fd, &topts); + ASSERT_OK(err, "test_run"); + + side_effect = UPPER(topts.retval); + ret = LOWER(topts.retval); + + ASSERT_EQ(ret, want_ret2, "test_run ret2"); + ASSERT_EQ(side_effect, want_side_effect, "modify_return side_effect2"); + ASSERT_EQ(skel->bss->fentry_result2, 1, "modify_return fentry_result2"); + ASSERT_EQ(skel->bss->fexit_result2, 1, "modify_return fexit_result2"); + ASSERT_EQ(skel->bss->fmod_ret_result2, 1, "modify_return fmod_ret_result2"); + cleanup: modify_return__destroy(skel); } @@ -50,8 +64,10 @@ void serial_test_modify_return(void) { run_test(0 /* input_retval */, 1 /* want_side_effect */, - 4 /* want_ret */); + 4 /* want_ret */, + 29 /* want_ret */); run_test(-EINVAL /* input_retval */, 0 /* want_side_effect */, + -EINVAL /* want_ret */, -EINVAL /* want_ret */); } diff --git a/tools/testing/selftests/bpf/prog_tests/tracing_struct.c b/tools/testing/selftests/bpf/prog_tests/tracing_struct.c index 1c75a32186d6..fe0fb0c9849a 100644 --- a/tools/testing/selftests/bpf/prog_tests/tracing_struct.c +++ b/tools/testing/selftests/bpf/prog_tests/tracing_struct.c @@ -55,6 +55,25 @@ static void test_fentry(void) ASSERT_EQ(skel->bss->t6, 1, "t6 ret"); + ASSERT_EQ(skel->bss->t7_a, 16, "t7:a"); + ASSERT_EQ(skel->bss->t7_b, 17, "t7:b"); + ASSERT_EQ(skel->bss->t7_c, 18, "t7:c"); + ASSERT_EQ(skel->bss->t7_d, 19, "t7:d"); + ASSERT_EQ(skel->bss->t7_e, 20, "t7:e"); + ASSERT_EQ(skel->bss->t7_f_a, 21, "t7:f.a"); + ASSERT_EQ(skel->bss->t7_f_b, 22, "t7:f.b"); + ASSERT_EQ(skel->bss->t7_ret, 133, "t7 ret"); + + ASSERT_EQ(skel->bss->t8_a, 16, "t8:a"); + ASSERT_EQ(skel->bss->t8_b, 17, "t8:b"); + ASSERT_EQ(skel->bss->t8_c, 18, "t8:c"); + ASSERT_EQ(skel->bss->t8_d, 19, "t8:d"); + ASSERT_EQ(skel->bss->t8_e, 20, "t8:e"); + ASSERT_EQ(skel->bss->t8_f_a, 21, "t8:f.a"); + ASSERT_EQ(skel->bss->t8_f_b, 22, "t8:f.b"); + ASSERT_EQ(skel->bss->t8_g, 23, "t8:g"); + ASSERT_EQ(skel->bss->t8_ret, 156, "t8 ret"); + tracing_struct__detach(skel); destroy_skel: tracing_struct__destroy(skel); diff --git a/tools/testing/selftests/bpf/progs/fentry_many_args.c b/tools/testing/selftests/bpf/progs/fentry_many_args.c new file mode 100644 index 000000000000..b61bb92fee2c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/fentry_many_args.c @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Tencent */ +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__u64 test1_result = 0; +SEC("fentry/bpf_testmod_fentry_test7") +int BPF_PROG(test1, __u64 a, void *b, short c, int d, void *e, char f, + int g) +{ + test1_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22; + return 0; +} + +__u64 test2_result = 0; +SEC("fentry/bpf_testmod_fentry_test11") +int BPF_PROG(test2, __u64 a, void *b, short c, int d, void *e, char f, + int g, unsigned int h, long i, __u64 j, unsigned long k) +{ + test2_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26; + return 0; +} + +__u64 test3_result = 0; +SEC("fentry/bpf_testmod_fentry_test11") +int BPF_PROG(test3, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, + __u64 g, __u64 h, __u64 i, __u64 j, __u64 k) +{ + test3_result = a == 16 && b == 17 && c == 18 && d == 19 && + e == 20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26; + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/fexit_many_args.c b/tools/testing/selftests/bpf/progs/fexit_many_args.c new file mode 100644 index 000000000000..53b335c2dafb --- /dev/null +++ b/tools/testing/selftests/bpf/progs/fexit_many_args.c @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Tencent */ +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__u64 test1_result = 0; +SEC("fexit/bpf_testmod_fentry_test7") +int BPF_PROG(test1, __u64 a, void *b, short c, int d, void *e, char f, + int g, int ret) +{ + test1_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22 && ret == 133; + return 0; +} + +__u64 test2_result = 0; +SEC("fexit/bpf_testmod_fentry_test11") +int BPF_PROG(test2, __u64 a, void *b, short c, int d, void *e, char f, + int g, unsigned int h, long i, __u64 j, unsigned long k, + int ret) +{ + test2_result = a == 16 && b == (void *)17 && c == 18 && d == 19 && + e == (void *)20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26 && ret == 231; + return 0; +} + +__u64 test3_result = 0; +SEC("fexit/bpf_testmod_fentry_test11") +int BPF_PROG(test3, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, + __u64 g, __u64 h, __u64 i, __u64 j, __u64 k, __u64 ret) +{ + test3_result = a == 16 && b == 17 && c == 18 && d == 19 && + e == 20 && f == 21 && g == 22 && h == 23 && + i == 24 && j == 25 && k == 26 && ret == 231; + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/modify_return.c b/tools/testing/selftests/bpf/progs/modify_return.c index 8b7466a15c6b..3376d4849f58 100644 --- a/tools/testing/selftests/bpf/progs/modify_return.c +++ b/tools/testing/selftests/bpf/progs/modify_return.c @@ -47,3 +47,43 @@ int BPF_PROG(fexit_test, int a, __u64 b, int ret) return 0; } + +static int sequence2; + +__u64 fentry_result2 = 0; +SEC("fentry/bpf_modify_return_test2") +int BPF_PROG(fentry_test2, int a, int *b, short c, int d, void *e, char f, + int g) +{ + sequence2++; + fentry_result2 = (sequence2 == 1); + return 0; +} + +__u64 fmod_ret_result2 = 0; +SEC("fmod_ret/bpf_modify_return_test2") +int BPF_PROG(fmod_ret_test2, int a, int *b, short c, int d, void *e, char f, + int g, int ret) +{ + sequence2++; + /* This is the first fmod_ret program, the ret passed should be 0 */ + fmod_ret_result2 = (sequence2 == 2 && ret == 0); + return input_retval; +} + +__u64 fexit_result2 = 0; +SEC("fexit/bpf_modify_return_test2") +int BPF_PROG(fexit_test2, int a, int *b, short c, int d, void *e, char f, + int g, int ret) +{ + sequence2++; + /* If the input_reval is non-zero a successful modification should have + * occurred. + */ + if (input_retval) + fexit_result2 = (sequence2 == 3 && ret == input_retval); + else + fexit_result2 = (sequence2 == 3 && ret == 29); + + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/tracing_struct.c b/tools/testing/selftests/bpf/progs/tracing_struct.c index c435a3a8328a..515daef3c84b 100644 --- a/tools/testing/selftests/bpf/progs/tracing_struct.c +++ b/tools/testing/selftests/bpf/progs/tracing_struct.c @@ -18,6 +18,11 @@ struct bpf_testmod_struct_arg_3 { int b[]; }; +struct bpf_testmod_struct_arg_4 { + u64 a; + int b; +}; + long t1_a_a, t1_a_b, t1_b, t1_c, t1_ret, t1_nregs; __u64 t1_reg0, t1_reg1, t1_reg2, t1_reg3; long t2_a, t2_b_a, t2_b_b, t2_c, t2_ret; @@ -25,6 +30,9 @@ long t3_a, t3_b, t3_c_a, t3_c_b, t3_ret; long t4_a_a, t4_b, t4_c, t4_d, t4_e_a, t4_e_b, t4_ret; long t5_ret; int t6; +long t7_a, t7_b, t7_c, t7_d, t7_e, t7_f_a, t7_f_b, t7_ret; +long t8_a, t8_b, t8_c, t8_d, t8_e, t8_f_a, t8_f_b, t8_g, t8_ret; + SEC("fentry/bpf_testmod_test_struct_arg_1") int BPF_PROG2(test_struct_arg_1, struct bpf_testmod_struct_arg_2, a, int, b, int, c) @@ -130,4 +138,50 @@ int BPF_PROG2(test_struct_arg_11, struct bpf_testmod_struct_arg_3 *, a) return 0; } +SEC("fentry/bpf_testmod_test_struct_arg_7") +int BPF_PROG2(test_struct_arg_12, __u64, a, void *, b, short, c, int, d, + void *, e, struct bpf_testmod_struct_arg_4, f) +{ + t7_a = a; + t7_b = (long)b; + t7_c = c; + t7_d = d; + t7_e = (long)e; + t7_f_a = f.a; + t7_f_b = f.b; + return 0; +} + +SEC("fexit/bpf_testmod_test_struct_arg_7") +int BPF_PROG2(test_struct_arg_13, __u64, a, void *, b, short, c, int, d, + void *, e, struct bpf_testmod_struct_arg_4, f, int, ret) +{ + t7_ret = ret; + return 0; +} + +SEC("fentry/bpf_testmod_test_struct_arg_8") +int BPF_PROG2(test_struct_arg_14, __u64, a, void *, b, short, c, int, d, + void *, e, struct bpf_testmod_struct_arg_4, f, int, g) +{ + t8_a = a; + t8_b = (long)b; + t8_c = c; + t8_d = d; + t8_e = (long)e; + t8_f_a = f.a; + t8_f_b = f.b; + t8_g = g; + return 0; +} + +SEC("fexit/bpf_testmod_test_struct_arg_8") +int BPF_PROG2(test_struct_arg_15, __u64, a, void *, b, short, c, int, d, + void *, e, struct bpf_testmod_struct_arg_4, f, int, g, + int, ret) +{ + t8_ret = ret; + return 0; +} + char _license[] SEC("license") = "GPL";