From patchwork Sun Dec 18 05:03:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "wuqiang.matt" X-Patchwork-Id: 3064 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1789650wrn; Sat, 17 Dec 2022 21:05:33 -0800 (PST) X-Google-Smtp-Source: AA0mqf6AB2SbF99gZgpnJyWwyO6qTPB86qCxHJMhfpfWG4baXMNw83xCm+D5qzHo70M27VSnQL6H X-Received: by 2002:a17:906:6c7:b0:78d:f455:b5dc with SMTP id v7-20020a17090606c700b0078df455b5dcmr30139048ejb.28.1671339933712; Sat, 17 Dec 2022 21:05:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671339933; cv=none; d=google.com; s=arc-20160816; b=Ro/6gel2XPndfhI6lG9mRKaNC5FWPlkAdqkh5xs8SQ6puREyhFNt8MDPiCcCqXYkx3 jU+yPYdCprgCVblpWITDSbRbyIKM9QQg3Tjhzki93kdzohhVp/0Ko/Mt4tPKna9g5s96 0qN3vsk2OtJElSICu1wy+4W6aCtUW4NgAagpzW0KkKiMHLoMz8NvnqQz1QJ2meyvZwmF AiPDlTvBdBdr4TGJ2kbVJPapDpoyr5SJk22TRDN/TIiL2MNODws4+ZLL1OEkfQ0lW8HA WqPd+SD00TLugBVravrFujvKocliHScoiKvQaBZZZPjs9fLt+bsXoSDCjOMJmx2WSveU 6+wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=inXEajzLzHrYa1MiEvHFJL5aAucCwzDSfIbGNe8cXzI=; b=tiSBXqPFo4CrHQPA3ioWKQPOjAnBdEODEbHN7AKfS9J1ccmE1C5Iav54xmfEgbcYao V1a83Sb14eQYZkNuTsEmYfVdPd1tkGnKbVrMMlIIt8bAwB2W9OQ5lB1r+2ff9p3Sh7jE 19lEr7YhqLag+byCnIuM3YmMHtPUiS/64Q6cc5zZX68chOCVYPFrBJIXxE8cMTowzs+Z cUIoTqXhmA7KTK9R1SkdurEH9ZvOYABV1LLXHwfXy4XnFAvfq7UbUSjdllKO50Tg9Ify yiNE1FtQpIMl26H5jI1XWAUx6EOpAbUBCuFE5+2gpDKLIxLJq+PvDvjEnSAlqpM49CJu PPDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=VLfCqZFm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sb27-20020a1709076d9b00b007aa493b067asi7035597ejc.396.2022.12.17.21.05.10; Sat, 17 Dec 2022 21:05:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=VLfCqZFm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230166AbiLRFEN (ORCPT + 99 others); Sun, 18 Dec 2022 00:04:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229693AbiLRFEG (ORCPT ); Sun, 18 Dec 2022 00:04:06 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8151CE19 for ; Sat, 17 Dec 2022 21:03:40 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id v23so889512pju.3 for ; Sat, 17 Dec 2022 21:03:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=inXEajzLzHrYa1MiEvHFJL5aAucCwzDSfIbGNe8cXzI=; b=VLfCqZFmlOW7xCOi34/MigZHTURtFYXxqVaVzshTbyjHAxpChZSc18esacXFpc3OvR wMfmU+B1IBMPsflH0izQL+vnyW7hJectNmU8FrSgV84jHL8980OJma3CVbw/h6H+kvf1 nw3hkEgdyG4wLBwNcYh1uRjQwHvFUVL65ITt9NVVjZb+/oCTF3H24OyPbF1QNwWenRuj G5uWok2LTXKv9PWcv8AZ4Ca77qN3b7tX4iWg6eWYBy44V8JGxNsv3VbaXP5OxIR1ReE0 J2dac8MOIFI/XRpFioybwfeAU3PYtvwg8TePok2FiLcq3slh+C4h6YNN9kuk49LUeCLq 6nKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=inXEajzLzHrYa1MiEvHFJL5aAucCwzDSfIbGNe8cXzI=; b=072HdspjYuii0NX/jWt6fBvRGtkmz/ctR3THrH40BFdpfx8kVgJ7k8G5276KASEZQr mr0uhB4AscDCCsqoLAGI2nd5yH2ROolgYg73VSJdW0piYOMERgvpJq52mOGfq+QqkIfD QH/xs3ezBGch2XjXz1OGX62PxxizqHlLL3N0arQrBdmdMIh7sg2Tj7MObKnsNB8XVero f5AyiqdZwpy1YLCrUzkKoFd1SRryJfqcRaSB+xUFm3iPn7Wa4RREFlwn/twccPZ3ub7o T3d7jtZyA3PnJ+dYaeTTcJc0MN7j2e0NJxPH/dOrLeb8kOtzhxICjR6JrdkKTr4/i4xO EV5Q== X-Gm-Message-State: ANoB5pnuYhJ+RpL5c0uOelsTH5uk9FVl8/mXRec5Uu6V/H7ekDJCVphc HqE4uI97aVAFW26jJIkQ4KkFmg== X-Received: by 2002:a17:902:d4c3:b0:189:c322:df3a with SMTP id o3-20020a170902d4c300b00189c322df3amr53080097plg.43.1671339820146; Sat, 17 Dec 2022 21:03:40 -0800 (PST) Received: from devtp.bytedance.net ([139.177.225.227]) by smtp.gmail.com with ESMTPSA id v20-20020a170902ca9400b001708c4ebbaesm4339348pld.309.2022.12.17.21.03.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 17 Dec 2022 21:03:39 -0800 (PST) From: wuqiang To: mhiramat@kernel.org, davem@davemloft.net, anil.s.keshavamurthy@intel.com, naveen.n.rao@linux.ibm.com, rostedt@goodmis.org, peterz@infradead.org, akpm@linux-foundation.org, sander@svanheule.net, ebiggers@google.com, dan.j.williams@intel.com, jpoimboe@kernel.org Cc: linux-kernel@vger.kernel.org, lkp@intel.com, mattwu@163.com, wuqiang Subject: [PATCH v8 0/5] lib,kprobes: kretprobe scalability improvement Date: Sun, 18 Dec 2022 13:03:05 +0800 Message-Id: <20221218050310.1338630-1-wuqiang.matt@bytedance.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752526942132924771?= X-GMAIL-MSGID: =?utf-8?q?1752526942132924771?= This patch series introduces a scalable and lockless ring-array based object pool and replaces the original freelist (a LIFO queue based on singly linked list) to improve scalability of kretprobed routines. v8: 1) objpool: refcount added for objpool lifecycle management v7: 1) objpool: implementation simplified as Masami advised 2) rethook_alloc: error codes returning supported (ERR_PTR) 3) MAINTAINERS: support added for objpool files 4) synced to latest 6.1 with x86_64/x86/aarch64 verified wuqiang (5): lib: objpool added: ring-array based lockless MPMC queue lib: objpool test module added kprobes: kretprobe scalability improvement with objpool kprobes: freelist.h removed MAINTAINERS: objpool added MAINTAINERS | 7 + include/linux/freelist.h | 129 -------- include/linux/kprobes.h | 11 +- include/linux/objpool.h | 116 +++++++ include/linux/rethook.h | 16 +- kernel/kprobes.c | 93 +++--- kernel/trace/fprobe.c | 37 +-- kernel/trace/rethook.c | 91 +++--- lib/Kconfig.debug | 11 + lib/Makefile | 4 +- lib/objpool.c | 372 +++++++++++++++++++++ lib/test_objpool.c | 682 +++++++++++++++++++++++++++++++++++++++ 12 files changed, 1292 insertions(+), 277 deletions(-) delete mode 100644 include/linux/freelist.h create mode 100644 include/linux/objpool.h create mode 100644 lib/objpool.c create mode 100644 lib/test_objpool.c