From patchwork Mon Nov 13 10:52:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 16501 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b909:0:b0:403:3b70:6f57 with SMTP id t9csp1120359vqg; Mon, 13 Nov 2023 02:53:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IGfaOBCGxtE8rej2JtTjE7CfWZswVol9e1zu6Xeb34gTg2hkTJTTCDL2deiVZa1+mWyU/h2 X-Received: by 2002:a05:6871:64f:b0:1ea:3210:3b5d with SMTP id x15-20020a056871064f00b001ea32103b5dmr9845160oan.40.1699872798869; Mon, 13 Nov 2023 02:53:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699872798; cv=none; d=google.com; s=arc-20160816; b=r0uleHMpxxag34SoTGZUDEQ6mFStOMFRh/jImZf1bLrqdGqhiyXTqFJtWjty2ZuN2v IOsXnFcQPgmosTQ3FPMYuyUPU4Ss3uyk0zxPveKOYCNrFDV8unBnwKxsfNSF9lebJ8/Z 23cAJCQBqClTgfp7/wbZBKqqOR8KDl1awGroz/GPHtY52Z0N0p8G/CV5tVLw0dJ6hAJB ZZtr8R87u5+Pa3Ff3D8tg51QGrha+VUcasVxRts6z8RSEWtejyoWn1YQrkpeFtUpyglv vKp5fYHBx6+0i+YbWB2KD+Vd1fppKJZQvLQ5djWd1I/Ix3iuR8DZZi7vArA4/JzvDj6k Qldw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=KZBgUsc1a+sMxZldszdOQr8LgJQCZkrcquIQgM/3+8M=; fh=bwsiIJU4ZySHCYgdbJrawkDYkx7XsqbTjYxIEnch01E=; b=viZG1ebBG5b5AgZ1ZApoxIz/NWUIElcZasPmsxsLgWbcp6iM4C2p7npAnhrM1GPywd A++haPyl2O5rgtxV7455P/mZUzWbJ6FFecLbxt1HitxWQHCA+H/QOzI0kDoBBCDJX+JA X8ZX+1obKDstNdgkw44Usl2JqyCLXEy2+eLPJO2kEqenTvAme5Efh88uWA7XBfEUGnmV 68BYYHli4A/VTzNOmv6+k1TS9FmS1Xd7gaXdCryyaiMHZfMjuPqgYSNmBY16rE5tXORe +8p57HkvP97SKcLJGhu5vkTl7PJuEl/fjnpuyYi3FSBrwCvs1kCrQ0bBXW4EmxBVCoRO Sk7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=PLigGecI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id u15-20020aa7848f000000b006b61871ae27si5276776pfn.367.2023.11.13.02.53.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Nov 2023 02:53:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=PLigGecI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 494838098FE9; Mon, 13 Nov 2023 02:53:01 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229633AbjKMKwp (ORCPT + 29 others); Mon, 13 Nov 2023 05:52:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229459AbjKMKwo (ORCPT ); Mon, 13 Nov 2023 05:52:44 -0500 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 000B210CC for ; Mon, 13 Nov 2023 02:52:37 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5afe220cadeso64797437b3.3 for ; Mon, 13 Nov 2023 02:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1699872757; x=1700477557; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=KZBgUsc1a+sMxZldszdOQr8LgJQCZkrcquIQgM/3+8M=; b=PLigGecIwx3SJjZmxuhEQ+bslIsr7bkmKspF1iIpHF8onVlmstn3JC+2BhS4XO28ZR ld/qth1Rfh674ILvMBlsLPB7uQgQqPSC5g8oo2hJ3qQMAN/zS1zhlZT0jzz3CcyVIIIN mXjNnt9DFxTiB48zsmhiMV2zRLF20LUnHtaiw2xmr47/17wLa5pCPYvsF3FCK0IksuH7 7XOBsGufa7+heORlHfqsQyzd9+wwwUx26Jy9wjsoG5U7jEaw+CelezsPQQJjq0bgsh2x 6d6QlqzbQhtpCRj0/G/okDTOdpsKMcOnCdyWRajxFDZs2I/BbbUik1d7pHiFf/oHNNpc eiwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699872757; x=1700477557; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=KZBgUsc1a+sMxZldszdOQr8LgJQCZkrcquIQgM/3+8M=; b=dnioSrE4vG4ppDLureNXtattoOCpBzMEn2Khm1xbFA5YdQCGF2UENvbAuOJeVSsclV B1pC5T1kth4kRwG599ViVyjIyeJNFhfQdZRThcMiGQNvPiZUF/c7DYC9y9sE7dRuJPwm pip4wIYLiIvSArbdU+hnSWKy5b+jFRxexSt4z7rncHnR5l3pLuhqRpoNYacXfr7pish1 iKh5517sGGTyBDQ6DItgez6DBDdTqiIGMhQNgKnXInOaYQks/laSpVsVB1w3TwMeude0 sGJgAjyW2OZ31LkczpUuYACAK1s2wilnlmcF0pschqG18ZDDBcfvWx9MAb+LkrBE7Z+t LtTg== X-Gm-Message-State: AOJu0YwGjd0bDfDDlRAmRqKjWgoVDNKTYZPqq2+AmhGvcHT9mjd2LtAa 75c/J7zDeppUJ3Plg7COx70vbnvKYcM= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:be07:845d:4221:b15e]) (user=glider job=sendgmr) by 2002:a81:430b:0:b0:5a7:be10:461d with SMTP id q11-20020a81430b000000b005a7be10461dmr132113ywa.2.1699872757205; Mon, 13 Nov 2023 02:52:37 -0800 (PST) Date: Mon, 13 Nov 2023 11:52:29 +0100 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog Message-ID: <20231113105234.32058-1-glider@google.com> Subject: [PATCH v9 0/4] Implement MTE tag compression for swapped pages From: Alexander Potapenko To: glider@google.com, catalin.marinas@arm.com, will@kernel.org, pcc@google.com, andreyknvl@gmail.com, andriy.shevchenko@linux.intel.com, aleksander.lobakin@intel.com, linux@rasmusvillemoes.dk, yury.norov@gmail.com, alexandru.elisei@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, eugenis@google.com, syednwaris@gmail.com, william.gray@linaro.org X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 13 Nov 2023 02:53:01 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782445819895227990 X-GMAIL-MSGID: 1782445819895227990 Currently, when MTE pages are swapped out, the tags are kept in the memory, occupying PAGE_SIZE/32 bytes per page. This is especially problematic for devices that use zram-backed in-memory swap, because tags stored uncompressed in the heap effectively reduce the available amount of swap memory. The RLE-based algorithm suggested by Evgenii Stepanov and implemented in this patch series is able to efficiently compress fixed-size tag buffers, resulting in practical compression ratio of 2x. In many cases it is possible to store the compressed data in 63-bit Xarray values, resulting in no extra memory allocations. This patch series depends on "lib/bitmap: add bitmap_{read,write}()" (https://lore.kernel.org/linux-arm-kernel/20231030153210.139512-1-glider@google.com/T/) that is mailed separately. v9: - split off the stats collection code into a separate patch in the series (as suggested by Yury Norov) v8: - split off the bitmap_read()/bitmap_write() series - simplified the compression logic (only compress data if it fits into a pointer) v7: - fixed comments by Yury Norov, Andy Shevchenko, Rasmus Villemoes - added perf tests for bitmap_read()/bitmap_write() - more efficient bitmap_write() implementation (meant to be sent in v5) v6: - fixed comments by Yury Norov - fixed handling of sizes divisible by MTE_GRANULES_PER_PAGE / 2 (caught while testing on a real device) v5: - fixed comments by Andy Shevchenko, Catalin Marinas, and Yury Norov - added support for 16K- and 64K pages - more efficient bitmap_write() implementation v4: - fixed a bunch of comments by Andy Shevchenko and Yury Norov - added Documentation/arch/arm64/mte-tag-compression.rst v3: - as suggested by Andy Shevchenko, use bitmap_get_value()/bitmap_set_value() written by Syed Nayyar Waris - switched to unsigned long to reduce typecasts - simplified the compression code v2: - as suggested by Yuri Norov, replace the poorly implemented struct bitq with Alexander Potapenko (4): arm64: mte: implement CONFIG_ARM64_MTE_COMP arm64: mte: add a test for MTE tags compression arm64: mte: add compression support to mteswap.c arm64: mte: implement CONFIG_ARM64_MTE_SWAP_STATS Documentation/arch/arm64/index.rst | 1 + .../arch/arm64/mte-tag-compression.rst | 166 ++++++++ arch/arm64/Kconfig | 37 ++ arch/arm64/include/asm/mtecomp.h | 39 ++ arch/arm64/mm/Makefile | 2 + arch/arm64/mm/mtecomp.c | 257 +++++++++++++ arch/arm64/mm/mtecomp.h | 12 + arch/arm64/mm/mteswap.c | 110 +++++- arch/arm64/mm/test_mtecomp.c | 364 ++++++++++++++++++ 9 files changed, 985 insertions(+), 3 deletions(-) create mode 100644 Documentation/arch/arm64/mte-tag-compression.rst create mode 100644 arch/arm64/include/asm/mtecomp.h create mode 100644 arch/arm64/mm/mtecomp.c create mode 100644 arch/arm64/mm/mtecomp.h create mode 100644 arch/arm64/mm/test_mtecomp.c