From patchwork Fri Nov 3 16:03:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 16225 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1145195vqu; Fri, 3 Nov 2023 09:06:24 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH69dUpop/MEboXeP9TzcNsi/pvu5X2PMJ2QahObJHA1DF+GE0Lf1FU/6/y/tu80kIFWSna X-Received: by 2002:a05:6870:f21d:b0:1e9:f6c3:8594 with SMTP id t29-20020a056870f21d00b001e9f6c38594mr25069830oao.2.1699027584126; Fri, 03 Nov 2023 09:06:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699027584; cv=none; d=google.com; s=arc-20160816; b=dOCbhlKiHr4VskSntEKXBVCjXRtEZ3lXIwvwayZ84krOTz7KqUWmRHO4679oGiwXoD NdZDa0Sf1iogcWSk8OyaNvTnstDuwar6yerayvHqm3tcMw3jM3ghXe5QvdoybvRNHxmC KQY4Qz0/ykPNYn91EgnocXIyRvepbfAkQ4dY9Q5EKCyEau1ZR3rEBZCk9Csx+CSd8H09 27ceadoZmFaSRqxcWCU2qEkH9uSaZvVEeiJKz/+0tp99uGO66wOGtJ7OwT6WbBI2CS4J 1xJVnzT8saHbK1+E9S6DpjddNXPPMxvkUM+mTxFiihSKwApmZ8eHBQBwrEvUylhuUGm2 Tx/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=gTzXxPv+gObuuXQua2/KBmP4Hxk1HM7FnDHk5jUfbw4=; fh=bwsiIJU4ZySHCYgdbJrawkDYkx7XsqbTjYxIEnch01E=; b=0EzpejV3s1EonwhcVvY8TyCxQPB6Y39cEap/XUi9JE063anSZo2Z8BaslFaIoz2tHK SGWWoAlPrKV+X0+PECM4XXOvKu3q6AyePHCWHwD8z9+koHYHnxcVghWW0E6T58228DJB mF6aAUOqKaJELfcgey85ppPULe7JUaaz4s5nZ/pM4wx3qgiflc0cqdOcpvB4T9RPOY0A QniM/Jp6osVOmAqYV1fu9n2ZXSL66FhsYLItB1grHaYwoDB1ooP7STt/FlLc9WRndeZU YWJ+uBADeYteYl6Y+F38y4osUEzTcU+IdN+mlYe0xoSitpYE3iHboeHlEY6J64NY+XX8 83xg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=bTQRZquv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id ho13-20020a056870498d00b001dd949bec1dsi814220oab.134.2023.11.03.09.06.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 09:06:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=bTQRZquv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 4645B80873CD; Fri, 3 Nov 2023 09:05:31 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345028AbjKCQDr (ORCPT + 35 others); Fri, 3 Nov 2023 12:03:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234376AbjKCQDq (ORCPT ); Fri, 3 Nov 2023 12:03:46 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C5C2CA for ; Fri, 3 Nov 2023 09:03:40 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-da040c021aeso2526962276.3 for ; Fri, 03 Nov 2023 09:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1699027419; x=1699632219; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=gTzXxPv+gObuuXQua2/KBmP4Hxk1HM7FnDHk5jUfbw4=; b=bTQRZquv+FPn1uDggpHuh+TsylPV1ByoIx70jjWT7FW0rcw970VgvYNzA3cvEz051I XmBDNFYBQN9EHjK7V7ByNMyMbvyAfyWVTdYdGcjZx6eGbgvmgoYSsOnrG+/ST3OJphOT DVRxHP/aEiKgjQ9CuRoYKRrubommkYUIUD1gMKPhsFp9GQYqNP/aTnG+L72agxhK6imD P/NE7F28v9Q3jXkQMzFkSDF5J6UtD+LzeTGnKfBxg8NY37sNSPXA2a+uACut0iB7PteK XcAY1I/bD5YI0tgzY+OF9mNQlEOnmpE//0eKmrD7JlpK0BeptMgcFUCYB6tIBqWbQGd8 6F8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699027419; x=1699632219; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=gTzXxPv+gObuuXQua2/KBmP4Hxk1HM7FnDHk5jUfbw4=; b=qkq7RcMvnOk8+R7R0KLrR5EtDjg9Op836j820yl7ZAn/fpGKx6jzaWhLr1xYCXoDn8 Y1lptn7YxMkrIJtm2eb2rmURzPxkQ5MXvzRN7B2tUGt7edQf/PlP7T+uL/O8rQjz53j4 UxOlzT75smGrHoSkCvbNr7MLQRLzwNCnnomKuDD/VTgBSZuPaLhtU9xTeyL05RAad5Jy ll0dL2vJ3NClNHJgElU0CoJbMl5N/+glawUBMyYFa55HrCxKSQPiAyq/15kneWtf1DAY LjDhTgd1+c3gtotVBcH4MsbsdPiwhYmhpvUCjSqmh4nGYSRmNkw7TupiZqSUTEoGwMQo Zg/A== X-Gm-Message-State: AOJu0YxV6XgL/oWBjR8h/MRXsLQ0BbMvCLZQeMBW2+TM70Iyz+t1mfdN kCFbHIo0+d+YvryZmMvcV5+ALyMHL+M= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:74c:1f8e:4661:7aaa]) (user=glider job=sendgmr) by 2002:a25:488:0:b0:da0:5452:29f7 with SMTP id 130-20020a250488000000b00da0545229f7mr425792ybe.4.1699027419431; Fri, 03 Nov 2023 09:03:39 -0700 (PDT) Date: Fri, 3 Nov 2023 17:03:32 +0100 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog Message-ID: <20231103160335.2464561-1-glider@google.com> Subject: [PATCH v8 0/3] Implement MTE tag compression for swapped pages From: Alexander Potapenko To: glider@google.com, catalin.marinas@arm.com, will@kernel.org, pcc@google.com, andreyknvl@gmail.com, andriy.shevchenko@linux.intel.com, aleksander.lobakin@intel.com, linux@rasmusvillemoes.dk, yury.norov@gmail.com, alexandru.elisei@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, eugenis@google.com, syednwaris@gmail.com, william.gray@linaro.org X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 03 Nov 2023 09:05:31 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781559548347939308 X-GMAIL-MSGID: 1781559548347939308 Currently, when MTE pages are swapped out, the tags are kept in the memory, occupying PAGE_SIZE/32 bytes per page. This is especially problematic for devices that use zram-backed in-memory swap, because tags stored uncompressed in the heap effectively reduce the available amount of swap memory. The RLE-based algorithm suggested by Evgenii Stepanov and implemented in this patch series is able to efficiently compress fixed-size tag buffers, resulting in practical compression ratio of 2x. In many cases it is possible to store the compressed data in 63-bit Xarray values, resulting in no extra memory allocations. This patch series depends on "lib/bitmap: add bitmap_{read,write}()" (https://lore.kernel.org/linux-arm-kernel/20231030153210.139512-1-glider@google.com/T/) that is mailed separately. v8: - split off the bitmap_read()/bitmap_write() series - simplified the compression logic (only compress data if it fits into a pointer) v7: - fixed comments by Yury Norov, Andy Shevchenko, Rasmus Villemoes - added perf tests for bitmap_read()/bitmap_write() - more efficient bitmap_write() implementation (meant to be sent in v5) v6: - fixed comments by Yury Norov - fixed handling of sizes divisible by MTE_GRANULES_PER_PAGE / 2 (caught while testing on a real device) v5: - fixed comments by Andy Shevchenko, Catalin Marinas, and Yury Norov - added support for 16K- and 64K pages - more efficient bitmap_write() implementation v4: - fixed a bunch of comments by Andy Shevchenko and Yury Norov - added Documentation/arch/arm64/mte-tag-compression.rst v3: - as suggested by Andy Shevchenko, use bitmap_get_value()/bitmap_set_value() written by Syed Nayyar Waris - switched to unsigned long to reduce typecasts - simplified the compression code v2: - as suggested by Yuri Norov, replace the poorly implemented struct bitq with Alexander Potapenko (3): arm64: mte: implement CONFIG_ARM64_MTE_COMP arm64: mte: add a test for MTE tags compression arm64: mte: add compression support to mteswap.c Documentation/arch/arm64/index.rst | 1 + .../arch/arm64/mte-tag-compression.rst | 154 ++++++++ arch/arm64/Kconfig | 21 + arch/arm64/include/asm/mtecomp.h | 39 ++ arch/arm64/mm/Makefile | 2 + arch/arm64/mm/mtecomp.c | 257 +++++++++++++ arch/arm64/mm/mtecomp.h | 12 + arch/arm64/mm/mteswap.c | 88 ++++- arch/arm64/mm/test_mtecomp.c | 364 ++++++++++++++++++ 9 files changed, 934 insertions(+), 4 deletions(-) create mode 100644 Documentation/arch/arm64/mte-tag-compression.rst create mode 100644 arch/arm64/include/asm/mtecomp.h create mode 100644 arch/arm64/mm/mtecomp.c create mode 100644 arch/arm64/mm/mtecomp.h create mode 100644 arch/arm64/mm/test_mtecomp.c