From patchwork Wed May 10 08:44:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinyoung Choi X-Patchwork-Id: 9129 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3474262vqo; Wed, 10 May 2023 02:10:22 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5pEvRIl0DY5H99uR4p+9TTxLSi7QrpliAzqdvKO2oAotlAgg70+AhB9jyMy+TQbxOeUHsH X-Received: by 2002:a17:90a:bb0a:b0:250:9aee:563c with SMTP id u10-20020a17090abb0a00b002509aee563cmr7687752pjr.41.1683709822409; Wed, 10 May 2023 02:10:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683709822; cv=none; d=google.com; s=arc-20160816; b=jmcckc0Odr+6ZnpIOxTeb5mwdMR2OwwKdgC+iFaiYd0XX1svp62kSc9sfb25V/2w+7 djW0HndlMNJzUCSZUJDj13zB8Npy2BcTTP3DhvxEVS8uke4NPdxzUGLIXOfBlZA0cK9X h4s2qV7Ua1YALe9Kv3oktbVuYk6CEXyRKvjORy7kW4DH43OIarDjf56K0nrNvoCSlpIc YZ4UnQ3jBLSAVDEHjv85vz20YGLGK4KxFeoucykMNRXNvN4uXPcv63ryW2+FhVrOUp/s BqomKr+b5qKEFkiLG71xKBGcF61VhpqltZ3SbJJ3KJOh3zbbRvt2aE1VFlQisv0nvff+ Dfbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:dlp-filter:cms-type :content-transfer-encoding:date:message-id:to:from:sender:reply-to :subject:mime-version:dkim-signature:dkim-filter; bh=tAUq+j8EFp9wNJLVYSWcrdn/ApjcBvFZlh9RLdWUG7I=; b=hY2Iey57S/dLrgJuC5Y1pd0yU6eXYM5G2mufKfSpMrZ5WE1Uf5Vg/32R4ausDcwHXt NhimKqzURpsxEPkkozPnIdssO8m3+x7Ir/aJtsFsr+1DaCAysswGHCrVs/8w2xqdop+D 9xkdvRygyJYztAtwuux75MsxZniqEHvxNn9BFrKVO6fajWiY3cjjzo0v10XOZXhxQjdM Pbf432jsCPW/uzmgyESHIrWnuFLmnmCWOJPGDgZ4yVjZjjZhcUOZQz1rlu0CQG+fa/2q CvdwpPZgV3y4pkau/ByKDdIqvb/yXxAei8wgts6NhJfjDnBlEVvfN+UbHxR5qKQMRjZS QtSQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@samsung.com header.s=mail20170921 header.b=MWMgaVYl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x36-20020a17090a6c2700b00233e301c780si17527183pjj.31.2023.05.10.02.10.06; Wed, 10 May 2023 02:10:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@samsung.com header.s=mail20170921 header.b=MWMgaVYl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236292AbjEJIoQ (ORCPT + 99 others); Wed, 10 May 2023 04:44:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229564AbjEJIoP (ORCPT ); Wed, 10 May 2023 04:44:15 -0400 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91B0FAA for ; Wed, 10 May 2023 01:44:11 -0700 (PDT) Received: from epcas2p3.samsung.com (unknown [182.195.41.55]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20230510084409epoutp04e68f893b71e03ca1a4a2e318baf7c4ec~du7tk4MUC2000920009epoutp04b for ; Wed, 10 May 2023 08:44:09 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20230510084409epoutp04e68f893b71e03ca1a4a2e318baf7c4ec~du7tk4MUC2000920009epoutp04b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1683708249; bh=tAUq+j8EFp9wNJLVYSWcrdn/ApjcBvFZlh9RLdWUG7I=; h=Subject:Reply-To:From:To:Date:References:From; b=MWMgaVYlXihfT/rwPOIiiOoLM/ut/4PfVguwspKk73ohVg0BsNladBI+o8whRzE8u cE536U/QmGmBg+s8Px+GUFp2rDqs37h/KYGNrLPjEnap53x7v1SQASe7oqTVxUG2hM r2VDGK6cqubpzoz/y95LGgLZ5+aVi136ap2xEwac= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas2p1.samsung.com (KnoxPortal) with ESMTP id 20230510084408epcas2p13848402d1760f70a20b32a4ef2aca3e9~du7tBLdKB0623706237epcas2p1z; Wed, 10 May 2023 08:44:08 +0000 (GMT) Received: from epsmges2p4.samsung.com (unknown [182.195.36.90]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4QGT8m0wdvz4x9Q1; Wed, 10 May 2023 08:44:08 +0000 (GMT) X-AuditID: b6c32a48-475ff70000005998-5b-645b5957ce54 Received: from epcas2p3.samsung.com ( [182.195.41.55]) by epsmges2p4.samsung.com (Symantec Messaging Gateway) with SMTP id C3.15.22936.7595B546; Wed, 10 May 2023 17:44:08 +0900 (KST) Mime-Version: 1.0 Subject: [PATCH v2 00/14] Change the integrity configuration method in block Reply-To: j-young.choi@samsung.com Sender: Jinyoung CHOI From: Jinyoung CHOI To: "axboe@kernel.dk" , "kbusch@kernel.org" , "hch@lst.de" , "sagi@grimberg.me" , "jejb@linux.ibm.com" , "martin.petersen@oracle.com" , "johannes.thumshirn@wdc.com" , "kch@nvidia.com" , "willy@infradead.org" , "linux-block@vger.kernel.org" , "linux-kernel@vger.kernel.org" X-Priority: 3 X-Content-Kind-Code: NORMAL X-CPGS-Detection: blocking_info_exchange X-Drm-Type: N,general X-Msg-Generator: Mail X-Msg-Type: PERSONAL X-Reply-Demand: N Message-ID: <20230510084407epcms2p123f17696d3c30c749897eeaf2c4de684@epcms2p1> Date: Wed, 10 May 2023 17:44:07 +0900 X-CMS-MailID: 20230510084407epcms2p123f17696d3c30c749897eeaf2c4de684 X-Sendblock-Type: AUTO_CONFIDENTIAL CMS-TYPE: 102P X-CPGSPASS: Y X-CPGSPASS: Y X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNJsWRmVeSWpSXmKPExsWy7bCmuW5EZHSKwc2bzBar7/azWbw8pGmx cvVRJove/q1sFotubGOy+Nt1j8li0qFrjBZPr85isth7S9vi8q45bBbLj/9jslj3+j2Lxe8f c9gceD3O39vI4rF5hZbH5bOlHptWdbJ5TFh0gNFj980GNo/e5ndsHh+f3mLx6NuyitHj8yY5 j/YD3UwB3FHZNhmpiSmpRQqpecn5KZl56bZK3sHxzvGmZgaGuoaWFuZKCnmJuam2Si4+Abpu mTlA5ysplCXmlAKFAhKLi5X07WyK8ktLUhUy8otLbJVSC1JyCswL9IoTc4tL89L18lJLrAwN DIxMgQoTsjMW/HzOUtAoV3Fu0zvGBsaz4l2MnBwSAiYS3Ud2s3YxcnEICexglJi96xBjFyMH B6+AoMTfHcIgNcICPhJ/9q1lA7GFBJQkzq2ZBVYiLGAgcavXHCTMJqAn8XPJDDaQMSICvSwS d06fZ4GYzysxo/0plC0tsX35VkYIW0Pix7JeZghbVOLm6rfsMPb7Y/OhakQkWu+dhaoRlHjw czdUXFLi0KGvbCA3SAjkS2w4EAgRrpF4u/wAVIm+xLWOjWBreQV8JQ7//Q42nkVAVeLo9tlQ I10kvk39BmYzC8hLbH87hxlkJLOApsT6XfoQ05Uljtxigajgk+g4/Jcd5qmGjb+xsnfMe8IE 0aomsajJCCIsI/H18HyoEg+J93dWMk1gVJyFCOVZSE6YhXDCAkbmVYxiqQXFuempxUYFJvCI Tc7P3cQITsVaHjsYZ7/9oHeIkYmD8RCjBAezkgivd2hUihBvSmJlVWpRfnxRaU5q8SFGU6Dn JzJLiSbnA7NBXkm8oYmlgYmZmaG5kamBuZI478cO5RQhgfTEktTs1NSC1CKYPiYOTqkGpvXz Axjzl50UnHCu///1K7UGX7wa9/O8qt40YdH1CWaMcXcOZzyNFFr2xbjambXpBVO+0nrvHr7q nW0br1j9nOU09+iZ3zGyEVO7Ur8tcU1+rT/lYBFfWqHUppf/DRyYX2kfTZE+PKO/2iqadaWM hK2D8Ry+x4vfJlseyOKWX189L2rOgu8Cb+c3sh2/HvrZZyprOoPjBS09K1uFXxGHTZ/fW+rk fya1Z2HDfiPFZvnt/ubxz1o3PRLoyVF1q3e9yqMotezeJ+mnHMZf7a89n3h7h0/gjH8ln+eX 273P32F65YdsgtLajXeXravxrmDnmCffzXLL8FltT0Gy29P1F+pbQx4rPH2zpSdV5YvpBiWW 4oxEQy3mouJEAEBWb41OBAAA DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230510084407epcms2p123f17696d3c30c749897eeaf2c4de684 References: X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765497710511199845?= X-GMAIL-MSGID: =?utf-8?q?1765497710511199845?= In the case of NVMe, it has an integrity payload consisting of one segment. So, rather than configuring SG_LIST, it was changed by direct DMA mapping. The page-merge is not performed for the struct bio_vec when creating a integrity payload in block. As a result, when creating an integrity paylaod beyond one page, each struct bio_vec is generated, and its bv_len does not exceed the PAGESIZE. To solve it, bio_integrity_add_page() should just add to the existing bvec, similar to bio_add_page() and friends. As the bip configuration changed, the code related to sg_list was also modified. (ref: https://lore.kernel.org/linux-nvme/yq18rewbmay.fsf@ca-mkp.ca.oracle.com/T/#t) Tested like this: - Format (support pi) $ sudo nvme format /dev/nvme2n1 --force -n 1 -i 1 -p 0 -m 0 -l 1 -r - Run FIO [global] ioengine=libaio group_reporting [job] bs=512k iodepth=256 rw=write numjobs=8 direct=1 runtime=10s filename=/dev/nvme2n1 - Result ... [ 93.496218] nvme2n1: I/O Cmd(0x1) @ LBA 62464, 1024 blocks, I/O Error (sct 0x2 / sc 0x82) MORE [ 93.496227] protection error, dev nvme2n1, sector 62464 op 0x1:(WRITE) flags 0x18800 phys_seg 3 prio class 2 [ 93.538788] nvme2n1: I/O Cmd(0x1) @ LBA 6144, 1024 blocks, I/O Error (sct 0x2 / sc 0x82) MORE [ 93.538798] protection error, dev nvme2n1, sector 6144 op 0x1:(WRITE) flags 0x18800 phys_seg 3 prio class 2 [ 93.566231] nvme2n1: I/O Cmd(0x1) @ LBA 124928, 1024 blocks, I/O Error (sct 0x0 / sc 0x4) [ 93.566241] I/O error, dev nvme2n1, sector 124928 op 0x1:(WRITE) flags 0x18800 phys_seg 3 prio class 2 [ 93.694147] nvme2n1: I/O Cmd(0x1) @ LBA 64512, 1024 blocks, I/O Error (sct 0x2 / sc 0x82) MORE [ 93.694155] protection error, dev nvme2n1, sector 64512 op 0x1:(WRITE) flags 0x18800 phys_seg 3 prio class 2 [ 93.694299] nvme2n1: I/O Cmd(0x1) @ LBA 5120, 1024 blocks, I/O Error (sct 0x2 / sc 0x82) MORE [ 93.694305] protection error, dev nvme2n1, sector 5120 op 0x1:(WRITE) flags 0x18800 phys_seg 3 prio class 2 ... Changes to v1: - Add tag to fix patches - Unification of the page merge process of data and integrity - Solve the build failure when CONFIG_BLK_DEV_INTEGRITY is disabled Jinyoung Choi (14): block: bio: separation to reuse a part of the function block: bio-integrity: modify bio_integrity_add_page() block: bio-integiry: cleanup bio_integrity_prep block: fix not to apply bip information in blk_rq_bio_prep() block: blk-merge: fix to add the number of integrity segments to the request twice block: blk-merge: fix merging two requests in ll_merge_requests_fn block: add helper function to get the number of integrity segments scsi: add scsi_alloc_integrity_sgtables() for integrity process scsi: change to use blk_rq_nr_integrity_segments() instead of blk_rq_count_integrity_sg() block: blk-integrity: change how to find the number of integrity of bio nvme: rdma: change how to find the number of integrity of request block: add helper function for iteration of bip's bvec block: blk-integrity: change sg-table configuration method for integrity block: blk-integrity: remove blk_rq_count_integrity_sg() block/bio-integrity.c | 73 +++++++++++++------ block/bio.c | 87 ++++++++++++++-------- block/blk-integrity.c | 132 +++++++++------------------------- block/blk-merge.c | 68 ++++++++++++++++-- block/blk.h | 20 ++++++ drivers/nvme/host/rdma.c | 2 +- drivers/scsi/scsi_lib.c | 67 ++++++++--------- include/linux/bio.h | 4 ++ include/linux/blk-integrity.h | 10 ++- include/linux/blk-mq.h | 5 ++ include/linux/bvec.h | 6 ++ 11 files changed, 284 insertions(+), 190 deletions(-)