From patchwork Tue Feb 13 05:55:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200230 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp350841dyb; Mon, 12 Feb 2024 22:08:08 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU3gi3rQ/AdgUJPinRNEVrqcAdocWos/MnlCO+hBuiSXjCDbKOg1bV809C+1KyFdwhfsd6GHtmy9XO9o9QeXZc0yHMv7A== X-Google-Smtp-Source: AGHT+IGNF6oHyRmZ2mPeEZHmjSqFkAyJXZENmAPhrEpZsDAoMET6dvssSvGhyHuTdk1mf5iKlPTd X-Received: by 2002:a17:903:2615:b0:1d9:893f:cd06 with SMTP id jd21-20020a170903261500b001d9893fcd06mr8987365plb.60.1707804488045; Mon, 12 Feb 2024 22:08:08 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWjuzBmTPPNvV0eGLR2t83gfTP8Lgo0M867Vg3zEoOv8jeWUAyTNWgSagztHNgT4ozxvN/sU92vEJ4ZP79WchsKxFn+lQ== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id e9-20020a170902784900b001db2dcf3288si887866pln.123.2024.02.12.22.08.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:08:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62967-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=gfdY5ZVB; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=Ji9zGoEg; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62967-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62967-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 496C1B22CD5 for ; Tue, 13 Feb 2024 06:00:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A77D722F02; Tue, 13 Feb 2024 05:57:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="gfdY5ZVB"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="Ji9zGoEg" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78F9D17C71 for ; Tue, 13 Feb 2024 05:57:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803825; cv=fail; b=kIxiYt0EGYlBRAm5RTdw39J4016o8poQvKh0qaHteSM5YF8zPbUzdkOE6uIyimnB1nYGpGkV3TJXkTic+Wn3ldmfj3VQCWhUxgGHiEwi2L2wAAYIunNrHe9VGQlRmpPPua8fT5525c6wPc/8fZ78SPnFUz46WElRiSf3GBxgU4A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803825; c=relaxed/simple; bh=nyw0dQ/ITJXQwzcEChSoBKjNXJVRNp1RpUeC5ycZArU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=GBsRkY4sbfN+e05yu3NpMAndHtV0VcnhGAvG4yVkjF8iv7jjpPdj8rC1+Gc8tcOg+4EpvtU7MeqKbZ6uWJOEb5Gdz9TPDvKTDoj49WbE0D3+/1xsJhlywtks8tPyrQduI6GmnC1pc1QEQ3ufrCPhxQc08MaUMd0hpVWxGR6e1W0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=gfdY5ZVB; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=Ji9zGoEg; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D4xLVU031838; Tue, 13 Feb 2024 05:55:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=E5vV9XE+n5/oDbOcyagGXUgyXsm/0ns+qpyDQ3ZtavY=; b=gfdY5ZVBVzDC4v1oG3rvP4a16majEVVKtSoIIWQ/Q8f+EUcqDfTlt/7D+o4Gg9r0Y/1P u/wpjKTPb5MRoLyLpsdN9kQRSiHxS9xE3nCAGNXlygU6rZTRVfsljctOGDuChhrGBnin VCk5Jw5Lug1iF/IjrwCgJtsscKapZjDg7DSyBVyp+4kZtJYBhgfzKBLuzeJS/GbJkqYf B55U57Tp3TCOIWqwgAnOb+XVVzrWGceGwr9qw5Al91s73b4QXplZjxjOZ5IJWDbcXNuq YlQay4nJc4xuuJA4A7FPBzf3Y5pWklFKo0Knwh9cH6gv5xMYBfY33Pqmb033FhmLpPTA ow== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm8351-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:40 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D40f8h031501; Tue, 13 Feb 2024 05:55:40 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2040.outbound.protection.outlook.com [104.47.74.40]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6tvnj-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EibWRUmxWJBNC9xx6FSnp5yWUWkkqQwVSM1a17R9Y0tsPM/ADzqTIJwX8jBhtYpdhMdIt31bw1v770ZI2N26ClJ4uf7MnUNORbTkZLMdJ50Pi3Y9oHa3xVgSrQFzcNdNfqVDj7/HutnQk+q0wcKT4V+RuJuNRuq1dT/uRBvE1Shrcw7iOSx0Vxw6EOufpR4CwrOlNsXVn6Qy7XXjlH/Tt//9jYJYXxxxZ83iG/43I9a4+N99JVfwU277bJsoK04CUNLqxlyhyPjZR/+MxiuzurjW72dfIaVke0P9DdWbKV4cB1mK0MLYg8eGHwYNpCQ6nFMZGs7wNmgAIN5Wrf8wQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E5vV9XE+n5/oDbOcyagGXUgyXsm/0ns+qpyDQ3ZtavY=; b=doLzOyNjp6F/+aoVbXHPTLLUF4dnxaWSCWftS0mAwK+6du740fUE8+u7fMWoHGX4Ded5TN2EvI/SQKRLQMhQz4AsM7sif4fxcRQEHN3m/NTNTeaCcJ/gcI0txBwL/wN0NaPLPdbA7WIjQ09bq8VBnnM3533LNhB82rBL6QbmN0NLLZofeBZwiUZN/fhbtsqk1fCAIjoPPCEQFWuI6sw7Oby6DgTD093sC7Qip3V4yrDawy5M5ACW7RJWKOABSUKm7+31l7amh8b6G4kGsQ8OCW4fA/97G5JQjXmp0quC1qLXeW2hMkwn74kjlMaZQ7qIvFfSL3c/R/dGDGutsmebrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E5vV9XE+n5/oDbOcyagGXUgyXsm/0ns+qpyDQ3ZtavY=; b=Ji9zGoEgZKF6NL6suUdkQj7HsFFriJNPEG/J9EdJwJ4H+YGSikMGY1i0dgvvZHqCOqN3EAx+CMMx58X5mxIkQTtPScwSlnHZ7ftDGGLEnetCDvCMxBcTy3yYbtj+VTkObUfabHYxq0Q+pNPYHGW4V1gZWk9mXRZZm/7jsMHOBRE= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:38 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:38 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Jonathan Corbet Subject: [PATCH 01/30] preempt: introduce CONFIG_PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:25 -0800 Message-Id: <20240213055554.1802415-2-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0259.namprd03.prod.outlook.com (2603:10b6:303:b4::24) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: 7cf215c1-cbf5-43bd-ea5d-08dc2c58673f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WAwMQBJbZGrCVr21B+6LDMLVcI5/C1G0ZXOf86faWdSjuW8pJENmqPSPQ/oS6ElV6EHCx+qPGuJKGnFThVq+GForpUeg0OBNNMKtqpegA0TxDe1BRsGuFEVI+e248e5azCdQ+72xwLaevc3Qyh0dRDeaSsnrCUTNeS7QYgIK9bgeopQ0povPXyyujS2QuBXRXtXDNGWD0gNJaoW4Vec9qftKGw3qTMJ68S8UWz4tVWq5GRYqzcp7bIE3k8DiFECT4oZNS/S2TqYXH9XZbBHMt34eZaIwhlvJ+rRWvBhal5UAlK2+0KQu4QtA08Hqxqz1/jpA9Li3Jm9dGMODxQ93i4NUr55oLrcT7/TYaj8c96Oa1eqBMhWGX3PEKoXYcga/ZclS6hyyTG2c9mQG2XTH1ZKfpMpdBqSLj1rCjmG/SxIkICieEUHSBsSqN1zUI2p1Yd6xuKjlH8nBbsbx/I+npWkd0euuRIXn8AN+Pat2EFkXfPzQiExT14S04cn1T9wnmSYqUBZDH/ojop0s0FS2dUZArwpR7QlLt+hNVodOSPc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(54906003)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hZiI1xt/HTD+JjwqKZBTUgAlbezX9t7+P3v6ilpYOYcU6HV7XaHFvG51zbXjLbO1ykpdZR0EZ3ZPi+4YFMpy2S1M/YA8Ph24JBprFaIMIeI/nW+T8lyj56eUd5TWlw82zfAK0v6EWZ0/z0rBMxySQUOfif6BUqEswyvkUVRitS3bjZcAif27WtXvSwO6W1CtGjfLU24I4RedYiv8tc4wzFnHzTq4eZqOm91Vh3f7soEPKhV4pdB1V3TKHlBM7KhLVSgXsLubR7ImAHAKdX0Aqzx+zKNqTiqHCHG0irFsIU0tocsmU3wIQhHmngOIiBpcfQazvyOU4aBaabC97Gc+9O187M2N3HlmnDQ1hOl+byKMhootmblXha6qeWfD9vpgNgGLisHU1IQ/fL1y2NDn9X0YcumK8FHmlrFVNcL51OAbLVMTKNuGjkcZmwKBc0gSKK5LwC0BtzfZjomGdUArjzmvc4lRgwkhCU4dKLyxA4YLdPq29byXkvKQItd/xwLVTYVMRbtVBfgB4RPqO1a117MzN8gaJGH76HeW2S8vuHyTz92pKNvqnW4ywZjjk/mqr/khvQHCC2TKfeTIpq2mX7w52H6tafbBVU0zaOf6RsEzctpds8b1t+KF2yTOKCc2IXvR7lEgOn5YXV2c/1kbDFr7egHuXi8oUB32wIzcoP6d+yiM1XRCdYegr73BuFCJmwzg+KWGaSAGzfGHVtAq8f9WWM8iRMBPLGjOm7o+JScD/Vx6m2jwF+dfd0D4TQxTvV0iMDWXljNGOjHwEVnAs5YDrj599qf/Dix7/1Fpr2kkIWD5zCcT1yo3uWoeGVPqvizlCALp2N3l74/bhE/F0Qkk3+K+eGJO1K+KRzLnQNgZG7Pim4+ztwEvHFYZ/iuqtetze3atUnz8Z44dsyrch9kaJiPMhI2OKm0C7uHeSR+lhQkhrSSPLCypQ8TdglnPjuBgXzZa0Rn1q3ipWHvcJKP0WIwl74LL8tYuO+TQ6z5Ig4lyuJ681DD3CuxtMqou0NPuI7n7YTZcmt2bB657NDK1/g0WXmrp34uZBMFsQx4VwcuMwuEWOxn2Oy43bKfLntotN55sgQnY5D+TV6/sGDxvU1zhm4jpimZPpFjgsSN0yyLwRnoP9Fijx43hTME9qruAIgocc9E/mkNt9zeZO1SlqhkDFU1YjUzDd2qEk/Wa9f6B0TSt9hp4AokYoSUcUUmndaJ6LWOmnM8mB3G87O84tsBaVXdw1JgoTX2Lp9BiRDCXCfhd3AJDk71zBXISqcoTGblXidc9X14WZvxAVPQ+zuBcXbiaxiT6NolYWRUiVk19Sj6Cl05qTo75CqFU4jgwczIExp4feR18cusgrE7gY9C6a6NILgpkzb+5Z9wlNwOedE33W6mqnhT8gq3zY4FybUsHKJOw3wLBWbEPGXi64U1l/Lg4p5rNYX2eFX/RGt4SzQTxVyr0UB5l0fJw7hUKAffuVo9DCTEy6/0a9cvG9Mq8xY9jsIMxZtjH+336ZBiIIcCYSeKoLM3z6sZ0uXsj0xmrwk3Skp/4TFY1naeVG8b+IGbChHPqR4vBFEl/hkCVJJluGcsfLa7yPOtx4wK56aSlbSS7OFXrustSRA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: KQQ8HBlWNiHhHI1iCH7pPNvgaSfikLhXoCkLWicwsqlvqiWt3oyuyeYoodr5yylA1CLVBH9SzxxHz1S1XTB7LX6yuPZMKI9tEy0Kesgb8h4ZZ7SPwNfQOHXyhNjxj6qmRy5hKTss9mD1P2DJV4zzEtxnkXOllzQ+DxlkewRj1PSjScvb8UqQ4vI8ei6uhzsHUnLmOOl0p5bm2QAqY2WI2zDqQ/xijWSFMgmeJ/Z0dICRVPoeCgqNvgmAmJkQh0eQQ1MEr3+68qUPSi/EVkrSTCnVlgV9WBO87dCewLHKbOM2mwdiznva2gS0tIdw3cdY0Hiun2bWfndWndCZpSQnccPsvc1PkJ3MHT/nxvzaIFsV6Tp0HaKnJnrDwR8MHMRezm6BFmuq5Hriw76bsmkML3HRLLowcVNXNyPv8Sj7urBfJZ8bVGOkGJZ+28Fwz/zkZuWjWZSSRTlL5C47TqNeK7/oSG4vmUVGcnMArwscIxLQHGss5LkDoTMhLhCzsG+ffD4DEDEiKYqcjp1/NbBgfE9KJuyFnb4CxM5hs7o1rcabyBGGn13aEMM24YUTOcilNtOFzs3fSHEU5ykMm7Jf/djLUm9jAp3CE+PcJ24/aXs= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7cf215c1-cbf5-43bd-ea5d-08dc2c58673f X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:38.2314 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IZdxymuRNiWIzFTO/EakXI7rtQERB0CT5ik1T0OmYgVgO886RLYrX0ZW5eV6pexU4pziT9Oj07hKx9TqrfaqHVTCNKHHBS3iRwdcC02NK4U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0 spamscore=0 adultscore=0 phishscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: PyWqVxLXTWfTmZsnaNMid_Oir7Ac7tcE X-Proofpoint-ORIG-GUID: PyWqVxLXTWfTmZsnaNMid_Oir7Ac7tcE X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762798647352473 X-GMAIL-MSGID: 1790762798647352473 PREEMPT_AUTO adds a new scheduling model which, like PREEMPT_DYNAMIC, allows dynamic switching between a none/voluntary/full preemption model. However, unlike PREEMPT_DYNAMIC, it doesn't use explicit preemption points for the voluntary models. It works by depending on CONFIG_PREEMPTION (and thus PREEMPT_COUNT), allowing the scheduler to always know when it is safe to preempt for all three preemption models. In addition, it uses an additional need-resched bit (TIF_NEED_RESCHED_LAZY) which, with TIF_NEED_RESCHED allows the scheduler to express two kinds of rescheduling intent: schedule at the earliest opportunity (the usual TIF_NEED_RESCHED semantics), or express a need for rescheduling while allowing the task on the runqueue to run to timeslice completion (TIF_NEED_RESCHED_LAZY). Based on the preemption model in use, the scheduler chooses need-resched in the following manner: TIF_NEED_RESCHED TIF_NEED_RESCHED_LAZY none never always [*] voluntary higher sched class other tasks [*] full always never [*] when preempting idle, or for kernel tasks that are 'urgent' in some way (ex. resched_cpu() used as an RCU hammer), we use TIF_NEED_RESCHED. As mentioned above, the other part is when preemption happens -- when are the need-resched flags checked: exit-to-user ret-to-kernel preempt_count() NEED_RESCHED_LAZY Y N N NEED_RESCHED Y Y Y Exposed under CONFIG_EXPERT for now. Cc: Peter Ziljstra Cc: Jonathan Corbet Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- .../admin-guide/kernel-parameters.txt | 1 + include/linux/thread_info.h | 8 ++++ init/Makefile | 1 + kernel/Kconfig.preempt | 37 +++++++++++++++++-- 4 files changed, 44 insertions(+), 3 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 31b3a25680d0..5d2bd21f98e1 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4662,6 +4662,7 @@ preempt= [KNL] Select preemption mode if you have CONFIG_PREEMPT_DYNAMIC + or CONFIG_PREEMPT_AUTO. none - Limited to cond_resched() calls voluntary - Limited to cond_resched() and might_sleep() calls full - Any section that isn't explicitly preempt disabled diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h index 9ea0b28068f4..7b1d9185aac6 100644 --- a/include/linux/thread_info.h +++ b/include/linux/thread_info.h @@ -59,6 +59,14 @@ enum syscall_work_bit { #include +/* + * Fall back to the default behaviour if we don't have CONFIG_PREEMPT_AUTO. + */ +#ifndef CONFIG_PREEMPT_AUTO +#define TIF_NEED_RESCHED_LAZY TIF_NEED_RESCHED +#define _TIF_NEED_RESCHED_LAZY _TIF_NEED_RESCHED +#endif + #ifdef __KERNEL__ #ifndef arch_set_restart_data diff --git a/init/Makefile b/init/Makefile index cbac576c57d6..da1dba3116dc 100644 --- a/init/Makefile +++ b/init/Makefile @@ -27,6 +27,7 @@ smp-flag-$(CONFIG_SMP) := SMP preempt-flag-$(CONFIG_PREEMPT_BUILD) := PREEMPT preempt-flag-$(CONFIG_PREEMPT_DYNAMIC) := PREEMPT_DYNAMIC preempt-flag-$(CONFIG_PREEMPT_RT) := PREEMPT_RT +preempt-flag-$(CONFIG_PREEMPT_AUTO) := PREEMPT_AUTO build-version = $(or $(KBUILD_BUILD_VERSION), $(build-version-auto)) build-timestamp = $(or $(KBUILD_BUILD_TIMESTAMP), $(build-timestamp-auto)) diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt index c2f1fd95a821..fe83040ad755 100644 --- a/kernel/Kconfig.preempt +++ b/kernel/Kconfig.preempt @@ -11,13 +11,17 @@ config PREEMPT_BUILD select PREEMPTION select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK +config HAVE_PREEMPT_AUTO + bool + choice prompt "Preemption Model" default PREEMPT_NONE config PREEMPT_NONE bool "No Forced Preemption (Server)" - select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC + select PREEMPT_NONE_BUILD if (!PREEMPT_DYNAMIC && !PREEMPT_AUTO) + help This is the traditional Linux preemption model, geared towards throughput. It will still provide good latencies most of the @@ -32,7 +36,7 @@ config PREEMPT_NONE config PREEMPT_VOLUNTARY bool "Voluntary Kernel Preemption (Desktop)" depends on !ARCH_NO_PREEMPT - select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC + select PREEMPT_VOLUNTARY_BUILD if (!PREEMPT_DYNAMIC && !PREEMPT_AUTO) help This option reduces the latency of the kernel by adding more "explicit preemption points" to the kernel code. These new @@ -95,7 +99,7 @@ config PREEMPTION config PREEMPT_DYNAMIC bool "Preemption behaviour defined on boot" - depends on HAVE_PREEMPT_DYNAMIC && !PREEMPT_RT + depends on HAVE_PREEMPT_DYNAMIC && !PREEMPT_RT && !PREEMPT_AUTO select JUMP_LABEL if HAVE_PREEMPT_DYNAMIC_KEY select PREEMPT_BUILD default y if HAVE_PREEMPT_DYNAMIC_CALL @@ -115,6 +119,33 @@ config PREEMPT_DYNAMIC Interesting if you want the same pre-built kernel should be used for both Server and Desktop workloads. +config PREEMPT_AUTO + bool "Scheduler controlled preemption model" + depends on EXPERT && HAVE_PREEMPT_AUTO && !ARCH_NO_PREEMPT + select PREEMPT_BUILD + help + This option allows to define the preemption model on the kernel + command line parameter and thus override the default preemption + model selected during compile time. + + However, note that the compile time choice of preemption model + might impact other kernel options like the specific RCU model. + + This feature makes the latency of the kernel configurable by + allowing the scheduler to choose when to preempt based on + the preemption policy in effect. It does this without needing + voluntary preemption points. + + With PREEMPT_NONE: the scheduler allows a task (executing in + user or kernel context) to run to completion, at least until + its current tick expires. + + With PREEMPT_VOLUNTARY: similar to PREEMPT_NONE, but the scheduler + will also preempt for higher priority class of processes but not + lower. + + With PREEMPT: the scheduler preempts at the earliest opportunity. + config SCHED_CORE bool "Core Scheduling for SMT" depends on SCHED_SMT From patchwork Tue Feb 13 05:55:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200210 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347562dyb; Mon, 12 Feb 2024 21:59:53 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWfCaaUXzdGPRjdur/2IcaisJHo+bhs18AiMhmUouMEIMxHCA/LO83ZpX9Fm5ax4a6qBR3ElaeslXMHnPGHVS5DoahaHA== X-Google-Smtp-Source: AGHT+IED6ksclUdBtnZC0gMpoD9XqxxDOQTUfsc1xOq7//TLnXMy6V9boYyJjUmP+8bl68tgsI+x X-Received: by 2002:a05:620a:7e6:b0:785:d82b:784d with SMTP id k6-20020a05620a07e600b00785d82b784dmr3843794qkk.48.1707803993607; Mon, 12 Feb 2024 21:59:53 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVR5aFs7zHc16EqzZIERs5fzMes3jx8QS/IAtYbd6kgFCeghaaZ6EOwkEPagaIk77XudbFvXa11f7nQ2AMP8hynBpYJBg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id b16-20020a05620a04f000b0078715e1404esi1910767qkh.622.2024.02.12.21.59.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 21:59:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62965-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=O0WXBb6q; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=cqIlgWAt; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62965-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62965-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 58E2D1C23423 for ; Tue, 13 Feb 2024 05:59:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A9E75225A4; Tue, 13 Feb 2024 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="O0WXBb6q"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="cqIlgWAt" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E562C1799E for ; Tue, 13 Feb 2024 05:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803824; cv=fail; b=BuVXZnTSiUEjEsF0n+j9/4sZxXkLcpU16U5tkOMCJ86CNTYdrkV+m6R3jNAjFjYi+drVMyRmPNtret2I8zBmKq9RS0BiEuPmwZnVFL3LH9lM9+cGB3QDfMkELbw0gDOZmd2M5BUOcMHWglpV1oei0HokCPjDLRTWnsaS14NJ9+c= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803824; c=relaxed/simple; bh=DnGMedV1Ansg9quglE5C3m0NylP3ymfVefhffZLDrUw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=mgbb31eCtg8CSuHWuJjcksfG6hq+aM6KkSfqt9RYMOhEztFru1Jtct+Ett1kwE4FugsBMsFey2koojTFRFfOBM6cwAGGG/ik3Iw05hqFoIsvjUyYN15GoArHP64f/rh+AbJbViri652C6oP1OQY6zWeG6E1INM1KeiDtcgArrKs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=O0WXBb6q; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=cqIlgWAt; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D50FvZ000842; Tue, 13 Feb 2024 05:55:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=+8Ea2qu9i3Vi60pd8gLPWOE6K0mWMB9i6eUsA1V3088=; b=O0WXBb6q/Zf0/lyWG9z86KBOfuqEBJIIkYJRIBFBPSfdA+raNABe3NwX6r69tMNi/0iO 5UdaWDs4CFJax1G2r+hc477Mly4bWtxcRoHK7QSLvVbrj03GbOeRDIJ0JAu4yXs0sTXm qKOtAAmnJsOmmKETkR92WXUCAi6ybeTjfa+9lb2SflkkBw0zRAs+JJzS1agqAzlGtk0C RKrPN3ZJQZezQkudsk+kRMw3FPHiYnHlIpozJwpA9L3bo5sdX3LVPcequaM7WO6QrpJS NzeGjt80IYT7UAEQZaiSQcoGV3w+w74cpufrbWjrEAXREZxFoHnSsv2PBRRMXxQO28Tj 6Q== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm8353-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:43 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5KFIL014988; Tue, 13 Feb 2024 05:55:42 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2040.outbound.protection.outlook.com [104.47.74.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k80v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cG/9/iLUFbdqid/9N3f2MpmC4aDdCJBK2bY+YDmRqtBI4P6PekRejdLJnEqJhUox+Wc92p+1jEkxQvFUNJHQUxzY3Zf32KNJn6rpAL1sElYU3zczdp0Vz17zCeHeAH+i11hxX9b83hySIIHZfJmMZr46lY/eV3VfBq2cczjjgyc/l9yiJyeJORgDFGcDRTK2N6I1li+yRND94QH0UOw+Lj5A2MbacfhvHgzl+XPGMS7znddWDgz9SHGzGVLiXKNIQEwKkrol3FlLZkkWON0XSB/4HmhXuCq5GJWprejh616kc7N+fmKoSrKWH/pusyDjehwUhMrwXANAtT2PFjq9Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+8Ea2qu9i3Vi60pd8gLPWOE6K0mWMB9i6eUsA1V3088=; b=PHpb+casZtIEQQ3xmP53D4Vzs55wtizbqyUgRcPYcTWDJjhJZemy3e2TCUmrn+Sd/P5oMW41PGAts0zl1joHOl4jUMNN9szCrgj6U0OCe2xPX2zSHXc5W74C9rSpOSzAZy1Ap5x6A0P9QQCN8VUOAvPXis3JGcULccG5i14tyDB7zfHn2+JzomX+wgLiyxi9NAbbnICYkdffAcN/6JOgjY72/lnceDIXhBtvIRbOULMdHRKf0dDssBmGxt6ZnEN3H5FE+FrmxQsCsrqt/QBRL0UfVAkFQPBm10blmo9e74uOkbI6cF0LLhiZ0hDRA58e/LBB2nwTx0DdMBHKjvrclA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8Ea2qu9i3Vi60pd8gLPWOE6K0mWMB9i6eUsA1V3088=; b=cqIlgWAtpHIcKXP2/3+9LhezJIkDC809+JJDOBWnc296WVrtNZiwfQZwfHq83TKgPtMU0ZtucbP5PTfOMR7zwEWH6gHpJr5EC70pLZEje1X37eCk8fE2pPtbUNHLPgfZEfdRvrNxGqPThKBY/ajJUwJxMpmuJeSr7zP+BbUHU9c= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:40 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:40 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 02/30] thread_info: selector for TIF_NEED_RESCHED[_LAZY] Date: Mon, 12 Feb 2024 21:55:26 -0800 Message-Id: <20240213055554.1802415-3-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0242.namprd03.prod.outlook.com (2603:10b6:303:b4::7) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: 7cd61c89-576e-43a5-c624-08dc2c586854 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q0sQEe1DVW6J/nWg4pakDKHGCVN/Jbz9jIDT0R1uuEKGbLeraN81I40MnGVwbvQkdNhJweRkQctwmI2pwosCkTzKDIfN5sRMbibsrwKz7OzhTfwbU+wOL7o/lOXK6k8StnQF3O8DizSU+V5yRU3xLgVMnGZjwH0I54Ym4Dx81jK2LZ4lggQwTG3l421rnr3UgDksXpRUqE4EHdtXEYNOtQfa+69gpJXFqg2NUZUpqcwEEDpPVKm1MKUnTX0yoC/XCq+HJg7Sw/aRGP0bO/58I+7r06tVMptdd+Nk7dR1D06qfDu4R/O3WsS7Dte54vU3H5+SU69u1g3xEv8BaNB2yBaQDNR3X2adW2dI07rtQd6jyoaRrOlwU3mWJGUCyOYNHAkbVipdHHNGU0bEcNEkapojwu+qz8+NI/xYj4rwTWTHMoOP7l02t6VJhE5D7nS5g+8UDdkKjGftCr2aCsWT8AEGIXoA7RXjafwRg1ZGgajzVUbPOu0vF2pebiZmGMyUI9LidkGTBGqODM8xlQW67P2zESRdL5NBgIGtUNwz/wI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ywXvu1QkMsGY+3pYohEGuqhJ+a86NDxG3huXjt6V4Ab6395jMu5wK4WjgqV9lF//jQvmnRCoyLpXAkbxESZfjxZY4aLabv0fFObJYKZd94BniFM6gAjLmwI+ZLsrV4w4xqESKUuf2ySl7MpOK6SZskZHR70TrxR7W9a9336IMm5tCM+cuzNrVIwmKCaDv2BICchqsFNyDIEtTWr82PDiTFENk6NsLisa12mZG1ARGFDcvaAphm9s14boWndp4+czjfQesbVQ/L9DW4XWVCZeEVUXDJQZSfQxYTE4iRfZFMS8HBeJsjylVvnVzqRlL0a7P3QPX6NTuOz6zKqmAWxwByLfmzny7WjWZ01J+5poHjK/et6L9fV2hEYYoOcfU2mf58Fw+Cgd1EiotvsCRTRcNsFJMPGtBmZX03iBbXrBlqxXgwo0PcVCYFsBFZau6EmHp3Yw2G+CQmf5w8DvuwJKdLaEQsAfcIsxB45MtuFLti909YmcfnvyHsm+3xuFfD0yUwESUoGYFMFb7X6sftcV+B7MDWH8GxKR6zt23pz+lDXb4wmRTqT0SWf6XtzcJDVIf4GlVWVxk1elSpnAOSPdtiUCKloVB5DgZ1MVh6RuiDiTfSuB8ym3sMIl8wErNTpueTU0lV0jwEwm784rbd1AkH9PDSvvmflwVVVgN3USlBryrgZedRdncbedvbl8xA19xqmcxvWPwQtyHahBlx61K7XA1ABZShRUbuwolmTuXtGbhGSMpJyWCWs7nBeC8okIsfljs62q273u3qyTZw29+G4IFcf0NUWSNBpZ8cnsiWyhZhdgdeW4JQD60yVsAQvETGZ2tx8l50u1WIsr9ICteQFoli93JW1Bx0mbVz3djCNR9A7fHW/NC9l9wRiGviA8Lu/gBkPm/ZKapwuF6tAt3jA1MQfWMkRjKgdHl7YOl515HtIHTvmW+MExdln1cIgRfrm98tvpMfaNzaW0HneYYoe3F3wLszeYxqLAJdC7vgKZyBMfAQUzQ7qMkfdsJEwiJ9ZtnZY1InsJrgWK8TXkhreyMpmgnGhNDs30XGBXP4ZYhf0U2w6wOtWnv055QwTtdcPhSuhw0FSCbHsYwOJn3Gd4b8BImOJRnjJHCRtbJgVYYzA6FiTlplX6kgZ6C+h+vWfM96tpkf+yykr3FJDZgp+2ETwp4kXKU8ayIUaCQMo4x4VXefjIU9MJMqa8hH9mjgd3798AwNeXqBB0yTUimLaVGe/uMA7iAMfZLYl9Iw9YyZWJSTqNMhMh/5xWARm7hj0dUDDh6LCDgZLJUgC3v/wpK0xehV46zqRVwI7cORSwYp8SjHxXUFeJyNV2O4L9R1BarryYT5bNHkzfCHF1BCO8MLAfbQKv3URuvDyXelFS/u8B0GWNo3Kp+u0zsltC+kMm/Wf83omtbcGt6bWZevJgGO7uH7J2YLviM2ARm3sZN72JB3xk0WfIQpUmnRzD/cAGdTFFL8OySyoMUc45eLfTrDTNLwUnsoPsNeq8cvhSZu3XvnqlDVrBieFR5A9g2CX62jf3/iqI2Jg733xoj+ma5DiPYA9JvZv5k2eejMBLN8+NUJtEdfKnpUMW3eHr8C+V3snMHEcQ8G6ACdeD/Q== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: EWhBTv+Y3d8OvKIOYUHnQj+LCBh2aWPMgE+SJ7sRIiHaIEd5aGRxvBwdl11jAa48rrIKf+5w5lx1syaodEogeB4SFASv1l0/UEibTYLAXR1JeF8AKAfDZuWkqet8lun+/6DTYkEzfqUy0TM2c7MQdhXdMWMT/vIZnbfrB5c//k3bkUVIaUWDvdskPkmwR96sXza4m7hNiQgnR0tVpxzSDj5qQ9iRXLJLgcTGfzLnEmgMpejHn3DsCupjdLAsSAwyWObfZ8c3Uu/8czUd4sbQW1BxKzeeUX6jtvsy1PPWNRf73w/mHkM3ZCdGJh3c/5RjlpzuLqTmoYOB3sghQK6Vbbg6tVa0RazIzxTqQ6ewWkStTLGa8l/jbX32EyVXbHEpnJLkhdegeGGEa7BkbzW6bzCZBF0e3gvgPJospFoFe+r/ZtReVXvAzeh6CmqXvOHDCZ60EGKhdrrAWEHeFxUr87a2m/KCjzar78f5YB2cXMPRDyZHZ3D0RZRHZ1v1KJLnOiiUzMXorqOgYuONcrFQ/CauVBw8ZtTgaSlCcAqNTap6vXk/BgCwU5eI11xXLfIl1X2faMcDFoJNvsZL7LLIPhj1B3zT73cUcKJfBX3sT4Y= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7cd61c89-576e-43a5-c624-08dc2c586854 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:40.0232 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VMG2oJ+mHYvdGt2trS+6/jlDR/6aFsfUD7Clc23twGc1lzmJS7cPxCEePHZ/7EZZH4Fj5/6RbbUm9h3L3HRlUO7ZugrwUJfyMPfl1rCGgiY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=879 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: wVdhc_L0mYVf_Y5b9n0fWsMjArRRlqKV X-Proofpoint-ORIG-GUID: wVdhc_L0mYVf_Y5b9n0fWsMjArRRlqKV X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762280065233273 X-GMAIL-MSGID: 1790762280065233273 Define tif_resched() to serve as selector for the specific need-resched flag: tif_resched(NR_now) mapping to TIF_NEED_RESCHED and tif_resched(NR_lazy) to TIF_NEED_RESCHED_LAZY. Note that, for !CONFIG_PREEMPT_AUTO, tif_resched() always evaluates to TIF_NEED_RESCHED, preserving existing scheduling behaviour. Cc: Peter Ziljstra Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- include/linux/thread_info.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h index 7b1d9185aac6..99043cbbb6b0 100644 --- a/include/linux/thread_info.h +++ b/include/linux/thread_info.h @@ -67,6 +67,30 @@ enum syscall_work_bit { #define _TIF_NEED_RESCHED_LAZY _TIF_NEED_RESCHED #endif +#define TIF_NEED_RESCHED_LAZY_OFFSET (TIF_NEED_RESCHED_LAZY - TIF_NEED_RESCHED) + +typedef enum { + NR_now = 0, + NR_lazy = 1, +} resched_t; + +/* + * tif_resched(r) maps to TIF_NEED_RESCHED[_LAZY] with CONFIG_PREEMPT_AUTO. + * + * With !CONFIG_PREEMPT_AUTO, both tif_resched(NR_now) and tif_resched(NR_lazy) + * reduce to the same value (TIF_NEED_RESCHED) leaving any scheduling behaviour + * unchanged. + */ +static inline int tif_resched(resched_t rs) +{ + return TIF_NEED_RESCHED + rs * TIF_NEED_RESCHED_LAZY_OFFSET; +} + +static inline int _tif_resched(resched_t rs) +{ + return 1 << tif_resched(rs); +} + #ifdef __KERNEL__ #ifndef arch_set_restart_data From patchwork Tue Feb 13 05:55:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200213 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347882dyb; Mon, 12 Feb 2024 22:00:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IFMlBnmQo9MnfGFX8c1hHIcC+8SrESepTR4MdFoya3DG5DpIcAOV3qjzh6zpgS5YEWBTjJQ X-Received: by 2002:a17:906:e257:b0:a3c:9206:608f with SMTP id gq23-20020a170906e25700b00a3c9206608fmr3961228ejb.0.1707804036238; Mon, 12 Feb 2024 22:00:36 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUs+DOGW/kjQcEglM+ECm21U3yvjeug1vUioC6fXQDnYW/CGujPNTBUnLa0+YKmi4XOyq8oUulraVu4KW218g4YxAzmtQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id r23-20020a170906c29700b00a3cea6183a1si612120ejz.621.2024.02.12.22.00.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:00:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62969-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=h80prFgH; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=IsDCwxRY; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62969-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62969-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id AC2921F24C2E for ; Tue, 13 Feb 2024 06:00:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B257224211; Tue, 13 Feb 2024 05:57:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="h80prFgH"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="IsDCwxRY" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A90D417999 for ; Tue, 13 Feb 2024 05:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803826; cv=fail; b=rR1ioOfffXA+4GJL/41tkI6daIq1eHIFJJC4frbPIAhgGNbmRfygfRG+10KQrerFzBo10omLMx1GHPsgE5HD/9qOUL3F2GtZwKvhGGZw6nafpjFeU1w9bZ+TIsO+hrtWUlEOV5VliINs2FQ0KBnnNFRHo1dA/43sTXaMEeRcROg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803826; c=relaxed/simple; bh=cZ254lLjxAdLrH8n5/CjJEIsc916JEPISVxZvXwcxZA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=QBnsVTVMomB1+0kTKS3peQmvsodpIzJFb0Z1PCNmtKU3X9rldB3lmRCN0DHe1QmIFeBN77GZuUzgkXNtGP36+L2BDVondn6aQ4E9IkNL2L7IAWl9PBqrZjRVaIzPLapj0N4rxb2WenTWj4m/7DX+NvMLt12vOlxGtp69GFZWgQ4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=h80prFgH; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=IsDCwxRY; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5neCO028863; Tue, 13 Feb 2024 05:55:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=HzqwUvkK8xB7rMUcd8xTT1/p/prwRC3nLM1EuhPBUtk=; b=h80prFgHjqy6bSWjwi3PqOZ6ti8lC1LMoOsm8mDYfZf7zPLiIApr4Jq91hZX6xpNhUj4 KymzF36hGneaeOV+TeEFgdt6TnPuA+BEuM3DHxFBHuFfJsllc3V6LE0s45zA6ofNcJ4U dlkwHFtzeRm7rLgtPasvSl/p3SOBlfRhsZaHBkwQVi7ZBGbui2+t0jkGisqA3w4eEGSF 6UM0P1fcU7IGp26TZSrmkOcsU1L/1ub+g3vck6u5sq0eU3mtwJDRacdTDC7jqjMVhPiq td0W+osIyE5ABNbd5ZRFKwQX0raIL31BASQlR1lxMov4WVD7bQKEj+eWN0fNatR3Cub4 7w== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82m480ge-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:47 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D4kTMT024017; Tue, 13 Feb 2024 05:55:46 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2041.outbound.protection.outlook.com [104.47.74.41]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3sq3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:46 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I0Uivd1y/XdBlW3zImbLhfEVK+QjtwvRxmPjJ1NSdbCm2CMjM3ruz0F8CgB48TwOnmIeaRDn14d/T+OpjwBhvTa9jRoYKx4eyL1QI4PjTJI/CHQBy5ZRNzVQWmJwQfHAZ3JDJezKC74AcCwMk9ETMvJhz67LiMZbUsw1I6yhPTf5tx2QOz9H3NYgJL/D1ycFVx7dSnrVYmMDt3Xe17vejerKvSpxIXdk607lQ8b8RWIpKD85NjM4ia1KlnFpIe0vr0o+vBcvYO+Cf9TvEhgtjti/uogfi0k7o695NATblFXNgvfwh0ANchfeegb9FgAXmbfOjAQrK2wOr6RWRrLIhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HzqwUvkK8xB7rMUcd8xTT1/p/prwRC3nLM1EuhPBUtk=; b=c/nZ6QXcl4BUP5vvjKpk0AsD0wWlgEYKytq45FlOuQTf/16Th1PmCpU8RV1Z62doFRd4Z1kYgOiSJNQSf87QvRTZP1SPhMHqwe8+97TKnG3NvpFRWIImsbX4DXi389uxL4x73tCnr72K3UL4W6as6XYNV50bRfFKfCdiSjMTRowObWJWKQtnw5IO3fKiGM361uWlGn7p8U9eG0QsmuH1U95sfx5Z7K+qQVMn4mUbzVSg20TyJBUCb9qvzin04wwPUzVs/fyMoLSKOGMVjFU9scUjfSVrxyEb6mGLN1mW5561hQnf4ahH+BxX8Hfvy4wvCGSqEJJN2C+XFWZJxKEKtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HzqwUvkK8xB7rMUcd8xTT1/p/prwRC3nLM1EuhPBUtk=; b=IsDCwxRYHttrUZ0BcU/7zCPcArzdSj3lPh0f3X9wlELod2CsMrw9Nv4ymKJG70JfKO2Je4ST2rYEhIPzQlQfe7fpz9zxGoZgXEbPdhErIRDF0PRBkFhLNBrJ1VT/P5gE8TYUd6gq3dj2k8kEDRVKFupAJQamTSe7k7oa6fKO+3g= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:44 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:44 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Arnd Bergmann , "Rafael J. Wysocki" Subject: [PATCH 03/30] thread_info: tif_need_resched() now takes resched_t as param Date: Mon, 12 Feb 2024 21:55:27 -0800 Message-Id: <20240213055554.1802415-4-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4P223CA0007.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::12) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: c1eed021-4c68-423d-6b18-08dc2c586b25 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OS1otCMje2i9YJIAGA+hcwp8fIWFbYz4ZJ4fh7DMp3noWfqndPfrdzYEqp8h3VZyNbTHD6exkkZ+6l9ZT+IY6g2Kgaf4gnbCZi5t/VQiqCprvQ6kQBtVxoHIp+2uUGzFg87waa8PCV39vIGUO/6L8S/a1NGp4D/iV+DnfpNSbnn9T3zexHPNurunCOFT1Oe666xjscJ6o/9ZrsxEagMO8CobA4G7nKfwxkmg8vpYc517OyJ/ZBG0le4I5NgBhvvambVVeT46NlfjG9Mn1ieXNIOqk99/iZ1rqAuPXTncRnyStcIPyK4N96Xbz1aDC74i0phPnAO4I3mgjahJt13k+ygYifp0nitQFdEIMAGP5Mksd3VEIr20ue49aeH+FFqqrZ5oyfjs1la+ERnDrozxAXWhB3FSZyXCuUB55Filrul/0/vM1Pv6gFusW28Mk6Il188Ayom0GM2SgJJ4zNvN0kvCdMMy+SpqedHHhHdVcgUhxMVUBs6gJPbKUrUdw4R0xM6YeCGQ2hyeUpAK9kThpwkMEXG5cBCy7e4ArFmbHhc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(54906003)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Oy1SKhBOpzUBEwbs5uPgDXM+VrHg6GnBVBg19fV+rJ0nmVbvA6IHBcd209ImrvzxLN2Wr50u+6kxoE1HmlWMmz2Fxg6GD2aNkCa1Kjj9gfF8x9fjVVtEuztnjHs2GmHq7vs0RYHsr4mGtTTkcFE5tRH59yYPW4R4MgPl932PzZlsHQYWQozZReTQciaZZgzE50rtu1l9dnt6XJKdFfpWJsuTW5I7MR1i91G6h7QMLklExSMgdd2Y1XND1xgR7RQUN1+Ynjm4m5Q8bmqGUX6/xGZ+T+Ti06H5hbGfmc82HDtiIPQlDYHGnIk55aKbDwDREYqnfUeQM+A8qqDVOKiK3kfFEMWkoWxjphcn2HjZGEd2F2Ge2mvlFwMF3TjJLx7m/LdyEZ/E8UKyEfYmtAabbpgSGdortQwUqRwDMqeqcKb81PlxTgiwl474Rh2+7+tiVWgcQrUCE+Hdodf05LLHJIaBpb+b51YuBF/jRyzIKIMLkbB+u+Zlls1jJ0aeHXoH9hM9l70aKWn+n9sE72m+2lKdSbZXXkUPvLgTLQ+auLYGLzyGsZ5JDaAJh13hcjzI88N8TeLhqGjI3vx8pCgB6NTzXpCbClw7gxYzihr33PCHFO6MPE3zeoCqsAL5NdDap18RN/DjtElODeXQhi3mcpimSu8mRlfBOEDEH4BoAuwK59zTeMV4eHNgGlFmOB1J+oB5cS3bS6zXyF8Xfv1kZVnIHtNFM0hlTykf90JfUArHBBEhnYu6NdkE5WI1EOR/ia/CbqD49iqtQLcI1m5VUHMzBc7Phl96bgicAhRY12/TdJwqQEcddl31dbICdVT1IsbyIjOX7tE7CdqotrAHAi+VmVpp6wUazCxzsj8dVGIP45zLOtWWF5+vFo93jqGSrsV+FMsSA659H+p8DvjJAE1tnSgc8zie71d2KMP7JOi1y2xAjhQ5LNTJNCjTP/HPb79z+H++EGb23+d9jnXJtmlEX2+Kvp57n4U91OSvVs9Rh0Dilv2C++gbj9TvqUmT51m3Kt9WfUBPH/j6QzfkpKwhLdrnrhV2KjaFyrnqbpmdE2bN+s6mb8Z/gLzOoESYf3Cusje2kHlB4P315FEJ2IVk4PsXT6ol8H2qjycWl84stFTL7qvD/D7zguBSVYhX0N44037B0qBB+XgBKId8yMktr31mQIEWz6fWDXHoFUm9EhKtKToD1x9DbcnbHPx5s2EDWAYNAcUctRh8tRRM7Pg1HW0a2D84f5an+OYIw7VFIdJ1qocGY+wVwsitx/6M0BCh9w+Eu27xCkA9Ojbw34Ihz0iWVzHqI+5UgYE5IbkD5igE1TrAd9/5RbARQB8MH9ABLAN3f+pxK+rU6+MRGIt7eE3Gs0d9p9FPpX1n39l/kcJlp2T8eEXrlCFVS1PbteJLq/1VlrH+ILzig9doYuwkEW1kIF+kXxoZVWO77fEgifPxZQr3dOuqIafAM99ot007Mwy/prXhHEqYBQ5vzxbKGZBaOSXm6s71SxUf/j6i9ABAaCR4klgvpQhr40OJr/tTH5Q4zE+1nDAvIA+P1TARy2eqyNOZcApVhTPiZXg9fWMMiXJ/Pfbe4gQ1/aUiLQLK8bZfytK6jwlfnc9+ug== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ofZUWZUjXWRuoTM9XbTQZaiaQ5EZHpdYbCMPy4/tsuLKVVXDCg8k7XZh5x5nq12Kru+zyVTEcbYlOw2vyVFtUXJrP/zIWM5Kq+KyY4z9cZx3LUtCMczLSEI2+vqx4Wc4eCdjvSWfoqY9N7DyWYRKtL3fEVj/4rl32gDS57XRItyM/JcdV8Z56YASVlomPxKYeiA96ZORcmmzCWi6inqoZgIVS9QDtxBwmXacYNxSrHu4wl3ST+RKfZd340mfo++ygVEWdtGtApxsLDkoFrRWHeZ3ZbNZq+rd9h14oRRLqQVwG8kvBB7TrPyiFSYKVOOTmepBnGz7gcoOXY74D76yMbLNXUE+knJ733y1HE/TxcWHLfK2e4JXm7pe3gCO+oZrbtcwJ2Ks7QI2wngXd46NVQKEChIGbziH3Y2No4vIe9WEs/+ui52U7eGCRSdFqoTNp7pkPZZTu6Qh/E4xrXDcHYk8keNJeCE/2VakylRuZLGbiuECqGKvFQBp0MfZk3W8FOuMVVVHbR6DWg45Fc4iHEJCudaLVsLS341l9TjieF+zHkWH72eWCWX3jX1I6Iflyxq81rD4LgjtxZYW9zgxQiHZhkP24S90AzKH76QC0V4= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c1eed021-4c68-423d-6b18-08dc2c586b25 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:44.7376 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NVuR4RS4xwfA4fUqHDi8Pc8kP25bPfPw1ggYVY5k6oCf8MllktotT8I0TFCvF14vz2LTZh8QQbsin80xPkOUhL3KsNZbo8cJNAHY1+IlaBs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: 47l9FIMoFeS6V7NQ2dqIWuSZ-F52eYTr X-Proofpoint-ORIG-GUID: 47l9FIMoFeS6V7NQ2dqIWuSZ-F52eYTr X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762325222706234 X-GMAIL-MSGID: 1790762325222706234 tif_need_resched() now takes a resched_t parameter to decide the immediacy of the need-resched. Update need_resched() and should_resched() so they both check for tif_need_resched(NR_now), which keeps the current semantics. Also define need_resched_lazy(), which as the name suggests, checks for tif_need_resched(NR_lazy). Given that need_resched() (and should_resched() to a lesser extent) are used extensively in the kernel, it is worth noting their common uses and any changes to them: - preempt_count(): we only ever want to fold or make preemption decisions based on TIF_NEED_RESCHED, not TIF_NEED_RESCHED_LAZY. So, related logic now uses tif_need_resched(NR_now). - cond_resched_*(): checks for should_resched() and preempts if TIF_NEED_RESCHED were set (and if (preempt_count() == offset). Hand-rolled versions typically first check for need_resched() which would also continue to check for the same thing. So, in either case relinquish resources only if immediate rescheduling was needed, not for lazy-rescheduling. - idle: run to completion is not meaningful for the idle task and so we always schedule out of idle whenever there is any work. Most idle code uses a mixture of tif_need_resched() and need_resched() (the first one especially in the interfaces defined in sched/idle.h.) This change moves all the idle code to need_resched(). Cc: Arnd Bergmann Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: "Rafael J. Wysocki" Cc: Steven Rostedt Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- Note that the need_resched() semantics here are narrower than earlier versions of this code. In previous versions, need_resched() checked for (tif_need_resched(NR_now) || tif_need_resched(NR_lazy)). However, there's a fair bit of code which relinquishes resources temporarily based on need_resched()/should_resched() checks. Or, in the case of mutex_can_spin_on_owner() decides to optimistically spin or not based on need_resched(). So, this version limits need_resched() to only checking TIF_NEED_RESCHED. arch/s390/include/asm/preempt.h | 4 ++-- drivers/acpi/processor_idle.c | 2 +- include/asm-generic/preempt.h | 4 ++-- include/linux/preempt.h | 2 +- include/linux/sched.h | 7 ++++++- include/linux/sched/idle.h | 8 ++++---- include/linux/thread_info.h | 25 +++++++++++++++++++------ kernel/sched/idle.c | 2 +- kernel/trace/trace.c | 2 +- 9 files changed, 37 insertions(+), 19 deletions(-) diff --git a/arch/s390/include/asm/preempt.h b/arch/s390/include/asm/preempt.h index bf15da0fedbc..97fda8e06b6c 100644 --- a/arch/s390/include/asm/preempt.h +++ b/arch/s390/include/asm/preempt.h @@ -114,13 +114,13 @@ static inline void __preempt_count_sub(int val) static inline bool __preempt_count_dec_and_test(void) { - return !--S390_lowcore.preempt_count && tif_need_resched(); + return !--S390_lowcore.preempt_count && tif_need_resched(NR_now); } static inline bool should_resched(int preempt_offset) { return unlikely(preempt_count() == preempt_offset && - tif_need_resched()); + tif_need_resched(NR_now)); } #endif /* CONFIG_HAVE_MARCH_Z196_FEATURES */ diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c index 55437f5e0c3a..7fc47007b926 100644 --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -108,7 +108,7 @@ static const struct dmi_system_id processor_power_dmi_table[] = { */ static void __cpuidle acpi_safe_halt(void) { - if (!tif_need_resched()) { + if (!need_resched()) { raw_safe_halt(); raw_local_irq_disable(); } diff --git a/include/asm-generic/preempt.h b/include/asm-generic/preempt.h index 51f8f3881523..ed98e6168b99 100644 --- a/include/asm-generic/preempt.h +++ b/include/asm-generic/preempt.h @@ -66,7 +66,7 @@ static __always_inline bool __preempt_count_dec_and_test(void) * operations; we cannot use PREEMPT_NEED_RESCHED because it might get * lost. */ - return !--*preempt_count_ptr() && tif_need_resched(); + return !--*preempt_count_ptr() && tif_need_resched(NR_now); } /* @@ -75,7 +75,7 @@ static __always_inline bool __preempt_count_dec_and_test(void) static __always_inline bool should_resched(int preempt_offset) { return unlikely(preempt_count() == preempt_offset && - tif_need_resched()); + tif_need_resched(NR_now)); } #ifdef CONFIG_PREEMPTION diff --git a/include/linux/preempt.h b/include/linux/preempt.h index 7233e9cf1bab..336cb76f0830 100644 --- a/include/linux/preempt.h +++ b/include/linux/preempt.h @@ -312,7 +312,7 @@ do { \ } while (0) #define preempt_fold_need_resched() \ do { \ - if (tif_need_resched()) \ + if (tif_need_resched(NR_now)) \ set_preempt_need_resched(); \ } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index cdb8ea53c365..7a22a56350bb 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2099,7 +2099,12 @@ static inline bool preempt_model_preemptible(void) static __always_inline bool need_resched(void) { - return unlikely(tif_need_resched()); + return unlikely(tif_need_resched(NR_now)); +} + +static __always_inline bool need_resched_lazy(void) +{ + return unlikely(tif_need_resched(NR_lazy)); } /* diff --git a/include/linux/sched/idle.h b/include/linux/sched/idle.h index 478084f9105e..d8ce85dff47a 100644 --- a/include/linux/sched/idle.h +++ b/include/linux/sched/idle.h @@ -63,7 +63,7 @@ static __always_inline bool __must_check current_set_polling_and_test(void) */ smp_mb__after_atomic(); - return unlikely(tif_need_resched()); + return need_resched(); } static __always_inline bool __must_check current_clr_polling_and_test(void) @@ -76,7 +76,7 @@ static __always_inline bool __must_check current_clr_polling_and_test(void) */ smp_mb__after_atomic(); - return unlikely(tif_need_resched()); + return unlikely(need_resched()); } #else @@ -85,11 +85,11 @@ static inline void __current_clr_polling(void) { } static inline bool __must_check current_set_polling_and_test(void) { - return unlikely(tif_need_resched()); + return unlikely(need_resched()); } static inline bool __must_check current_clr_polling_and_test(void) { - return unlikely(tif_need_resched()); + return unlikely(need_resched()); } #endif diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h index 99043cbbb6b0..8752dbc2dac7 100644 --- a/include/linux/thread_info.h +++ b/include/linux/thread_info.h @@ -211,22 +211,35 @@ static __always_inline unsigned long read_ti_thread_flags(struct thread_info *ti #ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H -static __always_inline bool tif_need_resched(void) +static __always_inline bool __tif_need_resched(int nr_flag) { - return arch_test_bit(TIF_NEED_RESCHED, - (unsigned long *)(¤t_thread_info()->flags)); + return arch_test_bit(nr_flag, + (unsigned long *)(¤t_thread_info()->flags)); } #else -static __always_inline bool tif_need_resched(void) +static __always_inline bool __tif_need_resched(int nr_flag) { - return test_bit(TIF_NEED_RESCHED, - (unsigned long *)(¤t_thread_info()->flags)); + return test_bit(nr_flag, + (unsigned long *)(¤t_thread_info()->flags)); } #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */ +static __always_inline bool tif_need_resched(resched_t rs) +{ + /* + * With !PREEMPT_AUTO tif_need_resched(NR_lazy) is defined + * as TIF_NEED_RESCHED (the TIF_NEED_RESCHED_LAZY flag is not + * defined). Return false in that case. + */ + if (IS_ENABLED(CONFIG_PREEMPT_AUTO) || rs == NR_now) + return __tif_need_resched(tif_resched(rs)); + else + return false; +} + #ifndef CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES static inline int arch_within_stack_frames(const void * const stack, const void * const stackend, diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 31231925f1ec..be53d164e267 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -57,7 +57,7 @@ static noinline int __cpuidle cpu_idle_poll(void) ct_cpuidle_enter(); raw_local_irq_enable(); - while (!tif_need_resched() && + while (!need_resched() && (cpu_idle_force_poll || tick_check_broadcast_expired())) cpu_relax(); raw_local_irq_disable(); diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 2a7c6fd934e9..0a9642fba852 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -2701,7 +2701,7 @@ unsigned int tracing_gen_ctx_irq_test(unsigned int irqs_status) if (softirq_count() >> (SOFTIRQ_SHIFT + 1)) trace_flags |= TRACE_FLAG_BH_OFF; - if (tif_need_resched()) + if (tif_need_resched(NR_now)) trace_flags |= TRACE_FLAG_NEED_RESCHED; if (test_preempt_need_resched()) trace_flags |= TRACE_FLAG_PREEMPT_RESCHED; From patchwork Tue Feb 13 05:55:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200217 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp348658dyb; Mon, 12 Feb 2024 22:02:14 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUfgBBtMiR6zlO8lDaTeEhLNdleUUrcsBl0Z3OPUqSuSO6lpooQhZW6I8i1PsNDD2SehuWxwpPTJ+JH68CzYtmxy2uYiw== X-Google-Smtp-Source: AGHT+IFBtvTVOTATARq+CzovZtlqrBXyEfYAe0DFcBEDfBJ/EMiO++SJ/EE4y062lo6p/1H7KcqI X-Received: by 2002:a05:6358:78b:b0:176:56b9:d4e1 with SMTP id n11-20020a056358078b00b0017656b9d4e1mr4371276rwj.23.1707804134489; Mon, 12 Feb 2024 22:02:14 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVpzQpeEwZybAqJsQOotT1FZp1BEymFuBdqrv7WjoWUuleGDkKRc9jK3PEeq3MYGUJ8O0dcvrqYStsshNMluwV9VJJt2Q== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id e26-20020a63371a000000b005db76738ae1si1498262pga.229.2024.02.12.22.02.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:02:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62959-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=DLJHjPlE; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=zqSIhpcz; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62959-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62959-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 9D8BEB25EAD for ; Tue, 13 Feb 2024 05:57:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BC8BE17743; Tue, 13 Feb 2024 05:57:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="DLJHjPlE"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="zqSIhpcz" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1C0B13AC0 for ; Tue, 13 Feb 2024 05:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803817; cv=fail; b=TMi2zkymnZ97bw3qltEFZeY5NGTVZ9CenzIDEem+Qkrt5xhQI95li1n/kouhPOJNhBkDp+M1bbBTBb+ULPQg5n5N0adHZd5CRiloHdLRyFzYXv89GjUzVkk57T17BaDZZztjBIwKZX/AOPmx78/b/Mylpno1uHPn4RR9M8mMkX0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803817; c=relaxed/simple; bh=nr0QPCu+GnZE5/UOpk39fhN9n2hnLDQiL8bisVcyU58=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=VGbPmfr044XLWB9Hh9MaKNwuRt7A+s64bMk5zqtQc39l9Mv9HpavofA4Xds68kLCtRRwgAcj6jZ9ciMRvaxk73CeuW4MQMh++4bVs00KFdETAQe6Ds/BZ7aCsFJ4bAgDgxHlKrrWeH6MHNVZDhlUpjhcTNkI+2LR5i7RrVAeMKs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=DLJHjPlE; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=zqSIhpcz; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ZIOu006358; Tue, 13 Feb 2024 05:55:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=xiTU2FNLYRY8SsmUttTPdEuTbpCyPg7TgvckCcFoWU0=; b=DLJHjPlECxOzILKjnBrMmu5WJ/RWbphE3NmWOu5riNTfhxg9qy7IGmKjxxotJ/OKdHjX Ivw9qb6uXoICrvgTntuJ8pPVv4en4/UuRCwsJT7FAXM5XZssTYy/2RX9KYgQ8UrOJHTc JAWJEqovPlH7nZxD7Uo/0QR4JzYldBWLJZ6X5PYR7CjoDWllhQ/uSTEawi2/XJO7+f9o lWyp1vgLFrW1ELw7LpVTFT8cUvXNjcI8omSPHFxEnbZikBdEMClNJxCtuyC8SXO1YXtK pNsGhxJYkczfSmDzygfXaWPcjjgftIaVzhw7N2H2wBC2ZCk9W72aQIVMhC4s9o2uYJCF Zw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82d7r1fr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:49 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3TKii014951; Tue, 13 Feb 2024 05:55:49 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2041.outbound.protection.outlook.com [104.47.74.41]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k842-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:49 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jII9NDdr4Pbzs+R3dAt/b1R9d1zoAjEWG7NyqNFIZVXjSqdljd1zXVJ1Yq8Oy08uDA4LRJJLqJ2L6YRAzmCioPj4/y3pgjicYnQEqg7fhmEl3+kmkVaVvVvQjFljGkcOUINZz3ITJ22my1nhBNRVoJ7rveH54cRnWYhsLAe+Gr5qQmTn7l+HOgqLOiBZ+mU+PbfS7ZHfU9TfSB3gZbg9lagoPFAE1VTGzDd90Kq3nqdLpHfFYUfJiWKWI7rWpwH3lBZ0vS5FPvue5pS4Bn+p5fUEowqL3Cy8pBHrii9WplIKTAFU7M+xy/9zw9jgybQVegQfr1+0W6qe+y2hyrIQ/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xiTU2FNLYRY8SsmUttTPdEuTbpCyPg7TgvckCcFoWU0=; b=YdNWB0C4+AcsTtEpo2MRJCzDucWkPnxfB8mVdnF5nXia/UcpDnSFMvj+5fApXfZpNwDlxb4h8kwh8W7Tr/O/JC1B0/Iv6a4ynHdUwdWDMNfrlYdBKs6uGoVvzPqHYtY6Xs/CQ/l7ArPDn+R+mcbRjHLNfQ66jdbEEwjjNx3NYp8VTP60GwoTCxUf0s/RCeoPWgvkjw3mx3+ZecvuflFEz6VJJ7DKajMR7o+cvALBj2K3/HMiyxejc8s0MIroD0zENDLraPYgfUo65oSprFvfA8xgl+W33wR6r2zmu5EWwDLwi6ned9USHTj/saFVsAD5JsqQQgZu+E0uzv23Rtz77A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xiTU2FNLYRY8SsmUttTPdEuTbpCyPg7TgvckCcFoWU0=; b=zqSIhpczPuJ0jt4KtdTe7gvQ2xrMgDJaiYotNVVgtbxrvYivEHZrSvurAhMhrC3fIIyju7ysC2pro9zjqBdJR10heQSjUgkj7iASmmsLxDCnELfYXQ9WLwwaJjvilJd+cAm1cePD2Ipu1tPnyu7myZUb/PXeExbI6WVvNc+niZ0= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:46 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:46 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 04/30] sched: make test_*_tsk_thread_flag() return bool Date: Mon, 12 Feb 2024 21:55:28 -0800 Message-Id: <20240213055554.1802415-5-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0038.namprd03.prod.outlook.com (2603:10b6:303:8e::13) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: 1557ed33-39d1-4fe0-20d3-08dc2c586c2f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w1ixs6QKQARmPzJdKLNBw7ua1+7+dFmLHxx1eifFvkq96a9nUulkokxc49foUZIn0M/gBsGzSxhagpDe0kvVywwiupsypXVcdwxxwp7O0QEBjAhyuppE0I9H1+vOUX4ZnEZqdCjVOrJjCZ4sv/RT983b7IUZrjPTGyzMxi/YnzqdNCLCcyWmz8cA5gpYSn6Yh+UYi2idvQksRcF3PpZCgLl2DoZ2aOZDNKOm9JL2EfQGYDXKSK2GvavHIVjKuKaSRD4BxSsNioqEfh3TDkwrw8DLQujqepSpQT9dL4JjI/CBsYDE3koc/+kIpNtRE1nCy5htTQtUzYXZjjTTxej0B5qNtMRzgrY5PgOfzLE63KVn5lo1SItFjNHKNBiRMpOvX9MFYvIJd3GVKbsGkolmpBRZJK8YHsf6UgcNd0AFz2zRQcZd4oRTs6tJf8+I+mRlLQ2Jh6pVzJycvx/SLVtbnda9dBYvdb8yb5dZsJaijzCz0Goa3xzYCWSf8iZLUjyNKDduIb9zfRiXzm+Sa9/mm8pJ9AQTFpt0iCXcHB/k2l0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2qDNptoTrbYsTHUotw+EcNcH9Id7OVQF+6+/g4sQb6a5t8yMAJQrfpwPA9plGJs90KsDBBEvtJhgGy2SOh2DBUjJINeuSgtX8COaA3/lG+szWJGvUC1whTLsRtwsQrWM6ARnrGdax/Ks1BZj94efh7vSH8t6GS54UYDl9X3PPioDAt7Cmkuk4+Bdh8e1xsbfrleLTW0QRdbSPlGyOihB5xy1jnnxEOuTTHHEkN2mxOGWME3nN/VCkRdS88Cc47UyvB2CyijAw6+gRNC/qhk2OEpUEEyn0ZKNy2ZKsa6lym5iB1BvaORzuV8frx9sWFsYVASW/HSHzwWFqY2v8dlbfUqLGWsxzs2c5XzjN4x+NhzMyjHq/bBLqbql4e+cBdlr9GMeC6xyj7nZ1fyIWhm38obvsbu3A8hTCey70mW61dsgpqVLSIsBjRPCw9eBnQ0ZwRVGtRX0euiHBb/VXAfiggne0ealk4VZys2N73RYCVygAzP5s0F/8kLWr/UNSagA/JUgTimqs+D1Lqn7eF3wu8gDhIfI2Lo1SvwYyejpLYNHYkDpkTHBC/W0lFt0GEyC9XEQHRW5Jdqnl1SASO+SKdjeGUffvq1D5BBApp0rG0ibkkRt7ejSEscHfpyw7dkDdRZO7/OXbIUoAmQTqcV1PHfGuE63prVs80JO1TKzqdej51Xhl6s57joM9rUcAiiM48x4JH2aLzBTvw0PthmtjIKTVu7NuQyHRqcImEaAPH5PXmICP0kto2VcFRYrHXXPOv7bO2NBi9xW1U4goxeb2L1EztZ+kWVvcMgvrKdiyU7zEntKmsbXdunZlrx+WeLdbV1OoBwI3rwRrmrG3P24+nAYGEo3VIAqCeb2tfJULw7CxxWyfsX8YaiOtE0txdNWSVfbHfAyRVTOeoC07SRlQTRXf48LBDmQrX8CJR5sdjEA53qcu+jr3Rvxvw4NEUkLC0LIMSnUV50ZI8qiRa/c/BhIxpreaIiTdE49XeMcLDD8KM45aN2oiyD+VHb65wHEkDsuRwerDNoXSeRyb4kYfVicoDSy9jCOzp4gR2ERJZROj/sJ0QIsiI8WEfGqWLuhIhpOVWfchxtFyfiLgi76kFdWWCuOCGugzGh4u0+ZIKvC3tCQC5Xg26eeagQzqi0l3Gb1Akc7DYLkTssxsDdtT4vhNdLqXLTxHkgqvO3gn2Se/8Kx48TP+D4+ZyWAmccYQINPHlvWW733SlmZWMvKCFs/xdRKKFcEWMVtlWF/Uo+irnjuZswxJ67GApXN31rYv+WDIQCFRvphFPaybHF8CGa5k4PzVMTnU35S9mC24gcSwJY76j2TMJAv5B1ERLtxovf2s/cqDAkQMJtiyRNXheQpDD0D0E1VPqfeDLcC1hQgr13beOU7UBzqqGb5oPnpd5qt1rVEEfili+RyG+9Hsd6pADlDeGJTpQd2COfqreI/F9Pja2dcrbe1aXzgm+yYZlsdk4lUJcLk7FkOOWv76ydi8DufEfYJgmGOXujrZ0LUYTN0i5RVaSU2+cS4c9LnVgT1YZXHRs9mb7cOEAbvF8q6ExWMRF3HUIXExYxGn2qA9ZvpDTleH6wRYnOJDuhKUDZDk/cqwod3UBf0NDxP8Q== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 1LxcDQci518h8kDIfKu8uTewckRw5KhRBxoaLbDZtTuewfHr+M+0PmMVQPSKkSPC/MK7S9SRFeBn8u5DYTDAfm+Uny3zda7x/9/mpkfRMUCTr5vB/fBfT8A7ROevKVzrLbxfp74ErCY961GgZDUek3XunIYGrDXeZ/sKh0PAXeEwb4zE2/3p6SuSpCll84ZCpz0z5bBSeG79oQUk9sDFgUeP7MG2kiZiBKCE6Ar+u8BH2wT02nOns+Wkr2ALTQY3BQMYidBUUt7sZ3Wtr8oESibYga+t0WoaMnJ4C0KXfqm8JyCqM3glIDbNIPLlUwE007JQfpE4HkHlq8ELxASv3IJg25a+RNZmJjxox3MA2e3zUqswWPaiO2fsHuBmAzDAs3sRQ12Sstk/hVHRYZwvzWs8+nxPNdVrinQy1rTjto7NN37mjC3grV39udOydgBVcC9+yonlXZtm0+RlW06BLU7540RN0cuv0PrOaJ7AfmG3XbIRHlCtn3tlFgzLn0bBcm6YOpA/VnlIy/OnEhGdSpTdCiERMPyIWq7XB3V9rKMyN84JkRsTh8fM31Sn05+Kqf/H7MJinpwA0v/j1IB6KgYdIDnp756/Z48Zq0RFr9s= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1557ed33-39d1-4fe0-20d3-08dc2c586c2f X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:46.4647 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6kQ0a1tAUSBgQjv3bWMaOC+WmjyuupLi+OBudjzbTdMdXAGtJa1Z9ClXmIvbLaXj+7vFkUzwQcpGIV5Rartnyu78ppztWlMD30lOY7++dxY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: FIi_gl6ubWUvy52-6ZdaPSlvRcs3nQEh X-Proofpoint-GUID: FIi_gl6ubWUvy52-6ZdaPSlvRcs3nQEh X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762427750224756 X-GMAIL-MSGID: 1790762427750224756 All users of test_*_tsk_thread_flag() treat the result as boolean. This is also true for the underlying test_and_*_bit() operations. Change the return type to bool. Cc: Peter Ziljstra Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora Acked-by: Mark Rutland --- include/linux/sched.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 7a22a56350bb..e790860d89c3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1933,17 +1933,17 @@ static inline void update_tsk_thread_flag(struct task_struct *tsk, int flag, update_ti_thread_flag(task_thread_info(tsk), flag, value); } -static inline int test_and_set_tsk_thread_flag(struct task_struct *tsk, int flag) +static inline bool test_and_set_tsk_thread_flag(struct task_struct *tsk, int flag) { return test_and_set_ti_thread_flag(task_thread_info(tsk), flag); } -static inline int test_and_clear_tsk_thread_flag(struct task_struct *tsk, int flag) +static inline bool test_and_clear_tsk_thread_flag(struct task_struct *tsk, int flag) { return test_and_clear_ti_thread_flag(task_thread_info(tsk), flag); } -static inline int test_tsk_thread_flag(struct task_struct *tsk, int flag) +static inline bool test_tsk_thread_flag(struct task_struct *tsk, int flag) { return test_ti_thread_flag(task_thread_info(tsk), flag); } @@ -1958,7 +1958,7 @@ static inline void clear_tsk_need_resched(struct task_struct *tsk) clear_tsk_thread_flag(tsk,TIF_NEED_RESCHED); } -static inline int test_tsk_need_resched(struct task_struct *tsk) +static inline bool test_tsk_need_resched(struct task_struct *tsk) { return unlikely(test_tsk_thread_flag(tsk,TIF_NEED_RESCHED)); } From patchwork Tue Feb 13 05:55:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200209 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347510dyb; Mon, 12 Feb 2024 21:59:39 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWIap+LhkDo+nn3NcsP1cAw6LZRokX6BWmQhWGvhtbn1Shk9X6HfWhUMSIwVsljWmuFf3GjynBdG/SqCv3YB/CpBNtR9g== X-Google-Smtp-Source: AGHT+IEnJ1JSDMlcMLkS1YLca44LpcUWy5msnlJqXlrPpeZmy+bKn4AxUrN+f/Lfjw6/QkJLUYL4 X-Received: by 2002:a17:906:c798:b0:a3c:f045:a516 with SMTP id cw24-20020a170906c79800b00a3cf045a516mr898397ejb.57.1707803979181; Mon, 12 Feb 2024 21:59:39 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVvXnfn+jSbyHwu5wsmp/+mm3J0aKIGGLSCjNC65MJYD87NJU05rcFIY9yITt6m6Eq2+BcI3U5cGzcf1JhUMD9mk54GcQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id g21-20020a170906595500b00a3bd699061bsi884786ejr.412.2024.02.12.21.59.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 21:59:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62963-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=jMO9LZ1D; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=IWWiDFTG; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62963-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62963-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 9BCDD1F24C67 for ; Tue, 13 Feb 2024 05:59:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9AA7E22319; Tue, 13 Feb 2024 05:57:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="jMO9LZ1D"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="IWWiDFTG" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09A241754F for ; Tue, 13 Feb 2024 05:56:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803821; cv=fail; b=jTtAjDZRysS+dQ+Z39q2wXGDJC7LziFdbzVzv3kFpR223R+t4gczTHiBL+weLBs7baYrS1gAWzRHhZ0a1uCYGkM52cOQBRjorLRqrHOLXTW0PqSe0K2sqRGWr5KUM+ESIkXAKz344FDpLQWX6Xdkbwhb9ibn4JMbJyVDi7UYWIc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803821; c=relaxed/simple; bh=l7U/OJUO6CI83OsMzPQK29qpXrrJF2BPUHGz5bQq0VA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=K5mnFU5o7WT0vtan6CaBy6FFBtP61taq3sqQiRxQ5Y3cVdNSYpaZ+VvLhmK7CDPnMmgzGMP+IRFmFrclWqtuIq2/JLJLBqQvzScUXRenBJTVh4dMgfgQfxwSQQmM2e6WwqbioRrXJVAfOgpu9yOMJlWMmPsjUYy15K18BwEtvKM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=jMO9LZ1D; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=IWWiDFTG; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5HWOV011785; Tue, 13 Feb 2024 05:55:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=JPqNy/ih8oh/yE7bYXGBNueEA0lXPAgRXgUwRoFWa/4=; b=jMO9LZ1DIRSlMKq26xHXfonLUvVF1gIdCagx4v7s4aZJhLRlI5vOid/a0xEGtnCay0aD IScWdejo3lmfWP99faielaaAuNCNTqEkxrJ90Y8KRxufY6FEKtPIfWg0rGL2i7iRy3y7 OEZ+jaRAn7LVlkHvqjYmfE22gVKz73vVHZEeoIxidYt2bS3h2gjLUFesIdHLNiDMDFiE B0Fom0Gj4XJSDNpDw/pMC6hqTjAKvwEjEmsRIBl2p9z1d/BMK+g+XhFTIywXHL5Xep/N EzTXynY+bYQ2T0uLQfFpSyOEEk2x1EfX86zVfEOWjjw1qoT/hIIdsRrAQuOzx0IP8Mp9 Mw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w824v02ms-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:52 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5cWbM015103; Tue, 13 Feb 2024 05:55:51 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2040.outbound.protection.outlook.com [104.47.74.40]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6ug81-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fsqZjtISKbZ9DHG8gNDZZKwDhP/O19V1mYh/wL5wh2WxV3fy2NSm7j25+kGXP2aU10FTgV9X/ZbhlAEb6SrZuN0ijcOgthKme6Rfm9kDEjDHjtismJiItsBKQ643QxvSLY0zIKzyKxXPQ6R4hjhlIkWp6Ikdz1U8Aex9DNr4GTWlGDGeadKaqhM1KvZq9IhZT02jI7xw34BwGU4sNcp7qG+GMWjK1RFPCdgNy0S4D78S+0sZez/C0ZcVnRhBV51JNHMa3mZKYoVtItggwk8o0G9YTznVmqefmV+HTtn8McgeLj57lt1KoS3uTex9Ve36WD08oxI7EBNW5x2BMshSLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JPqNy/ih8oh/yE7bYXGBNueEA0lXPAgRXgUwRoFWa/4=; b=ZAm8tndO6pWC2mGdmwyeC9jv0AMDElFM/8GIEggMsS+Dd5ncORqOv+wE8shg1tGSeJU8vibg0AtaMmQ7RhNlTF9m8rkw09LM2KH6keURCzwJRDkM+z/47da0A1/kXHL8NxD3qyYVDlO3w+J/RjMOxsyAWpq5tY6hUhxRC2XcJyXschDMO7FGYj68xWllb+AehJfABtObxVHwz0Mt5tvwfdDLbxYVmzOydtHKgCYQD5iqM1OVRQ3HfegQhft4iVtmV4CB2EE6J8f13rLkqKESAIHsrdEwLrZGklYADRcvxhVyUDOgsYpKJgKJ2+7mrFmfjwyFS25s0GB6qEY8hHNUZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JPqNy/ih8oh/yE7bYXGBNueEA0lXPAgRXgUwRoFWa/4=; b=IWWiDFTGOtAq1Qpil2xZph3Zkoil/znsV3shCA2JX/TsHiZukWe8tiuiOVTDTvXDfxPKzW7yJd5/xsIn0Tb8YqxdVHmAXEz3qBB12UKmBkI9So1mRIj0RLo4CqHNo//9ZlN2y115O8nP92dV8Rem2T29SdQpAuGeoygT638rigo= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:48 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:48 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 05/30] sched: *_tsk_need_resched() now takes resched_t as param Date: Mon, 12 Feb 2024 21:55:29 -0800 Message-Id: <20240213055554.1802415-6-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0155.namprd03.prod.outlook.com (2603:10b6:303:8d::10) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: 9d45fde8-3eed-4efa-6e3a-08dc2c586d87 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Jwz/h9L80ks88UKz5oM9xifha2ZFLWHjVx2QI8SB1BdbJL8O7RV0UIeoK4vrTj+LmTf1AhEm00ODslHndQG+U8ppzNWeKNKSjX/vJmAbuB/ahsWrY0qiFDNW/6aGoJ5dJMB4tPjnZ9ixWcFcA5q4wVdj40mO5uVQCf5xaZGI2DHqZHioAAvCdZzUomYYUi/pLoui8Ea8fLgnqOl3MR8tvll2lg833KWrWAOchx2CvLhTAp84NPbRAL1VatW9NC8h3ommgoI6Wl1yNrOqfJ0auBD3zcxOB/sVJ/q7bFkjlR6OXspSeNLC8S6YZzhZCzVkkVfP/TpuGtTAAA3q/OPQac6VuVhzsVkUgShLVFykCOGuAPf8HGts7h/oyJuXhspcTBbsm2j8snHRqgXCdtWDJvhyCyL/DiW4Y/ifMlVUWesQEhDvpWik5uExJpIiuav9l1190Bjhh6btQWECnbeyL7c1eR+nmxURaVw9H4pKa8Ac6ULA0Fc673K0jf+qqBqZ3kqmrXNo0FMJh+Iu4/a5ogtK8jZWOZfXbzE/kkrhlOk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(66574015)(83380400001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(30864003)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HgQOi8n25jPMszKtklGiand2Q7OzzxqoDLTbOPXt/Hy6DEnWedMhlEId6ICxVLFBNBRkU/Wp8KcrBiE4bSdi/nGKUZfvONJu9rWx/IPx3u5br0sRV6nHS6IiDgOqOFO4gDKhj8Bl1dEuOwNelh/a2KAUmLeR+dRg3rxcDH/e8ElLRfWRGA4xJd/e1gS50fxqxdZs0bR74lpX0XnjvUkctmYImuUQzFRcPsWuEPLdFqgFipGwoBI6BUeodO2Sd9bb0sii1hVAmdkdbUjqMeDpnQxxX4gQQYs/kkPEwICCrGBZ3YLfgFhxYp9/DxCbiYhlzzW6gB6PWBC8IBFJgkXqRgjwjKT6Pr9dA8siRzD5WXzCUuM47cEnS1LEbxCwGAVU9HoKq0SiCUL/IhVpf7+wQvEmSEjIV+GaMVzlnn2HbCtk7MfMgyAIICUPIw+9yYPFWIOALOYPbPTeIlPnCKU8P8QnNsMwiT++iyJIijFtv5Nqc1Bwr3S54xuHw9DYTf7CZCwhqbktpCdVgA3ao8cPIgJvVwjPaBhw0LTSUM6shtIW0NrBqt/h7dLEQVHCKExX5o08ZYk3M4fSMWXp7VODwSVHFNQ46tB0N1gxb7mW2zy4r03iBqlmv2tNyLNdBsBisuIjyycjj6Wa3adJZxZgWzSifXJAtthzWmk6shaxIjQxfqzLtUA5uOpdFFPAbyEzWslOfTVZ1pjXH5cK6EaZh6MmhnDV1mx0R4vWeXToKEKuY7drF0B8502gwfLQzuSWUGPbAmxY+PThkGp/Np+PXpqIg2VKFXZGr6J2f675nGLArl27EmWyL0Sw9fYkXWfhIHDvgcbqRkhGzgtxqecnNs/h+YNZiV/SKHMaYh3y+zz/zZCpiKHiOwSY9+zeNeFMT0IYUb+53X+OeMyOIUOxvoiGnBXt5PmHBmsgCUWqHfL2aze1MFEwL83rvQa+O8qpr8QCLXswa/JiaRe3hCjO+VJ+uht0Q0+U0zhTjkQW6q6Xyn6CfIaY6MUUfLWEw85DhopxPvwtKp3jsGL91V3JFxWzta2jQm3y0yLJ4JuQh8cw4trqA1IQoH5TtbOtB/3aeCUw58AH83jHwaPw+pfnn4yyY6EnTrADNSxpbL9pIl/J1uZdLBU8l9oGImZcGJM7EnZPg4xvG/bIIQd2RUQ0YfiDWDV/0pyMlKe41bHskKDrwplIDvJnY00ZjLJh0u/ZfZ7b53Nf7NpsHL5EJgKbefeN4ZhwAmF+VnYvnbLkXBm5fUffXzxlFzKK834TDjUoH+67j+4jBq1LT5yK1+Bu5Ddx6zsJpXwNKruMCbvE9T0G3nHM2nJsePFx5kL3e1BHt+o4yfeHJaVVLbcglWvcYY1HC40kg/LP2z7ZKccDWEL9t/UPHqW47SgPf7jAke1Vb3nRfGnzn7UUAA9lRhRZXoxC9ukydU1j3Pdclujc/mASSFKtqtM1XT+VeQSaUJsduiYeMKdKNTJp6mTvaHKEXTYJzxeFo4dZx8Xglf1Db9fQkwH+dHFs+RC+Nj9xmMzRR6fOd+n6K1bgj6qacJclVIJwU8Rd9h8a62dP3LIsSZy0Fq2/zdjcLIk/FIDHWNCd+B6756oKnLQB7oh7pqmYDw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Z7mn0cbk+Q0fUDICIqh9vXIGMuJHPCJD1CcWvM9/YtJx0bj6GgxCitTT+OlQhok2LdRUjmq8ZHaDKYGZNyicoSEpxto1era+aEiPqCAkMcs+NWZnpCVyEOvddmRpfQ7Zeh2RJHyGhR+kgTGxhOrnEnjfvsgKITxkMolvw6+zYGnZRRAeCEMkXhy0rxtTArioLlBgufRECyNLRVZY0ydsHhyhZl5gco4ILaGgieVqTwouHpC2e3CMhACVvAmf7FxsicP8/aW0vo3ISBR1OLc1CMTIyw8MFq49md7Sqv9ilJxvkpZCpI+XP7595O6cqeXm5NAtr5cCmITMcHbuoK0/GnQ0n0lY6bygwnL0lBj06XVUhZuEvmI+keNytHnNUDG2krY/1gvFg7O2rxwr3NHNl6BJLZz7hraR5RZlYZaVhK9Bqz842xxwAb+y/R7fw8P7c0dikhHz9D/oYwkNmZNpsseiOUIQqJLTyx61fuRVECHHCV7oSX1jdE5cCNfJundauaF69BTNSv3ivxzhgLQpuwpn07cgSTWrcyRecOhSEphJWMdE/hdeB3ZkzhixdY5TQTrK/HG1c2T7tKDc/owdsYX2HBy3wXdguilIrwTS7kA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9d45fde8-3eed-4efa-6e3a-08dc2c586d87 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:48.7880 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UgB16tS6Ax6jeCP0Gyf5XFQ5oiuIAben/mMOzlP04HR4Ysc8OT79hdkcKtLNml2gz8qw+Rue6ktHrpP/RFoplJYLC7knvQl3GrAVGVL9zQk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=0 phishscore=0 adultscore=0 spamscore=0 mlxlogscore=999 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: -q7hDUj8Yc0RVSd6qgIcCWYfOsl8fLEs X-Proofpoint-GUID: -q7hDUj8Yc0RVSd6qgIcCWYfOsl8fLEs X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762265319030644 X-GMAIL-MSGID: 1790762265319030644 *_tsk_need_resched() now test for the immediacy of the need-resched. These interfaces are primarily used in the scheduler and RCU. Outside of the scheduler, set_tsk_need_resched() is typically used to force a context switch by setting need-resched and folding it in. Update those calls with set_tsk_need_resched(..., NR_now). For scheduler usage, preserve the current semantics by using set_tsk_need_resched(..., NR_now), test_tsk_need_resched(..., NR_now). Note that clear_tsk_need_resched() is only used from __schedule() to do any clearing needing before switching context. Now it clears all the need-resched flags. Cc: Peter Ziljstra Cc: Ingo Molnar Cc: Juri Lelli Cc: Vincent Guittot Cc: Paul E. McKenney Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- arch/s390/mm/pfault.c | 2 +- include/linux/sched.h | 30 ++++++++++++++++++++++++------ kernel/rcu/tiny.c | 2 +- kernel/rcu/tree.c | 4 ++-- kernel/rcu/tree_exp.h | 4 ++-- kernel/rcu/tree_plugin.h | 4 ++-- kernel/rcu/tree_stall.h | 2 +- kernel/sched/core.c | 9 +++++---- kernel/sched/deadline.c | 4 ++-- kernel/sched/fair.c | 2 +- kernel/sched/idle.c | 2 +- kernel/sched/rt.c | 4 ++-- 12 files changed, 44 insertions(+), 25 deletions(-) diff --git a/arch/s390/mm/pfault.c b/arch/s390/mm/pfault.c index 1aac13bb8f53..4e075059d2e7 100644 --- a/arch/s390/mm/pfault.c +++ b/arch/s390/mm/pfault.c @@ -198,7 +198,7 @@ static void pfault_interrupt(struct ext_code ext_code, * return to userspace schedule() to block. */ __set_current_state(TASK_UNINTERRUPTIBLE); - set_tsk_need_resched(tsk); + set_tsk_need_resched(tsk, NR_now); set_preempt_need_resched(); } } diff --git a/include/linux/sched.h b/include/linux/sched.h index e790860d89c3..d226c2920cff 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1948,19 +1948,36 @@ static inline bool test_tsk_thread_flag(struct task_struct *tsk, int flag) return test_ti_thread_flag(task_thread_info(tsk), flag); } -static inline void set_tsk_need_resched(struct task_struct *tsk) +/* + * With !CONFIG_PREEMPT_AUTO, tif_resched(NR_lazy) reduces to + * tif_resched(NR_now). Add a check in the helpers below to ensure + * we don't touch the tif_reshed(NR_now) bit unnecessarily. + */ +static inline void set_tsk_need_resched(struct task_struct *tsk, resched_t rs) { - set_tsk_thread_flag(tsk,TIF_NEED_RESCHED); + if (IS_ENABLED(CONFIG_PREEMPT_AUTO) || rs == NR_now) + set_tsk_thread_flag(tsk, tif_resched(rs)); + else + /* + * NR_lazy is only touched under CONFIG_PREEMPT_AUTO. + */ + BUG(); } static inline void clear_tsk_need_resched(struct task_struct *tsk) { - clear_tsk_thread_flag(tsk,TIF_NEED_RESCHED); + clear_tsk_thread_flag(tsk, tif_resched(NR_now)); + + if (IS_ENABLED(CONFIG_PREEMPT_AUTO)) + clear_tsk_thread_flag(tsk, tif_resched(NR_lazy)); } -static inline bool test_tsk_need_resched(struct task_struct *tsk) +static inline bool test_tsk_need_resched(struct task_struct *tsk, resched_t rs) { - return unlikely(test_tsk_thread_flag(tsk,TIF_NEED_RESCHED)); + if (IS_ENABLED(CONFIG_PREEMPT_AUTO) || rs == NR_now) + return unlikely(test_tsk_thread_flag(tsk, tif_resched(rs))); + else + return false; } /* @@ -2104,7 +2121,8 @@ static __always_inline bool need_resched(void) static __always_inline bool need_resched_lazy(void) { - return unlikely(tif_need_resched(NR_lazy)); + return IS_ENABLED(CONFIG_PREEMPT_AUTO) && + unlikely(tif_need_resched(NR_lazy)); } /* diff --git a/kernel/rcu/tiny.c b/kernel/rcu/tiny.c index fec804b79080..a085d337434e 100644 --- a/kernel/rcu/tiny.c +++ b/kernel/rcu/tiny.c @@ -73,7 +73,7 @@ void rcu_sched_clock_irq(int user) if (user) { rcu_qs(); } else if (rcu_ctrlblk.donetail != rcu_ctrlblk.curtail) { - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); } } diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 1ae851777806..d6ac2b703a6d 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2250,7 +2250,7 @@ void rcu_sched_clock_irq(int user) if (smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs))) { /* Idle and userspace execution already are quiescent states. */ if (!rcu_is_cpu_rrupt_from_idle() && !user) { - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); } __this_cpu_write(rcu_data.rcu_urgent_qs, false); @@ -2409,7 +2409,7 @@ static __latent_entropy void rcu_core(void) if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { rcu_preempt_deferred_qs(current); } else if (rcu_preempt_need_deferred_qs(current)) { - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); } diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h index 6d7cea5d591f..34d0bbd93343 100644 --- a/kernel/rcu/tree_exp.h +++ b/kernel/rcu/tree_exp.h @@ -759,7 +759,7 @@ static void rcu_exp_handler(void *unused) rcu_report_exp_rdp(rdp); } else { WRITE_ONCE(rdp->cpu_no_qs.b.exp, true); - set_tsk_need_resched(t); + set_tsk_need_resched(t, NR_now); set_preempt_need_resched(); } return; @@ -860,7 +860,7 @@ static void rcu_exp_need_qs(void) __this_cpu_write(rcu_data.cpu_no_qs.b.exp, true); /* Store .exp before .rcu_urgent_qs. */ smp_store_release(this_cpu_ptr(&rcu_data.rcu_urgent_qs), true); - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); } diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 41021080ad25..26c79246873a 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -658,7 +658,7 @@ static void rcu_read_unlock_special(struct task_struct *t) // Also if no expediting and no possible deboosting, // slow is OK. Plus nohz_full CPUs eventually get // tick enabled. - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); if (IS_ENABLED(CONFIG_IRQ_WORK) && irqs_were_disabled && expboost && !rdp->defer_qs_iw_pending && cpu_online(rdp->cpu)) { @@ -725,7 +725,7 @@ static void rcu_flavor_sched_clock_irq(int user) (preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK))) { /* No QS, force context switch if deferred. */ if (rcu_preempt_need_deferred_qs(t)) { - set_tsk_need_resched(t); + set_tsk_need_resched(t, NR_now); set_preempt_need_resched(); } } else if (rcu_preempt_need_deferred_qs(t)) { diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h index 5d666428546b..9d4aa4fde5ae 100644 --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -712,7 +712,7 @@ static void print_cpu_stall(unsigned long gps) * progress and it could be we're stuck in kernel space without context * switches for an entirely unreasonable amount of time. */ - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); } diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9116bcc90346..41c3bd49a700 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -933,7 +933,7 @@ static bool set_nr_if_polling(struct task_struct *p) #else static inline bool set_nr_and_not_polling(struct task_struct *p) { - set_tsk_need_resched(p); + set_tsk_need_resched(p, NR_now); return true; } @@ -1045,13 +1045,13 @@ void resched_curr(struct rq *rq) lockdep_assert_rq_held(rq); - if (test_tsk_need_resched(curr)) + if (test_tsk_need_resched(curr, NR_now)) return; cpu = cpu_of(rq); if (cpu == smp_processor_id()) { - set_tsk_need_resched(curr); + set_tsk_need_resched(curr, NR_now); set_preempt_need_resched(); return; } @@ -2247,7 +2247,8 @@ void wakeup_preempt(struct rq *rq, struct task_struct *p, int flags) * A queue event has occurred, and we're going to schedule. In * this case, we can save a useless back to back clock update. */ - if (task_on_rq_queued(rq->curr) && test_tsk_need_resched(rq->curr)) + if (task_on_rq_queued(rq->curr) && + test_tsk_need_resched(rq->curr, NR_now)) rq_clock_skip_update(rq); } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index a04a436af8cc..b4e68cfc3c3a 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2035,7 +2035,7 @@ static void wakeup_preempt_dl(struct rq *rq, struct task_struct *p, * let us try to decide what's the best thing to do... */ if ((p->dl.deadline == rq->curr->dl.deadline) && - !test_tsk_need_resched(rq->curr)) + !test_tsk_need_resched(rq->curr, NR_now)) check_preempt_equal_dl(rq, p); #endif /* CONFIG_SMP */ } @@ -2564,7 +2564,7 @@ static void pull_dl_task(struct rq *this_rq) static void task_woken_dl(struct rq *rq, struct task_struct *p) { if (!task_on_cpu(rq, p) && - !test_tsk_need_resched(rq->curr) && + !test_tsk_need_resched(rq->curr, NR_now) && p->nr_cpus_allowed > 1 && dl_task(rq->curr) && (rq->curr->nr_cpus_allowed < 2 || diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 533547e3c90a..ae9b237fa32b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8298,7 +8298,7 @@ static void check_preempt_wakeup_fair(struct rq *rq, struct task_struct *p, int * prevents us from potentially nominating it as a false LAST_BUDDY * below. */ - if (test_tsk_need_resched(curr)) + if (test_tsk_need_resched(curr, NR_now)) return; /* Idle tasks are by definition preempted by non-idle tasks. */ diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index be53d164e267..cd25f71f43a7 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -359,7 +359,7 @@ static enum hrtimer_restart idle_inject_timer_fn(struct hrtimer *timer) struct idle_timer *it = container_of(timer, struct idle_timer, timer); WRITE_ONCE(it->done, 1); - set_tsk_need_resched(current); + set_tsk_need_resched(current, NR_now); return HRTIMER_NORESTART; } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 3261b067b67e..c57cc8427a57 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1680,7 +1680,7 @@ static void wakeup_preempt_rt(struct rq *rq, struct task_struct *p, int flags) * to move current somewhere else, making room for our non-migratable * task. */ - if (p->prio == rq->curr->prio && !test_tsk_need_resched(rq->curr)) + if (p->prio == rq->curr->prio && !test_tsk_need_resched(rq->curr, NR_now)) check_preempt_equal_prio(rq, p); #endif } @@ -2415,7 +2415,7 @@ static void pull_rt_task(struct rq *this_rq) static void task_woken_rt(struct rq *rq, struct task_struct *p) { bool need_to_push = !task_on_cpu(rq, p) && - !test_tsk_need_resched(rq->curr) && + !test_tsk_need_resched(rq->curr, NR_now) && p->nr_cpus_allowed > 1 && (dl_task(rq->curr) || rt_task(rq->curr)) && (rq->curr->nr_cpus_allowed < 2 || From patchwork Tue Feb 13 05:55:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200206 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp346959dyb; Mon, 12 Feb 2024 21:57:31 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV+rO/87bzvM49J6r1LdoILEwiyx8BB6NekXUkcXFyQPDykpJfi6j6vZGZv37sPsIAwYTl3wU1jNHINdijb4YnS0yaEuw== X-Google-Smtp-Source: AGHT+IEUBxtXhZPhufdzroIl+5jA2rk3i6JV8tU35Fkoh6OVi3vi6wlHYVgJd5yHW50Ibr3SeU2Y X-Received: by 2002:a05:6402:f19:b0:561:ea0:e234 with SMTP id i25-20020a0564020f1900b005610ea0e234mr1521271eda.16.1707803851312; Mon, 12 Feb 2024 21:57:31 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVqTwtj1QazF9WsEuJESrhbSbBUy+DkrZZbdQ5tP68MHVnIGS0fIpQkQnzkexB8f1b5vpyAQ5T9P84QLDEqdxhfjzidSw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id u17-20020a05640207d100b00560b059eb73si3336256edy.330.2024.02.12.21.57.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 21:57:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62958-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=NZN5sSpJ; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=dT87G+Yn; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62958-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62958-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id AF2411F24BA4 for ; Tue, 13 Feb 2024 05:57:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 17F0C15EA2; Tue, 13 Feb 2024 05:56:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="NZN5sSpJ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="dT87G+Yn" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA52111C82 for ; Tue, 13 Feb 2024 05:56:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803813; cv=fail; b=kIEnGsdWjcMRyLrziyfBqw/93XbyLwPtdaVC6Hy4ua7zljWQ6r83DsdI5thSNHowdiP2XbCwI0Y722N1eezP+dQGEfIXa5ZA+bDFrt+1hrbmFeLT9IZ4jl+d5Jeh2U+QYYYjWGXefbTlCUSJsq57MYhPoaD+K9oHZDakPUGpy0s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803813; c=relaxed/simple; bh=c7vDiRC40a2gfZYgLklAP7cQJ+DGoX/A63ETunIq6YI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=lDuAOk8S6aWfMUniWMn0TRoF0YEdXqrgQOUPHMq0/6/tTg1oEiYDXZQ3BxSOqBYU7XpC01HFKvuOaOkcdr2sj4VgYNhC7vGi4sNBAueu9pzkgMRbbF8Fe+Gb1fCn6FIgZDS0Ru5v2skX7HY5gFS3pP2oV6nlZnYWGAamSkVHlDY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=NZN5sSpJ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=dT87G+Yn; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ngWY028876; Tue, 13 Feb 2024 05:55:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=NJ3F9MMY35Q5cReOpKoUBDwzW5yjggD5Oj/M+HmYOsk=; b=NZN5sSpJoX1tZXOA8narfhBPwR+HuZV25gswItlHvfCahu3euGa4z9xaGLMFfx6Z+S90 7M9mgIEMJBXy93uzCwgZG7KCCeWhZfNNm62D0+xU6TVKiNpHCU7bdvTic0mr3WjIQtIH 9WzALiLXFSq+owJBvu2mG0xOn3D5EH+Al9aHhGXjhOsFshY6IaCx1K8/KPFMbKvE5Llg YSWa37tgTLgGM3hRWjQ7d94zArQOXrKNrpyk2kiN0N+Ucso++iK76WRHzHawPeofrZPU zxyYr04lm4TxiDT5HyAPohuBJiBazFGF9YugsYU916YGLEHUhEhff3brM6wvvUFkTmBO mQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82m480gr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:56 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5l6cL024619; Tue, 13 Feb 2024 05:55:55 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2040.outbound.protection.outlook.com [104.47.74.40]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3sv1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nbk+IngOZi1B+62OMyRqhaCeOzBnwAVYRYbuxzoVgSbZxE3B3SBHV69ud7Gs4MY0RxP9hbjFhC+RwLcGa43feT8q6BtzUHq0IojxyRYj8YNRtrEftal7nLtp/olE9VpZEcWQ4mcotQPwerQMYM+ywQ2iu3d/U5SnU/MnJMxoK9gk5aTZVoeO+PGT57ll1kNvN6THk9olH5jPIeeF+0Wibxc5nrcNN/vcWAbVL4iL4eWxjFlnXEBFKm/t20PWW2PdCajONZb3f+328IznfCg60ai0sbnqpWiDM08Mr9DBs7f5TEptEuBfCjp0NyNvnk9LmCKxsZCyD9HOTa6Anhb5JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NJ3F9MMY35Q5cReOpKoUBDwzW5yjggD5Oj/M+HmYOsk=; b=XMv9o+XUX/ivs5pEBCsMlwGNFC/Yl+R51UCHgfumMLw7K2xrCf5m5ooM3whBbfOun0jdv+fXN60ZeXGJkWiICiSlFIv3K1bzZkVEBE2vY6YJv9lngR+YP4tgwAjhhVZNlarKwD8KKdIpyPV1RxxCzk1r8tpoOQ0djF8GSRT0F1kDFzhMGOVmWa+6wFrPfwuAIympm/HaM7zgKg4c2TNi+TxOFLfxPxDGJiuhy6uGfpP0xtZG4RMdLeoZVeQSSMMe/oWc02OdlZuYE1hJ2qZCGBclPfIozO7SXOJqckML7+SoWACLTZjFE8eAvk6jkiF5oXLIC6u7YfDcxpMAnsUZ4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NJ3F9MMY35Q5cReOpKoUBDwzW5yjggD5Oj/M+HmYOsk=; b=dT87G+YnYvBLrT8VumphwhxnMYn0CPY1fku8nE8vlS0QIf0DSqsZAEF/2gjbQiKhXMia+9BbtFSm9KOYviZ+GCJpkqO7Wxyk0X24xSDnxVtjQPjTf3EnmvL8ZbfxQTuO1CMdpsyrwybqEPD9HUaoK0bClfii4nat/k55dw5iSjA= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:53 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:53 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 06/30] entry: handle lazy rescheduling at user-exit Date: Mon, 12 Feb 2024 21:55:30 -0800 Message-Id: <20240213055554.1802415-7-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0387.namprd04.prod.outlook.com (2603:10b6:303:81::32) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: f508e533-7c47-48c1-9dea-08dc2c587084 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tPdn2McKEWQtQ+JiCW5rVWAbpGBM4nEqrmkaRR7uabRe3DoesHff4Hi82Cxk2fe5WDaz0aUHYtS81efDjK/fkoCq5gcasH7+qPAzpGmHW1gv4MOkizUDs9E7HA9mf884RjM9Dn9Nf0cqlkzx80MpkmnoWlZM95milNvmpudcML4Hrc7itOVOLT7MRd1+lAJQCcnNrUb2iUJF/DJwxNyEtv/G2nTtoMAnEcwunOhU+2kqq5RkJGD+J26psAW5j6ZzRZf3hkB4YovBu7aq+IPpsutUturx1MADPcPWiFy1RVqX/qysSIVJHHPyQwI7ClKdtLlp0TCt0kBH8dj/ms+/hqxJQJGoHGzBmKEqFNZRxT5unLHWvdUat4WAXW43NOLxn2+tfMh6CgL2QTcZM02tz0tmkm/wkszYYhP2rodVzdoTWPipTblHLRYTS+iX7G327gfye8jy+8I5lvhQ834R9J/UfU4aa4+ZnFRIHSWaRrXeSt2Agr4yzjzctpvgFdWExT0Xzhf68Ipp2h/UVplC3RMJOmZu8HiJJlNRFsuUs24= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8TWLDvlHT1A9q6YQU+sr9xAvMKEd//CXz+eOzSqyrtO8wpnfMwmRmRqHdgJfEq8MOLgHJVzX/cOmyOPbbi4y9dVf2UxUpnZPh2WbJdq4dX7q97f/WdTbkzdo8cGwww1chQ615fzEw8lgmS35eYLgT154OfU+ArXbm+Ioaxcln6Wqz5qrIbs308YU17Fjb6ib3ty1lAM4gWM5utYAFJg2KfizsZp9gMBu+lr2jUIKc9BGcDmDmerQFRgAolaYcQW/8z90SvcLqPF65HrfvpLO/EcA0lJg7fvcduqVITz9UWsfXk7yqzlIjWFdPYQ8ceqVThNHgpCHv6r6KII48+o/slY0X9BV3OSjZeYIx7nWgHdeXGwO4EMX4CjooMREDudeI2jq5aS+fk51tE+VKfVcUHvvo8rBDql6l14ro5+2nz/rIQJAu9RjP/BKq8ECmsAeLleHJJt4Cjb/VszgcuS8Z7ojayhEMNWcKilOG005lpIoKF/c/65vZpaYQL8s00ozt35tgQM+VzXJtIyoNVqTriQne14nyV85ZyVjoL4zHlNGuBbnq6euBX5h+OoKBkh0jbwwWwj58d5L4GyJUUX+wGunNUT2E2GREPXBMTdA0FXF9SIk1aiYobAjY05BOwc+/WhaBLfryCdvGa1MZCEBWAbWtJfpIhDC0qeG472TRJ1QkSRJTD9UD2n4OfuaBnNK8IWX692+GBu9MwZYIDvQaWnZejvRFTrjzn8WGo6RCOCRkNfDvZAceqX4moIrGsbNgEFgCzPP5z1e49g8g6BbVF4G9oouYah/83XjTDwXXKe0crkxXRCX+ELZk/MOFWWWOe5gW660/QJWZrWcZ6xk0lx4dRuPBeo4BDmwW+t4rhnWvEbiQF3Ziv6VyyfCr9n1aaqziabXQWfRpA13chevJ7G9nihELovGZCKbQZpL9L1x+bN6T8oTmDusCkY1snNNz5EuGkF2u0W+wwrnhi9BbejpPggFu2RCCp2Grt7VAQyQBr7i9ixG7Zu/JvvDiUMJ5T89FJPiVNw3aUxObVzUYrWnugvW/MQKvCTV63T7u9xPCGiVCrVIjUjDJbi+dVWHDN6JM/J0OgGNkOdGnnFf+pruuRf6KymRYftFLlNrpJsSBWVy7yTg58YCEUHGmRGgY1ARLNyNpUTCkofGASD2IVuKIQSN88kPbcJL41Yzb0EkKoT/zbFYUwgF0xM55+ca5haw9VAhe/tMvHbzV3V81tkgq3Di9dIL2/vV+/kBu/hkuog8sRQ3lBG7TwhhbwUokrk21ClNhi6xSLTnXQInwQ7dK9/T62bm6m7j8ZI0ztTXu3rOMbFxkbazdITFxEhSaN0Gg51RMeqisP1gG3upyD1KLzLr8HNfNaX0nLV/AS5ju/qyRoKrMi2+mSLIM1U3VjNanUBRAteUiPSt8ywBl4QqgMYRVIgKSpb2la3vd2lJBabFRqLVjIdf5RcMi5Cvm3nZsrOzcFkCsL/WR4rSBQfM6ROvWYefdAL1BZaAj1GbmwCL6XYKgNst1jiDedfo3H7+bpmTaz5Ydf7lWTK+mcvaaD7tefYX30Bmh3BQrlv7mIhDuSImyfT6cnIh7LjftZ593zyTT3dfpdYq/8I5Jg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: MB5jCMiQN0XTQ+QLTc0uIMZ92Cx2s1y2m7rTyVOx2/rzwis4jLv6TOvzuoSYkJae3jHd+7rPUm+HYXN3uCqwWiU4Y1CtVbHHq3ry83ObmOePEFktDbab9evSL/UZUMbxG7ZAe3edLlcYFb2yzUX89L7cmcub5cyDO668QOYEnzTaN0LQmxdwkbcKuUU6+T2wtheszpAm6pLN507JLBpAHwO1FiuZtXVyY17RYTKLdOxCiE5jj5M53OWEkZL6ai2KFT3tgQTuYZcTMvDiyHCnL9LTY38ukHDeo11kJC628KpjyFuCYnv8RrQJX23o75bA+X2K6cGqUtjxcDd3csKdlTLqof2jOG+OJiiNYg1RHCcx+ubXqkTjqUCxmu5n5sTcFb9R+/G937weRvZ0228L5hFc1dP1P6w99rWJTrhg6OOrxe9XGyJmOq3rdLDlYGrOCgaxcpY4EmZETPrIeduA6HJRDQpfUJwUKzNKxcrhHZSrERARYGTcI19OkzzdAkdR9HYJK7LVQghqYzzEJ+PGul2PSuNoCo9HQMwyEO7KHgd3PwEjDhORtRZk0Dn4DJ0cNaoV9zZMqEiUWFA3Tu6e3argZA89VKAqliMM6pUlP00= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f508e533-7c47-48c1-9dea-08dc2c587084 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:53.7565 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3USngZK16H39tNBdEHl6eylsdjV8qTenvHev3Q0/SaCEZVM5ilffRz5ehGIeKLshxlqkBZposJ4rm8Rlhtg/g7wCp83xacvbjJ1Zv/d2WNA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=611 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: VjPYC7iAKuk3zqty61qdMbSb9u94d3hO X-Proofpoint-ORIG-GUID: VjPYC7iAKuk3zqty61qdMbSb9u94d3hO X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762131592404665 X-GMAIL-MSGID: 1790762131592404665 The scheduling policy for TIF_NEED_RESCHED_LAZY is to allow the running task to voluntarily schedule out, running it to completion. For archs with GENERIC_ENTRY, do this by adding a check in exit_to_user_mode_loop(). Cc: Peter Zijlstra Cc: Andy Lutomirski Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- include/linux/entry-common.h | 2 +- kernel/entry/common.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h index b0fb775a600d..f5bb19369973 100644 --- a/include/linux/entry-common.h +++ b/include/linux/entry-common.h @@ -65,7 +65,7 @@ #define EXIT_TO_USER_MODE_WORK \ (_TIF_SIGPENDING | _TIF_NOTIFY_RESUME | _TIF_UPROBE | \ _TIF_NEED_RESCHED | _TIF_PATCH_PENDING | _TIF_NOTIFY_SIGNAL | \ - ARCH_EXIT_TO_USER_MODE_WORK) + _TIF_NEED_RESCHED_LAZY | ARCH_EXIT_TO_USER_MODE_WORK) /** * arch_enter_from_user_mode - Architecture specific sanity check for user mode regs diff --git a/kernel/entry/common.c b/kernel/entry/common.c index 88cb3c88aaa5..5c5befb05410 100644 --- a/kernel/entry/common.c +++ b/kernel/entry/common.c @@ -92,7 +92,7 @@ __always_inline unsigned long exit_to_user_mode_loop(struct pt_regs *regs, local_irq_enable_exit_to_user(ti_work); - if (ti_work & _TIF_NEED_RESCHED) + if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) schedule(); if (ti_work & _TIF_UPROBE) From patchwork Tue Feb 13 05:55:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200229 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp350804dyb; Mon, 12 Feb 2024 22:07:59 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUh1BAkSDdwLusySossb4ckijMWGqBL+ZRm+4ahbxznXfZEl6O4HDcDh2+pmSMcKTo5jJ8LCeYnWjsNswvzStjQdhyafQ== X-Google-Smtp-Source: AGHT+IGmiydaybvAF+oUS8K/UEOie27FtreVPd7K9bsNoE0YFwl17TDlAzjk3F89W7Czxta5b8tj X-Received: by 2002:a05:6870:c10c:b0:219:84b8:40d1 with SMTP id f12-20020a056870c10c00b0021984b840d1mr8702865oad.20.1707804479779; Mon, 12 Feb 2024 22:07:59 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV8alWZ8Ca7eFa1Ttm4C3Efow001oolwC1/0rI2lyvD2o0uGpssM38FErAqUxlIgQ+Igh6KFHtUf9Aqu7EjeKKDpCmXeA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id p25-20020a639519000000b005dc42fd8c43si1475740pgd.267.2024.02.12.22.07.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:07:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62968-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=N00DEEaO; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="dPhkO/nt"; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62968-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62968-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id F0263B21833 for ; Tue, 13 Feb 2024 06:00:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 49FE122EEB; Tue, 13 Feb 2024 05:57:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="N00DEEaO"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="dPhkO/nt" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9B1C18044 for ; Tue, 13 Feb 2024 05:57:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803826; cv=fail; b=SKqYwWVdkQX8nrHdCm21iGef6xxwPqReft7RCJDjGl6sHS9zejjE+zDXPyjm+sPqcmAoGMU+qizYcUE3xxxQU7PZCOUq7E3v1xbR5EgOsu4a/SvvJIHY1avQNptSjPPGWAXbFPKq6rOQ/JcJWo4eCy3aZtoU+j2Uc3KUs9EvzGk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803826; c=relaxed/simple; bh=L84gfAYQ81E+gA8uIm1JdV7lY8twisDRcCpKSLCIelw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=ItwuuVpMCJiKXFpXGlu6IpKVWTIpdfQ3eC/1vv6bGGZ3cYBLD7v9HHTuhpDhB9/s6QnkBhM8C9grulIoMwp4j+uBfT6VMFJYb8gcdPotOPeGY1UK/6lLBH6b46T7mGbKdDnc+vA2KWNmTJbauJyp8x1zW53xZWA+1oJb/xm1FCg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=N00DEEaO; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=dPhkO/nt; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D4xtFE000302; Tue, 13 Feb 2024 05:55:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=e6EiB3lC+Ersv7Mw7KyxtkiBTvGLpqTJLPo7B7DSKZk=; b=N00DEEaORZARhZWb8JM+U9EtA1LnSbGmNoY2cuMd4n09LchHfBrAYYVrUcGbe0nCeARH 11ZeKBIDkIhY7S2bM6kJoNP7jJ3o9XbO9vzVd1TH3WB+ru5aN2jJYKd73iQRpMIIidsc 6+6IqeifSi19E5L6+9pkAkMaNL1Byc9P9kwDHgYDwm9RScHNF/NZzsHJxs0d8jA6m1MO ode4bB/cPdD7lL0gr/0EC/baSJjRh5boi/rhA05DGqXER4aV9vfCaXxyEvp0o3g1yx9U Gj2ooqaFbsx/Q7ntc74FSFhuRvY3RMF7GJqInXd0HZlTjrNIk+L5vsxOIvATgXIwASYY YA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm8360-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:58 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5VHF8000527; Tue, 13 Feb 2024 05:55:57 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2041.outbound.protection.outlook.com [104.47.74.41]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6ufx9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:55:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N8ZgWgyh8RyGZ7UxTANXS9eolursCa92nkNIiXmLSG9gu9UC4GkQCSaq22UY+LS357XYDoCYp4pyeGzSyECabC93Rn6sGwsg/JpNTAwycDdVPGDPW5b3xPUllrZHpZLviA5M5p4GsTfg/xQtr8w47nayklNWCo2Wh5rCHuN2J0yDzKddviqB3MMLnpZK0fJ6WGYW7FaQ6ZpidRChacscDWDPtp+ZqIS2WsyEiM4sve6KiQhLf0pfyBmsxGqZR9WmyOgBt+uGbZxlHvpYk4tE2KA6Alxux2WAHu8eWU1EB0sWrV+Flr2vpecbzyBnCpQhDye/VYm1aJS6DnVq0Zvneg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=e6EiB3lC+Ersv7Mw7KyxtkiBTvGLpqTJLPo7B7DSKZk=; b=Gm9FE8FLyTs2tfwKuAqTH3smL+TnAjjDOZBixLMHDbUHFAOFIfuQI7YEJGi3vPCdUGCV2F9dhycGeQt+KNTo9K6GOVXcOedwvLy1jYibNh9iS9sFPHve2jw0YlNQ8KawX9pxWKthD1OBrtBUmTNARpNobvwOB3zarnYJA0pzpDOxkdqFKxACagPLrWG5wJlbx43ieon/28GGwqdxCj5elGAfHZ0sMMImpVWsS2YmOlPzRPY0CYptRi9JDCtAQTPnRpgOjD9lVfWm6xHtC3UJZ5me83eRowT+qhWbdA1KrmlMoOMEfFuUZmE0CrHC/PlHow/wEEk9n68vYbl8tclUsQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e6EiB3lC+Ersv7Mw7KyxtkiBTvGLpqTJLPo7B7DSKZk=; b=dPhkO/ntAvbDnvKTwAtqUshsjCJy8vtEeI5L5kSemEGLZ2QTpEboXytbRr4C/zjRPA/VRjux/0kI2JBKHFMpNHvC7i6S5s/5XpRvBwcKYEzS8k3X6K6q0PF15Y6DNN1qWpSzQJ58R/q6ZoWzGsVqm9eYFYHO5VBQwarYvEU0cPU= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:55 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:55 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Paolo Bonzini Subject: [PATCH 07/30] entry/kvm: handle lazy rescheduling at guest-entry Date: Mon, 12 Feb 2024 21:55:31 -0800 Message-Id: <20240213055554.1802415-8-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW3PR05CA0006.namprd05.prod.outlook.com (2603:10b6:303:2b::11) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: e7e8c939-38f2-4950-f0c8-08dc2c58719d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fW7s31Iyo3hzAU7ycaasZORpDNrwnjntV0wj+a/xhHonXGGd8zmiYn3l0ioPkC6WLBngKGgsS3s7CYQqrGZZeVN/zQLyKozBjPHeT+t3ED7OLbnh+RBYCdDRtay56T/ecu6PlUIBIInoLwM8b01w1FuXoIoni01GVfTDzvu2Hspy7DqQqBKMLSvrKITGumKfVyNfVlEIzVSZax5azsd6dQlFJQNt75VGluqqvOVpCv4LWFvO6LtDgoWJN5E+08TS7+vjgKI7vEv0grCyNL9HGxOaO4GUyrUhi0DlEkUmOiKwbr4TLV2uNb634UUPc/lytwoy6Vv2VLeAlFxMCntCZXketFf+c7MKhaDiex/xdh6tXztcI6waeP66UvmvX/RQUw1ofsk401cngUdivbvYRfSPLLpDZD8Pt60xs/GxJfS6L9LRGmsHzK/KR5mNpcjn5l9mhhzx7Y6pRT2RcrSCzB9wIHvMHlDj6nLFCOEYffMPjfM5iBUIbQU9Axc03jeNxqyhk7MlLP0zvJcJnjoqe7Dv7WhUs13/mD0MpPfcnoo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(54906003)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uXjjGIct/j1Z6QBj7gEpfjAO8+p2Xzc/wEDMaEHwNyV+dpYxXbqdFMsyoZrowjf3i4gRZ2vx+4DTzYX3mMgRGmI2F+h49DtpZI72e39jNa/igKD1k3Q1FamHB2owSRQi/caCieb12YAYdVKLtRjDx+vfW2y5+ubE05zGNCGwibZ+GfOXGyQ2O7ohvYAE0hDQD9cbG8Ua9avnobXzI9ynAbZY7J2qYh6H5Sx23YLKgO5i+E0+riWBe6oUj1yRMFO3R6hw7xyPw8lix6FGi9YpUl+ONPA4EY1KlFQXeidLbzUt1vuQtesvtky+YJbDKVMbkre4QL9xl/0EdQpNYY+m5lPTNNnKTz3ex37rH5Vk15475R7Z+aQrVtiXQL+7kTeAtULp0drtCmcbYjjeVHiYlj7Lg+tqZpUv2sdKXcQuwFz+7JbrLvvsVHPezfnHI70zm3pEOzpVxoExYtSfppMA3nJc3CWQOcwqWshXiqeFDbcIDIrZkAxbDwLIL9MHyEEwB5XqCHc2p/W7spTaHLSdlESTDAGdcjiXUsEifdYmWM9f7Y2LmFCHMzZq+XQJYNvrWQyKJUX/0vtZoz4fd4lROdlYHH5OTkb2W6S0IgJG60HvpCZPDiJyCE609f30rf7tv8JU6e5y+90RGkJiDyVdiwfTszF42IY0gx4RX0jramW1JtYI67DdrquAxZrruAyIGWBpagGPHxDXEYKwTgPXdy6kbaWGsdzwEEiFhucjfban+JcbGuHoOzm0kocrDPR8Jeup02pCkoAe8HaXtW4OvgU7CWM30NYrIQlKj7wrA0q6mbWpg3HbW7Op58qM0GKPaYChqw+xr/GEbL9TSEItrQ6kj7YmXUk6JEHoQoDF7D1bTpYQ44+r54GbtXiDDWJnjvdN0C1gFrlnxHFS7sY5GorcJPD5I+FErhPQbKt3P8BM00OLlJ5+glSEIq0oYf9dy5dMyZZol0WtmMSJLU9Iz4AOyaG3Qi4ALUCuDtPYJdAzwWDO5Yq+nw69JRO/mMv0j1HL1UmRehAVPaFfvbvfaX/Z/Vh4xBlZt4QC5vDGEpiQXs0FCtSUIqOxOzvCZ+JHzJoOSv8gZ2ZR+CsttKK+Ej/uH2laU7lmQuRIt2ncc8Hf6LDXa//zWJ90tSk/c/FgB/2wHP3UHOVLBHuAprAgwlM56LfT21l9D8Xtg091PknyhyYZJ9nVvr82JywKvnMAZr+/ezMNU3q7vQl9Qqy0os2RsDJrcau5fDqoGxWVlSAcfjsABRjKRXWHXlx5qXN2OSlHzk60NJ11N0pMgWZMSqjEhDltOvgJ/Ci0vzJ2dIVC5eZIKDzIahrl7dh0LcsChCzhTNDeuFxvxAJl1u+kjf8GuK9hxyVf34iUy5gmN+1Nn8mw522sjnjWAfhwASbioYQai9cfD6i0rRkLFvyvKnwWf2i8eQQXbkqosT+fH8tWC0h7vl3NYqDUDKcRLfL/wEyiNi5iaaUAn7eN5nKaCZ09aKZGOCkqXJ0lzW9JUAUGuQDB5m88ZuUzU/rytBitBZ8UTgAxs9rD59Ry8QW4Rtz8zEikb7CZL2U8MMp0olNsF/UI7v6hnz6KONnvasR2Z0msJozjW6Yp8ODdLs1h3Q== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: A1OCSDJWHJPUzHM3SL1fRFGKl0gxD9Faqk6T9/lDrZvbm44VpD958Kb9+Tr6xgq/HAEVcCwHFInmRuvUGb7FRFvMvIWu7vNMWw76Fj8wR9h8OfVJbXabFj2HsTVIWEI6eSNrmxZM97TZdKF/SGrIGRAMifWSTaMo+GcHstB5fW6yfpZsnU05WAeIm2oWfLTeAG4y7h3Lx+tROsoCm7CKThnPzc/RcybZFnrXh9+KGsQmpuyt1MBp5e7ymMb2m9SnNLCmh74Yx3zY4qDtwzNwXwrB3FAs4LlCrtFFSpl2cH7WCFx8gwS1OOe25ZFKjouBITTZ0c5tmPUUQ/+VxlNc0OslY/Ubb+JdiNX+IXwKFH+m6WZ6BbKXE2FxilvylbShk6xYEkkbnxFoGtmDVwr7lsOCWAVqObx7fvOqFGWaxG3vJI/rR1uO0b65g2KMnrbG+AjlXEiN6IybmL9tzoUJDIhxNzvbV/73Eu8jTdLAMhGU5HsUOQeBbPZh62ojg9quPVCcazyb2cABr1OZtKshCXHZpJm9WkbCoiZCg9OHZWKAp/LVw4pqAJGECHvwJ4r/8K6/LrUO671CX0I5SE7Yg7TTGn76UKZ0t7L6pipcKgg= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7e8c939-38f2-4950-f0c8-08dc2c58719d X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:55.6332 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NYRjJ5vvaA2PLH4R1G18TbBEXmRCNnH+CH4vImvfUghwYg+ti4UDfO3shLlP0iU3omBtmNOPTt3Ud8d+efBOgdm5qWeWLcpfwWT39IQJHqY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=718 phishscore=0 mlxscore=0 bulkscore=0 spamscore=0 malwarescore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: DHA3pgLUIH21BP735zBhJ29t70HzAB8d X-Proofpoint-ORIG-GUID: DHA3pgLUIH21BP735zBhJ29t70HzAB8d X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762790174174758 X-GMAIL-MSGID: 1790762790174174758 Archs defining CONFIG_KVM_XFER_TO_GUEST_WORK call xfer_to_guest_mode_handle_work() from various KVM vcpu-run loops to check for any task work including rescheduling. Handle TIF_NEED_RESCHED_LAZY alongside TIF_NEED_RESCHED. Also, while at it, remove the explicit check for need_resched() in the exit condition as that is already covered in the loop condition. Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Andy Lutomirski Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- include/linux/entry-kvm.h | 2 +- kernel/entry/kvm.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/entry-kvm.h b/include/linux/entry-kvm.h index 6813171afccb..674a622c91be 100644 --- a/include/linux/entry-kvm.h +++ b/include/linux/entry-kvm.h @@ -18,7 +18,7 @@ #define XFER_TO_GUEST_MODE_WORK \ (_TIF_NEED_RESCHED | _TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL | \ - _TIF_NOTIFY_RESUME | ARCH_XFER_TO_GUEST_MODE_WORK) + _TIF_NOTIFY_RESUME | _TIF_NEED_RESCHED_LAZY | ARCH_XFER_TO_GUEST_MODE_WORK) struct kvm_vcpu; diff --git a/kernel/entry/kvm.c b/kernel/entry/kvm.c index 2e0f75bcb7fd..8485f63863af 100644 --- a/kernel/entry/kvm.c +++ b/kernel/entry/kvm.c @@ -13,7 +13,7 @@ static int xfer_to_guest_mode_work(struct kvm_vcpu *vcpu, unsigned long ti_work) return -EINTR; } - if (ti_work & _TIF_NEED_RESCHED) + if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) schedule(); if (ti_work & _TIF_NOTIFY_RESUME) @@ -24,7 +24,7 @@ static int xfer_to_guest_mode_work(struct kvm_vcpu *vcpu, unsigned long ti_work) return ret; ti_work = read_thread_flags(); - } while (ti_work & XFER_TO_GUEST_MODE_WORK || need_resched()); + } while (ti_work & XFER_TO_GUEST_MODE_WORK); return 0; } From patchwork Tue Feb 13 05:55:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200211 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347662dyb; Mon, 12 Feb 2024 22:00:09 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU1Bjbr2TQC8y7kx87ZNWSgQmE+P32tlBIpAW+Iji4tzC+VB1ZiknBeOutwDaBGFivD/CiJMTrFe+nAyM2va6qUhBOdyQ== X-Google-Smtp-Source: AGHT+IGu/lUUhyf/jUFz3F3HVyhQL11osKZ0yaachNNGg2NT00zq8ZcmuiZTBWgeAOHga5sChwEx X-Received: by 2002:a05:6358:895:b0:17a:d4c0:d59f with SMTP id m21-20020a056358089500b0017ad4c0d59fmr8063347rwj.4.1707804009227; Mon, 12 Feb 2024 22:00:09 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWURf1fxtZT23kNTB1+27B9+vxY5/Cdxr2ON/4GEy9IofIBJ+AXguZ2adlZGKWnn/cA3/GJA4gk9nJN/klaxwKhT1Ht1w== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id mb14-20020a170903098e00b001d9cc9f85a7si1424084plb.566.2024.02.12.22.00.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:00:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62966-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=okoVFJZW; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="vBS7hq/G"; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62966-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62966-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 20A30282302 for ; Tue, 13 Feb 2024 05:59:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 19C1D225CE; Tue, 13 Feb 2024 05:57:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="okoVFJZW"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="vBS7hq/G" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07A88179A4 for ; Tue, 13 Feb 2024 05:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803824; cv=fail; b=TNVE2EAy90FFrBwhDbc6rJrNHSeDpEI9VKwe02Z+6sF3ZriNbJu53EhbddQqdkDjyaIr5liOSsBujK0YC0Di8307A5UhfsW9g41ImMhIbseC4CgOuE6fqCMlDUeecwIR+mowMmFpvVHnvvwk6s2GsTtKjietEyFXqkAYT79zWeA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803824; c=relaxed/simple; bh=Dqa8KlGXrm0C3zFw5hkLi/g8R6Y6VLTIxlmx2O+YFnU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=BIxJIjB7/LQxzjE0+dACQl5/rXmY/00Ue5XDfg5hGfV9bkeLRYaB7oUmCXfSdSC+Y6NSI0P4PHYW+avtMjXohkWWC9aav3fjQF3fwCoKU0HVLSQ+rf3PO9ukDwko/h33V5Iov7F1XSEHguCTJQP2gGkEmeAWUXz/a6AAy/mIkeI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=okoVFJZW; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=vBS7hq/G; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D4xjrA032720; Tue, 13 Feb 2024 05:56:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=kwO0FTgMGeCJOFyBocC/+VIY0AfnF7GdW+hu2rV46/o=; b=okoVFJZWkridKZgN5EBJWxXV0eSRad6kMThpCN8yuGRzpuAm9/5OH6gdfLeTVk1xsmqt ychH/tDFoQfdPMh6edJ4vNma96VMsY7pwO0vhdHX/mXDq0D6VEvCvHEP5fexFMDb1Qyq pW3D1W+F8VmZjh5hnXn9pJYQsvDqNBqTiRk4KMAFCwMWS5ILFsyPLCubJDcJ4JdOvYyw dW6c9J6B50M8OMiZORyBTShTEg/oETD58GqALGorPRo868MVTkadx0htch3dcOjAriAL jVchf6nGpEYYsQ1cByojrZu1XX9OJfn4lEik3Vwh2yqO6sFrhO6oD4jUF9C13Ss35T5c bg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm836a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:01 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3JiMx014926; Tue, 13 Feb 2024 05:56:00 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2041.outbound.protection.outlook.com [104.47.74.41]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k88v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b+yNPLLFxmWy5cfjBkglVp7iTnrHosZy6hA+5GtUKem3YM9kBV+imLBx5vFC1rRhPXw0i2EMiyHkDBs4ClmO3EATenZ+0UFzf8WWjuL3fimRSAAMG/m3ssTU9w7Q1hYAJwCph6j/V2Njw3JK0JgzQfDezw64PUEmyMUjfChHNwM+O+fnkSiHPiTavW9k3SpSZesA1Uvhb74UsNPPfcHmw+KalfWavlV7yJai6gVjFozsKN8G7P7MysnCzKsuFrO4KdRTQwKRiEl1uRDrj01jN1x9iUReQAq3AZIqGwKLVyrJVChIdOfJmPyVb/4U5o1hwIs0/ExR7xElfxCjt0ekww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kwO0FTgMGeCJOFyBocC/+VIY0AfnF7GdW+hu2rV46/o=; b=QhaaOf4zKgqq9WvUG+AkczwmPBwGZQBoLi//hYzUtyTeY9wpbp+79dqwh+hjXlcAEq+q5wXbHioPPrnsEMi6mWSYIR2yyV1xL0/pZ4ius+d3e7t2TIGd1BSGdlAx0LuRWgXLnZsYBtm0YtUQAUxWbAP1iaOBj6aQ6s+6ux7DwCe3BCV/4PTwfzmbXnpNNbgNMHKYMvK+SYSq882sJHNeesfgTqDQcNucuM2U8BgLGGerxmF552DJxep9dW45z7voLalfkgEPEdJU11UcQcq8wOG/FNfHHTi6Cpbq/cZJDEnMrIAHd7DriAWDsS/tpexxlkTIBIW3ybzv1YChk4akLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kwO0FTgMGeCJOFyBocC/+VIY0AfnF7GdW+hu2rV46/o=; b=vBS7hq/GnW9YP+7t5IWMTqRcnP9JGlY+5J33RmFakxAD7RfINp01l4v5lfu6ffaUSCP86lWgzcBuuNvfvw0+hcgkTf97TIgz5n3S2bO1Puv2FLb9PPXWag3RtRodjGteTUdlrbCS9Y1+f6dLczfUEjJhfw7diawzw28YfP7aEao= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:55:57 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:55:57 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 08/30] entry: irqentry_exit only preempts for TIF_NEED_RESCHED Date: Mon, 12 Feb 2024 21:55:32 -0800 Message-Id: <20240213055554.1802415-9-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4P221CA0006.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::11) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: 05749a48-769a-4857-1aee-08dc2c5872b7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2hSL9Q3h/PlGi+Rs06jSWaXNZXaHH8rh/HsSDo3/3m//eF+UPbG6E4X3QO8F0oSxAwJhBex3/3Z37AHTnr7jF0N372Md9zdEU9AV4o1WGqT7IuPue7oTXwwQYXRJxxOZcHoQQ3uVeEhNkiP45r8LR708JPlsmYMFOJP6SsR/dl9NasJP15MvokZ9HJUJQ6W8zsxq1R8DQ0QFAOf4sTrdJj2EhY+O9v0/owGi+OMmwLM3PJXXJ2+z3mw/hT3aZcmUYKcBK6VHw7SMwpVrZc/icgL8XqAfIWNOrZI5+30zKTNEcS3ua0aR3LMFrnge5IDhTXutCVmKWRDcN7GNZaOUpS5w6qqdA1yXbsTSrDKx0CWsoeX7AmxW/B3MH8Nrqbr/DDQC7AiiqCcwj7gSTIlZXKropQAzEIONdM6WLAPXeDQG9ZnzJ8vNRZO74okotkkaCNgA01PUkOwSyDU1mYzZmTmtoLs35e6i+9FXKMjoudwyzTGwK25lz/3NEZibvrtgpyUTxEuvzbFq+1Wm+psAm7RXh3QkCldWqaD4GTGwBA8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1T+Ntj9g8AxJA4LQcVE8f4qbCyp/cTgETX0tpaben0XtfqqCl4Cv2NA7K2PSGYOTp5jYFVJnbCQVfBDO2c1FUYRV6Er9Ae9yF897h1rB/6X3hKLpORBlL/ccekd/C4K9jzGW9ByBaoi6Sx580VVKO0vNTE4Oe0TueM79ijv+WNZ5ir9l9MT8/SBYjm07RwBbHQug6M78KY2xrwIiTTkI6w1vwDrVrKU1xylgScCCJqkiHaxfDo1glACWus8Xcen0W93zMpehSK9XUAiAX2BgjyVUXf+GE9rgqErgabzNAh/lAp/raHUawe6VRQfbxmzL+KV+wVfuXTXK18s2L94ZWgP4aRhciHc8mENsFnv4dOi4jHc2BRHDIKvHKB9NLcZK/KjM+kX94lohJEcFkVlmkk8lReoTl7fj6yCEanY5DiAgCIzmsP36Y/25vOqjTG7zvrVLL3fb1aBv2eCRSUYFYJumzhWhFAIYoic5An2wIWKJJRUZdOXM8KKJaVmtkz1VSFftCZ+hXFLP0lJJd0YNpj5BkJxZw2d4sLW3WnptsvML0iv7HfOuy8YlCjV8/kUe/oKv6WAroErJ9IftJlTYVt9ZuYbRO6LmUIWK64vsf3Fr35vUabR1Bg/3i5UwR831pwr4J7/cDpDQvV6djH5axuOfIQE7pq+h41Rs+oVqx7XIqhMLJVi/4OBRIv1BiQCUECW1UnNrpjjZbASzmmW407eqEnrMcIlJcVVmnkSijBneOOJJ6CSBdF74FMeGHXJ4qNhuzEiBY30lbQ1aqPH5SS6R0qT4cjHUOoE95RDSRZee4l1FeoAa5BWH0p1ymrlf9btuVLsGtsG0HiAZoMHUvc2jc9+9JYu7cQc2vNsSpyUjNYeAn7lDq4MgOre7eYUdcFdnNlro0dUw0mpmo0r2iikf2J8eskn3fBLO/Pud2xE8kmhcPYMzzIcJHv2tV7O2unHozhZrHFwDgFnmbjjBDw3eKn0nQG4hL5GMUO4taumo0Bt8AHwl6vGJe+jYAbg1FQl2OHrvMib/E0KWgBBL4jWodqlQn6F3ESyZik+QI1tDfX2FPvNTt1NjvCodyrnfAPr2K0mDPHInjBUKZJpA58tTervN2NjMYGa5pkMtdfsYEYIxQnfh2GXbnee5OD4qYDmrLSyTCz1WHTa8iwfc8irJ7bj2taEX29S7DQ6dKM1w2Y/uzT+5xU0plN2wltTVzOuuywY578rPL2lxhe6O8WbMvvjtwcixweyQRDNsS4u8xoMCkaA0ze7IlhSMt5QdO+M2N3rs41xA/BUs8RRthCxFNgfIHi/Z1sNLytmbvFog9RGexh7+IvFzdB8KjNeOuVnPeSsGvONOTSveS2mjyinbJzj5TIFy1SfuI0mW6D5gcxBaB7x2kVq7GHzNc9I/nQ0l+blzdXovztfZ9mfqCeR/z8MzplOOeSeebDguBJmEsL4XLgtRddgiMbUm3Gc1OhHlBKc2JA7onj2xMRgLT0h6dAc1yUvHA13dugup/b67w6gdgsiokn5/kau6HNCVzbrsIB/HOjVUBwMRMSmIPdYXsmjGsLbQHO3vGM4DVp4RQXz75uQo/2f6w4MlHlNNuW7oD63kMfYwxEXprWz8YA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: NQ2xxY43zMYpT9zs8ZG+BEkrmvGuEkmPEvr0eKS53DdL7QkM/QF8VT1AeTOgMiicJTJHF0PIZrzfRYMuEBZDFAPp6xnEohBzA+cvW4XD63MdN+Tgoih5j5ZcAurNhYszWHjK7DgHHydpcm94LKSKjk5+0uF+ot38DfusY1JLN+bNWQYSeyElromFseiUcl/ghklquXhbuitDtI4rP3COJPMSYrhJVuWRX/FZV3GGzJyl/CwZy+r0KVNx8cg5vb2c6T2nvEEIl59a3rnoX5smdCRGgSYUGDL2d4kLK2KSURZ3TJ1/4xAzlkmM5kaqrzLJZOpSLI1MT2KufEoVqgLMmT8DHxq4OSd09Z3ZT+NK8NW4cFUMVegjUPl7DRGlxs0rW9eZHu5ioxu5o/FbX+SL3MM/7Sd9Cd/3KevZ5Sboa/VQH6/srirDllVU+xzQm6dksKsONWIHH0zsdGTVP0b6yT61m32LLcET+SoiqPZWaT2RUQh3ctbdXWp8jj2sKyvbvNnYpklhLEB+zLfY3kqucdgIS7bwyIgCwAMPgTaVpuloYeOsREfpbFpQNfvYPEIRluIlhE10W00EiQnut0kTO+lp1u+4Qze1+RGJGuxdlcU= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 05749a48-769a-4857-1aee-08dc2c5872b7 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:55:57.4443 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oZ2n5w8iRshsoccNMHjPT6AyMKULH04074s2jCRd0ihFZk/9dB1dA7v+hUqHkeCau3jXVAGp7/6Is8PFswd4juIGYazsLJgQ+vQRVSVgtUM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: 4mymwDjJQ9DjY87fN3ZM0kNxtkQgaRBP X-Proofpoint-ORIG-GUID: 4mymwDjJQ9DjY87fN3ZM0kNxtkQgaRBP X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762296786895262 X-GMAIL-MSGID: 1790762296786895262 Use tif_need_resched(NR_now) instead of need_resched() to be explicit that this path only reschedules if it is needed imminently. Also, add a comment about why we need a need-resched check here at all, given that the top level conditional already has checked the preempt_count(). Cc: Peter Zijlstra Cc: Andy Lutomirski Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/entry/common.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/kernel/entry/common.c b/kernel/entry/common.c index 5c5befb05410..e1e6d7c92ca2 100644 --- a/kernel/entry/common.c +++ b/kernel/entry/common.c @@ -301,7 +301,15 @@ void raw_irqentry_exit_cond_resched(void) rcu_irq_exit_check_preempt(); if (IS_ENABLED(CONFIG_DEBUG_ENTRY)) WARN_ON_ONCE(!on_thread_stack()); - if (need_resched()) + + /* + * Check if we need to preempt eagerly. + * + * Note we need an explicit check here because some archs don't + * fold TIF_NEED_RESCHED in the preempt_count. For archs that + * do, this is already covered in the conditinal above. + */ + if (tif_need_resched(NR_now)) preempt_schedule_irq(); } } From patchwork Tue Feb 13 05:55:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200221 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349146dyb; Mon, 12 Feb 2024 22:03:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IFdLnG5G2yQaJFk3TdsHsEjcq5685rrHEFnTrs1jz7dGNAqth8VWMpprSbkO/BQ3gTkKNmi X-Received: by 2002:a17:903:8cb:b0:1db:2b07:34f4 with SMTP id lk11-20020a17090308cb00b001db2b0734f4mr2018222plb.13.1707804209523; Mon, 12 Feb 2024 22:03:29 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWLs6re9/KWzsaBmIDH0J9YGSdyTX73V3bk5qBxKPgZYh2E4o4dSLS/hFOiDIg2FiS04rZxnY97Wg0XAwjQLa9up4YPng== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id e5-20020a170902d38500b001da31b60e37si1425350pld.651.2024.02.12.22.03.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:03:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62962-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=KR2Kl+MH; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=XuMLjMjj; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62962-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62962-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id B1760B2663F for ; Tue, 13 Feb 2024 05:58:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 666AF1B7E7; Tue, 13 Feb 2024 05:57:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KR2Kl+MH"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="XuMLjMjj" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5272B17558 for ; Tue, 13 Feb 2024 05:56:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803821; cv=fail; b=sS4FCE/f1PF229O76ZhEor8qz21sHL0Y3JfliRD1OrQgm/Ji1joFR0qXUBT8T+XO0jyPI1bm6TUhmcvLoAB68oAbfaeikAQRYDibCIL3K/ce11AX6+ol7LTIRaos7Rw76qK2LcdnjFB2LBh2nUODC7ThTZekXuKZ/V6YZbwnJNQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803821; c=relaxed/simple; bh=yHMRxHn+WSGnJ4XEWqVfaLwJkb/Gh+LmRbNfhdGcUB8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=hIQaN/wKfPIbIEd6IYQ9KhRzPi9D30AIU7LwOW1m9MpHgJAgUi1U3kl04GA8KapfEIhgWsli0NrOThLvQvw2v/ZFdub4zjUj5y8YmRWUFaBjBURiG2RoFrCNdRi2CcajV413pvliuYHLoPtRTX7tTlEmSumfb1Vx7OLlvbRX7jc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KR2Kl+MH; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=XuMLjMjj; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5n3sK013728; Tue, 13 Feb 2024 05:56:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=DvwUEy75MTO450t/awgfNqDNaLpyUzAb+VPNe3a0neQ=; b=KR2Kl+MHZNJLSOHkkNj2umq0HyCE0e+iItBUyqb0ck2tnIhU3OGnYukHgmrJ4MO/TzHg hO0zZdRWaWV4q7RvtdrC/NVjbooYP1gtq78no+HvYFTYvlZLi650XXwjHt0wXZXCabI/ TOEHOwJ+a1FH7EfbKs4sfra2oOEOS6g2WGD3kDWJMiJzV0uLa/0ooDOs1bMMf7FrVxXD DRyj4DBQLWkmDNJoOUDUt7dBCPn5lQ4LSxNF/SUcgW8TsF2yevILtZ2bRQ+4GHR3j98U l3LJO/RZBr4XhDP4WDkCmyQg6ZFgMX2sRfU2ag3CWYsLq6RL4EtUcSwA74utpKMdMTYx Bg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w816tg49f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:05 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3XGS6014943; Tue, 13 Feb 2024 05:56:05 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2040.outbound.protection.outlook.com [104.47.74.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8ak-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E4LSVVlaEqVWTIrPCh8UFJuQ0FocAHp0iyEWOnhL1L+C6/Rgvy9tX14aT5fDLVlRHq1wkJMCNFio7ISKUI0q6cTzJ8nWZZdk0DMfG4YYIjb3RPeAN4+McBhWqJ3dxeU1nxLBweiOrcTTTg5DvUgofG5aoJR6G9WsMH/hsusOjTA3mfvpwKMFsvXhboYRJhlNyXvpIIXhqXYxuMxxvyP1pWys+lyejdNKtx1od4Zgp0qkyHZKZTh80d6PenpX1DE7NglkuSYTykxpGQuREp3c4fRNiy4y665RI9PwCIQz6SQAhvYcJF8FI4ZKoe88+RJ+nc6SGrJKI6QwPLSFOZR6eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DvwUEy75MTO450t/awgfNqDNaLpyUzAb+VPNe3a0neQ=; b=kwglg8ZUsX2WxUzl4q7f39ztpyd1x/p4Kqwk2QfJDoMnZf8D6kSmNGO7KyTFP1IM8Fl09GuF209KMnwlN2S8ki3ObYkmrl0cQrxlc9KDaoCCn+1ThTur5s/GG500v1wYNW1OmihDE0NUl1ZYCZpQem6yo9lD36qMNq/naNvPWYFAZylkqsfl0gZkwMDI5s72TXFmgBD7KrReHG1UDkloaYZHSOCstUYhOBcyJmrQ7uuDBh4LOU9WST8Oim2sEQX8JKQK0VG/nEBDmGwmBz9UDi7A8mkmbhMVarXlpiwK/W3vUAoxuFQffwIn0/AxGq7NLfMCos6291MMK8d7XzJxig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DvwUEy75MTO450t/awgfNqDNaLpyUzAb+VPNe3a0neQ=; b=XuMLjMjjrwq9Ws30Nbzfk8Lbel2RCLbXYHq9RgpX7aX+ZvDgWoiZdk0PcqCLvuHDyNYgDAn9XW/KeFDpfRopKi5m04XXx9Yuth3EfXOx7/bsFzwn3P3sQomwQsUvi3iSGqi721103bImjeixNWPKnbpbzoBHbJm3b8F/mwopEiE= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:02 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:02 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 09/30] sched: __schedule_loop() doesn't need to check for need_resched_lazy() Date: Mon, 12 Feb 2024 21:55:33 -0800 Message-Id: <20240213055554.1802415-10-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4P221CA0025.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::30) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: a1edaddc-6855-4a5b-e3ea-08dc2c587585 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aedJBq8fOYfvwOf0jYRyQEacjym0lsCBCPhhMFeNZG0JPHxp5ru7utd/mCt7kCIFROaGVbs5dfAB1tkM+0X4LKYiqk1D4cYDzZVyj86qKrwTmA2lckUZoIIVjXJLSFILL2w/E/TlUDJ5wsob9JuH+DebxuFzghlqWnSSL9npMhkyqheNpZ8DG3mkDryW0FJ9wemeAK0UPjGcqEcNxX7Gm08S/b7k61lctH8ToZ4vYgXIeceMK6JBdcpYFlmHYmnUZ15fGw0h82jE2vYYqG7pEk7t9F4KOT0pzqp7804j5+KgudHhNz3WAnVm3AMX+9ccENgPsPsoVY6Fq2aGvhkH//mF9OEcoGgN2qnHDdaj6rhP0HUUAOqbaGLcMLUm1/a5UMu2PXYgVG/jPcP3olyEZCOq1h4sRRHIw8/hKIUcBZ0Z1q/L6zC5ZkWtlI17KWFTVeJmhVn2/rw8O2s7u1ntBnJ5SmsE4Z1Ehp3RzRQNeX5OkhnDzuovR9mzcPTXiztMUmj3Jni9ECZm+qGv62oa8e6JdUVr3XDRqaKYJ2Yexd4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(966005)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DHP7XuVuCq0lBPQ4UuesfaOizcu8K2u5E1dO8HvLX2jiGC4oZxV0MjRKsWEtwHinddM6DXFY5gg/ijGn60giILpBt2ABSJ/kxeDmebjs9pkKBtLa43NJ1i+n4Lh21vgh56ha6eKGJkUOoXKoDsCJ0sF0Zo1YqfgHQ4T5+C5z6hQZe4TQtM09Ja3lWDdYv3DYUT+nnpHAzmrMf+BiFR+WrHaDHuWLQ9xpy3OLveqEGFVl+l5V7+NB0ZfXjxxJE8GGiOusT11wkz9tL3pa4GEBHIRzlUI3Z+aRztdwnc655sJlYrQst+Kppb/QwONEQnSla0w/5d+IJHpcRzoQLi8XzD9MiDRyX+dWbwUqz5mde+ZzGciKOCAnbctHlmEqtbVYSfWliho6zU+oUajtkw9V89vLLYnjAUzwj+ojZl07Qbwtiim/sCCANZEla4BrrGu6Bbr9YZM7GzG4pZLV/kvhP2sExixBPGRAHw8aGca4Phe55ymGXxLM+zygvY+1StoPjc03VoaJBi186Likbs4z8QPzw41dQ9CSP29I7/OIuuzklLGBYkifdZJsa1XMQolxGB2VmAtUrbkEGeM81FqZvoK+8JDHOz2VcLXG/+mMfCK29I92XR1Ixw5J7xBL7RtInViUsJOq3TrNL1RYwfxL/K9/TAkxmlFICXiDCssCmqxpS2yOaTo+EEW1WPA+44UqrWsXtEwePlL/i2uwwlE+CuJQHmNy/HM0EaU249MZKz4SPsHpq6d5HnH3Kb1Iefb/iWmkkv9SyvIA802O8BYsGJ3DvS8A3YqHGrSMVGZNc36nR20ooozgOHx2GiZPgzLd+4VEveS6f6JFpOQfdrifN5ZuM1P8BA8OZWPeJiBI88r7ldhT6yYgcBSlXwHxyNncS6J3kUchCq/XCkn6GsMB8Str+dk/cmtPlmSJ39ERePcGN5cOuj1Ga7ZyM1Or+CGfhkY18rN7OYwKHQIfb52KrIABYVsZLCXuQx+/IMcwzDdu7UMm03Rvpx+DwCeLGKcndxulpfajTZnhz03nBM6kRS5bVoPG+UKBdP0TX0cwJWiuEmsQngI6CwsX5AFOiuHyo2f0yrVJx+m1oR/POoPFc2ZG0wlSuaLCLIujlelBxZtOQm4+seYIqx9R8TDXFzGu/H9RK9VtiCrWPm24gozRboKm2m1v1VQ2m6eZCwCSMsuPigiaYKh7cmVwNO1pi+hdjtHXk0MvMoAQfvtBDPyFQ1DyxZlg7XBmvI1KptouHBudr9DvzUEgFjnxdTLt6CNnTpUA0zXuBMP2Z4vEvATvo9ucppT42O+WSnqkzS8S3PMBBXSslFQn6xeClj/yUZCLOUs9f5YBWs3a1sXKCkgzmP9SwZhcs5qA9t5Esa0kTue5wxcL8xGGUvUdJh1XqxbCwlsMzveG4t353mB4rqxlbFTMsldUORnaxxhkoVhDJ73UZ5BrYkVrOPUui6XBE6LBVnELElmc/b7meZfxdtdryM78BEQ40LXXoX6UQ1VQo9upDLXXU3xKKBgZrfTAfpdnY0EMDjf2WVNElZi1m5C1j2/oqQA02YQF51eYlH/Zl9JSBNeb7V++2KOs0/ATWA4ZQfGY0yZVKRh/RkQsoQnWyw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: W9ShxVOdxY3UIQ9q0vjaNLlfrGjlqRb6FSnzOyfZDsIPNnqgeF7WAbRw71iMrbzrd8nyXQkn33oufqO+YB8FwJYLDXrSOPNiY6EuXZ+R8mO4RkM8d3VLA8hxxUnjohfHvR++e3z6DtmZHd2GZ2Dusrrlkf+ze1VIrvkVXNRVaNIxpqhJHR/RW0vAtRv1eHt/giSBppH0Ey5vKxKhAGmnfrFajq4WcsOKbZhOZAc3dPlfwjqcVbcsUVQyunvUo1dEp0MOwJuI0C8ot4fHun954zeZO9dJsTqjsWglsE5AKrKN48vZ2sfxx3iq6oR1D2LOZsEFHQ7EwmVkbB2zAzhRy3GzNDgAdTCjcwwOjroXOiQLohRwkBHVJLhsqQw7j5sXXP84NH1zQNmNxIuaqIs5QKilhvWMGDnCRqOjxGzYg5tyY0kHGg6uGSqO33lhILbBckRG3lGB7dKsseodVcogPvkPOLNmCWI+ZaCp6g8FRdMP15We8421MXqrGWzXDINSDma0kIhFqQesD/R89gqfKFKb6phhpbnZbv/+Ya6KSneHzpbEJlPba7IERgfYh38q45sGDAvixshTTim1J/r6Jl1eCgTnqudhgMCW5fQwJ1A= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a1edaddc-6855-4a5b-e3ea-08dc2c587585 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:02.1458 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yYnHackW7j9ejiNcZVoaicbLHDvJ2owoCSj2JRePtoBNpNkMl+gAvfqgzD+RuDVl8KeEg1lLX9UGJtIrWgHZyjlFHY7xrxZGHq+sx2V/XZo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=540 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: W1PZE2iODLlVkADUtjFxfwW_DvZYUoqN X-Proofpoint-ORIG-GUID: W1PZE2iODLlVkADUtjFxfwW_DvZYUoqN X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762506532606710 X-GMAIL-MSGID: 1790762506532606710 Various scheduling loops recheck need_resched() to avoid a missed scheduling opportunity. Explicitly note that we don't need to check for need_resched_lazy() since that only needs to be handled at exit-to-user. Also update the comment above __schedule() to describe TIF_NEED_RESCHED_LAZY semantics. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 41c3bd49a700..8e492d20021c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6573,18 +6573,21 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) * * 1. Explicit blocking: mutex, semaphore, waitqueue, etc. * - * 2. TIF_NEED_RESCHED flag is checked on interrupt and userspace return - * paths. For example, see arch/x86/entry_64.S. + * 2. TIF_NEED_RESCHED flag is checked on interrupt and TIF_NEED_RESCHED[_LAZY] + * flags on userspace return paths. For example, see kernel/entry/common.c * - * To drive preemption between tasks, the scheduler sets the flag in timer - * interrupt handler scheduler_tick(). + * To drive preemption between tasks, the scheduler sets one of the need- + * resched flags in the timer interrupt handler scheduler_tick(): + * - !CONFIG_PREEMPT_AUTO: TIF_NEED_RESCHED. + * - CONFIG_PREEMPT_AUTO: TIF_NEED_RESCHED or TIF_NEED_RESCHED_LAZY + * depending on the preemption model. * * 3. Wakeups don't really cause entry into schedule(). They add a * task to the run-queue and that's it. * * Now, if the new task added to the run-queue preempts the current - * task, then the wakeup sets TIF_NEED_RESCHED and schedule() gets - * called on the nearest possible occasion: + * task, then the wakeup sets TIF_NEED_RESCHED[_LAZY] and schedule() + * gets called on the nearest possible occasion: * * - If the kernel is preemptible (CONFIG_PREEMPTION=y): * @@ -6802,6 +6805,11 @@ static __always_inline void __schedule_loop(unsigned int sched_mode) preempt_disable(); __schedule(sched_mode); sched_preempt_enable_no_resched(); + + /* + * We don't check for need_resched_lazy() here, since it is + * always handled at exit-to-user. + */ } while (need_resched()); } @@ -6907,7 +6915,7 @@ static void __sched notrace preempt_schedule_common(void) preempt_enable_no_resched_notrace(); /* - * Check again in case we missed a preemption opportunity + * Check again in case we missed an eager preemption opportunity * between schedule and now. */ } while (need_resched()); From patchwork Tue Feb 13 05:55:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200212 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347743dyb; Mon, 12 Feb 2024 22:00:17 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXP8xLNpvsSJqZoHP3I9OKCU5fA6sKrsBHXO3DdjAr1rylMykIXBfLtz3aZTFo4p0VlB981zqh4yEVpm5LgrGDExS2xHg== X-Google-Smtp-Source: AGHT+IGetRwqt2V4Gas3ZBSIYCkCoBGAhaHFZUkR/Tbo/+jLhMARVsmjQufp14eocB3YghHEJwSJ X-Received: by 2002:a05:622a:188a:b0:42c:7deb:b6e2 with SMTP id v10-20020a05622a188a00b0042c7debb6e2mr4332690qtc.9.1707804017391; Mon, 12 Feb 2024 22:00:17 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXXh6FEftN9t6Ynho9waJl1+NqNZTW8k0GjcHlf48xne3kavDZ1d7oMPVDW9j5WUbwrHMr5NNh/MP8EoqL6YFdiy87s7w== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id p14-20020a0cface000000b0068ed1dbe6b5si1219807qvo.94.2024.02.12.22.00.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:00:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62964-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b="l/pyXAhm"; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=aDW4HWn1; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62964-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62964-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 213DE1C23416 for ; Tue, 13 Feb 2024 06:00:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B401222625; Tue, 13 Feb 2024 05:57:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="l/pyXAhm"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="aDW4HWn1" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DAE017995 for ; Tue, 13 Feb 2024 05:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803824; cv=fail; b=AkerbR8K7XwvkCSe/I79hw1Kde/EQC5+WjrhtiMcVoW5OpUpctjOTSQfUr01ZrDDP4z76sj1Vue2+ASglvov4095jFjx2lcIJKvajULMvAmdLy/5FLZY/X5FJs7rz+v1vBCSkgxEPFOGjcPPBV0T5i8aOsZMNrjH9seBOhLk/5o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803824; c=relaxed/simple; bh=uyJvof1Y84DCTs62xNaN2/yfkDP/JhPAVxfDuZlMrSM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=ZNqFD3Jna3PpHUm+ifJMnU1ZFOV5oK1Bcwz37EqmLI/p77Wr2QNHtSvWECik6zqMLsDOdFFttcWgtsbBMOCuN6ROIwK2PXZye306C/IwrEZ2W33prHwhTdYKAekmyVwGVE+Zo4sdBJCG5PeeWqfqbZnm2ZKe6UVtZIeDQyYZFCs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=l/pyXAhm; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=aDW4HWn1; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5nJTS013833; Tue, 13 Feb 2024 05:56:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=tWjhyKlLEwGO7Nsfh/khG/DAMH36BSalzUK3LRdpXrc=; b=l/pyXAhmdlpeLjZ4bQO1NnAmmto20mPaDytTyfobcsqL2XWa8mnBmmXb4n66Qmnyai8q D9myqvSvFclfyBecsj3pZzH/Fln/dQbEb5Se2qtVXD9bwlzH3N6QVla8xEY+DYODxVjM OYf8Ge9Bh/wtNbOCAN1+zfFPNhZtwAqCISBD9jwOMrELQ4iVgaguLbQ0ACwrfTpe0Zjj Ed09FWhdHJPFU0zrKNTCuzIM7NU1Ce+3AbTLMsH9lgBFmKVLVBZafka0esDiT2eMluZK Ut77/6p58PZr+VxvF838hPjzIUNbxKo1SxHG3DDdGEydC3mEL6i+kAtnTT4eTln7JTnl SA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w816tg49g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:06 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3XGS7014943; Tue, 13 Feb 2024 05:56:06 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2040.outbound.protection.outlook.com [104.47.74.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8ak-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Exx6NSDUFTxbB7mnjSyhnFSqWkWPJ9LRs/uHuOvCfOk5v88bqRCWh4kFfL61zBtyWz7HD77VoYPGeY61qRXfEngh6TvWchX3ay04zioKPkg3pT4V4g2q1PjrBXYJ3a79sKPVgZ0CRY3M8dyDV1SUlgHtbXrSxYrV+yaiiTnyRSHdHXRdYMBmvPXnWfBig3bmxdc8xfFR0GzA2fq6jwmAQ1WW6FqiBhnle/2ME2S43+3cF1WxpGYC4s0wlfvIW/ucXtu0MD+8jfwN37RyPDh3oZjE584Au79Dmub5PXYGjDO44x0NHCuuX7yt+Pe4IktmUBX9aI/OMU9gNjqMsrWiVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tWjhyKlLEwGO7Nsfh/khG/DAMH36BSalzUK3LRdpXrc=; b=OTL/CxM8KUBctyAdOiF95OtG+zYbafVRoC9O6YL4WUGJTQptk/OBblxr8o/k/qNevKBSWY86ZhM2ZpM9VodLeQpmAcruvbTbkVWCg52Cj89tZImvRUxrRZm9DauyE8MgVb37XD44TqKVBDG4REHSap60htBY5qbel/sZjLGfkIQiZeQ+nvgcED1AtIL2wxIj1B6pINd0WWWkCgv49LdGdGoJty31ZXeXCiX2mE5vKOONWWgpnlyuWl7GXsha+Ik7AJ2L8V44Pp8+YYihC8FIhejwlFwVkEmV7KE2ss2hwaa2cXi27KqZODR7I6lCvb4Di0nspi+bAUU5zwU04ck8AA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tWjhyKlLEwGO7Nsfh/khG/DAMH36BSalzUK3LRdpXrc=; b=aDW4HWn1BB5dxq+ojHuO39Gt9rfwuCoWEY8aGzKMLlECkhDE1riipMubw2Y+nPkY1Nmni+ORjtZbVUf1tAUMqY3EvX9Wu15Dvg9Q0/rbOyQ3TcdBf5T6gyzEvZORHW6kHLC4YAi/pIzSHBObDlOCyIB8rpX5aPuoC5rEegaPrDQ= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CO1PR10MB4642.namprd10.prod.outlook.com (2603:10b6:303:6f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:04 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:04 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 10/30] sched: separate PREEMPT_DYNAMIC config logic Date: Mon, 12 Feb 2024 21:55:34 -0800 Message-Id: <20240213055554.1802415-11-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4P221CA0007.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::12) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CO1PR10MB4642:EE_ X-MS-Office365-Filtering-Correlation-Id: 11b4eefd-9be8-4fa4-a75b-08dc2c58769f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3SdKWUc+6Q0NH7Ehhj8rN08hfbh6GC0aWTpXSCGYg4SevNzmmOCQi1sl0wmerUMLz6vAqsSh4EYtLFtBIiHGKSmSS3A6dvRIt2mA8N7Xjs4W0/3JK480UEMwIYiuuFXgaqITtAlT0Rx2enWzzw+AGhnylQG2Zsaq9gz1qK/P30/W18w+w+4Fe3i/sUHAs9QMu8g5IyKXB0DQtCPpCxuVFJ31Xk404iabEV96xrdfbm2aRh57C6gh5nWqKvjlwuatoh9Q8+eu/Eddgo+4yRR4qYPEiyD3DWDkx2m5PMDDktb03kAbhX1YNveJqbWny7ALZpqq32oruiSExBo4+ueuD1jPMku7mJES+oP9q6VfTh82WwyUIapfHjAwGgfsnSPW6yyM9udMkeag6Wg2V0ey01brDUovV2ZBi4/5jjlhI5ffLOnhlnco8XEWI5XRfTqdOD5qV6Az8Cy8IuCcf4p5bB4b/tDQHoRbzpScn0ZIMFCt31CfRVvIFAptEQBiS3vzGf2Rftl17t37nc58cKAVoyzpAlOA4814TNiAfcQ2p1pS4i0/h9PAS2E8Gkr/BMIE X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(346002)(39860400002)(396003)(366004)(136003)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(6916009)(2616005)(41300700001)(83380400001)(26005)(1076003)(107886003)(4326008)(8936002)(8676002)(66946007)(66476007)(66556008)(6506007)(6512007)(6486002)(316002)(6666004)(103116003)(36756003)(38100700002)(478600001)(86362001)(2906002)(7416002)(5660300002)(7406005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wXQyaeMmlWuM1M5v+rJPy1VQJgjbVBnNiLIMJmfHCavmAtira3nWOLkckifdOQMYHWoJflLCBWTsbir+bNIdrCndDCOUKVjat70c5bHjsIYgA3z86/K/8IKephZT1+zxrkRSJKqh2AFWpKHacY7vzj5U0XyAsQX+pYCEwrJgWMp+gxyhpjCrOlchQLm9g7J1us8KKqnjSx0CIlMNRQfsDd8ex6JkvFOO9zDwMWQ96PHi9zw6bEvMTEK0GQ/4mvQY5kBOwm3GWVTlY3pBIMCK2aS7klgkCVdf83rrAROBXPMpjTgJU/WksN+VVqzvV+uIxmu7Nq7AhLTWXSdcBVfUhLHGJlNsX+nj5uF/XkRzwGD44/585mDhmGMFf44Q1DbnvN1SgQd10L4pMvvAewBecjnMaEJZTYlrZ1jA0lCk2CRvOb/d1S8dWqnIaRV/pN1ePTC+JXnfnqR1u+NMPJSISDcyaRQmHKgyGkWRWz6DYaTBDvC6mhk8++Fw5l5q8AkSS8YH9YNXmKUDjW54oHEA6tmHcKxuBUucwyF7rHO7WLr0/OU1RKW9NXlnnofMV/zmFqf/iPRYaM1YvzziDFPo3v96PFfutTqIJgpyR3N69Ahn/ng3OmXRffCH/uqlHFbC3QERwNrHpIyGptR3iXvarj6AW/Ldz4lCcGrDecRxT+abcAZJ124rx8bpIY1zvsklBPxLrWAQgolJJl9jZpsqxRRWspVft+JQAdubHgJZiJYnskux6ARpRNdLvU/mzoXsOSPePIEM6mbI9IRaJXSjmgrRNtqgLfqOpGCA1Y7EeUXMMRcPG+rN1A4UHteqMs5E2o3w/JBfrMH4NQFIJJuyl86eddxboRyJ6P73SEv32yk8WUFFjVKvpg0Wz2i3/+d6fDZKQgk+OGHsVkbFj+RWqXNZC1Q1rrhmXj9aHq7VvZR2lOFc6a5ik+RuyrXMve7tBWzHs11ACphT5Nvfrs95W0g1LGCN+gRSC6gVwWWnb6BYXsXcLmm9Uib/lZAb50QJZGvEJZboXQVz+PY7DDvaxxXt0RaIiXO+OInaMVrcvoNNq99Xnwel/qEr8cGEQE8zZ0jSvK+zvApZHrw6wACSjvOiMg3YnwnwPjgu92zLSAyn/B6gSmrRH/9O/XgzKqbGZJI3BUSF1a51n4u6YfX/yGQxJ+JJPGDW63LRrau0gdGwKWWzOPJ7XI9JA5sPFrnEaFG3u83WbKipCmbCImCwxRPEfXjZpnk8rrQYvps98ZCKZnl1fsWfw3t48k6d0DqEtXrU4Oo5Kt5fCoTaXbzOyHUmZwQKiOkl8uar/RejrtzSxzBBEDQgKJR4Hdei2RHLMLi7/oEph5ZElSKAUYssb5CHxTvhqTHL7kIDJRHVTV0FcZzw+0l2F7SCOX3yH2V9bhRtKeRMhme8D+eh4r98FMWMlQoU/7yNIb4crInSRGHCY6INQg3NxnxAsZjnnwDuSxPluV/wj/ZKgm+AqKQAoOlz8HeAMLtDM6vS/02eJRZ+aw9TEYATnMDh7ymblPDoE4jdPzb6vINeHo33AHm4Y/suyIw/k7/YzVb4BoyBFInyyAwZG42zovqlohd5A2iG3G7zAV0GbhAXgEeT8lFazA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Ptrz4irqTFYNzEu9jNP1aP+WO3dF9iVlU+nHiNmr5YXi+FozzKx1azOxoyhgkburuwQAhKWGfyzDmH+/vKyjLZs+iG7YH5FSXhTCKA4AV2cuxqlErNycAkoPcyhlgSjTXozwUuzTxhQolxo4BsPokszB+I4UUYl/BLIfhmXe30m9SA7g6LcV5SpMz5zK2kCSQrEEw7H6rhs6t6LNflct+QEpU+EW4cpMFLziH+0IdJ8Hm+vUdyTV9V4eJNZdE7iZMTh+cfAQYakqxxhlpb6hXSPnNH+mXsCOiUbdH3dSyqNqB68G7i4uY0scDNEM7mjSpIwAb/mODEB4Fliy10P4hUUFvYReG//kNNCk9yCICF0qdAvh11dRzRXEX9KgMD1lFF78vDnpCG/mrtsvPfWv6oSuhOanuVyvViRGuklC3QDiXYs6AUYr2YTt918vYXms/Q88mbaT1xy82/UHnt3AdxbvRjNE2XSX+tSPabNee/EA2NINk2PwTkN7vJ8iiac4Bi215uj/9P5HuNBF0H6O4yBiPdjhqTa3x6J4VcyscC78T+8Q92j17CSuRvwmlO1wih1G4cP4JwWa64xvN1mXkoV7eViVwkd02Wiu1B6leLo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11b4eefd-9be8-4fa4-a75b-08dc2c58769f X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:04.0526 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: j3i8slpFIc537vDm2XVC1SVlIVe+cKVACkgEg15fZFHseYsGCZBmsZTJTTZj2t7XKRlZUvuIsJzVVoyAScXqbRMv0iuB43SNVDJP+oVZTZo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4642 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: FzI2GUJiZPzgGZvsBg7zonksnnwLlYsW X-Proofpoint-ORIG-GUID: FzI2GUJiZPzgGZvsBg7zonksnnwLlYsW X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762305214671959 X-GMAIL-MSGID: 1790762305214671959 Pull out the PREEMPT_DYNAMIC setup logic to allow other preemption models to dynamically configure preemption. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Signed-off-by: Ankur Arora --- kernel/sched/core.c | 165 +++++++++++++++++++++++--------------------- 1 file changed, 86 insertions(+), 79 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 8e492d20021c..6c5de6c1da27 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8692,6 +8692,89 @@ int __cond_resched_rwlock_write(rwlock_t *lock) } EXPORT_SYMBOL(__cond_resched_rwlock_write); +#if defined(CONFIG_PREEMPT_DYNAMIC) + +#define PREEMPT_MODE "Dynamic Preempt" + +enum { + preempt_dynamic_undefined = -1, + preempt_dynamic_none, + preempt_dynamic_voluntary, + preempt_dynamic_full, +}; + +int preempt_dynamic_mode = preempt_dynamic_undefined; +static DEFINE_MUTEX(sched_dynamic_mutex); + +int sched_dynamic_mode(const char *str) +{ + if (!strcmp(str, "none")) + return preempt_dynamic_none; + + if (!strcmp(str, "voluntary")) + return preempt_dynamic_voluntary; + + if (!strcmp(str, "full")) + return preempt_dynamic_full; + + return -EINVAL; +} + +static void __sched_dynamic_update(int mode); +void sched_dynamic_update(int mode) +{ + mutex_lock(&sched_dynamic_mutex); + __sched_dynamic_update(mode); + mutex_unlock(&sched_dynamic_mutex); +} + +static void __init preempt_dynamic_init(void) +{ + if (preempt_dynamic_mode == preempt_dynamic_undefined) { + if (IS_ENABLED(CONFIG_PREEMPT_NONE)) { + sched_dynamic_update(preempt_dynamic_none); + } else if (IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)) { + sched_dynamic_update(preempt_dynamic_voluntary); + } else { + /* Default static call setting, nothing to do */ + WARN_ON_ONCE(!IS_ENABLED(CONFIG_PREEMPT)); + preempt_dynamic_mode = preempt_dynamic_full; + pr_info("%s: full\n", PREEMPT_MODE); + } + } +} + +static int __init setup_preempt_mode(char *str) +{ + int mode = sched_dynamic_mode(str); + if (mode < 0) { + pr_warn("%s: unsupported mode: %s\n", PREEMPT_MODE, str); + return 0; + } + + sched_dynamic_update(mode); + return 1; +} +__setup("preempt=", setup_preempt_mode); + +#define PREEMPT_MODEL_ACCESSOR(mode) \ + bool preempt_model_##mode(void) \ + { \ + WARN_ON_ONCE(preempt_dynamic_mode == preempt_dynamic_undefined); \ + return preempt_dynamic_mode == preempt_dynamic_##mode; \ + } \ + EXPORT_SYMBOL_GPL(preempt_model_##mode) + +PREEMPT_MODEL_ACCESSOR(none); +PREEMPT_MODEL_ACCESSOR(voluntary); +PREEMPT_MODEL_ACCESSOR(full); + +#else /* !CONFIG_PREEMPT_DYNAMIC */ + +static inline void preempt_dynamic_init(void) { } + +#endif /* !CONFIG_PREEMPT_DYNAMIC */ + #ifdef CONFIG_PREEMPT_DYNAMIC #ifdef CONFIG_GENERIC_ENTRY @@ -8728,29 +8811,6 @@ EXPORT_SYMBOL(__cond_resched_rwlock_write); * irqentry_exit_cond_resched <- irqentry_exit_cond_resched */ -enum { - preempt_dynamic_undefined = -1, - preempt_dynamic_none, - preempt_dynamic_voluntary, - preempt_dynamic_full, -}; - -int preempt_dynamic_mode = preempt_dynamic_undefined; - -int sched_dynamic_mode(const char *str) -{ - if (!strcmp(str, "none")) - return preempt_dynamic_none; - - if (!strcmp(str, "voluntary")) - return preempt_dynamic_voluntary; - - if (!strcmp(str, "full")) - return preempt_dynamic_full; - - return -EINVAL; -} - #if defined(CONFIG_HAVE_PREEMPT_DYNAMIC_CALL) #define preempt_dynamic_enable(f) static_call_update(f, f##_dynamic_enabled) #define preempt_dynamic_disable(f) static_call_update(f, f##_dynamic_disabled) @@ -8761,7 +8821,6 @@ int sched_dynamic_mode(const char *str) #error "Unsupported PREEMPT_DYNAMIC mechanism" #endif -static DEFINE_MUTEX(sched_dynamic_mutex); static bool klp_override; static void __sched_dynamic_update(int mode) @@ -8786,7 +8845,7 @@ static void __sched_dynamic_update(int mode) preempt_dynamic_disable(preempt_schedule_notrace); preempt_dynamic_disable(irqentry_exit_cond_resched); if (mode != preempt_dynamic_mode) - pr_info("Dynamic Preempt: none\n"); + pr_info("%s: none\n", PREEMPT_MODE); break; case preempt_dynamic_voluntary: @@ -8797,7 +8856,7 @@ static void __sched_dynamic_update(int mode) preempt_dynamic_disable(preempt_schedule_notrace); preempt_dynamic_disable(irqentry_exit_cond_resched); if (mode != preempt_dynamic_mode) - pr_info("Dynamic Preempt: voluntary\n"); + pr_info("%s: voluntary\n", PREEMPT_MODE); break; case preempt_dynamic_full: @@ -8808,20 +8867,13 @@ static void __sched_dynamic_update(int mode) preempt_dynamic_enable(preempt_schedule_notrace); preempt_dynamic_enable(irqentry_exit_cond_resched); if (mode != preempt_dynamic_mode) - pr_info("Dynamic Preempt: full\n"); + pr_info("%s: full\n", PREEMPT_MODE); break; } preempt_dynamic_mode = mode; } -void sched_dynamic_update(int mode) -{ - mutex_lock(&sched_dynamic_mutex); - __sched_dynamic_update(mode); - mutex_unlock(&sched_dynamic_mutex); -} - #ifdef CONFIG_HAVE_PREEMPT_DYNAMIC_CALL static int klp_cond_resched(void) @@ -8852,51 +8904,6 @@ void sched_dynamic_klp_disable(void) #endif /* CONFIG_HAVE_PREEMPT_DYNAMIC_CALL */ -static int __init setup_preempt_mode(char *str) -{ - int mode = sched_dynamic_mode(str); - if (mode < 0) { - pr_warn("Dynamic Preempt: unsupported mode: %s\n", str); - return 0; - } - - sched_dynamic_update(mode); - return 1; -} -__setup("preempt=", setup_preempt_mode); - -static void __init preempt_dynamic_init(void) -{ - if (preempt_dynamic_mode == preempt_dynamic_undefined) { - if (IS_ENABLED(CONFIG_PREEMPT_NONE)) { - sched_dynamic_update(preempt_dynamic_none); - } else if (IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)) { - sched_dynamic_update(preempt_dynamic_voluntary); - } else { - /* Default static call setting, nothing to do */ - WARN_ON_ONCE(!IS_ENABLED(CONFIG_PREEMPT)); - preempt_dynamic_mode = preempt_dynamic_full; - pr_info("Dynamic Preempt: full\n"); - } - } -} - -#define PREEMPT_MODEL_ACCESSOR(mode) \ - bool preempt_model_##mode(void) \ - { \ - WARN_ON_ONCE(preempt_dynamic_mode == preempt_dynamic_undefined); \ - return preempt_dynamic_mode == preempt_dynamic_##mode; \ - } \ - EXPORT_SYMBOL_GPL(preempt_model_##mode) - -PREEMPT_MODEL_ACCESSOR(none); -PREEMPT_MODEL_ACCESSOR(voluntary); -PREEMPT_MODEL_ACCESSOR(full); - -#else /* !CONFIG_PREEMPT_DYNAMIC */ - -static inline void preempt_dynamic_init(void) { } - #endif /* #ifdef CONFIG_PREEMPT_DYNAMIC */ /** From patchwork Tue Feb 13 05:55:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200207 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347131dyb; Mon, 12 Feb 2024 21:58:11 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUWZQM5A847EFO2V5/aE/EDTRV3Z/XN/S68cYM2qwHaXDDAC1uzrB7Onz9BYZWMuhgla2ilBTNsWo4Ue1oWT0j24+nlvA== X-Google-Smtp-Source: AGHT+IEpN06sG2sEX77y67YrPgmfXtOSBqjzej/Oa9uBIWYHz5oLe7Fy+L3hrg5iOcMZV83QLmTd X-Received: by 2002:a17:906:5ad8:b0:a3c:9bc7:7a0e with SMTP id x24-20020a1709065ad800b00a3c9bc77a0emr3319526ejs.9.1707803891489; Mon, 12 Feb 2024 21:58:11 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVFOjDiMVZ1Xhz2eqMVpYuxU29wTN0ZAE5/0rzvmjf5bBDzcHIb+h19URsOE6eoSNfF9zOUohIcDdluf6Toeiqvhb7kUA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id r23-20020a170906c29700b00a3cea6183a1si612120ejz.621.2024.02.12.21.58.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 21:58:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62960-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=Gzec7MUb; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=udJQEnS2; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62960-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62960-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id DFF961F24A85 for ; Tue, 13 Feb 2024 05:58:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8698817994; Tue, 13 Feb 2024 05:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Gzec7MUb"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="udJQEnS2" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C515C13AE8 for ; Tue, 13 Feb 2024 05:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803817; cv=fail; b=oopYkkjggJTIeLcoBeEwRxJJ8kZVgXgXVN2w6lQ/m5sesi5M8ZZZg4JQhvGntOUkKwcgwkYQWclunV+Gecsi9BrwKxmf0jzPjcxZ0R0kzV+5srBdbTMDrAngdWgAF6mZGGSzqoeNyusUAsa4AXrtTVU9vtr+SHEmj7byUVGSMeI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803817; c=relaxed/simple; bh=EjX72QId1zVFFp7qM0r2GQLoHgmWlGmnGOOnzaAPdmI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=aDwcWq4w/aOcqFTmed/OdDnee8I0a6JEzyn4O0hA/500hlIGUwoD3cVFKiW8CtTATSk4MaSjuBV8AmzJ8OJEMBL+hdhsZV+tYBvxbciwOmICAGre+orWms2FKzwwRjH/LWoxFPZgiIqg5W8ITxYtczytdpLVvMfFWIveKUdcq9M= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Gzec7MUb; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=udJQEnS2; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D50uK0002098; Tue, 13 Feb 2024 05:56:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=A2GLpj6phi1v9myea55hgaJ0LAaKEWQELk7JINqSxmw=; b=Gzec7MUbs6bVmqj00w4n2+eWxR3sYUk81GHwa7ohTnSTXs9GL6EK9rih4awPUoGykQ0C TRxKiGiRr7VZcnxjMVkttRTw8h4JQGXN8JKOEXYaYCNyC4VK2tD2mMTreOIrGaWcNHFw gpISMd49MFO2evjHdyRtLDe97IbuiO97DI8dNaW4imdH0E+lxebq5wUrU56IF7H/fv5P FWJ3Dh+DWQn73UrKiKOEjUVJ5LGYmt9/oYEpgEh85FCigmF2CCEklV3JWMVXtKt62YiK QAf7rVCwYqfZ5akB9pUXHJNWOQ9K4AVkhJRfug9ysUkAkQLOCXiK0miub4u5STbpbbjL 7g== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm836m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:09 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3Ug28031505; Tue, 13 Feb 2024 05:56:09 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6tw4m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QDuMHwID/Zoe5WCrUmTc7DwGstaJKyDIlPfxtQZ1Miuj1QKYYmyD0mndGFa5MDCtCAWjNQwVF7KRgJBdjRpe8rjTKwdejHtPxL/14OaIkK7gYJDTlquRiF1iVvsvvLXRRop9bZvO3/dsCXCkW0ONG/j2kYQn29OXhUEjfwbXXdhcsYL0kUgvcwXcZjxRTcS52tPF0YB10Wh+7wxt6W0hwmhytGshBZF+1SpNNTGYczwEfd5bXEBgll0Ujz9mDoI+BIgf3FNWxRKtlIFiw2/rTagvjuJidiOW0Tnq7/bUjLKcvRdHfdyeeWVTdFEmwaSaeuTQOons0LA92AKuAeiPIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=A2GLpj6phi1v9myea55hgaJ0LAaKEWQELk7JINqSxmw=; b=X1pP3TYpQ4KEufm6pIk9Y0SdXXx4J/JTbVnVB6kLRBSXK/vipKNCPeohpCytgLIr+odC9uAscChG5MpU6iMvhTa0qFeGvSzEhX5LZDfxb0WTwk2IPvbrzDnyLaZ0nU0gBBv5NRPFCEBHb+05WLC1n/uXSDdRlPFdmPWvm103zvxeM8+TK6IL7J75AJmDkBbqY/JG2TKfQ4ygUeuvcc4cQ3Xu41KX1mUdO1z4SciJZyj+/A4HzdfTUNWROp0fOebcYBpKIu8gWUUZIfDUhAK8xvDwxhWbFXGJlJo5beMBbxlGTBSCRQETD1DVWBHgkx37oLuGF/zPzJK5ixd0/FSfwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A2GLpj6phi1v9myea55hgaJ0LAaKEWQELk7JINqSxmw=; b=udJQEnS24k4OeBY7E2VcQLrZNXD7nqSse2elE3jX2MJzeG++NYac2V/waDSYWdkwQ+qQpHS0c46Cv11gPg3ahyQMiq35mBNFDPbM/VSZ/7XOCIDh0VZ9i2mi1KXsFKpG2WMMh7vgULRmA5YCMJff0EP0DZ943M3C99qHLy1ji4Y= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:06 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:06 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 11/30] sched: runtime preemption config under PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:35 -0800 Message-Id: <20240213055554.1802415-12-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW2PR2101CA0024.namprd21.prod.outlook.com (2603:10b6:302:1::37) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: f63e2cb2-2915-4831-65ab-08dc2c5877f8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KrBtfX2dd0EVm33NpEcKDkyrdvuvUXrW/KAzWbJ1OL/JexrX+y5qqBrFjvyVTbEXGfd9Jt6q8eo7jfenm5kKF+gKcp2T2DtbXtWpWiJvdoB2SjDuF2/HioAVO/UWEHFyjWYndY64AtGoeOoB0PPqIb+DIDMUReS/wlRO+iYZWbYVn6bp4MC9PP6A/1bSJY1360HVKtNT4eS7QoqDduZqrpqkBO2I5gtqO3TKlUiyk0ANh6GPfjljiPv3vPqheBiYznJc8cD9Ncfy5JYSuZfbFMFKVrxM1sDX8xDVW/KSUYOpKDi4Rbk4m5j/ykIhuFZJ+cgdNg3ob+EkPebF9fs3uY6gzBsihJehrGK8YJB08Iztq/qRyj7uitzwj0TEzkIN3mzM1Io8IyDmGMIIrPkrMrg0iH3deitN83NMfqyKc/Z6qLkmYqTZn1tXSw6vBg+4jYNtrZADC/EnpeMDxB1Ws0ZmPbQFOwOVVOpChPYWYS1bOuQ/mjekkETOYCXOP2LB79STqrqRo5vDzGxhzC75cqoulZ7U8chrEuRYkulEPFaZPk2SGcq+hSCgMJWrPP0x X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LRcTc3Sdn1rwDwDvt4fHVFcvRO6bFocPWEBirqeZfVr1T02zNPqucb5+heguFgb5hKJ79iMeF+Q9C7EJdZ25jfLxrcWBx3mzwuyxCrK1VUcW32E+hSH+vVqfHl9HQ4QwjTs9ZJEtETS37PqY3lJyWqftuCs55pP1ZfLDzj44ldXmA/8G7FdcMvjPcKoPFaU7+nhTNVbuXiwY5NfhRQULrSn1WUVArLM4O2r3lqoH7wgsv/FnK1p5BYR3JQ0J/I+yc2ukNa4IfAaDvUr0u3oPeUT2O7rtuJFdOJfLDwijgvDyBspIQXtMWrpdcuU8g7LMRMzaZ+lnt860emmYM8mYjp3kWoL2Azp+cm9wuL6bJbeEm73Tj+96PWiuB52JgO51wyZdQmf/Hovi7lCzD5+H1W2OFJXMlxNObMIvDUlw+Mn+JqxKdt1FYBdDYKwK+N74Tr62yqT2gCVzTjwvMrSSWMcRFQ+IFJyyYhx25cg5ZJ4Ztw8H102ZPYEdgOsMoyTVdT48RCJ7fUDyMfuorIjiaV2e30vFqZKyo5z1jGJa6hywEcTwgNqH4U5zgGo3LDlRkZG8x8NQmDiHk0SM5SEq/gtL/NcMf4atLHIo6ofXdmutPQF0Q4i7EzEux56egP1JNRsZ6Is7VrU+hrN5LikXziNCmtdWHaitsQZv8Ggmxo02Tcmp1j8q4SluPY1dxEY/YGxEAQJuS0vyaFe0fnYDpTB/CeNghz5R2I6/fmIkxBmmaRVjiX1TzfnHxI9GY6Ba2y3k+ProijSv1mq2PelmWOWwzOlKM2D0OZIjmj+f/yKnDfSWICTQsKBQjC91S9606Y32OBJTgbAtRn2+Rl0OPlyEbJTpsRlJw/VTByzQ1zalcFzNkJIne8Yp54pyURxVvHlr3viRFjDw4g0EHgES11r8N2YLNpfFkGxAgy9jOYwky3eas00iS8u6GWfci3m/gHd61/Hj5Agr9lVpE+5fEkTMkqYt55jcx5GfGfsDwJj0XSQq4C9CQAFuFqaS036dBf75pzGHKur31rW/nSIdk7rTYYzZo1IC+J+MqlBFxUMQDiibQo0GbQrh94P8IoVcupPIfkjDJ2gHCp2++OPZE+8BVp2Kl3CcWjocI3W6wR5cRDcriQPxkJHR8midzIu6b7p2VqCSFoWrHgbO1WHIFZmfSPwZAin0bqZBvf7EzYsBiASw4qhtfzuffXT66TddMiOXMD9sbLRp/+88S3dCF4j/RZaqJZfsPK0ewSdZLe2Blti3u51tHYBA+CnsP+s+2OyJCJkLkGbUGa4KqsURrBhgXYObtgIb0ocy+43KfXJU6+Z6npakSzdAPIzAN0lxwCx0YCoefaMpyVHp6xjAPCRCXiQ6iqmyR2YpD+PoYrDrYwHgDZsJUqJ5K+QB9x9j3+IGQf2x97fKeUNXMmf8pKN4z0JKIPrwJ/oUamxFcxIi7t8NCTpuaYR3QKUgOLC3E0RkwgJMSSZfudTR0OkfhSwguVC1Y3rS52DM4jlHPldUMWq0z3dqSosOZoaeAgJBv3z9zDXgsHGmjQhxSrbezfW1hY9wWYP7odsbCCfQGYyYb87AeiG6fJ8nl+jFG/ZdEBPNSqp5QhrNYYpZhPz5OQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: HKPax4B0fWvznOZZyCC4MzUMBqlfZZ7lALHv3ZAfeYVr5k+CeNiHuGat+DnqY+ZXHx6VJD66rUhtSwhrpV+XGkQEiBxA5+ZmF2LWQDaHoKS01t6elpIy1i7JcRJtpjwa9VoC32dJeoqY/l0v1umzxt0EUxaQxXQh5dcJQvesZHjxgrYPCZgLl8jA1wpgh+vCS/bBMyPD48cUvT9hZbdlo8DOCHsM00XLYn9Mbz/mhFGicgJ8YOgNEJo4JPIckcAxqnuh4Iz3PDi5hHu36f06jpA1OoNoMfY5iQtc9svk8jX8rha85VhK1rmzWFX13b6YWZ+RRKoXXU6tbIdxj5y88SF5pZOliSTy0HRbUkDqUuKr5LY59vI1gztIlmNPBL8OE/9ZGzHzy3cCSvs9fwmCoZPGcMWNabF3qepITMKTENYc6zJacn5/+EL/NSV7HOO6k5VcUJgeaFHLv7MlyZ415ZzGId01lSiRTkNRqBZTKvHa+ZfIsJ8YLBPqTAIjjxzBbFBv85Psv8SSDCDoS3agwBxW/0M4U2TbElRS8vWFLxXA2/OvrcdnORV+0EmdmuVKLVJQ76u7gg4m4ukOjU+rxYeS11hT+ejGJ8VPOmELXwI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f63e2cb2-2915-4831-65ab-08dc2c5877f8 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:06.2510 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: w0JNEXwhcrD8zFsY5T/zCQ/Rq+Do9S99PDG6RXv3sDbNWbK+f8DMK4y+L53VDwXQiXX+5pNYJZKl1CI7m7kllAx3jRrCCx8Z56Q3leTFEu8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0 spamscore=0 adultscore=0 phishscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: ebCTFaiJy3eEBBRGgSiCMsKWx5aFFUyc X-Proofpoint-ORIG-GUID: ebCTFaiJy3eEBBRGgSiCMsKWx5aFFUyc X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762173173104039 X-GMAIL-MSGID: 1790762173173104039 Reuse sched_dynamic_update() and related logic to enable choosing the preemption model at boot or runtime for PREEMPT_AUTO. The interface is identical to PREEMPT_DYNAMIC. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Signed-off-by: Ankur Arora --- include/linux/sched.h | 2 +- kernel/sched/core.c | 31 +++++++++++++++++++++++++++---- kernel/sched/debug.c | 6 +++--- kernel/sched/sched.h | 2 +- 4 files changed, 32 insertions(+), 9 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index d226c2920cff..58e6ea7572a0 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2073,7 +2073,7 @@ extern int __cond_resched_rwlock_write(rwlock_t *lock); __cond_resched_rwlock_write(lock); \ }) -#ifdef CONFIG_PREEMPT_DYNAMIC +#if defined(CONFIG_PREEMPT_DYNAMIC) || defined(CONFIG_PREEMPT_AUTO) extern bool preempt_model_none(void); extern bool preempt_model_voluntary(void); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6c5de6c1da27..e5d631197f35 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8692,9 +8692,13 @@ int __cond_resched_rwlock_write(rwlock_t *lock) } EXPORT_SYMBOL(__cond_resched_rwlock_write); -#if defined(CONFIG_PREEMPT_DYNAMIC) +#if defined(CONFIG_PREEMPT_DYNAMIC) || defined (CONFIG_PREEMPT_AUTO) +#ifdef CONFIG_PREEMPT_DYNAMIC #define PREEMPT_MODE "Dynamic Preempt" +#else +#define PREEMPT_MODE "Preempt Auto" +#endif enum { preempt_dynamic_undefined = -1, @@ -8769,11 +8773,11 @@ PREEMPT_MODEL_ACCESSOR(none); PREEMPT_MODEL_ACCESSOR(voluntary); PREEMPT_MODEL_ACCESSOR(full); -#else /* !CONFIG_PREEMPT_DYNAMIC */ +#else /* !CONFIG_PREEMPT_DYNAMIC && !CONFIG_PREEMPT_AUTO */ static inline void preempt_dynamic_init(void) { } -#endif /* !CONFIG_PREEMPT_DYNAMIC */ +#endif /* !CONFIG_PREEMPT_DYNAMIC && !CONFIG_PREEMPT_AUTO */ #ifdef CONFIG_PREEMPT_DYNAMIC @@ -8904,7 +8908,26 @@ void sched_dynamic_klp_disable(void) #endif /* CONFIG_HAVE_PREEMPT_DYNAMIC_CALL */ -#endif /* #ifdef CONFIG_PREEMPT_DYNAMIC */ +#elif defined(CONFIG_PREEMPT_AUTO) + +static void __sched_dynamic_update(int mode) +{ + switch (mode) { + case preempt_dynamic_none: + preempt_dynamic_mode = preempt_dynamic_undefined; + break; + + case preempt_dynamic_voluntary: + preempt_dynamic_mode = preempt_dynamic_undefined; + break; + + case preempt_dynamic_full: + preempt_dynamic_mode = preempt_dynamic_undefined; + break; + } +} + +#endif /* CONFIG_PREEMPT_AUTO */ /** * yield - yield the current processor to other threads. diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 8d5d98a5834d..e53f1b73bf4a 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -216,7 +216,7 @@ static const struct file_operations sched_scaling_fops = { #endif /* SMP */ -#ifdef CONFIG_PREEMPT_DYNAMIC +#if defined(CONFIG_PREEMPT_DYNAMIC) || defined(CONFIG_PREEMPT_AUTO) static ssize_t sched_dynamic_write(struct file *filp, const char __user *ubuf, size_t cnt, loff_t *ppos) @@ -276,7 +276,7 @@ static const struct file_operations sched_dynamic_fops = { .release = single_release, }; -#endif /* CONFIG_PREEMPT_DYNAMIC */ +#endif /* CONFIG_PREEMPT_DYNAMIC || CONFIG_PREEMPT_AUTO */ __read_mostly bool sched_debug_verbose; @@ -343,7 +343,7 @@ static __init int sched_init_debug(void) debugfs_create_file("features", 0644, debugfs_sched, NULL, &sched_feat_fops); debugfs_create_file_unsafe("verbose", 0644, debugfs_sched, &sched_debug_verbose, &sched_verbose_fops); -#ifdef CONFIG_PREEMPT_DYNAMIC +#if defined(CONFIG_PREEMPT_DYNAMIC) || defined(CONFIG_PREEMPT_AUTO) debugfs_create_file("preempt", 0644, debugfs_sched, NULL, &sched_dynamic_fops); #endif diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 001fe047bd5d..34899d17553e 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3229,7 +3229,7 @@ extern void __prepare_to_swait(struct swait_queue_head *q, struct swait_queue *w extern int try_to_wake_up(struct task_struct *tsk, unsigned int state, int wake_flags); -#ifdef CONFIG_PREEMPT_DYNAMIC +#if defined(CONFIG_PREEMPT_DYNAMIC) || defined(CONFIG_PREEMPT_AUTO) extern int preempt_dynamic_mode; extern int sched_dynamic_mode(const char *str); extern void sched_dynamic_update(int mode); From patchwork Tue Feb 13 05:55:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200215 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp348240dyb; Mon, 12 Feb 2024 22:01:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IHIrZ/qZIC9ovln3cSEe87PGLenEcWS9EjHCUZV0V5AhxW1AR28W50mVXUNxRSwmPJ+va5j X-Received: by 2002:a17:902:d506:b0:1db:33bb:1757 with SMTP id b6-20020a170902d50600b001db33bb1757mr1077199plg.57.1707804079659; Mon, 12 Feb 2024 22:01:19 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUc+cGXbzuHHscQGSXzIz4U+w2BNfjKqQLDvV+W49WyTZd1KNjc1w8uiDPBCfR/eCJk8fiZmIIYJxVRFQGbZ1JFHpIjuw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id x8-20020a170902a38800b001db3613aacbsi556392pla.617.2024.02.12.22.01.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:01:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62971-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=dIIatEMg; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=pm82z+2Y; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62971-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62971-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 6CD4D282254 for ; Tue, 13 Feb 2024 06:01:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0F999250F6; Tue, 13 Feb 2024 05:57:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="dIIatEMg"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="pm82z+2Y" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6707225CB for ; Tue, 13 Feb 2024 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803831; cv=fail; b=MuSbiob5H5eqBqQFXrjTHyBrMIYR8EcTvRS6TraS3XPFVRAKkA28j3Y124ZVui1lEps4h16QWSzfMc+JpSA7St/Adu2VVfz2+4glJhvBzTfUsmOqmIKuw0sgiJY40C20DVdDwN6eCRKFMKv2anU5MvIcImcZ3u/eP5lKUEPkjwU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803831; c=relaxed/simple; bh=k40h/VbNGJWXmNNYS7xe7vLbqntRE9hDXq8zzQkVs+A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=ihN4bQ69T1+ixjOAWXDRHcdOMTuV5mxM0Nhq/tqHFLfj8I9WO8EoU6b87IlaCoueVMV2hE4qxvWfSfO35/qFGwdUJ5qfdKTu7ran+Rm82hea+GQ17BzJXPyt3z00Wj737d5SjpK2iqpvu9TByng9qwMATqWeXlmSv9pz1VBn2uQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=dIIatEMg; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=pm82z+2Y; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5Gh5K010530; Tue, 13 Feb 2024 05:56:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=VhKFn9KscEGMvCXQCivvG9esouJ9r7Y9KzLpBubsktQ=; b=dIIatEMgE6PqeYkNlMjYBgItgxRy8GEx1J51OAu+hlyt5Vn4BrRFYkmd71Qhajoxz0VL +prc79jvUnwkf+RsVKO9Mzhn3tI+yjIK3TjhDY+LTLnWyphAabvQwv4HjBvsOX7HK67t ml8vCnivPnh+aOAooiPhtmI/ry3iOidvFdqdzro8DTaKA5W2guTuWygvolTV5foI+qJ9 n3npHURqZ5u71dW/yi4s3Urz2kNHh/Scn/oNgtPq+Ntt/IkOOGDY+T9WciDPODDIR6Qp c3bQZDTTDe0kYiqyOV6kfZ6WG75HimDMsK+etoEFmj+tgdry/3fRZEB997MAKFyz/G8p LQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w824v02n7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:14 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D42gFX013836; Tue, 13 Feb 2024 05:56:13 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w6ap9gnrf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S74oHCcnndEw7mHFtmqbFm7zsPncPahCJVbgJF1T9J1w9REB+p1mukVewXFuvyYviwdoyFIU4b/nne/YvBE7+BcIju8KGhdBem/3J6dkcvRk1rIqwgCrhnOZ+vR9KDuqXTOjXBLMk5YpzqbBUz+zWbtkv5xBBWJUCkGiJ6T5yP9aUsmzOZlXGECIvSQXCaUHQjqlFiocpV1FB5LUhAqyGNpusJ0GsvUmMG9d6a9DcpSAKGgAWHNPAYQVXvwtAJPLHCOe62XmD9+m3E1hzaX0rTR17i4cPOlxicmjWmbn8P/PyqM8IrA8gYRLXPcTA+b36EOUxuVTGppAutzmQy6L7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VhKFn9KscEGMvCXQCivvG9esouJ9r7Y9KzLpBubsktQ=; b=Ype8FQdgd3XU7m1T0YdvRq+aLPYHn1G3K0LhZn37H7C3S1W2FxI4M3Wr4dPkW2+r5hVGlUr1hy6tCpR+OIRk4mwHI+lbQ20k4wjMFUQ5S18jOVx93UW6H0/Y/ERQ7e92/QeWZ/LDs8sOvB86gLPZ9oH+1B7gXPOAC3pAvj7HlGYlC8xeZIXVX6kweKQIM6wvGDQ5hoBDkhPCkde0q14F7+VqITd1k9JfU3U77Mck58rRQ7gUdHbBadrnPKpxRul+odvUEm5pyETAResti6jBRESV//vMu2wNsxWs7j1jzb+oN8QIC1shn1v31hB667CAoJSVX16O+2SvlfTUl+U0Bw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VhKFn9KscEGMvCXQCivvG9esouJ9r7Y9KzLpBubsktQ=; b=pm82z+2YtOBkrfsDwPEyLEV9XVU0pBXML6StYMbB9Q4LU+SKEqT/alj5CdIZWCnEFNE3IVil2eswgf3G4OVNv44zlRORj70f/lB58k62JmRBsyPbcuESr8CqWJ9jL7rWxIAS2j9DI/a+T/md5BJNMdsT8CUoNdKEWx912rsZ5po= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:11 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:11 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 12/30] rcu: limit PREEMPT_RCU to full preemption under PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:36 -0800 Message-Id: <20240213055554.1802415-13-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0059.namprd04.prod.outlook.com (2603:10b6:303:6a::34) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: d8f6811a-4a49-4a32-f4c1-08dc2c587ad5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b3ifi4FSh0pCe7+W1CWtFp8EtTcTfQEgKIhFUB3omeQR+jSYpAKweiXDUCR8HSjU34QJsCeG90sDGCYjViklnjRkjVPHKUj4cbQeds9sYYjALAjRk1n4BFYOdlX93b1G5rugHxSprvBCHdo4Fzjw6vdjJaZQ2popZ3jj5BdiLuN64Oyx9JjPv6cYUO2xdVZdMY21HKpL2MWMxDIt/oCdm6CISn/xaGFcZii6HZr8wIyg67sNjLLK6vXZYum1FW2HKaeB2ponGwb6fBHFYekUQMGzQWnIFsE5p6eZaMmmXVBycJRvr2t3qxGqjCZeq9DrQNDQ5Uu6rx+QO7nBF+J7liM/3mdqcjt4h6DGwLropGSL1T3gy+sK73jElxIorMlck/19p41GKoAMqV6AZaIrZ+b8Mamg6N9CdQY1HKDA1cMPF7L2MKhpFIkFLJgpUS/++b0w17ZxQvmCCRtRgFUJffz4OGK9P4UMz2JEyFIYayw2WZ3PeKBsIhz9GNCw1GaDr/zhltSxGyRn7IUaOXWRgkvXu6ihOUXgUUWcYObXpUdbQTHy6tRXMKXSLMTtOye8 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KoHyIqeNRwkYHy6EV4olV+lqg5RqmXsVqCIyQMxgSWKFWz3HPCk21YoNGWtCUUwidoszzZ9rYoct6EjiJjmvMpR3F0nDEXccVQJPi2dYc1n+AuXhG62byKFTJImLDvX4c7n79wbTWTTlm4Lep4yrtK92J7yTZ9QA79DtETFN1O5hdWTicxsSuIlvJZ6IbtpDZSChcISngRoCLt0S/VWh3ht07Xf9V77oKgjWS+xMI2yOJtxxLySnR+Ni3UGBJ78hg5PsyLhU8pdMOl6HVG5Tawbr/w3zWYX+FjEmhQ2ETpx1S1AH58vsBrf0lA5oVIb2pqT3e3Ljy4tj4vXe01WlveRfAA9IZfnfK8djE+cR8F1ytmzzkpe5nu5OtZG+KQKNm+RS15CY57D4zk8vAXbAOMEaP7zvGxj5JDqPHB/MizuWzAgFFRTNcpWBpzHMCZCeq6EwUu5P5hqiceo6hH2X4CRYNxCzu/zFA9Sj0ujdmj6lYjB8jbqXENyJ9P+JbBSF3qfHfGtDMteXZV91FqcVWFhOggPJ9dZ6j/BLFpEVWmR8iRj0c4Y/xgenniUS/0cLsLZ1V5Nldf6JhQYE7M4roAArgEkAHxP70XlZbdSJxHdhzqSqHWNhtImRM0QjPsEyLVe0QUOt9pW8ZCN02a2xe/PSrFMi/NuKDFxbp8OKZcZJfe9UMLRW4zd3g1VV95N+wX01UUPIpqt8i2B81YQsxdkg4CEj97dISFyBGy3yd/dHzOgz+aNewBoeO14o2yfzgS2r2BaX44UcoQuz8U6i+w8uKYKfwXFiW7SPqYjHswUNgAUqw0e17MDRhwDIAOcgy/lE2dDSlIpYNZoGLH4kWAdjyUAUhxuNVWiJFhi20ThzVXUtZqSegVXXRiIkoiq190O49IgjAUKXpQr6TWhyel44GxkkLAfA/d2nZWgax0JCiw8Y87vykfUPGRpMTHtu42PXiO3tCQLOindTDqj6ucV4hT+2RL+cRam+v8x8E2jc2CE+FHp1G1/MVDD+59E7JSvkkPjAlfhI0OXnm50h4izF1Wzrt3nsewLeT7saRdiToE0/bmVmDQHwrjPUY2Qx7nLcv8SDQ0JK0TL/pPSQ0Y8WHbissmgzVwNa1pfvp2G9fxJIGDIdoqQAKumLnbJCoIefZ0zv+FKteDJRkOUiDjG03N7mh2TpUCbFODAC8bnpwSNfCQhnMDHIf9tOofcSzQNzgeTkII+85bKNL6vG7gGnUnSPsTEk5id8dQQ6dYuDBreEanrsYrLmCNDU05H52G9zjxbHIk8rsTrKbe6db+Y41A+jtpnQjqbLu3AkeOnASGFYNTij3jIf/3K33YYTQmNuU1KzLNk+oDFPqZyPSl8/TQ4rRzmIqlvjGxab+4MdDFRbpuc7sYDvdgoGqjRuXI83f6alSWTMRVushBHpWfYzd74cYvuXGn/HGWeBsx38JbF6yweYNB7Uj2kCxZmVxsVGl6NarfyuF3ToD/B2nwcdDCf7YfA41C2EipEhfkuiOKwO+tRE8warsu6wim5ogBDs/0VEA+0fIOaJ1DMa0p4gp7nhSgDlZeDjkDzk3Pf+MyF7oB5/Em1/6VY6OTaoK9eFNxBx63jIAUeBBfZgPg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 1lP+DIq/f8qFCdr54UFgbB1syscx3z+VW02IjS3x5k94gRB+0oRy0gDmdnSnrJgLtJHnZWV4Fu5xrAD8StKdWGTTkJPSRUwHMfZ2jds40x7lZx41B3toiSWwlQ5LX3qlL1ouZOWmoLXEg5QsD/BukvhmNZlamGCbcH41iZ5cl9QP6vEZjLAVLGLjWsttne20lXiPcMIbCaCzNtveg5+udFV4Ch7Cj6drUOpoLQX2rkNSNR/tRqEsF0I95CXNrL478A/w7yFHLwHK7hku4LpBygtBFDV0s0B3hdPt7Z9VSw9ZGMv4QQnu26JEvFbHKxRoil+tLUd6koAt9Vu/XCT6Ka3BvL+hVJbOPyaGtgQ4qy0Wk2sCGl4z7H4UBt6Rfo28HsFRPcaQVGSQsEEMb3jVSiFAHzlmPOahRBmwLq4Vi2jfIE78c5D6fUM6/4Kcqd/WYiWD9H76bD46/GdlAR5SE4fkR75VyubK+rtcEnmktxupUASs31PPJEsvm46Vw4EU/4MDb+kVNHLZk+hg6CCyI9eVYJCgdFLiZKSIysL9wramCH+jS0VMtlzmjNlPNu/XR9JWLbCiq9e3Vx5nwEjQUdZ2Q/nVww2eAJRFjU1ozfU= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d8f6811a-4a49-4a32-f4c1-08dc2c587ad5 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:11.0550 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UX8CIX4mU5hC8HlkGgvsuafXtCDbML+lrMO7ZbJpDUZat2gq1oe3ntcZ3ltMsWGYMQJzyWrZ3OfV2BUHnDWZPXKsiSGjdgpr7RT3/1hnmNo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 suspectscore=0 mlxscore=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: WhdQ0jabJ7kk1VqGXCI2FSXlFvXEltro X-Proofpoint-GUID: WhdQ0jabJ7kk1VqGXCI2FSXlFvXEltro X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762370615159412 X-GMAIL-MSGID: 1790762370615159412 Under PREEMPT_AUTO, CONFIG_PREEMPTION is enabled, and much like PREEMPT_DYNAMIC, PREEMPT_AUTO also allows for dynamic switching of preemption models. The RCU model, however, is fixed at compile time. Now, RCU typically selects PREEMPT_RCU if CONFIG_PREEMPTION is enabled. Given the trade-offs between PREEMPT_RCU=y and PREEMPT_RCU=n, some configurations might prefer the stronger forward-progress guarantees of PREEMPT_RCU=n. Accordingly, select PREEMPT_RCU=y only if the user selects CONFIG_PREEMPT at compile time. Suggested-by: Paul E. McKenney Signed-off-by: Ankur Arora --- kernel/rcu/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig index bdd7eadb33d8..d0ecc8ef17a7 100644 --- a/kernel/rcu/Kconfig +++ b/kernel/rcu/Kconfig @@ -18,7 +18,7 @@ config TREE_RCU config PREEMPT_RCU bool - default y if PREEMPTION + default y if (PREEMPT || PREEMPT_DYNAMIC || PREEMPT_RT) select TREE_RCU help This option selects the RCU implementation that is From patchwork Tue Feb 13 05:55:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200214 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp348231dyb; Mon, 12 Feb 2024 22:01:18 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWlbQcIda62FtkV9iOT9malsBQsRcwty9A063hAi7ViUF3yhxtFexbqtXj6RHuUCLa1ZRcEyT5ESQdPv73DZMLb35RbBg== X-Google-Smtp-Source: AGHT+IFg1KmH273XIEhbf8BtrRKf3mxMhAi6AI+JwnjIDMu1alzdy64tHNCWv6ybrbTW2Oc9SOIE X-Received: by 2002:a17:90a:cc04:b0:297:1113:9af7 with SMTP id b4-20020a17090acc0400b0029711139af7mr5074193pju.19.1707804078460; Mon, 12 Feb 2024 22:01:18 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVonLystNzjddEQ9U5b0hwUBqHtY+Cr7eLM8WUfwgYgzn4wjK5+lE2mEazNy/rOC4eUn7oUYU7+oaU/Y3fS7RIfrxhcYQ== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id 75-20020a63014e000000b005ced4fbeb75si1448380pgb.581.2024.02.12.22.01.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:01:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62970-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=n7jkOAOS; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=QN4+elrC; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62970-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62970-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3BA3E282663 for ; Tue, 13 Feb 2024 06:01:18 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E75D4250E2; Tue, 13 Feb 2024 05:57:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="n7jkOAOS"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="QN4+elrC" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E15EF225AF for ; Tue, 13 Feb 2024 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803831; cv=fail; b=HhcDQd2dwpUddTy83NxaNNyeFA7sm1qaVwfhnLQBMKcbZfsTqFYVvUH0R2hQB03GF4zsdDbjOLgVsra6vx9KiazwFnSabLkADpBooZBUJM1QQi+8WQL5b6OG5g6Vf41kBEBPqBUTGvXbLmCOr64Yv7a8128aKBs9FeAUNcNzvTo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803831; c=relaxed/simple; bh=ptzY8xYhgaz8pOJ1Q9OkCaHt6nskL8/KOpzXWY9aYm8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=J4iUjjC2pQfM1juuCfDdGICc8ftOaU0UF5tGMi9Y2pxnxpiSyRj5pYoZg3zkzT+l12OOOVH+5wVygI80A84fOzHiIdMGgWUGk0eYxcxhbUoy3nZ1o1UQU5m5Ogikwf484SmSudusQ/dt0KmeBi9Pv2zOWCJSh3UsSoroDkHFw+g= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=n7jkOAOS; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=QN4+elrC; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5UC7m005104; Tue, 13 Feb 2024 05:56:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=R+WAiT+QYHc5DleZN6Kcu5t68C7UX9ZuJGb5W6WXKpo=; b=n7jkOAOSGPrz0xOT1ygWIpy+pLV5AxAn3lG6PMeyPUz/WZ5fxPTNig+90pGs+nyHj9/w CDZOrWT0gTlwjzoRX/PqlUI/WRMsaEQ2PoXs2HYpUJUT3bz5ZzfhbZX4VouBcB+gvaB+ SSVywMXTOiobmXEANm+7mZui7wLz9tO0W4tid67jzlDBJxkzUBPw4/rmcDPDGoYOEIUw nG9REbNgp/te0c9bng7pBQVrZiP+nJdgS6BL0OHbzq1rnAO359pCXn9oXoKv0gSp0qAm hV/GZkrEUGRzfF5lz7TjvRIlmLU0SZFjkNt5Pn8RwVoPH+rF0htnrY3J0UKl8Dv4uF3I Iw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82atr1rb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:16 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3xouA024776; Tue, 13 Feb 2024 05:56:14 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3t4r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b/kv2IrIUvtDW5ojLg5dusdL9b6Gp9f8hnijBOsIG724+iItKRjI5sXFknRoj0JIcbJjx7LLZLiM8+BBXe8ygbc9Rn1kWmG/KRyEDrjV4Zomw6W2HwFqca3qotsmgxG5aB7u2TisisdEQSbpPXNt40OnQmbv01SjmlzkDK3qvrWklkEwPcbLW1CEmBsG6h9HZA2K5lHRgNr+21or6m1qtwApZ8kUCsnix2T00a/YIJkwneXimG02o4aqLU5sgydTPOIVwBfyK+KdkX9MINXdNr8lRSdJZ3bs4VPG9/lOz+BtJduVipYiRjsPMXRUJL6M5PEpee2vZTYOaK9Nxftxxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R+WAiT+QYHc5DleZN6Kcu5t68C7UX9ZuJGb5W6WXKpo=; b=IOY9FhR2HafvR6ttJvzSBiXpkOVScWbmQrq+qcJ3IHrzJnoHDM86p+cnxDtW8zIwH+HWagTd7ZdSFsMuYOnB1VfpKXPHtJNWTXeT14t9r443iQrfbdS+/iIl/0pL+AFArSWCV3W1vLTYk7Aua6goLqqxaq2yvhAIaBpa4oh6ItopCe2K74vrQu9y6+TOwlWBDbFXZY8fk7ApEgTjBXdrsrP5NgmJi44omoNUVlyQXTPN13AJHrsVZjfXZtMm9+TaYmWYPeulj/2suvyTbF+H/jHKDTZEelyJuD9XvhMRMr1pBpDefvGkF7jGSg6f5fGXOCi9Szg9s6WvNls+HORBOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R+WAiT+QYHc5DleZN6Kcu5t68C7UX9ZuJGb5W6WXKpo=; b=QN4+elrCL01cFwWVmOrmsqg0vS+t/cWAcr+grF9SUA/1Qyz2+ZAOrV8xdvjPMkdquFAGbXSpXTcTjf0IQyDvQ9hd7QRVlAv9u99Un9FWGI5n1AXZe9y4+8AzauzY02tlgb/kB+hxXbEOoFH+jqUDCo5Z/ZcrTuh2k687NRrG6Ak= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:12 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:12 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 13/30] rcu: fix header guard for rcu_all_qs() Date: Mon, 12 Feb 2024 21:55:37 -0800 Message-Id: <20240213055554.1802415-14-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0032.namprd04.prod.outlook.com (2603:10b6:303:6a::7) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 4ebc9aeb-94df-43fa-e3de-08dc2c587bce X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cm7Ob36Ume456s8OVBxqMxQAhzlfAYE6A7nJMgKIiUf9/8P5hgavI0vw3mUvzvmjerjLPB7tFTr4x101b0YhI00ary1tPLbCUsYktbI02vKxg1wyXohbGEZYLnHzGQgdgsBfH85LwH7YkExr8tnfA3aq7xmhOs5jyT9PB6OULfc6PyoYPv63s/mhU4eR391aSRnBRpCwf2EQBt+p59BjZRHWZYLgtEWGYMgke1ARhLGRGQyiWBfVgqd2VoycXSTnmlzJu8zxnC0n9uzOhA3mxCl2scKFjlQJ4odpWgYtI8MQ1kmi7WF9uVi+29azoso5yOOkIGHthNS/e1aUj9JkQKeo9D+rNIz0QEYclMMJxhQl9eFh4hA6ZbDOf7n5ps1Ns/VFP7re9Q6YBA1J26jfDHVzLeOLcr8DhcHAGyXhWu3xLpV1zvxqFmm1a+ezgOFw9PsRopx0I6ZBkKC1NQ0mdzPAGbTtR4UyswN0957zWn73KjoVztiGZ7mAy8MHugIkt9khVxngZ73/8RiGmpEQ5pGcdvEqwjl4TS37+xAMVt1amyJB4ueJ+3+UHqqPd4AU X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(4744005)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: UuQm/tVZCPCqWJy3I5mnxdzne6BEEM3GdH6UTHjiihUamFesCHP5SqqHSW6iHNIc88nM9UOCwweCwHBYHI06Cy1+Bow05fnmwi+/Rpz2j3w7Tt6UEvlO4G8PFHU39cIYZsAOBJgr9XPseH+C8lXDWb7bRslLVtXLUNWxVLZnQcgS8p8RaPqTN3m2LErTixMwMAzLDg5xZ6TjzLfQLP0piJ372XyUkOs8WJHmFFK8xU14jhkkNY9pP+YQzzVIGW8s3iuBP1PtBrXzitc5+LEdo+FK46iuwZMVji6nWd13uHYvSPgP4GVt/jzTqg2Oi0sxE6Lx7L+1LcK0UIhj2ZXFDAmWyuGSvCYoQ7tygas56XHNIYywxXt1akoB81DOuIIMz3OTMS24NfmbM7Mq2whPuetmlyX60aEjtyCaFSV+oyCKcmKObVvDVP5YbZ0lOpS4t96LEEn+VGjQ708eO/5xgut9oD/INBE3yJ3Tj36Lx6rfO92hkmchrokpbDOb3I2vGD61H7nQUuUhvHxVtM7NEaCYjE6325z9R3RTlr0LTatxZwxkYnomMTxyMSgVAZ7JEvbJCNLLOKD6Dir1f87qFnhcs3irZAf1mxHb6Dkl0eazQZ66rho184v9PBX3AIgGgARR3gpbHZ8NBsHYFP5umPNbIAbO/YADwSyZyiEqMkHjwcabN66zAitvQbeyLslLRoRKey3m1r9ov28WeWUxiNhL25TEmcc77MRHQNsW4IZuBChQ9xy+OyNJ8hiDpoeIA6FgiGX0+O7SJT+jToUAT4oVUUZVxKlov2GBIalP4I0/icl5ZXSIYBzDwr88XXzTIPBPBGg+G/+76Ft/+gRK5o/a7+k+xMNDFaWmfmlNsOrRzJdkrnGA0+dxzKpri8w+/rXHp4KB1Nr02yDqwHsCf8ieHVZlSY/Ytjj2WNkmFaFCJvKLmaBgtIXGVFysTfLf+8p33oibCPNh5psAjLQ+Up12OTaHY0s+nXyH+GzZwoDkQL9TAaneAhw+srKFb0Edc8x481/q2mRuosbUODF9+BueTbBefgqMg07xjvgmpYUUGg6d7IC6Rt9gW44RVaR5ygL+sPCsWB55oPyI7jYmYfi2ar7GJ0I6pAJO6khw8DikIoyL0enkZPqyrwvvdAb8978+o2917epZdui+PI9Vh9hBMUi/Pb+vvwJWgfoA3niet0EjV6ill32kLaO7DVMsSMNEL0ETKmBZwPNVVDK0Ah+rvnTR7r7Vih8rftQuxcaErHczJ9+nSAXXqKXhebbZKadC9G7LXLAhqTr76Z1o4d5PI6mZWj55jrG44JCSUyUj81lVduKZrssfU9vwMfyhRhyuycyt4cPeZ0bdPJ5LxiSYd9y6tDyYczE9jz/XFrQwhwbUA39LChSiD2Pa5BDjroyN4/6nMW5FKDhpguKO4kod0kvSXyClcXebkQufUjD0vj3tlMDVOiGlr4w+GsAyoHuZFRDslU91nFuUb2uo+BZzj5D9U0CvD4BCZp/+c6c43GFVVZmhpJEbsUZidWbnM+HGLdGkH0g/R+1w+cP+JoOwV3R6m9O+dzHp68JModizthla9tkITCYpgFl2kJTNN6ZCZzwZZuqbrX2x8Ymvvg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: VNXR8A/83GXNONEHjN62F7ZB2LNXJPcWcPIkbTbm9G42Kir68BoeuqL1RI8ERGGjMlgWMWWTdhr/1VD7TGpc7p446Pn5dPRlehlO9ZWnNsro3Y61tkhD5tZzVSI0S61Cr2iDB1BsJEUUbC3z8sjupd/PFEvmSRzClowHs8YciraQ9iHnfSOmYzCekrHb5Myv7WyWAJfijPGONxuGFerf3Y8P+Dbj+35//IRGWXhE0sa7jh6btgtKab3izDo+x3U1zf9PXE83Ey4vqBDGcU/kM/t46WBuS2deLJjaS8qXqH7TeK/+vlxoUJ/HodYTwFYRX/tDJQtAFM1ivf3JXmHmPSuExMvlzvOdX9IZDeliuSJQgT8K55fdte1N+PqOT/Pq2aahY7TjV3g69asop11dy6tucP3yrPBX80NWRT28lScjxcqgLnRlx6r2Hnsp56Pngmy/T6WclivTlFXJXMS9Zpltuw5KG4zVLnzPwKQJkEwUzjnH1r6JWNHXxLWCyd9eArJLxeRT7rc2YMsGAgObeprRb1zf3ImCGyZcruR+luQGPFBPb2LhMSX/cnCchfy4Up8Gks5E0bqPLxzB6werGDKtAVBBga1DxY04Yuv6f/s= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4ebc9aeb-94df-43fa-e3de-08dc2c587bce X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:12.7414 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kPiyZD87XgGxyro5HWGY+PCCc7oPXhBN80o7ODrqjdrmMmdZMACQQw2ZH3xZEsTHn3G2U9eWIMoozXjXSOSmf61GzNSf8bqXJMdcbh//TmU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: eMTZOi3O761qKBSqv-ELMEMHYC5OtEne X-Proofpoint-ORIG-GUID: eMTZOi3O761qKBSqv-ELMEMHYC5OtEne X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762369191534375 X-GMAIL-MSGID: 1790762369191534375 rcu_all_qs() is defined for !CONFIG_PREEMPT_RCU but the declaration is conditioned on CONFIG_PREEMPTION. With CONFIG_PREEMPT_AUTO, you can have configurations where CONFIG_PREEMPTION is enabled without also enabling CONFIG_PREEMPT_RCU. So, decouple the two. Cc: Paul E. McKenney Signed-off-by: Ankur Arora --- include/linux/rcutree.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 254244202ea9..be2b77c81a6d 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -103,7 +103,7 @@ extern int rcu_scheduler_active; void rcu_end_inkernel_boot(void); bool rcu_inkernel_boot_has_ended(void); bool rcu_is_watching(void); -#ifndef CONFIG_PREEMPTION +#ifndef CONFIG_PREEMPT_RCU void rcu_all_qs(void); #endif From patchwork Tue Feb 13 05:55:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200208 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp347278dyb; Mon, 12 Feb 2024 21:58:44 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU+Rl/c4iAUjBC7mbK753dd6iWTxr5OSzVpjHGLWGc8dkrTh+lzjmenvwSeImOd3F7NBGw99ZHW09jLF3Cn+VAawp/USg== X-Google-Smtp-Source: AGHT+IEBU/l99Yh3dmtk5H0hKaHBgrn9DzmGac6lUFx9pAnYHqeleNA7ywC7eO5PS/1yMRsbSpWR X-Received: by 2002:a17:902:e543:b0:1da:22d9:e7be with SMTP id n3-20020a170902e54300b001da22d9e7bemr10797339plf.23.1707803924492; Mon, 12 Feb 2024 21:58:44 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXrxWYkU2GzFRmDTdugAdYseFrSsqMi0i1/Ak5wIbidT7SIXZi66+Bwj8HjbxDcj7aMIr4iFoxcdQQujEKAMRSCCbjD8Q== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id je19-20020a170903265300b001d974e1376esi1453874plb.494.2024.02.12.21.58.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 21:58:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62961-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=nN4XiDc2; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="Hq2hi/ky"; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62961-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62961-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 436BC282208 for ; Tue, 13 Feb 2024 05:58:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BA3271AADE; Tue, 13 Feb 2024 05:57:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="nN4XiDc2"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="Hq2hi/ky" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 527521755A for ; Tue, 13 Feb 2024 05:56:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803820; cv=fail; b=l+7F5JmUYjNzZbGBgEzFdtgIf/q4TsygTE6apt27DJtR59+8gUiMpkt3Nn+bqhsW1T9wTfA2ddjex7OWUtPZCcdlObabGwaLkJBE3ptZbCZF8goVpP4A65YXkGkOFEnpeghxh95CCIHHFsKljHPsjP8zwk/34z9kpBboeU7Fi8A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803820; c=relaxed/simple; bh=c5aoa06uLaFqzpqTyRQALJlmMtWX84tL8prjP6W6Gl4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=bB0XArtno2Dao8jthOMAoX/g6zV5yo8t+rpzkzo8MrLLSXL9hGur5OX34blOh3ALtz41PK1kbL75iHPVlqSf3KQaToorsC0U4ZUrCVvD+6jQg9Piu//ivuvllbePnhESr6niyUWlkhR7qAAjOcV1I9EPh8W5DBkAjWWanqgDg60= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=nN4XiDc2; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=Hq2hi/ky; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5YtxR005928; Tue, 13 Feb 2024 05:56:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=BWOjIhuEiE1aoMAgsLTjONO2NO8uIgFA2F8rCNlfaqQ=; b=nN4XiDc2HtprlrKdWEwL+gnTAhDPfA7ZzmzyphZpedjj4VPES8VRHwdJYc/2VBIoAuEs hfcfJ0XFQoSLV4f5r0la7PgoiycsvlkHC8SwJkh60BEjZGY9+9XUKgGcLJZRzXqweVIQ vJkdYvlOk5TxQd/cKLQOJffh/DTEIOpKeofVxkNPm17Ewan03Xi8We2N0caNyQuxXYLo 7cXvgLJsXEG6Daj3WKR2RPpK39XUlISksN7xRqIERAad+y4vM+d61CzEZobwmedaHPzx e8XwpNr0dMoG5AP9Asdclejq7378tnnCmlwMS3b8pTLOYbhhiqqA0RF+lBJ3ENdV+Qhv vw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82d7r1ga-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:18 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5EER7015017; Tue, 13 Feb 2024 05:56:17 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8gv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hh3utZR/4J5tE7FzVARTnkDYbUBeG7cjwUIr93dPxQyoTAUI1BdERHpZRfztm0fsEKxnMQnVL94kw/+Q6DiGdbT5H9016JzON/pNL2Ce+wNlw4MqCvHLGMo637WagunS8320gKWdAO71ym+CxiHUUwZNEmDZL8N9xd5bnLiu/4lO/9NM+ul7sHAq88kvPQ0mgiu1uSlzuv4SF82HqYWxC6lirP3XPfjh9TY/G/zc725uzBIGoeavPdpmqUrNQKWsVpgj/yDDcz0PMHYzKsCw4bzDpqr6eM6EjT4bp9Pz3fquODQln3hz2mEOvFKsHAOC1BJ2h6Fkehot4FfxkD5XgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BWOjIhuEiE1aoMAgsLTjONO2NO8uIgFA2F8rCNlfaqQ=; b=ZZgFxl92Me2sA2oR+PUSiq9+RPoqgMa8PfkcyRkMlgHmPWvbNcwjc1Hwl7QKVNIIOlnGREuJFf9hPm8XoJhAHhowDug4ePuQej3NH9luUrVQyAE3e4/asLwubModuWTqVVuQyg60QWjFpB/tcKQCuBVgzZWTeE0yxrkYxEvj3X6fgP+CGVRG11GCYkkQs944tCAaqWqzvhuWor/U8AvCFlvJF1fbvUigpkQhzNwlVKohtFFVh4hLzOSDnZyGZ38fkqj2PWSQdVlEi8ZLazR9nBrKFFhNlcpiYfToj1hEv/vDjO9I0myWMdWJ8hLXhRW3v8g/usnsdNzR79zax15unw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BWOjIhuEiE1aoMAgsLTjONO2NO8uIgFA2F8rCNlfaqQ=; b=Hq2hi/kyp2+kIxDHb6F8y7GBRD11xe+S28ithVhwPOdh6Uvk0m2vj+qRjKdX6XsQpzxBemMjF2kdMQwwPiSq/E1a2Nmhg8DYX35vM2TwmTu5ClATPuIFa+x9fqL+2SqVhF4EAdJOG86Fg9KzL4ECJ+h9GF0sPuIdLB4YZk6xOfc= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:15 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:15 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 14/30] preempt,rcu: warn on PREEMPT_RCU=n, preempt=full Date: Mon, 12 Feb 2024 21:55:38 -0800 Message-Id: <20240213055554.1802415-15-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0293.namprd03.prod.outlook.com (2603:10b6:303:b5::28) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: a941a476-0361-482f-f550-08dc2c587d55 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OAgf4K81EcG+nLCxrarAXGWccZcFIF0kl3MtSPuSrKjH1CvRayE2MY+ti5qoC5es7wDa6RDxf4Q6rKk4nCrwv9LlD9byEMH7kYzR02+rV6x+CFHajCrkwPaxTWD9XHNqexfG5ZsWtonPj7bYCBsv/5TjrTQXeOva0zV8WZ1+Kz7u3gjKvYuzaXS3INsMJbtvlsuWw6WawC/wUqZ9DzH283RjL2N2wV5Be899kUQfPtSa6ymfD9jl8jJcrm5ro7lRFhD37gpRHcdjP4lQtqLCBu9EFau8dKp+lLOzam9TUvASn8GRGpKDYgoCjfNl4tXCCmiop8ufofmm6z58HvYmxaEd/LemutU8WzAs+fvf5wH2Hs7rcmgiFdQ4bSkm2KCu0HIdX7zYZslwNA7lC0/rYoblyppQSZDKIRtxSSa8efCeyIUduo4fZtG0lX4Np3jgI+MQf2VJBMvBISkkcpE1oYg/Ocjo3sPsRXWZ+JJcNRVAKCzKvxZOPmSi6VtOse4+v8hC/GSZ5a0hKQvbhzMSST3CmHmbzETAOd0caSY8bHo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4fuBaLbJwFFoa85otIIr8NVml3pcW/VwBgaQ7kt3axj6gB71p/Gk5oaxpCnhEAT2eCNcC4fHDgiA4sImpN0J5tWGA4XLvbfUFFYOQIUYOqQp74+Be+VC1LViNdP/tW8lFm/MY0v4nHZx5oUKyBmYWHGAqC2dfwWOqBpDButKHnEjlVmCYZXgb/gv4wReagnh8UXQTFlQ8lEeav8NXvttewJvg7hVbYunsWrtMWtyA6ZQT3LO1Ddo+lah4qsbhzMPJJapSAYMpVxXGHLAcpF/AbE9KyJSmhvMr303CKhj3iumH9ORYz/DftEqNIs1zGySt5ys8nz4i6M0U6y9TkZALdT9bDm+oTNABfGD+IBFi4VhbbcSwRkHZU0zaq35LvpgUNY/JLKRmS9KwjBEH8d8Z8QnuwSS9mapiyurf5fl4XCLoqZZKNAhfbykXRD/ulk16EL1bK8cwN66x0PgchX31IgqMgCuMSiYvXz/PrSrrFbyOixAt2UWBTtZB9ddHPoPjYO9P5d4XoqLsFbDT7fkkp+iTkM3gs4msLn2JTESfrS14jrZYZxr41muCeYYFqlglBVXIudU5hFzxfz/gWOrQ5PTC93QWd21+h+lviD8CCsUVX1yiRPjidnrQwLqxNad+CBep6xlzSU96DPyYX7n6r/ouw15VKgnrgdBzKV6tWQfwy1biguskxKR4W2vQivnOWQov/b5PDaBv33B/6UQ3TmbELXMbcyBPyy/5Q/Bb3ICAXrntj4rfYCLlrmYqf398CMokcQ3+dvouCJYf0GcM7dZpsKnswOKE7G02DSeBomF52lJDPIaYTWO/gJk6fLDWkKC3lI2I4Tb1d+kBQjPyXcr70sWIPGBzgPNbAS2LQxhMXpnpr9KpcZJYXs+jgB1Jz+gWtqVaARotpe7Sorvy6r64J/a4OeC+3ruUxrHspGp/E97tRaqLBrF4VMowNxR11ljOpm9lDdsFUSxkdfTtuO+lvT19yqoLc1HtCsoe/R8716B3RNdYESHMTeMeltoqSRDC+b6TsEUiz1wmJ/zutCvcHQEHtO/lwlfxy9k71Cm1DDCU6napcpB4JlWJUElMFxSxhdllG+7z0jsMfuJ2ACNczdgVQBa8R4HxvUzeAPXBHuD0RmmXH6RuSThehVbIgcGQReBAYCmQwFn5oAuy8mXXcg/V7AmiQy2hQbsRh2O7dikXVxlM+uqjHTS3s5ZPAlmKqQphIFOuTMEHFPsE9UgJftDtM27Xy8TCG/CrwGenGbIEc2U4FsSkU10FDylnQYQlTy9Sa+QqXycCexRNl1Ua/7JYVn8bFmxjYBd7h1yUUp+NMaM5i9B4lVUZw/PXbZ6/Jy0EV1fwL+cZP3t3x9+8M4YLApjBoe/ZsBB8LXwltCli+U5arTmGh3DmgniAkIhpGvERkKZllyEX0v5AHvx2RVKHK2uKlh1fFtIz5P2R1X/IhANHEYy6sZIWFPSOW7+cVZk/SBQNDISaLII02sGcxCNRXMhCNOE7WWxEwuhb/4kQX5YKE/22PBliXx7sn5YbzCO+TFHXAnqkUEC4cUftsqCrx+hiJXcTaCXvfjHBNLzl9oywpzIcUd8HDRVdulm33CteigEtPyxhsq1SA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: HfR41PFKv3pFuse5klcH37QBJ45XRjIDlrjNvB28atbzO1Dzq5H5/S3LqXESQMwJ2ShKf7lmzN5/lLHujFDbuH1OBM+fzebYqBQ0JV/H5hdjDFRMBKW8zMvUTkcI557jLeX8wqkogEsu0O8GYHpivRB88Kn1nvANMf4I7G2MyrNsk/LR3XAwG2ms11zDJ3OoADiznNFgi0PMY3UYOVwt6ptcreXydZbYhQlqQMDd+iCIiTyxHm8EMShunYFoMFAGsgtjr8LuaZIZHFlkNFNZc2/u2wiD4mzTaAHTpHEeSaoogJrSDN0QC+FrEyUhCiaPwkMmBD8QQ9ho2Y5Z+EwQAFzKjaGOUTlJ80hF+9DGbTafDugSptr6c3rnkUqL+c6/+MUsdxvGfBL+nhleZlplav/tFrFX383dE6/YKCVIzGi6L8+NT41ljG5QBy2QIOGEA6nLGc+Q3IsSM4sZI/Th2/yt920SNiD3jSFYeZN2XMRLIUPGEWS80XRxy8lRjun3n1GO7j/0GRLjyEuQgi/vJbK3yZEf/flITe1CAkEP7NfBdlpUFZL38fjNZ08cVgwTZi/VEh2Uc1Xn5Lj7Kaqpb6cMYHjjVJgKrkQvO/5WoJI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a941a476-0361-482f-f550-08dc2c587d55 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:15.2580 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: g9TZVWkoLhsGLvVv3EHHg3MCRSaRikGInhDoJZk/7Y77b4E3TK+EzUJbgxWuXaolmtKkWpfMy2/9j/DSwBoq55sCdyL3a27GGBNr8BuJ9XI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: pWfdXM9vxQe9m_RDSgPOOkT7fAqry-9v X-Proofpoint-GUID: pWfdXM9vxQe9m_RDSgPOOkT7fAqry-9v X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762208194453215 X-GMAIL-MSGID: 1790762208194453215 The combination of PREEMPT_RCU=n and (PREEMPT_AUTO=y, preempt=full) works at cross purposes: the RCU read side critical sections disable preemption, while preempt=full schedules eagerly to minimize latency. For now, warn if the user is switching to full preemption with PREEMPT_RCU=n. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Suggested-by: Paul E. McKenney Link: https://lore.kernel.org/lkml/842f589e-5ea3-4c2b-9376-d718c14fabf5@paulmck-laptop/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index e5d631197f35..425e4f03e0af 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8922,6 +8922,10 @@ static void __sched_dynamic_update(int mode) break; case preempt_dynamic_full: + if (!IS_ENABLED(CONFIG_PREEMPT_RCU)) + pr_warn("%s: preempt=full is not recommended with CONFIG_PREEMPT_RCU=n", + PREEMPT_MODE); + preempt_dynamic_mode = preempt_dynamic_undefined; break; } From patchwork Tue Feb 13 05:55:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200218 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp348770dyb; Mon, 12 Feb 2024 22:02:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IEMg83MHWKQVhspXM9BUw7S0FAAbx6fR7tkSiFvqmoNZOwj6FRDm6jWiRErL+KuCz4Xa4q9 X-Received: by 2002:aa7:c68b:0:b0:55f:31f7:4279 with SMTP id n11-20020aa7c68b000000b0055f31f74279mr6037925edq.12.1707804149360; Mon, 12 Feb 2024 22:02:29 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXUX2ZjY0LyaTjsAMGrDUNyC85yiKpnm6ZoKQH8RMm2DtiR9zKk7bqiDWs0qPZmy6Sdlz7QiN4dHITGa0hWlsFtkeIS6Q== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id a20-20020a05640213d400b00561cc26deaesi1224899edx.304.2024.02.12.22.02.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:02:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62974-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=AaALZoWW; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=BpZxWOUP; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62974-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62974-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id C73281F22BB9 for ; Tue, 13 Feb 2024 06:02:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1DE8232C8B; Tue, 13 Feb 2024 05:57:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="AaALZoWW"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="BpZxWOUP" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAB2D28379 for ; Tue, 13 Feb 2024 05:57:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803840; cv=fail; b=tTic3sN8wV/8u78sMePaDe+YRQVF5z92WIjwCICRygfH+hpzH0p3cqtznjo3p6Q+kRrBQ+LQ5sSQMxCmoqo1h7fT2drcG6fKrx71Uzx8kGaOW+a4qy753jppJmpk9mzvMiBPzjYlEUsG1sTw1YVBQeicrlKJ30xvO6gDM8VaWi4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803840; c=relaxed/simple; bh=JjC6I1Yt5TepbgP7pOwxCPr07L5TXN3oQLxBWuDrOlw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=jlZUENOLAtEb9P93KaTfU3gWcpKQAp/TYZUu+Mfof3Ob0GmOqOpu28bgM2HhkS1XgoziAICmoRd6GYXvNdve5gYS5uIi905GUtYWsB1rT+SGQ3ZYXdYHP59DC8T1fFeaxgB7uJ9gg02+UoC0OqJl0Ts8mfkiWVC/GgR9Ps7uNGM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=AaALZoWW; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=BpZxWOUP; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5n0bQ013674; Tue, 13 Feb 2024 05:56:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=yJLNdEfVBhdcfZaQk2QRGE/kfdPCbqx9mH+b0XBRGZ0=; b=AaALZoWWj4y0oUL0YRSnQCrckyqQYqRd3acYPnXkn+twVhVe+brXy0O3hqIJYSChxEnI z4mqNG9OiBTWtEwIFZ5nK4FQXoU6qm6iTnxO9+w0EilnN5W4TsEtOArW5T//AGaixiLM ECxUHqLtB48SM0ZY8eP6q5wKiPQpiLqTP0jD8fcpLWGIkCp7T/zWy553B0JZ9fme0D4L +54yW1Evv7x9oHLGH1a59T7g+sQqtBk6SJfu1aEQEf2z94PsyloUfuN3y3raXyIu7LVK xKgjtSqYzg4I7H2pcKCA40Cvb69MKAFm7UnznMkn2N8+LWR+zWKXBT1eisX1DNGjw2KN JQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w816tg49y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:24 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3YlWG015085; Tue, 13 Feb 2024 05:56:23 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8jp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V8WixgJtk6rYj36Q5uweHWUseXTzJ9wUhSOPuILpSVrTNTQm+YGquXSe1XC2V7uaLq2PS+EgToGzg0mWunyZrLUe03XiOUDwb7V3VhzkYVcD9Nb49SUkfgg5It1Rt4OTipOA6VUFk8ZpAIMCfaCirppQb+re/a0nX1pL003YS9iOXeHoSXbXMb3rJmOkjju9eBHtF3GfNzmXWLTBQ8b/S21QjNxlcWlA1VO27vYEERMYCP+WTJlZcOZEiVXyKfbrbZC270OqbjKDG0uaeh4dlzBvwd4RSzI3fEa8Rd93yFWJHiL/KOYk+HnZdyUZDXLKw5KI8eQ4GVwM8Pqn7+joOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yJLNdEfVBhdcfZaQk2QRGE/kfdPCbqx9mH+b0XBRGZ0=; b=mLGKxBJgVTT0avov6eIVyUBE/qxX/r1mKOrMbCQtO2o9g5ZEDFe4+vH6bPgfl19s09vcSsPT/iFb0IoYu/GVjQ/XaNIDjdKIyE4TOss3Hf6+TyqQPg3BLNAI5K5zAJJPQDUt5xb17L3hiZ+7oV1gKdCfX0c3fZMSYv+9YnKn3F3W9Q6IGuQJ9ik4HpYx7emAT0xNdYucSGwJb/nNWFpMBDG59CiWlywhJvcS8XvT6uwNf2oVyFGgnkTRwzxtaxdx3uHnzRBcWx6pYqJUMSklDAVfgFsC7LIJMlkKeYwGWIQV1c1v7VyqSBTyjovLhTwLro3aH7NihgX8I9u8+oAH0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yJLNdEfVBhdcfZaQk2QRGE/kfdPCbqx9mH+b0XBRGZ0=; b=BpZxWOUPt4Qe4cwMxGBM6jX9yVCYi/iJzcatA+jaIUz5Bs53u2Qp1WjtEBlaW91EA7DJ3MJGwssuT6OQZ0XoE/U0FMwkZW0HluEsI3oDbJ/7ojAztPsFDnhcJiRk0Z50CDEAI/Sj32duYItVhbt0O5JPTtyGYjICePYxnDgVxyc= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:20 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:20 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 15/30] rcu: handle quiescent states for PREEMPT_RCU=n, PREEMPT_COUNT=y Date: Mon, 12 Feb 2024 21:55:39 -0800 Message-Id: <20240213055554.1802415-16-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0229.namprd03.prod.outlook.com (2603:10b6:303:b9::24) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e167990-1449-4efd-9094-08dc2c588034 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VG9ABPqmMxjxVpZSt9tzrUIyyueOGRoTEwBmfX139M3jsfcmaMf0OI5yvabT9F7HH/M2Um/PZ4foF+lt53G7URijPermP5Ivvn3nH08Ws9rf0ZxCGBfhmhIWhcEr8kSBmWPNuRaPNG6vPKx2Oz28qWW6myJna5ZU2XWTNGRp5TB/XgaQrqeImkZVHRcuJMS6TqIsqBZPF7xF3NRDRNZk2oFlh+mQKS+KKDrt5F39WYHwbctRUyhUZZW0KLLQYM7icJkfkQB/xk87UJFpvb4HCPo1bi/lx0WKIA26a7+r6runrjfWbW1s1acoVxTwM0T94NmAyDeGlaaBdSMjnsjm14lfCnG/l5lrRjtrUbF1nvaLP5QJ4eyYP9UZ433OqYtE2Jy2To2xtSHuVKl+Ux73BN1AWupVjbGTs6htXaVhjcvnRSAR99PVRrB2gMPCANMurW+WtrJZiljHOpNQi5U9bkF8aimseRyB/q678fZ0a5DC4m++Fb1/A61XWXEcFB69tRnTHXJI35L4npGek6HT4isoDu3gIuC69J4tIXpHZu+MCRiUWOPUvyqCpUSroMJB X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: n6eOdKp+2mVk7mkEIT4aESeKMKe10KEBEqeE1Y7flc5EbKc7FhmaKo+Kp6X0pnqjVcVq7V/I9qPvM+GGNyvcG3ixA1kTks5fD4BkvaI3T3wlgz3why9aOraeOgf7mBhf2NTq9hyNoS84IM4SoBra1kEM+DSsCYfQbOO9tmwnXl0yKOsNj3unhn3FrhEbTjDT4FGlMTudrca06rjtAoQuEJkOeyGzMtfPvPVLWtRWrm2XN6siFMni7p7HbWRQPM5f5UjkdxmWBWOroPy+RQ8htxSEv0g37uoCKU9fyQtO1nv/kOasNqCEXf94BRd17kXp17lRpzba/tCjk9XvTToaK5tbJDy1RfE1t1Fj6ZX4xdc3a5A/yhgfCEzAdthPQHgy5JPRzELqiIiKPCcx79ottdhjZqTwjdKGzBXevyN1YpU2ZJ5Owv5E2MFn/gY6xUPmDI+/gANxeisk6ejS2BHhEoKQsdOUAjR/ltRjFoynuYysY0ww0ZC3QSPWYuCHMOLosHExxAilEH/9LYoqDaR+abIrPLMEgvEg/WfdEFGlfAdHh4+A3EWtarIcePnEM4MmMS+/squbVOHjiZbpNDndz1CcTbshf2D/4/9knQVz3XM60CXMTSh7fZijGSUig3OP4ZqVLxEE7udcI+ps9/WoQnX6v6NMq/oOzwW8THGmxGG8WxgfIzZUxaFpBGsYlR7M44Fqmz72CfCETYDVEwJqEcJ6n2O0O585pUEa+LqDixzoPuO2s6axOCqCfslHzvfsex22WfLm9mNruIfGZ4NqaqxDJUEDKrNMYqHMU5635mQ9biYTtxREH0mXXU6dBZ454me07b64KN7bTB1w2kZVfkurqVHoXBRns59J3LXySWukaZN5GdyQj3yKHhJFxJBjGmrTuiQwNNzzo+fzOQ3COtIphsJ8bRAa5BjOEyhjQ8z3LwHc+1csgGBYiDl/dcurnzCrpGud/lnfwATNYlK8vN1nHG8GCY9Z8kYYEPlh5Jzcn0a85Z0/zvmHEYi/BRV+Pvhd9YWCvG0qV93xxz+SLsyOxugxXI036QbpR6Lro/7NTd/MkygLrokmWXJTaGNnq3dJWD7ssdXBBzZfFtgjr04SrA433U0IYXmhyIVKH+P1GscU28Kmb1uozbLZ6baxXwJTUA3D2epMb1baGH0PqZ+UR3B96cabHY5bB/3GIqwgac8FMaTIutCRA/ITYUpoRWdtg3Rw80Io6R8quSuQyYh1Sfl9C+0Sm9Q1+MmYnC5X14qNrxkumP/TuZS37wq7J6XiXvQNK7tsJgt/4gNhc9maYXRxGCSWM0K2cCRsBZzd7aI2g/+lFC99DKTtzgAXUzwMo5oYk8IBPk/tJV2XIbkupqpt86EHn7EYlZFdScXuKXExvE7HGu5Clvo+XwExo6FLR8C+DSmPHO3g2kiOKn6aEk5h0Z7MppLY5ctllNqNFrUDGVTnKqkJcFLz9ypb/lDc7M9u9qcKqAfX/H+7bYjFADmOhjI4OUFCiGMBGuilU8dFgwhsRHDuxfq5MQKH6OmgyZu3qrtzVBZZfdkrv7TdmOZ5gMuR+sAeUcL9enOSH9j3YvdEPyfoVJQHD99/TQUboyqJ9yAqs8eQFvRHcQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: f9pMC49EDpKuUz6+tlePZPIO1frqqTkUdfywS9wU1lWrmBON3JDEEAoqnXYLF2mYpxEd3kFoNRGv4Y5qUTXQzbl/Om+PEEk8jRRJUZpEBWf+5lADMtRNQQ3vwELvi4pWuk2o5hHWPp+YusTfwZI0Fzf20RT2SpDXU3gGefOCKKHB+OmabUrF/POIm7Tw5BdvEZcrJouD3gNGe2CZGneydkIJbMLs5E/eWFGM4GW5rJUBRA1dHUkdXHNsmez9QXFfUu4qBRcMFfrHJi8Kt+G1SgNp0cLIow5w9Ja4REQgiQz5sJv8sWSM54lQAGHlEr7b42kIdW+SvfX2CcDGNjkZ8h/I+WPOXyL4zIVWqpkH1ATKfVGc1RwkmMapBBgH0I+4yIxmZOWL8QuFwWxjhGfsywYQLWP2nAINHsOA/vk5ycWWlaHiR/XWnpQauvBKJxQlbbs/oPc/+awOGn2dsWc14UBJZPm9I2oOKkvY5LtqBrV43oP5Ls+tktmM086FMjat+Q75sNViJXemHkEOtXTqHeqds9eFUDTnTGhIldBUHO79EyKZ6dMmQNIt8+RIsd+iLoIkJOslBUPP6qMDjNEES8l08iJS9LRQnK5lAlhi5nQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e167990-1449-4efd-9094-08dc2c588034 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:20.0803 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: liU8PpXs/k9jGfapHxe5XQngIFJlxxM2N2ZP0o1Q7StfitM3p0QfG0GEH9roFzQgkEd142DF7Mpfkrf9hoZ++BIXTEc7cb8vkcNiBQEquCA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=969 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: -jvbt2DS2DVl02mf2QdxwccquCrtnsUh X-Proofpoint-ORIG-GUID: -jvbt2DS2DVl02mf2QdxwccquCrtnsUh X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762443750611864 X-GMAIL-MSGID: 1790762443750611864 With PREEMPT_RCU=n, cond_resched() provides urgently needed quiescent states for read-side critical sections via rcu_all_qs(). One reason why this was necessary: lacking preempt-count, the tick handler has no way of knowing whether it is executing in a read-side critical section or not. With PREEMPT_AUTO=y, there can be configurations with (PREEMPT_COUNT=y, PREEMPT_RCU=n). This means that cond_resched() is a stub which does not provide for quiescent states via rcu_all_qs(). So, use the availability of preempt_count() to report quiescent states in rcu_flavor_sched_clock_irq(). Suggested-by: Paul E. McKenney Signed-off-by: Ankur Arora --- kernel/rcu/tree_plugin.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 26c79246873a..9b72e9d2b6fe 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -963,13 +963,16 @@ static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp) */ static void rcu_flavor_sched_clock_irq(int user) { - if (user || rcu_is_cpu_rrupt_from_idle()) { + if (user || rcu_is_cpu_rrupt_from_idle() || + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && + !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)))) { /* * Get here if this CPU took its interrupt from user - * mode or from the idle loop, and if this is not a - * nested interrupt. In this case, the CPU is in - * a quiescent state, so note it. + * mode, from the idle loop without this being a nested + * interrupt, or while not holding a preempt count. + * In this case, the CPU is in a quiescent state, so note + * it. * * No memory barrier is required here because rcu_qs() * references only CPU-local variables that other CPUs From patchwork Tue Feb 13 05:55:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200232 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp351885dyb; Mon, 12 Feb 2024 22:11:08 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVkEJx9FSTVQRTAIVrRAfeoBVWTdlboypLsiozlLDIYhNXMH/NK36a7+TsysffhZJMFcr24KMW3HUByIL1HDVQTEEB+/Q== X-Google-Smtp-Source: AGHT+IGBacLcWSQcBbaEt2+uCff9HqQ+4+LOyiYaVLlcHd3NT5Jdfm8atlPvWbTf+jpVw06VioFi X-Received: by 2002:a17:90a:9311:b0:296:530:996e with SMTP id p17-20020a17090a931100b002960530996emr2348004pjo.20.1707804668745; Mon, 12 Feb 2024 22:11:08 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXYxNM49Iusxdzzkubq0/F+3u7vuDDZCsNYgWYpsWYS0mx8+sHkxLBpRYJYMMJWW+2Z+sZUUFW/hLnRlZnR9WFbgY4Bag== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id o27-20020a63921b000000b005dc42fd2ea5si1391101pgd.889.2024.02.12.22.11.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:11:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62976-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=KAzrDiPL; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=BMcJod57; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62976-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62976-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 742FBB26119 for ; Tue, 13 Feb 2024 06:02:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7EDFC36AE9; Tue, 13 Feb 2024 05:57:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KAzrDiPL"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="BMcJod57" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 028B8286B3 for ; Tue, 13 Feb 2024 05:57:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803841; cv=fail; b=P4C/81GbDGTj2zP+5HpZrzWjJnlTQP2ijgjNpWxnw/LcccZ6sx1MAyIqvfvBFQAmE9jtWhAjveOPEBE48lRL1hiGlgq/k6vVw+fkm90/zqp07JQvO0TzSmAFY+nOhfK99sk3rOE+CshTwFzJ97p/HBjVYqGB1oSlgZe8c2rXqfs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803841; c=relaxed/simple; bh=a7f2TJ02TLEo6PvKYzByepJ/+WZwJxpPpbEWeVWUJto=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=EwiAyOR+mc6bMaaZ9UvKfK2e03l0Kx1xlRZIZ1Gw8AKRsBltyvThItjKM/JOblSo1iBDzUMecaSwLyMn5ghnlt+COrLc2hPLAgahJMVUp3hOFDlBl/ewJDySTTa2lOsUQwjMHvz/fbvfFAAsIFlU6c3qxQmh6P014S+Z7Fz64ww= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KAzrDiPL; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=BMcJod57; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5U7DI004760; Tue, 13 Feb 2024 05:56:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=pP0wdiV3fIR9BWpzqWZ/WT1vd1nOf9518eTuA0ndXxQ=; b=KAzrDiPLH1ZruqvlqVqix+k3Ka9Gvn43KFsl+hHOc2vKdNKqVs5LU7x23JfGwvw305JU m57HD8NEw62dNOd9fyu4qTkhT07UHUGHpRj5592q5398uKHSExVf1mr9XxaH/NSoNq9P FlXBDtV4+wZ6EtEedAfo+eFeIaSfqzId9FazPQlyuU3K+c6h2jBzwBpAlDz/eDQo3HDc GBFYn3Tk9HVDYd851ZKXdFKPHfa2bcBgaApjTgYbi4Bys9Y/o1gBlNnOMghbmTENf65h hcvY5O+dJs3eYJn3z/u6k0GMEbTRdTQ8R97zqUQk4WRwGcGcea0aUq36YADLZfVmA3/G Dg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82atr1rj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:25 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3YlWH015085; Tue, 13 Feb 2024 05:56:24 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8jp-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C5nIbkRQNyU3ftg74LzZRIFFikCvIlguuA313o/nTP2tRc+kM3PSok0GHvIaeQfBkQR+vGmuGJOOMccemr/az7Th2++KMomAwoQ6P0kKLX8gODcy8VRfAagYfLvhnWzobKGjXERE2D+znea8DcL2QxWi2CUUQVWgd1bnMUqenmpssU8rq6vgL5dSSIKg+ZTtNLD083pdIqbGS14Y3lgIBPcul6ZxFw+vViHQxWImO0CtAuEbkzLw5LtGsTF3BJtacAdcFYH8p0eNgrdxFGlj9th0mT1uafdNnaqPyyJzktSf4bBK+f69cxXIJAA+YWAfSh4Apsvni0e/NK4wvbxg2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pP0wdiV3fIR9BWpzqWZ/WT1vd1nOf9518eTuA0ndXxQ=; b=RxaPa7c/ni9k718RuqzcGaN46sViMNI4SRTPJZB+aC0ze1q4szU5SMO8i1oXP6sKy0kY6GWFjMhZXg019KY6Mp4vWtJRUzaPj1Csf+Wmq9qeXMfJ+8cb75RGfcKMEoVblh0qkP62WlcjM2US8D3IgHuD3jiQDpjhur9emS5gF3MlZhPTbXxwmLRAkmDMpdP+6N6DiJR0Fn+AkFJTS8oxyoA94dRomjLP1hUQv7cnoOGcgQX1l4WGhZ/+mtM4Y6JC1nJ4Yy+67pAWQlQpTta5wFCpJzT+IFwZuY8EY/tfUKFmp0Q6IBbnAQGC3S6WKoJwAMZcIz/nMN0QzZleeWvpRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pP0wdiV3fIR9BWpzqWZ/WT1vd1nOf9518eTuA0ndXxQ=; b=BMcJod57cdDu6dX0fYiMVFxXB3kceWjMqOtZLTCCTO0bjQcjTjscCS7lvp/4296AdVeV2jNGr6IuEkLeZbY+NGH0scjj/0YPCSrPWZ6jfhfae9wNsWL5nzpJWIpTgpqqEMKZ0b71CJJse+/h4V7N77V44Fd1OgUO0EDyY4Y20L8= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:22 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:21 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 16/30] rcu: force context-switch for PREEMPT_RCU=n, PREEMPT_COUNT=y Date: Mon, 12 Feb 2024 21:55:40 -0800 Message-Id: <20240213055554.1802415-17-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0231.namprd03.prod.outlook.com (2603:10b6:303:b9::26) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 2ef417ca-7dab-499b-404f-08dc2c588132 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: V1ZJC/eCjNaZnnp64tXdvAf1ckz7Z6aCRfxm2pE8YmzRxXEPV33CumUgKRS9ihRjV7soYbScOp3PMT8WMs2MSDKvLLZUc3Qb3FPzEJyIwQjBjzr7Q0Ajavq/rYzuWvpEtY8hTw4HQ4DlDWjAVAH386U5Di4u6+ibC6pws/qwKrYU818i0wn9yXAKXJhGeXP24XOFBQCa1KTLayk8K6I4n6+Z9cXctEKPlC3IFBxXLpge/UrjlmZ+eis0CzkolrsvNrvkD14susGSKWbbRA5Z5WwQRECOqKIXaxSpPD40AXE4l/53/Zi1TGCZAKTeyfCRikv+oXPa39Jtis0sqGbYmywf5GHAqp1Ha0Do0wTT3hbbhc4XjbJDMk35uSBUuJVrsE+FeIns0R17j7Gf8M3w+6H90cWMG5sip14V9cI6AhH+epNMOhkdBUlNIqoV5G8Yh1b01oagX07y1wgnYGM5ZtAgKXWHPWz8ORazM7cblnjq/BppturFS9ukQYUs3X6X4HYAVSUBvQPbW/lBqXEmz8ugacZnC/OJZZ2F0l+1Mq7UJ2npw1Jeac9C8eyS7IBy X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: o32WEnGmqiNCucWx2jXVFtl2292qdfDEM73CT4EHxea9Iu8ksn+DUPFSMU+5kpn1zQycYfGf3ld++J3H/0TdDjlATB5czS8pK3Yb+lf1kJgqOITzHnusq9BHa2Mq3oX9chEW8TPB4joB3A9lkP1eitMqN7YonYyQO0rmvnAdLhnWijqsEACmM1mt0Sq2CBxdMo7b8A25Lm5qTgFOsplXUk+OxX2TQ70n3rBpCJcozN8JQaVAxV0H0utU6s96CjtX3drvkkwSEING/GYI6Ve9D9jFrjVtwvOFtPYtOJCvQeaSNSRxMPcPX1NIq+9LbVUE80ALkMO+GQkHJQBmoh0oxCVVElyCUtZMGXV8U11A6OTHOp15mkxeQZjT6DCWp92laGYY+HiWaR+UdE8dbGK1wmTktOgsM+WgejizzcT6XPKLwAjn2fcB7FqaQiw3Uzij98HLvQ+yFiVwrIEi1NKD9w+MXMwnwtgxg6ymzcnLnXx+wS9me+GOjHUaoRJ7tl1q9E8RY2wXtrjaZgaJiHD7jOvYnt4eEg+b0Jw3NZSqhUgxLceyR2YQFsDADilnj87VqGcv/4u60bDPIjjcvQdA0UDMDQUxs2X2cW2BQb+J+BGnrb6IVDepiPvsWxICsTtAxTNFq8S17DHzF9gAmay+YldyWrKvBHvcpj3H1qcfaiVjlBcqv+YTbJ8+3ec+VN+6T+lMC/CcCNOHACuHTppCwBWIBPhF8HNTejn2wD07yvCCH2qBzneMkYcDa+WKMfdkjKdrnN3A/dHjYSmDg1n8lQj34W2RsZsb10itE+4c0/aedoGWCxFUKbop6oJAy3YOd9jLbSTbkDVcom7wzyVpeJltSrb8D10diBmJzdSaABLYfiXrLKwbD3huVYu0PjYiQ6JN3AqESwLb9UbzAKa0wdf0YTXhQsvD0U/X4+PM449r1zw+9CEW7dkHJCJeDLZAuUZe/NZ3MizPR7vkowcx4O+TiLr2FVRJ6aLRjHy82l2KZeTj9J1tcZyzuy4+8CngzsHa/vSn5HY6k2KKAv9zP0hBs+pCDY1M+bZHrxSiwbBynVv3P9SzkJdO8a3RcFTlDeJQcwIcaI7T//tH+zDkKQiZqg4xr6LGNlnFA253P7SZAmceWPQMsKy7iky+K4rIniRyebGuNvQE5gaB3BMPkWZBh5sMLhwUqW8JjMM5pSFGemGE0xNBMwjcRTAJht9JQx0uX2za60YwMfBon4n8T2vaoIFKyekXHcQw8opaZbUtenxeTmLQUVuTtQX9OMXO+cVN5CAXtGhK3B2OcF5Iw4kzlwtzIn0yvYp+ruB27s2h8bpcCTa+d8Xh0owV0kXybrfmGWaU08wvGgTHbmS2Z3+ad9xEFw2qSJvI40PscFHnZT/7pMp5pCJGKZfctvd0Z7l3PJ4TX1BgNckjeXHdmxSt8EKLz+aKf0sgKA9lBGEjoW3qrRCuoMj/w7yVQEMGgerDX8zcf8bwaD7reeQNjTIcNEkENbXB6enbQyeTK3EwiqXymFSCeWg0skB0pPljspKov7rYB0kKM4dndvSkeElb7mw5hUmxZxh8PVv6D5yXm6Y3mDS0N/l67hEH1sXPGG6aG0nrWiTxMMhGjNCbZQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: CXbT5kgumPGx+0kZIUFd0cn4TA6/9PqgBkzLGhumucgzMKapmTAZIPofIW4WrAenFPB65ORA/vd/k3f1CjCFmdGJogTZiE4NsfxehjU9YX1KAKjS8adRDzHAzlCqru3+hRn1EauFq+zn16eEoMOh/onV89BcDD8ctXp0+INrQZUk6HzB44MYqqaw+qtvAC439P/2znjlbDpdmd3l3xj7L92NluVNbLZod5ERKFsGagc6R/hRRd8SbiLsCNw7glK2I1haaTVnvx4BneuhsCKUBp7kF/vSarG1HNGODSUAzSiDlz/Vc0ypCoBaYX9FTG3x2NtHKiyw0CrGGmd6gLI26vBpnrJzAisppiJ8T8uJKAhXfV3iIienJR/j1gZXLsyGvOeogKtCXv14ezYR6tDrvgaIBLHm5iuMmUZ92CaPu1q+0zxVI9ihP1U8hElC8jCDTbMtIq3F69LzfKZ7wa9p995BjzdiwQJ1r73PYXKBQAFrTWgFFaQCKjsChoKozs8IHigMWLCUWOSMWJHh5bP0FGe3G1KBtUi6elAob5rTejh6QIiZOjxoqps+7OtfTXTdM7ZR/0ksp9pBawrdwnGogMdasVnZHiAW39scCORl4tc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2ef417ca-7dab-499b-404f-08dc2c588132 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:21.7522 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /AanC6tZ1iM/77zGaUMrVMvaQJ1gFNNEaGTQ7l2ilqajE4LKLYNvsq8kGqk847V6AKCUxrWMRNl7cWV/MiKpq3I4WuOaOeBETXiQvoIAMXk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=869 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: OOuDlOiIhYaTyzydUUHdPAsdvzDIVCcl X-Proofpoint-ORIG-GUID: OOuDlOiIhYaTyzydUUHdPAsdvzDIVCcl X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762988247595297 X-GMAIL-MSGID: 1790762988247595297 With (PREEMPT_RCU=n, PREEMPT_COUNT=y), rcu_flavor_sched_clock_irq() registers urgently needed quiescent states when preempt_count() is available and no task or softirq is in a non-preemptible section. This, however, does nothing for long running loops where preemption is only temporarily enabled, since the tick is unlikely to neatly fall in the preemptible() section. Handle that by forcing a context-switch when we require a quiescent state urgently but are holding a preempt_count(). Cc: Paul E. McKenney Signed-off-by: Ankur Arora --- kernel/rcu/tree.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index d6ac2b703a6d..5f61e7e0f16c 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2248,8 +2248,17 @@ void rcu_sched_clock_irq(int user) raw_cpu_inc(rcu_data.ticks_this_gp); /* The load-acquire pairs with the store-release setting to true. */ if (smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs))) { - /* Idle and userspace execution already are quiescent states. */ - if (!rcu_is_cpu_rrupt_from_idle() && !user) { + /* + * Idle and userspace execution already are quiescent states. + * If, however, we came here from a nested interrupt in the + * kernel, or if we have PREEMPT_RCU=n but are holding a + * preempt_count() (say, with CONFIG_PREEMPT_AUTO=y), then + * force a context switch. + */ + if ((!rcu_is_cpu_rrupt_from_idle() && !user) || + ((!IS_ENABLED(CONFIG_PREEMPT_RCU) && + IS_ENABLED(CONFIG_PREEMPT_COUNT)) && + (preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)))) { set_tsk_need_resched(current, NR_now); set_preempt_need_resched(); } From patchwork Tue Feb 13 05:55:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200231 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp351879dyb; Mon, 12 Feb 2024 22:11:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IEojVpYvtTZsNQguPib3OA4Cs1HWLWA53dJDnMR27qwih5NWnGo8Iuw2Pn7eBK5NMwCiDpZ X-Received: by 2002:a17:90a:498a:b0:298:b24c:4f79 with SMTP id d10-20020a17090a498a00b00298b24c4f79mr325212pjh.40.1707804667774; Mon, 12 Feb 2024 22:11:07 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV7OfVfrf9ndeZZOKYOmCbAe/DUppv1v32TO5F57q2ZW6irZWEJdkNv++dVlF01NWvP5Qav7rYpmiSmSfwkmm5m2R4Jdw== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id l29-20020a63701d000000b005dc4a7c2325si1456716pgc.824.2024.02.12.22.11.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:11:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62975-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=LKEs+w7S; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=PuyMM9V2; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62975-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62975-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id CCD01B26F26 for ; Tue, 13 Feb 2024 06:02:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7505C36AE4; Tue, 13 Feb 2024 05:57:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="LKEs+w7S"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="PuyMM9V2" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6647286A6 for ; Tue, 13 Feb 2024 05:57:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803841; cv=fail; b=owUB05VJerq2JSJfiFC2/S0+HwaJopEtn1ZZKoXiwmAORyJsReB+GHqkjNZ/Ahj4IyfP4EPFSkRiZWHjzMng2mSoO8+l+8BxghIY+z4AXV7yiV+JXWE4W+X7Wmg65vcDJgi+v4EDfGhD+LtQghNoTwNfrVLGt3Rk5Bi9qQaUExg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803841; c=relaxed/simple; bh=6TlW/dK4PJiKA1H+OKwfpAOdooAvrFke8sAXfycZwxI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=mdh8hiB6DNnRCyQCRugEmN6ZuDsur33VrMshueHDoIZnoQs6AWMsBrvpOUfqAfywSs4e6flLfQP1ud+SKCfxBFyYQeK3jXf8OagEiA/9AaM/sQFg6/sc4rljcAimT/I/bUb1HvlLkk54HdE7bd4CEzr8k6bBafQpUqXI3x5Xif8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=LKEs+w7S; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=PuyMM9V2; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5U8md004768; Tue, 13 Feb 2024 05:56:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=ghLuSp8KTJpodZ6jrwIUcvXLLNeYMHBmB+G7KWE+Mng=; b=LKEs+w7S3Un8LpIDv+XF5d+0/cKBUfODoD5Rm1VBmFV9cVVZhtPb531O9I0X1iqXvpWd WzjJf7Q93JIMQ7/ryO7OsuioBw2s+uIStfMHOaD6GFXOkDSSx3d99xnRCYf4JKKQcMCG kZIxhxQrsPIj3XFNr6qc5d/9+jUqPjaQGRX3A11b0C97D3rNTlt43US0sHCLl7Hq036Y e0hXcrhs4BF0lX9+FMUfLeIzSDKCYMHmxxMtZ3paUsLpLINYEoBXTNAvdxyODlzIFgsL /XZgMXiq0SA/vsPdbdmvPHWYnw4jxKSgXxme1NW0F8rIIwPVO5qyMuHl18TUVKX4CaOu pw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82atr1rm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:28 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3TU9l015098; Tue, 13 Feb 2024 05:56:27 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6ugpy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZK9z6yzm0cgBECRMalG1PXFegC8ZaUu2aLMwAtYKoPQ4ewJBlTpKITbN8CGdkMPRhFPMYCnJABdWG7yYvf0BR+FmLLvOHJzef30I6spEPOXab5VqiTYvbYYcyA1lXAhcTpLqSGor6weTyRvk2WwLJtLFrh5PZgZQhT137st9u7qlnwnhDlg9K4z6uYsFX0EfVbY9zZIicXoU2GEcYHl+uSrjuwZSIRwtzVAoJ95mEpEvxFha/FOCOUOCK5rE+x5wLwQZ3sncBdUll4Lv7lQ9nS43N6BZTkAoBpDObKT9CQ7MKuBmJxBx5hGVrpqoAep1P4Lhm4LmbF3niKz7k7dUyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ghLuSp8KTJpodZ6jrwIUcvXLLNeYMHBmB+G7KWE+Mng=; b=h9pO3C94RKxWMVTRcYnhwgtNnyJ+N/A4O88Aydyb9gkTdtb07kTF4Qibk870FxR6lSu/Ugfcza4HbNkjKHHT5cLUC1I1tkTFWjJ78BRVrPNNcj7MGFoFXD5V5HHDgHRfVbym+dELWe6QJwLmN/eTWGulFsrUK3inP+J5L9NK4YTGMZSo+5ucvU3+AS7UrU1PAUj9T/4mxekgNqWHCxeUQW61up37wYPwdnysmq2/d0ciCENcHNExZjqN7sjllmua3kM+A4/BsS+PH9MBmAUHqa7tHZ0YHcZ5M+Xf2oLiZaxuH5VPRR2ghQkkhcW3SNox7i5HZFgigV1o9Zb6RpCJwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ghLuSp8KTJpodZ6jrwIUcvXLLNeYMHBmB+G7KWE+Mng=; b=PuyMM9V2H1Oe5QugFc6hbxKcn6U2JVPsoB6agMgrHyOtL1CCMs+F+gmx7/DYDDWxOyRy9uCVjbFY1HxXnafBFOS3PcG5MkTM7ty3u0yBuHFXM525z1oqw4J3G6ab8W6NuUGIbb0wyEV1gYYyeTsgmf2Ss97UtWY5XV5yDS3fz6Y= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:23 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:23 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 17/30] x86/thread_info: define TIF_NEED_RESCHED_LAZY Date: Mon, 12 Feb 2024 21:55:41 -0800 Message-Id: <20240213055554.1802415-18-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0016.namprd03.prod.outlook.com (2603:10b6:303:8f::21) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 84ca4265-a3ba-4535-934e-08dc2c588276 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1jXoN8w+7Q+282Y7dJZ8DhYJnQea6stuM6zxHvc5+PQUkhxw90pZMW5hCpGrf6m3XDRAOuEe64nMJiUdecJ579+fA37yHlTpZki1R/gyiEr0xyn63sFxyh6KjZKfeX0yGtUbBw8l2SCqDBvi4CEPzqV2S0aQjVrp+vJrpduPO9K2Y8TNgNzFEDvPg8FRNBnq90fFxNe4R9nQCwEMyvVDZvMWw1mioQ+CUUkPFslQQoaFOy+csORe/e3O8xM5p3bKg8dow6eRo5qych2UbEOf4hYu2s2uZKp9gURPVjnZQPGMFJa/+Y6pdNgagkStbavpWH25zFOfJTXD7d9Mv0C8uwSVDlhjIRjYDwAjeCjuj/FdEXdIRgOv4qlUrObecgO2UpPEFFLyfblcofHreGbg3TybuK1KiV6T9Oz2ePmrrxsNnXXV+IHbDZkUuchag8oHcQriZO7zI9isAFv4wm2ifLMdUPtjZedjaY1bWy+276DX9Hd+juQC8Sy9u2P9fZTtMFQ/5jV7F5nISxuzi8Ge6rhK3Uj9Y5TLMarWnLg1k3c= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ypxaAEKBu4xNkE/aQkIRMZK7hvuQUJpwU1ASZyO8y9dSjyCQUWTLpxm3EPaGJvoVORsC7u4Dq6t9KhZKCft2C0w56BRO0qCKmQrHtOF/l5fbIhHvvqYA7NkZkJUffzuhVCJ3kuHi0Fsjj9hSEBI8E3ac/l8XXWeb33d3lgSagb1b+yR6597UHgGTS+baBPVWP3z6zcAiEPNuQgDZSKwyqLAPMmkA2RcX58XZV6SIMjq7B8guOw0bdyH+XUaNyyzIQ2LEYZ9e2fAD82Da7ShlvqFVDboEYcL230tAWTAlj6D54McblT0BXU8vJKPY+hiy6lqgwsqUT2xggjoy83WxLthc3QfRim8wIf8QN+3dI4paYyxrby7O4GMaPGnORyhegSRVU7ys/6vm0cl1fKU+tMImoUBci8lEdmiqv7UL8ifvaZQMFidWKInfqRn+MYaVI94jS5zFmSJ49jsZtEZNlJCPgAworWhqHSsGLYrj4CXhEu1JiuoYgGE5v/C4/pds8KZ0WCmiQIWiQ8dPGOGYgC2Ml+DK4fkPY2/xfR3dfsJcCQPVh6x0oq2G1kEFUkfc1OdsLp7UCGGIx2ujsnsZuFEb/1SJFQaQbwAK6fMQSsHwWD2mwHkbK4xmAO0bpwoPau5jc72/0yYOQDFuJ7K4ZjD65ywwXjTDNLPOTKeLqgdSUciPLOp0MliJIL/DkNp6CFg3QNlJ89vRa/52cJi+EdkjDHV8yJ9MHZn8uD3Uk4/hQ9jE0c9U7yMtIOni5UwmvD39d0u7hgemOu2cH01VEN+K5UTUAcaBdUIBQiMFWiecr54NmDWfbFfgB8Fo4slW7zOka0dA4CoMCuMQWqjq2y4po8nuAI2fg6HNLyIbxZgVl0noY4LJ4NDGB40OYPO/Kp1ukJ5HYGWbmUjMFlyorazW7WI4OWoEBAKsC4ykJmOSdGwe452iBWx0tJNjyx8dlQ28nAtt6qIsZ8fC9HOB4g4qj1LvmhBrUYyURNPM1XiE6QS45fdep9rGQ/L3lw6IuQ4hbrELyyJXUQC9wc6SXC+hokrFUwc/sqyjECZW0vkSOTBrH75K7XxkdHkGWI70yCsSYTnwqIfcIXxvcK9+hFKLsxyJDx8euVEVwiRL2WWKg85pyf6YfCGeBa9KKMIwicElNN3a0bfTGJd4zG+ncfwiMArUlFj36X1GvzJ+ZV5Og9CxBKj1sVpFQv09EW0gvqk2M3teN11s3dIk78Pprh1FwylIJxZDMOdDsErAB3oCf7shjAXuN53Ol3Y0bA2UEuVAsTifR2/alENQMzV8iZr3kPJs6AGM9sSgg6Wb7NOKQtOaN5h/P79zG7Q+S3vhSM0hCyc88ZEAM0JrwRoNbOz7e2MCU0H0FYuo1fnraSdBZ4vAVn7SbTscfFofYP+g+BdnVWUwK2u6FXyFaDne8F/9XjTf9mBh+yBUBSSINa6JBL3B3qWYInSsTNmG/Enw7lcl+LJ9YHN4tRIOkouxdwP/wLh0RQWHztiYzYt4Zue/1oAC9G+r9CIuRvZV1QJQ+IpntIy3qRqGPUEGmyG0ViSCbvHACwaarAdvvFEF5ZGlj4yPSmBUPnIMJYoJjkx2pRAhtLtA6AvDUEGNVNhvCg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: CtjKQKJTsInfw3AsMvSfCPj7NsdHqxQoSfrlwEAlv+BHiLlkEzRo39sriomM44SGL/qfYMhz5Mxdl3QjjK+x1i86R0Tzd6dodDwrGqwuSV59zCbKfy1q4N4NUOpvRuMGywysOuJX/2jL01p0PWoqCjty1LCyh08ntaZaFLNv+3ZY9g0F5U8kt3rv3k9VLVb3/rBrm7Exahg+gamzheTEKxZHMoMF2BieNBfQ2KB92l6L3fhmUKuxvaXlMqN6SH/7rEt+R4BsPUg3lOsslwrwKf4EQdfvzbY+97BnjiYePKm2eLEyXAotylJfmVQ8y8lvq49R5tp7pw9BL5DbSVvLUYyh0ZJaxH5I1x1e1uzxJIYQLOSEmslRdTUZvF0N+KOm8qv0gY3qoiByYQeim45TkYFRX63DO1a5LoUhVmO3jYyCmrTaj7PV7fhV77lI9ug6zJWKk3/8uxxyHNN0sWeCbxJKze+k5Zsea4eabuzvjpArysfXBiB6xVYWUT7oAMyXH6TetB5nhz6x44VYg3tlh4BMxv1VlMN5/k6SlWk5PkplNR08eBcN2V7jCH++prmTwnjhQv23nQ0qbAkNhaKB2+FW4cDJmdayJ4ixRHOKre4= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84ca4265-a3ba-4535-934e-08dc2c588276 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:23.8659 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SvuYWSOliyLjRxN19mxmEPnZxSS83Xawkx80trG1dtaWCnjC4GC0+eHWum8gWu5+OnYX5lj5olPbuE+Q3oGqZzJJEUn5K+azzdGsD9M27Nc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=0 phishscore=0 adultscore=0 spamscore=0 mlxlogscore=740 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: rHbhPWj06GGaTjDbAaENr7AO63uuHNow X-Proofpoint-ORIG-GUID: rHbhPWj06GGaTjDbAaENr7AO63uuHNow X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762986984667318 X-GMAIL-MSGID: 1790762986984667318 Define TIF_NEED_RESCHED_LAZY which, with TIF_NEED_RESCHED provides the scheduler with two kinds of rescheduling intent: TIF_NEED_RESCHED, for the usual rescheduling at the next safe preemption point; TIF_NEED_RESCHED_LAZY expressing an intent to reschedule at some time in the future while allowing the current task to run to completion. Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- arch/x86/Kconfig | 1 + arch/x86/include/asm/thread_info.h | 10 ++++++++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5edec175b9bf..ab58558068a4 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -275,6 +275,7 @@ config X86 select HAVE_STATIC_CALL select HAVE_STATIC_CALL_INLINE if HAVE_OBJTOOL select HAVE_PREEMPT_DYNAMIC_CALL + select HAVE_PREEMPT_AUTO select HAVE_RSEQ select HAVE_RUST if X86_64 select HAVE_SYSCALL_TRACEPOINTS diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index d63b02940747..88c1802185fc 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -81,8 +81,11 @@ struct thread_info { #define TIF_NOTIFY_RESUME 1 /* callback before returning to user */ #define TIF_SIGPENDING 2 /* signal pending */ #define TIF_NEED_RESCHED 3 /* rescheduling necessary */ -#define TIF_SINGLESTEP 4 /* reenable singlestep on user return*/ -#define TIF_SSBD 5 /* Speculative store bypass disable */ +#ifdef CONFIG_PREEMPT_AUTO +#define TIF_NEED_RESCHED_LAZY 4 /* Lazy rescheduling */ +#endif +#define TIF_SINGLESTEP 5 /* reenable singlestep on user return*/ +#define TIF_SSBD 6 /* Speculative store bypass disable */ #define TIF_SPEC_IB 9 /* Indirect branch speculation mitigation */ #define TIF_SPEC_L1D_FLUSH 10 /* Flush L1D on mm switches (processes) */ #define TIF_USER_RETURN_NOTIFY 11 /* notify kernel of userspace return */ @@ -104,6 +107,9 @@ struct thread_info { #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) +#ifdef CONFIG_PREEMPT_AUTO +#define _TIF_NEED_RESCHED_LAZY (1 << TIF_NEED_RESCHED_LAZY) +#endif #define _TIF_SINGLESTEP (1 << TIF_SINGLESTEP) #define _TIF_SSBD (1 << TIF_SSBD) #define _TIF_SPEC_IB (1 << TIF_SPEC_IB) From patchwork Tue Feb 13 05:55:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200219 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp348892dyb; Mon, 12 Feb 2024 22:02:47 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV2r2LDH2+iGnTJz24s9QE3AbZVe/92eFCwwid7ESuPLuFQEONwVvulsTBKH5uq7EneQzRcXtTJVb/ZQaZrh0Gba8JlTw== X-Google-Smtp-Source: AGHT+IEiL2Foh4jlpBuC+W5i897idmTfHScqfqp7JEtlwtArgIMvcDr1DVovvZEAlhtQt0lZ0qKQ X-Received: by 2002:a2e:b8c7:0:b0:2d0:e35a:3ba5 with SMTP id s7-20020a2eb8c7000000b002d0e35a3ba5mr7398556ljp.46.1707804167365; Mon, 12 Feb 2024 22:02:47 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV4EkhzV9SBhZZYwIXQquxR2BIlgHkwQxEWVHYlJcELtRJUtTiiyl6Wvwr397H050PNAWGugiZ6Z7udmyO9LNlpcr2nfg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id a20-20020a05640213d400b00561cc26deaesi1224899edx.304.2024.02.12.22.02.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:02:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62977-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=BIdPjjH+; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=HiN2KVxx; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62977-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62977-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id C691B1F22B30 for ; Tue, 13 Feb 2024 06:02:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 007AF36B17; Tue, 13 Feb 2024 05:57:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="BIdPjjH+"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="HiN2KVxx" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2200A286BF for ; Tue, 13 Feb 2024 05:57:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803841; cv=fail; b=cuU05bPH6SnHZBLi81ugH7N1Z+5+EbI+DOBn5GZquzt9dwudP1i+nf0zBsElAIQ3Wq/zr7M6tFNXgaYJwdJEqKVXwCKzs3l5RoH76TZBAd3JCnQdu/1nG7gW/Cu89vE+rlkFm4voAi5mVix81XU52bYdN8k9mMxpRmpVI1aAE0E= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803841; c=relaxed/simple; bh=e7K4s8NV1Z3WsjSixol3ct7UiyOrjWtYHg5qnvnaGdA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=Cl+33tTVnXp1V2Zh4WMrJWps79AG38xCeiS+8OFUwZOkFe9ZyW9kz+xHy8D1LDuuLWzAAGZvNFEn+seBGQfK5W1t87U/K26l8oRWYRXwxc0t1inlPHVEbKRA521GDBp9u03re5xjfA0WJL8rArdLu92MDgzvtzE8m9GensUkWT4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=BIdPjjH+; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=HiN2KVxx; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5TnRS003524; Tue, 13 Feb 2024 05:56:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=gMxzyHoRAAFPam1OfV6WHYNcwGnxqQvmOgjxa1LeBwI=; b=BIdPjjH+7RpPj/h0OJXhpg+FxRkskdnrOqBy2UBuXH0d/Lf+VToFdpBbT8Z3HpkY1GKJ 015EZxUtDNUT11CD92G0opNql9rl5v3CU2zq9KJqxjynAo1SIMyKyApbbigGG3CZklhh EmzihM7bKxKC9ckiC2/UdnNN53A619FbY/9Cr6cnF2eeParThSBil0SFEXZvHKmOnqfc Xq+j845elcyaBRTsa4jGvTpFcoSRWOu2IdCzn8qM/4r6ul1irB940sxwEIMKOhHfCBol V0jPmOcIrrLYYxKquKMb7djl40iBIDJK1gtVXXjiaVQkyXlgPFP0cQI+DKG3kh/WwcfO dw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82atr1rr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:32 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3VCaG024670; Tue, 13 Feb 2024 05:56:31 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3ta7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c0Oxl/dULl99a80+zDn54rpb42tEE9ESpiwBW5S+t5EIjRI4EHHvYngLjDZcW0zkJ/3PY9hPFJy4R3+mGWe5JlzvdODWTrWtet84Ydhuto1uEmN/CMQYZiSGrvyyAGRbuM9WSDq3oVAHBn2+VuDoRy261NbmS0B9Uqp6BYBY2TCavSL6XwjEiOODUVwZ5CXCsbozwq9d2wu8YNr8iAu+9fDkpYLK3noUEcVHcyfUMKaqHh3WkBwqXeNm4sfq7tnc9DUbVxaYXTLyslZ3Qs8a0YdifSQ8hB1ihm0qeGScs3ZzQXorUaAA8hnOcyL/MApHzte/2PI55mMiSurBE+s4pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gMxzyHoRAAFPam1OfV6WHYNcwGnxqQvmOgjxa1LeBwI=; b=SK5F44kvV7bVsY0f/udTuxDzes6c6lHCQgY+s/Z7gk9Op+S/MsdUEVxGvPakpdvU2aqu3rakyLanYc7cz9e9nmB6taB/I/BV5Hb0QAkKBT4Wn8nRYXVowhh/DbtC6IO1CE5r1gO+TyDR6fLT4RqS4uQqYIQ/Izf+LcRsrYNWgRlE90DdGdSCfruY25jGmOZu33n9GgRjEUue3FjnDgyShSNqxHl29eq3WhncJx9YKb7963MKEcOT6zse/VBUBdbL3Yy2boM/5IcaxSzjwWDU82WMGGMEQYDth54jWpJ98pfzNXEJk8Okks97ENxXcDDaIp4nFGLIBp+ZOPPULfxvIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gMxzyHoRAAFPam1OfV6WHYNcwGnxqQvmOgjxa1LeBwI=; b=HiN2KVxx+A8Se+1l1+pdq65LxZFMtLUyuq2c6UaZ+g4W2z7biNsq/bCgYtkDV5PQco6YwVAuiHHUm874hQYxxEh8lGDoaDkUTVdtYPL+5cR/maMEKbPumjkXEZiJOp+HIy3s77yLmSuTBENCfofH9+zyq/NYclVFpzNDaVzYh1I= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:28 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:28 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 18/30] sched: prepare for lazy rescheduling in resched_curr() Date: Mon, 12 Feb 2024 21:55:42 -0800 Message-Id: <20240213055554.1802415-19-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0041.namprd03.prod.outlook.com (2603:10b6:303:8e::16) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 4d28e10f-5a89-40f4-3e55-08dc2c588545 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wL0dyXkrCY0mnw3Ap1ZdwN8ygtBYarv5SdVAHvYTT4eqQLAPJW4dZmEJxGDJfsZogcC1TDdyNLd1YveUagGBBAkqvqi1H9tf+PY6cj9qKwsl1eo+MB0QHKMmUZxf3noWtYquuG6MtO+zx9/5zpEuy9YtmZXaRFQurb6rkR/vJHOPNzLGGOWmyhERCFHmYq9VrBN4Z81xFgOyfsu4yaeuGiaLs0cmxflc4bXQhwWl5NB6T2B8A1oy4ljYZRMGI6tnWNf/2pYT08vCwkfJp1H67riNe4bjdj6AOcOoFPk9iAOTJAOo4HZTSOAG/T6DiKknfDooubqV+ux/y2M8MU4tvliXenDBxIzugEIyUuq46VR4o7WB1Rv3U9oBU+hA9Uk8wkPdpNddMYg+9nM0AuRCevV4NJ6JMOSNs9AlSfI7s7oH46Nwq5zZRutNXwxhZ0425IkrFIMpZrqLNONPzf+8HpH4m3vwl1v7vcBToXs7TQ1KSDovohuuj4O5cVwf5Kiwa/NVE7yd+LvXrDQbefHbNneXrgfY9ymhhCM6ox34oHU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FE+mOfg0I2EiujKadPkY6tXFf/j+qpNm1JREOYyly30G0Ijq4GEGCBHUvh47BHKZus6oTI3c/MDeTcgXXztSLYWbs+7okXyyjIz77a2U23IzrRQ7Xvi7wJnUc0WrrLHI49Shu1GSGzeyhqdQTRztBTacAOuAJOQgieiJz6pL7fQj1HaZMBmsxc6gVD0VYDJq0UEWkoNRCn2Waetg0iVsghsaKawfw5YaRemwtECa9p2ysl6XIoW2P35e4sz/EWE3Cwr4oPk8Hgs6S/nD62kw9E82F6AcHSftbbBVS/qAJg10ireEzGUyH880rlRYroGICx1WARTgBLgtZHgJD6TthTk5jeMKOZQKpzmuMDhCXeuYZpfLz4ls5veE+eTYtRnKhR1yCSQZU4c5pf2tN4qaSj3BMbgZ4vAo3hpt8En4caNYRQlQujK665uQYTNdI7mL6KFah3rDIVK1gCni0FKjrVMEVgnUDdslWaSHUY/e9LRtFHF3k3H2IHGpG5ZX1W/3kCD90qB8Cle9sCpj09brkp7c1Rdny/xb/9tuQNLowkZ+pjS3i3WfKVidebYWK+VDGXHWfiOYsChTQCC5ehb4GwEIAcdMQVsEKqLoPgO6wxD8HbBHc1B1Ya+zDvi397q4F31WOZvgU9t+e1kEhhBj1pszZ5J9h5qocv56WwOSgY75zoTgY3NHw/A8K8N+WMO+e7s4EH2XUMZ/vf7lNpH14Li2xLjx0prlQD8d6cWKnQYrA/+jEUFkJHjnS+ly0W+NiMTXUI/g6Sex4PDx1iUvCE3qS3a9R1LzdrJzkQHiknP52jmyHEmACXlsbuGr7sKDVf+xmR6npHd7Orge/ZhkhC91KLjhQQYTAycmJVSNTI74O69hSw88e9czzdKfYZvmMHmX3uVysxSd8du1u/wbXAyAjFe/1L+tHnQ3EOO857iSHlEhxUKSCEGyOVTNsZVzV+C8khxlTdZuWBcFcvboTWBQnH2x/mMX2edYWPzqJ3HvgR59i3mDttHPjYYOuykbn1DqBoNSSdWx4r7mN8y2FdC9r6CVA52o8v3KT8xI64fMQYJIp7FpLuIkoYjWSsZPYB2T/wzXHVD/dEYj8ZIR5R8zQiZONQY0JK5+Waj53pGtxfdu03mH1N64NO2cTDJH2FGkxIrubktaTjMLyGN8xwszrkdpZh0zkqzsJGIgkmsCVqqVqckpgvhA7U3UuFUFKtbH0l3GQad1VMwfISjQnp1Kz58wZeiladZ94tQeObfRf468u/0Cv0k1kvSxOKZ4c3dw8CB7EOMRZ9o0hPHvedgFLTUwcqj1VqrQwZBnfX2hfrCucZ33NmfJ2Q4igXLaJUgMdALZy3t/7s0Hf7Sy5lRfDPNncHhOh1B5Zo2IfA1A/fgZYuVGWdoEOIyIgtsuWdvMnAOTHwb1ebb8+TEgMIRShflzgSO/AnMu/vQ5JGxmG2nXjF5R8FrP+hs14oV3aY67k7Z7ODMtCMNhSP+RdDp1rHGWADInsD7u1hpxKLA0p9yQYlY4UKexzrbeNTKtR6A1DmV4t5atd8xQmEa0ZivqS164bwF3SOhegYVj6JnbywvLUJASRaxgNBQYU1Y2Q9gITrYCkzdK9FnWAK25Qg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: yTc1hO7N6cX5CxhAJ9054Oy83zDqEWVuPk9DllPp8P7dzEg/gld8KwOJAVR8n9s06L6cvOYTTXQPs2IP3KjLkjbCEknho3366dbiYaiXpQnIvhJ3MWY6RkD2IAZEFZ3my+gUuJRJDiSV7xvP0c6ajn+e3qgcid8W7wCHVAsM30xNmgHTz80BsrhWwikdz3YVhtjxv/qkc+D/k0NHF50dexoZnHiNKzwaXMXDroNmTIV+Qwte+/AJBCwUBdhwjWusCwsY+Omv22RHRodnPgONmvICyx2TiVagzYcAphLZ7wVMSwVnVsTrqvB75RkSkQt9NNez0xAVBBIjywizCdV7X1H2+YHG/WdAXB8+0zca91OEA16gCUBoLM0Te+COLCJ6CW9f8TE/SUzFfyHmMhJxgYvCAp/NPqgYEj/g7t4dTmKltY5iKqIiTUjgYmVGgEUnjuT5O1YR7rnLh/+2DF50mdNj99iTmgtjsuwXQMNMCm9zir0sZNYT7zU57iVoUazk9azVqNhYFt5Olf6BR43IiRaZ9FMmZcAj50e/86imspw1CNF8gE7nfH4+k/WzX29dh4DffUpnPEHdi6WH+R7dMXcivjysJUrGXEI4l8drkCg= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d28e10f-5a89-40f4-3e55-08dc2c588545 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:28.5715 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7qy/k51NDLiOm54X5u8gXajwqxZ788YX/rHCrZZXjdyDvop+EuyYQz4M+Bich3P93sGuhI5JySwmVpIB1VINJyOMreI9QJKeUNTX1mBb84I= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=837 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: jueS3Y7whRVoqIPPCuyzEXrYZkYfmTaJ X-Proofpoint-ORIG-GUID: jueS3Y7whRVoqIPPCuyzEXrYZkYfmTaJ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762462445947026 X-GMAIL-MSGID: 1790762462445947026 Handle NR_lazy in resched_curr(), by registering an intent to reschedule at exit-to-user. Given that the rescheduling is not imminent, skip the preempt folding and the resched IPI. Also, update set_nr_and_not_polling() to handle NR_lazy. Note that there are no changes to set_nr_if_polling(), since lazy rescheduling is not meaningful for idle. And finally, now that there are two need-resched bits, enforce a priority order while setting them. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 41 +++++++++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 425e4f03e0af..7248d1dbdc14 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -899,14 +899,14 @@ static inline void hrtick_rq_init(struct rq *rq) #if defined(CONFIG_SMP) && defined(TIF_POLLING_NRFLAG) /* - * Atomically set TIF_NEED_RESCHED and test for TIF_POLLING_NRFLAG, + * Atomically set TIF_NEED_RESCHED[_LAZY] and test for TIF_POLLING_NRFLAG, * this avoids any races wrt polling state changes and thereby avoids * spurious IPIs. */ -static inline bool set_nr_and_not_polling(struct task_struct *p) +static inline bool set_nr_and_not_polling(struct task_struct *p, resched_t rs) { struct thread_info *ti = task_thread_info(p); - return !(fetch_or(&ti->flags, _TIF_NEED_RESCHED) & _TIF_POLLING_NRFLAG); + return !(fetch_or(&ti->flags, _tif_resched(rs)) & _TIF_POLLING_NRFLAG); } /* @@ -931,9 +931,9 @@ static bool set_nr_if_polling(struct task_struct *p) } #else -static inline bool set_nr_and_not_polling(struct task_struct *p) +static inline bool set_nr_and_not_polling(struct task_struct *p, resched_t rs) { - set_tsk_need_resched(p, NR_now); + set_tsk_need_resched(p, rs); return true; } @@ -1041,25 +1041,40 @@ void wake_up_q(struct wake_q_head *head) void resched_curr(struct rq *rq) { struct task_struct *curr = rq->curr; + resched_t rs = NR_now; int cpu; lockdep_assert_rq_held(rq); - if (test_tsk_need_resched(curr, NR_now)) + /* + * TIF_NEED_RESCHED is the higher priority bit, so if it is already + * set, nothing more to be done. So, the only combinations we want to + * let in are: + * + * - . + (NR_now | NR_lazy) + * - NR_lazy + NR_now + * + * In the second case both flags would be set simultaneously. + */ + if (test_tsk_need_resched(curr, NR_now) || + (rs == NR_lazy && test_tsk_need_resched(curr, NR_lazy))) return; cpu = cpu_of(rq); if (cpu == smp_processor_id()) { - set_tsk_need_resched(curr, NR_now); - set_preempt_need_resched(); + set_tsk_need_resched(curr, rs); + if (rs == NR_now) + set_preempt_need_resched(); return; } - if (set_nr_and_not_polling(curr)) - smp_send_reschedule(cpu); - else + if (set_nr_and_not_polling(curr, rs)) { + if (rs == NR_now) + smp_send_reschedule(cpu); + } else { trace_sched_wake_idle_without_ipi(cpu); + } } void resched_cpu(int cpu) @@ -1154,7 +1169,7 @@ static void wake_up_idle_cpu(int cpu) * and testing of the above solutions didn't appear to report * much benefits. */ - if (set_nr_and_not_polling(rq->idle)) + if (set_nr_and_not_polling(rq->idle, NR_now)) smp_send_reschedule(cpu); else trace_sched_wake_idle_without_ipi(cpu); @@ -6693,6 +6708,8 @@ static void __sched notrace __schedule(unsigned int sched_mode) } next = pick_next_task(rq, prev, &rf); + + /* Clear both TIF_NEED_RESCHED, TIF_NEED_RESCHED_LAZY */ clear_tsk_need_resched(prev); clear_preempt_need_resched(); #ifdef CONFIG_SCHED_DEBUG From patchwork Tue Feb 13 05:55:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200216 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp348524dyb; Mon, 12 Feb 2024 22:01:57 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU7Lgx/fYS6hlnG6Jjti7hk5/mQhdv0uQ2leNpjFPGKBnwoMyhn7v54wZ0Tkn6Pnncj9+fu59gTOtaZMdZuKqVXqpvBJQ== X-Google-Smtp-Source: AGHT+IFV8fiAoJ5mq4tKbsg5oQiW7q081MIHg2xcWMMfdgH4T148ipXINvg26GV+xXwt4Bbhr81T X-Received: by 2002:a05:6a21:9214:b0:19e:5dda:6bd with SMTP id tl20-20020a056a21921400b0019e5dda06bdmr2111089pzb.8.1707804116807; Mon, 12 Feb 2024 22:01:56 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWFrOUyCLxhatsjepcQxU4zXr2J7dzNdukdOcEgv/9tGAcVqSeAGjziZOM0Y+N95ysWMUjqF8fF4TeXR4R5MUgrrb5DSA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id k7-20020aa79d07000000b006dd881750d2si6118660pfp.46.2024.02.12.22.01.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:01:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62973-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=NBVXKK75; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=UhqLIKIB; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62973-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62973-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 9099228197C for ; Tue, 13 Feb 2024 06:01:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D5792BB04; Tue, 13 Feb 2024 05:57:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="NBVXKK75"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="UhqLIKIB" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 337BF2511E for ; Tue, 13 Feb 2024 05:57:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803837; cv=fail; b=jBWfmIr0YUyh35UWUIn2Or9PFTB7U5y1pycoD9eLFn2rCsk+zQCF7JEvkJkg+85+aAjHiPmklzOEx/L+3gALiWVdTNuX3GpljH/aAJp/jR1jrSWbbxfZia0zzGjK+k3g/z50Pa1zXsBR/MwfjatIBrh5Sz0syn11LJB7IqkJi4I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803837; c=relaxed/simple; bh=/90XYadXFTygsq5FGeOpqbO2rUJpNQfPfSSpfzZRxIU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=eQDnJ2DK8H6ZSL104nmr4UbfryqHAbk1stQgQ580h3X89ivjZ64c7H0m6tuE0j4Z1TPigKQacQTp2psxLn/LX785vQ4sySnmm4FfVjtXp9KwRcyA/wYsxS++ZRxpzgZdTg/GCq8VrNKl4suobMdU5W5xFwXfuOs/Vh76bfiyb8M= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=NBVXKK75; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=UhqLIKIB; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ZIP2006358; Tue, 13 Feb 2024 05:56:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=tX53l1bK2rnTg3wwG1qVz+NSwR/puAnNCzHLHDiJUvQ=; b=NBVXKK75MemBbKZSf17WMA7TF78fMY+fGRn2F2+GkuaWHhK1G/TSIhpHI0ZXCQKUO0ey bpzLeHfHWzCWoolIQVIxCDENHl8lg1o8BR/n/AvYv9EHyA0ypZVZMl7luqQI9qNDRiiI PtZiSFESGtl6Sp91XbtVdaaE6UEtTj+D275QDxx9QoH3/OW9m0WhxssdKFWKiYQ/BQEr EtRIP/uPhvBN80KSstTnVlHrh44OUxjRLw/8xPqZ50t2v5+n5MLxwVaVmH7+UEufbIM/ h7t+hZQEYIfJdk00IGXej0gMmUdVscjqyAZ5ZAY9E+/2ti9Qx0TFSh8PphXr8jpXTPRT VQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82d7r1gr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:33 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3VCaH024670; Tue, 13 Feb 2024 05:56:31 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3ta7-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:31 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cgbr7cbNsRU9E8FF75eXiLFrOXhxbKcCz10sJ/yCZHD2skV5Jll54sRM7Dwgs+0V3Aw11bM0kQ9dSLZfuHG0mWHnyvhVlvBpk/9xVUlDmC8Vm7Z7QSJXN5ZFGZSSRTrKJQ6aRXEKGkYpsyo4Da3KA6NMLkiZb/6fIGedmTNqJJ0nSXdU/KuOH3PqqtU8hkOz3XY2tb4vhvaDg4RJSf/bmFb7JixGNw/tuO/1v72E2aZaoB9Z0ts/5mIIq8A+iMVVPGJPfOZP76dgqAGSbUWbZ0i00qB/Bxq5cCw6KasBxNejoLIDm3LgZ5I49GKd3lBJ5LI0ZjqYpeQuS92WJpz2Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tX53l1bK2rnTg3wwG1qVz+NSwR/puAnNCzHLHDiJUvQ=; b=K/CyvT/aXCAIXFAHSOfMu5ENeFdND1uUgECLWDOFu8fC29w+F0WEbQBI38P2qvSJR4vpwwG9gRXs0RGdOSaK5DeTl5WplyR82QpaJ/z0VCuZ60a7O3aaBCdZ516/bJzyOWnOhbEzpHceFaGYPdVNft3Zu568QxD9kaL7qllHinYIY983M3L02+X8oaySLGm0BM8Gf+SH8F7mvP0MOWOGIi+3uusBUfHlbNOrJTizjxheYTHxIISytVZXK3N0V+NWam9x0VAt0Oy65x/KGX0p1E+63yYKSv41yOmYRmE+bbOyWal1bXKoKp6EEiWV74Nym3ZeTU25ZQbbzX0ISVrSug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tX53l1bK2rnTg3wwG1qVz+NSwR/puAnNCzHLHDiJUvQ=; b=UhqLIKIBp2H7mSgADOXYJqwICV+tNcVh/krO6rmqqYhPQCMonLO2zlmdRm9fWF/bte7IHvBGCLOdcnm3y+O8hl7ItjR2Q9PmGkOJP9MT4r2rCdgYubFDRJHxNnHCGKq6Q10uEhrHDX0JZyJq6eMGmVD77F6SQE4OEhWB10JyhUA= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:30 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:30 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 19/30] sched: default preemption policy for PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:43 -0800 Message-Id: <20240213055554.1802415-20-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0122.namprd04.prod.outlook.com (2603:10b6:303:84::7) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 8444f64a-3861-4f97-659e-08dc2c588640 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xCMRmwIdmSKAv/MS8NRRKEZ6Aj0+5qVDYrNuSyqgO43dGuB/RwKxUdi10q26AuTzetlEJQd+SzWMGMSzJxMEhUcq5lFsNqwkIqyBcfEX9LsNaxhOnBYW8gWs0vMgZa8iTMHB4DluKoecndyptGinY828qtSdlr0eeWiYjEQYtAO+JUHxe19WC3HENWW32tBsoZYPkd8LrjxgROQlyPue/W41buedBQnjfOkRxCmndD20O7BP3ta66bSkH8gNbGm+or7GyURrcoVH3/KIFUSrxSmQl7dWksj9WLOd8qGMTcdLeE9qftrElOVD3WJnkRaH0xHBJTyYuqDIi6Ol7+/MNZ9qwkPJ5oRC2j/GgVIhoAkWG+InMASq3++52Yv6HJ3w5FVW3X1wL+tSdoJlgZphuMaJub/G8a4ogonVNLGJx8xDjXE2qPaBG854ZRwRO4Pz3/fempoyk6P9k3flTux40Yn/4+aohrLle9ssEqPDtEQaXC2eAjNNWtbjkIrARqMcfapucb/ggUI5LgjTCElGFj19dPO695ilOVqojcU9hCs= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: L9eTBW6uGl4TzERnqcviUqv6YJEqR8Io86uyaNN1mFCP8tD9ojvj74QwPwWWEshVqvDrtgtgy717sRdNxDSc6+/kj4EnFziIPYMlN7mcbtTfPmDgITNsh+mcIJSIiSh5MruC/vuJ52zoPticgRiw/zkl2lPlhkI5mdZctvuIMnDP7WI5o/+iHMrThg2WzTXvMuk55bz8flSLcBMPHX5q8343QSn6UAATMG5Vqmr/CotTGwYs/I8OAhyv36l8SKcaYrUw/oOTwyG5qzecfs7vVDs15jSc1VnOaoH4YfaRRNvNMRdSNP6F1ZGqpSKnd17ebVv/7aRe8JFcztyNHoJWmRCz2Rs6la9aqLluSXxmd+qUpgfN28u5+21QwZqckDs8Mxv+yMxoJ9ZWky/K597NzblkyUDfX2Bq4gDrOxwUoRHcD0iQgS+pnkItL9Z5F3Be5fs+U/ZIuB6M2vRbBnF7wHbNHYQH86LIJzU3x/lw+NX+fn+0fnp0Pg0PYYoDZA/uDVgwZ9HIW++0SKmcfFhCgOa4MWQVAmfdtntB6k2nBBshHs+bSb14oR93j2oGQ4Wfhw4mXixOxJDs6/TPmh/SpPyabMNAOy45hPB3uVlIe1era4mPOvdClNRIpV6Sqrp0f91IBFCBLSnDIsQLlA2mAluU2lnz2HmS3t94llBYu42IYtsndjyQjhhb93p4AQP0kuVQ0jNhOn9yr0IjQOh8qBIt3ArtB//tV4wvOaOF0TVthl+E8UFkGMh86ZkCf2N0hK+nrIhJI2y4D4Kd53gzAtin23LQipYDSP+AOvQQHuyOKLCdacy73kXTu0kA2BgZffiiZm2ZWNHst7uRDdf5TviH8NBm6/ZEGseNBkObi97+y9PkiaLnt/EfNB19p1XryaAN3TknF1vj5/R2BVvKrjeAD50rF567mg/40sC/qjj9iBwbnzKaeF/txLnzUVRSsnpDuPeeb+Fy3HrphbUpNRmVT6uZmpxwjPQuhTL/nc1+XUOuM1bZi1Kghq+d9xQ2k3fmnHnlRXyBrEA9Bmw7B3OKu8IlSItJRhHX02WabrUKLQypSX0Z1v7P7bhrefWcyP+/EEjZZUtJF3JtcKCK/mPCiYW39gMqOFkxI0IcRhnGN5FJOPJNDjpdEfYWHNVr7evXsO1RzEwNZpNj5k1SkMpCtLwIWyKeONeBArXwSJFdDCgZwTYRP3OuKTO+h2qV39Zk/CfdeVPo8dx0ez7+nATBd2ylnQdA3qf62c4ebCnpMMymJT2AtRYVcDu4ia/A4Flcbhvkj7UY0D+h5B+wjWjHSSVkzJfg0Rgqv/cJH0Ne9sEL4oevsPayBwp5ig1GwrOFxTOa8rbrUvPrJqbTI8W8UbwMbs+aOyxkuDbOgLpFgbc6wXJtMMbk0hhKRMYgaISqnD+T4zrKnIdNl0jZh+Jz2aj3SrLGW1e3R19s+0HhncQoziuVgtI71agS6wvgdkv5dE2Y22Kixdozaf3gDC1xBkRlrrBG3fwNnl+uvybfvs1RP7Sd3qmxKnLMsfpnzCPyO5OI6aL1bBeg28X/Fyb/UQhcGChZgQm4xTJK1qMdwsJj9hcRDMjW1aVAPBy7REj0jJYcuBLRRP8Vg55VNw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 7/9WKKaKC5t/FUUCnvuIGXJpn92j+6rh3c3NglyJQ0a89XjAEOliHZQGswRFWeTFOiqVGVOOlUVYXNTdLlHH+RaniKYUHK986Oio/GWVW+3g9Gp9H1nh0PoT3ON5TcqrMpBr2rw1ZjNFmp9xUfl8X3jEKdnizmsheWeLIA/d27XezXe2pmK7A47XkqjXXJbrjMz7/ltORGqI4MMJ85qO1HVaA/qscPiDaxaURVYDwIlasq6vuZt6ShTlRYTxQNzp19CkXD8TCBCblv0kFhJy2L06Ibh3DnA1/KuakZTLdHxMOxaTof2XsH1B9a9wlnX/islu0bBl8i/5vzxCkJMO1uv5hXBLyyYM3UrSKQz9ibylsUOVJQJjeYmaIy6UQHALOCI1G/OSf97vV6MzwJAD0Ix9ZdPEdw4+bQ8rkvhcMANKbg1AbYwXcCGpvXpdKFgGsYTry9b5UIMuX1YrWb3PLXf0oSh8jM/vEN+PIGpuWGlo3mAAU8DZsv3YbRae/MaZDSAFn+RQ5EskECxYI8mBbP3XZsKkfWsVJ4zt2ecs6FUV0xnTYs9kqQMsCUqQmwANrz5el9kTq9s5qdapzVa4QRCZAqoOsooKUuHL23mMCg8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8444f64a-3861-4f97-659e-08dc2c588640 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:30.2165 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Bv5ztFBTOIFEiG7FMm+iJ+hjYmSI13jHwz1ph5MRQg2J5aola0uwSftOkTSEc7Cl+1aLe/8V9MCMl+kFnqJSt2Ly9BRAUdELZsS/ERH1ld8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: 3SKrLiFqn9WTmiQ75gJCVKIRdXPRzCQ_ X-Proofpoint-GUID: 3SKrLiFqn9WTmiQ75gJCVKIRdXPRzCQ_ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762409566862358 X-GMAIL-MSGID: 1790762409566862358 Add resched_opt_translate() which determines the particular need-resched flag based on scheduling policy. Preemption models other than PREEMPT_AUTO: continue to use tif_resched(NR_now). PREEMPT_AUTO: use tif_resched(NR_lazy) to mark for exit-to-user by default. Note that the target task might be running in userspace or in the kernel. Allow both to run to timeslice completion. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 31 +++++++++++++++++++++++++------ kernel/sched/sched.h | 12 +++++++++++- 2 files changed, 36 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 7248d1dbdc14..6596b5e0b6c8 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1032,20 +1032,39 @@ void wake_up_q(struct wake_q_head *head) } /* - * resched_curr - mark rq's current task 'to be rescheduled now'. + * For preemption models other than PREEMPT_AUTO: always schedule + * eagerly. * - * On UP this means the setting of the need_resched flag, on SMP it - * might also involve a cross-CPU call to trigger the scheduler on - * the target CPU. + * For PREEMPT_AUTO: allow everything, whether running in user or + * kernel context, to finish its time quanta, and mark for + * rescheduling at the next exit to user. */ -void resched_curr(struct rq *rq) +static resched_t resched_opt_translate(struct task_struct *curr, + enum resched_opt opt) +{ + if (!IS_ENABLED(CONFIG_PREEMPT_AUTO)) + return NR_now; + + return NR_lazy; +} + +/* + * __resched_curr - mark rq's current task 'to be rescheduled now'. + * + * On UP this means the setting of the appropriate need_resched flag. + * On SMP, in addition it might also involve a cross-CPU call to + * trigger the scheduler on the target CPU. + */ +void __resched_curr(struct rq *rq, enum resched_opt opt) { struct task_struct *curr = rq->curr; - resched_t rs = NR_now; + resched_t rs; int cpu; lockdep_assert_rq_held(rq); + rs = resched_opt_translate(curr, opt); + /* * TIF_NEED_RESCHED is the higher priority bit, so if it is already * set, nothing more to be done. So, the only combinations we want to diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 34899d17553e..c3ae70ad23ec 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2462,7 +2462,17 @@ extern void init_sched_fair_class(void); extern void reweight_task(struct task_struct *p, int prio); -extern void resched_curr(struct rq *rq); +enum resched_opt { + RESCHED_DEFAULT, +}; + +extern void __resched_curr(struct rq *rq, enum resched_opt opt); + +static inline void resched_curr(struct rq *rq) +{ + __resched_curr(rq, RESCHED_DEFAULT); +} + extern void resched_cpu(int cpu); extern struct rt_bandwidth def_rt_bandwidth; From patchwork Tue Feb 13 05:55:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200220 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349080dyb; Mon, 12 Feb 2024 22:03:18 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXJsdnf+yaLMnsp8hsnPjus3kUykssCzMD3SYvl9OkYOAzI0J09cA6rae9D2v6ENmh2DJf/QTXTbdQOZ3LEdaooMDHM9w== X-Google-Smtp-Source: AGHT+IFOT8r06ZlZCbcQEx5bgj+v/A7bv+sx1OP8uDWFSx03mMp03Uw6S+/pmYgFfqwtyfPupBMR X-Received: by 2002:a17:906:27c4:b0:a3c:7b20:2dfb with SMTP id k4-20020a17090627c400b00a3c7b202dfbmr4924603ejc.1.1707804198028; Mon, 12 Feb 2024 22:03:18 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXNU518sGfI50ddAFXQQ1HFvJ9Bwl06s3IJgB/JQhKCYMCQsN/1FgasCf5QbQ+1RCS2YO1Lmimgs4RXrVsXxfsoJwtFJA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id mb16-20020a170906eb1000b00a376270dd1dsi906475ejb.740.2024.02.12.22.03.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:03:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62978-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=KfGumtUg; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=G82c0uqf; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62978-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62978-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 6B27F1F24AAB for ; Tue, 13 Feb 2024 06:03:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7794538DCD; Tue, 13 Feb 2024 05:57:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KfGumtUg"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="G82c0uqf" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6F313717C for ; Tue, 13 Feb 2024 05:57:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803850; cv=fail; b=IwEGvCIsE9oXPyWG9uFHljjzfmwdqxLEOE74Cz+bPBc+Empbc7hxLFUygO8C+Mlg+Jxd3Q2DAwSfTcOTX4S2O762epWzoWOq8zHIvCRWNeo/0ira6DRsQIVWqfokp/SGpsyNErs9jF4+bEI9WxEPqMkv5KcHiLUpSSYcRvVGAi8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803850; c=relaxed/simple; bh=Ft8y8OngKbzwDsZ+tglqAARM/IdDnwoeRLwfRMUNrxw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=ON9vp5vjQ8TKq/v9WhYjX4T5+scwSddM0/LE6Z8s1edjEvEc+gv4oRY/D5BwoDFOTg0xoy5Y3QB6wN5oVkpDlrI5f6xGYp+VYp+iY+MxouuMPhvb57DpVWohJuKv347ilhhcQgJJe06jtv2aa/BhUeG1Hss6t7R7fyiQWbz1wFE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KfGumtUg; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=G82c0uqf; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5UMxs005162; Tue, 13 Feb 2024 05:56:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=tCiSuEUWQWMydIOayxrE3q9Dd2AiL5VU7dfnq3OK/JQ=; b=KfGumtUgsXEtXjM7v0gQnCsvzDuOptWAUUAGUE0C3eDPc9NXxEmByRGsDob+kK3blXdh 003VEzF6pnGhjc4NUrETa+fbGp/qkxHHmsKFTvxmdfaIM0QbuEUZVlMNPexdZPOh2e8Q fJ0viA+jipSXXy+c1Qg5KZjbdpxhAWit6LHI/hJlBwyGfBDwWtnIWuOn72+y6/ZwJwRC agRF6EW89n1auocDgpuq16t6RI2cudTKF8cGo6m3r1YGM31jL+1vJkmiiWlv09M4P1zT xzS/c7EvSO2Eag8q+61Sj/P0TvHPR6RMHg/3nZ42rZEba77mvXmRbY4O7K8vaA0yPn4V kQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82atr1ru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:35 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D3XGSJ014943; Tue, 13 Feb 2024 05:56:34 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8q9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K8JZ3eNVjl1qS4Y3FGhQN3c/hPwv3j34Td+3fM6XeeC95QXfGtDUUSEnYprGEkiCkIoA/wEaveJp45LkwZ7KhApholBlWKgXdCidFQZ0QMPVF3evpZDkp7sH4UJQSDB8DO5wxIajW/iZxZRpLNEHboQmkX3BtgTr5blBb6WOa6yQlXtxcEODZ18LY0q9g50FVwZqldVAhCBpfC/+VotbC6YC2TVaYl9FBr07ln1ruCuXSNInTNtOphOcMkhtZ3EZbOyid5fGJ5OoG6sZckr6oD7WtOM9ee3Ts0Yp4Zcmc5BXEOzqS+x4+FD+BFL7BzFGo33XFtMD3gT1b4JiQLazFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tCiSuEUWQWMydIOayxrE3q9Dd2AiL5VU7dfnq3OK/JQ=; b=b73R07z1rNUyWLxCmW1GeyPZyDbiY+tKDxbA5+KMBp597/RUquMc07S2GwHvXiy3lJGUxhdNBof0vf8uZknen4ZRkFwhB5cLDIv6Hp0dfnkaA/WVSwoGeS0UQvOagPxN5gSBo1RGzDQrIeSjQB9XbF4Ezd7iDf31TzBN2FaGnJjXAIQzSFeEy+0+QgxUKLH6d3lm+SPaPWMXKcD9f60Y6tD7GLdDvJjDjg6ImJ+LUFH5BJAj1LfCqc1VQRPLUMMZT+2IPwbPE4r6I/u9XYb8Wo4vTvw34w8RUjBT4oxcbNWdaiGNadStmU/roQUlL7kMIcRY1XgGWGItDIVbKlOq1g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tCiSuEUWQWMydIOayxrE3q9Dd2AiL5VU7dfnq3OK/JQ=; b=G82c0uqfc+uqjyumw8EE9WWfFzI5hicbZKnGB5civqtE2O+LHdWo2oPFbx266CcCeCTdGusZO51/Y+2lO5HFQZCqaPSDbe1YLiPMi/lhi696+IYaETaoaB5QHAsxT3ybS7ANgLPyY8pugE67Thh36qq9NRYQYacZvT/JWQTDzeg= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:32 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:32 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 20/30] sched: handle idle preemption for PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:44 -0800 Message-Id: <20240213055554.1802415-21-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0278.namprd03.prod.outlook.com (2603:10b6:303:b5::13) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: a4ef8248-25fe-47ab-bec8-08dc2c588754 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ybne6AR9CBy73Q8qm1BnnuYoXspEX8rxYLNe7W66wv5Hiuqo83z6VCyWr7dSVLeiBdqSveP0nvzdBRwKPNRRvUH3U9mR2zRYnXpuj0tn1Opisn2fPw1T2ndd7D1bpTcLutI1D3l7kh4GIv4RteJz7JFPQt1nqitC2Fj7XxOY0brQ8n4e+y1yeonmRO0NxMfzCDkBDsnN36uV7RyUMZURRl/Az44sxW8W98j7D7/NYYVGJBf8ghzGB5lKIYtdfKMionSNHuKwN8AZ+DoaFkPD0H6iaYjt3dhZ8+7jXGMA29BqKPI0UQII381S/gfIZ3okqhkyEMteJXsqQviyKtq1O6O88jKZeKa6VMJi+gVV3vZWNdzxjPLL5iYLUzPy6X2uC0QYZTQbZOEZI5mgSXtvEQmFh90zElgbHc2gB+j6JAPccPW3H6eRve87q5ISozqu7xpNZYPLccsYA75ZWBOYAKJ4Lr76qommJYAe8z/8ikVZ/Zan53+JnLEC1lqNzSHINil8FmzEQ3YgODpEyh/SYkVqoY2c/0BsGa1rp5hgUw4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: T7Za8WfefmA+6MzSSIJAnhLsROl8Ikb+5lNNN05lNjo75gTZsgcFP4L4csbJ40Gd6XzwApAU+O9IvUk48etYwRqLONSKaz9SIjPCVSOtPVnIqTZcR6b9FCKtKI4ZMWruoYIrD8B5y9paIZajpDdZ9Em8N4+xNoAv/EbORa6+YvRCjNlXeAIPpbDuMM5TqJCJZFGyelz6p4U7Yq2wLn8rn9L4qBer8x1Wv9TNbNPYhxl1EWVce7nIbqMF6lY9s4lK9XCpW5BEIn823BxGaKyDd4vbi3pgwP1kyDEjM8AG+NUEiVYc6PD6JbLJ0Hz0KP8OiZhfwG85wWeMz8jMsXBiU3BhEYS4OfxT/mvFT6FujMQEykOjy048DmvPlg79Ej8Q3okNbW6Ny0F77AvvGQdPh4OEh2JtlYEHUujoJKS0H99TGo9sBewEqVruzXdHajxUlYG5rdOygC+VORyTvaZCaTFeohyVjvDdp9Z85ES/X/WPDIxCuM0DnigsUDdJiM1FsxlFF4UNDdxryfFDCUQ7ifgPRE8Xgj9GN/twEoT0vq+vROpf9zRXq3OlzrNkeD93SufLDKJxggHkLNwhMR+InuhazivHV168tyDX0VHHB/yGMwd8zjtCl44hJ99kqd4xooyjH8Z2JnyRbfjnbRinyLGPqax8AkN1cpwC5Bt3aLJIhJBRx50irbEG/IElUu34Z96h6JrRoA3RpbD89hQ8PU6QjL1crUcyfjwNO5udeCCbH9ikOQm23zJSD27kmii3tXM5M+Ucc5paxxv0ySV/R6tGiaVZMRcqOwLIwniVEuaXPPWveH1Ihv6CLql+7i8VIUqF6KkC7lOcW17N6q0KMra61GzW/Odjmv0bi/4cxbGH5cSxU9wvA8Myn8wc5skMgoSTYOXP+cK27F3zNL/x0xosynERUub++XjzIla9G8R8JKBi80lRESFcgq4+j4xfWu4xm9fT+pidviBQHDaL6NgcY8MaSvQxFhjMTV5QK07erU2jEXovEMbqwSmkZ6Q/5fz8ALL7oG5YUli+cyKB0fk5Dj80bCD7jKkK8frjv4uvNbmSISxhDLeEHtcw54TZJTmuUgWCyx2a32HLtFj0CjiTW6VJdCfGWpfMUEODDSWdWAgSD4v5IWbJPoeXM3LZMSuTZbbiN03uobD24IqoVd3HFnzoJDVyU8igVEuL3cIPm1c2tU29gb3Vd4/MDu8LWYtoat7OTOZ1XqtEOu0652A8ALeIUH3rH9gKAWUWG40hJIwGc8B12+ziub1Rv6hNGOWQpGCQ5CoFjiKvhrkHdQeaqWGoPS3Ep4EUIhuL8Fa3a949b5LeJCMq7j/D9hSDvavZATb3ZSMJemPIv++Dlw6jtx0xk8u5xvsDfsqse6hp84Hp+Q0d66s/WeWvFlO7JS1U5PpCMYQUHgY+w2cP01kuOxpYL2LR53Hgvaq4jE0bglJ0ApW0oeyGrHKYcNMGkt1VoEPflG/WeULNMZtYZsTFvywVM9GLd+34qSKAb2Q1QKeOUlMi1rYV5SJauOJeIKARmDw7BwS3Q7AXTmV/hnbURFs/4vstR8PegcF8HGM4QgbTFOzQkzKkwLjn97W1MRy2p/gL3BGP+9VSyk9MGA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: XBXYo81hww6PCJCGn+O/lL5W0k+P9JVV1evr66s8N5Z3vk4ux8SfpJxNNASf1xQQYoOfKHzMu9axMIGYHL8a75j3kuBJE4dyu3VYaLSfr+VFGc463T4Sa3hhmNbx31OII5HiDgeZ7lmY+6ZhTz23c/Q27fDuMIVDA8mOFB1pqXbXO/ZKmWuB9EfbeVcRHFfOqipdgcu09LmrtVbWoZBpD7TKmBPUgZbcJS9KBSMGZy9TOLXCKQ9DIDV1APQzS52lVrUs5Jbc4kvzdWKVgbOF7JwoAAlhv05ycREeNG5K/rz70iVG/M5AfrU2RU90+qzozEoK0SLsAOCOaV0Gq6kcVctu2ul9Es7NhwdQGYkwNUDReVaEe5x1jpDz1jzUwm8P96dDtqXJrpAx6AZ/DypryZcMX08gRMHsby+1BHf/MQ1B0E886oQoso+ON2L2PKJcHqZvwTjo8vHKDge41ELPX2sns8+Wn88FPnQOLeJlsQg41bnon0MYgCjy8+dEXhwWYuYOI+S7oaBgExU0so2qRVppAArD0DiFkOp03JkmZ1l5TO9lwYVIBJfJGWOb979JF87DX+cJfAXUooWCtxAepGhgZl1xb8VjyiB0zt87TRM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a4ef8248-25fe-47ab-bec8-08dc2c588754 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:32.0170 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: A6gJdwz4w1k+fV7RTgAchFtWEOCPdkExNbyRp+fW1xUsYQexi8gCxUZ0y/dd79FnypNLV6KMEVeS6IcTB5pqlJdpLAU7eOnmym4X0Ur99zg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=984 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: z7dvOnMISgqXLOoahKatk7ADvv36kQdz X-Proofpoint-ORIG-GUID: z7dvOnMISgqXLOoahKatk7ADvv36kQdz X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762494687332244 X-GMAIL-MSGID: 1790762494687332244 When running the idle task, we always want to schedule out immediately. Use tif_resched(NR_now) to do that. This path should be identical across preemption models which is borne out by comparing latency via perf bench sched pipe (5 runs): PREEMPT_AUTO: 4.430 usecs/op +- 0.080 ( +- 1.800% ) PREEMPT_DYNAMIC: 4.400 usecs/op +- 0.100 ( +- 2.270% ) Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6596b5e0b6c8..5e3dd95efb73 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1045,6 +1045,9 @@ static resched_t resched_opt_translate(struct task_struct *curr, if (!IS_ENABLED(CONFIG_PREEMPT_AUTO)) return NR_now; + if (is_idle_task(curr)) + return NR_now; + return NR_lazy; } From patchwork Tue Feb 13 05:55:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200234 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp352498dyb; Mon, 12 Feb 2024 22:13:03 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWRvFAXkPSx/7jCD1KthHkTKFL1oTYgqSMzE08bkhefYDWxUcdL6dgkPSwGXUC3QumCzqxpkXjx55rXLpPT5A2K7hOT7w== X-Google-Smtp-Source: AGHT+IFoY9go39BC1cfwPP1GqFwQsBK7VKm6pQekwybEcKfY1091Ojw7q1V7gjCIkv89CBTY7PWT X-Received: by 2002:a05:6a20:9f9a:b0:19e:b614:48ad with SMTP id mm26-20020a056a209f9a00b0019eb61448admr7868200pzb.5.1707804783294; Mon, 12 Feb 2024 22:13:03 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVThFGir6OGlmBUvPhuLjkzMhTEB9IVOmV+kxkT+vf3ZfKpEhrf/9fCliHnVhwByGMXNYMDMIo7MnxaECpCo6HDziigUg== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id b8-20020a170902d50800b001db331108d2si766304plg.379.2024.02.12.22.13.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:13:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62981-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=Pqe+mfvq; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=jlQHVaky; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62981-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62981-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 9252EB2722D for ; Tue, 13 Feb 2024 06:04:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7E83343154; Tue, 13 Feb 2024 05:57:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Pqe+mfvq"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="jlQHVaky" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53B913A1AE for ; Tue, 13 Feb 2024 05:57:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803861; cv=fail; b=qEdGuschzYH92W8DYkkOtQad8XvfGEfZh86XwUsseKuOy7w2dkyhCHgYPjrhNC9lrNaI8fpGKCHtSlLOgLcDeEC4xE56QJ8me9ViGeY2ZFDD2HNnyIZMg22s6EN+X6wOLFOkrtAZJgt1LmXaFvjaYChvlqzykS5JDgFXEwc2GLo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803861; c=relaxed/simple; bh=nBKwVvQJxQh1s9wUNZRJwrqUQwS4G2njDeHrQGd1xqU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=g/BpK9DHrVEY3RBWhoaoTtt+AakRQFEn+VhFhAHs3Co3YhPikIka9fKHBwlQIve6+NQjJ9wQoHtBbBqUyQq4HH6oIIK4674DUSggnraDcjKI/4vilMqNq1LrIhI5nKylzjcHIDywfyNxaBp8p6H2qwwTcm58fC7tGK0vTzHjQjw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Pqe+mfvq; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=jlQHVaky; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5UAgf005097; Tue, 13 Feb 2024 05:56:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=I6D2XlQiBTqdezvAHzpaUa9Foics35VeMmOvva/QEmU=; b=Pqe+mfvq8ptPBJxK0yeZK6R12G/udsmExzIbF2gDFPrm0Vn+aSF6mGAf3bXternconXX 6TLeaMnKsEvfdXn8b1N4oUnrkfDycgW9ccNMiDm4hh6rfVTVrF/KPq9LFmFPylBzDN4c vsjuE09NnZKed/o2J1RL2xhe9F5gAcQBBclvA5qk76QivJ8aGj1OztoKPJbQc2dGb6Wc CUS0RaWwr0Oo9jzZUykYfdXrlfUK+pDd/dfmMwK3njUH00c53HTi6pO4r9ZDMb3haSg0 rOd8TkwXwwFT8JYEC50WaA5/jjTrM2ebLxhdJA8flw2/Fx+siX4AVvJ9EUQOzsf207Q0 1A== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82atr1s0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:40 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5uZxs014991; Tue, 13 Feb 2024 05:56:39 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8sr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HxkzY0aUq0MM6Ow9Ww6VHGjJsfzVYm/nYoDtZ6Dsl5LEZizMAhdEGC8YNANX7UvvPjFB9FNFE+qVOX9wtFHfIDDBMmdy65/oRnEBCEB58/FvJXW18LGRV5l8GzCQjxoLVjLVyOqHmiHGbw2SxYYE985Vh00sYaX8Ft6NR1tySC5EDYeGHQf3AngLw59TjLb839g8Qr1EuPROixyEJKjNslhW1nq+NAW1dGzfplvuqTSBjOARnrxYU6hZUXJW8Iw7aF4BUuvbEctYRo7mnEd+X4QQL5M7yz3BAxXG4jRIcF5JR1TO8df9DZ1tIk8WvlLDE8CuOIjshSsmOTgjgPdCKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I6D2XlQiBTqdezvAHzpaUa9Foics35VeMmOvva/QEmU=; b=GXdv9AY4L2cZS0xOy3M4M63MjtAhQUg4h57F0XtlagUHTAqpq9mI2VR/Yw451u8vhZ7kKO2MXoP9zJS6KQ2oFPcVzHh52dwm0yxrJ+MbnzFxldC4yVemTNUHOPMMTFUqyVXkWSRbiDpNp2iskXLw62lQCnagLmmCP3oIMiK1y+QIHC/N1wxY1Hu/RISV70sKRFKOtTPQWvfsjlv7nqPFREGh79Ol9WTczWNE/irueU6PqRF3wMok8hkECIEiy8tmUg6ACqp6EW1Osuoy4MInkMSE3ijDZHrr5XObwZ0s4mkHy3SzoFTUMQGIkOU9exN1Fk32t1nY0zzbP1Yisnbl1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I6D2XlQiBTqdezvAHzpaUa9Foics35VeMmOvva/QEmU=; b=jlQHVakyLC8vuGfnR52C9VaN8DjwwS5PLaBEZdOoaytum2+d7Ox1ud1psU58CHcCIXZT3Manb9CgCyj2GecPi4GNz++uj3N4SJfbcrDFENN08DRonWScA0tYq/orqA4GOPqKe9fjClwMgxzUL/igVZwGH4RnWk+oSpn+KSNe0As= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:37 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:37 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 21/30] sched: schedule eagerly in resched_cpu() Date: Mon, 12 Feb 2024 21:55:45 -0800 Message-Id: <20240213055554.1802415-22-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0271.namprd03.prod.outlook.com (2603:10b6:303:b5::6) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 62d67274-e41b-43dd-4fc0-08dc2c588a4b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ojfQ7FooKbd1gFw9GhN5OBBuUrXJC27rA5z7iBXk5h47K6dCniRIy44AWaakodwuhqejsrg3/78u4xAlPOyUdfpam1ANdpKcWg08Dajw5AAIVukwAlkREZmbzcIUfxbZTYb0BPk5iIa1UBPz0D6wgiqKNdCFgzLZ4aT9ak0IgDBiu1uZBzrgMp5xLzQne/2CePNaZ3JnDi/+LH5hx+bzRHscYkM3qWCVPfgLM9kqkoJvBtWTGoFh3hPKmPkJJe7IdA9l4vdxDNTR1B7wCWmDEr9M64v3DhYMwI7ioTWGemWf3esgQXx5dpe+ZYLBZdmXzXOg2jNDLe9YHRIKlNsk6F7epJuPeYO1pQv8sTwu9bONFPE4PsLyp1H45ryOVQBJnvcLiRgd260KGYO4SIpw+449CHUTl4I/t0fkGDvb7f5ybd0X0dlKAWX/euUZCzuyCU9XPRH/wStOWPhlgqmse5m9u8Ic57QRPv6WVmv2skHAQtC6KQ3hDiutkt7elsm6epkZTaI0TAQUy2zktVYdXj/xSiN6JdWVguzFcM3WVziZmMuzOQBZ8EhXufbVgJiR X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: EMosoI7PFq1cO8twuAmOxV2baxrtE1XLQqUgH+ECIcZXjNr76OBzOtoEg/Pv1qYi9BnS9m5tVgHY9motCpfQYsFWE4kuzu+mp6rx9k0tt0ORjpRjFrMarmLwbJIWn7XwwNBYbCT12kLkYPRYPF86yva/7S8IRzOKYfI+KKlDJuRVwzOR3L8YdFLavr/yv7ujX62Ekuv8mTKoa0Xdk6vij9TQqOUIIYKJ0JmyFjsod1LTTmvMA95KyJ72BGb5fKEAIH25BEEhkJQYV9TXJ18zucUL75E0KlS5qri1WTupGpdcEZrJt/sUSmkozR+PEn06Id5ZD2740V9ikt1XIToR3YRpf59K0j4QN+3ITeuQTtLf8/iZo5KTY3FFTU6mP4QWzIORN6ZhR0ZG5dMNVpVtch+SKzGzg1bH5YVcV18FS615iIEwJWHOAoK+Vm3krVmse+Fc04XihHO9zbdRitWMtsDrIRHym5mubvsL17wsq2ro6qZFQCwh5vKCgZe5i92sO7nRAu23iAgIstRmpsCVhgR8gT0GoukWt90J7zDJyQrTYjL8biJd79FVBh4Y0wx5X3KlBJClSTrvlyW9uScbVpau2AscS4EhyGmkyKbE3p6cSr/ONa5lNDGyNZuMBKWUaeer7RAzhdssWgb4WQ1YKom6gm93zuhHVwt/+cfumrDJHo7/yNj3nn2dr9zcH/HbMkmNmcATMYKf+3aHHlLZSc0Ps/WCCE1MUy/W+uIKHiUVeLS9xDn37rEODUG+FfO+/dt8IPJmJ7mgkDX5v5W3OwjgIVQeQcpWblZXuoV2aN/6iPHT40W/WCC9cMwwGuG2OkaQpiEDN960mrqR08scAcWTy3/zsawy0wQHWv71tbIgikPwm8P8Q55X9i7r8DtNgwMZAZFZ2cxm6HsgH/N0SODTyk999DCIuU2jSI0h6muqMdyF/xwJ+cePWrQcb6Y5XxCDTMGd59LzMX1ZmDbSCuQoPeEEzW0hnRlsImrrui7GjHT6+IF6N/UDC7ceEdjfbhfnksxRprF6j8dZth/BAd6ZRt3N13MN+M2UVhV4ORxfcjnbqzZtiobCj2OWJBxD2kryAQH75DnY7JGyLBOqAx+eWOga4CpPCcr2YnOv/VSm5nTggLaDAB7hTrNmD4foxlqytVBXKlzGQ7cXv7W+tho+qdE3Y9dMi8KKvkZWJUDsJ9WBwEeKJtMFpX/rtzL1KsOquNSbXRE1+lzT8AkYkV7Cl9kKsJaMxUVM6+dfmhkU7vi7bfiooQo+yywGSZxON8wyvQGoLdIQ0R1NiTL/ymwMTNFnZ8FqyGo7TTgwXJIkd3C6JnhYGnpteKjB8eu4EsmyW8c9jeA/kXH4HBgY6W99ikU1l26Jie8ZLwLQH0as+1sbRcm+TXfkMpDypZn/1Jq8QFSf3iyG1/J7Znm82LKMfcf2tPrJfwpuX8hsOPhEdEBoORsIVA/Ocu5VsokqmP3a8vQXn5NHZdV2eqAFuHZR04SgSt6uxfwtumofz8SzvinsKkpgh1aK9YFowY3Vo1XZWrXRQgjefMIpFxCHWLkVb7TkmlpnpsWWo59cbvltLidlAPkHWAIBnpU2eR0/s+TD2V2ejxaI7IU9rkzIYA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: uvgLDQwRrp78BvzFju4XRaGBeqzrLTP32DnZ0LratToHAnaCEn03s5q8284gz/bd973bwhZZOx5YHlZXuc89uyYB2eO07jrHEl8CrZAEDgPmT5kfD66qGYQ7BY8pHgGKuID8TRaxlBhH1RMDNK7UbisgyAnoU6ts8ct94fWOlLw2z3wCfeQavdkKM/kpjSlV30tg0NvqTLH/TcV7TuaJZxHMf52MDXOs4X1n3aVE1f4/9ImoU/0ePUO5aREt9Z5IIHJUaYMqugvrwn1jtos7IwR8eCzG69WmZXVVQUTnTkjH0dTYPBkguYDm+fWSPtcdBgfU30JmysiXGcZ9nRWnpp4zeZxQ3P+kxXgPbedj8aa9Xm7ra7LTMT7no/37P+2qM8/H81rOBZbMvdgokhKfq4M8e9cHX7Q6ZxY5QT0ghcykpeIpdt3yXdZuaHXKmBOyLYlGLrStAnfvLJtmM42wyPk8ES/XmLQTBEy7LJYbyN8vdRG60BYnBNLsAeSH/8NskGEhbBoHXt4wXzQ+AynrSkUuJG19QW/pwzNEG6jMld2B+FhUbl+p1y8J1I9IjRRxir9GDOnYRjYWU9MEWSwufKW0k9dedsKYAZDDNhdMGTM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 62d67274-e41b-43dd-4fc0-08dc2c588a4b X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:37.0147 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9Bhp3U08bZ40yMG+a3h4JGY0c2Fi1201NmTUD5zQYcql83HejxX1WA42fxFVWtWL8gPz1OaFoQ5a0jtI7RZujYFqIJungYJRM1TFRVSo950= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: utIk-zCDd7SrAha4fZjevqfXPB6NuRcy X-Proofpoint-ORIG-GUID: utIk-zCDd7SrAha4fZjevqfXPB6NuRcy X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763108761297174 X-GMAIL-MSGID: 1790763108761297174 resched_cpu() is used as an RCU hammer of last resort. Force rescheduling eagerly with tif_resched(NR_now). Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: Paul E. McKenney Signed-off-by: Ankur Arora --- kernel/sched/core.c | 15 +++++++++++---- kernel/sched/sched.h | 1 + 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5e3dd95efb73..de963e8e2bee 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1035,9 +1035,9 @@ void wake_up_q(struct wake_q_head *head) * For preemption models other than PREEMPT_AUTO: always schedule * eagerly. * - * For PREEMPT_AUTO: allow everything, whether running in user or - * kernel context, to finish its time quanta, and mark for - * rescheduling at the next exit to user. + * For PREEMPT_AUTO: schedule idle threads eagerly, allow everything + * else, whether running in user or kernel context, to finish its time + * quanta, and mark for rescheduling at the next exit to user. */ static resched_t resched_opt_translate(struct task_struct *curr, enum resched_opt opt) @@ -1045,6 +1045,9 @@ static resched_t resched_opt_translate(struct task_struct *curr, if (!IS_ENABLED(CONFIG_PREEMPT_AUTO)) return NR_now; + if (opt == RESCHED_FORCE) + return NR_now; + if (is_idle_task(curr)) return NR_now; @@ -1106,7 +1109,11 @@ void resched_cpu(int cpu) raw_spin_rq_lock_irqsave(rq, flags); if (cpu_online(cpu) || cpu == smp_processor_id()) - resched_curr(rq); + /* + * resched_cpu() is typically used as an RCU hammer. + * Mark for imminent resched. + */ + __resched_curr(rq, RESCHED_FORCE); raw_spin_rq_unlock_irqrestore(rq, flags); } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index c3ae70ad23ec..3600a8673d08 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2464,6 +2464,7 @@ extern void reweight_task(struct task_struct *p, int prio); enum resched_opt { RESCHED_DEFAULT, + RESCHED_FORCE, }; extern void __resched_curr(struct rq *rq, enum resched_opt opt); From patchwork Tue Feb 13 05:55:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200233 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp352184dyb; Mon, 12 Feb 2024 22:12:00 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWdBojLME9qyMmF4SFEaas9wSfwiOaI9FF8+GPt5MRYAE3By2DYp4KvotuZw0ZzCVL/dR33oILUM+kfTRsITJU1PTkuyQ== X-Google-Smtp-Source: AGHT+IFxsgLZ3dkuSNz6sSAscevE9bj/tv/ecNkSG4LjczMG+H3gSC879YKoQ9fZiYjiTdjjJ25y X-Received: by 2002:a05:6a00:a89:b0:6e1:15:ac41 with SMTP id b9-20020a056a000a8900b006e10015ac41mr55072pfl.8.1707804720283; Mon, 12 Feb 2024 22:12:00 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVQT1ODifsNeyaPTGBmkv9tDkBnlYHfzN2/GWXawU4Uq3ZGeM4cOS+7XPso5LXv1hP9bduhAaBxYv/XTZzyormz374AYw== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id x187-20020a6363c4000000b005dc8554d500si607614pgb.62.2024.02.12.22.11.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:12:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62979-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=NTz8Z3mc; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=TeNA2R6H; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62979-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62979-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id E7608B2703E for ; Tue, 13 Feb 2024 06:03:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1F12738DDA; Tue, 13 Feb 2024 05:57:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="NTz8Z3mc"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="TeNA2R6H" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAC5D3717D for ; Tue, 13 Feb 2024 05:57:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803850; cv=fail; b=L4hinT0koQgi9jee+XmUVQQRmfjq9IOryzvJbiVvHZh2qSPu8tfASfio7ZHol7q/4KsXqzCuF2x/m1ZEn25txEOp0a1H5nPMCql90Ru+0GAa+Zyh5t7di3hyPRrvj0J9/sZJqW21sXoZYGhXxTeYyUEb7TaXP/rsBdjkNtfj4eI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803850; c=relaxed/simple; bh=XDjQhS9lPGXssL8EbAGlcBJie6Q34X1PmYxpIbDppPw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=dg+7JnQxZBMWZCIHwjA6v8smjwX0y4wlAITbcwRg+QYNF6ot8MVmhn6Kk4hdKPzTt4TbIUL1mnCgXDYBS4cZdbjnQ3IUTpNCqrzWe+zLntD7JohcyuMAYqslZ4+BZRupehY2fd8Os2mzbbxMXNQRXFtiFyPjb6VzOKZFVBMna1o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=NTz8Z3mc; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=TeNA2R6H; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ZIP4006358; Tue, 13 Feb 2024 05:56:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=SdJNpFEBBZnwiy8gXkBt3C6Eqpxv3bPK+R2Sr8Od6hI=; b=NTz8Z3mc3gTBZU5+l2q+u6rA1QZseRy1tKou+FsH3f69i2uEJ1BdfxTwhFmn0jhfwjeO xi2qoHA25meFB2XfOylrehHvvy3pz4n6rb+L7PRAZWgQlCGq/SoJLzjJobKuoP/GuyWk DNQ76ysYsfkPRaqs4lKEZAngyT617MaEJqaREfY806bXURVOwIHZOyYQnF5WFxTADoXM S2Uwg0S99x5qApgL8OSMuc9EPCFqO+Df8zQbHqprgS8yq8Aic1PycvPIA1zWWZj0zFts 0No3l2POstDJZbncUfc4y3f2tTiH10VH3wED4vSjzvKJ5d0t7BZaQYdo78tCMs3Pigrg gA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82d7r1gx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:41 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5uZxt014991; Tue, 13 Feb 2024 05:56:40 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6k8sr-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=neENiLYoUyei/fCK3+BqBKIdWbN+IV4doYpijYGYsl+M5va8k3n81SmLLY6Jprnl8W6eZF8uYqC+tp2uR/k4TYGV80UqbpiwYc4kbw9HLUo7Ter87VertG/rPe5gff5aWsjaadrv7DL7kMEoa+j8HTSDbLrkSMJTANWIbSEJ6HE2T9F/Gg2C1pkrtlt5v0zuunr2nIPZD6ZD3KHbdpQpwSnRf/IRUTDdYoeNE7LOdvRE3C07MTi+I2AFpteWEahHoYlDOrKRWRC5wWUqKjBujU73O7yaYaCrkggZu4jg5U0ZBRAHL4vsq9RMuAbxF/wN8IlkPXGMY9wr+40Sm+VxZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SdJNpFEBBZnwiy8gXkBt3C6Eqpxv3bPK+R2Sr8Od6hI=; b=MGje3ZTq0DuwyTcyj+sExP47lSFMjjbz4vqbGDIJD+KEYx40v8Bi0Q7EUI+GGO/ANw+zxzILtf/5LdnjlNwq+FS1CNQvw2fZG4k7hMRx1VW650dk1e0Waz/5qvk6pzGdR1lxxSaO5jDQHp5iq65/JPHFBgdDo29oKmceBngr0k+7P1t6knKAhiolukwOJzeoJjg9O0/4qt99oyEP5BRnKClwsAfs/qOFbXylXOQfDkbAGR7ld1X4NitJwXpQYL6epWtgmXB+x8rfLB1ZXDGu0GG+13I9uUnjriPtALoUdL6T2ro4/3iuupp7kYS6MXhz9f4SBlljtB5+ahoToAzj6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SdJNpFEBBZnwiy8gXkBt3C6Eqpxv3bPK+R2Sr8Od6hI=; b=TeNA2R6H959YY18xtJ7+vU+B50Jmj4GOKfDaU5eYZVvZ1o8xAWbkqBx/nnzSM22eYpzSNVh4ZAMjsL15t2FYhWnnx7ON4rcb1zKO00ji66GLjtygA5u+5MVx3Ps6RbjrImsFWRXhEFrACHjXWXRVlX/SeAYNI1ftcBX0iPxq18Q= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:38 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:38 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 22/30] sched/fair: refactor update_curr(), entity_tick() Date: Mon, 12 Feb 2024 21:55:46 -0800 Message-Id: <20240213055554.1802415-23-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0137.namprd03.prod.outlook.com (2603:10b6:303:8c::22) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: e95b355e-8ebb-4a54-4956-08dc2c588b5f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gQPazGmPDoBTCMszzvf/YOQ8ICqf7z2OVWC/605Ox8tinCl8jsCW9FLDsVdjq/Jh6R+6AeGCT3cBZ5MuhCRLTYRsQRxHBAeqgNgxfPBiyUu3ULbnVrnQ31io5jCj+SrYfdUs50yyeUTNC3D3iaKjQpMzA9Y5Fr/vPfJGjyHKTiksRBcjM4m8VmixwvG997r9KUMUpDP1YwxZXrXG/iHwKPNjV7BjFoDkf5PWbWRxlRG7unnzMiLv6vhgyUvdbIh6mYQxfsxLDAx6lXmtUlN4cfVST7sxVA/nDKpubGu3fVstLXnZzTboTGM0WCw3XGLUxTu3I/dOWtHKm0HvcF8bvOJCgK4eTp2dqGL95szOkWL6ERW7snqBPi0auC/q/CpylfaEB0IkE5pCKu6muusr+IHXaUItOP4NlyMcZwtbB2kqbOaueJSDeAm2gqFyO47w3HjX30JtBFv2Knvo9JkA0D4C8iydpcHGjjMeG2CQnMc1Odao8PpbOlB8AfoSqP+/zyvHYGrbKkTCpgGuYCjunH0MTpto1reWB3WDgPBzwQjHOcS6NBVD2ivbUGTzf3U4 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(15650500001)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: iEZi7uGslFmOkVWCV2Hi45kmAQL6GS4UxfsPQO9ZDW5px9wCGd68B/SGXy9Ol2MOLmH1toU6UHnBJ/5LJH/wK+rs0RdGeTlMxAf65gadDi5Xdpqf8aKRshg1rmawGZUmxAzjATtKVmQBrZ76HozfJ+c9CmVdwQf/B5vB9pgzGogvomG56f/hz6+3X+OWhZLrEJoR2P3c5t1dLLCTQPVpHqoAi3lW37GRv1xPA0uTUHHIc30890/ciqHROtaYZbSBoWENagVAxYVAcjmxBRsjvUKoUcmZN1uhK3aiBa9a822MzQfzsf5I6/y6frTGPCGd23FNJbMR0K95oEr5uqYHGtkW7WAZuEVOPqFD+ts0xDiuaPrVjFbG5PYs3fq0LcNRIwR2Vb+2JPtapTzY1EP2E17OPwKqE8yp6tL42d+zEUSJW4vOmfhkxCqbH5KsKpIZRbOEIZ6Be4VXSh2qs9B4fRyfi+EYeQ7xxNWILA663q8I9VgTOjFF/U2giVxnMZ0evEDPuB2hcLsSFGJg4Pp3KJd+6MZet3ynunk0DTzm0DgvHjV0p9wASIFD0uemh8CpgpBZ+PHMJL4+t5dzYoMrE/BwrDyHMuRUgJeThbQFIL4qedydcmeOn1UYRDZZJ3wjtmUm9AgBCOmdrBpygeSN5jYqnS7+WBk7tgiiOf8bJyAFAHQ7RwiO+3y0ZkkQT+gTZj5S4Hxz0VTPpuImCp6udx714s+f0AN7UkI5fTsomLl+yWtE4L7Xin1AxZliZKNh9kzhdIPeSYnwbm5tl+krqWvFGEJVi1k5RsmO9fcBPupWBMMHg7ppWMtlH4bo6X1jVEeM4ihwgRtI71FZcwEuYFKATdlWflGJOYn2K7T0XoabkUp2mynslB/Rol3s5BT2qlCCW/hBGdkgC15DGPlQkXSVKnuLyqLWags60voAZ3XDC35CLNtvyJtltLpfIYa7kiklRZm3+IAWxQPwx4SNPPRMG9775sL34nt7H9DhiVl8acwVLvl7chjVLN7BvCQstrp9HPg28xedfjK1ReaArzo3O3cxM/rzLTm5k2vnp0DTXYkvh/2ohWHxIuaVRcI78XjvPxA779RC+Q34x9Lr84H0i9GW0yOZsUyIW70l88pQUMFRTi4XoTfO4X826+OUnruo1lAdzV7xd5bli3gz2H2zSln1OTPouqsVtyrw1BV7uuT9ZntQ+wIuDuQMqZQRYVRaNpvpL5R/oV5QFFSWcE5W9wTkpllPBV6J1piso+FTCTJXrZ+omqzXs3PopJDVaJatE3MlwNoYYC/xzmcmTNyZsy242Xv4PzBh86rVPDwpAjIMiJoVDqoIbr1ITazENQk5y0f81YUXIxZQXcZFu0SxLIlFdyWbR6LC/+C+zoFyU2BRJMxHYWrrwqhJSkieHX8YubPdIlMKtMcDqFn6arQPLxv+xv59VSUntwcWEFjbC1dJ467BGAEXM3OeNIZiym3rLocdLaiykI8nhvyt2j8+5PikLpfPvqExx4qGOmPlN8STFnk0Nvhse6pYFez3R5u00H4dvz86yH+8dKb4j35rHHXkDzO82oHWIEOkNETehZwVKB/wdx8GARNmfVWWsiE08GeNgZYBSRsoMl+STg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: mkJNLTLXgJMFiUFXR7xs57U1ooJNsqw6CvUQG1fFj0lZ4tuXygAq9lTco+lT23lSz9UvDl7a0r/y0bsquBE7DENYaH7mdX4FGDdh/W2TrL1Kx0/I8RX9cGiMmNyTfUDbz5/9lNytlRDrQ5NrvXlc4zHB/+QMZKiKeOW1eFIB1G6Cd5bLckaSJRV4DcPhTllGn3/3K9mq1RlfnUUKlGL1wTJAPZh0xcbXiulxQRP/qWNNyMEuVLrNReBfckRdtbeU+NUfS31YD6H0I/XvMc/pzk8wUNdpDMyw4r2hClrA7qaLeXh9UR704lq3dwTK8gzRxYY3Dk4b9IXiUVsthpw0fGWHydSMDAhe9cl+Q7wBMCv2wO5z1Mo1hGeN/um2wQMecA3VOlPIksm7x2ewrq6uvqgoT8YSQEAsGm3/k87Q0qcgCcSuoFO5YkbxP1AEUJr9AzQD+T5yYaFosgNiekpXZKAGlRYK/QRlaf1exHp/8EFaFOKpgDf/FzD7N+W2gQhC0lj8cpcP6y0E14spZf0N4hoMDUMKwnFYOhWuMAqvxdF5RZz+0R1WkJRwagW7j5nN/sen86xxhWZZ+NZGKCAsHDDuyGz6WwWXG75qhJhVF3g= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e95b355e-8ebb-4a54-4956-08dc2c588b5f X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:38.8141 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: P4ujcLPPrnHArHoIid3mQNREUFDIApEe++vvlaz2udrgUdO5jBpoEWBUibpDDtFiURNe6Zp3yKwKFS4UIUA15RmPEY3a2Zttg0b8e7nldgo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: 0RdEmUMVz6onagjUqz27alULBnpRVvBV X-Proofpoint-GUID: 0RdEmUMVz6onagjUqz27alULBnpRVvBV X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763042520600856 X-GMAIL-MSGID: 1790763042520600856 When updating the task's runtime statistics via update_curr() or entity_tick(), we call resched_curr() to resched if needed. Refactor update_curr() and entity_tick() to only update the stats and deferring any rescheduling needed to task_tick_fair() or update_curr(). Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Suggested-by: Peter Ziljstra Signed-off-by: Ankur Arora --- kernel/sched/fair.c | 54 ++++++++++++++++++++++----------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ae9b237fa32b..278eebe6656a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -975,10 +975,10 @@ static void clear_buddies(struct cfs_rq *cfs_rq, struct sched_entity *se); * XXX: strictly: vd_i += N*r_i/w_i such that: vd_i > ve_i * this is probably good enough. */ -static void update_deadline(struct cfs_rq *cfs_rq, struct sched_entity *se) +static bool update_deadline(struct cfs_rq *cfs_rq, struct sched_entity *se) { if ((s64)(se->vruntime - se->deadline) < 0) - return; + return false; /* * For EEVDF the virtual time slope is determined by w_i (iow. @@ -996,9 +996,11 @@ static void update_deadline(struct cfs_rq *cfs_rq, struct sched_entity *se) * The task has consumed its request, reschedule. */ if (cfs_rq->nr_running > 1) { - resched_curr(rq_of(cfs_rq)); clear_buddies(cfs_rq, se); + return true; } + + return false; } #include "pelt.h" @@ -1153,26 +1155,35 @@ s64 update_curr_common(struct rq *rq) /* * Update the current task's runtime statistics. */ -static void update_curr(struct cfs_rq *cfs_rq) +static bool __update_curr(struct cfs_rq *cfs_rq) { struct sched_entity *curr = cfs_rq->curr; s64 delta_exec; + bool resched; if (unlikely(!curr)) - return; + return false; delta_exec = update_curr_se(rq_of(cfs_rq), curr); if (unlikely(delta_exec <= 0)) - return; + return false; curr->vruntime += calc_delta_fair(delta_exec, curr); - update_deadline(cfs_rq, curr); + resched = update_deadline(cfs_rq, curr); update_min_vruntime(cfs_rq); if (entity_is_task(curr)) update_curr_task(task_of(curr), delta_exec); account_cfs_rq_runtime(cfs_rq, delta_exec); + + return resched; +} + +static void update_curr(struct cfs_rq *cfs_rq) +{ + if (__update_curr(cfs_rq)) + resched_curr(rq_of(cfs_rq)); } static void update_curr_fair(struct rq *rq) @@ -5487,13 +5498,13 @@ static void put_prev_entity(struct cfs_rq *cfs_rq, struct sched_entity *prev) cfs_rq->curr = NULL; } -static void -entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) +static bool +entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr) { /* * Update run-time statistics of the 'current'. */ - update_curr(cfs_rq); + bool resched = __update_curr(cfs_rq); /* * Ensure that runnable average is periodically updated. @@ -5501,22 +5512,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) update_load_avg(cfs_rq, curr, UPDATE_TG); update_cfs_group(curr); -#ifdef CONFIG_SCHED_HRTICK - /* - * queued ticks are scheduled to match the slice, so don't bother - * validating it and just reschedule. - */ - if (queued) { - resched_curr(rq_of(cfs_rq)); - return; - } - /* - * don't let the period tick interfere with the hrtick preemption - */ - if (!sched_feat(DOUBLE_TICK) && - hrtimer_active(&rq_of(cfs_rq)->hrtick_timer)) - return; -#endif + return resched; } @@ -12617,12 +12613,16 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued) { struct cfs_rq *cfs_rq; struct sched_entity *se = &curr->se; + bool resched = false; for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se); - entity_tick(cfs_rq, se, queued); + resched |= entity_tick(cfs_rq, se); } + if (resched) + resched_curr(rq); + if (static_branch_unlikely(&sched_numa_balancing)) task_tick_numa(rq, curr); From patchwork Tue Feb 13 05:55:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200223 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349477dyb; Mon, 12 Feb 2024 22:04:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IEKv5ZdCTep2rmNMquLIO3A2G38ewT76tegpxHamf7ZJ3QJh0PiWxR06d2pqyvINKdw3oWw X-Received: by 2002:a17:903:32cd:b0:1db:299e:2567 with SMTP id i13-20020a17090332cd00b001db299e2567mr2795217plr.53.1707804260965; Mon, 12 Feb 2024 22:04:20 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXU8MxW3kJhgYGbY15L70wJH/0jmNxMpYzo2Gj/1j8OYIU5xLAHKT1nQEf/yXqWUSWPMX9g9e3KLn6PxijEI9JnOjTb1w== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id f9-20020a170902e98900b001d99acc35afsi1470754plb.192.2024.02.12.22.04.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:04:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62982-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=Qf2sXnb5; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=pxKuNoSp; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62982-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62982-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id AD767283749 for ; Tue, 13 Feb 2024 06:04:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8947E44360; Tue, 13 Feb 2024 05:57:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Qf2sXnb5"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="pxKuNoSp" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07B553B290 for ; Tue, 13 Feb 2024 05:57:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803863; cv=fail; b=P+vgoo3wZeuJSSS+9l4eaFrJ7qSrXQ7H1M7rG3VuuA9G4RdvBcn+GN+lAz80B5LFKRJiJcT2rESJ4cT1P4u79WNDofSOzfsig37oe1neUE/57EHjNV3AZrRZgMkte6dEqHxCQ63pNLTtoHyGWWRvcfHffHD+4TEaHHT5sxyyOTs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803863; c=relaxed/simple; bh=wo6GlQe+fRdKnWucgtncVQ/AxrfSTxoZxHao/cBgfuk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=JMa/D1n4J3BoLbCHgvuUryuTAjnuLRQG2PFv5NNXumIkNhrTSqPgrb5tevsJQSAC6bSAynH/SkbHbj0ToOAUYVgNrCz0awq6IIH+yBsQ8Aq5TW3dmjt12xd0hJ+S0px0bjhHwZzfELdnrDG7HNHXpmTVG3HhXgb5FlvosH4dxvs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Qf2sXnb5; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=pxKuNoSp; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5nC7O013768; Tue, 13 Feb 2024 05:56:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=2iwGN19TdbWEak2p/YVPf2El8bxLr84wbWeeHyyM05g=; b=Qf2sXnb5EyAeDNCAWcVAKtIIWYB82A9abeY+9Q6qQmBSUyuAkVLklNEcBL7MQWky9Tux nUFkDEgFQyDTdYrYxktk320r2UNfWLDJ9C/og9cAR4y+35cXCyL4sVaquMLEWRTcF2PF 9BKtQR9bD1mJSYIdLb/BY5e4PXScF/j+oX5AoM4m0pUcFwigj6XB2byhP5CZsQKRihAh Ya9sop1y4v8zBSCJvsW09EjxF7oErS112rV/ZxIGNNV0X+f1o8Mq18IUjPtGHRJWV81x 6Tc+8/DjoKPcAmh/9wX/tON546xRc/XmroDA0SQ+VT2ePm6jwQgoKUzomxdkdn+XPZg/ ZA== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w816tg4ah-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:45 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5cLmh031604; Tue, 13 Feb 2024 05:56:43 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6twk3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HAGznIBtIX+nujt7dZYvv8obm6cNXTydgs9xeOu5qRNx9tNPXJ8Kn2vpaE0VTikADUHcfuEl17yLIhd/rp1Wzp1BEnJbFC6Waq6Gla06onE3H0IWQE0pLvkuE4szsbNN/RvyV/lmT0Kk0HF+jpoPPQYCDpZ7iLTzryRzXCTvZ7QzMuBu+9k5dQdAmdepDvztC88HS0YBS7fYr+JP7CTtY6DYBGMg6uVJ9L3n68rFUB0JE0gwk09nCtDUUYg4dRgXxuOcU47qMx/zX0gSc73L/lcMiVulilv3FmQkVl+0NtOfXp7N1OIGd8D6gG0vD1zf4OVL4L/fZeMp6KDIfj2dRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2iwGN19TdbWEak2p/YVPf2El8bxLr84wbWeeHyyM05g=; b=fif3qHsJitt7+QzVFNqATH9d/IQuQemDFlOsAs37byujnsO7B017WOITYnA1mYhkNKGBGyxwWQu0NuQXXwA7RJOCaHZ40suRp4HL2JEP1lUIysUtmnkGl4fuCN8doeE71ATJCOPEHMdc3ukMiXlTiEksuZwhJ/ZCGfXb85smZbEiu0AowizeiedEtmrwBzAwdFbCn63wo3Jlwr0d+Fs5Z5yh+dIA08TK16vQ6WElJMUpomFlIIzccgn9cYcI7n3Zhmq4qHxexdDEU82NLP6fxpV8Zes7mj49d5HAn+/5fr4QQzlla59IaZIzS3JFavE6/zZJdKf4oDzyl2hpk8CjoA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2iwGN19TdbWEak2p/YVPf2El8bxLr84wbWeeHyyM05g=; b=pxKuNoSpa2hGSZuQhPZOCIcb0at2didyHpXW1eTsJi3rz5JVJVHFfnMGCJVQQNpkza6UAq33wGEKDyX3HeWkyXaRt6LByAlq3jqXKD710cpq1y4WJ0/08cozOoAoTl6m0Kxhmcrldja8z8xPuT51paEvyp/nxV37inyUu4sZRQs= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:40 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:40 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 23/30] sched/fair: handle tick expiry under lazy preemption Date: Mon, 12 Feb 2024 21:55:47 -0800 Message-Id: <20240213055554.1802415-24-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0151.namprd03.prod.outlook.com (2603:10b6:303:8d::6) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 21ad92d1-6259-4387-0160-08dc2c588c9b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +lHfsbbYnrduRU6cRj8Ihf5pK2UPIBPqYOANQ27bfIIMmIMQTwmPxZYeEiZpj4Ixe29VlwdaC0GykqHiJgKJqDU/s/0wKaXqzDQLAF8q6wl67aL16VyJd7S7XfWSGHvanuZTuCrGJw3dDhyrsIOUR7PbG0w+AGIiH9k68+eTYejnmyYUNEIxYInBC1HATqQJDfvhoalKdHr8CaBvJAyy4shiDIU2EFU2ZmboqTOSe7bBsZGUgz8FV4zyL866Li7qoC3QvJYQsL3ImPDpv3vMZGFpplf0C4hHejccfuKQklZScAvdqOpUrgGSzKbobmsQkU0CRDmiBJmEgts/9q/pR+zKteZDS5Av9+RrA9NAlmH/7W0EzSXLkKXPwSKu/AN8d3qqQ8l8dGbBh8S5ivNGaVkUdACgi93/wPh4V3dVKRaN6CHOhhvPIG8JIf/oJ56LwPo1vSQBzgLx1uy5+GZughbK/98iBJYB+kcFX+tL9LKp9YK4Q5k7j1ubEIXy6tBXv+bt95ghLJosML/awv5NLb+hL+EMPhvcqAYke/REEEg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DxsEqEqIk9CiFL65mDfVKA+SQz1tYjD/0M8kcgoFOA6FfsrmkMzq/6VYaPtzRdom6hYNnAk0GKU/KzfQ0TBL45j24DMke2w/xekMJVfvEJMMrYn487VR0IkfNL5EcF3haBx9YTXH2Di/WLAKVQ46JiV+MZuwLBX5dQlbQx8uGPGUpcFL1KYfFXk57Ns70OJVBmqvjFGCjFo3zbD9rm7HY+2x2UNZe0sAmpfkWAoY5E11u7xiuzRQL3pjs9kV7uonQ/t0sO8g0zTOU010bo06f/ECCO0UVUSkA6QP+JAJiD3QkOz8edFUJ7kbT6eAjG3QvJZAZGBaJSmfjWr0qhuhA6b0HxaO3plcOsRs8Indyu8i3m9IeIqqJ8rCFGs79TI4iBW8gwuDrQzfEVW065clt/kWRBBUPaY+Fr/xskc0H+D29FPCIH1o+qpFu0JRGxdN2RBOmIvSoMq/tvQe2prbemYnsKL6LMMKWkQ/fKCT4afQNkmjO1tGknK9spvuW6Te1b4nfcXbgkUsvg08S+aLWVgxU8II6QWZmAH9Gkf7aWIupBD/Qe3Go81QsjCwU9g2xrFeEbAn58+Z89oE5ONJjWoGc9JPs6oTSbix0IEdQ6gEoREC2mYbVzg+2ZiYTkKjWF8u3GX7k+arjGePZm3Y4kE444fCg7Fsu9fD5yWaU4WQWCX218edQ9Nkldq9x6b8Zho8TxeM6/LvG6sgHL2w2p/eJDyfak9M/X8lbxSeJVFG0+ejOa26YGa7bTXSFcHEXtt88CNGXM14n6l4aYcyHGhC7f+4p4NLXGQvxyPS4+DypLJuVxhXmxh94G2HlZlyqVnv0r+h8WM1vXpC0YzlLgV4u6JIAtGeNKUpKUoLL8MqUtHr6cT9lozJZfo1bx6Q/VcOacAUWcLFvOcRc2C/ol+dx7dZG7e7j0OZ2KXIMDkCeijn5wcQeBLaIm7PABZVZN9L1zRjAZjPf/cLJKrvUzvcZ3F/JDICwdh+UCiyRgWCDhfhl3Eq5aBrmcV0fgemRygn6V/vP3tCuIXBybR0MNL5H1e5Q6CqL3Nm2S/LaWdPV8SeA0kpPHkCIDgsItkSMEmNFqmJLcrNNAjFDFZfs9xfUY0AZ0F0ieTGOtex6ONqGuUEub9a+cGS1INUrO+5V/hSKAzHRC6Ej9HAUwoJi6uLpkU/MrM6pwRWRGIwmvVBZWKLklZsOyxTNTDoqjEGLikACIdjqdSnzt0TS3vDZtxBSHOr1UTS4wjXLYDa93FwXiY//sdGvaAvw/ciZrfi5EwR6sQ0wIbrlniCzoHyrIHRz4ZDJ2EqFMSjCkN8ARqya3X6XYAd0py5Lb3Iu9yBWxP/woTsGsOTg1uYmMvJw4zoaWn5ZcEexK6NrzWqdvaJso46cMwMVmxG5kl91sGsCeQRK05EGw7t9dY6wCE9vOOalGoAx1rucJLY6/pIvAPFqHUnL+2/s2KClx57p5T5CsEUqLco4w+dEYzyjysjVWgv0NDe/mIhqxZ1O2u4C4qm3Vq55TqCPywcqI8GLyGnToSx4H+1fhtl7qFPYtJcFDdMVnFLJg4tX1nnCTAx/v+1+m4+IPHy9gvFGNwAMOOUibHNdbCjO6RXYu95MfmC9g== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: aj0et3LZ0UTuw/dhA+geDfJQOXmwmY4878wmHWw0hVt1AV7/yb5+ROrFnPjjl8mDOHStRELzIP4nKfcA74zqrtjVzXMqH1sC4KpT14OzHeIFRRBoup3pFxQRl3ZAy4RLy73jiQLslc+nbm1AUWq4MX2dTRRmXxBVGSFqQeICU/5Lh7pUzOpDr8mOkAc0CMCjRqm7b711+WyT2rIcSF6s8hLNe14a07JsZx9W0LkN8lUAirAqCJtRk4rI539Z8G8vRQyNoyD+bnmPApWePzjud3IryWAsWYv+R2d7/R+Z5uTqehvXLsTinufo2ZbAu6ywKbUig+a8VGsiKWbUmGlWke5L2/+RhujPbNdZRmaeBEeGFKWWPkAe060VYWeyG28J3hHqt5iKg38weIE/0m81I3I4wGqiZyUJ/ahzHzYB5p64wxBItfVMskdpApcwecePOpUiEAsYZ0ZHzyBe2bsdyVnl3zPZUP9OvipZsC/HbKGjLt0q2zAm0qAZ3n/H+B2fdLsaq6Ak9DyiHBCvrNNWfoA61Ni6aoFpyYXvk61Yd/w2o3VZmvIqKJOQu8k/2K8zGX7dvn8jusvMrhYMf66easEiO03l5Vmkpn5jU6XaCJ8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 21ad92d1-6259-4387-0160-08dc2c588c9b X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:40.8677 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Uz/+7iWBVy1M8V+sSv/bs4VZnFn1K/zWJ24NNgwLyj8PNwGFH3AW84Cpsel7G0IPfnXdZWJMp1zbP+dzot6QB4lKG1tbu31Jby0WDZrXrr0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0 spamscore=0 adultscore=0 phishscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: am0ulBMsYDuITzzGysj9mPcae3IDlBxw X-Proofpoint-ORIG-GUID: am0ulBMsYDuITzzGysj9mPcae3IDlBxw X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762561118008701 X-GMAIL-MSGID: 1790762561118008701 The default policy for lazy scheduling is to schedule in exit-to-user, assuming that would happen within the remaining time quanta of the task. However, that runs into the 'hog' problem -- the target task might be running in the kernel and might not relinquish CPU on its own. Handle that by upgrading the ignored tif_resched(NR_lazy) bit to tif_resched(NR_now) at the next tick. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- Note: Instead of special casing the tick, it might be simpler to always do the upgrade on the second resched_curr(). The theoretical problem with doing that is that the current approach deterministically provides a well-defined extra unit of time. Going with a second resched_curr() might mean that the amount of extra time the task gets depends on the vagaries of the incoming resched_curr() (which is fine if it's mostly from the tick; not fine if we could get it due to other reasons.) Practically, both performed equally well in my tests. Thoughts? kernel/sched/core.c | 8 ++++++++ kernel/sched/deadline.c | 2 +- kernel/sched/fair.c | 2 +- kernel/sched/rt.c | 2 +- kernel/sched/sched.h | 6 ++++++ 5 files changed, 17 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index de963e8e2bee..5df59a4548dc 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1038,6 +1038,10 @@ void wake_up_q(struct wake_q_head *head) * For PREEMPT_AUTO: schedule idle threads eagerly, allow everything * else, whether running in user or kernel context, to finish its time * quanta, and mark for rescheduling at the next exit to user. + * + * Note: to avoid the hog problem, where the user does not relinquish + * CPU even after its time quanta has expired, upgrade lazy to eager + * on the second tick. */ static resched_t resched_opt_translate(struct task_struct *curr, enum resched_opt opt) @@ -1051,6 +1055,10 @@ static resched_t resched_opt_translate(struct task_struct *curr, if (is_idle_task(curr)) return NR_now; + if (opt == RESCHED_TICK && + unlikely(test_tsk_need_resched(curr, NR_lazy))) + return NR_now; + return NR_lazy; } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index b4e68cfc3c3a..b935e634fbf8 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1379,7 +1379,7 @@ static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64 } if (!is_leftmost(dl_se, &rq->dl)) - resched_curr(rq); + resched_curr_tick(rq); } /* diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 278eebe6656a..92910b721adb 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12621,7 +12621,7 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued) } if (resched) - resched_curr(rq); + resched_curr_tick(rq); if (static_branch_unlikely(&sched_numa_balancing)) task_tick_numa(rq, curr); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index c57cc8427a57..1a2f3524d0eb 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1023,7 +1023,7 @@ static void update_curr_rt(struct rq *rq) rt_rq->rt_time += delta_exec; exceeded = sched_rt_runtime_exceeded(rt_rq); if (exceeded) - resched_curr(rq); + resched_curr_tick(rq); raw_spin_unlock(&rt_rq->rt_runtime_lock); if (exceeded) do_start_rt_bandwidth(sched_rt_bandwidth(rt_rq)); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 3600a8673d08..c7e7acab1022 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2465,6 +2465,7 @@ extern void reweight_task(struct task_struct *p, int prio); enum resched_opt { RESCHED_DEFAULT, RESCHED_FORCE, + RESCHED_TICK, }; extern void __resched_curr(struct rq *rq, enum resched_opt opt); @@ -2474,6 +2475,11 @@ static inline void resched_curr(struct rq *rq) __resched_curr(rq, RESCHED_DEFAULT); } +static inline void resched_curr_tick(struct rq *rq) +{ + __resched_curr(rq, RESCHED_TICK); +} + extern void resched_cpu(int cpu); extern struct rt_bandwidth def_rt_bandwidth; From patchwork Tue Feb 13 05:55:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200222 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349211dyb; Mon, 12 Feb 2024 22:03:39 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX46hYuuPP+gasUPZ6NQEPp5RtxPWgZr/pdIGfbWNWpUa05qKoeDiNt+68S1ld7RcRvSJT0KHfbJyMcpffv8TzfgGgf2w== X-Google-Smtp-Source: AGHT+IF6JV3pyUZI0dj3tSIAd0HfzZSyRdnnJv0IVP1fk4YAF8H5TSvPPHH9Iu+feHpQZ55ytNP1 X-Received: by 2002:a05:620a:214e:b0:785:da5b:31df with SMTP id m14-20020a05620a214e00b00785da5b31dfmr3240494qkm.73.1707804219127; Mon, 12 Feb 2024 22:03:39 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUBq1m/2VYCzx8zLsV6IBSV+kuPnRmBpZzB8xbAFg5ofyLePc/3y7/xwgv1IxfqdUE1JrXjTc/ZgrmOSPSXe8/RmY/tWA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id o15-20020a05620a110f00b00786f0d7488fsi2196910qkk.87.2024.02.12.22.03.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:03:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62980-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=QzwuVQct; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="yH/zQ/MH"; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62980-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62980-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D87661C22A5A for ; Tue, 13 Feb 2024 06:03:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2365C3D0D4; Tue, 13 Feb 2024 05:57:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="QzwuVQct"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="yH/zQ/MH" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70B0538DEC for ; Tue, 13 Feb 2024 05:57:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803857; cv=fail; b=J5EwoiZw3dDT9E1KJjIDpXjKuZOVi+zv0OO4dMdAAs3TEsuCppuWkK+VwWO8fgVPIp6hmTBgyoH4uucSq5DtzY1E1bzJ9dXZXUEtwXrzcEIcutnWw8Z7xsf4M/PEfyDPhNk9RYmzGs8BTtzsyygnn+RwyaUN5s8TYW+4BWm3iKs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803857; c=relaxed/simple; bh=Mww0LhqcfHd10l812lIJRHN3IvxONYhQShLcipo36aU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=HA3YRNOBUVsqj3nvSd8oMUP9UGubGMa3tNHfsllxNW6I5O/pS2YPC2EYmdNWT/AkA3nPGgITdmypWvWZkrOD78nPDzTKeaMfI4Y0h28O90Uk3ANMky121e2BRcfIsxR12xxY4gEDFehtzPvyp8kJiGcIoAVcwSRgT0zD00jgEa8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=QzwuVQct; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=yH/zQ/MH; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ZIP5006358; Tue, 13 Feb 2024 05:56:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=bs2ravu0cwCgK2dzh8jR6t2SDdXIwliCw+KKLAjFaWE=; b=QzwuVQctClNof8rtg4wkipnz87y12TJ0Cz8TDUkur4o6RNXdPb1IblVZRLBfY1jm2eVF JB/QigdS621cZhW/hOA1ap6oGjQtxWqOj+GT7gkmBKzntwifDoBs3CJhtaJ2S53J/PYC 2Fh6zb620VcEGd8stvouVQjTMQMpEMjXANrKCiXugVWYXBQu6xmetef0Km4w2I3yl/BT iw2H9vbeFjlbso2ROk6+OieVZS0hkmyljpdEZ7N4ZRrrq8zUMt9Fc/NR2xuGotbQFGAB +amWjfUG7eLPVsUpxap1PI4XYpqxkLatG5DoeTbR9NMEm5bU3NQKL/2F4ftrCu/An8pK xA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82d7r1h5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:53 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5aOFG024597; Tue, 13 Feb 2024 05:56:51 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3tmc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jPck2n7hizzyrZJv+KYUoT5iVQB3A6ewZ4IYxZa745G4m2/izzFVJw+WVW3xOwvL85LrC0tiKRCtftQikiXorQ8j0ve21SzbyT0ijwZKrid9tJNELkf11gske/oLj+6at63iVs6raBidT7oo3UVa47aOlyw3TcPb3/Dur4AE+OYZI8Jd/ySk+dlm1AfpNmM8MKpr0ON/IIbfMKyrIUGHDv89UKHIJeN/hYvdHnYW5xHPkGGhp/txM/UG+VhSNUt4C15WymhC04exwHurcUy2C04NJDaf539SjBlzxBzwRGPDn5KOc1WQwWLPNGrGcTg/Z8iBqRMedethQu9mFoy5Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bs2ravu0cwCgK2dzh8jR6t2SDdXIwliCw+KKLAjFaWE=; b=oV9lT+/faygPmqdf21gTG0nCU298RaIk6fZqNG3jHUtnwMIvl9pcvkD2RE8jXgzJ7JQDOjvvr9oFCz24sK8McWz7ebA1cV+LcX4jnOeTZBq+dtX13g0PQ2Eie+U3Y5I4rCTU3bThERnA+b7ffijKfvz87AaIHsMLZpztsgf0H2eXCtkQGwYmkkxVtStow71Zgsu+QwfCDPXTM3F5VAupkYyOYb2V65CLAykZ/KZ0ZBBDE9aN7VD9CGrYWFW4fGq/qmYGCaCX9yjxFrgLPNl9WC+Vaq/K+PrjiGvCo9AMJow8abJIdYCrldNm64QG9/DCWsz2Dzh1XRzFBcHQuU3/nA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bs2ravu0cwCgK2dzh8jR6t2SDdXIwliCw+KKLAjFaWE=; b=yH/zQ/MHEXLJgQgJM0ic7yDu5EfyjeUxTCSmqZIrXb4I+/e4ngi4B/KRW2g/ZJK+zq//PKoa31VVhJR1dQQUA/oWZJp21f3ri0sgPO9uOF14RUjaeRFXvgoiSM6we1jteyzEeDPudBJ4Mj8phuTjMTGaFH9B80OHXrTfVqU1ubA= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:49 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:49 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 24/30] sched: support preempt=none under PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:48 -0800 Message-Id: <20240213055554.1802415-25-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: LO4P265CA0091.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2bc::8) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: acc2334a-a718-4af6-f99a-08dc2c589178 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5wUsi1Wf+ADw2T+0gYYXJPCyw2naIeCUR+cwpP7I6o0CqJZuMDo74o1NWiuAmMaqqt8Qc8RFQF6HAWCkSQDwYPYNTPID1wZDTG0L1JP22XOqpuj5qeXnK7PAzNrM6YwI9N/107qD5r1rPe138w3hprSUiLrn8SSmbR1Lv/RsvTrfVJzQ37rJcpE1e/WaE6SwVShImLUlxpusZVLXXGzGXELkiet2kmuAaXQzu4N76k+fDHHCrDvCRRoSNWFRiS/vNbQuvdKpi2cPZ/KOayD8zfInMsw6m+F//xmwV1B+xOzyu2Gc3nOSTwCTpyCBvh4f1CLVj7gadr9gNrsRV31yD0VXPl3AwIOgAfqufVmezCoOTe11cZVQDvfhXuGak07Ruc7O5pdP5EEJt8uzmiGahOM4xaoL+WYIfPoI8xGm7oRJCSW/fO2I7VAUMq1yZdBn/j2vgt6jwmZTSDD/Pjgfdlg9866n2xvOQNevlBKql1II5813ELiPZjql8bLI+qdHN32jtZwK75UxhFs5qgB4vYEyNWE5zzP2uIAhOPfXNY4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HSf5bciRkztbCdhUqBpGpkrDAeJ7cNlnZT2LhwXgTxS63BuMeMpo469AYDdeegPIk/0MwMVJQLg78cr2dny+P01Ftxy8W+hriHl93cZG1kfqHT3gViiIBbXkDZeBiJHMgwzDVCMkj6aXkDZh+ysovrdT0BIZn0VWKN55qes8tWKnvFEhIvqSrW/Y8W3qI3FSC93PauF0zjG3lXYFYQzyvLtdNYl2e5mxV/9+BmD4KzV3TSiGJ9FReeaR830SCI/JI6mo8XZzTOof+iijZff5+M5VYjY0mdxs3zkOOSAWU6S/CkozqCVGMiEy6qaYxQq2j5p9KARFBxFLM5C9nL6GUFC0u+Q8r+m+rxd9EK1uAwGEg3WxNAv5F6TZnPLHlEGzyvNMJj6O5t+xPiyqEBvNqE3YMxc5UM57DRkxqB2o0E/SI7wYVBhsNhoSPeYLcoDgXRovIb3ecsUlTjTKY9sCf3SVw8CN41NXHqrkfmwE22os1AOhtDo19Qny4708RA6GMtDcVJQqcnkjyRt9KJds/96hP4aRbWC4c6jPCDDzkD89HN/wG7wPeN8K9Fv7vawIiUo2r6Ec/hKXImqODLm6HA7s/eXurP55HymUfsWwcVG5AHRBpHgBN1/UAZhgkwgrlTZKTaRjzvgy3BMhUSCj8NEptylOykTakk29PAzW9dGWs2Mxvn1IKo4sJCZY1paaVudl5inwwWgjZLoQKiHMQ1Pg10kcSlududwxqqLCrRxwEcyIu6q9mrGKaXk8dJ0ZdF0MvZh+h/hkTXts0uU+qrlSbpjCb7BPBZKKcln2MJEAAW+PXcRGSe2B5oMAcGgoV5p4SavInUi3kMcMeCRKIVLY9v/5TO3Tsckl3hJIwmsePDOpIha1oX/m9RYLy8KDW9usymDTxi5yyimfwjanIzqbp7IJth+bUrCUduGAYlLujiSh0pskdIxOip7RsbRNncfpjLn3ddeNQrd6xLhhQrCWCAAbisagoJZFl1Pc1p22XHcKY0ry1GyitsAhyL/+ek5+orWwZ2RO/V0se4kPeml7EVjHJb01xBeRzH19MVgdy/jJHA6M6ALaUZiN0/YYauBlPKLVduar/N6N7h5CnSQo15h56JZ6okOjTcltIWR6jMc1UsbRXlSK3l5v3C+lmHtJ7XtKbZ0LEnUwlHk7/1MpbomWjel5WSpNh7NsJdxoOa8Ea7Y1dE0a3tQ+La0X5evLGpuGlAZI7y38/A56rgrq36JAGB6g0Zm/SSGcYOHiY62o6Rm2Aeuv0cB/bMDbxxFQHadIcaRlQfFs2RAb4faqTKZljH8iERMluf24gdXybiNkjhqUWYVnE2iXWrBsv8XWz6nsmODJWqDvzbB8nI98cOzfxqCbCxQfEEJNOtw6zz0kxx5+3EqetZil2dkLsToBZUR+j8l1X8Nv1lCtg5V5faNx1JHDZ817JpJzOu0HlHVDSgitxFPRBktAiNwKi0AMwC2adqwRn/AsN5Z7ALREq5CjoL9gCjO7cJ6MPKMrnsXafGuwbT+5YhmJvUC2T8GTIpjs8mqjrR0240ti/NYg+bEZff8S+0H0yv/vDfXV1JiN6pToEWIo2o8N2DGyq+1s4BEhnTMFrpuHEt8Wig== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: lRHOTcHJvHDdogDE5NOKvzw7e/eRfdL79u/2wYZ0sfNkJssOuqHpsbwj78UR5wa2vKtUbvVyCRUW24PLAoWsk3XWhVjDBkgJpzN3VXIlwB/SqaA426znzHLULAe7SGoRrCpYevUBQfhGDFsNKl1O073cOvdOy0q5YFU+kUJWRBXHUBtUhzmIT58K7PjodK1tmd4PCZddEaRjPJNeRIXFIJvE64L/JPzKfrGDTGzvOOiUMh1AEU29f7bGOhRik5CclddgzZ/O4mfMEnq0rnz5FhHw1A5iKCmimnC9DRBozCEX5W4pi/M/Il0kS1OArwP/cxLO/jLzY01z5sCBHUAORMK24PDN+HQTLCcddMGqf9Wi/iw7ddtqdLJtv6PkzQtXmd36b06YTe3BzDiKRFeoOl4BpBHZ0YSZwjUmW36mqFM3GxBAx824bM5NlEsLUpuiuvWMSHGX0lq6FfacBdv/oq6AJ4qz8qXkLDcpIRzAicfDrVM2A/0E57ezUFvsMPe6D5ySLWasOK81IQR3+nXfwQuo4QNzKtgYgeLoV9kLtB8g9U8slinwbbc0RocxQ/8v8Tz2htAG+p7JxjQlLjCdsmKr+ORrR0uZsGjbiK5K79M= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: acc2334a-a718-4af6-f99a-08dc2c589178 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:49.3059 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zGkd4Me5cL0Fz8It+vEz+xsF1PBLHIjM97PW7q6ikGdhUeBv/P4i72+zVR8TA5/g54OytM+lizScj39TtfC64j26fP0OMO4KWdo6CjHADms= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: 3r-F3ECbcNT-BSRGFXVfrcseIKsoCuz8 X-Proofpoint-GUID: 3r-F3ECbcNT-BSRGFXVfrcseIKsoCuz8 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762516678057428 X-GMAIL-MSGID: 1790762516678057428 The default preemption policy for the no forced preemption model under PREEMPT_AUTO is to always schedule lazily for well-behaved, non-idle tasks, preempting at exit-to-user. We already have that, so enable it. Comparing a scheduling/IPC workload: # perf stat -a -e cs --repeat 10 -- perf bench sched messaging -g 20 -t -l 5000 PREEMPT_AUTO, preempt=none 3,074,466 context-switches ( +- 0.34% ) 3.66437 +- 0.00494 seconds time elapsed ( +- 0.13% ) PREEMPT_DYNAMIC, preempt=none 2,954,976 context-switches ( +- 0.70% ) 3.62855 +- 0.00708 seconds time elapsed ( +- 0.20% ) Both perform similarly, but we incur a slightly higher number of context-switches with PREEMPT_AUTO. Drilling down we see that both voluntary and involuntary context-switches are higher for this test: PREEMPT_AUTO, preempt=none 2115660.30 +- 20442.34 voluntary context-switches ( +- 0.960% ) 784690.40 +- 19629.42 involuntary context-switches ( +- 2.500% ) PREEMPT_DYNAMIC, preempt=none 2049027.10 +- 35237.10 voluntary context-switches ( +- 1.710% ) 740676.90 +- 20346.45 involuntary context-switches ( +- 2.740% ) Assuming voluntary context-switches due to explicit blocking are similar, we expect that PREEMPT_AUTO will incur larger context switches at exit-to-user (counted as voluntary) since that is its default rescheduling point. Involuntary context-switches, under PREEMPT_AUTO are seen when a task has exceeded its time quanta by a tick. Under PREEMPT_DYNAMIC, these are incurred when a task needs to be rescheduled and then encounters a cond_resched(). So, these two numbers aren't directly comparable. Comparing a kernbench workload: # Half load (-j 32) PREEMPT_AUTO PREEMPT_DYNAMIC wall 74.41 +- 0.45 ( +- 0.60% ) 74.20 +- 0.33 sec ( +- 0.45% ) utime 1419.78 +- 2.04 ( +- 0.14% ) 1416.40 +- 6.07 sec ( +- 0.42% ) stime 247.70 +- 0.88 ( +- 0.35% ) 246.23 +- 1.20 sec ( +- 0.49% ) %cpu 2240.20 +- 16.03 ( +- 0.71% ) 2240.20 +- 19.34 ( +- 0.86% ) inv-csw 13056.00 +- 427.58 ( +- 3.27% ) 18750.60 +- 771.21 ( +- 4.11% ) vol-csw 191000.00 +- 1623.25 ( +- 0.84% ) 182857.00 +- 2373.12 ( +- 1.29% ) The runtimes are basically identical for both of these. Voluntary context switches, as above (and in the optimal, maximal runs below), are higher. Which as mentioned above, does add up. However, unlike the sched-messaging workload, the involuntary context-switches are generally lower (also true for the optimal, maximal runs below.) One reason for that might be that kbuild spends ~20% time executing in the kernel, while sched-messaging spends ~95% time in the kernel. Which means a greater likelihood of being preempted due to exceeding its time quanta. # Optimal load (-j 256) PREEMPT_AUTO PREEMPT_DYNAMIC wall 65.15 +- 0.08 ( +- 0.12% ) 65.10 +- 0.19 ( +- 0.29% ) utime 1876.56 +- 477.03 ( +- 25.42% ) 1873.63 +- 481.98 ( +- 25.72% ) stime 295.77 +- 49.17 ( +- 16.62% ) 294.41 +- 50.79 ( +- 17.25% ) %cpu 3179.30 +- 970.30 ( +- 30.51% ) 3172.90 +- 983.26 ( +- 30.98% ) inv-csw 369670.00 +- 375980.00 ( +- 101.70% ) 390848.00 +- 392231.00 ( +- 100.35% ) vol-csw 216544.00 +- 28604.60 ( +- 13.20% ) 205117.00 +- 23949.50 ( +- 11.67% ) # Maximal load (-j 0) PREEMPT_AUTO PREEMPT_DYNAMIC wall 66.02 +- 0.53 ( +- 0.80% ) 65.67 +- 0.55 ( +- 0.83% ) utime 2024.79 +- 439.74 ( +- 21.71% ) 2026.12 +- 446.28 ( +- 22.02% ) stime 312.13 +- 46.14 ( +- 14.78% ) 311.53 +- 47.84 ( +- 15.35% ) %cpu 3465.40 +- 883.75 ( +- 25.50% ) 3473.80 +- 903.27 ( +- 26.00% ) inv-csw 471639.00 +- 336424.00 ( +- 71.33% ) 500981.00 +- 353471.00 ( +- 70.55% ) vol-csw 190138.00 +- 44947.20 ( +- 23.63% ) 177813.00 +- 44345.50 ( +- 24.93% ) Cc: Ingo Molnar Cc: Juri Lelli Cc: Vincent Guittot Cc: Peter Ziljstra Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5df59a4548dc..2d33f3ff51a3 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8968,7 +8968,9 @@ static void __sched_dynamic_update(int mode) { switch (mode) { case preempt_dynamic_none: - preempt_dynamic_mode = preempt_dynamic_undefined; + if (mode != preempt_dynamic_mode) + pr_info("%s: none\n", PREEMPT_MODE); + preempt_dynamic_mode = mode; break; case preempt_dynamic_voluntary: From patchwork Tue Feb 13 05:55:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200224 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349597dyb; Mon, 12 Feb 2024 22:04:44 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWyufBcrJRLlfZbphUdf7ZxcLSXTn9skTH7wHb6WhXy7ejpiCSUcbuMWnYx+Qm78kIf/q3IyR97CyA/qKCDB9FTvrBg2w== X-Google-Smtp-Source: AGHT+IFbVP8438Uf1e7H/8WivrwaGHqaayoGnnU7do6FGLmK6HGb8KZVXQ3PaJK1OwLyvf44XvmF X-Received: by 2002:a81:a50f:0:b0:604:7009:8267 with SMTP id u15-20020a81a50f000000b0060470098267mr7396435ywg.39.1707804283825; Mon, 12 Feb 2024 22:04:43 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWX3D3Y3+y+iXkjagqdL1Y7OHDrv4eLsWyUzmX1rE/1kvo0mJMmpiBb65nU2EvqUajMxxuWfyc7jRFVfq2o51qsyxCbHg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id d19-20020ac847d3000000b0042beee75f61si1970620qtr.568.2024.02.12.22.04.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:04:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62984-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=Gp19mtXg; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="az/145iQ"; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62984-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62984-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 8EA9D1C23534 for ; Tue, 13 Feb 2024 06:04:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 13B1B5024A; Tue, 13 Feb 2024 05:57:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Gp19mtXg"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="az/145iQ" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E1FF46551 for ; Tue, 13 Feb 2024 05:57:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803872; cv=fail; b=WFvMSqztqe6/PSPrcs86duHG84JRonXOnMrUUH2JKgD+4bFuG02kZbAW82XTHCHS7Mu3/9RgTlD8lF4OQyHZRMjIR1bq/OQ9o+JHEoRlItZKZHReN39nMYKIb9NkhCRtCuxcQge55ylYQx2/9NiF9Ptns56+k+BGKd1HC8GryRc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803872; c=relaxed/simple; bh=Uxj2dlGr5k8t6r7cvGC9E90vmUoirWWv+uI4l5YRcv8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=GcfD7JS2qUDTUzLp0ZXVihAcnXn6QrMmrvVapgzRi3SbhELk3z+C91oRRoCGe7RFYa0l+HRlKsLPBERXnxCPsF+whjvGFQvg5bq8XflJIs8K9RI+edc2ada1kLxXjYBHFRYlFdvOXHBMv1V0cZu4+7K0BLgm0Otg91EoSS5eXZs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Gp19mtXg; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=az/145iQ; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5n3sQ013728; Tue, 13 Feb 2024 05:56:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=dXEtnZOhyxE3K0Iw78IS+ki4QyI9cMz/QAPWPNumPqY=; b=Gp19mtXgBZISZS7y/4WN+oW6/ZYzDvbOi7tbRqmaqO2kPoKQ3bLpZALAmhpu7mEbXqpa 4G4krJMyLWVEYuK+GUFTs22I4WIEEhWbh5hdBw3BjJggh8psr6vaZ8+3zszmGG3FTnfs KyuIkdZsNxM+5gvf03GF5TcFrEtC/vs9AFp4VI5SPuKQiNFfT/5MlfVsDlJKhD4LrdQu tg0MjoHu327xCukbQN8E6htzHaBrfJ/tPXoDqnxEAZYIV7TXsiUhFxHKIQbCrDnBzT7i d6ZU48EiGBM+VHlyF5XIBpzh6QNv4j9k0OQKEjasleIBES4o7mcEDpp4wsKOmPZ+NcO1 JA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w816tg4ar-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:55 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5l6cf024619; Tue, 13 Feb 2024 05:56:54 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3tnr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YFsP+z8oL/FEbfmlndIhOYyAjGfYrsjupYqU4URyR24grF+8ZeZf2QLJvz8HEsd1/kQsWYO284OMCgfXs1Wet6Lm+RRNgCl+APvxn4uqY9S6qhpGw+7T+fvMfB0OQL3+pkP2XjlWTeuhk5UJs47PEX4zgPXl4gZRXV0nHdGxwSAprhNyMb4o3jyrchjPe81bvTBBRKtuPdE7hJtxqyIi4Ikl6kILiZWLILYb0rP/u+ikYCCc2NImN8Ykha3cce2C6GGuCdbnfyUqzWj+iEAiycbRAtZUeUp7ieZjm05Lq0AsSerRMIhCRE3C+86Dtd4GKzVfebXV8llNTNu7ZUifJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dXEtnZOhyxE3K0Iw78IS+ki4QyI9cMz/QAPWPNumPqY=; b=Phqosc9K3Rl17ZuQOfc/2coTNzBmzf3NfQACneTdBRhvCc8RAO/6XZnVq61itRuPs/3HK56V0BB90/sSUsp5sCpoZMyy6qpV1VvoIZZduOfcYPLMJVNaiKM+QZ/HWpFnVDZHBXc1P56C04Y/xpDreF3G+2O5kc0ZL8eM+mHfAWOq7aKr3OOjg/l9K66k32e+qPb0gglNbOXHNMCQSxaDrulPMW2OP0oN3DKR2jBNdpsnl0IELDP/yusZ6a9Hc9U/xlp9R7uFpph+ZFe9nGuA3h+VOy9pgHvnIWoYMU1F+Ser/hhMy/RyVyQIvMgn7wOpqBZvm5cSrXbrTNK2dyd2jQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dXEtnZOhyxE3K0Iw78IS+ki4QyI9cMz/QAPWPNumPqY=; b=az/145iQkf83OnJnUAvndR5xrbQjq3XWgndlowrGoZi/U8xlRVec/pjE9jpjl/RzUeDWWDATA1C1odIWMopG4RAECpzN+SiIAOBsAb8tVuKHkP5eQAMJYocLhYq4b3au4dwvz+xF8XiW0bNWxEeLezbbVWTBL1H5HSgx5b9vU64= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:52 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:52 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 25/30] sched: support preempt=full under PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:49 -0800 Message-Id: <20240213055554.1802415-26-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0133.namprd04.prod.outlook.com (2603:10b6:303:84::18) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 74c6010e-a1f1-48ad-59f9-08dc2c58936f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LGs/JCLc7VAjmR3lqI74SWapFV4kSHCq43a8qqHAJ1Kz5lfcNlRnOQa46s+CZWkcD2N562JrEmM8qm/J2nkOCSS4b4KE/TU/gbbN1OzEejkngbEdhLol7/Y5JwOM4RDLyqma65kpXG1R2+sbudLeHPFy05WWeQ/7RgUsqa0RR5nEXDmd35NolwXF5TF9V8gcqwpp3+VbyYYngG5uSgLjd8fVQBsk0uM5wyQp4HYu+b6VYSA9rsZxPMwAnxktY+ecyapVcpZ5gASS5UktAvQjnvAguu2Zgb3RJkyuAsbXErBwG5w/2Y0hLiga3jHvMyJyKn9eB322Q94hDrxNtoMGYpluQOxotUU6uP50p3gpQIiNIQqgzfx29qQT7bL/hlzyCjuch6CcEJa49+tZTCUTb4iZenWMob7P7DFzZSUEF2swoeQfcF1ZDv0eCr6RWdPpMdD69iHxMShoI7yvpNyL8uksf2rL0DrVeywCKWlAECOX8QxhZBx91FbgW0Hp9WKwR6e3XqEYWyyVCPXSYij2F0x/DxyAiQStW2ALg6bH8HI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Boa+y92NZDD2976/cpbHYNPGn4ANxNqFm6YKvViyh6SXwYkzh8Zmv+yKnQ5eqjAjB7QyTvPZpQvyATtyJTr9oUHZwFh3ArVxlNkHuIxFtRzS/i6UefWlfk2sIpL7e9cMpD3y/RU1hHVXq5SFeGIcPsrCW2N+Aw6mZ2/ZapY5j4wO9iDv2OOjm4XF2HiQNuwJu9sU8PARH5VJ0LP7n368sxaP8fZTyUJ1HNh97ummila6K4ZLWesur3xqbItLT/9UW1cvTAk13HHw41FqFKyTZ8lYo+Iq160+JWuLDrvXI1E+4+jbzlI3zBBj9rmkL86O92yCCm5Wczn7de2aBoFzQakBNE2R96ZI1JFLLnP4C0IZfDV44O6mrABNMoIiNj7BZ0JMsMKVmYYLZKbfHHPBjk68Lr6twuRQguv46ysCkwIAwEiaCkRY4AAE5hemL1hBLY9H4fqcqGyibNyssnEwjSw3d0Wh+IMyCX3jsV3PBzx2hBquAsRP70zSRzdQmm3Kjv68DRl1PXoENvuUNnxkeMFMctLYHyike1pGAMP6cmW31PuB+omzdCxAA/+yhRGkfMRSLql1ewIivg6eJEc+TPRibmwIXKQMoxHyP2tJgVQs0YdM9UJFdmkEsSVM1KShkjPSZSJ7cIByDpYemaDU0gDijlnrN7+z9bKGy4hYqS7pKyAHmDZ9w6LjPy2wdo9gXFUIO4GEJXuY/hMOGI6HCIEKmTKprMTs38O6g+E1QRwSZwavB2yMbsR6ONANm3gsdpT54F8OkwydxYfGnzdvZV0u5o9xycha7JjurIfZ6UEls1u34N32m8PLyOFafyLAIQ3mXEXVAQI3ieFCz3xbo1Y1HI1euAIbwk2FOYzwqycWhQTFQBZCGTJcZdXIHBRbTVu+yKPrMZ/brpexbeUqyEsNCLp+eDgp/Yc1y5sEkHio9jNzRr+HQZh8viYdjLtQ6ZrVIqWiJllpCerg6wfduwtpkLIpR4m+ZozVbS641vl0/BWiBC0Ucvi/ob6E9CbdxwQpPPbyj45hGqi3c8z6G5s7I0eFR78kheVPpv+4zOOxkeuhTKQ8IOz9mH2PUBWb8c8Hq2Xa/WlgmhmkF5bxaa4TVljfNdrvIPycZFMkz6w3fTAWyq6TH3pqHPiGnbpC0+MbuyY8FhvZPHQO4a6KIO8YAlrRPsTGoyxQbg4gWpGtUtYeKpdhg04JSuZWN5C3/hvyW0cS7EkhXQauXKa+QpJQIQOXpoPIJ4LBj6LzF8RCGCiBLRFxR0a9Ql2bYDisg1VBt/OXZ4msJAqO8jAfVB6eFkDyd+fbx616slo+eAatg98tW1VM3/STvWcpQZQl9JpcFEwbL2wtAiPt3Cuz8BmGrhrorCrsALgLhsGzYWoj+RraKd+OcwAs/Hg2K32D8taczFw+7mOyaIgs8RzhZyf3ah3h2R2I5vJ2RjD6YuqOhwP57Xp/OHonXlWJsRYvNMKHIjhoJvL9jv2Wufry6orjfmeZM8dhYLULJ8bhClezaqSsxGf3YTziblUI0bwQDaasn6o8P27FZ+15JYZ09SZoYdoeej8OfMYgUHS3Tjn0i3TxumQ6GV0xDhA2SO8ljS3ajvrkM9wq64b31tsw7A== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: yGIOG4kiyCdNTndN497U/nwtfLyS1HzhINwAQjt9HplmrkMWXt802xEU2sMsvWwvmevlCxVW4zmTpM8wKJt9zjQtC4SIF23/XIAxW8pCLS3SakzVMaQ9bwN7WzcGoomN0XVxgDKi7hQV4J3vrQhAxxxFRuR9R3hfa+IqmO3R25N65BPYJX95nc//AENkyXn9YTI/K7+Rn7/uRHvURGnM6y2/Z+xaBiBrSZMEVsHhO1jGpy8TZwAjtxNhDigMtBuz6kOqWPNCUGfOAWAny2sas2dHYUd/yIyUnl0LkmdulzC4azJInlNOtoEeC6PLzahEUebG/afUjIsq9LifHjF8XwD0FfnUJ6qekNmENjx+aksEHoJAoNCHsixxbC0D8K+sPaGlqpGJce8EYkH9cMDH4QeB5v9nwimldyXF2yZtPC1wA0VZ2k8+fqMEDuxUMD1tj3fH5RpSHodVjisPR8fokW1DKXxGRhKG/Nrq3xU9VFdGp5RiEBujr6OTsTBMA/qtRwVGjOHxE5dIVsN93HCiDzVySllItY315+04qlUCVKZu1MfCqmHw9RdvduzCraZCqdfWPxCkfi2ZH63eZEtGaA1jkkOWbFN3B/wZjGkGKQc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 74c6010e-a1f1-48ad-59f9-08dc2c58936f X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:52.3302 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Mau2xC+6ge5I4wp/qyCAR/11J8N6QTscMK+xrhwAhH/i1Co3T27EQoa3ZEg+IjCyz3JpRIQTzm2NtZbk1LLV03RAaXM4oL+HaxmcwZD5YaU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: _dC-pdLwH5S_86MnE3DooFnxG4vNtnWz X-Proofpoint-ORIG-GUID: _dC-pdLwH5S_86MnE3DooFnxG4vNtnWz X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762584680405139 X-GMAIL-MSGID: 1790762584680405139 The default preemption policy for preempt-full under PREEMPT_AUTO is to minimize latency, and thus to always schedule eagerly. This is identical to CONFIG_PREEMPT, and so should result in similar performance. Comparing scheduling/IPC workload: # perf stat -a -e cs --repeat 10 -- perf bench sched messaging -g 20 -t -l 5000 PREEMPT_AUTO, preempt=full 3,080,508 context-switches ( +- 0.64% ) 3.65171 +- 0.00654 seconds time elapsed ( +- 0.18% ) PREEMPT_DYNAMIC, preempt=full 3,087,527 context-switches ( +- 0.33% ) 3.60163 +- 0.00633 seconds time elapsed ( +- 0.18% ) Looking at the breakup between voluntary and involuntary context-switches, we see almost identical behaviour as well. PREEMPT_AUTO, preempt=full 2087910.00 +- 34720.95 voluntary context-switches ( +- 1.660% ) 784437.60 +- 19827.79 involuntary context-switches ( +- 2.520% ) PREEMPT_DYNAMIC, preempt=full 2102879.60 +- 22767.11 voluntary context-switches ( +- 1.080% ) 801189.90 +- 21324.18 involuntary context-switches ( +- 2.660% ) Comparing kernbench half load (-j 32), we see that both voluntary and involuntary context switches, and their stdev is fairly similar. So is the percentage of CPU taken and various process times. # Half load (-j 32) PREEMPT_AUTO PREEMPT_DYNAMIC wall 74.45 +- 0.39 sec ( +- 0.53% ) 74.08 +- 0.20 sec ( +- 0.27% ) utime 1419.68 +- 5.12 sec ( +- 0.36% ) 1419.76 +- 3.63 sec ( +- 0.25% ) stime 250.56 +- 1.08 sec ( +- 0.43% ) 248.94 +- 0.80 sec ( +- 0.32% ) %cpu 2243.20 +- 19.57 ( +- 0.87% ) 2251.80 +- 11.12 ( +- 0.49% ) inv-csw 20286.60 +- 547.48 ( +- 2.69% ) 20175.60 +- 214.20 ( +- 1.06% ) vol-csw 187688.00 +- 5097.26 ( +- 2.71% ) 182914.00 +- 2525.59 ( +- 1.38% ) Same for kernbench optimal and maximal loads. # Optimal load (-j 256) PREEMPT_AUTO PREEMPT_DYNAMIC wall 65.10 +- 0.09 sec ( +- 0.14% ) 65.11 +- 0.27 sec ( +- 0.42% ) utime 1875.03 +- 479.98 sec ( +- 25.59% ) 1874.55 +- 479.39 sec ( +- 25.57% ) stime 297.70 +- 49.68 sec ( +- 16.69% ) 297.04 +- 50.69 sec ( +- 17.06% ) %cpu 3175.60 +- 982.93 ( +- 30.95% ) 3179.40 +- 977.87 ( +- 30.75% ) inv-csw 391147.00 +- 390941.00 ( +- 99.94% ) 392298.00 +- 392268.00 ( +- 99.99% ) vol-csw 212039.00 +- 26419.90 ( +- 12.45% ) 211349.00 +- 30227.30 ( +- 14.30% ) # Maximal load (-j 256) PREEMPT_AUTO PREEMPT_DYNAMIC wall 66.55 +- 0.34 sec ( +- 0.51% ) 66.41 +- 0.72 sec ( +- 1.09% ) utime 2028.83 +- 445.86 sec ( +- 21.97% ) 2027.59 +- 444.89 sec ( +- 21.94% ) stime 316.16 +- 48.29 sec ( +- 15.27% ) 313.97 +- 47.61 sec ( +- 15.16% ) %cpu 3463.93 +- 894.12 ( +- 25.81% ) 3465.33 +- 889.04 ( +- 25.65% ) inv-csw 491115.00 +- 345936.00 ( +- 70.43% ) 492028.00 +- 346745.00 ( +- 70.47% ) vol-csw 200509.00 +- 32922.60 ( +- 16.41% ) 187447.00 +- 42567.20 ( +- 22.70% ) Cc: Ingo Molnar Cc: Peter Ziljstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2d33f3ff51a3..aaa87d5fecdd 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1035,8 +1035,9 @@ void wake_up_q(struct wake_q_head *head) * For preemption models other than PREEMPT_AUTO: always schedule * eagerly. * - * For PREEMPT_AUTO: schedule idle threads eagerly, allow everything - * else, whether running in user or kernel context, to finish its time + * For PREEMPT_AUTO: schedule idle threads eagerly, and under full + * preemption all tasks eagerly. Otherwise, allow everything else, + * whether running in user or kernel context, to finish its time * quanta, and mark for rescheduling at the next exit to user. * * Note: to avoid the hog problem, where the user does not relinquish @@ -1052,6 +1053,9 @@ static resched_t resched_opt_translate(struct task_struct *curr, if (opt == RESCHED_FORCE) return NR_now; + if (preempt_model_preemptible()) + return NR_now; + if (is_idle_task(curr)) return NR_now; @@ -8982,7 +8986,9 @@ static void __sched_dynamic_update(int mode) pr_warn("%s: preempt=full is not recommended with CONFIG_PREEMPT_RCU=n", PREEMPT_MODE); - preempt_dynamic_mode = preempt_dynamic_undefined; + if (mode != preempt_dynamic_mode) + pr_info("%s: full\n", PREEMPT_MODE); + preempt_dynamic_mode = mode; break; } } From patchwork Tue Feb 13 05:55:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200235 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp353205dyb; Mon, 12 Feb 2024 22:15:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IG5WkOenfb7qrfFl+AadjJZ+a1HerWZsoTMM215miagC/E8SsiP3nWXSOrQJOw4eAblGe+8 X-Received: by 2002:a17:90a:f193:b0:297:eca:8381 with SMTP id bv19-20020a17090af19300b002970eca8381mr4923206pjb.16.1707804926761; Mon, 12 Feb 2024 22:15:26 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWVlfUkPvpJ6RCM6bXzqJcyWWID2BW+bCRpKaOBeMfuOnwglw+tlrmr5eeIEoVVSAr4rbDtu0v94BPI/Dkzgs7v2zUxEg== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id mi10-20020a17090b4b4a00b002970e69b1c2si1539461pjb.130.2024.02.12.22.15.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:15:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62983-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b="SPvN/kE9"; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=DJySch8q; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62983-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62983-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id EA786B2221D for ; Tue, 13 Feb 2024 06:04:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E0BB045979; Tue, 13 Feb 2024 05:57:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="SPvN/kE9"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="DJySch8q" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C5463D977 for ; Tue, 13 Feb 2024 05:57:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803864; cv=fail; b=LSn1TBYAoO0wtSoW1zU5SHLnGrTwKwveQdqwYPmDn6dlc/8bki+/jjq51WIytFq2PQrjluMp4tbvxF3SzBA2uyO2tfCg57ZRUsg7evpBjyh4uzYczUDqFp9RpXg+yJ8LlFmiHZXhuLkfVGCwN5q1s3/uUAzAlZ5kLrvgkVbxPU4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803864; c=relaxed/simple; bh=Pqnh9+Y4CihM43PeZs1AqTHwvhoLciD4p8HqIMV80K0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=mPtTVX7GstHrv5ttumqgz+1u7Lm8xjGSVKwkoawZpw8BHwYE3BIuSiJESIyf6uLVKt8sPEr1e2oB6VgS/BZmKFJHLaGVbH8V5gtjRzOGnIIUzzMKDSm48QCEJ5RXkfbPAIFWIFbnhoTkgUSZa62X0h0XAsVJJhkrDeb9l9+3MCI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=SPvN/kE9; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=DJySch8q; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5Yp9q005664; Tue, 13 Feb 2024 05:56:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=clyARajQghUUYAlB1jitbQuT4jxObCTuhFBJ9nurdGM=; b=SPvN/kE9I5Q8byuOXxRxz95vdFG00usP6WBUQE/HmugvfHhJc4SbDT0s0RSG8We3bKlV Ff04BZlwt8aCPh28RmiEj07MKitYaynnJTAntuu5KpfXpGPuJsob4rUIeR6RN5UgbZAI UIuo/uildaoCVvTv25y5t8DBLootKniWneJZC7GlVKCYnCAG4klLMSPMflmlLayA5Elp kBYNxkdR+Zsa9CE/VKBTNK6mNrIYzRzYoFOkzEoD3FBrKIlHdNa8bTll9g5MEc3PGoCk ozBJ0KO1ZAPqRvp+vZoj7QFm4sZglVimrEOve9oI3PAZADq+DAjDSbXF5YAJFbDf0o+e rQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82d7r1h7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:58 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D50lL8024685; Tue, 13 Feb 2024 05:56:56 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3w5ykd3tq0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:56:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KumzYutSRi/y0zEcLhOzPZrYQ8eyD3RJUncs96i14da2m3n23h/2ZMt/tORXF28uZzKhnZov48XQvnjc6dMkVm8VtleO48oaDDrlvqorsbr3xP41Cb6HveuCaj2Dd9KdEvWPKVNigsMI69UMvDwpa6txi7PAhPz8CY6nlcIhoURNfHzk6Ac1/dsnmNJ5Ef+DHWQdwsftbG6lBbrY3JkZee75QXzExScbZS4wrP5LXa50NTdOKstUPZvri9iVdpK20uF+S2GJt7fDKy7WuMJ/e24PMAEiUXO3a/x8gVFKIcuePaisxykJ0oDTube4qiOkSegaL8HhB3ArSFddzWLuZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=clyARajQghUUYAlB1jitbQuT4jxObCTuhFBJ9nurdGM=; b=N9fCHJrhDxIKfan7TS4c4S6rvcdOEUb5VQ+1zdoGRelBdgNViYjTSrs++FQDFWmxRAVlPDySONYLHCvK/fv097oOycQUmtlf6gUQ7Pa9sbC2XcSdVXIbXkcw7Qu1q3BkCH+SRQLYNUBSQrmAWoQl4obgVYsM5WLcdabQA1hNw/FvHXysdbUPtBbvXaKqA9l81ahTWQPL3aANlUVcBL+jm9ZmQw/v7R/ILddpjyiOGkziZnE57NlAU9BJD6ARyQ8qVO9Pnc2YEDMtkfZG1b6kLvf7Xk8j+LbDI7/zHyqCmTTc2GoWX7flXbuhTYKZXkP6bWO1nA0FgyrfwAvBfXE5cA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=clyARajQghUUYAlB1jitbQuT4jxObCTuhFBJ9nurdGM=; b=DJySch8qqRCMHLlIDVGG4Wq1YowksDLnI5LOhhqaWBhMRoDOQ73mlfWWGkzhk3+C7alpJXOH+9hA4WaGD9FnTbyQQk8/WNtR6iDEGMqP/heQT9HM1xYFvFJaWSA8YE+25+mOTUAr/JpWuv2JQ1+xdnbdN9/PnP2s66asD508DAE= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:54 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:54 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 26/30] sched: handle preempt=voluntary under PREEMPT_AUTO Date: Mon, 12 Feb 2024 21:55:50 -0800 Message-Id: <20240213055554.1802415-27-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0327.namprd04.prod.outlook.com (2603:10b6:303:82::32) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 6ee8780b-6ebe-4707-1058-08dc2c58949e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7MbNXA9RVJ86zBz98E6pbyp6kC4w352wnBR296a2Ttz6iDUWznMLHCIUxcjPa9iFLJxFojkAmNSvbqVKKvuyO2kyux1fZBLNYXZLnbDSoc2TsQa/osyGafhtrtpLFbWFNzekGsBjr2V41J3vHM+Qv3R6WophfpIlzw7yd2l8keJEmjZBkj27/F1arDNA3XGPREhTU2pQsE5w+OZWgeQmkxJ78UfClcMFwb5HoyAWxU90Rf9iheJoYifXdCXhkMQ9NOlBP+9FDJgV7AhqS+tJrMv/GwtxruHIztxDiv1DVG6le5hdLeumFAI0tMcOzUJVlFslJEjHtznZR7r09ksRgIGBhevoF5tJYYUR+Zjnkb2qkbuXzdq2Ki5JJXwHeVRMSNDP7Oc1OKXUqyy60IqU/npX0yVCMxSHLohRlBrR8mJAs+ETxuOYVCsRmubpNK3Vn2IbabZ6ilP4YrxGfbpQoqdeJYCvPduXNgayNAdsozxBMPw+YBNlsKUCFGNrsXxTypB6fY5hX+hVwykfml5S9N+Y6+Y9wZ5ITG5riPDErBk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QA8SHkvFpc0LXDfJxkWZmSQSowBKcIuU12tf3UQoroCafu2KoVNlT6mJ8dAGle/3z99AkTObNCFnurCzbfeVxu46C16hogD98D/AL30jwdjuUhm7QZMDu4gIu8i8z52Fwsd4/N7OBtEI08GFd0ah2hpsO4S//y35VFo4NwAjdeygS2swZchoaaHokteViUzMVLTY4t5Gcc2GJyqXt4IyI431Chu/qmkR/3nMonl6t9xe/rmtuhbQaxX4aD6pfWlegdoecBv82QoQdDyGZ6l7GgqctI2tXv5EDWjTzVhuJ3t/6Epp09LVTpEO7xxzl1+CQDUAkJC3JvrVER/Sk0XGGoocopUjP5QOlPNvMyahY/0gMERaCKbmdHOpxZvdyooDu1rJ5K5L0ICbpdTWFQ1UK6ZBps2rCS0tUFjxzIIsPxnQeXHjQmyphx8NuhhF95b3LycVguLYYR4xJhuYuGaST96B4kl2C9hmwuDVjjU0QCFQuB8cpYcDBkNeneN61eHAXiYSKZVwJtY21qvTxnj2wulYyTbpsdWaXPNLdFhFx3u+az9Mo7JYZUTRZCOb8Smxzqkn2c2AlV2g6WbeaL6JOVyHU4BDKG3Y8fizR5t7u3pSrnjjQFADbCZPDDm1nGolgBxXQsv2Xvb0V1lCXum7yIJBvYhNMeANzAxf/xeo9oRaE8wGsS+HqYcUgfPaHZ4SZ6rp1iAJryeSxlLQQfv+LzE4iLHfeIVDDH4FMqqhr9NxkSajudo1kPiMNDa4CI6yJsuSXK1SxCiXrDDQht6UpFcWmVKJhyuew6jkCNDzctfD0mPyi50IY6Jetm3oVttDM2b9A1KolYMAYs2EFkw3nbmT/T8KI5Gn57U5wid9O1BhzsZ2DF/2157iC+QUdnAwGvrtyexhjFgonyp2zFALULwhura+0GWOfWLz3WjtGBFsvDR/firG+np0Ww8zRBqjIebc8JWYVFcQOx5xHC4AlQW1KfIPqPQU9/hP4VuVpKpR4eckhqyL8Hu4i8LXL8fIsSxBOBoDgYgz73lWMDvFOAu0Blkb40MpHhgNREVhDphsJO0K2u38j1dyY2u8NmOTt9jDXAE8PtDzG6DJND7dwl3bjxyTioLxgWohZDdROy7ZocLW6NYaYX+WZzoSNlRoy1ETdNUKrTkG6cGVBiQxYoU2mBD7bit6aicD7Lz8bU8q4bsGlzyOV33zjBsvoK0oXhiUFvb2nQzrZPRb0DPF+s4op3a71FWtJMdm8IKzoqnHVTtKxraswLjhPZCY32hgwrEClcwmGLjiVa/uMK8FFOCPSe+gl9P61Do+jHMxOsCgNfIFQ2QCa3vO7I6y8DFg7+T5DmLsokSyZD31l8sJ0qxBxAPOrY5hWE+UVsn/+gDpHynWUrtHcZHBPO2AZyIiAlpxP6bwH8Y0BBZWfvW8DHp+kK4kDsGNLDjhwCaQSMLdvGlTFu93/60FeOV9oOvQM8GVrIKlHzTrx+5ueun7XwC19wGVuLtGhrAUsxhyKQb96EygahiwwT/H4sKO6vYK/xEbSORTX9gyUWEg38MMoM2pQkmQyCamaLY179cMQp73umFXnLlIFAcpEiHSrvm5H60o5yfLmW9XEHco8xHN0Q== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: O66dd+qtRsgl/XznNtVtu++9Rde8IBKsGrhwWmdzPZEIVi5uCjwAp9bc6Oj+K/aQpbdG8zjd3nCv5oX6wzrWSz79JvZkETY2iiAOk4uXXhNGL9s429M1h4GY4oFhji7U60mhPXCeIfTsOzxurT4WkfLHuhwq46kiMvlwHhF9MZm+BAzUFNPZGyZPsdoXmG/Ganu/oLTRwZjc1QTga5UXVkgye9z7GDsKLRb75wgJmg+SE8pq57Au8RJPLUcknF72cnk3uILUWKiY27SM4zRP6u2zo6tw5kMzWM/o0JRXp/odYyiH/b7JYE5WYcxvViB+uIiT9k1Etf/j//vh1ynmoEc3JuZYi5EggZuq0jOLvXpu4HbOyErA/s0lson2G4o4z+y3OwMkezH9B33unvf8LCQgPeAtA3NDbuH+pnh3ZnKr120VTY4V8LUn+Osy2GZy8JqdeyU1CAjDu2A4kU8Bt3ISeenS4IZz47FIxanOVxe1YWKBLNw4j+8BgkOsJ9cMlhDhjwp4j8kk6HLOFW/I0MMz3TCLAqbntq+tcFmmPLl1xcKBtRgtORxamTfCQ/S45YBdEOOvEEQeDmExK4u/4VYvUhr61L8ZDPBXVTGPaos= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6ee8780b-6ebe-4707-1058-08dc2c58949e X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:54.3772 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KLFpFcsInx4peieXlAwZPfbGv6awtLeQ2T44wSBphZj612jq2t80X+RIErPtl89aheiAkSzRb70PSxc9TS9EqUfCq/WGTYMvnFs9x0p6d6c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-ORIG-GUID: Cyj1bxwmvirPQuAjSU394l4Ust9ZeRJK X-Proofpoint-GUID: Cyj1bxwmvirPQuAjSU394l4Ust9ZeRJK X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763258685300480 X-GMAIL-MSGID: 1790763258685300480 The default preemption policy for voluntary preemption under PREEMPT_AUTO is to schedule eagerly for tasks of higher scheduling class, and lazily for well-behaved, non-idle tasks. This is the same policy as preempt=none, with an eager handling of higher priority scheduling classes. So, compare a SCHED_DEADLINE workload (stress-ng --cyclic) with a background kernel load of 'stress-ng --mmap': # stress-ng --mmap 0 & # stress-ng --cyclic 1 --timeout 300 PREEMPT_AUTO, preempt=none stress-ng: info: [8827] setting to a 300 second (5 mins, 0.00 secs) run per stressor stress-ng: info: [8827] dispatching hogs: 1 cyclic stress-ng: info: [8828] cyclic: sched SCHED_DEADLINE: 100000 ns delay, 10000 samples stress-ng: info: [8828] cyclic: mean: 23829.70 ns, mode: 3317 ns stress-ng: info: [8828] cyclic: min: 2688 ns, max: 5701735 ns, std.dev. 123502.57 stress-ng: info: [8828] cyclic: latency percentiles: stress-ng: info: [8828] cyclic: 25.00%: 6289 ns stress-ng: info: [8828] cyclic: 50.00%: 13945 ns stress-ng: info: [8828] cyclic: 75.00%: 25335 ns stress-ng: info: [8828] cyclic: 90.00%: 34500 ns stress-ng: info: [8828] cyclic: 95.40%: 41470 ns stress-ng: info: [8828] cyclic: 99.00%: 85602 ns stress-ng: info: [8828] cyclic: 99.50%: 301099 ns stress-ng: info: [8828] cyclic: 99.90%: 1798633 ns stress-ng: info: [8828] cyclic: 99.99%: 5701735 ns stress-ng: info: [8827] successful run completed in 300.01s (5 mins, 0.01 secs) PREEMPT_AUTO, preempt=voluntary stress-ng: info: [8883] setting to a 300 second (5 mins, 0.00 secs) run per stressor stress-ng: info: [8883] dispatching hogs: 1 cyclic stress-ng: info: [8884] cyclic: sched SCHED_DEADLINE: 100000 ns delay, 10000 samples stress-ng: info: [8884] cyclic: mean: 14169.08 ns, mode: 3355 ns stress-ng: info: [8884] cyclic: min: 2570 ns, max: 2234939 ns, std.dev. 66056.95 stress-ng: info: [8884] cyclic: latency percentiles: stress-ng: info: [8884] cyclic: 25.00%: 3665 ns stress-ng: info: [8884] cyclic: 50.00%: 5409 ns stress-ng: info: [8884] cyclic: 75.00%: 16009 ns stress-ng: info: [8884] cyclic: 90.00%: 24392 ns stress-ng: info: [8884] cyclic: 95.40%: 28827 ns stress-ng: info: [8884] cyclic: 99.00%: 39153 ns stress-ng: info: [8884] cyclic: 99.50%: 168643 ns stress-ng: info: [8884] cyclic: 99.90%: 1186267 ns stress-ng: info: [8884] cyclic: 99.99%: 2234939 ns stress-ng: info: [8883] successful run completed in 300.01s (5 mins, 0.01 secs) So, the latency improves significantly -- even at the 25th percentile threshold. This configuration also compares quite favourably to voluntary preemption under PREEMPT_DYNAMIC. PREEMPT_DYNAMIC, preempt=voluntary stress-ng: info: [12252] setting to a 300 second (5 mins, 0.00 secs) run per stressor stress-ng: info: [12252] dispatching hogs: 1 cyclic stress-ng: info: [12253] cyclic: sched SCHED_DEADLINE: 100000 ns delay, 10000 samples stress-ng: info: [12253] cyclic: mean: 19973.46 ns, mode: 3560 ns stress-ng: info: [12253] cyclic: min: 2541 ns, max: 2751830 ns, std.dev. 68891.71 stress-ng: info: [12253] cyclic: latency percentiles: stress-ng: info: [12253] cyclic: 25.00%: 4800 ns stress-ng: info: [12253] cyclic: 50.00%: 12458 ns stress-ng: info: [12253] cyclic: 75.00%: 25220 ns stress-ng: info: [12253] cyclic: 90.00%: 35404 ns stress-ng: info: [12253] cyclic: 95.40%: 43135 ns stress-ng: info: [12253] cyclic: 99.00%: 61053 ns stress-ng: info: [12253] cyclic: 99.50%: 98159 ns stress-ng: info: [12253] cyclic: 99.90%: 1164407 ns stress-ng: info: [12253] cyclic: 99.99%: 2751830 ns stress-ng: info: [12252] successful run completed in 300.01s (5 mins, 0.01 secs) And, as you would expect, we perform identically to preempt=full with PREEMPT_DYNAMIC (ignoring the outliers at 99.99%.) PREEMPT_DYNAMIC, preempt=full stress-ng: info: [12206] setting to a 300 second (5 mins, 0.00 secs) run per stressor stress-ng: info: [12206] dispatching hogs: 1 cyclic stress-ng: info: [12207] cyclic: sched SCHED_DEADLINE: 100000 ns delay, 10000 samples stress-ng: info: [12207] cyclic: mean: 16647.05 ns, mode: 3535 ns stress-ng: info: [12207] cyclic: min: 2548 ns, max: 4093382 ns, std.dev. 116825.95 stress-ng: info: [12207] cyclic: latency percentiles: stress-ng: info: [12207] cyclic: 25.00%: 3568 ns stress-ng: info: [12207] cyclic: 50.00%: 4755 ns stress-ng: info: [12207] cyclic: 75.00%: 15187 ns stress-ng: info: [12207] cyclic: 90.00%: 24083 ns stress-ng: info: [12207] cyclic: 95.40%: 29314 ns stress-ng: info: [12207] cyclic: 99.00%: 40102 ns stress-ng: info: [12207] cyclic: 99.50%: 366571 ns stress-ng: info: [12207] cyclic: 99.90%: 2752037 ns stress-ng: info: [12207] cyclic: 99.99%: 4093382 ns stress-ng: info: [12206] successful run completed in 300.01s (5 mins, 0.01 secs) Cc: Ingo Molnar Cc: Peter Ziljstra Cc: Juri Lelli Cc: Vincent Guittot Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- kernel/sched/core.c | 12 ++++++++---- kernel/sched/sched.h | 6 ++++++ 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index aaa87d5fecdd..aa31f23f43f4 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1056,6 +1056,9 @@ static resched_t resched_opt_translate(struct task_struct *curr, if (preempt_model_preemptible()) return NR_now; + if (preempt_model_voluntary() && opt == RESCHED_PRIORITY) + return NR_now; + if (is_idle_task(curr)) return NR_now; @@ -2297,7 +2300,7 @@ void wakeup_preempt(struct rq *rq, struct task_struct *p, int flags) if (p->sched_class == rq->curr->sched_class) rq->curr->sched_class->wakeup_preempt(rq, p, flags); else if (sched_class_above(p->sched_class, rq->curr->sched_class)) - resched_curr(rq); + resched_curr_priority(rq); /* * A queue event has occurred, and we're going to schedule. In @@ -8974,11 +8977,11 @@ static void __sched_dynamic_update(int mode) case preempt_dynamic_none: if (mode != preempt_dynamic_mode) pr_info("%s: none\n", PREEMPT_MODE); - preempt_dynamic_mode = mode; break; case preempt_dynamic_voluntary: - preempt_dynamic_mode = preempt_dynamic_undefined; + if (mode != preempt_dynamic_mode) + pr_info("%s: voluntary\n", PREEMPT_MODE); break; case preempt_dynamic_full: @@ -8988,9 +8991,10 @@ static void __sched_dynamic_update(int mode) if (mode != preempt_dynamic_mode) pr_info("%s: full\n", PREEMPT_MODE); - preempt_dynamic_mode = mode; break; } + + preempt_dynamic_mode = mode; } #endif /* CONFIG_PREEMPT_AUTO */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index c7e7acab1022..197c038b87c6 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2466,6 +2466,7 @@ enum resched_opt { RESCHED_DEFAULT, RESCHED_FORCE, RESCHED_TICK, + RESCHED_PRIORITY, }; extern void __resched_curr(struct rq *rq, enum resched_opt opt); @@ -2480,6 +2481,11 @@ static inline void resched_curr_tick(struct rq *rq) __resched_curr(rq, RESCHED_TICK); } +static inline void resched_curr_priority(struct rq *rq) +{ + __resched_curr(rq, RESCHED_PRIORITY); +} + extern void resched_cpu(int cpu); extern struct rt_bandwidth def_rt_bandwidth; From patchwork Tue Feb 13 05:55:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200225 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349716dyb; Mon, 12 Feb 2024 22:05:04 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVXJ7rugJ668nxfE60ltCNEKLNK2LJ/fQIedA/b0nUjbtGJkcg6Xo2wJqnnbd/iLY+C5UtwTVHejow30NB4doJjcQuD9Q== X-Google-Smtp-Source: AGHT+IHOSnd+eOz1+erfAnOP7y2CrD9ggY4RHEL7T7MXdWOFJyjZaPFlwausJ0FaCaho1TUAgEK4 X-Received: by 2002:a67:f75a:0:b0:46e:c364:4241 with SMTP id w26-20020a67f75a000000b0046ec3644241mr3762323vso.32.1707804304470; Mon, 12 Feb 2024 22:05:04 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVKyw9SEk+cKu46LAjFAXRvh8WzfoSYzubm46VfwwIA0Oxwbtg6tbESTCC9q7TVmDK/DtNJnr9PWKycsGtn5bjfZEXVHA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id a13-20020ac8720d000000b0042bf6462ae9si2020114qtp.734.2024.02.12.22.05.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:05:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62985-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=J0u5NpBY; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=aglmZ36X; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62985-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62985-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 235F51C23541 for ; Tue, 13 Feb 2024 06:05:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DBF8451C3D; Tue, 13 Feb 2024 05:58:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="J0u5NpBY"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="aglmZ36X" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E89C50268 for ; Tue, 13 Feb 2024 05:57:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803879; cv=fail; b=NgaiFwdilISOkCSAKzOIsnF138xROe3qQH3yzkYwUIdFVK3dilAIZ2RVJALQAHXwEKIJamBycpsY4uNTjo+rBcyQ3dsb07Wji7hLEUzEiQPcL5iJQUMtknqOHZZ4YpWxn2CeEkTvnT7/UEyu+4ooEVeRz/c5gNxCa55Nhnqbzm8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803879; c=relaxed/simple; bh=FNQMu8YhDbr2MUMb66k9ed7hDgUXcs2hgjnC5X1ydDo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=ZN40BV3qwCkhRjCpncnaqON8OSgGZ69VrJhmiZB1/HomkAzGcKGiPHy8P1ter59vK+zh1U/1XUw5bhRcaYvotk5DUL5FgeBmHpHa5k30svb2Ec0gvaJf7oUt6iG9uCkf7MwvTQ7kbdU3hSdXGZ4JHktfmxVzjrgPFBHeT/LcBp8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=J0u5NpBY; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=aglmZ36X; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ngWi028876; Tue, 13 Feb 2024 05:57:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=gE7irObtoq0nTvIx4+u0CN8B39f4Bzk1KjFzpyKSFBU=; b=J0u5NpBYSvIX41YUBDOf7w579rY59Pd0HnVbH2khmMXRCsUSvVV70ji2xJHDVXHxPGq5 y8whhnhEmJlblaeVnc3IGv9jxMZl/zHeFpSqcvsf8PxIHjpzacZz4NQrp//n4eIpmTh9 KIGHKwBFlYbwmHoRgVpepz30GPTlBj2zkm6+LsE4avRNhiVCmY04WUxx3+tWRACJnqu2 m8sccUs7A2BWe7LFEEwfoVhBO/cBXk730zN3mcYsRrzv8ApYuaYLn4N47YZRR9LGxzwa pcMUQUHR9vhCoyZeDkYr4wF5uIEdtgQe8bVgxbniApNm0mM0R3WNeZ+NH/O9QSRXkIIh Ag== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82m480kk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:03 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5oR4G013773; Tue, 13 Feb 2024 05:57:02 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w6ap9gpjd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UeCsUur9sG5Mym5f3wv1Te88d7opzIjYBmW969w8c3NdfPMk0/w5bkb46u231/sSO5jxQ7TmXCA+7WLeFvrZyDZw7rCv0xV0twOeYRKKpKy9iuyIP3hSmzQ7sljmTfl1OohNZ5xB9nRRlO1rvpHIKYnBjFg/1Eoh2bqyMrkDJsuDTFH3dcXTDJo0cPacxtG703RlhxLZuTSuqdPIeuG7mEWll3lZdaD3HMBHyd006bF4zRFI5tudW2Ewk4iM+YKJn7OQKuXk4+a2HltMAFRNvafFcsjJPV4ZHBIqeFmTbcuw/YY6OSz6yUFynFei8e+lpmN7H4hgHFHiC5wfaV8JCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gE7irObtoq0nTvIx4+u0CN8B39f4Bzk1KjFzpyKSFBU=; b=Ow2OdIjQcUkDJ/YurjBIwy5TkbgQaPmk3+7SJahBvzOr4W3v4hVjk4juAWRLmlF3l5asgsL9+qeS8xkVgnXF5y+hqY60DVmHm1jTVL1BcT4kA3iLTorlwR87jzQGDShgUGUx9lJgKcmaX/K1wtk2xpxutcDWIMwR7M1+8B7p/TeXN7YRTnioeHMOSnU+bxaLaCwr0mKOwEpV1q1ArACEaMbo/zry3nlITpxV554G3aJV+qrPgCZtdarQs9RKkk6TwQlwh8z74FB+NCqDN0r9IOjSgRgBwqOq6sNm2SF+tyG/41UMB7+cCJZU2YlcXVRN15az6wbg9A0uNyiuBMnH5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gE7irObtoq0nTvIx4+u0CN8B39f4Bzk1KjFzpyKSFBU=; b=aglmZ36Xg0/lsjq9NsAzoRR3HhQ7d58t2JlZLAls+AZKXRQTbcCZE0Dk3EGtPsmxgDwj3iuCI6AGvbFR8Y9w5f9KYis1zDMw3febEEbqCcquQG1SUGRTbTDwHYJ4FccSb191wTPD1xMSqOeFnQG3TVGRomIHu8ZYvJE+x1gzAc4= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:56:59 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:56:59 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 27/30] sched: latency warn for TIF_NEED_RESCHED_LAZY Date: Mon, 12 Feb 2024 21:55:51 -0800 Message-Id: <20240213055554.1802415-28-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0172.namprd03.prod.outlook.com (2603:10b6:303:8d::27) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 3a6d2261-1dde-4e92-cee4-08dc2c5897d2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d78Ckh9ipsAlfoOgv6f7fU5VJ8LjYPn+kz7I+Db+Py+FWWQu0ju1/LncrVjA9/PzWabS1pXyiNLeXhcdzBl/yu9DaRFBjCjcJSZi1mZp5/kQP2s7UwVSIzh7HdBeu8uuNPtJ3j8kPtyhA4ZsDTEUWh2IFWsGsn9jo4qAaKOnHIs9Q9bTpR01N5Txr6tZHJVdV0Tn9P5gZmE5jykB6w2OjKqEAjOMLvL322Vpw0jqxyuvGpH97DhC1CVjVN8MMR6isMoeE99bmguY5toIRnj+zdkLCUeR3bBAgXCVO5qFxeRlsYpnJdtZKDg459BVqnTR3amONOevnczH7L0lWfDIaiXyu0oGx2ax9mQrwVRGc56Z5Uv1pyt+wXS/IANtdhvctGASmMlqshPzVnk0q8SrYlgl0Zjtf8wKeWSfgz3xmB5OVRmGdo5t6A2Xj83uHTTipT36B8U/tcyBcM1EtrSGBBCJL6ZNFJwFJ7zY9/IuVPue3lf5pstAhana0X6jJqH0dghBZtnEI5Ducx4EQE5seFCsXmn6A8QNkahqI/HjttMNoNXiDnktBJe6siv9ozOG X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(107886003)(2616005)(36756003)(83380400001)(6666004)(316002)(6506007)(6486002)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bPyW1ev6xKZmLdJ3AzbSmFKjh+/Rv4P1ED6E0oRPh56fREIa0uu+9tZ1Hms22d5zodxgI0WydA6sTBVDHEl3FljVVSIE2VMrfoJHAmFHSPMHD5zKP0yj9HTs1A5lpLi+iJotSxwkSQNhNYA+wG6/tRw1LSrKhap3ZhUKEZ9i+RqOWXiOKRiBZ73MRoBjobP7WYNKW7IfZIRodhtkz1Pe4a5zVR4fELWWxoQQOrsR90aNn3kmrjZ5bA9memuxrk9FbH1Cbd/xjj55PZgmXWVOrl5uOSK07tHyEZrW+zk6Gus6IhBEJosOJ0doDhTawqNbQA1NmRHOcS2WkBif/vwJo/6Yu+ExBk5T0Zh7SC3AmycGlkdCSuJIvXjh/fcI6NMfk+3we1qCPGOvM153N0qBMuTm0RSz9ozgBU14lpE8TTK5c/7hXjO/UUdQhmEQOulOLuORShMxcA3vKgAgXx1wkHtQ2j906K8cojliSixthc4qOJmIw5CKG8joNWfBd4j8kh7E9PrIO8dkxxTbv47XNafVy6YVeGBzcNQVuNOEzDWy8KEQ3ktLaNAAD8Cue5EL0RMm7nk3+13xqVXi0bctE7v6AEzy428X28DY71LpkE5LGxlLcXNZ5UlHzO/KL6SGIWjkJ402C7D/STuWVaeJXhNwL0x5dW8Ej5aXHsbeTjEguAbhfzYCBMRaeKAXwJpmMLaNjQtYBG3Bje+ZggAMtV8pokhe4niRQ3zmN2N3tq/Yw0y/j+d6tm3kIS0Dp8GF2CxH7Yk+ZTJt012dWyVkA6OnnxzS2fnXyINijTWRD9UVJ95GAP3RmAVIwE+piQRZ4LQZ9YFVKbfYakrqcR7xuhk1AOv0g2nEJGTaRhVoBGwLVCJd/xSCGUirepliq6nx2K/8J2rqp51UgIJ/AzlrrLrrrAGOT80o9OkG3EFljpLDlfsc6bIdBE3e1QSunMdN/1gVmg3nO+/27AdmgWqOOL+nexJ2c0DY8VhB50Ab/RAb15mFeF0ldFsMoMaUMQ6SYHe0q8es/tD2YQO5WiO0lMdNpOWpHPsU1MuVnX9L4wFuiqMF4uBSYxncH/udCCvn3fpFXtwDZ7xPc9p4g5lXishOBLFJBlRa4mH9/HpneXCFD9/doHo5Hgti5ZtO+XQk7Lj02EdhovZreaB+vHgFraMcHii2ndmylQ1/+igc39Ts55IzDL7M8Xunoy+cUegCBtJjemyOix9cnTJ3kk5aH3NXaasgsg1Di4iLQ1oBwS6ZDiwFRytrd1e/B9Dn+gUsduyJccPVbJFFKKX7ikDLp2c/E0O7f0cXT5WXXj39d9rGY8AM4VelilyI50KT8v6pIV/suyCo8Rml17nO2M3JAjJnvSV+pmktp6VKyNcvSbLROIiw+gq61XOLV6M7LRQwxj5EAFKku6satGKJe+Dlk2oolJtU5VcU9Q7t51aoUy8g9VDW1NgHtZYLfdnAvu6TVgYQLfoCIvWzVEQj3cQMRkeeHnQR8B+liRcsNc2GpR9UnASTdc/+s5MGDr+tHfB/JIbHJ/UbluGyoEtAKnawqJ9mB167kpjFPccBx3SS7S2aEJUQfhyzklGuZscBh1cKNG4++yjX/cd+DCnyJKpvnQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: CyAPiJElFVgU4zVTdiiOIxqzXvSHzsuVrYHFear/520Zjv3Lchx6bcrRBp5fody0hkqgfgw3sVCcP1g7F4aSv1SjIpVRu3jrn7FJsiu1oB7wrbi5RdrvVdYy7s6ab0QQfELMHtUfyEHtWfH0n1pk7b8gyPwTn2Zmk4BLjzDmikqHkW9baanfBkkrygpAO2WagRYxWIND1m9BkwHanYt5CSub1xdxVYpETZ8excNbqwM0HsKUZxsUkUryyXmamZMCKYTgy7+M4zW1iV/RnjkzYarv7nCU8CwQ4OqARSZQ/aSxIg6mfYXVEIYtl1/VmWG1cHJe+/f3JRitUtJsuQLA14fdt96CjtM1fSSfdGQJRIgw4Ayh6+ccNthN6x3Zq1tfWp7PaskB5dWySYdwxJ/QtJ2bJHfzGpZKmJc4smyxO8rXJDdq0aM98cEg9dk3Bc0OubhF3oy5m5lPE2bSy1XPegtSseI4WSxg3vMMCnKdhsLVwmIvpwFAjHgWv60e9Typ6vrlkOxQ+N1R876NP9Il6UOnBHiaW0vnHJUwqxVvpz1aATi4ZV4qp5TgX9PmvTaweM48cFKL6LHuFVsQSETsUrTKUpIr+Rx+Fb/q0FHAjOI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3a6d2261-1dde-4e92-cee4-08dc2c5897d2 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:59.6990 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ItYNgoKl+mfjsCVE796iDedzpAUTuTh7XvPpTZ0r3KmTEMKszNHOcxDTcrtX1dxuRIhLls+8uBeRyLvrJx+mtNBp2CTOV3xYr8u0/qDFPVI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 suspectscore=0 mlxscore=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: UB6fNBix8VZudmMcdR9WmDz0v3Tti47a X-Proofpoint-ORIG-GUID: UB6fNBix8VZudmMcdR9WmDz0v3Tti47a X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762606786539557 X-GMAIL-MSGID: 1790762606786539557 resched_latency_warn() now also warns if TIF_NEED_RESCHED_LAZY is set without rescheduling for more than the latency_warn_ms period. Cc: Ingo Molnar Cc: Peter Ziljstra Cc: Juri Lelli Cc: Vincent Guittot Signed-off-by: Ankur Arora --- kernel/sched/core.c | 2 +- kernel/sched/debug.c | 7 +++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index aa31f23f43f4..cce84ba0417f 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5670,7 +5670,7 @@ static u64 cpu_resched_latency(struct rq *rq) if (sysctl_resched_latency_warn_once && warned_once) return 0; - if (!need_resched() || !latency_warn_ms) + if ((!need_resched() && !need_resched_lazy()) || !latency_warn_ms) return 0; if (system_state == SYSTEM_BOOTING) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index e53f1b73bf4a..869a09781f2e 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1114,9 +1114,12 @@ void proc_sched_set_task(struct task_struct *p) void resched_latency_warn(int cpu, u64 latency) { static DEFINE_RATELIMIT_STATE(latency_check_ratelimit, 60 * 60 * HZ, 1); + char *nr; + + nr = tif_need_resched(NR_now) ? "need_resched" : "need_resched_lazy"; WARN(__ratelimit(&latency_check_ratelimit), - "sched: CPU %d need_resched set for > %llu ns (%d ticks) " + "sched: CPU %d %s set for > %llu ns (%d ticks) " "without schedule\n", - cpu, latency, cpu_rq(cpu)->ticks_without_resched); + cpu, nr, latency, cpu_rq(cpu)->ticks_without_resched); } From patchwork Tue Feb 13 05:55:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200226 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349912dyb; Mon, 12 Feb 2024 22:05:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUssGE80ja0KaEg8kZ+TfFB+8fxjCQx0hBisF0ohlFV+PLXLCcRZ2VUz6lXehgyMqewlvXeAWlEqb2H2AyUDU3rP3tvjQ== X-Google-Smtp-Source: AGHT+IGz7ncc53iLkSPNYtgC8nGVQbYrF8nq0WLtYXAFKpmW91mrhCa7MkAtIeGo47bNEmztGpbY X-Received: by 2002:a05:620a:2ae3:b0:785:db2d:d894 with SMTP id bn35-20020a05620a2ae300b00785db2dd894mr3055160qkb.35.1707804330405; Mon, 12 Feb 2024 22:05:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV6gs56wymVmRlSQXSWYKBzSTpfPRPDhDy2aTyY8gx7VVEysGOHdJ4A241FDXgpKgFaSmRH4Zp1X30qBXJIor+vdv6kQA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id vu14-20020a05620a560e00b00783322de747si7701287qkn.339.2024.02.12.22.05.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:05:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62986-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=lY6q1Inl; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=0GHqqeSx; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62986-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62986-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B5BC21C22F71 for ; Tue, 13 Feb 2024 06:05:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 82F3D524A2; Tue, 13 Feb 2024 05:58:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="lY6q1Inl"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="0GHqqeSx" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ACEA50A77 for ; Tue, 13 Feb 2024 05:58:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803881; cv=fail; b=nvGPME5marCEh8Bp7yuKFravG/0pBiz3kM/t0KRtCqdyYZSgFXvO+jBNm3uPviJMbMhxj5ROcYycSmr0nMonS2wvX9iAlB2xNIG8QDUrd61bH0TorDe3VU3LUDb4o0ilEGS1e1KT+Dw9kCK2IMTUHAtvnhUWe9bseGUQgDbAoD8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803881; c=relaxed/simple; bh=HWMnr0IL3/+UnCwwU/rSW3D61iUcgJaEY3VB3Gw22UA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=flCRZpkgy7Gq0/0yOLfyiyT5Jme6TAs34ppxWLy3StAlhCon2ld8aG5qF8/JFDLHfvLRbJhgmTTZjyvlojcVXZEXIU4BNo2N6Ue85i0s/LFJbHfoFJs0hSQ+W73ipHXFg1AGIjcvkRLcY+8iYdOcMOaKwoK7q3gVIuREguDVnYI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=lY6q1Inl; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=0GHqqeSx; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5ng4d028881; Tue, 13 Feb 2024 05:57:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=2ISIDIE+DWdtLOJcbwLdBeVJe6nQP559F6WAgS9bf2A=; b=lY6q1Inl3m5SZnA2NrDjUiHZw9YdBlBPAKIDo1E3Op3xWlAjhK3zbp5iQtE7ghKNe7Dr JGzyabCKvdSD2995DcT6LZG81s8+UhxRYNGoFrtXvIdxQ52B7keeVvF0SXCKyi9Y7mCH WDNSFwPGsqxm2iDDIJVbSUNdqM+AfzWGWfuDnnrdaG2STibOIMJAoDMMHu4ZmsXsa01T rSGfcirJYkm/9fXUFfdrEmiYSVRpBeNu2SB/fbf4rFRBue+QK23htK24G6jy5ovodCxG WlsuLN0NKzT7BP/nUiVR252VuCsKWOcVwbhsINT6B3x+rVMUE0StjyagOntiFRNAmhhw VA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w82m480kx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:05 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D4QMOQ015080; Tue, 13 Feb 2024 05:57:04 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w5yk6uh6d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hYdRrHnuD4POoNXv4xnBuftpInZJqvHTotsdjI6pkPNHBUbJUgO1UUvjDsNWkH45Ij5hapX8lt1awpBwRF52lNErL0DtsBnzZq/pGTXqBWVBIYU/KbZB3KaaCKRXZbm2RBM3GszVS+qMRXmn9anJjw7U4UieMptaHySnmhLMIA7e4URSDT3npPCKjpyxlNg6CKQ6J6MQyYQQEVguE96j8gvnc/vKMtmrRQQB1oRZJvNgMwBud8+sFafZIiyVVhWnRpfmKZ21Ar1fZtFYfxfa6fvRtHt+HlYzeiiTPt+Ra94VNWSx93Eo4LJ1yaMy62JDMj3K1z3P3PLnEKw1ybtWtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2ISIDIE+DWdtLOJcbwLdBeVJe6nQP559F6WAgS9bf2A=; b=Ifr+Lr7j3BmjX/NF2m+UTa2JBOq83+2fdxAj01O8bT9cmK8tQbRGrQ0p08dC7Kw6m24Pc8oYYH0BEuSELNRv2mtp5TpMzCce0MG0KaEeMQ2yP/Rf+7qdRuL1HuTTvNHdHFDVnboAFfT2OilTs8yaKayGMygr4xpGXQe7/Z3zflfBagskYEhDPirBdF/h0DP7jM1DjA2dkBXXFCSJLu0ycbWGt1/ol0RiiKUYfQxSZEAjk0fcqATf1Po17SIYunLDWGzAH4a7z4Hlb+EkB9vtldU5oky9ow/Yb1rjxBrN5k48EZwRNU2+3+yaJlF2hm21A5ZE/cLxfiqtmVMvAuZ/1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2ISIDIE+DWdtLOJcbwLdBeVJe6nQP559F6WAgS9bf2A=; b=0GHqqeSx476Pc/dEUK41si3+GqSe+UcWRlN24VQ/BD1SAyjow+qrNKUM4Bjjexe1QUpO9P7faSkVNfZH9FQQF0pDRZbdgEJE7tyLctqn/lWJyceSFOYQNvH6JbgknCc05xmlNsc3Q2t/nYpcMUKBtzhbcixYp4ArTneJSiTf7kk= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by CY8PR10MB6906.namprd10.prod.outlook.com (2603:10b6:930:85::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb 2024 05:57:01 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:57:01 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Masami Hiramatsu Subject: [PATCH 28/30] tracing: support lazy resched Date: Mon, 12 Feb 2024 21:55:52 -0800 Message-Id: <20240213055554.1802415-29-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0165.namprd03.prod.outlook.com (2603:10b6:303:8d::20) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|CY8PR10MB6906:EE_ X-MS-Office365-Filtering-Correlation-Id: 828b5498-a7bc-47f0-5333-08dc2c58991b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: adkPco3e8A1sUf0m6vrr6QRN0g+XgKcef4qX4SacAM/Fd81Fu7MtgwYSsBUj7qdvl3uu3LunceHy6IVCOu+eCrSuMKpwCo4IemokO+xgTLLLBBL1G7/SGnmmmwxbAnH3VME7uzYsLmfn9boHh+ua+l0RdiMpBhKuJv53AsvZhn2sGFj3YGkUy1+Ph2h3gZei66pevBD3sR2P9p2dfSfWapX/Oa4rUdutTdTT6TV+c/Twbf05JSMhFGPmCvj6Cg4Vk0VXB9eomVJjQ/zhuAfVBt5Mvpji3stChQ0BMuw4Y8D8Mk+Ur4RWWGKHOZcpRdCzlzDIGW5E9UtR6DkcMvaWz3fKcq60qjB1HzN4I90rVn5GAvEm8TCeVvwwZjfN1nhuH2qUMhMQsBMCUcvrBFOPniu6g0Rrq+3xyHeumodDec+thLw63+xN0rybbpUmbZduZPVwgmmA38TQAq+RZGRXdZIWju8Ngpk0pIvioyx/cKzSyNxmFf/g29W2bNnQIHP7aY94H/0QKO0riAhCbD57u+1fOcigfqb+fL4WTwcNHTY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(366004)(346002)(136003)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(2906002)(5660300002)(66476007)(66556008)(66946007)(8936002)(8676002)(7416002)(7406005)(6916009)(4326008)(86362001)(41300700001)(1076003)(26005)(2616005)(36756003)(83380400001)(54906003)(6666004)(316002)(6506007)(6486002)(966005)(6512007)(103116003)(478600001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qoiYsXh639pejsSB5FoL6ve6YV6FHEVKhPm5a+LoVfgMtY7zDMGfjxmnb++XtwJ3WnMts4E8uOrIZgqcSqYMBmwaWkrCyfc+PYzfaBt+mOab1tLMxDAdo95F8Y4dQL4G8862RtLPQ3oCgDUoe5GSbmMPCui7LPZGqhfKs0rN1AUD9TA0XSpyK2KKfpiBScOA4m/jb5+H/MsNreFLMCu/u4c8OXrJOtMFiK8EcA+1REsNyHvDzRg/2wr3fw0q/gYc/pKZnIvS/NY/FesHdoyRfJzZxyArKlh5mtM7frCE7eEGhToHPDBl9gI9mg/DayxaEFpR1YpnR9EN5MyhGOP0iZXxSMy53wSX9CG2KsNXf3BeBX+NxENIVbvVe6J6ghh1cAY0c96EuKnDrYMAHjRiA0A1LjOlWUSnq11uhVbM75fg6VC3X6WsN5k2kcuigVPvsuwdn/FX32qv22nv18UdhxJJQq2YXaGQpr9+XYUjPUsPPDDVVMD5NfDFauzSg19Cm3IgszSoec3FkZVE11pR54iwqDd+oqDM0DMaplslM6W+qy+hO8w5FvkAreR/Vog1iOdz8PlnLDvP/FpRN+Wk9WrrKHSVIfNQDTiomndXfkr0s/3S1LZmhTquktqowMUM52l8UvYnFOWI39y5ky1zWWI4m0Cmgv8n9GrgknUytFdMjqPXD3ELfzpre0ebdrOvZRD41JjWVF6BczWH9Y7SPQGxdm/4HglhnTNd+RE8/rW/q6Y26z3xW2RvPLk9slIPOkes5Qjl5xljkYbeBwegaZCaz3Mz4rP0wgzFQFhMIGkvaaWcWyX6WWWz0bzvcHZnRubVqZcl+QJ3xdrRYWeGudidGmO9iTe7/Qk7Vyo8yWNg3OxcedbJcHMsyNOU8+V1KIYPS4Qa/RUXTT1MBocnoDD26BpuFCbZKoRRbctwtcu2z1q12oQBlb6hJuuC8sWW4Vv9i7I1a7kVkaDG7pi9byrX2h2UdV/KsAAD/kYfrjBJmlcCK3uZk/eqfSVHrZtP/DoYhC70SeSbp+X5accRZ7iu4L6fz4mCC+jcpdvNRPkPZXwnE+ZmRiZKO685+CWMkphSU8CSOHhQEF4zNkWwcN08KEG/uC47jGU7CyfDw/ss+WjGIdmj2uTUBIHLNAucdSfYrsyIEnLv0X2B7LYYGsELsRPZfKHMbddnjzGbL0K08paTc/YpDRRJ26Axc7/UL8+Kbsu04SftblAhEwqPRRULjSPwEdN8iGHawq64fq+4uz5tC8R7gkXUnzeDhGitwoiGdTE3mcLe+1FkNSTRIlW03HmyQO96TebsO3C12TDrK0SBGy4pboXm2UWvMv9srNyvOSdAB6B08taCzpCnBVAlzuJH/vliqHN4IQFtfP6ELL9IBaDN/deCkEMEPTImZ9/iE0dy9fYSGIgHZQsKxCeJCeCaLi4fhhoalZX45TIkFcZBqrHh2AeF4mEu1565kb1YhfrqWIviItWdXDImnlYkggnaG8M21VAgUOwZfC58VvVDLyLGfpiBJCjYKRCdpHGtNjbVgv/WEjAJrqV4LI6NxlwI+uUFW6OgOSxQNy14Z81fQ+iNIQEdd9raqUwkprl+ZgRI2Dm6bFhj+Kgtiw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: qJqrWrvKio2Ushb/Cid6HMKzH0wiJoF+cs2xEQhGe4ixVpcFZYVM7KrG/n2NIXChjcxjxeqgRVDxFCN/m51pmxMIcaQUMmq4VZJzQmv9HoO0DxVeXa6/VmZLw9O0vKgNiT5CmQcIXpPbNOSqliEYR5WY5ig0pYAZ1LXqJCwbOdyVJx9KhpzVqdNA+R9dV/ZXdBQGuJQAPam9AtKlHbeEmUVwjBah5fqm3iDVpUAt/1cooarJO8O6I5JAozJ/joOeG7d9T/4mAU9Q8Yv5ujOIxnQYZ+TqZNC2EI14mG2q90Hvtx+0umf8vhf/isCMwfEAXaD6LSG66Rfd3s2C/dLNYOaa5nZ/+scJ1NLFUHUYHXLjJva6wurMHJh+Ya8esPiX4ROdcNJ2uNfczhhq7fdt7hMPU0o7grGo6d2t3b0lZEcL/msJDw2NNu0+vFMG/omz/PJBmAU40w7TzoEn8jXwSD51pwjSyxr6rjtlT54NZRf6bUw0vXpAv30Pj7OjvtzR6H5hKoA75x3sVjIZ/v3YEEjETT7SzKfreRbltrZA+FLNlze+Wm8/ONWGazA7yNJ2St4o5fLHXQDm+LjnVVNUgj6QOw/gAew6Oddpz1ktdcA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 828b5498-a7bc-47f0-5333-08dc2c58991b X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:57:01.8305 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KfkWEoOpRn+l9IyvJDd8MZvom6wQPYlSL2X1xcmZGbSBfhSbsHxa3JAzI38lR4fsa+o5/jdbj7ZbNFNHKDNIqDpho0FfmM7c/ha9t5XdPXQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=0 phishscore=0 adultscore=0 spamscore=0 mlxlogscore=999 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: yD1AjVAKFwJC5zOj-JdfyINs1mc27Kqd X-Proofpoint-ORIG-GUID: yD1AjVAKFwJC5zOj-JdfyINs1mc27Kqd X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762633120420000 X-GMAIL-MSGID: 1790762633120420000 Tracing support for TIF_NEED_RESCHED_LAZY. trace_entry::flags is full, so reuse the TRACE_FLAG_IRQS_NOSUPPORT bit for this. The flag is safe to reuse since it is only used in old archs that don't support lockdep irq tracing. Cc: Steven Rostedt Cc: Masami Hiramatsu Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- include/linux/trace_events.h | 6 +++--- kernel/trace/trace.c | 2 ++ kernel/trace/trace_output.c | 16 ++++++++++++++-- 3 files changed, 19 insertions(+), 5 deletions(-) diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index d68ff9b1247f..0c17e9de1434 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -178,7 +178,7 @@ unsigned int tracing_gen_ctx_irq_test(unsigned int irqs_status); enum trace_flag_type { TRACE_FLAG_IRQS_OFF = 0x01, - TRACE_FLAG_IRQS_NOSUPPORT = 0x02, + TRACE_FLAG_NEED_RESCHED_LAZY = 0x02, TRACE_FLAG_NEED_RESCHED = 0x04, TRACE_FLAG_HARDIRQ = 0x08, TRACE_FLAG_SOFTIRQ = 0x10, @@ -205,11 +205,11 @@ static inline unsigned int tracing_gen_ctx(void) static inline unsigned int tracing_gen_ctx_flags(unsigned long irqflags) { - return tracing_gen_ctx_irq_test(TRACE_FLAG_IRQS_NOSUPPORT); + return tracing_gen_ctx_irq_test(0); } static inline unsigned int tracing_gen_ctx(void) { - return tracing_gen_ctx_irq_test(TRACE_FLAG_IRQS_NOSUPPORT); + return tracing_gen_ctx_irq_test(0); } #endif diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 0a9642fba852..8fb3a79771df 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -2703,6 +2703,8 @@ unsigned int tracing_gen_ctx_irq_test(unsigned int irqs_status) if (tif_need_resched(NR_now)) trace_flags |= TRACE_FLAG_NEED_RESCHED; + if (tif_need_resched(NR_lazy)) + trace_flags |= TRACE_FLAG_NEED_RESCHED_LAZY; if (test_preempt_need_resched()) trace_flags |= TRACE_FLAG_PREEMPT_RESCHED; return (trace_flags << 16) | (min_t(unsigned int, pc & 0xff, 0xf)) | diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c index 3e7fa44dc2b2..5e120c2404cf 100644 --- a/kernel/trace/trace_output.c +++ b/kernel/trace/trace_output.c @@ -460,17 +460,29 @@ int trace_print_lat_fmt(struct trace_seq *s, struct trace_entry *entry) (entry->flags & TRACE_FLAG_IRQS_OFF && bh_off) ? 'D' : (entry->flags & TRACE_FLAG_IRQS_OFF) ? 'd' : bh_off ? 'b' : - (entry->flags & TRACE_FLAG_IRQS_NOSUPPORT) ? 'X' : + !IS_ENABLED(CONFIG_TRACE_IRQFLAGS_SUPPORT) ? 'X' : '.'; - switch (entry->flags & (TRACE_FLAG_NEED_RESCHED | + switch (entry->flags & (TRACE_FLAG_NEED_RESCHED | TRACE_FLAG_NEED_RESCHED_LAZY | TRACE_FLAG_PREEMPT_RESCHED)) { + case TRACE_FLAG_NEED_RESCHED | TRACE_FLAG_NEED_RESCHED_LAZY | TRACE_FLAG_PREEMPT_RESCHED: + need_resched = 'B'; + break; case TRACE_FLAG_NEED_RESCHED | TRACE_FLAG_PREEMPT_RESCHED: need_resched = 'N'; break; + case TRACE_FLAG_NEED_RESCHED_LAZY | TRACE_FLAG_PREEMPT_RESCHED: + need_resched = 'L'; + break; + case TRACE_FLAG_NEED_RESCHED | TRACE_FLAG_NEED_RESCHED_LAZY: + need_resched = 'b'; + break; case TRACE_FLAG_NEED_RESCHED: need_resched = 'n'; break; + case TRACE_FLAG_NEED_RESCHED_LAZY: + need_resched = 'l'; + break; case TRACE_FLAG_PREEMPT_RESCHED: need_resched = 'p'; break; From patchwork Tue Feb 13 05:55:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200228 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp350013dyb; Mon, 12 Feb 2024 22:05:45 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW0AxWGjeCkaKSocupx1+OsOnV01zOWpH5MpcH51EF/yupNZh2BtXd4wOWj+orIKTna5JSenRlGz70gEz5IrkAs6MjCmA== X-Google-Smtp-Source: AGHT+IE8nJOT1UT8/bsI9RTogE57DSAwOV6FEVziVMEZGB0n22WUF2aGfkNIT527WTnAX36dCgEy X-Received: by 2002:a05:6402:33d2:b0:560:dfa:61c6 with SMTP id a18-20020a05640233d200b005600dfa61c6mr5516861edc.20.1707804345032; Mon, 12 Feb 2024 22:05:45 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWZ2yMDRtmB/WoPAmnzeQ3CEoV+f25Uv86ekZ8dLH4TZVa9bzgc83/jtxu+2QVduisA5icr+UOVkbYFg7w1zJyqKNEpZQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id x19-20020a50d613000000b00561ba9ca3b3si1656641edi.633.2024.02.12.22.05.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:05:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=QT18keuO; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=JQVixab4; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62988-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 79C3B1F25108 for ; Tue, 13 Feb 2024 06:05:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5CA96175A9; Tue, 13 Feb 2024 05:58:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="QT18keuO"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="JQVixab4" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B5DC15EA2 for ; Tue, 13 Feb 2024 05:58:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803910; cv=fail; b=lbu6U6xHqxCPM5CMlEe2PD8qB2FWgzUx2Q+Gl7yJx6qZAXsobdkpLUQAaT6b/EeNJu16ph+d+T2SggiBpjhk3YGwIg/dxzjoEM2m44MKeU6ufAB77Gbk/yVG/1h3tDTnId3IE26WvCqoebtwA5veyft611PV2i4ZNGDCJCubFrU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803910; c=relaxed/simple; bh=Hh7g7P3QkviaacmI8bsuKVGg/QOjNTgaUsSh4DIuSqU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=YqX11bPRU0Rg12R4pik4cypD0e1Pg2kIoHfeGEKEpW4FtN8cOLhK0cin3SQj0BPIz3KoqPsCTa1aW/IvIvLT7yypY5hn3SowCrH7+7myS0qSkO9zmPMwZuXOFTebjQWCp8wD163ZbXZN+J930PTHKjeJlj2APfX7un4dgJb/dD0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=QT18keuO; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=JQVixab4; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D503mB000622; Tue, 13 Feb 2024 05:57:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=1gzu+2Xh2YBhraKNYVXIbFY8o0B8DJKjS8/hWnD2p+M=; b=QT18keuO82Aj3lX+zVQYXUNsa2n+7+ffNRhBtsXVB3BDpzkFnPqG08Vnp7r1xvRBl6tl RNGXDl0kWCoqk3c/SLCV8yUVN21jbQDlDwfAnlcHl5eqtnmuIp7M6l/vyOZ/dKBXW3BQ I2WOTbfiAJIg2X6pjChjMmCOiCWrl3n0Mjri3Ba9JRlg5bGHRL4JTtWFaO8Gi6naefV5 lBTMrH5OhIjXmxbiGFSpoGE4OIJRyxyutXSPdeKFZFTNAsvXBxpS5wfmYYn5WZYYbFJX Da7OT5hAzl5EjNQbqhSkkv8+jERYwhk5VN8XuFj3yp7WuQB2W8fLUmvX4rAaq3I9oqVq BA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm83a5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:28 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D5QHSY013775; Tue, 13 Feb 2024 05:57:27 GMT Received: from outbound.mail.protection.outlook.com (mail-dm6nam12lp2168.outbound.protection.outlook.com [104.47.59.168]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w6ap9gpxx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZL3DlOrC26ntJOet7+3V1yzRKtft6mzNcJs+4NBFAUwZNq9LcJwFB0nJuo+b26NrFspHtddD25ywm4ye8OosyWIu0sQ9a+CtE8He1f42jnd9E2ShChhL1UxfW7EgG92tDgZCDiXEKUGkojASPg967iPr8QOmzkrcQxkdCcnXEpyWrhibzfuQnIms/NXFWhLijtDfTOj4ovKYnYFmexlgrLFe1XDUa2gnTYvpANeogUOBKeeLWE+vsU1ZmRWmDc5x3ePwLFVhpX9Aya+DkeN6iRvZ4ltdVbfqwYVtYT8MPtVyiSudPxPNyT7cGIHaethdomXGIfNokBZ2NfMO+258NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1gzu+2Xh2YBhraKNYVXIbFY8o0B8DJKjS8/hWnD2p+M=; b=Q1Yej9TwzWOkrf+OlWfpufTMQJ9x9LLRi0NoSh688jXyRKeHV4IIHvS+VN+5MkmMHSjx7d8vN+v3jS++4w886qMdgkhwBzAEGi2TI33XbsixF3uKKglBpw3ilJuZGj8zVKy7WTqndbrslFuotZvXggg19mrQoxvsj8DGkpYo8ODYBHD28qVqBhCul0JF5aQv3vQsnqzb9O9xCEdCJ27ZFIXT5nXCrXLL5H4x0WjkIWitIwJHTn1dwWLvS3dEh2X3OshdKqldnvqdPbW1wWMmhQmfP5yXCkMjsW19FNHauAJXGvfFmqLvTLIK62jpmJLksS58u3FqDXhDiJp8dOGjvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1gzu+2Xh2YBhraKNYVXIbFY8o0B8DJKjS8/hWnD2p+M=; b=JQVixab4MviVW41RCfQrMbGR7rmpgplKmHM/9Iiwgp4xhlbyEg12cyWlpDYm4oW3hiUHzmQKj8R3Oe++tPdwgOgVme9cDv1O7dXO0TpXhkczvnlHi7SHeXQkwsvBvAca4/T9OW1hAAsMydm2Oj4QdsXbTD890VX4A+MT7JsOdLg= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by MW4PR10MB6533.namprd10.prod.outlook.com (2603:10b6:303:21b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.32; Tue, 13 Feb 2024 05:57:24 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:57:24 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora , Masami Hiramatsu , Jonathan Corbet Subject: [PATCH 29/30] Documentation: tracing: add TIF_NEED_RESCHED_LAZY Date: Mon, 12 Feb 2024 21:55:53 -0800 Message-Id: <20240213055554.1802415-30-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0177.namprd03.prod.outlook.com (2603:10b6:303:8d::32) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|MW4PR10MB6533:EE_ X-MS-Office365-Filtering-Correlation-Id: 5b7bbc1f-26b3-4622-75d7-08dc2c58a66a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pj4Dbc/s50EIdSYxxBlH0G7qSNx/kofliQl4lmqqsGpiBU5s7WFwvO2qkVOw4M849iP1g/bvumC/m6QWE+NjLxDscpDhYgdUmrczSkEV+LVoWwF4PwIizAuQ6Uq6vY+YptGxx63cxPhgwip3PMspJblF+4wM8Fo+Ykh6L7pk1VwW+oumJ2ZNJHdvvcNIaeWvirha8oc8Mu/Pa15FFTemwAX7b6wzyDq+riAkRKjZg/9mJdJ8Uw6bMhSsMKICsIr+OWmoihiR9FZUwXcqwG7v0FXmyL3ymr23dP0eduH+jOtr725HEb/zTHFJAodjOJCWGp8P8XUhODemRy6A43tYy/k1HecV6FQby+7zU3eYZgK+DGEuOdHnWfLjpnBk70j2CreAJqhMGlGbhxIDeYmOl3IJvs+lly4R8NhhFcunANadxdVLMtqa/+xvoWBgsaVGAq2RuHyIdP/UFngFXw2p7RMit6uVKMwz8EmgSVGSMXv1xMO2oE9FgFIxgLYiLRaQwCV9CVW+FfY0FlIv79nL5tt8vE0cdJAixDIsUuauCTE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(39860400002)(346002)(376002)(396003)(366004)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(7416002)(2906002)(66476007)(7406005)(8936002)(66556008)(4326008)(8676002)(5660300002)(66946007)(86362001)(103116003)(36756003)(38100700002)(6486002)(478600001)(6666004)(6506007)(966005)(316002)(54906003)(83380400001)(6512007)(41300700001)(2616005)(1076003)(26005)(6916009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TZrf0EvXLm7y3QZo/8N75mbtRYGQ2wheZTJDBJRtwIqi9yygF2Syk5mNWKrk0X5JewftGLyaSRbBge9LJCyIc/u5XFMYLlmrL4KGvSmQAq6C6YcxkgJZtLk1zckUArNlORPdVyTo+fSeGaQ0gh+Lcugzy6xUDS8zyChFlOHs7D9mF/LTlPtAlopZihvefPtUqBKQKOoOyw95AcHVg2cCRtV9UvJElGCA/fsGjNlBKpPt5WATfZLFaD47xfLS0XRGQA17t/e7i1F5k9hN0NUSOw0PS5ErbJ/dAJjmYmNA0GJ2BWgXwpr0v/8GnMGy4cCmmAvJRTN+xGdmZgsQ3kD1GDSZQXe4x9luMK7v3fIEaiAMjbtKxDT1BjpDZ1ggz6IqCLZj/EjCo3ICKTO96PPKzbYBpGO8HlFoSxeNgf/TQadJ2EcgWXVgNDf3pxmIiblZ+XcUoZhVEAWHWvwkQEEdLBvc4Q/MbnxPIFKQBE4TRFudE/DZR0dJKJwY5e6nDXpHbLhqK7Ah1QYpz9Rq6BD1F80U6b+nEW3meCkVvE3x4E0vlWYP2Ym8P8VBNjXY4w2mCKbfN3JYmeVCdpHKbVFBwNzAeLvtDtdF2hait8wkPsiFdrrVCSVCZwE4Q/74YnZ/A3KH82Oom6IccxO6q+yAdtw8sFfJyJiPkT7YyAWtBa0np6emVxdDXK5+qpxci+66a6OWeGJGn2XxzGlEOVBfE7kOnh2NKsVg02C+bZQBN24GaWEKmtir9BBrjC0B1IvNK7tIKd970S7Vntmqo0z10A+SEhUEjf5Oy5NRF9heQajdVQkYpPZyAjioHNWJ9NY1H8kc4Pb1YxlHOC25fkK3eAYCBCh7uKe6f45dErdyNFz3icmqeP8DGjFvuCu/ncL/dKoh3QOaxZM4vYhDpXXEfC6av4YeEDjvoK/ka2OIoMfoCdk77niysq4Rua4dEPwREWtrGmvJ/7QvziFGQfAW6uPEjasjWRoKYrhzYpr8sdBlA6La/76udYpnBSgOyhrjupBJyBqjXDP6ql8GthrE57ehrZw5IxSESq2W3jVaYOi1S6Vl03qgVrRmjewbChPwmfVRcfJ6CZ8qZl4dAtoW215oW0nqmTHF/E2dpwWdAhW9SDaQD9ikV08onQkXs3xIYpwjTEGaxEl6RWHhh0y6lWrwr0lbvDCuFYn/KmBil6bERHHipErDadZ8jFuzXgwtNC9vr4QLiFzf/irapSitJ1xthcced0AvFXT0Q4naR/q2t7c9GwwQIztZsOO3e09Y34rubabpjWV8n61hVkK2PjJIbnUspzbBg66tBWljxpBL+3+WQljc/5m1fZewKeVqhwfk/VIqwx2FJvox/Iwvgl1KYGwGhxiAkBWJw5r0go19krSCx2y6+B5Rk8ROoUObGGTR+YYmEaI5mA6fkr3FSVOWGDsiTHcbqonudPCw17WtQO/mKw//8NJCA7QYv/piH+hfvt8ifNMDVVVIiAoJu9nRmoXg9aE++6fNDf8Ha36XH6OCg7L+fM6NNwEzFR0DBxJbeHhtfEVSXUhDZqaOz2/ogVHLiiobAIaIlYAGeodrtROw3coO0OPkb/CW+Efj3+aINiAK+KRCqclfELOMXA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ZBPDhbN/YMpX7p/npeQhnMfBrbexS+Ap6uZ+tah2j1xBZ30zC4k4freu1/AhxDeKVmpEXmKOXELImICFVZxyh7+5JCQWXu+PxytNZoAznd8U4QQVWDN49sOWiVikqqmGO/i+8RBcO/yKo2I+Fpe5ckOcQZdDlqIw4VKUkKQkW2gWXEzT+CFQNEXhmMT0xrs398ttUrsdtwwuqQk2wEjFbYi+HgE84zzmraRxOn4j6dPpkVPokAf/IeSgMus4RzGWtonsbO7KRPDeqMDj6ArXb2LoMe2WBdiFEaRL/4mU+rA8E1UAyzE8Aui0pCXAlDVRAgiDM9KwUbAQO9uPso3Xqsn20lul/BsSGEKYGKwikAKZHq6Xcu0ZAy1/QNyaqiTmZJE6D/h4JQTstmoAqsgyS35RSEJgDOzzbsrYNlNcmUIwHHdOQcbygwe54l/azL5BiAV3CkZtxNIl9pksSIM91guJKPS3hiOKBdCajf7yXtd6UedPvjEjsfQK0nQYe5ULMxGgeKfiFA6xhPwZMxm99JQPWzNT/TLuiIz802JA84z+OcTL4nqvwYukpeOdZxll1DjVhyoU09XQmb/y/z0StfbUafcX8iJxvR58pSKaVDM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b7bbc1f-26b3-4622-75d7-08dc2c58a66a X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:57:24.1526 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nQWyrluQLfoIJh/iLxZlNyLv4TsBs5xULm0NRdhAtrFrmlxda7OCjJFi5QGTAEmMBIg9MvSRoV56kC0CzvNVaZcyiOMkL32qBZeTBOkX8MI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6533 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=762 adultscore=0 suspectscore=0 mlxscore=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: ABa53rnHVsNa9P3CQUJK6M0iaJwDRILq X-Proofpoint-ORIG-GUID: ABa53rnHVsNa9P3CQUJK6M0iaJwDRILq X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762648785847205 X-GMAIL-MSGID: 1790762648785847205 Document various combinations of resched flags. Cc: Steven Rostedt Cc: Masami Hiramatsu Cc: Jonathan Corbet Originally-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ Signed-off-by: Ankur Arora --- Documentation/trace/ftrace.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 7e7b8ec17934..7f20c0bae009 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -1036,8 +1036,12 @@ explains which is which. be printed here. need-resched: - - 'N' both TIF_NEED_RESCHED and PREEMPT_NEED_RESCHED is set, + - 'B' all three, TIF_NEED_RESCHED, TIF_NEED_RESCHED_LAZY and PREEMPT_NEED_RESCHED are set, + - 'N' both TIF_NEED_RESCHED and PREEMPT_NEED_RESCHED are set, + - 'L' both TIF_NEED_RESCHED_LAZY and PREEMPT_NEED_RESCHED are set, + - 'b' both TIF_NEED_RESCHED and TIF_NEED_RESCHED_LAZY are set, - 'n' only TIF_NEED_RESCHED is set, + - 'l' only TIF_NEED_RESCHED_LAZY is set, - 'p' only PREEMPT_NEED_RESCHED is set, - '.' otherwise. From patchwork Tue Feb 13 05:55:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 200227 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp349910dyb; Mon, 12 Feb 2024 22:05:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUYgl5Lm5bP/LkGfDyTXECn20QBnM2kk7exr978G5t5WPpsleU0kuJKeMzxivvJDNePBYpuK6EF9I+rMN7tffJLoa4kuA== X-Google-Smtp-Source: AGHT+IHDHICHf2yyeCUoU8x41yyEhW03Ux5hdjYoBEzZblEwCUvGmf0bToH3PMsaBte2+PuUwbL5 X-Received: by 2002:aa7:c88e:0:b0:560:ecd5:968b with SMTP id p14-20020aa7c88e000000b00560ecd5968bmr6329868eds.23.1707804330281; Mon, 12 Feb 2024 22:05:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWNcbsxwnoUW/TO/lVOjpGEKiqfwT+W98MW8/1OG0857daji5P/asFfs/rTVqYBR4bZVk9PKO6hPvl3RuvPybjbWP7k8A== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id x19-20020a50d613000000b00561ba9ca3b3si1656641edi.633.2024.02.12.22.05.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:05:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62987-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=aNuN8AYI; dkim=pass header.i=@oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="divfJC/h"; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-62987-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62987-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id A7BED1F2508A for ; Tue, 13 Feb 2024 06:05:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4A601171A1; Tue, 13 Feb 2024 05:58:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="aNuN8AYI"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="divfJC/h" Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F22E1401E for ; Tue, 13 Feb 2024 05:58:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803904; cv=fail; b=oDWXlEOYJHftY2tyU0anVPqcAU8CThQVea7dSI8hj1y0E1m/5zQCfx6BZcqUOiC6fy9LDZNVCxAzMrBmDdqM0On2pqd9sDBkAHWuPavUygJD+ntDoP7Pmlklbu+0uh8F7/AXDfZDJATlstEcNdxILDhqe0wmaYtCrqQ5zS5cHF0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707803904; c=relaxed/simple; bh=VMkEFwCUYdfN30GB7Gfq311dRmUkjGK9iGLckSrn97I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=jt1Lw6yq0DSdJyzERmw26RnMY3DfIG+QXlMVpsVU9yC41XoXJvLkm+mD93MindnJ8gLQa1O5QCcWUF1Ew95qHPpOwYRodROB10TBnl1/g75zwoywk8eZyRGnRxRJzTSh+nwXansudA4b1udjuHmXYDwC5QSC6adGZ43WiJw9ffo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=aNuN8AYI; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=divfJC/h; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 41D503mD000622; Tue, 13 Feb 2024 05:57:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-11-20; bh=kMw/WFtoqiI/n/zOwO4wSzAXSx4Y9p7q9wdCPvVW994=; b=aNuN8AYIw3eDXzv0EjIKTPXCam8MOdefHD0yJPnCT7s0g7mfk5pop9mZIr/G7CZRWl5x N0TGdJtEbOpEgGA8GN1E8zXJTQ5EA9W5ndYPYSsjMvFO3I27nPIBMpbD8d+1BOP1MeYe NCZ/vXleHhX30WZA7qTnTzirITqnFBAjlHwqCoUte9FT980FRyf7qIGhXnHPEb43QTQ9 /kmJb7fTM9wdRrVfIOwIwjP3GXxHFa7AizIgZeJGRGWs5XSXpI/+kDKDB9h6p1rnggF9 E/l/677P8HgzU79sD7qVGXDyxkfLe44yH3+PKsWk6hWUALB7EFkkc2kt7s4nc2/6S3CV 3g== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3w81vm83ae-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:33 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 41D4hSmG013763; Tue, 13 Feb 2024 05:57:32 GMT Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2169.outbound.protection.outlook.com [104.47.59.169]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3w6ap9gq0f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2024 05:57:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D+hP+t/ta0Z5HMt5ieoY29iQv3kvHObDGUbZKxx2W5OJCEIR2++bTWqsrNdAUU/y2X+yXcUOekejpAjPyO365G9YYhyUnoTjmQJaaQtcCfQkm7CugwK/aC/MHjVwcmK76OXuvpHqc1o2ntHPWicjAB1/ElZr34clERVl3k0/yYE2pOYgxJP/t3875bymnA/g1KX0ZYbBmqVW5oqnyc/xRZQrx97j5GUTmSmR9nttx5kT1c2LeKeHw1bQGDZdPPwhpUPsPqXAf2NdksHz2kOZTFrRHb7AM2MI/OxetR6DXQUHKweSzvylarhRPhEyJZbDbP5qc/z2qsYS6Khf28C7Ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kMw/WFtoqiI/n/zOwO4wSzAXSx4Y9p7q9wdCPvVW994=; b=fn1lmBM05MkiQCPlHnOB96M3FNNzYfhPcrJApe2aqkiIu2RF19AUixur1Rp5yJHALTWylQCfVjK/QTLWZ/2Z0FhvsxmjFN2YdiBHgbkHB5xkIzWel0mohVpsLMTNIPHCQOfV1ZNSb9YJoZYAfUXABez58rNQb838jtZ023XaVVaeEjI0XV9du6aPzbNIHnyL3/sQU3bqgYGoBsQBDTGA4iY4DjjjNnfM/wBRZkj0rim//V/QalLYs+iErzzTkc60tpMqBqGupcafq786WuN+yLbI9xLgkmm1V8CzCugGwZu3Oi1qvfDhOgReoPJwN+A9jN+p+1Rdz8zIVQt1TBj42A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kMw/WFtoqiI/n/zOwO4wSzAXSx4Y9p7q9wdCPvVW994=; b=divfJC/hSgpLQTC8N/M+l3sQ5/FhbRqZ4E81PkDSiawPQCQK/Yd+oXJ08aAc0cKjI804NaloIJIqMRRuFV7mCwFm1UtpglqgwkjVYu1BY8cFId5//qBamp40DH0RLxUCvibisTIfpRp2NwiA1F67fMSRnTGAueqQA0fO2qZako4= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by MW4PR10MB6533.namprd10.prod.outlook.com (2603:10b6:303:21b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.32; Tue, 13 Feb 2024 05:57:29 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::91b3:fd53:a6ee:8685%4]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024 05:57:29 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 30/30] osnoise: handle quiescent states for PREEMPT_RCU=n, PREEMPTION=y Date: Mon, 12 Feb 2024 21:55:54 -0800 Message-Id: <20240213055554.1802415-31-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com> References: <20240213055554.1802415-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0111.namprd03.prod.outlook.com (2603:10b6:303:b7::26) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|MW4PR10MB6533:EE_ X-MS-Office365-Filtering-Correlation-Id: 73debee4-726d-4864-19d4-08dc2c58a976 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GRrwfrLXNwvhUR6rAtpfea6YAz/SZjcimxseOTj3JMy+jItWecz97e8IHaYK7fDvaM5oCKh5u+tK1VWJ9jGKIZ8s4xGfANvZ4UdfMXtOcndsL2nx2UelCMdb8Rp5hb0FNR+DMOsVQi1Yl/DzbhfjyjSMTKxj+ROOJ/eRiWTg/3I46tYy01ZWHi7dP6agKoky3Lpw/lOrOizHHx3cIV5jfcjw1M4eP+x3OuSrmmVeeKM9TWPgZDsWmCGCQq3uWHcY4+JAVwF3a6bgJO2xp3cwmSQqUN+CcUt3R83UTt3DQj1yK6WQqVPJYifZ/DqtGBomZktxDKC0yGyESkqaGpgdS2Da/gte2/PLXFJyuNf+5Au/dm6ofziOys+ufgoUJjhiqKkTfiqSX9wJ0cqhu81TvYPsItoEWocjtA/hNHvvmCClAki0M7rbn4OeGmDItcC9d7+O2yC7T+7LWZzDS0KPd/rq5MW/cMMAvXJErCUJ1lZ0ADRcyeR89iNzYou/VtET3sCGL/wupiKyd8WuLj8X+BxAY/D71W0fOeFN0XDVrprXsEpjjYNti/YXPMOLUO5i X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(39860400002)(346002)(376002)(396003)(366004)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(7416002)(2906002)(66476007)(7406005)(8936002)(66556008)(4326008)(8676002)(5660300002)(66946007)(107886003)(86362001)(103116003)(36756003)(38100700002)(6486002)(478600001)(6666004)(6506007)(316002)(83380400001)(6512007)(41300700001)(2616005)(1076003)(26005)(6916009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ER5dELVe2W0RwnrUBRDi3fCx2MsL99JHRxHQASaibRPU3ufNJhnD6nYCMvhvSu0UeNtmflov82EyFOB164VGppiv5Vw4rfknPj6neDaxAnQaleX4kqTt/IlGE9DwvpP9Wp9r/kj3QRfAjDHNPSK1Pl4DBHqRRNzz6amT/rwE8ziRO19VZ+vEvXvdexP73bknNc7S4sFTfwkUUP4NK2WBYIUKLg2cViELYNIjddhrG1CQH0mGTNp4maUB+w+Ip6IZXzxxnMNUcbEoAZz0Y9MKfg2cX8dccq9Yrj0FCq2lOE04UJXH6Ahu2BGoOTvk5J9cZNmbh4ffvhMC/Q+IkB5cUtolKykHqzv+YYDaRJt0aicx+bEDNef42PTPbomklvGOMxcBYbj8bg/fQgcSlUzdHKE56zXZfx8TzS6uB0I7+UazOvv7emd6H5dW87MClRhOvdDGkOBW7HLGgJtU40v1dylBCYAkpxBY6qOHvT0YcRKZF/h4H/lPTNNss/xWRqSADVEUPREsGjU7j1xs0NZ44mOuEpgUjNYYzCZ7rt9pmJPX9MDql1h1aQmSNBFi0xcdj9WIn/mzEfwkEl8t09aJTu/AFgW4L0UozrvtvlKJBVgVeOBGZmKKyMVUQlWZszQigPDcPfLGUVsJL1KyZZm30Hk/lWuOWpsfjm85GRIg3vD1zQjBvRjtpNlCmwD5T5cPsQcXj/KRRH7qqKJpKLXUZq02s+/wIYahpGAcQJ+5o5ee7L/nccyRcMzp/beqj+uUDFecc5wlqrp5Ij+AQvidPOB5Vg+c7tmxwTE91pLIu6Vd/SI3ar4wvXTHYJyOhWPkMLRrhFNL+K7WG7fjZQnQ6/+v2LNwUSRtjsTJ/zSPeH/pxqjpuy5QcK+qu9BQ+vgeI2uqlU4Z2ZqcGt2iTBx1HBgZVbSBe9PAV++vJahMbYXQHC1qhv1G6obXHN90UDfmiogm4AL3DdFRLyyuF33bsQHYvWathxZpmaeqUFxfS3BkD8FxeCHI87nxD9wahIuzuQBXy06RjwiPeGqC8mFLfp/kAQZYc52USKJ6tAJKkFV6GXzxqdulWATF7RX2jIm0lrqohbxa5d6HNBUCl7HVCDevYW4DMR4XiQ1PlxXLJbQJBm7DcwqnfqhuS2rI1i94iE5KQP22CLubIeJpBEMHSattlMXLA5xJ619lCngAsnT+0Wn8yn4ATqYJLfi7vxFJvmUtx/oLKwc/dVW3dBris6BMZrb2AaL3HNcLOrLEwyUyJFOx0oOjTDiFTGFY4nPsp0frqKFixhEiCGVJFwtrjU3mGf9a3Swm+MHISz3eybRDveUpjBlqxr1TRcyOTgW/7kj8rFeZODaB8tKZjtamtW57MXm3HvVstZAqyqrDyrapW0xCX1L5fMi1/zxIeHUmykLKhjOKrDEtJPihW6sHUbIOn3U4GgMeHbpTYk6JZOe9a/HqTJ6WdCvKrQTxb2DspaBGpD0czTsmybFW6QMcVPxYzoAbSGiN52PiwLmKcDwHObWwepYNv3REcx1T2C3L8xSROMQAT0muplYHKk0YAHIeDWSepNxPGzPIfkXe5Mmi3l+U3KYa7ZTtQ6hGvnvTmdZfh9SWVOFVJIB4w60wFg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 0xg/Tbx+3QMbu2i/eGVnvJJE+Hm/4eFqRwLtF8mlt5bqa+EK+gWcX2ay7ivgr5gijcpgrcY7ErZGhP8qSgcItJpt8AJX5+8n8OgliB1TgwT67iQzKCQmJxKeFnHj0aKeaL5l5UPYZwB7+VY/v13zleLf1menhnV9di3J4KAvXeFoFmkSh/HW0RPVWOc+YZ5BSQDI78DaSlsHSW8hBEcgjVqZUbZ3azvWsCSqyQd5goloxWpdwLJU2pJSxxTQaecZSr8Owo4uh5Gkip3YQFUlI78CQMpnk+jz8x8L/nHqTtUpMS0gRw1SKu0z+V64/5QbWQyyrt2smptxNumEAl+YtHlO7GcEP5PyDzsZEVEiL9N8stU/IL8si4oPtKcZEMiSsbWh/QdxFL/OR30sMtupqHlz2wq8N/6LgJ68thUIdHIsEx7AMNTmVQCIDOAmfX6zjNFuZFDRHfCvnHFFQFXdIXbpf9jjxfAewJLUW8M+9hbJbseB/f6psVfs/U2DwTHWFsgMbBzzSWGJ7cxbFz15gbP4jgGlVM9Aasqk2pIW9zUjiDGBUfyiHYcJOB35BAlOPcbjOzAxkwwFmPiUYF+bZz8pVKqMuj4772rAkNLipCU= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 73debee4-726d-4864-19d4-08dc2c58a976 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:57:29.3080 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dqWpVJsp4fU5DcecyGSK0w/gw5zhTw4qq+JMQwYHZmH6VugxRBBpwBJkUBMpn7+KiKZJw23+lhoI5B+vbbePELDcrl6E0cc0KnhsIdo1+zI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6533 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=942 adultscore=0 suspectscore=0 mlxscore=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402130043 X-Proofpoint-GUID: NfzWETDx5tKhze4yfwCRpx3cjxSWr0Sr X-Proofpoint-ORIG-GUID: NfzWETDx5tKhze4yfwCRpx3cjxSWr0Sr X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790762633349509948 X-GMAIL-MSGID: 1790762633349509948 To reduce RCU noise for nohz_full configurations, osnoise depends on cond_resched() providing quiescent states for PREEMPT_RCU=n configurations. And, for PREEMPT_RCU=y configurations does this by directly calling rcu_momentary_dyntick_idle(). With PREEMPT_AUTO=y, however, we can have configurations with (PREEMPTION=y, PREEMPT_RCU=n), which means neither of the above can help. Handle that by fallback to the explicit quiescent states via rcu_momentary_dyntick_idle(). Cc: Paul E. McKenney Cc: Daniel Bristot de Oliveira Cc: Steven Rostedt Suggested-by: Paul E. McKenney Signed-off-by: Ankur Arora --- kernel/trace/trace_osnoise.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index bd0d01d00fb9..8f9f654594ed 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -1532,18 +1532,20 @@ static int run_osnoise(void) /* * In some cases, notably when running on a nohz_full CPU with * a stopped tick PREEMPT_RCU has no way to account for QSs. - * This will eventually cause unwarranted noise as PREEMPT_RCU - * will force preemption as the means of ending the current - * grace period. We avoid this problem by calling - * rcu_momentary_dyntick_idle(), which performs a zero duration - * EQS allowing PREEMPT_RCU to end the current grace period. - * This call shouldn't be wrapped inside an RCU critical - * section. + * This will eventually cause unwarranted noise as RCU forces + * preemption as the means of ending the current grace period. + * We avoid this by calling rcu_momentary_dyntick_idle(), + * which performs a zero duration EQS allowing RCU to end the + * current grace period. This call shouldn't be wrapped inside + * an RCU critical section. * - * Note that in non PREEMPT_RCU kernels QSs are handled through - * cond_resched() + * For non-PREEMPT_RCU kernels with cond_resched() (non- + * PREEMPT_AUTO configurations), QSs are handled through + * cond_resched(). For PREEMPT_AUTO kernels, we fallback to the + * zero duration QS via rcu_momentary_dyntick_idle(). */ - if (IS_ENABLED(CONFIG_PREEMPT_RCU)) { + if (IS_ENABLED(CONFIG_PREEMPT_RCU) || + (!IS_ENABLED(CONFIG_PREEMPT_RCU) && IS_ENABLED(CONFIG_PREEMPTION))) { if (!disable_irq) local_irq_disable();