From patchwork Thu Feb 22 09:22:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "zhaoyang.huang" X-Patchwork-Id: 204634 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:aa16:b0:108:e6aa:91d0 with SMTP id by22csp133072dyb; Thu, 22 Feb 2024 01:23:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVjtrY7JZd12HfCEkyUsS8uqEgsUMdpLaOOuBrMRYSoM09cwqILfgu4NajvHlGqbTNTtw6QXndsnjsJFv2TTLbjRGiJ2A== X-Google-Smtp-Source: AGHT+IEmMMiTkGY37AF493pwhstsYAWTqf23TQQj0FG56APMzc6/FUFDgfuTzeTI/5XYHasKaoNs X-Received: by 2002:a05:6402:1656:b0:564:7007:e14a with SMTP id s22-20020a056402165600b005647007e14amr7605073edx.6.1708593825818; Thu, 22 Feb 2024 01:23:45 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708593825; cv=pass; d=google.com; s=arc-20160816; b=mkZXbNot/IaGF7QH1mlTUqgNwWz30SaBCY1Bm5KLrRKBP+n5bpmsqbOLJe4k3NJypu 8BiybUKvtHW5/LqqaY5YAtq122P14o3AZ7PoSPXZE0DLM1+4ActarppXzrWpFsZpk0IG QEcDAXWjSOVq3lmPqWhdAXaqvE8aMtmbnMTZaWldD0X9hGA2asuhuJSdy44+xZz9/3BZ O0uy0AkMuvSQ/ykbCDyQdKeY66N6rXD9XEtepfvsizpzvFC15G+GOtBIEM81RWBPmRFC 41YDOyosowFZd8uWP3dfGWUxib7RPQF61sTrM4AP3ZVznUoO2UlprQVdYQcq29XdIR86 9qaQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:to:from; bh=mBVGE3oq6L4FTwSx0yXLnJzxWxQWt32Z6YBwwNSN8Lo=; fh=mzA9NCJw0RTHkzOWmkgDF6mTAW7l5j//L5ineSKIwXU=; b=YnA/tbYEM4XeDeTeYRwcSJjb4G6rLl5NHUFGOMIg8kBbc6dEqqukthuEj56sA0GxCc 4jaOOS3jc3BNB/qVs+FeKT9n5cS2OOejB9hjo7hmBQYd7rA8BhYNr8hzfa5TP7UcMmb6 c9xrJ4WvzIju+QequDqqnkRihEvxALN28SBfGBdhC9e11JUEr6qmM2JwZXnzOsi5z0dV fLF2b8A3J/rguZ6FrQ8UfViIHtuIl/sdoaaVg3mgY9RFaYq84XdLBTErmyWf4XRwz+DX hdsE4jWQIJQ7qPpaXM0CUVLG6b+9JNVciAChVweX5s4bPmsUe6edbKAcVIo+fGyJppXR kRfw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=unisoc.com); spf=pass (google.com: domain of linux-kernel+bounces-76211-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-76211-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id t21-20020aa7d715000000b005640a717b42si5033679edq.170.2024.02.22.01.23.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Feb 2024 01:23:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-76211-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=unisoc.com); spf=pass (google.com: domain of linux-kernel+bounces-76211-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-76211-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 6CE511F24A7C for ; Thu, 22 Feb 2024 09:23:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DF93F381A0; Thu, 22 Feb 2024 09:23:01 +0000 (UTC) Received: from SHSQR01.spreadtrum.com (unknown [222.66.158.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BD64376FA; Thu, 22 Feb 2024 09:22:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=222.66.158.135 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708593780; cv=none; b=UpXqGiQ+p7rQMLvGwJFT4aalbBqj2vhsmSFKCfUrwFIXZfJMEOQ5VEVFfTTZvJkZ3qmYLoSLThsX++e699Dc5jPd1gD/aj9ApIMIy2oL5GjKsgZf8yFzOcfyPu2qBwIE4EtZsXgj/WbEiXy/6AVGPTAvCWbLPvcRJy0FHhXz6ms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708593780; c=relaxed/simple; bh=Y9MCsGargnZMdYbvCa0RXPzvIXitJU5Yea7omJUm5oc=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; b=Oz9IU/6AG92nfJdjC59l/b9VIZ2eZKKmCe+0OunesMfdjVCAhbnH7c2QJXg/PIad61OZLsliJj+gjeZ1x+KAfWArWpPb+XQwV0PiQWCaU65i74UY7wdr8Z2S9hGUxNplezvN8yhG01980GrlaaAb6M6R///vrd0L/Ub3iy9aEgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com; spf=pass smtp.mailfrom=unisoc.com; arc=none smtp.client-ip=222.66.158.135 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=unisoc.com Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 41M9MOXf042231; Thu, 22 Feb 2024 17:22:24 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4TgSMH4w5yz2KL7sp; Thu, 22 Feb 2024 17:21:47 +0800 (CST) Received: from bj03382pcu01.spreadtrum.com (10.0.73.40) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Thu, 22 Feb 2024 17:22:22 +0800 From: "zhaoyang.huang" To: Vincent Guittot , Jens Axboe , , , Zhaoyang Huang , Subject: [PATCHv2 1/2] sched: introduce helper function to calculate distribution over sched class Date: Thu, 22 Feb 2024 17:22:19 +0800 Message-ID: <20240222092220.642294-1-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL: SHSQR01.spreadtrum.com 41M9MOXf042231 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791590479743945531 X-GMAIL-MSGID: 1791590479743945531 From: Zhaoyang Huang As RT, DL, IRQ time could be deemed as lost time of CFS's task, some timing value want to know the distribution of how these spread approximately by using utilization account value (nivcsw is not enough sometimes). This commit would like to introduce a helper function to achieve this goal. eg. Effective part of A = Total_time * cpu_util_cfs / cpu_util Timing value A (should be a process last for several TICKs or statistics of a repeadted process) Timing start | | preempted by RT, DL or IRQ |\ | This period time is nonvoluntary CPU give up, need to know how long |/ sched in again | | | Timing end Signed-off-by: Zhaoyang Huang --- change of v2: using two parameter to pass se_prop and rq_prop out --- --- include/linux/sched.h | 3 +++ kernel/sched/core.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 77f01ac385f7..d6d5914fad10 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2318,6 +2318,9 @@ static inline bool owner_on_cpu(struct task_struct *owner) /* Returns effective CPU energy utilization, as seen by the scheduler */ unsigned long sched_cpu_util(int cpu); +/* Returns task's and cfs_rq's proportion among whole core */ +unsigned long cfs_prop_by_util(struct task_struct *tsk, unsigned long *se_prop, + unsigned long *rq_prop); #endif /* CONFIG_SMP */ #ifdef CONFIG_RSEQ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 802551e0009b..b8c29dff5d37 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7494,6 +7494,41 @@ unsigned long sched_cpu_util(int cpu) { return effective_cpu_util(cpu, cpu_util_cfs(cpu), ENERGY_UTIL, NULL); } + +/* + * Calculate the approximate proportion of timing value consumed by the specified + * tsk and all cfs tasks of this core. + * The user must be aware of this is done by avg_util which is tracked by + * the geometric series of decaying the load by y^32 = 0.5 (unit is 1ms). + * That is, only the period last for at least several TICKs or the statistics + * of repeated timing value are suitable for this helper function. + * This function is actually derived from effective_cpu_util but without + * limiting the util to the core's capacity. + * se_prop and rq_prop is valid only when return value is 1 + */ +unsigned long cfs_prop_by_util(struct task_struct *tsk, unsigned long *se_prop, + unsigned long *rq_prop) +{ + unsigned int cpu = task_cpu(tsk); + struct sched_entity *se = &tsk->se; + struct rq *rq = cpu_rq(cpu); + unsigned long util, irq, max; + + if (tsk->sched_class != &fair_sched_class) + return 0; + + max = arch_scale_cpu_capacity(cpu); + irq = cpu_util_irq(rq); + + util = cpu_util_rt(rq) + cpu_util_cfs(cpu) + cpu_util_dl(rq); + util = scale_irq_capacity(util, irq, max); + util += irq; + + *se_prop = se->avg.util_avg * 100 / util; + *rq_prop = cpu_util_cfs(cpu) * 100 / util; + return 1; +} + #endif /* CONFIG_SMP */ /** From patchwork Thu Feb 22 09:22:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "zhaoyang.huang" X-Patchwork-Id: 204633 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:aa16:b0:108:e6aa:91d0 with SMTP id by22csp133024dyb; Thu, 22 Feb 2024 01:23:40 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXKdG8BQudEVx5hT50qgNBqqNwupRA+SZGaMkMl8ZoqZDRdyTHAL5gI8w2HFV9Wk3e81wO5RhjGsnNx91uPNnMuuGcY7A== X-Google-Smtp-Source: AGHT+IF1xWK3es9CmOjGox47xkH9C1v1ZQu1dRX/gZXqqpUQblccFJ2mxww6zLDb7gcqo9SWxrjm X-Received: by 2002:aa7:c484:0:b0:565:789:3058 with SMTP id m4-20020aa7c484000000b0056507893058mr2830212edq.23.1708593820573; Thu, 22 Feb 2024 01:23:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708593820; cv=pass; d=google.com; s=arc-20160816; b=k4cIWI/9S8uSUUaCFrAE8lW/kt/S0RKOINX+rGCg1/Ddt0XUAiWVEg9ETNaycX0ty5 L2/Pw+C++DubGCdM5Ug5DL9989HWjLQ0i0hAgT/6Eht4nqrlOTYUhwtiGxeLNFwvF0Td OBtuEA9LPV2s00Josx/eYbqkHKfYCVHfGTL0WxEvouT800HGilKNzFI3KWn1ZKMJYkj9 cf9LVFS352egqzDgFJZJFnG95NIXyvcoKNcEG5zPNDw/k/rpZChEWGXJ27vqTIQ97p/M jlMY67K4Fp2e9WtUW4VXBcJEDO8JBziZDlAhcFIsAo8kdEYiHubipjrWG2Qt61t4AUsM Pu8w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:to:from; bh=KZllBMXuVtNU4QcMF7eAMltIsn3pZ1PsiImqZIhnD1w=; fh=mzA9NCJw0RTHkzOWmkgDF6mTAW7l5j//L5ineSKIwXU=; b=1Iu2jcz1XyJbUBx/WE9SOZnG/eER7juP+5/bEPBV162ZYIoOJY03GlJkBUlygT1u2J Kf84sDt3wnBXayf2kTrIrugvZfgyXKTquZQ00TZ7fkwD4VJDVdzURvYOwLsAYOj/YViC zFQ201W5MIUXH07C0InPmWxzepaDXO+j7Aif62r3l4sKeUqxN7oCs6oxrrRz5Jh0zuHx g92pWNd87lH/c2ii2pg874YJgmr2ui7Ein1MNcyr+uGzATPEdl113WWxEBSNmQBsXg6o c4ZGEvkQQgPak+z9Qw/oz5vl+xNoQdngPEUwHXeCpYPVgdyldmL/ZMANUjV0fe9IhIUH gtyg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=unisoc.com); spf=pass (google.com: domain of linux-kernel+bounces-76210-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-76210-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id c11-20020a05640227cb00b00564d52c8ce3si2107918ede.508.2024.02.22.01.23.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Feb 2024 01:23:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-76210-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=unisoc.com); spf=pass (google.com: domain of linux-kernel+bounces-76210-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-76210-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3110A1F218BE for ; Thu, 22 Feb 2024 09:23:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7F9B839854; Thu, 22 Feb 2024 09:23:01 +0000 (UTC) Received: from SHSQR01.spreadtrum.com (mx1.unisoc.com [222.66.158.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39A99376FC; Thu, 22 Feb 2024 09:22:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=222.66.158.135 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708593779; cv=none; b=Uj7kPLpMP+XEgMTKb5xjaWX5RbrPPnbWT2L/nyKhT5N6d3mHWJi2ZafDe6a4YoMg2LisKNG99BDWkDvbH2uevkMFClVFeFgE/CpKHvSZbbzuKOVineYGS02Qg592evK9lt0zxvlbFvpVQCELg3E+nPO3r19oO5dSFVmvhEOd1K8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708593779; c=relaxed/simple; bh=ujbmUgxGOpenZbV+oP1/IFMPiRVz/EeLf/b+iP4JGxI=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LuTSYzSVuVFn8MWNtAwpn9Oh5kptv5znEN8TYG3MZuLiO9d217l8qDLcrpW3vYljmzFnMpKH6k7KqNwIQynSCvSxGNstUMq+cdGIGZvld22LHdq00SdIO80H261g465/ZhXcf/NRKEl1qP5wgvpgVedegrVqUD0hDNcZ9yKviv4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com; spf=pass smtp.mailfrom=unisoc.com; arc=none smtp.client-ip=222.66.158.135 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=unisoc.com Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 41M9MQYU042352; Thu, 22 Feb 2024 17:22:26 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4TgSMK4dw3z2KL7sp; Thu, 22 Feb 2024 17:21:49 +0800 (CST) Received: from bj03382pcu01.spreadtrum.com (10.0.73.40) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Thu, 22 Feb 2024 17:22:24 +0800 From: "zhaoyang.huang" To: Vincent Guittot , Jens Axboe , , , Zhaoyang Huang , Subject: [PATCHv2 2/2] block: adjust CFS request expire time Date: Thu, 22 Feb 2024 17:22:20 +0800 Message-ID: <20240222092220.642294-2-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240222092220.642294-1-zhaoyang.huang@unisoc.com> References: <20240222092220.642294-1-zhaoyang.huang@unisoc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL: SHSQR01.spreadtrum.com 41M9MQYU042352 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791590473869078976 X-GMAIL-MSGID: 1791590473869078976 From: Zhaoyang Huang According to current policy, CFS's may suffer involuntary IO-latency by being preempted by RT/DL tasks or IRQ since they possess the privilege for both of CPU and IO scheduler. This commit introduce an approximate and light method to decrease these affection by adjusting the expire time via the CFS's proportion among the whole cpu active time. The average utilization of cpu's run queue could reflect the historical active proportion of different types of task that can be proved valid for this goal from belowing three perspective, 1. All types of sched class's load(util) are tracked and calculated in the same way(using a geometric series which known as PELT) 2. Keep the legacy policy by NOT adjusting rq's position in fifo_list but only make changes over expire_time. 3. The fixed expire time(hundreds of ms) is in the same range of cpu avg_load's account series(the utilization will be decayed to 0.5 in 32ms) TaskA sched in | | | submit_bio | | | fifo_time = jiffies + expire (insert_request) TaskB sched in | | vfs_xxx | |preempted by RT,DL,IRQ |\ | This period time is unfair to TaskB's IO request, should be adjust |/ | submit_bio | | | fifo_time = jiffies + expire * CFS_PROPORTION(rq) (insert_request) Signed-off-by: Zhaoyang Huang --- change of v2: introduce direction and threshold to make the hack working as a guard for CFS's over-preempted. --- --- block/mq-deadline.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index f958e79277b8..b477ba1bf6d2 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -54,6 +54,7 @@ enum dd_prio { enum { DD_PRIO_COUNT = 3 }; +#define CFS_PROP_THRESHOLD 60 /* * I/O statistics per I/O priority. It is fine if these counters overflow. * What matters is that these counters are at least as wide as @@ -802,6 +803,7 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, u8 ioprio_class = IOPRIO_PRIO_CLASS(ioprio); struct dd_per_prio *per_prio; enum dd_prio prio; + int fifo_expire; lockdep_assert_held(&dd->lock); @@ -828,6 +830,7 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, rq->fifo_time = jiffies; } else { struct list_head *insert_before; + unsigned long se_prop, rq_prop; deadline_add_rq_rb(per_prio, rq); @@ -839,8 +842,21 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, /* * set expire time and add to fifo list + * The expire time is adjusted when current CFS task is + * over-preempted by RT/DL/IRQ which is calculated by the + * proportion of cfs_rq's activation among whole cpu time during + * last several dozen's ms.Whearas, this would NOT affect the + * rq's position in fifo_list but only take effect when this + * rq is checked for its expire time when at head. */ - rq->fifo_time = jiffies + dd->fifo_expire[data_dir]; + fifo_expire = dd->fifo_expire[data_dir]; + if (data_dir == DD_READ && + cfs_prop_by_util(current, &se_prop, &rq_prop) && + rq_prop < CFS_PROP_THRESHOLD) + fifo_expire = dd->fifo_expire[data_dir] * rq_prop / 100; + + rq->fifo_time = jiffies + fifo_expire; + insert_before = &per_prio->fifo_list[data_dir]; #ifdef CONFIG_BLK_DEV_ZONED /*