Message ID | 20230316220638.983743-1-dlatypov@google.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp25539wrt; Thu, 16 Mar 2023 15:22:59 -0700 (PDT) X-Google-Smtp-Source: AK7set+xF23ntP08zAXqXgLoQUNFYj6YRIfPK23cU8V38qsdIfGrWxzaLxRwOJrT0PNXvXMLtxHW X-Received: by 2002:a05:6a20:8ba1:b0:d5:21ab:6b with SMTP id m33-20020a056a208ba100b000d521ab006bmr5893154pzh.2.1679005379229; Thu, 16 Mar 2023 15:22:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679005379; cv=none; d=google.com; s=arc-20160816; b=gdOc6xXfYMLRGt6Rh8l+erbdLynx3CQ36wixomQpGpaEmr5NNrIxGJspZfQ6GM/Ad9 5Wuzowj3Nd5kRBkz9iDCmSVjJeQyl+pSuX4A31AU8+SC9zhtd3If64PJ/QIAc7kDNP2J OrHwchkJGeWPZR+mJEc1xrxjXkBI3aawTw4HWna7+j8kY6UYVLMRBTwDRogDM9tfxAgy qKSSkGjPgsYzT3lKIkk4BmNmpPFjbNPWtolV2TkafmiEyaVIWEeegBfKjnoBXSSb9XG2 +/S1h9glCO5AaPicBffiBMYTqZQHXPkYecs/NTXPPsULcYVbX6dNJ6I+DrbTqMgDD1B7 JEtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=5w/bUTaQvv0dE6Z9kgV2X6M6GsXpoWgjbDaqKfzvjzU=; b=HVMaKN4BAAB7QEBdmhIdSRpzZx5Q4v2HywEnkmT6I8eogE90yAGJf8pIIQKwa3ssP9 cQSrXrx6j9hGoLvOeT6XTEpvVO8D0aq8tqO25DeyylcQa8JWZVgpZahPDZHELgWyyN0M 1/I8jGFE/O7JwfnaJOhKGq34miHKFQu529vo18fX/Zryuf+BoIHLgAxvNbbd0cz+ZoVo 29S7h3HIN0FjOqGZqOa+KBs+f5uani9y8h5coDiAj3vYjXGtEctct4rnXYCE0ckWdcf5 xxOSu+BSGwCB6WM+aUwz6s7AghheBtDBCR1M3dTZ66EZ/j355d21ijmVr75TLQZPBc0l d1HQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=rIZGB5so; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m2-20020a056a00164200b005a9d0456018si603603pfc.52.2023.03.16.15.22.44; Thu, 16 Mar 2023 15:22:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=rIZGB5so; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230106AbjCPWHK (ORCPT <rfc822;pwkd43@gmail.com> + 99 others); Thu, 16 Mar 2023 18:07:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229874AbjCPWHH (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 16 Mar 2023 18:07:07 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8734488D85 for <linux-kernel@vger.kernel.org>; Thu, 16 Mar 2023 15:06:47 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id d22-20020a63d716000000b00502e3fb8ff3so794929pgg.10 for <linux-kernel@vger.kernel.org>; Thu, 16 Mar 2023 15:06:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679004407; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=5w/bUTaQvv0dE6Z9kgV2X6M6GsXpoWgjbDaqKfzvjzU=; b=rIZGB5soI+YB2bwAFuUr9c+JWOpF0HrtgYe3XbqVePLIx5T7nAyu6sQkhvFb9ikBQo uFClwlCH26lx8vZzECJagMZ6U4LSE1qPlJW30AN2ao4CJ1fuFznQS06bR8Bm0RnSqNLs wYU7l/meIRfAoJGOe3R6K7zG9/xz+efX5pu0hxL4ei087YNIi/m8pAX03ul+Xt7Q+AjD 4DyGkhls6+O8D5kD7nRnFqprRdqELs0uJnLd6uV6/H5HUFtZ0tGmwFSeYhbzBPkqU9i/ mT/xyNImWDH4rmzcoZm942ZfNEU40kmejnt1+Xhh1wVM7wbpKI94Zm/l2SnrGI6R8LPj NsXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679004407; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=5w/bUTaQvv0dE6Z9kgV2X6M6GsXpoWgjbDaqKfzvjzU=; b=fiKMFlnlq/KYHoI6eG0owExfIgv+B0Gag+eyjPWub5L/lVoOQNx2TlOhOhyMzZpFuM sVeHfo8um9u5UYIraq++xWpTkFRtwXgxHTRKQznhIPZZIYBpfzzf7Dl+4KumI0JJyWdC iB502rKjlkCXZJj9y0VtGbxrqToOnBdhbYwpBlp9pY6D3EOmrQFZwq95vkSBlk3pKwB0 vUTOX8CVzdVHeU5/DXFGkkYAQ8XbLXNxEyiXCKuLhu0SbxHR+eZG2QRGEUGKqsy5V1fk 6GguC9EphE3UPkkhd8xO+OSpzrVNU3gz8R9UMq8LLUTVLvvWuk5xtsXjNtR65EoHm+xh YLTw== X-Gm-Message-State: AO0yUKWGLH0yqlp40+w39BjTwisdNlDNvLh+AhevMH3ga7Xz1KHJOFfa rkckEt83tClP+ZtcuZbz2rCeJK3SgzGDPQ== X-Received: from dlatypov-spec.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:3f35]) (user=dlatypov job=sendgmr) by 2002:a17:902:d2c7:b0:1a0:4664:319c with SMTP id n7-20020a170902d2c700b001a04664319cmr2318672plc.6.1679004406964; Thu, 16 Mar 2023 15:06:46 -0700 (PDT) Date: Thu, 16 Mar 2023 15:06:36 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog Message-ID: <20230316220638.983743-1-dlatypov@google.com> Subject: [PATCH v2 1/3] kunit: tool: add subscripts for type annotations where appropriate From: Daniel Latypov <dlatypov@google.com> To: brendanhiggins@google.com, davidgow@google.com Cc: rmoar@google.com, linux-kernel@vger.kernel.org, kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org, skhan@linuxfoundation.org, johannes@sipsolutions.net, Daniel Latypov <dlatypov@google.com>, Johannes Berg <johannes.berg@intel.com> Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760546903598388926?= X-GMAIL-MSGID: =?utf-8?q?1760564744437720940?= |
Series |
[v2,1/3] kunit: tool: add subscripts for type annotations where appropriate
|
|
Commit Message
Daniel Latypov
March 16, 2023, 10:06 p.m. UTC
E.g. for subprocess.Popen, it can be opened in `text=True` mode where it
returns strings, or `text=False` where it returns bytes.
To differentiate, you can annotate types as `Popen[str]` or
`Popen[bytes]`.
This patch should add subscripts in all the places we were missing them.
Reported-by: Johannes Berg <johannes.berg@intel.com>
Link: https://lore.kernel.org/linux-kselftest/20230315105055.9b2be0153625.I7a2cb99b95dff216c0feed4604255275e0b156a7@changeid/
Signed-off-by: Daniel Latypov <dlatypov@google.com>
---
Note: this is unchanged, just added a 3rd patch to this series.
---
tools/testing/kunit/kunit_kernel.py | 6 +++---
tools/testing/kunit/kunit_printer.py | 2 +-
tools/testing/kunit/run_checks.py | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
base-commit: 2c6a96dad5797e57b4cf04101d6c8d5c7a571603
Comments
On Fri, 17 Mar 2023 at 06:06, Daniel Latypov <dlatypov@google.com> wrote: > > E.g. for subprocess.Popen, it can be opened in `text=True` mode where it > returns strings, or `text=False` where it returns bytes. > To differentiate, you can annotate types as `Popen[str]` or > `Popen[bytes]`. > > This patch should add subscripts in all the places we were missing them. > > Reported-by: Johannes Berg <johannes.berg@intel.com> > Link: https://lore.kernel.org/linux-kselftest/20230315105055.9b2be0153625.I7a2cb99b95dff216c0feed4604255275e0b156a7@changeid/ > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- > Note: this is unchanged, just added a 3rd patch to this series. > --- Looks good. Reviewed-by: David Gow <davidgow@google.com> Thanks, Johannes and Daniel! -- David > tools/testing/kunit/kunit_kernel.py | 6 +++--- > tools/testing/kunit/kunit_printer.py | 2 +- > tools/testing/kunit/run_checks.py | 2 +- > 3 files changed, 5 insertions(+), 5 deletions(-) > > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py > index 53e90c335834..e6fc8fcb071a 100644 > --- a/tools/testing/kunit/kunit_kernel.py > +++ b/tools/testing/kunit/kunit_kernel.py > @@ -92,7 +92,7 @@ class LinuxSourceTreeOperations: > if stderr: # likely only due to build warnings > print(stderr.decode()) > > - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: > + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > raise RuntimeError('not implemented!') > > > @@ -112,7 +112,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): > kconfig.merge_in_entries(base_kunitconfig) > return kconfig > > - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: > + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > kernel_path = os.path.join(build_dir, self._kernel_path) > qemu_command = ['qemu-system-' + self._qemu_arch, > '-nodefaults', > @@ -141,7 +141,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): > kconfig.merge_in_entries(base_kunitconfig) > return kconfig > > - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: > + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > """Runs the Linux UML binary. Must be named 'linux'.""" > linux_bin = os.path.join(build_dir, 'linux') > params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt']) > diff --git a/tools/testing/kunit/kunit_printer.py b/tools/testing/kunit/kunit_printer.py > index 5f1cc55ecdf5..015adf87dc2c 100644 > --- a/tools/testing/kunit/kunit_printer.py > +++ b/tools/testing/kunit/kunit_printer.py > @@ -15,7 +15,7 @@ _RESET = '\033[0;0m' > class Printer: > """Wraps a file object, providing utilities for coloring output, etc.""" > > - def __init__(self, output: typing.IO): > + def __init__(self, output: typing.IO[str]): > self._output = output > self._use_color = output.isatty() > > diff --git a/tools/testing/kunit/run_checks.py b/tools/testing/kunit/run_checks.py > index 066e6f938f6d..61cece1684df 100755 > --- a/tools/testing/kunit/run_checks.py > +++ b/tools/testing/kunit/run_checks.py > @@ -37,7 +37,7 @@ def main(argv: Sequence[str]) -> None: > if argv: > raise RuntimeError('This script takes no arguments') > > - future_to_name: Dict[futures.Future, str] = {} > + future_to_name: Dict[futures.Future[None], str] = {} > executor = futures.ThreadPoolExecutor(max_workers=len(commands)) > for name, argv in commands.items(): > if name in necessary_deps and shutil.which(necessary_deps[name]) is None: > > base-commit: 2c6a96dad5797e57b4cf04101d6c8d5c7a571603 > -- > 2.40.0.rc1.284.g88254d51c5-goog >
Hi Daniel, On Thu, 16 Mar 2023 15:06:36 -0700 Daniel Latypov <dlatypov@google.com> wrote: > E.g. for subprocess.Popen, it can be opened in `text=True` mode where it > returns strings, or `text=False` where it returns bytes. > To differentiate, you can annotate types as `Popen[str]` or > `Popen[bytes]`. > > This patch should add subscripts in all the places we were missing them. I just found this patch is in the latest mainline tree, and it causes kunit failure on my test machine like below. $ python3 --version Python 3.8.10 $ $ ./tools/testing/kunit/kunit.py run --build_dir ../kunit.out/ Traceback (most recent call last): File "./tools/testing/kunit/kunit.py", line 24, in <module> import kunit_kernel File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 42, in <module> class LinuxSourceTreeOperations: File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 95, in LinuxSourceTreeOperations def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: TypeError: 'type' object is not subscriptable $ I further confirmed reverting this patch makes it run again. Do you have any idea? Thanks, SJ > > Reported-by: Johannes Berg <johannes.berg@intel.com> > Link: https://lore.kernel.org/linux-kselftest/20230315105055.9b2be0153625.I7a2cb99b95dff216c0feed4604255275e0b156a7@changeid/ > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- > Note: this is unchanged, just added a 3rd patch to this series. > --- > tools/testing/kunit/kunit_kernel.py | 6 +++--- > tools/testing/kunit/kunit_printer.py | 2 +- > tools/testing/kunit/run_checks.py | 2 +- > 3 files changed, 5 insertions(+), 5 deletions(-) > > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py > index 53e90c335834..e6fc8fcb071a 100644 > --- a/tools/testing/kunit/kunit_kernel.py > +++ b/tools/testing/kunit/kunit_kernel.py > @@ -92,7 +92,7 @@ class LinuxSourceTreeOperations: > if stderr: # likely only due to build warnings > print(stderr.decode()) > > - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: > + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > raise RuntimeError('not implemented!') > > > @@ -112,7 +112,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): > kconfig.merge_in_entries(base_kunitconfig) > return kconfig > > - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: > + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > kernel_path = os.path.join(build_dir, self._kernel_path) > qemu_command = ['qemu-system-' + self._qemu_arch, > '-nodefaults', > @@ -141,7 +141,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): > kconfig.merge_in_entries(base_kunitconfig) > return kconfig > > - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: > + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > """Runs the Linux UML binary. Must be named 'linux'.""" > linux_bin = os.path.join(build_dir, 'linux') > params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt']) > diff --git a/tools/testing/kunit/kunit_printer.py b/tools/testing/kunit/kunit_printer.py > index 5f1cc55ecdf5..015adf87dc2c 100644 > --- a/tools/testing/kunit/kunit_printer.py > +++ b/tools/testing/kunit/kunit_printer.py > @@ -15,7 +15,7 @@ _RESET = '\033[0;0m' > class Printer: > """Wraps a file object, providing utilities for coloring output, etc.""" > > - def __init__(self, output: typing.IO): > + def __init__(self, output: typing.IO[str]): > self._output = output > self._use_color = output.isatty() > > diff --git a/tools/testing/kunit/run_checks.py b/tools/testing/kunit/run_checks.py > index 066e6f938f6d..61cece1684df 100755 > --- a/tools/testing/kunit/run_checks.py > +++ b/tools/testing/kunit/run_checks.py > @@ -37,7 +37,7 @@ def main(argv: Sequence[str]) -> None: > if argv: > raise RuntimeError('This script takes no arguments') > > - future_to_name: Dict[futures.Future, str] = {} > + future_to_name: Dict[futures.Future[None], str] = {} > executor = futures.ThreadPoolExecutor(max_workers=len(commands)) > for name, argv in commands.items(): > if name in necessary_deps and shutil.which(necessary_deps[name]) is None: > > base-commit: 2c6a96dad5797e57b4cf04101d6c8d5c7a571603 > -- > 2.40.0.rc1.284.g88254d51c5-goog >
On Sun, Apr 30, 2023 at 11:15 AM SeongJae Park <sj@kernel.org> wrote: > > Hi Daniel, > > On Thu, 16 Mar 2023 15:06:36 -0700 Daniel Latypov <dlatypov@google.com> wrote: > > > E.g. for subprocess.Popen, it can be opened in `text=True` mode where it > > returns strings, or `text=False` where it returns bytes. > > To differentiate, you can annotate types as `Popen[str]` or > > `Popen[bytes]`. > > > > This patch should add subscripts in all the places we were missing them. > > I just found this patch is in the latest mainline tree, and it causes kunit > failure on my test machine like below. > > $ python3 --version > Python 3.8.10 > $ > $ ./tools/testing/kunit/kunit.py run --build_dir ../kunit.out/ > Traceback (most recent call last): > File "./tools/testing/kunit/kunit.py", line 24, in <module> > import kunit_kernel > File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 42, in <module> > class LinuxSourceTreeOperations: > File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 95, in LinuxSourceTreeOperations > def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > TypeError: 'type' object is not subscriptable > $ > > I further confirmed reverting this patch makes it run again. Do you have any > idea? It seems like support for the subscript wasn't added until Python 3.9. I know support for subscripting other types like re.Pattern was added in 3.9 per https://peps.python.org/pep-0585/ but it doesn't mention Popen there... This patch also added typing.IO[str] and concurrent.Future[None], so those might be problematic too. Can you check if the typing.IO and concurrent.Future[None] changes cause problems? (I don't have an easy way of testing against older Python versions currently). If so, we should revert the patch. If not, we can undo just the Popen changes. And in either case, we'll need to update ./tools/testing/kunit/run_checks.py. Currently, it runs `mypy --strict` which will start failing if we revert any part of this patch. Thanks, Daniel
Hi Daniel, On Sun, 30 Apr 2023 14:34:09 -0700 Daniel Latypov <dlatypov@google.com> wrote: > On Sun, Apr 30, 2023 at 11:15 AM SeongJae Park <sj@kernel.org> wrote: > > > > Hi Daniel, > > > > On Thu, 16 Mar 2023 15:06:36 -0700 Daniel Latypov <dlatypov@google.com> wrote: > > > > > E.g. for subprocess.Popen, it can be opened in `text=True` mode where it > > > returns strings, or `text=False` where it returns bytes. > > > To differentiate, you can annotate types as `Popen[str]` or > > > `Popen[bytes]`. > > > > > > This patch should add subscripts in all the places we were missing them. > > > > I just found this patch is in the latest mainline tree, and it causes kunit > > failure on my test machine like below. > > > > $ python3 --version > > Python 3.8.10 > > $ > > $ ./tools/testing/kunit/kunit.py run --build_dir ../kunit.out/ > > Traceback (most recent call last): > > File "./tools/testing/kunit/kunit.py", line 24, in <module> > > import kunit_kernel > > File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 42, in <module> > > class LinuxSourceTreeOperations: > > File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 95, in LinuxSourceTreeOperations > > def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: > > TypeError: 'type' object is not subscriptable > > $ > > > > I further confirmed reverting this patch makes it run again. Do you have any > > idea? > > It seems like support for the subscript wasn't added until Python 3.9. > > I know support for subscripting other types like re.Pattern was added > in 3.9 per https://peps.python.org/pep-0585/ but it doesn't mention > Popen there... > This patch also added typing.IO[str] and concurrent.Future[None], so > those might be problematic too. > > Can you check if the typing.IO and concurrent.Future[None] changes > cause problems? > (I don't have an easy way of testing against older Python versions currently). Thank you for quick reply. Reverting Popen changes only as below fixed my issue. So seems typing.IO and concurrent.Future[None] chages doesn't cause problems at least for my use case. diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index f01f94106129..7f648802caf6 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -92,7 +92,7 @@ class LinuxSourceTreeOperations: if stderr: # likely only due to build warnings print(stderr.decode()) - def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: + def start(self, params: List[str], build_dir: str) -> subprocess.Popen: raise RuntimeError('not implemented!') @@ -113,7 +113,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): kconfig.merge_in_entries(base_kunitconfig) return kconfig - def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: + def start(self, params: List[str], build_dir: str) -> subprocess.Popen: kernel_path = os.path.join(build_dir, self._kernel_path) qemu_command = ['qemu-system-' + self._qemu_arch, '-nodefaults', @@ -142,7 +142,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): kconfig.merge_in_entries(base_kunitconfig) return kconfig - def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: + def start(self, params: List[str], build_dir: str) -> subprocess.Popen: """Runs the Linux UML binary. Must be named 'linux'.""" linux_bin = os.path.join(build_dir, 'linux') params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt']) > > If so, we should revert the patch. > If not, we can undo just the Popen changes. > > And in either case, we'll need to update ./tools/testing/kunit/run_checks.py. > Currently, it runs `mypy --strict` which will start failing if we > revert any part of this patch. Those make sense. Thanks, SJ > > Thanks, > Daniel
On Mon, May 1, 2023 at 10:15 AM SeongJae Park <sj@kernel.org> wrote: > [snip] > > > > It seems like support for the subscript wasn't added until Python 3.9. > > > > I know support for subscripting other types like re.Pattern was added > > in 3.9 per https://peps.python.org/pep-0585/ but it doesn't mention > > Popen there... > > This patch also added typing.IO[str] and concurrent.Future[None], so > > those might be problematic too. > > > > Can you check if the typing.IO and concurrent.Future[None] changes > > cause problems? > > (I don't have an easy way of testing against older Python versions currently). > > Thank you for quick reply. Reverting Popen changes only as below fixed my > issue. So seems typing.IO and concurrent.Future[None] chages doesn't cause > problems at least for my use case. Sounds good. Sent https://lore.kernel.org/linux-kselftest/20230501181610.2617032-1-dlatypov@google.com I was hoping adding `--python-version 3.7` to the `mypy --strict` invocation would help, but it still fails for me :\ $ ./tools/testing/kunit/run_cheks.py ... mypy: FAILED > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg] > kunit_kernel.py:116: error: Missing type parameters for generic type "Popen" [type-arg] > kunit_kernel.py:145: error: Missing type parameters for generic type "Popen" [type-arg] > Found 3 errors in 1 file (checked 8 source files) And here I was, hoping it would complain about code incompatible with python 3.7... But at the very least, that patch should fix the current problem. Daniel
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 53e90c335834..e6fc8fcb071a 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -92,7 +92,7 @@ class LinuxSourceTreeOperations: if stderr: # likely only due to build warnings print(stderr.decode()) - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: raise RuntimeError('not implemented!') @@ -112,7 +112,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): kconfig.merge_in_entries(base_kunitconfig) return kconfig - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: kernel_path = os.path.join(build_dir, self._kernel_path) qemu_command = ['qemu-system-' + self._qemu_arch, '-nodefaults', @@ -141,7 +141,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): kconfig.merge_in_entries(base_kunitconfig) return kconfig - def start(self, params: List[str], build_dir: str) -> subprocess.Popen: + def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]: """Runs the Linux UML binary. Must be named 'linux'.""" linux_bin = os.path.join(build_dir, 'linux') params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt']) diff --git a/tools/testing/kunit/kunit_printer.py b/tools/testing/kunit/kunit_printer.py index 5f1cc55ecdf5..015adf87dc2c 100644 --- a/tools/testing/kunit/kunit_printer.py +++ b/tools/testing/kunit/kunit_printer.py @@ -15,7 +15,7 @@ _RESET = '\033[0;0m' class Printer: """Wraps a file object, providing utilities for coloring output, etc.""" - def __init__(self, output: typing.IO): + def __init__(self, output: typing.IO[str]): self._output = output self._use_color = output.isatty() diff --git a/tools/testing/kunit/run_checks.py b/tools/testing/kunit/run_checks.py index 066e6f938f6d..61cece1684df 100755 --- a/tools/testing/kunit/run_checks.py +++ b/tools/testing/kunit/run_checks.py @@ -37,7 +37,7 @@ def main(argv: Sequence[str]) -> None: if argv: raise RuntimeError('This script takes no arguments') - future_to_name: Dict[futures.Future, str] = {} + future_to_name: Dict[futures.Future[None], str] = {} executor = futures.ThreadPoolExecutor(max_workers=len(commands)) for name, argv in commands.items(): if name in necessary_deps and shutil.which(necessary_deps[name]) is None: