Message ID | 20221126162211.93322-3-mailhol.vincent@wanadoo.fr |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp4718877wrr; Sat, 26 Nov 2022 08:25:36 -0800 (PST) X-Google-Smtp-Source: AA0mqf6c3QIObth6fQSbu+TjE2CRDQeHIMzNsnSfbRJNMkwdiEcr4zMz7SzFNHbxYogki7rwFLc+ X-Received: by 2002:aa7:db4b:0:b0:46a:c6d3:a237 with SMTP id n11-20020aa7db4b000000b0046ac6d3a237mr7536483edt.132.1669479936831; Sat, 26 Nov 2022 08:25:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669479936; cv=none; d=google.com; s=arc-20160816; b=GGxLHBh9/ZfanI4wrY8iyzUjD0oFLbOnPsnhyryCReBmOc4wUEWptdtvX/Q+bE7JnZ DWPdAzxSfNXGnS0VVv21EGiFsU+DV3+/oGrM+emrVNaDA0NQ7lLXGlD+YoQ325fAjEID FxKVO4i/2XqT5CITgig4h1Tz2yByL5vLuTD5FjYCbFn94CeCuHZKqbO9T+dUtcwPxywO nNwKqoW5wzwgc2w6ROYel/LLUV9lRfvDc6aMKDZy0UdZrgwUOcEP9Q8kbWDfNXPj/dFt iiqnWTpOiQTXUxK6iA+AgNYGQEVjAfH19f6Nd252fJcNE2I50tFLr8p1yJ6l+4XfxduH ljrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=FmB9Jwaiin8tKBRiu/35E9rAp2jafIlqnhIKGIJ7dMk=; b=d3Lk/IvOxHFPeARAucw00CmnBPXdzRdtNNuXkqRtClGxQnk3QS0kt4nFyJ/UpndwXk mQsO6rfa5KlpWalh/V9kelZnS9gJ9vruv1uN1WHE8lWHZcED4KVHF5vB2yxLj4HvcTSk KXZeV6MC3IjWDk62Q556wHNhvmjH638481eFf4s3yqLeiCEyugZN5dVMDoM2DkGxBOSf SdYC0AESoBqKA5mmmUlxwHeilNplQAV+H1FGVsIBJNXLfkmzcwlSmmngC72v6ayCez96 vAsKjvgSxLTu8BcN1udebq8cFfP372Ats9cPZu6igfEgvgnzS6DXKmhFbyQE6mYQPhw1 qHbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=khULKVn7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v5-20020a50d585000000b00467c3cbab04si5940505edi.66.2022.11.26.08.25.13; Sat, 26 Nov 2022 08:25:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=khULKVn7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229661AbiKZQWs (ORCPT <rfc822;zxc52fgh@gmail.com> + 99 others); Sat, 26 Nov 2022 11:22:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229514AbiKZQWl (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Sat, 26 Nov 2022 11:22:41 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5017D133; Sat, 26 Nov 2022 08:22:39 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id z17so1849424pff.1; Sat, 26 Nov 2022 08:22:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=FmB9Jwaiin8tKBRiu/35E9rAp2jafIlqnhIKGIJ7dMk=; b=khULKVn7mhBorIgJ9/uSv1GtI9gWeAf7IdoCqRohNUR3WBjwfEG8IRQpjhUFMcy/pF bZUfl5luaKyGxgWulR/VhNJE06b6pbsTiAcTuWh2zfW7cBzfeQBnyllksrnct6RuRZbs 4YejzBFsRwKBoUC8ir9dG4ru0k8JluVmTAqU/ZbCPGwzdPGccBSn+wZNcxsDGhM7lQZG k+PbJtI0sAUGEBZsMCTP6WaSfg7Vbkl4umo6qg8T7mEz9QkwpvT1PWDyfkwycKKLOTh/ MjJqiibil70zUkopTYedSZ7NMce7bCMAFfPTkiA7JQ4guX5sFdEcB4GQ/kspK0PkWaZ5 JwvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=FmB9Jwaiin8tKBRiu/35E9rAp2jafIlqnhIKGIJ7dMk=; b=1EOziRpsfVc7Cr3G+akDuRslHTKyyYDmQYrYgzu2OY3c3PbZVuQ8ma4SUgiD1c5A3i lRU4gMFmmtAM4m5k52FF6Yn/wZxJHlrNTn9mixzLJ6yr8zN8d4bYIHQUs1dt1nKlO71b v8B7C65XknCtOQHPu1WMd45aCDY2Z/11vMiImElDd4JdLHmH4qibnarqZBgDczgSjQ/V s3+9IzgLDiL4bTMv5MCAUJEGhD8ft21z99VDZQzVjW931o3CBE3xrzk4QWa9RypbC9Xq AbPjY0SFuE06PRNQShLL5KrbBPFTwi+XzdhpNmFUzmTxAORVMULFSFcOrvY4aHEVE4bS okUA== X-Gm-Message-State: ANoB5pl6voL9R3Z7EW5stRWpdivMddQlOrmQCGlxs3Zaepq/zjf3adEI WGCrP5dSA/3w0i1eISZ9ayQvjDGOF+REdw== X-Received: by 2002:a63:1f63:0:b0:460:ec46:3645 with SMTP id q35-20020a631f63000000b00460ec463645mr40054183pgm.92.1669479758992; Sat, 26 Nov 2022 08:22:38 -0800 (PST) Received: from localhost.localdomain (124x33x176x97.ap124.ftth.ucom.ne.jp. [124.33.176.97]) by smtp.gmail.com with ESMTPSA id y14-20020a63e24e000000b00460ea630c1bsm4169601pgj.46.2022.11.26.08.22.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 26 Nov 2022 08:22:38 -0800 (PST) Sender: Vincent Mailhol <vincent.mailhol@gmail.com> From: Vincent Mailhol <mailhol.vincent@wanadoo.fr> To: linux-can@vger.kernel.org Cc: Marc Kleine-Budde <mkl@pengutronix.de>, linux-kernel@vger.kernel.org, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, netdev@vger.kernel.org, linux-usb@vger.kernel.org, Saeed Mahameed <saeed@kernel.org>, Andrew Lunn <andrew@lunn.ch>, Jiri Pirko <jiri@nvidia.com>, Lukas Magel <lukas.magel@posteo.net>, Vincent Mailhol <mailhol.vincent@wanadoo.fr> Subject: [PATCH v4 2/6] can: etas_es58x: add devlink support Date: Sun, 27 Nov 2022 01:22:07 +0900 Message-Id: <20221126162211.93322-3-mailhol.vincent@wanadoo.fr> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221126162211.93322-1-mailhol.vincent@wanadoo.fr> References: <20221104073659.414147-1-mailhol.vincent@wanadoo.fr> <20221126162211.93322-1-mailhol.vincent@wanadoo.fr> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750576593978685394?= X-GMAIL-MSGID: =?utf-8?q?1750576593978685394?= |
Series |
can: etas_es58x: report firmware, bootloader and hardware version
|
|
Commit Message
Vincent Mailhol
Nov. 26, 2022, 4:22 p.m. UTC
Add basic support for devlink. The callbacks of struct devlink_ops
will be implemented next.
Signed-off-by: Vincent Mailhol <mailhol.vincent@wanadoo.fr>
---
drivers/net/can/usb/Kconfig | 1 +
drivers/net/can/usb/etas_es58x/Makefile | 2 +-
drivers/net/can/usb/etas_es58x/es58x_core.c | 13 ++++++++++---
drivers/net/can/usb/etas_es58x/es58x_core.h | 6 ++++++
drivers/net/can/usb/etas_es58x/es58x_devlink.c | 13 +++++++++++++
5 files changed, 31 insertions(+), 4 deletions(-)
create mode 100644 drivers/net/can/usb/etas_es58x/es58x_devlink.c
Comments
> @@ -2196,11 +2198,12 @@ static struct es58x_device *es58x_init_es58x_dev(struct usb_interface *intf, > ops = &es581_4_ops; > } > > - es58x_dev = devm_kzalloc(dev, es58x_sizeof_es58x_device(param), > - GFP_KERNEL); > - if (!es58x_dev) > + devlink = devlink_alloc(&es58x_dl_ops, es58x_sizeof_es58x_device(param), > + dev); > + if (!devlink) > return ERR_PTR(-ENOMEM); > > + es58x_dev = devlink_priv(devlink); That is 'interesting'. Makes me wonder about lifetimes of different objects. Previously your es58x_dev structure would disappear when the driver is released, or an explicit call to devm_kfree(). Now it disappears when devlink_free() is called. Any danger of use after free here? USB devices always make me wonder about life times rules since they are probably the mode dynamic sort of device the kernel has the handle, them just abruptly disappearing. > es58x_dev->param = param; > es58x_dev->ops = ops; > es58x_dev->dev = dev; > @@ -2247,6 +2250,8 @@ static int es58x_probe(struct usb_interface *intf, > if (ret) > return ret; > > + devlink_register(priv_to_devlink(es58x_dev)); > + > for (ch_idx = 0; ch_idx < es58x_dev->num_can_ch; ch_idx++) { > ret = es58x_init_netdev(es58x_dev, ch_idx); > if (ret) { > @@ -2272,8 +2277,10 @@ static void es58x_disconnect(struct usb_interface *intf) > dev_info(&intf->dev, "Disconnecting %s %s\n", > es58x_dev->udev->manufacturer, es58x_dev->udev->product); > > + devlink_unregister(priv_to_devlink(es58x_dev)); > es58x_free_netdevs(es58x_dev); > es58x_free_urbs(es58x_dev); > + devlink_free(priv_to_devlink(es58x_dev)); > usb_set_intfdata(intf, NULL); Should devlink_free() be after usb_set_inftdata()? Andrew
On Tue. 27 Nov. 2022 at 01:51, Andrew Lunn <andrew@lunn.ch> wrote: > > @@ -2196,11 +2198,12 @@ static struct es58x_device *es58x_init_es58x_dev(struct usb_interface *intf, > > ops = &es581_4_ops; > > } > > > > - es58x_dev = devm_kzalloc(dev, es58x_sizeof_es58x_device(param), > > - GFP_KERNEL); > > - if (!es58x_dev) > > + devlink = devlink_alloc(&es58x_dl_ops, es58x_sizeof_es58x_device(param), > > + dev); > > + if (!devlink) > > return ERR_PTR(-ENOMEM); > > > > + es58x_dev = devlink_priv(devlink); > > That is 'interesting'. Another interesting thing I found is: https://elixir.bootlin.com/linux/v6.1-rc6/source/drivers/net/ethernet/intel/ice/ice_devlink.c#L866 Because devlink does not have an equivalent to devm_kzalloc(), that driver uses devm_add_action_or_reset() instead. But any other drivers will call devlink_free() in their disconnect function. So here, I just followed the trend. > Makes me wonder about lifetimes of different > objects. Previously your es58x_dev structure would disappear when the > driver is released, or an explicit call to devm_kfree(). Now it > disappears when devlink_free() is called. Even before that, this driver used to release es58x_dev in its disconnect() function. I changed it to use devm_kzalloc() last year after discovering its existence. https://git.kernel.org/torvalds/linux/c/6bde4c7fd845 >Any danger of use after free here? devlink_alloc() allocates one continuous block for both the devlink and the device priv (struct es58x_dev here): https://elixir.bootlin.com/linux/v6.1-rc6/source/net/core/devlink.c#L9629 So calling devlink_free() also releases struct es58x_dev. > USB devices always make me wonder about life times rules since they > are probably the mode dynamic sort of device the kernel has the > handle, them just abruptly disappearing. > > > es58x_dev->param = param; > > es58x_dev->ops = ops; > > es58x_dev->dev = dev; > > @@ -2247,6 +2250,8 @@ static int es58x_probe(struct usb_interface *intf, > > if (ret) > > return ret; > > > > + devlink_register(priv_to_devlink(es58x_dev)); > > + > > for (ch_idx = 0; ch_idx < es58x_dev->num_can_ch; ch_idx++) { > > ret = es58x_init_netdev(es58x_dev, ch_idx); > > if (ret) { > > @@ -2272,8 +2277,10 @@ static void es58x_disconnect(struct usb_interface *intf) > > dev_info(&intf->dev, "Disconnecting %s %s\n", > > es58x_dev->udev->manufacturer, es58x_dev->udev->product); > > > > + devlink_unregister(priv_to_devlink(es58x_dev)); > > es58x_free_netdevs(es58x_dev); > > es58x_free_urbs(es58x_dev); > > + devlink_free(priv_to_devlink(es58x_dev)); > > usb_set_intfdata(intf, NULL); > > Should devlink_free() be after usb_set_inftdata()? A look at $ git grep -W "usb_set_intfdata(.*NULL)" shows that the two patterns (freeing before or after usb_set_intfdata()) coexist. You are raising an important question here. usb_set_intfdata() does not have documentation that freeing before it is risky. And the documentation of usb_driver::disconnect says that: "@disconnect: Called when the interface is no longer accessible, usually because its device has been (or is being) disconnected or the driver module is being unloaded." Ref: https://elixir.bootlin.com/linux/v6.1-rc6/source/include/linux/usb.h#L1130 So the interface no longer being accessible makes me assume that the order does not matter. If it indeed matters, then this is a foot gun and there is some clean-up work waiting for us on many drivers. @Greg, any thoughts on whether or not the order of usb_set_intfdata() and resource freeing matters or not? Yours sincerely, Vincent Mailhol
On Sun, Nov 27, 2022 at 02:10:32PM +0900, Vincent MAILHOL wrote: > > Should devlink_free() be after usb_set_inftdata()? > > A look at > $ git grep -W "usb_set_intfdata(.*NULL)" > > shows that the two patterns (freeing before or after > usb_set_intfdata()) coexist. > > You are raising an important question here. usb_set_intfdata() does > not have documentation that freeing before it is risky. And the > documentation of usb_driver::disconnect says that: > "@disconnect: Called when the interface is no longer accessible, > usually because its device has been (or is being) disconnected > or the driver module is being unloaded." > Ref: https://elixir.bootlin.com/linux/v6.1-rc6/source/include/linux/usb.h#L1130 > > So the interface no longer being accessible makes me assume that the > order does not matter. If it indeed matters, then this is a foot gun > and there is some clean-up work waiting for us on many drivers. > > @Greg, any thoughts on whether or not the order of usb_set_intfdata() > and resource freeing matters or not? In fact, drivers don't have to call usb_set_intfdata(NULL) at all; the USB core does it for them after the ->disconnect() callback returns. But if a driver does make the call, it should be careful to ensure that the call happens _after_ the driver is finished using the interface-data pointer. For example, after all outstanding URBs have completed, if the completion handlers will need to call usb_get_intfdata(). Remember, the interface-data pointer is owned by the driver. Nothing else in the kernel uses it. So the driver merely has to be careful not to clear the pointer while it is still using it. Alan Stern
On Mon. 28 Nov. 2022 at 00:41, Alan Stern <stern@rowland.harvard.edu> wrote: > On Sun, Nov 27, 2022 at 02:10:32PM +0900, Vincent MAILHOL wrote: > > > Should devlink_free() be after usb_set_inftdata()? > > > > A look at > > $ git grep -W "usb_set_intfdata(.*NULL)" > > > > shows that the two patterns (freeing before or after > > usb_set_intfdata()) coexist. > > > > You are raising an important question here. usb_set_intfdata() does > > not have documentation that freeing before it is risky. And the > > documentation of usb_driver::disconnect says that: > > "@disconnect: Called when the interface is no longer accessible, > > usually because its device has been (or is being) disconnected > > or the driver module is being unloaded." > > Ref: https://elixir.bootlin.com/linux/v6.1-rc6/source/include/linux/usb.h#L1130 > > > > So the interface no longer being accessible makes me assume that the > > order does not matter. If it indeed matters, then this is a foot gun > > and there is some clean-up work waiting for us on many drivers. > > > > @Greg, any thoughts on whether or not the order of usb_set_intfdata() > > and resource freeing matters or not? > > In fact, drivers don't have to call usb_set_intfdata(NULL) at all; the > USB core does it for them after the ->disconnect() callback returns. Interesting. This fact is widely unknown, cf: $ git grep "usb_set_intfdata(.*NULL)" | wc -l 215 I will do some clean-up later on, at least for the CAN USB drivers. > But if a driver does make the call, it should be careful to ensure that > the call happens _after_ the driver is finished using the interface-data > pointer. For example, after all outstanding URBs have completed, if the > completion handlers will need to call usb_get_intfdata(). ACK. I understand that it should be called *after* the completion of any ongoing task. My question was more on: devlink_free(priv_to_devlink(es58x_dev)); usb_set_intfdata(intf, NULL); VS. usb_set_intfdata(intf, NULL); devlink_free(priv_to_devlink(es58x_dev)); From your comments, I understand that both are fine. > Remember, the interface-data pointer is owned by the driver. Nothing > else in the kernel uses it. So the driver merely has to be careful not > to clear the pointer while it is still using it. Thanks for your comments! Yours sincerely, Vincent Mailhol
On Mon. 28 Nov. 2022 at 10:34, Vincent MAILHOL <mailhol.vincent@wanadoo.fr> wrote: > On Mon. 28 Nov. 2022 at 00:41, Alan Stern <stern@rowland.harvard.edu> wrote: > > On Sun, Nov 27, 2022 at 02:10:32PM +0900, Vincent MAILHOL wrote: > > > > Should devlink_free() be after usb_set_inftdata()? > > > > > > A look at > > > $ git grep -W "usb_set_intfdata(.*NULL)" > > > > > > shows that the two patterns (freeing before or after > > > usb_set_intfdata()) coexist. > > > > > > You are raising an important question here. usb_set_intfdata() does > > > not have documentation that freeing before it is risky. And the > > > documentation of usb_driver::disconnect says that: > > > "@disconnect: Called when the interface is no longer accessible, > > > usually because its device has been (or is being) disconnected > > > or the driver module is being unloaded." > > > Ref: https://elixir.bootlin.com/linux/v6.1-rc6/source/include/linux/usb.h#L1130 > > > > > > So the interface no longer being accessible makes me assume that the > > > order does not matter. If it indeed matters, then this is a foot gun > > > and there is some clean-up work waiting for us on many drivers. > > > > > > @Greg, any thoughts on whether or not the order of usb_set_intfdata() > > > and resource freeing matters or not? > > > > In fact, drivers don't have to call usb_set_intfdata(NULL) at all; the > > USB core does it for them after the ->disconnect() callback returns. > > Interesting. This fact is widely unknown, cf: > $ git grep "usb_set_intfdata(.*NULL)" | wc -l > 215 > > I will do some clean-up later on, at least for the CAN USB drivers. > > > But if a driver does make the call, it should be careful to ensure that > > the call happens _after_ the driver is finished using the interface-data > > pointer. For example, after all outstanding URBs have completed, if the > > completion handlers will need to call usb_get_intfdata(). > > ACK. I understand that it should be called *after* the completion of > any ongoing task. > > My question was more on: > > devlink_free(priv_to_devlink(es58x_dev)); > usb_set_intfdata(intf, NULL); > > VS. > > usb_set_intfdata(intf, NULL); > devlink_free(priv_to_devlink(es58x_dev)); > > From your comments, I understand that both are fine. Do we agree that the usb-skeleton is doing it wrong? https://elixir.bootlin.com/linux/latest/source/drivers/usb/usb-skeleton.c#L567 usb_set_intfdata(interface, NULL) is called before deregistering the interface and terminating the outstanding URBs! > > Remember, the interface-data pointer is owned by the driver. Nothing > > else in the kernel uses it. So the driver merely has to be careful not > > to clear the pointer while it is still using it. > > Thanks for your comments! > > > Yours sincerely, > Vincent Mailhol
> > But if a driver does make the call, it should be careful to ensure that > > the call happens _after_ the driver is finished using the interface-data > > pointer. For example, after all outstanding URBs have completed, if the > > completion handlers will need to call usb_get_intfdata(). > > ACK. I understand that it should be called *after* the completion of > any ongoing task. What sometimes gets people is /sys, /proc. etc. A process can have such a file open when the device is unplugged. If the read needs to make use of your private data structure, you need to guarantee it still exists. Ideally the core needs to wait and not call the disconnect until all such files are closed. Probably the USB core does, it is such an obvious issue, but i have no knowledge of USB. Andrew
On Mon. 28 Nov. 2022 at 22:45, Andrew Lunn <andrew@lunn.ch> wrote: > > > But if a driver does make the call, it should be careful to ensure that > > > the call happens _after_ the driver is finished using the interface-data > > > pointer. For example, after all outstanding URBs have completed, if the > > > completion handlers will need to call usb_get_intfdata(). > > > > ACK. I understand that it should be called *after* the completion of > > any ongoing task. > > What sometimes gets people is /sys, /proc. etc. A process can have > such a file open when the device is unplugged. If the read needs to > make use of your private data structure, you need to guarantee it > still exists. Ideally the core needs to wait and not call the > disconnect until all such files are closed. Probably the USB core > does, it is such an obvious issue, but i have no knowledge of USB. For USB drivers, the parallel of what you are describing are the URBs (USB request Buffers). The URB are sent asynchronously to the device. Each URB has a completion handler: https://elixir.bootlin.com/linux/v6.0/source/include/linux/usb.h#L1443 It is important to wait for all outstanding URB to complete before releasing your resources. But once you are able to guarantee that any ongoing actions were completed, the order in which you kfree() or usb_set_intfdata() to NULL matters less. Of course, the USB drivers could also have some /sys/ or /proc/ files opened, but this is not the case of the etas_es58x. By the way, the handling of outstanding URBs is done by es58x_free_urbs(): https://elixir.bootlin.com/linux/v6.0/source/drivers/net/can/usb/etas_es58x/es58x_core.c#L1745 which is called just before the devlink_free() and the usb_set_intfdata().
On Mon, Nov 28, 2022 at 02:32:23PM +0900, Vincent MAILHOL wrote: > On Mon. 28 Nov. 2022 at 10:34, Vincent MAILHOL > <mailhol.vincent@wanadoo.fr> wrote: > > On Mon. 28 Nov. 2022 at 00:41, Alan Stern <stern@rowland.harvard.edu> wrote: > > > On Sun, Nov 27, 2022 at 02:10:32PM +0900, Vincent MAILHOL wrote: > > > > > Should devlink_free() be after usb_set_inftdata()? > > > > > > > > A look at > > > > $ git grep -W "usb_set_intfdata(.*NULL)" > > > > > > > > shows that the two patterns (freeing before or after > > > > usb_set_intfdata()) coexist. > > > > > > > > You are raising an important question here. usb_set_intfdata() does > > > > not have documentation that freeing before it is risky. And the > > > > documentation of usb_driver::disconnect says that: > > > > "@disconnect: Called when the interface is no longer accessible, > > > > usually because its device has been (or is being) disconnected > > > > or the driver module is being unloaded." > > > > Ref: https://elixir.bootlin.com/linux/v6.1-rc6/source/include/linux/usb.h#L1130 > > > > > > > > So the interface no longer being accessible makes me assume that the > > > > order does not matter. If it indeed matters, then this is a foot gun > > > > and there is some clean-up work waiting for us on many drivers. > > > > > > > > @Greg, any thoughts on whether or not the order of usb_set_intfdata() > > > > and resource freeing matters or not? > > > > > > In fact, drivers don't have to call usb_set_intfdata(NULL) at all; the > > > USB core does it for them after the ->disconnect() callback returns. > > > > Interesting. This fact is widely unknown, cf: > > $ git grep "usb_set_intfdata(.*NULL)" | wc -l > > 215 > > > > I will do some clean-up later on, at least for the CAN USB drivers. > > > > > But if a driver does make the call, it should be careful to ensure that > > > the call happens _after_ the driver is finished using the interface-data > > > pointer. For example, after all outstanding URBs have completed, if the > > > completion handlers will need to call usb_get_intfdata(). > > > > ACK. I understand that it should be called *after* the completion of > > any ongoing task. > > > > My question was more on: > > > > devlink_free(priv_to_devlink(es58x_dev)); > > usb_set_intfdata(intf, NULL); > > > > VS. > > > > usb_set_intfdata(intf, NULL); > > devlink_free(priv_to_devlink(es58x_dev)); > > > > From your comments, I understand that both are fine. > > Do we agree that the usb-skeleton is doing it wrong? > https://elixir.bootlin.com/linux/latest/source/drivers/usb/usb-skeleton.c#L567 > usb_set_intfdata(interface, NULL) is called before deregistering the > interface and terminating the outstanding URBs! Going through the usb-skeleton.c source code, you will find that usb_get_intfdata() is called from only a few routines: skel_open() skel_disconnect() skel_suspend() skel_pre_reset() skel_post_reset() Of those, all but the first are called only by the USB core and they are mutually exclusive with disconnect processing (except for skel_disconnect() itself, of course). So they don't matter. The first, skel_open(), can be called as a result of actions by the user, so the driver needs to ensure that this can't happen after it clears the interface-data pointer. The user can open the device file at any time before the minor number is given back, so it is not proper to call usb_set_intfdata(interface, NULL) before usb_deregister_dev() -- but the driver does exactly this! (Well, it's not quite that bad. skel_open() does check whether the interface-data pointer value it gets from usb_get_intfdata() is NULL. But it's still a race.) So yes, the current code is wrong. And in fact, it will still be wrong even after the usb_set_intfdata(interface, NULL) line is removed, because there is no synchronization between skel_open() and skel_disconnect(). It is possible for skel_disconnect() to run to completion and the USB core to clear the interface-data pointer all while skel_open() is running. The driver needs a static private mutex to synchronize opens with unregistrations. (This is a general phenomenon, true of all drivers that have a user interface such as a device file.) The driver _does_ have a per-instance mutex, dev->io_mutex, to synchronize I/O with disconnects. But that's separate from synchronizing opens with unregistrations, because at open time the driver doesn't yet know the address of the private data structure or even if the structure is still allocated. So obviously it can't use a mutex that is embedded within the private data structure for this purpose. Alan Stern
On Tue. 29 Nov. 2022 at 00:50, Alan Stern <stern@rowland.harvard.edu> wrote: > On Mon, Nov 28, 2022 at 02:32:23PM +0900, Vincent MAILHOL wrote: > > On Mon. 28 Nov. 2022 at 10:34, Vincent MAILHOL > > <mailhol.vincent@wanadoo.fr> wrote: > > > On Mon. 28 Nov. 2022 at 00:41, Alan Stern <stern@rowland.harvard.edu> wrote: > > > > On Sun, Nov 27, 2022 at 02:10:32PM +0900, Vincent MAILHOL wrote: > > > > > > Should devlink_free() be after usb_set_inftdata()? > > > > > > > > > > A look at > > > > > $ git grep -W "usb_set_intfdata(.*NULL)" > > > > > > > > > > shows that the two patterns (freeing before or after > > > > > usb_set_intfdata()) coexist. > > > > > > > > > > You are raising an important question here. usb_set_intfdata() does > > > > > not have documentation that freeing before it is risky. And the > > > > > documentation of usb_driver::disconnect says that: > > > > > "@disconnect: Called when the interface is no longer accessible, > > > > > usually because its device has been (or is being) disconnected > > > > > or the driver module is being unloaded." > > > > > Ref: https://elixir.bootlin.com/linux/v6.1-rc6/source/include/linux/usb.h#L1130 > > > > > > > > > > So the interface no longer being accessible makes me assume that the > > > > > order does not matter. If it indeed matters, then this is a foot gun > > > > > and there is some clean-up work waiting for us on many drivers. > > > > > > > > > > @Greg, any thoughts on whether or not the order of usb_set_intfdata() > > > > > and resource freeing matters or not? > > > > > > > > In fact, drivers don't have to call usb_set_intfdata(NULL) at all; the > > > > USB core does it for them after the ->disconnect() callback returns. > > > > > > Interesting. This fact is widely unknown, cf: > > > $ git grep "usb_set_intfdata(.*NULL)" | wc -l > > > 215 > > > > > > I will do some clean-up later on, at least for the CAN USB drivers. > > > > > > > But if a driver does make the call, it should be careful to ensure that > > > > the call happens _after_ the driver is finished using the interface-data > > > > pointer. For example, after all outstanding URBs have completed, if the > > > > completion handlers will need to call usb_get_intfdata(). > > > > > > ACK. I understand that it should be called *after* the completion of > > > any ongoing task. > > > > > > My question was more on: > > > > > > devlink_free(priv_to_devlink(es58x_dev)); > > > usb_set_intfdata(intf, NULL); > > > > > > VS. > > > > > > usb_set_intfdata(intf, NULL); > > > devlink_free(priv_to_devlink(es58x_dev)); > > > > > > From your comments, I understand that both are fine. > > > > Do we agree that the usb-skeleton is doing it wrong? > > https://elixir.bootlin.com/linux/latest/source/drivers/usb/usb-skeleton.c#L567 > > usb_set_intfdata(interface, NULL) is called before deregistering the > > interface and terminating the outstanding URBs! > > Going through the usb-skeleton.c source code, you will find that > usb_get_intfdata() is called from only a few routines: > > skel_open() > skel_disconnect() > skel_suspend() > skel_pre_reset() > skel_post_reset() > > Of those, all but the first are called only by the USB core and they are > mutually exclusive with disconnect processing (except for > skel_disconnect() itself, of course). So they don't matter. > > The first, skel_open(), can be called as a result of actions by the > user, so the driver needs to ensure that this can't happen after it > clears the interface-data pointer. The user can open the device file at > any time before the minor number is given back, so it is not proper to > call usb_set_intfdata(interface, NULL) before usb_deregister_dev() -- > but the driver does exactly this! > > (Well, it's not quite that bad. skel_open() does check whether the > interface-data pointer value it gets from usb_get_intfdata() is NULL. > But it's still a race.) > > So yes, the current code is wrong. And in fact, it will still be wrong > even after the usb_set_intfdata(interface, NULL) line is removed, > because there is no synchronization between skel_open() and > skel_disconnect(). ACK. I did not look outside of skel_disconnect(). Regardless, I think that removing the usb_set_intdata(interface, NULL) is still one step in the good direction despite the other synchronisation issues. I sent a patch for that which Greg already pick-up: https://git.kernel.org/gregkh/usb/c/c568f8bb41a4 >It is possible for skel_disconnect() to run to > completion and the USB core to clear the interface-data pointer all > while skel_open() is running. The driver needs a static private mutex > to synchronize opens with unregistrations. (This is a general > phenomenon, true of all drivers that have a user interface such as a > device file.) > > The driver _does_ have a per-instance mutex, dev->io_mutex, to > synchronize I/O with disconnects. But that's separate from > synchronizing opens with unregistrations, because at open time the > driver doesn't yet know the address of the private data structure or > even if the structure is still allocated. So obviously it can't use a > mutex that is embedded within the private data structure for this > purpose. ACK. However, I have other priorities, I will not invest more time to dig in the usb-skeleton.c Thank you for the answer! That was a long but interesting diversion from the initial topic :) Yours sincerely, Vincent Mailhol
diff --git a/drivers/net/can/usb/Kconfig b/drivers/net/can/usb/Kconfig index 8c6fea661530..445504ababce 100644 --- a/drivers/net/can/usb/Kconfig +++ b/drivers/net/can/usb/Kconfig @@ -30,6 +30,7 @@ config CAN_ESD_USB config CAN_ETAS_ES58X tristate "ETAS ES58X CAN/USB interfaces" select CRC16 + select NET_DEVLINK help This driver supports the ES581.4, ES582.1 and ES584.1 interfaces from ETAS GmbH (https://www.etas.com/en/products/es58x.php). diff --git a/drivers/net/can/usb/etas_es58x/Makefile b/drivers/net/can/usb/etas_es58x/Makefile index a129b4aa0215..d6667ebe259f 100644 --- a/drivers/net/can/usb/etas_es58x/Makefile +++ b/drivers/net/can/usb/etas_es58x/Makefile @@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_CAN_ETAS_ES58X) += etas_es58x.o -etas_es58x-y = es58x_core.o es581_4.o es58x_fd.o +etas_es58x-y = es58x_core.o es58x_devlink.o es581_4.o es58x_fd.o diff --git a/drivers/net/can/usb/etas_es58x/es58x_core.c b/drivers/net/can/usb/etas_es58x/es58x_core.c index 4c97102202bf..c6e598e4800c 100644 --- a/drivers/net/can/usb/etas_es58x/es58x_core.c +++ b/drivers/net/can/usb/etas_es58x/es58x_core.c @@ -16,6 +16,7 @@ #include <linux/kernel.h> #include <linux/module.h> #include <linux/usb.h> +#include <net/devlink.h> #include "es58x_core.h" @@ -2174,6 +2175,7 @@ static struct es58x_device *es58x_init_es58x_dev(struct usb_interface *intf, { struct device *dev = &intf->dev; struct es58x_device *es58x_dev; + struct devlink *devlink; const struct es58x_parameters *param; const struct es58x_operators *ops; struct usb_device *udev = interface_to_usbdev(intf); @@ -2196,11 +2198,12 @@ static struct es58x_device *es58x_init_es58x_dev(struct usb_interface *intf, ops = &es581_4_ops; } - es58x_dev = devm_kzalloc(dev, es58x_sizeof_es58x_device(param), - GFP_KERNEL); - if (!es58x_dev) + devlink = devlink_alloc(&es58x_dl_ops, es58x_sizeof_es58x_device(param), + dev); + if (!devlink) return ERR_PTR(-ENOMEM); + es58x_dev = devlink_priv(devlink); es58x_dev->param = param; es58x_dev->ops = ops; es58x_dev->dev = dev; @@ -2247,6 +2250,8 @@ static int es58x_probe(struct usb_interface *intf, if (ret) return ret; + devlink_register(priv_to_devlink(es58x_dev)); + for (ch_idx = 0; ch_idx < es58x_dev->num_can_ch; ch_idx++) { ret = es58x_init_netdev(es58x_dev, ch_idx); if (ret) { @@ -2272,8 +2277,10 @@ static void es58x_disconnect(struct usb_interface *intf) dev_info(&intf->dev, "Disconnecting %s %s\n", es58x_dev->udev->manufacturer, es58x_dev->udev->product); + devlink_unregister(priv_to_devlink(es58x_dev)); es58x_free_netdevs(es58x_dev); es58x_free_urbs(es58x_dev); + devlink_free(priv_to_devlink(es58x_dev)); usb_set_intfdata(intf, NULL); } diff --git a/drivers/net/can/usb/etas_es58x/es58x_core.h b/drivers/net/can/usb/etas_es58x/es58x_core.h index 4a082fd69e6f..bf24375580e5 100644 --- a/drivers/net/can/usb/etas_es58x/es58x_core.h +++ b/drivers/net/can/usb/etas_es58x/es58x_core.h @@ -674,6 +674,7 @@ static inline enum es58x_flag es58x_get_flags(const struct sk_buff *skb) return es58x_flags; } +/* es58x_core.c. */ int es58x_can_get_echo_skb(struct net_device *netdev, u32 packet_idx, u64 *tstamps, unsigned int pkts); int es58x_tx_ack_msg(struct net_device *netdev, u16 tx_free_entries, @@ -691,9 +692,14 @@ int es58x_rx_cmd_ret_u32(struct net_device *netdev, int es58x_send_msg(struct es58x_device *es58x_dev, u8 cmd_type, u8 cmd_id, const void *msg, u16 cmd_len, int channel_idx); +/* es58x_devlink.c. */ +extern const struct devlink_ops es58x_dl_ops; + +/* es581_4.c. */ extern const struct es58x_parameters es581_4_param; extern const struct es58x_operators es581_4_ops; +/* es58x_fd.c. */ extern const struct es58x_parameters es58x_fd_param; extern const struct es58x_operators es58x_fd_ops; diff --git a/drivers/net/can/usb/etas_es58x/es58x_devlink.c b/drivers/net/can/usb/etas_es58x/es58x_devlink.c new file mode 100644 index 000000000000..af6ca7ada23f --- /dev/null +++ b/drivers/net/can/usb/etas_es58x/es58x_devlink.c @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Driver for ETAS GmbH ES58X USB CAN(-FD) Bus Interfaces. + * + * File es58x_devlink.c: report the product information using devlink. + * + * Copyright (c) 2022 Vincent Mailhol <mailhol.vincent@wanadoo.fr> + */ + +#include <net/devlink.h> + +const struct devlink_ops es58x_dl_ops = { +};