From patchwork Wed Feb 7 07:24:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 197786 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp2061629dyb; Tue, 6 Feb 2024 23:33:30 -0800 (PST) X-Google-Smtp-Source: AGHT+IG9vKMDyHR7FayPEQ5AqfgaAgnbF628Qs5VDm+XItnGc5XXEBFMANO1SjD3F9hQ+XbhbTJs X-Received: by 2002:a62:ce82:0:b0:6dd:a072:867 with SMTP id y124-20020a62ce82000000b006dda0720867mr1798290pfg.15.1707291210155; Tue, 06 Feb 2024 23:33:30 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707291210; cv=pass; d=google.com; s=arc-20160816; b=Lce+HsFC5+0XYk3l8UXwpbw1rs8hZbz8u9qD4DO8deAlQ6zEkF87v1k7iiA8cdUDLd /sjpu+EhCzof97FZCKPRVBo4GET9Dsygbyof4y/Et1W2MRrYOn2ylR2Epg/rzzSXjozi LyfJZUP3qLIKLfEKcq+J7cxrFA2IZUoW2GOLkrMZzKLFt0SdSzVn+3PW2/s2GLhHqhKB tKFOvle4kyNZaZ+sp+zrUYwXk5pphf/E1GVF/XG7pJM4PHwLagS9AIz+PQYIjQK7yxB1 w7hXA9Avp6OUQzqU8aZzITQorrTtEKL5vSezBC1+U6eexdL84lODVYUzJ3GI9BhEVffy uOqQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=QNN760rSp2aLl/aqcBN1X/32QXU4X/5iWMdhFZ7Jr6Y=; fh=eI1+GQbDW7jYL1h2XL8YYDYat6Yb0cYpF9/tf/4/PS8=; b=AfMnIItcuh0hoOAOkyQPzkRaMg6Klomt6CvokpOxgMxs49HPL+tqexiAvxVuKs7/ex vUPsrEqgt3RZrGtVIZhzT5rIkqnw1zPd1X0bvEPk8LCXIMF3R4zM6O37CeGKTiEbqB3P ZwTb4LdewhBCgrk8DH40ivAIz0gN/J9gGEklGUi51cQxFe5eeBPrQdOXp5CWeKLFb3aR W74faF9L1DkxvCG4MZl9AyTB0wPch0bJOp8mw0XL8C5OzRR9N68ODG336M76FFmV9d9h wy15jrO6nBATnU91HAIiUiGj6B6IheNFLFCKirZpHnqEOxutHP3DPB43Y5Kc8gY/kbik FH7w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=g0dpA67q; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-56076-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-56076-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=2; AJvYcCVOuM4RCZPBUq6piCD+SmJai0/MsYOhXUJRO1pkttMCPV7/6JccZEA2dk4QsnvUM0ekOrV4aHoMT1bYaYP+5V2hEQ+rZg== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id o26-20020a63921a000000b005d8ad5aa264si837903pgd.798.2024.02.06.23.33.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 23:33:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-56076-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=g0dpA67q; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-56076-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-56076-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 1C4E2B24B2F for ; Wed, 7 Feb 2024 07:29:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D637820DE5; Wed, 7 Feb 2024 07:28:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g0dpA67q" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F56C2031E for ; Wed, 7 Feb 2024 07:28:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707290908; cv=none; b=qHlgnq+1fnnDWrBF3TvS5w3LWSbAimeKPjLT1NndaXW6jm12hvYqgwURa/ST/M+Np1hY6A1agMz0piaAoND952UKK2ia70QhtHuI+GgEu7zqSexqmf0rIbscZk5j+ZvtBjN3tWdSuP4RWOHVx0U+SNIDAfgTpdZFXJ827DZInlE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707290908; c=relaxed/simple; bh=bWaLjtidBr05TM3D/n6lluQMqun8vTEheO+oRY9qZFc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZB34D5fI4IlhBkFuyVnp/+CJb1fHRIsffPoNENGG0L7YGiyrQqRdT7ykQ18P4bapiRYZ47jybWUOdmKPpz9cTdIEmtYEd73MOWieRqo+0WAp/ACHE/hishgnXNuO7XUm6AFWS+jFRMWs+U/SVyQYL9b3gOqddzjEHvpqM2zi4iI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g0dpA67q; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C9A13C43394; Wed, 7 Feb 2024 07:28:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707290907; bh=bWaLjtidBr05TM3D/n6lluQMqun8vTEheO+oRY9qZFc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g0dpA67qPnQmmuH5cjhc5BndXI09AW43QOWyyduSQG5elWe3RG0Wmq2ycUfQh+qmE 8QlJsjkZSrNDFyszol3FV/sSxkwhOdQ1+zTbBYy/TLep+hgGpecQlYPorbKhcbez5M RCbbnpTNpj9pmMHtS1oU91nFgpGSKv0eNqyUnufn/MOGbCDJnhOphOy6jOva/bkOJ+ jFzzP5ZHmhQqklhc+E0r28jFbopXqyK7CmJYfnGTkSlEvl2l8YO42TXJ77ovR/s4au swyvN7C4avpqkiCBHd4dvAqPo8bnOBmj/+SctTwEbNtIhnl0VWK3QWXwGwZB8pAMp7 Wu7AcsINtBzTg== From: Saeed Mahameed To: Arnd Bergmann , Greg Kroah-Hartman Cc: Leon Romanovsky , Jason Gunthorpe , Jiri Pirko , Leonid Bloch , Itay Avraham , Jakub Kicinski , Saeed Mahameed , David Ahern , Aron Silverton , Christoph Hellwig , andrew.gospodarek@broadcom.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 4/5] misc: mlx5ctl: Add command rpc ioctl Date: Tue, 6 Feb 2024 23:24:34 -0800 Message-ID: <20240207072435.14182-5-saeed@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240207072435.14182-1-saeed@kernel.org> References: <20240207072435.14182-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790224587633248940 X-GMAIL-MSGID: 1790224587633248940 From: Saeed Mahameed Add new IOCTL to allow user space to send device debug rpcs and attach the user's uctx UID to each rpc. In the mlx5 architecture the FW RPC commands are of the format of inbox and outbox buffers. The inbox buffer contains the command rpc layout as described in the ConnectX Programmers Reference Manual (PRM) document and as defined in include/linux/mlx5/mlx5_ifc.h. On success the user outbox buffer will be filled with the device's rpc response. For example to query device capabilities: a user fills out an inbox buffer with the inbox layout: struct mlx5_ifc_query_hca_cap_in_bits and expects an outbox buffer with the layout: struct mlx5_ifc_cmd_hca_cap_bits Reviewed-by: Jiri Pirko Reviewed-by: Leon Romanovsky Reviewed-by: Jason Gunthorpe Signed-off-by: Saeed Mahameed --- drivers/misc/mlx5ctl/main.c | 98 +++++++++++++++++++++++++++++++++++++ include/uapi/misc/mlx5ctl.h | 12 +++++ 2 files changed, 110 insertions(+) diff --git a/drivers/misc/mlx5ctl/main.c b/drivers/misc/mlx5ctl/main.c index e4e70359dbe8..c02b80efffc1 100644 --- a/drivers/misc/mlx5ctl/main.c +++ b/drivers/misc/mlx5ctl/main.c @@ -245,6 +245,94 @@ static int mlx5ctl_info_ioctl(struct file *file, return err; } +struct mlx5_ifc_mbox_in_hdr_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_mbox_out_hdr_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + + u8 syndrome[0x20]; + + u8 reserved_at_40[0x40]; +}; + +static int mlx5ctl_cmdrpc_ioctl(struct file *file, + struct mlx5ctl_cmdrpc __user *arg, + size_t usize) +{ + size_t ksize = sizeof(struct mlx5ctl_cmdrpc); + struct mlx5ctl_fd *mfd = file->private_data; + struct mlx5ctl_dev *mcdev = mfd->mcdev; + struct mlx5ctl_cmdrpc *rpc = NULL; + void *in = NULL, *out = NULL; + int err; + + if (usize < ksize) + return -EINVAL; + + rpc = kzalloc(ksize, GFP_KERNEL); + if (!rpc) + return -ENOMEM; + + err = copy_from_user(rpc, arg, usize); + if (err) + goto out; + + mlx5ctl_dbg(mcdev, "[UID %d] cmdrpc: inlen %d outlen %d\n", + mfd->uctx_uid, rpc->inlen, rpc->outlen); + + if (rpc->inlen < MLX5_ST_SZ_BYTES(mbox_in_hdr) || + rpc->outlen < MLX5_ST_SZ_BYTES(mbox_out_hdr) || + rpc->inlen > MLX5CTL_MAX_RPC_SIZE || + rpc->outlen > MLX5CTL_MAX_RPC_SIZE) { + err = -EINVAL; + goto out; + } + + in = memdup_user(u64_to_user_ptr(rpc->in), rpc->inlen); + if (IS_ERR(in)) { + err = PTR_ERR(in); + goto out; + } + + out = kvzalloc(rpc->outlen, GFP_KERNEL); + if (!out) { + err = -ENOMEM; + goto out; + } + + mlx5ctl_dbg(mcdev, "[UID %d] cmdif: opcode 0x%x inlen %d outlen %d\n", + mfd->uctx_uid, + MLX5_GET(mbox_in_hdr, in, opcode), rpc->inlen, rpc->outlen); + + MLX5_SET(mbox_in_hdr, in, uid, mfd->uctx_uid); + err = mlx5_cmd_do(mcdev->mdev, in, rpc->inlen, out, rpc->outlen); + mlx5ctl_dbg(mcdev, "[UID %d] cmdif: opcode 0x%x retval %d\n", + mfd->uctx_uid, + MLX5_GET(mbox_in_hdr, in, opcode), err); + + /* -EREMOTEIO means outbox is valid, but out.status is not */ + if (!err || err == -EREMOTEIO) { + err = 0; + if (copy_to_user(u64_to_user_ptr(rpc->out), out, rpc->outlen)) + err = -EFAULT; + } + +out: + kvfree(out); + kfree(in); + kfree(rpc); + return err; +} + static long mlx5ctl_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { struct mlx5ctl_fd *mfd = file->private_data; @@ -270,6 +358,10 @@ static long mlx5ctl_ioctl(struct file *file, unsigned int cmd, unsigned long arg err = mlx5ctl_info_ioctl(file, argp, size); break; + case MLX5CTL_IOCTL_CMDRPC: + err = mlx5ctl_cmdrpc_ioctl(file, argp, size); + break; + default: mlx5ctl_dbg(mcdev, "Unknown ioctl %x\n", cmd); err = -ENOTTY; @@ -328,6 +420,11 @@ static int mlx5ctl_probe(struct auxiliary_device *adev, goto abort; } + err = sysfs_create_link_nowarn(&mcdev->miscdev.this_device->kobj, + &mdev->device->kobj, "mdev"); + if (err) + mlx5ctl_dbg(mcdev, "mlx5ctl: failed to create sysfs link err %d\n", err); + mlx5ctl_dbg(mcdev, "probe mdev@%s %s\n", dev_driver_string(mdev->device), dev_name(mdev->device)); @@ -348,6 +445,7 @@ static void mlx5ctl_remove(struct auxiliary_device *adev) struct mlx5_core_dev *mdev = mcdev->mdev; struct mlx5ctl_fd *mfd, *n; + sysfs_remove_link(&mcdev->miscdev.this_device->kobj, "mdev"); misc_deregister(&mcdev->miscdev); down_write(&mcdev->rw_lock); diff --git a/include/uapi/misc/mlx5ctl.h b/include/uapi/misc/mlx5ctl.h index 9be944128025..1e4622c5979f 100644 --- a/include/uapi/misc/mlx5ctl.h +++ b/include/uapi/misc/mlx5ctl.h @@ -12,9 +12,21 @@ struct mlx5ctl_info { __u32 ucap; /* process user capability */ }; +struct mlx5ctl_cmdrpc { + __aligned_u64 in; /* RPC inbox buffer user address */ + __aligned_u64 out; /* RPC outbox buffer user address */ + __u32 inlen; /* inbox buffer length */ + __u32 outlen; /* outbox buffer length */ +}; + +#define MLX5CTL_MAX_RPC_SIZE (512 * 512) /* max FW RPC buffer size 512 blocks of 512 bytes */ + #define MLX5CTL_IOCTL_MAGIC 0x5c #define MLX5CTL_IOCTL_INFO \ _IOR(MLX5CTL_IOCTL_MAGIC, 0x0, struct mlx5ctl_info) +#define MLX5CTL_IOCTL_CMDRPC \ + _IOWR(MLX5CTL_IOCTL_MAGIC, 0x1, struct mlx5ctl_cmdrpc) + #endif /* __MLX5CTL_IOCTL_H__ */