From patchwork Fri Feb 24 10:53:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asahi Lina X-Patchwork-Id: 61257 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp836617wrd; Fri, 24 Feb 2023 03:01:13 -0800 (PST) X-Google-Smtp-Source: AK7set8qWkIiSvSPdVyjS4LxTCSPo/7onhATR6VodyZSx0Ig+G5PUlwbeYS2CTFLUwZERPbaHqz+ X-Received: by 2002:a17:906:eb18:b0:8b1:7de3:cfaa with SMTP id mb24-20020a170906eb1800b008b17de3cfaamr26240604ejb.3.1677236473446; Fri, 24 Feb 2023 03:01:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677236473; cv=none; d=google.com; s=arc-20160816; b=QIDMoPZKQe8AaXWW414Obl6miiHPJnfF0ZEbv/mZbSHqh2+nfJskWkWTiXWrvMirkm PozZsckBUWiDGX2hGdN2BAZtPag8DY1xy7W4YrbllOWX6/HIS7NAgFC+jiWWZuqK/0gA r1FXFX62saE17zqjSrd1DKuLEdkuNmaVZPRedN0OOOxO/iVwux4SN9I+fSVGGopWu+NE TPEtnPOEmKzC4J/1qK8H7Cvl09uhmit52JPa4ndT0PzZrZvww6WiLurr1Vaq5T9nj4yi wp6AKYpoyPxALohGktGLuT+iPKTPr4xqnt8G+9fNLo6gQA3puyl9/tP5njQnWa6UKoV8 a+Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:in-reply-to:references:message-id :content-transfer-encoding:mime-version:subject:date:from :dkim-signature; bh=CGDgMHbDokWuN0dMhHPf86b9XNJRRAjiaM4QUdz4Wvg=; b=raFzs5YJq5CjN3Kv2yFRPrvbBZf/9ho4pIXXLnpLpIoMqccRA74xtA10ncwqpRl1Nf NHJcRDGffcsYinCino5t0scRCPeunDP47A/Uj8A/26OUp4KgGIY6rhKScYuR4vGYs+yQ pwYVOm4kAJOACo2HUwPH85UtbD9NY5ZpqE4H+7PP95NztAxFqBrMzcJY/jcGjTkNudJv dE++6sUf2sTIIiWT8ZGbrHckb7p6rk25W2OAYp4Vb+2whkfrS8BMyQTYe9kBrcIKOJis /jlihZcxjKTImpNZJf92xM8L4X8i4WNxn5xe/OKrpYgPnsT01gfBfvBjdXnr8Yfs7Zy6 ZFfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@asahilina.net header.s=default header.b=A6Ru0K8e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=asahilina.net Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fl5-20020a1709072a8500b008ce419809a0si14491042ejc.217.2023.02.24.03.00.47; Fri, 24 Feb 2023 03:01:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@asahilina.net header.s=default header.b=A6Ru0K8e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=asahilina.net Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230073AbjBXKys (ORCPT + 99 others); Fri, 24 Feb 2023 05:54:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230247AbjBXKyV (ORCPT ); Fri, 24 Feb 2023 05:54:21 -0500 Received: from mail.marcansoft.com (marcansoft.com [212.63.210.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8418164E16; Fri, 24 Feb 2023 02:54:02 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: linasend@asahilina.net) by mail.marcansoft.com (Postfix) with ESMTPSA id E917342627; Fri, 24 Feb 2023 10:53:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=asahilina.net; s=default; t=1677236040; bh=s8yg5Qe3Cf1DPHC1UIkVpTHEHGDsFbQ4/QodiHzArtM=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=A6Ru0K8ekFVsvGDAELnpxa6tTrj8c587gyepAco4OyLOpVCnGZFAAC8pzTvEnr8BK ERgbLZBGIJZOiyYEOPzwWb7HiM6Gnb+/buqxYEmBZCTjAyUKHAL3jljF9FMyC1EvdD Q795JgevAi5UnOrlku3tg1TYdXWnX6kb5468++fDi4o0VbLkfBEN4nPCX34XnnvcO9 QX7UOjo47PKDFa4+uFUN4bf4lwt7021J0XN8PI57gXxiUDMt2LhCypHcqSw9oUHMwK ZwIz4ZOANwzmOk3L6j4WCbNzqVfi1XeGxtWC2lLies3au6RfD7cxjEXS9kW/WMZbel Qz4VyAEbABKEw== From: Asahi Lina Date: Fri, 24 Feb 2023 19:53:16 +0900 Subject: [PATCH 4/5] rust: soc: apple: rtkit: Add Apple RTKit abstraction MIME-Version: 1.0 Message-Id: <20230224-rust-iopt-rtkit-v1-4-49ced3391295@asahilina.net> References: <20230224-rust-iopt-rtkit-v1-0-49ced3391295@asahilina.net> In-Reply-To: <20230224-rust-iopt-rtkit-v1-0-49ced3391295@asahilina.net> To: Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Will Deacon , Robin Murphy , Joerg Roedel , Hector Martin , Sven Peter , Arnd Bergmann , Greg Kroah-Hartman Cc: "Rafael J. Wysocki" , Alyssa Rosenzweig , Neal Gompa , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, asahi@lists.linux.dev, Asahi Lina X-Mailer: b4 0.12.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1677236013; l=11585; i=lina@asahilina.net; s=20230221; h=from:subject:message-id; bh=s8yg5Qe3Cf1DPHC1UIkVpTHEHGDsFbQ4/QodiHzArtM=; b=czVp61paD5NoMTz4vwcTr9lDdsqWjE73xnF7YjvYkIghtw6eScQB6OZFQccW+TpLk15O7XiLU mYioNvRS1+RBDOqpGHh4UMxmwD3NLQvmeW/kCm575MbOzlXnNhTeCpU X-Developer-Key: i=lina@asahilina.net; a=ed25519; pk=Qn8jZuOtR1m5GaiDfTrAoQ4NE1XoYVZ/wmt5YtXWFC4= X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758709912523429363?= X-GMAIL-MSGID: =?utf-8?q?1758709912523429363?= RTKit is Apple's proprietary real-time operating system framework, used across many subdevices on Apple Silicon platforms including NVMe, system management, GPU, etc. Add Rust abstractions for this subsystem, so that it can be used by upcoming Rust drivers. This API is safe under the expectation that all RTKit coprocessors either have no unfiltered access to system memory (SMC), are behind a DART IOMMU (DCP, etc.), or require other unsafe operations to be granted access to arbitrary system memory (GFX, where the coprocessor page tables need to be mutated in bootloader-allocated memory blocks in order to map additional memory, which is a requirement to get it to do anything interesting beyond basic startup.) Note: Although ARM64 support is not yet merged, this can be built on amd64 with CONFIG_COMPILE_TEST=y. Signed-off-by: Asahi Lina --- rust/bindings/bindings_helper.h | 1 + rust/kernel/lib.rs | 1 + rust/kernel/soc/apple/mod.rs | 6 + rust/kernel/soc/apple/rtkit.rs | 259 ++++++++++++++++++++++++++++++++++++++++ rust/kernel/soc/mod.rs | 5 + 5 files changed, 272 insertions(+) diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index 88c65431d3ad..c920d6242e3a 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -10,6 +10,7 @@ #include #include #include +#include /* `bindgen` gets confused at certain things. */ const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL; diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 9944086d7e09..78108cbbf814 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -35,6 +35,7 @@ pub mod error; pub mod io_pgtable; pub mod prelude; pub mod print; +pub mod soc; mod static_assert; #[doc(hidden)] pub mod std_vendor; diff --git a/rust/kernel/soc/apple/mod.rs b/rust/kernel/soc/apple/mod.rs new file mode 100644 index 000000000000..dd69db63677d --- /dev/null +++ b/rust/kernel/soc/apple/mod.rs @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: GPL-2.0-only OR MIT + +//! Apple SoC drivers + +#[cfg(CONFIG_APPLE_RTKIT = "y")] +pub mod rtkit; diff --git a/rust/kernel/soc/apple/rtkit.rs b/rust/kernel/soc/apple/rtkit.rs new file mode 100644 index 000000000000..595b9b3dda96 --- /dev/null +++ b/rust/kernel/soc/apple/rtkit.rs @@ -0,0 +1,259 @@ +// SPDX-License-Identifier: GPL-2.0-only OR MIT + +//! Support for Apple RTKit coprocessors. +//! +//! C header: [`include/linux/soc/apple/rtkit.h`](../../../../include/linux/gpio/driver.h) + +use crate::{ + bindings, device, + error::{code::*, from_kernel_err_ptr, from_kernel_result, to_result, Result}, + str::CStr, + types::{ForeignOwnable, ScopeGuard}, +}; + +use alloc::boxed::Box; +use core::marker::PhantomData; +use core::ptr; +use macros::vtable; + +/// Trait to represent allocatable buffers for the RTKit core. +/// +/// Users must implement this trait for their own representation of those allocations. +pub trait Buffer { + /// Returns the IOVA (virtual address) of the buffer from RTKit's point of view, or an error if + /// unavailable. + fn iova(&self) -> Result; + + /// Returns a mutable byte slice of the buffer contents, or an + /// error if unavailable. + fn buf(&mut self) -> Result<&mut [u8]>; +} + +/// Callback operations for an RTKit client. +#[vtable] +pub trait Operations { + /// Arbitrary user context type. + type Data: ForeignOwnable + Send + Sync; + + /// Type representing an allocated buffer for RTKit. + type Buffer: Buffer; + + /// Called when RTKit crashes. + fn crashed(_data: ::Borrowed<'_>) {} + + /// Called when a message was received on a non-system endpoint. Called in non-IRQ context. + fn recv_message( + _data: ::Borrowed<'_>, + _endpoint: u8, + _message: u64, + ) { + } + + /// Called in IRQ context when a message was received on a non-system endpoint. + /// + /// Must return `true` if the message is handled, or `false` to process it in + /// the handling thread. + fn recv_message_early( + _data: ::Borrowed<'_>, + _endpoint: u8, + _message: u64, + ) -> bool { + false + } + + /// Allocate a buffer for use by RTKit. + fn shmem_alloc( + _data: ::Borrowed<'_>, + _size: usize, + ) -> Result { + Err(EINVAL) + } + + /// Map an existing buffer used by RTKit at a device-specified virtual address. + fn shmem_map( + _data: ::Borrowed<'_>, + _iova: usize, + _size: usize, + ) -> Result { + Err(EINVAL) + } +} + +/// Represents `struct apple_rtkit *`. +/// +/// # Invariants +/// +/// The rtk pointer is valid. +/// The data pointer is a valid pointer from T::Data::into_foreign(). +pub struct RtKit { + rtk: *mut bindings::apple_rtkit, + data: *mut core::ffi::c_void, + _p: PhantomData, +} + +unsafe extern "C" fn crashed_callback(cookie: *mut core::ffi::c_void) { + // SAFETY: cookie is always a PointerWrapper of the right type, passed in new(). + T::crashed(unsafe { T::Data::borrow(cookie) }); +} + +unsafe extern "C" fn recv_message_callback( + cookie: *mut core::ffi::c_void, + endpoint: u8, + message: u64, +) { + // SAFETY: cookie is always a PointerWrapper of the right type, passed in new(). + T::recv_message(unsafe { T::Data::borrow(cookie) }, endpoint, message); +} + +unsafe extern "C" fn recv_message_early_callback( + cookie: *mut core::ffi::c_void, + endpoint: u8, + message: u64, +) -> bool { + // SAFETY: cookie is always a PointerWrapper of the right type, passed in new(). + T::recv_message_early(unsafe { T::Data::borrow(cookie) }, endpoint, message) +} + +unsafe extern "C" fn shmem_setup_callback( + cookie: *mut core::ffi::c_void, + bfr: *mut bindings::apple_rtkit_shmem, +) -> core::ffi::c_int { + // SAFETY: `bfr` is a valid buffer. + let bfr_mut = unsafe { &mut *bfr }; + + let buf = if bfr_mut.iova != 0 { + bfr_mut.is_mapped = true; + T::shmem_map( + // SAFETY: `cookie` came from a previous call to `into_foreign`. + unsafe { T::Data::borrow(cookie) }, + bfr_mut.iova as usize, + bfr_mut.size, + ) + } else { + bfr_mut.is_mapped = false; + // SAFETY: `cookie` came from a previous call to `into_foreign`. + T::shmem_alloc(unsafe { T::Data::borrow(cookie) }, bfr_mut.size) + }; + + from_kernel_result! { + let mut buf = buf?; + let iova = buf.iova()?; + let slice = buf.buf()?; + + if slice.len() < bfr_mut.size { + return Err(ENOMEM); + } + + bfr_mut.iova = iova as u64; + bfr_mut.buffer = slice.as_mut_ptr() as *mut _; + + // Now box the returned buffer type and stash it in the private pointer of the + // `apple_rtkit_shmem` struct for safekeeping. + bfr_mut.private = Box::into_raw(Box::try_new(buf)?) as *mut _; + Ok(0) + } +} + +unsafe extern "C" fn shmem_destroy_callback( + _cookie: *mut core::ffi::c_void, + bfr: *mut bindings::apple_rtkit_shmem, +) { + // SAFETY: `bfr` is a valid buffer. + let bfr_mut = unsafe { &mut *bfr }; + if !bfr_mut.private.is_null() { + // SAFETY: Per shmem_setup_callback, this has to be a pointer to a Buffer if it is set. + unsafe { + core::mem::drop(Box::from_raw(bfr_mut.private as *mut T::Buffer)); + } + bfr_mut.private = core::ptr::null_mut(); + } +} + +impl RtKit { + const VTABLE: bindings::apple_rtkit_ops = bindings::apple_rtkit_ops { + crashed: Some(crashed_callback::), + recv_message: Some(recv_message_callback::), + recv_message_early: Some(recv_message_early_callback::), + shmem_setup: if T::HAS_SHMEM_ALLOC || T::HAS_SHMEM_MAP { + Some(shmem_setup_callback::) + } else { + None + }, + shmem_destroy: if T::HAS_SHMEM_ALLOC || T::HAS_SHMEM_MAP { + Some(shmem_destroy_callback::) + } else { + None + }, + }; + + /// Creates a new RTKit client for a given device and optional mailbox name or index. + pub fn new( + dev: &dyn device::RawDevice, + mbox_name: Option<&'static CStr>, + mbox_idx: usize, + data: T::Data, + ) -> Result { + let ptr = data.into_foreign() as *mut _; + let guard = ScopeGuard::new(|| { + // SAFETY: `ptr` came from a previous call to `into_foreign`. + unsafe { T::Data::from_foreign(ptr) }; + }); + // SAFETY: This just calls the C init function. + let rtk = unsafe { + from_kernel_err_ptr(bindings::apple_rtkit_init( + dev.raw_device(), + ptr, + match mbox_name { + Some(s) => s.as_char_ptr(), + None => ptr::null(), + }, + mbox_idx.try_into()?, + &Self::VTABLE, + )) + }?; + + guard.dismiss(); + // INVARIANT: `rtk` and `data` are valid here. + Ok(Self { + rtk, + data: ptr, + _p: PhantomData, + }) + } + + /// Boots (wakes up) the RTKit coprocessor. + pub fn boot(&mut self) -> Result { + // SAFETY: `rtk` is valid per the type invariant. + to_result(unsafe { bindings::apple_rtkit_boot(self.rtk) }) + } + + /// Starts a non-system endpoint. + pub fn start_endpoint(&mut self, endpoint: u8) -> Result { + // SAFETY: `rtk` is valid per the type invariant. + to_result(unsafe { bindings::apple_rtkit_start_ep(self.rtk, endpoint) }) + } + + /// Sends a message to a given endpoint. + pub fn send_message(&mut self, endpoint: u8, message: u64) -> Result { + // SAFETY: `rtk` is valid per the type invariant. + to_result(unsafe { + bindings::apple_rtkit_send_message(self.rtk, endpoint, message, ptr::null_mut(), false) + }) + } +} + +// SAFETY: `RtKit` operations require a mutable reference. +unsafe impl Sync for RtKit {} +unsafe impl Send for RtKit {} + +impl Drop for RtKit { + fn drop(&mut self) { + // SAFETY: The pointer is valid by the type invariant. + unsafe { bindings::apple_rtkit_free(self.rtk) }; + + // Free context data. + // + // SAFETY: This matches the call to `into_foreign` from `new` in the success case. + unsafe { T::Data::from_foreign(self.data) }; + } +} diff --git a/rust/kernel/soc/mod.rs b/rust/kernel/soc/mod.rs new file mode 100644 index 000000000000..e3024042e74f --- /dev/null +++ b/rust/kernel/soc/mod.rs @@ -0,0 +1,5 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! SoC drivers + +pub mod apple;