From patchwork Wed Aug 23 14:14:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Stubbs X-Patchwork-Id: 136669 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a7d1:0:b0:3f2:4152:657d with SMTP id p17csp490478vqm; Wed, 23 Aug 2023 07:16:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHEawDWYC3hlFzScm/VGsbRAyF38D/7aJNf6zqa8j4/Sw8U65FBGXTEXqIGEC6Xcup1PJVI X-Received: by 2002:a05:6512:12c4:b0:4fb:9712:a717 with SMTP id p4-20020a05651212c400b004fb9712a717mr10028175lfg.13.1692800169902; Wed, 23 Aug 2023 07:16:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692800169; cv=none; d=google.com; s=arc-20160816; b=mZSP4MkAUHzOoZTneiu76DYJP6hDEI4rEpnwKFd1qB/nJDyWmAiWHIfPja/tGG5SHP HpRR/jSh+OTJVQUnrCSozQBI7tQFQqV0xE1UjwkryUQ1HYCTyCWLVlvcokxUjhZoM/Wd F9MX1Df5NXNC7OpYMzVTDQGp851ha+lw0bwZOF+ozBvSHeSz1A9GEwO8mxrZ14e/y+DH 0ftVFWWilKbErNqPBjH6MudjXx+pIPRZWBAqVBlEUOMbjFAesrpG9UrPCq1uUS0uwvR0 mQMRaoeq3rAwZgKS2jeDzsCLATY8ST3MjmMu0DC2bylY3yenEG+whGO1IVvyZf9BPYwL 6qiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :ironport-sdr:dmarc-filter:delivered-to; bh=sg+ZcB2HBJfyIOHSc1hqGi3fiAxUJ6S6S+RYcZVUROQ=; fh=9xD62naDDQ44CEuxrRV0ts5PrjZies6B3X/DCYdxTRw=; b=yp8b5u3Jcw3HJ2GI8KvS9btQXaMj7w5LMVWYLMWndsg4Y8h9aC3QGGQg2zLOS8mTsK w0+YvwJQms2KGxnXcPia2HYV0hoJ3vOfPWHMutCSkbHJIw3QCILhxOaPPB7DuwMTZI9M 41tuRT4sserdeq3IBxbaAOEoQRr6loHSrzzEQJWVr7VctP/lzUTp+Ptf4yAUcmNepQPy ya72RztqLX2eOLdap+uxj8ii68+P/OxI+ih+3rmOUU7vQdn//JCL5/mZLZ5cX0QuDbcZ 1iedUiWbKR67XqYr2QbS115X3kvDiucdvj6FTD3h4JnF6oGLYp8UXI3Pa3HI2g8x1QSc U44A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id q15-20020aa7d44f000000b0052553cc3360si9354289edr.118.2023.08.23.07.16.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 07:16:09 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1AB413882155 for ; Wed, 23 Aug 2023 14:15:11 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from esa1.mentor.iphmx.com (esa1.mentor.iphmx.com [68.232.129.153]) by sourceware.org (Postfix) with ESMTPS id 5E6CE3857C5A for ; Wed, 23 Aug 2023 14:14:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5E6CE3857C5A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mentor.com X-IronPort-AV: E=Sophos;i="6.01,195,1684828800"; d="scan'208";a="17132438" Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa1.mentor.iphmx.com with ESMTP; 23 Aug 2023 06:14:42 -0800 IronPort-SDR: jGVyU7XgPkJjXWtd6ei/ztiC7OWJdK0dpCJBZg3/zeoJcXo0MAB2yNC4QpUdxv3NrG7gSEM2/g vnbhprb3t0CdEfuXSux7+iIkfACGmSwbBrf5sudJ4X1W7B8D+DTSSZ23wwyuysGmrGTrhP5/tT HtEGQroXfDnWVTuSG2730aAK+0w0qzijDgB5PG9TG389gIwt/Cv3kfSN+lSqAJi3qBFQeLYwIO dwnq+mo3jBkxqUTfDGvfgMHGAcPfYObZuiCjC1e7+i5IfzcLU2FRbB2X7OkMa5x3T08FgitfMl Y+M= From: Andrew Stubbs To: Subject: [PATCH v2 2/6] libgomp, openmp: Add ompx_pinned_mem_alloc Date: Wed, 23 Aug 2023 15:14:22 +0100 Message-ID: <20230823141426.320160-3-ams@codesourcery.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230823141426.320160-1-ams@codesourcery.com> References: <20230823141426.320160-1-ams@codesourcery.com> MIME-Version: 1.0 X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-15.mgc.mentorg.com (139.181.222.15) To svr-ies-mbx-11.mgc.mentorg.com (139.181.222.11) X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1775029630928312748 X-GMAIL-MSGID: 1775029630928312748 This creates a new predefined allocator as a shortcut for using pinned memory with OpenMP. The name uses the OpenMP extension space and is intended to be consistent with other OpenMP implementations currently in development. The allocator is equivalent to using a custom allocator with the pinned trait and the null fallback trait. libgomp/ChangeLog: * allocator.c (omp_max_predefined_alloc): Update. (predefined_alloc_mapping): Add ompx_pinned_mem_alloc entry. (omp_aligned_alloc): Support ompx_pinned_mem_alloc. (omp_free): Likewise. (omp_aligned_calloc): Likewise. (omp_realloc): Likewise. * omp.h.in (omp_allocator_handle_t): Add ompx_pinned_mem_alloc. * omp_lib.f90.in: Add ompx_pinned_mem_alloc. * testsuite/libgomp.c/alloc-pinned-5.c: New test. * testsuite/libgomp.c/alloc-pinned-6.c: New test. * testsuite/libgomp.fortran/alloc-pinned-1.f90: New test. Co-Authored-By: Thomas Schwinge --- libgomp/allocator.c | 58 ++++++---- libgomp/omp.h.in | 1 + libgomp/omp_lib.f90.in | 2 + libgomp/testsuite/libgomp.c/alloc-pinned-5.c | 103 ++++++++++++++++++ libgomp/testsuite/libgomp.c/alloc-pinned-6.c | 101 +++++++++++++++++ .../libgomp.fortran/alloc-pinned-1.f90 | 16 +++ 6 files changed, 262 insertions(+), 19 deletions(-) create mode 100644 libgomp/testsuite/libgomp.c/alloc-pinned-5.c create mode 100644 libgomp/testsuite/libgomp.c/alloc-pinned-6.c create mode 100644 libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 diff --git a/libgomp/allocator.c b/libgomp/allocator.c index 6007e64f580..39ba1d07bc7 100644 --- a/libgomp/allocator.c +++ b/libgomp/allocator.c @@ -35,7 +35,7 @@ #include #endif -#define omp_max_predefined_alloc omp_thread_mem_alloc +#define omp_max_predefined_alloc ompx_pinned_mem_alloc /* These macros may be overridden in config//allocator.c. The following definitions (ab)use comma operators to avoid unused @@ -76,6 +76,7 @@ static const omp_memspace_handle_t predefined_alloc_mapping[] = { omp_low_lat_mem_space, /* omp_cgroup_mem_alloc. */ omp_low_lat_mem_space, /* omp_pteam_mem_alloc. */ omp_low_lat_mem_space, /* omp_thread_mem_alloc. */ + omp_default_mem_space, /* ompx_pinned_mem_alloc. */ }; enum gomp_numa_memkind_kind @@ -612,8 +613,10 @@ retry: memspace = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); - ptr = MEMSPACE_ALLOC (memspace, new_size, - allocator_data && allocator_data->pinned); + int pinned = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); + ptr = MEMSPACE_ALLOC (memspace, new_size, pinned); } if (ptr == NULL) goto fail; @@ -634,7 +637,8 @@ retry: fail:; int fallback = (allocator_data ? allocator_data->fallback - : allocator == omp_default_mem_alloc + : (allocator == omp_default_mem_alloc + || allocator == ompx_pinned_mem_alloc) ? omp_atv_null_fb : omp_atv_default_mem_fb); switch (fallback) @@ -762,6 +766,7 @@ omp_free (void *ptr, omp_allocator_handle_t allocator) #endif memspace = predefined_alloc_mapping[data->allocator]; + pinned = (data->allocator == ompx_pinned_mem_alloc); } MEMSPACE_FREE (memspace, data->ptr, data->size, pinned); @@ -935,8 +940,10 @@ retry: memspace = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); - ptr = MEMSPACE_CALLOC (memspace, new_size, - allocator_data && allocator_data->pinned); + int pinned = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); + ptr = MEMSPACE_CALLOC (memspace, new_size, pinned); } if (ptr == NULL) goto fail; @@ -957,7 +964,8 @@ retry: fail:; int fallback = (allocator_data ? allocator_data->fallback - : allocator == omp_default_mem_alloc + : (allocator == omp_default_mem_alloc + || allocator == ompx_pinned_mem_alloc) ? omp_atv_null_fb : omp_atv_default_mem_fb); switch (fallback) @@ -1180,11 +1188,14 @@ retry: else #endif if (prev_size) - new_ptr = MEMSPACE_REALLOC (allocator_data->memspace, data->ptr, - data->size, new_size, - (free_allocator_data - && free_allocator_data->pinned), - allocator_data->pinned); + { + int was_pinned = (free_allocator_data + ? free_allocator_data->pinned + : free_allocator == ompx_pinned_mem_alloc); + new_ptr = MEMSPACE_REALLOC (allocator_data->memspace, data->ptr, + data->size, new_size, was_pinned, + allocator_data->pinned); + } else new_ptr = MEMSPACE_ALLOC (allocator_data->memspace, new_size, allocator_data->pinned); @@ -1240,10 +1251,14 @@ retry: memspace = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); + int was_pinned = (free_allocator_data + ? free_allocator_data->pinned + : free_allocator == ompx_pinned_mem_alloc); + int pinned = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); new_ptr = MEMSPACE_REALLOC (memspace, data->ptr, data->size, new_size, - (free_allocator_data - && free_allocator_data->pinned), - allocator_data && allocator_data->pinned); + was_pinned, pinned); } if (new_ptr == NULL) goto fail; @@ -1277,8 +1292,10 @@ retry: memspace = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); - new_ptr = MEMSPACE_ALLOC (memspace, new_size, - allocator_data && allocator_data->pinned); + int pinned = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); + new_ptr = MEMSPACE_ALLOC (memspace, new_size, pinned); } if (new_ptr == NULL) goto fail; @@ -1333,7 +1350,9 @@ retry: was_memspace = (free_allocator_data ? free_allocator_data->memspace : predefined_alloc_mapping[free_allocator]); - int was_pinned = (free_allocator_data && free_allocator_data->pinned); + int was_pinned = (free_allocator_data + ? free_allocator_data->pinned + : free_allocator == ompx_pinned_mem_alloc); MEMSPACE_FREE (was_memspace, data->ptr, data->size, was_pinned); } return ret; @@ -1341,7 +1360,8 @@ retry: fail:; int fallback = (allocator_data ? allocator_data->fallback - : allocator == omp_default_mem_alloc + : (allocator == omp_default_mem_alloc + || allocator == ompx_pinned_mem_alloc) ? omp_atv_null_fb : omp_atv_default_mem_fb); switch (fallback) diff --git a/libgomp/omp.h.in b/libgomp/omp.h.in index bd1286c2a3f..66989e693b7 100644 --- a/libgomp/omp.h.in +++ b/libgomp/omp.h.in @@ -134,6 +134,7 @@ typedef enum omp_allocator_handle_t __GOMP_UINTPTR_T_ENUM omp_cgroup_mem_alloc = 6, omp_pteam_mem_alloc = 7, omp_thread_mem_alloc = 8, + ompx_pinned_mem_alloc = 9, __omp_allocator_handle_t_max__ = __UINTPTR_MAX__ } omp_allocator_handle_t; diff --git a/libgomp/omp_lib.f90.in b/libgomp/omp_lib.f90.in index e4515271252..699f97053de 100644 --- a/libgomp/omp_lib.f90.in +++ b/libgomp/omp_lib.f90.in @@ -158,6 +158,8 @@ parameter :: omp_pteam_mem_alloc = 7 integer (kind=omp_allocator_handle_kind), & parameter :: omp_thread_mem_alloc = 8 + integer (kind=omp_allocator_handle_kind), & + parameter :: ompx_pinned_mem_alloc = 9 integer (omp_memspace_handle_kind), & parameter :: omp_default_mem_space = 0 integer (omp_memspace_handle_kind), & diff --git a/libgomp/testsuite/libgomp.c/alloc-pinned-5.c b/libgomp/testsuite/libgomp.c/alloc-pinned-5.c new file mode 100644 index 00000000000..9c69dbb7cde --- /dev/null +++ b/libgomp/testsuite/libgomp.c/alloc-pinned-5.c @@ -0,0 +1,103 @@ +/* { dg-do run } */ + +/* { dg-xfail-run-if "Pinning not implemented on this host" { ! *-*-linux-gnu } } */ + +/* Test that ompx_pinned_mem_alloc works. */ + +#include +#include + +#ifdef __linux__ +#include +#include + +#include +#include + +#define PAGE_SIZE sysconf(_SC_PAGESIZE) +#define CHECK_SIZE(SIZE) { \ + struct rlimit limit; \ + if (getrlimit (RLIMIT_MEMLOCK, &limit) \ + || limit.rlim_cur <= SIZE) \ + fprintf (stderr, "unsufficient lockable memory; please increase ulimit\n"); \ + } + +int +get_pinned_mem () +{ + int pid = getpid (); + char buf[100]; + sprintf (buf, "/proc/%d/status", pid); + + FILE *proc = fopen (buf, "r"); + if (!proc) + abort (); + while (fgets (buf, 100, proc)) + { + int val; + if (sscanf (buf, "VmLck: %d", &val)) + { + fclose (proc); + return val; + } + } + abort (); +} +#else +#define PAGE_SIZE 1 /* unknown */ +#define CHECK_SIZE(SIZE) fprintf (stderr, "OS unsupported\n"); + +int +get_pinned_mem () +{ + return 0; +} +#endif + +static void +verify0 (char *p, size_t s) +{ + for (size_t i = 0; i < s; ++i) + if (p[i] != 0) + abort (); +} + +#include + +int +main () +{ + /* Allocate at least a page each time, allowing space for overhead, + but stay within the ulimit. */ + const int SIZE = PAGE_SIZE - 128; + CHECK_SIZE (SIZE * 5); + + // Sanity check + if (get_pinned_mem () != 0) + abort (); + + void *p = omp_alloc (SIZE, ompx_pinned_mem_alloc); + if (!p) + abort (); + + int amount = get_pinned_mem (); + if (amount == 0) + abort (); + + p = omp_realloc (p, SIZE * 2, ompx_pinned_mem_alloc, ompx_pinned_mem_alloc); + + int amount2 = get_pinned_mem (); + if (amount2 <= amount) + abort (); + + /* SIZE*2 ensures that it doesn't slot into the space possibly + vacated by realloc. */ + p = omp_calloc (1, SIZE * 2, ompx_pinned_mem_alloc); + + if (get_pinned_mem () <= amount2) + abort (); + + verify0 (p, SIZE * 2); + + return 0; +} diff --git a/libgomp/testsuite/libgomp.c/alloc-pinned-6.c b/libgomp/testsuite/libgomp.c/alloc-pinned-6.c new file mode 100644 index 00000000000..f80a0264f97 --- /dev/null +++ b/libgomp/testsuite/libgomp.c/alloc-pinned-6.c @@ -0,0 +1,101 @@ +/* { dg-do run } */ + +/* Test that ompx_pinned_mem_alloc fails correctly. */ + +#include +#include + +#ifdef __linux__ +#include +#include + +#include +#include + +#define PAGE_SIZE sysconf(_SC_PAGESIZE) + +int +get_pinned_mem () +{ + int pid = getpid (); + char buf[100]; + sprintf (buf, "/proc/%d/status", pid); + + FILE *proc = fopen (buf, "r"); + if (!proc) + abort (); + while (fgets (buf, 100, proc)) + { + int val; + if (sscanf (buf, "VmLck: %d", &val)) + { + fclose (proc); + return val; + } + } + abort (); +} + +void +set_pin_limit (int size) +{ + struct rlimit limit; + if (getrlimit (RLIMIT_MEMLOCK, &limit)) + abort (); + limit.rlim_cur = (limit.rlim_max < size ? limit.rlim_max : size); + if (setrlimit (RLIMIT_MEMLOCK, &limit)) + abort (); +} +#else +#define PAGE_SIZE 10000 * 1024 /* unknown */ + +int +get_pinned_mem () +{ + return 0; +} + +void +set_pin_limit () +{ +} +#endif + +#include + +int +main () +{ + /* Allocate at least a page each time, but stay within the ulimit. */ + const int SIZE = PAGE_SIZE * 4; + + /* Ensure that the limit is smaller than the allocation. */ + set_pin_limit (SIZE / 2); + + // Sanity check + if (get_pinned_mem () != 0) + abort (); + + // Should fail + void *p = omp_alloc (SIZE, ompx_pinned_mem_alloc); + if (p) + abort (); + + // Should fail + p = omp_calloc (1, SIZE, ompx_pinned_mem_alloc); + if (p) + abort (); + + // Should fail to realloc + void *notpinned = omp_alloc (SIZE, omp_default_mem_alloc); + p = omp_realloc (notpinned, SIZE, ompx_pinned_mem_alloc, omp_default_mem_alloc); + if (!notpinned || p) + abort (); + + // No memory should have been pinned + int amount = get_pinned_mem (); + if (amount != 0) + abort (); + + return 0; +} diff --git a/libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 b/libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 new file mode 100644 index 00000000000..798dc3d5a12 --- /dev/null +++ b/libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 @@ -0,0 +1,16 @@ +! Ensure that the ompx_pinned_mem_alloc predefined allocator is present and +! accepted. The majority of the functionality testing lives in the C tests. +! +! { dg-xfail-run-if "Pinning not implemented on this host" { ! *-*-linux-gnu } } + +program main + use omp_lib + use ISO_C_Binding + implicit none (external, type) + + type(c_ptr) :: p + + p = omp_alloc (10_c_size_t, ompx_pinned_mem_alloc); + if (.not. c_associated (p)) stop 1 + call omp_free (p, ompx_pinned_mem_alloc); +end program main