From patchwork Tue Aug 1 14:31:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Dapp X-Patchwork-Id: 129323 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp2709907vqg; Tue, 1 Aug 2023 07:32:03 -0700 (PDT) X-Google-Smtp-Source: APBJJlEguTEwitiWRYamJBRZwkS4uORr0A9YZpqKycPTdgmb0zXHEyqWaUXjOSEb1e6v6WJt7QaU X-Received: by 2002:a17:907:2d0e:b0:99b:4867:5e1c with SMTP id gs14-20020a1709072d0e00b0099b48675e1cmr3392451ejc.28.1690900323195; Tue, 01 Aug 2023 07:32:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690900323; cv=none; d=google.com; s=arc-20160816; b=fC1GT5sibwY0k1pG1Cjf/DrqWm8sMZXb0CHDbC9Xut9mkWgUcyCpAl83CJwN0tJkit KASqzTRMWFJLvw0cAPUjaxJcgwvjamSeXHu+CKVTDPzWm81cZhO+7VdBpYhXp7Xna0x/ kdZZ3tUL8L5Pov9fFVtAEBKNsQWpgc92UyHsADQ6l1i3jHOUMkrX3g3oROJINHTCrb9k RoIRGo8w5osHxLXV1cu4WizYnawd4z/Ol9tccDobGkydxyn9L2HQbe9g+gHSHnDqAynb 5wuY369moeNw1O3fyDGqRlL4Ps8q7PhqIkFM3b5tn01pUGowGHKVXwaBuL+DSGNxrq4G GhPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:to:subject:content-language:cc:user-agent :mime-version:date:message-id:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=I0puHFtBi1jvCw0AHafbZfLcST6pgx9Zn8YT9tmQBW0=; fh=6vehXveSqt8KtaLXp1l9mYVT8WQLh53403zyfqHIw+Y=; b=mw9GwAzdb58IOEsRDah1tN+k1XpVvb+dSIOsf7125KY6qZQYnkIB8gG7Lc/XD7hG8T S+MF3AwDdZdg9TS5xcuyvCP/EHA6OSosZVUl35Zi+91F1VB17ET5m/mY77w9M5mldlvl c7Fvvv15SxJUEyWbLim4d4+eNq2esltOKMw1KKTU3pMlse6erHEwJbcOc+y4kNFF+elO GP+puxUxC4N05KujSnj0z2b4AXELTWhyPg1yRKtSmuP4uNVH+um1P/sWRW4ftA+1udFh J9j8lX1b9dZxqlrZ0Pv4u5vfikWimdbMdNuVITJaMkZGlEw6LlPqjF6Z+a7h3ytecWnU 34lA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=jDmPoHnx; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id fi1-20020a170906da0100b00987383af65asi4923526ejb.582.2023.08.01.07.32.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:32:03 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=jDmPoHnx; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6DB003858002 for ; Tue, 1 Aug 2023 14:31:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6DB003858002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1690900318; bh=I0puHFtBi1jvCw0AHafbZfLcST6pgx9Zn8YT9tmQBW0=; h=Date:Cc:Subject:To:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=jDmPoHnxkEVp5Rn4HZ38LoJ1VKIUiPdUF0tY140aFcJgRJ9eLJEzU3m6BlOhKBnyu cS3keKDlE9n++hQM0USCwks/4+vQTS9ygkTte3fNgw+Z1nG7MMLNu7FqVDQonqiOQK S8gIAvMU+J/RJmK4zJCmdULHC5DXr7msEb443y4Y= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by sourceware.org (Postfix) with ESMTPS id 5377E3858D28 for ; Tue, 1 Aug 2023 14:31:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5377E3858D28 Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-31297125334so3896521f8f.0 for ; Tue, 01 Aug 2023 07:31:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900266; x=1691505066; h=content-transfer-encoding:to:subject:from:content-language:cc :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=I0puHFtBi1jvCw0AHafbZfLcST6pgx9Zn8YT9tmQBW0=; b=T6Zv+Xr0AQOpCVhFXWgcXdsLzHoDJ/5HJs/dZbYGgZD3/GAU81PcU6uE7YIG+XolUk Hf2oEIh5slfPOu0hZGysfNebaJVrq4rbhxkDaYhQDzHTRNxSrP9j4ZGNCheYYWr9Laqu VAI0qkEu0Ia9tV83okRxP+0qUCeFIExnn/zOes6M7cap5fDV05G1GUXCHDt96Yexk+57 qlM72ryoCK3ORD/xVAtdkKstEgSK/RxGHMbgwnq1CtiUq5+nhMMTS9ra6qt23ssKEtCM KQOCbZTHZhekcblBsjJkpbCAbaDHdnRn24ePVw34Crt19B5bfl+0SfXxfE+mazY5oqIJ e66w== X-Gm-Message-State: ABy/qLYw/3zY4Hk3JTVoCP59JgiszNKIJbVjpVwO7TH5S+8hZ5cAw8Pk egvpnK9JcDqjJZ+2y1SvryN6il3W4c4BiQ== X-Received: by 2002:a5d:4f8a:0:b0:317:5c36:4539 with SMTP id d10-20020a5d4f8a000000b003175c364539mr2333747wru.25.1690900265344; Tue, 01 Aug 2023 07:31:05 -0700 (PDT) Received: from [192.168.1.23] (ip-046-005-130-086.um12.pools.vodafone-ip.de. [46.5.130.86]) by smtp.gmail.com with ESMTPSA id z1-20020adfd0c1000000b0031424f4ef1dsm16461986wrh.19.2023.08.01.07.31.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 01 Aug 2023 07:31:04 -0700 (PDT) Message-ID: <163c273d-3c01-8ece-21a5-b6ce88174ac0@gmail.com> Date: Tue, 1 Aug 2023 16:31:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: rdapp.gcc@gmail.com Content-Language: en-US Subject: [PATCH] RISC-V: Implement vector "average" autovec pattern. To: gcc-patches , palmer , Kito Cheng , "juzhe.zhong@rivai.ai" , jeffreyalaw X-Spam-Status: No, score=-8.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Robin Dapp via Gcc-patches From: Robin Dapp Reply-To: Robin Dapp Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773037496976939096 X-GMAIL-MSGID: 1773037496976939096 Hi, this patch adds vector average patterns op[0] = (narrow) ((wide) op[1] + (wide) op[2]) >> 1; op[0] = (narrow) ((wide) op[1] + (wide) op[2] + 1) >> 1; If there is no direct support, the vectorizer can synthesize the patterns but, presumably due to lack of narrowing operation support, won't try a narrowing shift. Therefore, this patch implements the expanders instead. A synthesized pattern results in e.g: vsrl.vi v2,v1,1 vsrl.vi v4,v3,1 vand.vv v1,v1,v3 vadd.vv v2,v2,v4 vand.vi v1,v1,1 vadd.vv v1,v2,v1 With this patch we generate: vwadd.vv v2,v4,v1 vadd.vi v2,1 vnsrl.wi v2,v2,1 We manage to recover (i.e. create the latter sequence) for signed types but not for unsigned. I figured that offering both patterns might be the safe thing to do but open to leaving the signed one out. In the long term we'd want full vectorizer support for this I suppose. Regards Robin gcc/ChangeLog: * config/riscv/autovec.md (avg3_floor): Implement expander. (avg3_ceil): Ditto. * config/riscv/vector-iterators.md (ashiftrt): New iterator. (ASHIFTRT): Ditto. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/vec-avg-run.c: New test. * gcc.target/riscv/rvv/autovec/vec-avg-rv32gcv.c: New test. * gcc.target/riscv/rvv/autovec/vec-avg-rv64gcv.c: New test. * gcc.target/riscv/rvv/autovec/vec-avg-template.h: New test. --- gcc/config/riscv/autovec.md | 66 ++++++++++++++ gcc/config/riscv/vector-iterators.md | 5 ++ .../riscv/rvv/autovec/vec-avg-run.c | 85 +++++++++++++++++++ .../riscv/rvv/autovec/vec-avg-rv32gcv.c | 10 +++ .../riscv/rvv/autovec/vec-avg-rv64gcv.c | 10 +++ .../riscv/rvv/autovec/vec-avg-template.h | 33 +++++++ 6 files changed, 209 insertions(+) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-run.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv32gcv.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv64gcv.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-template.h diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index 7b784437c7e..23d3c2feaff 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -1752,3 +1752,69 @@ (define_expand "mask_len_fold_left_plus_" riscv_vector::reduction_type::MASK_LEN_FOLD_LEFT); DONE; }) + +;; ------------------------------------------------------------------------- +;; ---- [INT] Average. +;; ------------------------------------------------------------------------- +;; Implements the following "average" patterns: +;; floor: +;; op[0] = (narrow) ((wide) op[1] + (wide) op[2]) >> 1; +;; ceil: +;; op[0] = (narrow) ((wide) op[1] + (wide) op[2] + 1)) >> 1; +;; ------------------------------------------------------------------------- + +(define_expand "avg3_floor" + [(set (match_operand: 0 "register_operand") + (truncate: + (:VWEXTI + (plus:VWEXTI + (any_extend:VWEXTI + (match_operand: 1 "register_operand")) + (any_extend:VWEXTI + (match_operand: 2 "register_operand"))))))] + "TARGET_VECTOR" +{ + /* First emit a widening addition. */ + rtx tmp1 = gen_reg_rtx (mode); + rtx ops1[] = {tmp1, operands[1], operands[2]}; + insn_code icode = code_for_pred_dual_widen (PLUS, , mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, ops1); + + /* Then a narrowing shift. */ + rtx ops2[] = {operands[0], tmp1, const1_rtx}; + icode = code_for_pred_narrow_scalar (, mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, ops2); + DONE; +}) + +(define_expand "avg3_ceil" + [(set (match_operand: 0 "register_operand") + (truncate: + (:VWEXTI + (plus:VWEXTI + (plus:VWEXTI + (any_extend:VWEXTI + (match_operand: 1 "register_operand")) + (any_extend:VWEXTI + (match_operand: 2 "register_operand"))) + (const_int 1)))))] + "TARGET_VECTOR" +{ + /* First emit a widening addition. */ + rtx tmp1 = gen_reg_rtx (mode); + rtx ops1[] = {tmp1, operands[1], operands[2]}; + insn_code icode = code_for_pred_dual_widen (PLUS, , mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, ops1); + + /* Then add 1. */ + rtx tmp2 = gen_reg_rtx (mode); + rtx ops2[] = {tmp2, tmp1, const1_rtx}; + icode = code_for_pred_scalar (PLUS, mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, ops2); + + /* Finally, a narrowing shift. */ + rtx ops3[] = {operands[0], tmp2, const1_rtx}; + icode = code_for_pred_narrow_scalar (, mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, ops3); + DONE; +}) diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md index 37c6337f1a3..409f63332c9 100644 --- a/gcc/config/riscv/vector-iterators.md +++ b/gcc/config/riscv/vector-iterators.md @@ -1968,6 +1968,11 @@ (define_code_attr macc_msac [(plus "macc") (minus "msac")]) (define_code_attr nmsub_nmadd [(plus "nmsub") (minus "nmadd")]) (define_code_attr nmsac_nmacc [(plus "nmsac") (minus "nmacc")]) +(define_code_attr ext_to_rshift [(sign_extend "ashiftrt") + (zero_extend "lshiftrt")]) +(define_code_attr EXT_TO_RSHIFT [(sign_extend "ASHIFTRT") + (zero_extend "LSHIFTRT")]) + (define_code_iterator and_ior [and ior]) (define_code_iterator any_float_binop [plus mult minus div]) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-run.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-run.c new file mode 100644 index 00000000000..7ca193ec2f2 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-run.c @@ -0,0 +1,85 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "-std=c99 -fno-vect-cost-model --param=riscv-autovec-preference=scalable -lm" } */ + +#include +#include +#include + +#include "vec-avg-template.h" + +#define SZ 256 + +#define RUNS1(TYPE, SCALE) \ + TYPE a##TYPE[SZ + 1]; \ + TYPE b##TYPE[SZ + 1]; \ + TYPE dst##TYPE[SZ + 1]; \ + for (int cnt = 0, i = -(SZ * SCALE) / 2; i < (SZ * SCALE) / 2; i += SCALE) \ + { \ + a##TYPE[cnt] = i; \ + b##TYPE[cnt] = i + 1; \ + dst##TYPE[cnt++] = 0; \ + } \ + vavg_##TYPE (dst##TYPE, a##TYPE, b##TYPE, SZ); \ + for (int i = 0; i < SZ; i += SCALE) \ + assert (dst##TYPE[i] == floor ((a##TYPE[i] + b##TYPE[i]) / 2.0)); + +#define RUNU1(TYPE, SCALE) \ + TYPE a##TYPE[SZ + 1]; \ + TYPE b##TYPE[SZ + 1]; \ + TYPE dst##TYPE[SZ + 1]; \ + for (int cnt = 0, i = 0; i < (SZ * SCALE); i += SCALE) \ + { \ + a##TYPE[cnt] = i; \ + b##TYPE[cnt] = i + 1; \ + dst##TYPE[cnt++] = 0; \ + } \ + vavg_##TYPE (dst##TYPE, a##TYPE, b##TYPE, SZ); \ + for (int i = 0; i < SZ; i += SCALE) \ + assert (dst##TYPE[i] == floor ((a##TYPE[i] + b##TYPE[i]) / 2.0)); + +#define RUNS2(TYPE, SCALE) \ + TYPE a2##TYPE[SZ + 1]; \ + TYPE b2##TYPE[SZ + 1]; \ + TYPE dst2##TYPE[SZ + 1]; \ + for (int cnt = 0, i = -(SZ * SCALE) / 2; i < (SZ * SCALE) / 2; i += SCALE) \ + { \ + a2##TYPE[cnt] = i; \ + b2##TYPE[cnt] = i + 1; \ + dst2##TYPE[cnt++] = 0; \ + } \ + vavg2_##TYPE (dst2##TYPE, a2##TYPE, b2##TYPE, SZ); \ + for (int i = 0; i < SZ; i += SCALE) \ + assert (dst2##TYPE[i] == ceil ((a2##TYPE[i] + b2##TYPE[i]) / 2.0)); + +#define RUNU2(TYPE, SCALE) \ + TYPE a2##TYPE[SZ + 1]; \ + TYPE b2##TYPE[SZ + 1]; \ + TYPE dst2##TYPE[SZ + 1]; \ + for (int cnt = 0, i = 0; i < (SZ * SCALE); i += SCALE) \ + { \ + a2##TYPE[cnt] = i; \ + b2##TYPE[cnt] = i + 1; \ + dst2##TYPE[cnt++] = 0; \ + } \ + vavg2_##TYPE (dst2##TYPE, a2##TYPE, b2##TYPE, SZ); \ + for (int i = 0; i < SZ; i += SCALE) \ + assert (dst2##TYPE[i] == ceil ((a2##TYPE[i] + b2##TYPE[i]) / 2.0)); + +#define RUN_ALL() \ + RUNS1 (int8_t, 1) \ + RUNS1 (int16_t, 256) \ + RUNS1 (int32_t, 65536) \ + RUNU1 (uint8_t, 1) \ + RUNU1 (uint16_t, 256) \ + RUNU1 (uint32_t, 65536) \ + RUNS2 (int8_t, 1) \ + RUNS2 (int16_t, 256) \ + RUNS2 (int32_t, 65536) \ + RUNU2 (uint8_t, 1) \ + RUNU2 (uint16_t, 256) \ + RUNU2 (uint32_t, 65536)\ + +int main () +{ + RUN_ALL () +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv32gcv.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv32gcv.c new file mode 100644 index 00000000000..e2754339d94 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv32gcv.c @@ -0,0 +1,10 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-std=c99 -fno-vect-cost-model -march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include "vec-avg-template.h" + +/* { dg-final { scan-assembler-times {\tvwadd\.vv} 6 } } */ +/* { dg-final { scan-assembler-times {\tvwaddu\.vv} 6 } } */ +/* { dg-final { scan-assembler-times {\tvadd\.vi} 6 } } */ +/* { dg-final { scan-assembler-times {\tvnsrl.wi} 6 } } */ +/* { dg-final { scan-assembler-times {\tvnsra.wi} 6 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv64gcv.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv64gcv.c new file mode 100644 index 00000000000..210c0dc5460 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-rv64gcv.c @@ -0,0 +1,10 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-std=c99 -fno-vect-cost-model -march=rv64gcv_zvfh -mabi=lp64d --param=riscv-autovec-preference=scalable" } */ + +#include "vec-avg-template.h" + +/* { dg-final { scan-assembler-times {\tvwadd\.vv} 6 } } */ +/* { dg-final { scan-assembler-times {\tvwaddu\.vv} 6 } } */ +/* { dg-final { scan-assembler-times {\tvadd\.vi} 6 } } */ +/* { dg-final { scan-assembler-times {\tvnsrl\.wi} 6 } } */ +/* { dg-final { scan-assembler-times {\tvnsra\.wi} 6 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-template.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-template.h new file mode 100644 index 00000000000..9c2a6f1b9cb --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec-avg-template.h @@ -0,0 +1,33 @@ +#include + +#define TEST_TYPE(TYPE, TYPE2) \ + __attribute__ ((noipa)) void vavg_##TYPE (TYPE *dst, TYPE *a, TYPE *b, \ + int n) \ + { \ + for (int i = 0; i < n; i++) \ + dst[i] = ((TYPE2) a[i] + b[i]) >> 1; \ + } + +#define TEST_TYPE2(TYPE, TYPE2) \ + __attribute__ ((noipa)) void vavg2_##TYPE (TYPE *dst, TYPE *a, TYPE *b, \ + int n) \ + { \ + for (int i = 0; i < n; i++) \ + dst[i] = ((TYPE2) a[i] + b[i] + 1) >> 1; \ + } + +#define TEST_ALL() \ + TEST_TYPE (int8_t, int16_t) \ + TEST_TYPE (uint8_t, uint16_t) \ + TEST_TYPE (int16_t, int32_t) \ + TEST_TYPE (uint16_t, uint32_t) \ + TEST_TYPE (int32_t, int64_t) \ + TEST_TYPE (uint32_t, uint64_t) \ + TEST_TYPE2 (int8_t, int16_t) \ + TEST_TYPE2 (uint8_t, uint16_t) \ + TEST_TYPE2 (int16_t, int32_t) \ + TEST_TYPE2 (uint16_t, uint32_t) \ + TEST_TYPE2 (int32_t, int64_t) \ + TEST_TYPE2 (uint32_t, uint64_t) + +TEST_ALL()