Message ID | 20240124122523.384659-2-rjones@redhat.com |
---|---|
State | Accepted |
Headers |
Return-Path: <binutils-bounces+ouuuleilei=gmail.com@sourceware.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp951153dyi; Wed, 24 Jan 2024 04:25:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IGw37yuS1nfcgdrsKW9DJwroKPrhaIOISrHuY+Qv2I6Mb1SlrNAQ/vcysMSgBAWwI+7R63+ X-Received: by 2002:a05:622a:1309:b0:42a:3274:cc72 with SMTP id v9-20020a05622a130900b0042a3274cc72mr1748756qtk.12.1706099142574; Wed, 24 Jan 2024 04:25:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706099142; cv=pass; d=google.com; s=arc-20160816; b=jzqwT5McL477qt58iZrv7mTzdinlYg1f9s4H1fUBymPFGjdD9iA7nJ1vkLo9jaTcrm AbClKRkPs1B9VLxf26jkbxXxCUrpPcd9UjM724Bmgvh2OEnlfsImQDURXIQA+GuEkPzE e/d7m6S2Z2wH0WphiyJadAhobufEb5pQc89LQJeKAPxaNKGsDjsTn1y/HbsXagDteKnF b/7Di6NSu4i+ReUL+Lp2/wiB5QW3IsPMcjAFQI0GEbeIePdpM3rmCQ7DI/XE27lG040p gYdMWR33hAcwPtZymJNk+grF4EeyzGvVKSHQolOPCBgIl7VaPz6C6ACsVLqBkeQTlQRz n09g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=MwviDOBA+wX1uVl0OhhrVmHLW/P2XOHrcpxHP4jOB28=; fh=r9stD0P3Vp8+80xVgHnPxGhbW/56gL/04uIfctklMLY=; b=Jyxq2I+Dg02y9wFsZ4A/FWO7UfsF62foVH5nSWO3BBgkC8IrLuZZgfQdJSZXB5uB+6 VRcofmIk5rFbA1Q+EaU0PX5+2uImCiVWpFkznZI6OHIi92DgkDDdMlD2Vn0oS+ynTScM KZkpn+IoxYXyot7S6JXOW/+gkJu5XFdUpno/WeP2qi104E3ZtxoQYQ5upDYq4KYveHgg 6k666e2yf1jN9srqXiTKbEzZofs5imcAJH/ve2yaDc0Dt93WLdehoVV2xpI5nY3yO34V eYe3Aah2xjquMyTsU9IkgHhPJIdNIBM+Vu535zu6OPpoy/9S1t+6ViTIvsWtd6he0OlV T87w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QauHigQ6; arc=pass (i=1); spf=pass (google.com: domain of binutils-bounces+ouuuleilei=gmail.com@sourceware.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="binutils-bounces+ouuuleilei=gmail.com@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id k20-20020a05622a03d400b00429cdea863bsi10364659qtx.208.2024.01.24.04.25.42 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 04:25:42 -0800 (PST) Received-SPF: pass (google.com: domain of binutils-bounces+ouuuleilei=gmail.com@sourceware.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QauHigQ6; arc=pass (i=1); spf=pass (google.com: domain of binutils-bounces+ouuuleilei=gmail.com@sourceware.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="binutils-bounces+ouuuleilei=gmail.com@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 58AD83857C66 for <ouuuleilei@gmail.com>; Wed, 24 Jan 2024 12:25:41 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 865323858C35 for <binutils@sourceware.org>; Wed, 24 Jan 2024 12:25:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 865323858C35 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 865323858C35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706099129; cv=none; b=jcnXG97CSEQq5iQMcauXnli9mMCzCtg2olOIzdlOzvzGafihd+f/Qm+4ox018Pi5uNW2PlEOBP59wqUVnksk9nAgjzFnBAOkkgc6Jzqldq/hS+y5vN6aDmCOBQiaeb4okUKrFj0MnhQ8YkNVF4lKrmtmxaJSRzUhqOy4GN93iPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706099129; c=relaxed/simple; bh=k+1JJH9Ynr1OyB6+1AqsukKRPsWtfXqopvr/FF0c8mk=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=nO4svQOw8jg4E7KT4AYhHwr1D0L5FMWNcdahANbwFaqeATYHGD8eQuYFzELkAaRh4TZ04h3vFX/IdndXmUMw2XSfFb0GPy80iYgaUFhqU+aBY5MP9ryIM94SF07Z4nz0Is8cpXbWpS5t7Imp3HQG0KRSMsHmTtgv4Av4FRSuac0= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706099127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MwviDOBA+wX1uVl0OhhrVmHLW/P2XOHrcpxHP4jOB28=; b=QauHigQ6/Kepts3JfB39ge5DlTDOs2Y8QtdNqOEIF1YHfZn2MWztTWP9wOtq7bWEkhaOrz Sj8DQM3T+rDV9H7uOKoJRMKuDn7AUc4Z/CfgpMOQRXpomONswc1S3kjEYFksTi+wwZafPK r2Mwdcl5g5HfbGztO/j2UUSwl2Z7q2U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-504-drhb6mf9NZOwCWbbh_LKZA-1; Wed, 24 Jan 2024 07:25:26 -0500 X-MC-Unique: drhb6mf9NZOwCWbbh_LKZA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C5CFE85A588 for <binutils@sourceware.org>; Wed, 24 Jan 2024 12:25:25 +0000 (UTC) Received: from devr9.home.annexia.org (unknown [10.42.28.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6535A3C2E; Wed, 24 Jan 2024 12:25:25 +0000 (UTC) From: "Richard W.M. Jones" <rjones@redhat.com> To: binutils@sourceware.org Cc: rjones@redhat.com Subject: [PATCH v2] binutils/windmc: Parse input correctly on big endian hosts Date: Wed, 24 Jan 2024 12:25:23 +0000 Message-Id: <20240124122523.384659-2-rjones@redhat.com> In-Reply-To: <20240124122523.384659-1-rjones@redhat.com> References: <20240124122523.384659-1-rjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list <binutils.sourceware.org> List-Unsubscribe: <https://sourceware.org/mailman/options/binutils>, <mailto:binutils-request@sourceware.org?subject=unsubscribe> List-Archive: <https://sourceware.org/pipermail/binutils/> List-Post: <mailto:binutils@sourceware.org> List-Help: <mailto:binutils-request@sourceware.org?subject=help> List-Subscribe: <https://sourceware.org/mailman/listinfo/binutils>, <mailto:binutils-request@sourceware.org?subject=subscribe> Errors-To: binutils-bounces+ouuuleilei=gmail.com@sourceware.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788974614527274920 X-GMAIL-MSGID: 1788974614527274920 |
Series |
[v2] binutils/windmc: Parse input correctly on big endian hosts
|
|
Checks
Context | Check | Description |
---|---|---|
snail/binutils-gdb-check | success | Github commit url |
Commit Message
Richard W.M. Jones
Jan. 24, 2024, 12:25 p.m. UTC
On big endian hosts (eg. s390x) the windmc tool fails to parse even
trivial files:
$ cat test.mc
;
$ ./binutils/windmc ./test.mc
In test.mc at line 1: parser: syntax error.
In test.mc at line 1: fatal: syntax error.
The tool starts by reading the input as Windows CP1252 and then
converting it internally into an array of UTF-16LE, which it then
processes as an array of unsigned short (typedef unichar).
There are lots of ways this is wrong, but in the specific case of big
endian machines the little endian pairs of bytes are byte-swapped.
For example, the ';' character in the input above is first converted
to UTF16-LE byte sequence { 0x3b, 0x00 }, which is then cast to
unsigned short. On a big endian machine the first unichar appears to
be 0x3b00. The lexer is unable to recognize this as the comment
character ((unichar)';') and so parsing fails.
The simple fix is to convert the input to UTF-16BE on big endian
machines (and do the reverse conversion when writing the output).
Fixes: https://sourceware.org/bugzilla/show_bug.cgi?id=31283
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
binutils/configure.ac | 2 ++
binutils/winduni.c | 16 ++++++++++++++--
2 files changed, 16 insertions(+), 2 deletions(-)
Comments
Hi Richard, > The simple fix is to convert the input to UTF-16BE on big endian > machines (and do the reverse conversion when writing the output). Approved - please apply (mainline and if you wish, the 2.42 branch as well). Cheers Nick
On Thu, Jan 25, 2024 at 03:18:23PM +0000, Nick Clifton wrote: > Hi Richard, > > >The simple fix is to convert the input to UTF-16BE on big endian > >machines (and do the reverse conversion when writing the output). > > Approved - please apply (mainline and if you wish, the 2.42 branch as well). Thanks - I guess I'm not allowed to push the patch though? Rich.
Hi Rich, >>> The simple fix is to convert the input to UTF-16BE on big endian >>> machines (and do the reverse conversion when writing the output). >> >> Approved - please apply (mainline and if you wish, the 2.42 branch as well). > > Thanks - I guess I'm not allowed to push the patch though? No you are allowed to push it. Sorry for the confusion. By "apply" I actually meant "commit and push". Cheers Nick
On Fri, Jan 26, 2024 at 10:08:32AM +0000, Nick Clifton wrote: > Hi Rich, > > >>>The simple fix is to convert the input to UTF-16BE on big endian > >>>machines (and do the reverse conversion when writing the output). > >> > >>Approved - please apply (mainline and if you wish, the 2.42 branch as well). > > > >Thanks - I guess I'm not allowed to push the patch though? > > No you are allowed to push it. > > Sorry for the confusion. By "apply" I actually meant "commit and push". Thanks. Note that someone will need to rerun autoconf for this change to actually have an effect. I didn't include that in the patch as the change is rather large and all in generated code. Rich.
Hi Rich. > Thanks. Note that someone will need to rerun autoconf for this change > to actually have an effect. I didn't include that in the patch as the > change is rather large and all in generated code. Thanks for doing that - having uncluttered patched to review is much appreciated. Rerunning autoconf got the 2.42 branch will not be a problem - I will be doing that as part of the release creation process which should be happening on Monday. Rerunning it on the mainline should also not be a big issue, as long as I notice your patch going in. If I forget, please could you ping me ? Cheers Nick
On Fri, Jan 26, 2024 at 11:51:07AM +0000, Nick Clifton wrote: > > Thanks. Note that someone will need to rerun autoconf for this change > > to actually have an effect. I didn't include that in the patch as the > > change is rather large and all in generated code. > > Thanks for doing that - having uncluttered patched to review is much appreciated. > > Rerunning autoconf on the 2.42 branch will not be a problem - I will be > doing that as part of the release creation process which should be happening > on Monday. Rerunning it on the mainline should also not be a big issue, > as long as I notice your patch going in. If I forget, please could you ping > me ? I pushed the patch for Richard.
diff --git a/binutils/configure.ac b/binutils/configure.ac index b03e36c9e0e..dac72c1bdd4 100644 --- a/binutils/configure.ac +++ b/binutils/configure.ac @@ -31,6 +31,8 @@ AC_PROG_CC AC_GNU_SOURCE AC_USE_SYSTEM_EXTENSIONS +AC_C_BIGENDIAN + LT_INIT ACX_LARGEFILE diff --git a/binutils/winduni.c b/binutils/winduni.c index 5b659764948..f19de4f8cb3 100644 --- a/binutils/winduni.c +++ b/binutils/winduni.c @@ -771,7 +771,13 @@ wind_MultiByteToWideChar (rc_uint_type cp, const char *mb, if (!mb || !iconv_name) return 0; - iconv_t cd = iconv_open ("UTF-16LE", iconv_name); + iconv_t cd = iconv_open ( +#if WORDS_BIGENDIAN + "UTF-16BE", +#else + "UTF-16LE", +#endif + iconv_name); while (1) { @@ -844,7 +850,13 @@ wind_WideCharToMultiByte (rc_uint_type cp, const unichar *u, char *mb, rc_uint_t if (!u || !iconv_name) return 0; - iconv_t cd = iconv_open (iconv_name, "UTF-16LE"); + iconv_t cd = iconv_open (iconv_name, +#if WORDS_BIGENDIAN + "UTF-16BE" +#else + "UTF-16LE" +#endif + ); while (1) {