Baobaobaolin.com
date
entry
046
topic
tooling
rev
—

In macOS's bash 3.2, "$n個檔案" doesn't print 3: the Chinese character after a variable loses a byte

echo "上傳了 $n個檔案" ("uploaded $n files") prints 上傳了 ��檔案 in macOS's bash, and with set -u the script just stops. bash 3.2 decides where a variable name ends one byte at a time, and the first byte of 個 counts as a letter in macOS's UTF-8 locale.

In shell scripts written in Chinese, it's natural to put a variable right against the text, as in echo "上傳了 $n個檔案" ("uploaded $n files", where 個 is a measure word). I'd assumed this was fine: an unbraced variable name follows ASCII rules (a letter or underscore, then letters, digits, underscores), 個 isn't among them, so bash stops after n.

But on this Mac that line prints 上傳了 ��檔案, and the 3 is gone. This note is about why macOS's built-in bash 3.2 pulls Chinese characters into variable names. In order: reproducing it, the reason in bash's source, how often it shows up in this machine's logs, why it's hard to notice, and how to write it.

Reproducing it

The environment is macOS's built-in /bin/bash, GNU bash, version 3.2.57(1)-release (arm64-apple-darwin25), with LANG=zh_TW.UTF-8 and no LC_ALL or LC_CTYPE set. This is the only bash on my PATH, so #!/usr/bin/env bash finds the same one, and /bin/sh selects /bin/bash via /private/var/select/sh.

$ cat b.sh
#!/usr/bin/env bash
n=3
echo "上傳了 $n個檔案"
echo "上傳了 ${n}個檔案"
echo "上傳了 $n 個檔案"
$ ./b.sh
上傳了 ��檔案
上傳了 3個檔案
上傳了 3 個檔案

Looking at the first line with xxd: after 了 and the space come the bytes 80 8b, then 檔案. 個 in UTF-8 is e5 80 8b: the first byte e5 and the digit 3 are both gone, and the two leftover bytes don't form a character, so the terminal shows two �.

With set -u, using an unset variable in a direct expansion with no default like this one is an error, and a non-interactive script exits. The error message shows what bash read as the variable name (the � is really the byte e5):

./a.sh: line 4: n�: unbound variable

This a.sh exits 1, and the next line doesn't run. I tried more characters: ), ,, 。, 、, 「, 的, 秒 and é all lose their first byte; 語 (e8 aa 9e) loses two; 媺 (e5 aa ba) loses all three, leaving nothing; ASCII : is fine. Running the same script with LC_ALL=C or under env -i (empty environment, so the locale falls back to C), all three lines are correct.

The cause: asking isalnum() one byte at a time

In Apple's published bash source (the bash-3.2 directory of apple-oss-distributions/bash on GitHub, patchlevel.h at 57), param_expand() in subst.c finds the end of a variable name like this:

for (t_index = zindex; (c = string[zindex]) && legal_variable_char (c); zindex++)
  ;

c is declared unsigned char: one byte at a time. general.h defines legal_variable_char(c) as ISALNUM(c) || c == '_', and include/chartypes.h defines ISALNUM(c) as IN_CTYPE_DOMAIN (c) && isalnum (c), where IN_CTYPE_DOMAIN is 1 under conditions such as STDC_HEADERS being defined. So in the end the C library's isalnum() judges each byte separately.

The trouble is how macOS's isalnum() classifies those bytes. A small program that switches locale with setlocale and calls it on every value from 0x80 to 0xFF: in the C locale, none are true; in both zh_TW.UTF-8 and en_US.UTF-8, 65 are true, namely everything from 0xC0 to 0xFF except 0xD7 and 0xF7, plus 0xAA, 0xB5 and 0xBA. Those 65 values are exactly the Latin-1 letters À to ÿ (minus × and ÷), plus ª, µ and º. isalnum() was never meant to decode UTF-8, so the problem is the combination: bash judging byte by byte, and this classification.

A common Chinese character in UTF-8 is three bytes (supplementary-plane characters are four): the first in 0xE0–0xEF, the rest in 0x80–0xBF. isalnum() calls the first byte a letter, so bash adds it to the variable name; the second byte usually isn't, and the name ends there. A variable called n plus byte e5 doesn't exist, so it expands to an empty string, and the remaining 80 8b are printed as-is. The second byte of 語 happens to be 0xAA, so it loses two; all three bytes of 媺 are among the 65, so the whole character disappears.

Whether newer bash versions still do this, I have no version at hand to test; this note covers only the one built into macOS.

How often it shows up in this machine's logs

Using the system grep, I searched this machine's Claude Code conversation logs (JSONL under ~/.claude/projects) for errors of the form "an ASCII variable name, then non-ASCII bytes, then : unbound variable". Counting only tool output from before October 15, it found four, in four log files last modified between September 23 and October 15. The misread variables were FILE, PATH, GH_ROLE and UP, with messages shaped like:

line 61: FILE�: unbound variable

Today added one more: writing the screenshot wrapper in 045, my error message was "沒有截圖(chrome 結束碼 $rc)" ("no screenshot (chrome exit code $rc)", with a full-width parenthesis), and I got rc�: unbound variable. All five showed up as unbound-variable errors, meaning the scripts had set -u on. In scripts without it, the value quietly becomes an empty string, maybe with a few garbage bytes, maybe with nothing left at all; whether that has happened, searching for this error message can't tell me.

Why it's hard to notice

First, the shell you test in isn't the shell that runs the script, and the two break differently. In the Claude Code sessions I use, the Bash tool actually runs zsh (see the tool called Bash runs zsh). In zsh 5.9, echo "結束碼 $rc)" prints correctly; the same line in a #!/usr/bin/env bash script is what breaks. But zsh isn't safe with Chinese either: echo "$n個檔案" in a UTF-8 locale prints an empty string, and with nounset it's n個檔案: parameter not set. zsh's documentation says that with POSIX_IDENTIFIERS unset and multibyte support enabled, any alphanumeric characters in the local character set may be used in identifiers, so it takes 個檔案 as part of the name; full-width punctuation isn't alphanumeric, so $rc) is fine. The result: full-width punctuation works in zsh and breaks in bash 3.2; Chinese characters break in both, differently.

Second, the locale. The same script works in the C locale, so if the effective LC_CTYPE of a scheduler or CI is C, it may keep working there and break only when run by hand in a terminal. That's an inference from the env -i result above; I haven't checked the locale of every scheduled job on this machine.

Third, static checks don't report it by default. ShellCheck 0.11.0's default checks give no warning, exit 0, on b.sh above and on another script with "$n)" and "$n。". Only with --enable=require-variable-braces does it report SC2250 for every unbraced variable (a style suggestion, not aimed at this problem), including harmless ones like "$n 個檔案".

How to write it

When a variable is immediately followed by a non-ASCII character, always use braces: ${n}個檔案, (結束碼 ${rc}). That works in both bash 3.2 and zsh. If you'd rather not remember the rule, write every variable embedded in text as ${…}. printf '%s個檔案\n' "$n" also works, since the variable no longer touches the text. Running with LC_ALL=C avoids this particular bash problem too, but changes sorting, character classes and other locale-dependent behavior, so I don't.

To find this pattern in existing scripts, scan bytes with perl (*.sh covers only the current directory):

perl -ne 'print "$ARGV:$.: $_" if /\$[A-Za-z_][A-Za-z0-9_]*[\x80-\xff]/; close ARGV if eof' *.sh

I ran it over the 15 .sh files in four website repos on this machine; no hits. The pattern skips $1個, which is fine: an unbraced positional parameter takes only one digit and isn't affected. It will also flag strings in single quotes or comments that never expand, so hits need a human look.

If you remember one thing

macOS's built-in bash 3.2, in a UTF-8 locale, takes the leading byte of a Chinese character or full-width punctuation right after $n as part of the variable name. Without set -u it prints the wrong thing quietly; with it, the script stops; ShellCheck won't warn you by default, and zsh breaks on Chinese characters too, just differently. When a variable is followed by a non-ASCII character, write ${n}.

Revision history

  1. First published