Hello All!
Built a GoldED+ that holds text as UTF-8 internally. Nothing changes on
the wire - it writes to echoes in whatever XLATEXPORT says, as before.
What it buys you:
- messages in different charsets in one echo are each shown in their
own, all at once - CP437, CP850, CP866, KOI8-R, ISO-8859, UTF-8
- on unix and macOS no more luit and no more separate screen session in
a legacy charset - it runs in the native UTF-8 terminal as it is
- in Windows Terminal and conhost a fullwidth character (CJK) takes the
two cells it is due instead of being squeezed into one and shifting
the rest of the line
- charset comes from the system, nothing to configure
- conversion goes through iconv; on Windows through the codepage API,
on OS/2 through ULS. No more collecting .chs tables for the pairs you
need: these know them all. DOS included - the build in the archive
carries GNU libiconv, statically linked
The first point deserves spelling out, because it is the whole reason
for the exercise. Text inside GoldED+ used to be single-byte, so
everything came down to the terminal's charset. A message tagged
CHRS: CP437 or CP850 had nowhere to go on a CP866 screen - CP866 has no
umlauts and no accented Latin at all. You got question marks or garbage instead, and German, Polish and French echoes read like a lottery. Same
story on a KOI8-R terminal. Now the screen is UTF-8, which has room for
all of it: a German message in CP850, a Russian one in CP866 and a
Polish one in CP852 sit side by side in one echo and all three come out
as written.
CHRS: IBMPC is sorted out too. It is an old, vague identifier: first it
meant CP437, then it came to mean "the codepage of the machine the
message was written on". It used to work by accident - nobody
configured an IBMPC table, the bytes went through untouched, and under
a matching terminal that was the right answer. Take IBMPC literally as
CP437 and a Russian message falls apart. It is now resolved properly:
the session charset when that is single-byte, and on a UTF-8 session
the DOS codepage the locale implies. A ^ACODEPAGE: line next to it wins
over IBMPC, as FTS-5003 requires.
CHRS: ASCII is taken strictly as seven-bit ISO 646-1, per the standard.
If a message claims ASCII and carries eight-bit bytes anyway, those
become question marks - but that message is mislabelled by its sender,
and the old asc_* tables did exactly the same thing.
Recompile your nodelist. The index held a name in 36 bytes, which was
36 characters while a character was a byte. In UTF-8 that is seventeen
Cyrillic letters, so a name like "ǽѬßá¡ñα òα¿ßΓ«Σ«α«ó" - 41 bytes -
no longer fits and got cut in the middle. The field is 80 now. That
changes the record format, GoldED+ will not touch an old .gxn - run
goldnode from the same archive.
Two new keywords, that is all:
XLATCONFIGSET - the charset GoldED+'s own files are written in:
golded.cfg, the language file, templates, tagline
files, the help file
XLATAREASET - the charset of area descriptions in your tosser's
area file. Unset, it follows XLATCONFIGSET
Both belong at the very top of the configuration, ahead of any line
whose value is not plain ASCII - values are converted as each line is
read. The usual case is a UTF-8 terminal with an older config and
language file:
XLATCONFIGSET CP866
There is also a GOLDED_CONSOLE environment variable (Windows only):
cells or stream, for when GoldED+ guesses wrong about how your console
draws a fullwidth character.
And if you do want to write in UTF-8, there are echoes for exactly
that - UTF-8 and UTF8.FTN.MESSAGING. Ask your uplink for them and
write in any language you like. I keep a group for them:
GROUP UTF-8;
MEMBER UTF-8, UTF8.FTN.MESSAGING
XLATIMPORT UTF-8
XLATEXPORT UTF-8
ENDGROUP
Everything else stays 8-bit as it was.
Built - fifteen of them, DOS, OS/2, Windows (MSVC6 included), Linux,
macOS, Solaris, Haiku:
https://github.com/evs38/golded-plus/releases
If you build your own:
git clone
https://github.com/evs38/golded-plus.git
unicode is the default branch there. Builds as usual - cmake, or
make PLATFORM=lnx; see INSTALL and docs/building.txt.
On unix, curses and iconv are required.
Curses deserves a word: this is a transition period and it is easy to
get wrong. Wide-character support used to live in a separate library,
ncursesw, and in many systems that library no longer exists as such -
since ncurses 6 the wide calls sit in the ordinary ncurses, and
ncursesw is either a compatibility symlink to it or gone entirely.
pkgsrc, for one, has no separate ncursesw any more, only ncurses - and
it is the wide one. On older systems, though, they are still two
different packages, and there the wide one is exactly the one with the
"w".
The rule is simple: if your system has ncursesw, install that. If it
does not, the wide calls are already in the plain ncurses - install
that. The build then tells you what it found and whether it is wide.
iconv is part of libc on Linux; on the BSDs and macOS it is usually a
separate libiconv from packages.
On Windows neither is needed: the screen goes through its own console
layer and conversion through the codepage API. On OS/2 it is VIO and
ULS, both system. On DOS you need libiconv for djgpp if you want to
build with ICONV=1.
New build switches (make form below; cmake takes the same through -D,
with a GOLD_ prefix on the last two):
GOLD_UTF8=0 hold text single-byte, as before. Default is 1
everywhere except DOS
WIDE_NCURSES=0 the old 8-bit curses API instead of the wide one
EXTERNAL_CURSES=0 use only the system's curses, not a packaged one
EXTERNAL_ICONV=0 the same for iconv
By default on unix both come from the package manager (/usr/pkg,
/usr/local, /opt/homebrew, /opt/local, /opt/csw) with the system's own
behind them: a packaged ncurses is usually newer and wide-capable, and
a packaged libiconv knows more charsets.
On OS/2 conversion goes through ULS, the system's Unicode API. It is
picked up by itself when the toolkit headers are found; without them
you get a build on the tables. The DOS build in the archive is made
with ICONV=1 and libiconv for djgpp linked statically, so the .chs
tables are not needed there either; built without ICONV=1, DOS stays on
the tables as before.
There are bound to be plenty of bugs: GoldED+ is a huge construction
set grown over decades and nobody quite remembers all of its features
any more :) and almost everything that puts text on the screen was
touched here. So find bugs and write, I go through all of it.
* Originally in GOLDED
* Crossposted in UTF-8
* Crossposted in UTF8.FTN.MESSAGING
* Crossposted in FIDO_UTIL
Eugene
... It's full of stars!
--- GoldED+/BSD 1.1.5-b20260828 (NetBSD 11.0 Intel Core Haswell)
* Origin: FireFox Station (2:5075/35)