Going the extra mile for Windows XP

April 9, 2015 Zydeo Software

I’m just about ready to tag the Github repository with release-1-0, build an installer and finally, officially, really usher Zydeo 1.0 out the door. But while I’m waiting for the digital certificate (the very first I’ve ever owned) that I need to sign the installer, I’ve decided to do some last-minute dusting and testing. Caution: there be dragons in Windows XP’s cupboard.
TL;DR: Zydeo will run on Windows XP, and look nearly as good as on later versions.

What’s special about XP?

The most special thing about it is that it’s officially no longer supported by Microsoft. Sure, if you have XP installed, your PC still functions, but there are no more Windows updates, no security patches; as new hardware comes out, there will be no drivers for it.

Second most special: XP has just about always been a pain to work with, particularly if you need Chinese or other East-Asian scripts. Much of that has to do with fonts; the rest of this post is about the details.

Theoretically, it should work.

When you start creating a new piece of software and make decisions about the technology stack, one of the questions to weigh is, how far back do I want to go in terms of supported environments? It’s tempting to pick newer technologies because they often make life for you as a developer easier, and may enable you to create a better tool with equal or less effort. The other side of the tradeoff is, newer technologies will not run on arbitrarily old systems.

Above everything else, my goal with Zydeo was to experiment with user interfaces. I chose a fairly non-standard approach to do that: a windowless UI with all painting done in a worker thread. To make sure I can spend the most time working on the UI aspect, I decided to stay completely within my comfort zone with the programming language: C#.

As a consequence, the main choice with Zydeo was the .NET framework version. Whatever has been added to the language since .NET 2.0 is pretty irrelevant to my work on Zydeo; some new constructs may make life marginally simpler, but they have no bearing at all on my real challenges: text rendering and fast custom typesetting. I chose .NET 4.0 for a combination of reasons:

• It’s the highest version still available on XP;
• In a very Microsoft-y way it’s equivalent to 4.5: if your program runs on 4.0, it also runs on 4.5;
• Windows 8 has 4.5 built in by default;
• Enabling a lower version, e.g., .NET 3.5, is a tricky process that your users must do on Windows 8.

I also went to a great length of trouble to make sure Zydeo ships with two free Kaiti fonts as a fallback in case DFKai-SB and KaiTi are missing in the system, as they inevitably are in XP.

So after all this groundwork, I expected Zydeo to just work on XP after a seamless installation. Here’s what I got instead.

Facepalm 1: Segoe UI

It took me less than a minute to find the reason: in Zydeo I rely on the Segoe UI font all over the place. It’s a beautiful, and beautifully balanced, typeface with the extra benefit that it hasn’t been ruined by excessive use like Times New Roman.

Unfortunately, Segoe UI is not available pre-Vista. This one’s on me.

OK, that’s a relatively easy one to fix. Google’s Noto Sans is nearly as visually appealing, and I’ve already experimented with it for the English in dictionary entries at an earlier stage. So I go in and cook up code that supplies Segoe UI throughout the user interface if it’s available, and substitutes Noto Sans from a private font collection if it’s not. Then, voilà…

Facepalm 2: font linking

You see I rely on an under-the-hood behavior of Windows in the search area. For the input field as well as the two buttons Zydeo specifies a particular font (Segoe UI or Noto Sans), knowing full well that this font does not contain Chinese ideographs. The system drawing routine (GDI+, to be specific) recognizes this on the fly and picks a different font instead. In this case, it picks the wrong font for half of the characters. I would even be willing to live with that if it picked the same font for all characters, but no; for some you get Heiti, for some others, Songti. And you have no control over this at all.

This is called font linking, and essentially it’s a good thing: it makes sure you get every character on the screen, instead of tofus (empty squares) for those symbols that are missing from a particular font. The problem is that the process of picking characters is not really documented anywhere, and that you have no control over it. The closest I got to a description of what’s going on is this post by Michael S. Kaplan from 2005.

The problem is, there’s no way to persuade GDI+ to substitute all Chinese characters using the same font. It follows some half-assed “logic” to decide if a character is simplified or traditional, and chooses a font from a different family for them. That’s not the kind of thing I’m willing to put up with in Zydeo.

Attempt 1: Force SimHei

OK, so let’s give up on the whole shaky trick of relying on GDI+’s built-in font linking and force the single Hei font that’s available in XP, SimHei, onto those buttons and the search input field.

Yup. The page on Pinyin Joe that’s a such a wonderful source of information about Chinese fonts in Windows is apparently mistaken about SimHei being proportional. By all appearances SimHei is fixed-width for Latin characters; see this improvised test from MS Word, with the same text in 11pt Calibri, 11pt SimHei and 10pt Consolas – that’s as mono as “spaced” ever gets:

Update: On a whim I mailed Joe about the error. Within a matter of hours he updated the page, which now says monospaced.

On to…

Attempt 2: ship Noto Sans CJK

I really wanted to avoid bloating Zydeo’s installer even more – at this stage it already clocked in at around 18MB because of the two free Kai fonts that I include as a fallback, Arphic and HDZB_75. But there seemed to be no other option; if I want a pretty Hei font that also does Latin characters well, Noto is the only choice I know of.

So I forged ahead, and while I was at it, I also changed the installer so that it doesn’t include either the Noto files or the two Kai fonts but downloads them during the installation if (and only if) they are needed. Of course, there’s the stupid vertical alignment of the characters in Noto Sans CJK, so this meant more than just changing the font to use; I had to add custom offsets to make it look good.

I did this on my normal Windows 8 box, only to end up banging head against wall when I double-checked the result on XP an hour later.

Facepalm 3: GDI+ and OpenType

The reward of all this sweating was yet another one of these messages as soon as I started Zydeo:

You see there are two ways to create a System.Drawing.Font object: the first works with installed fonts, the second uses a private font collection.

Now, there is a subtle difference between the two methods if GDI+ cannot deal with the specific font you are using. If you install the Noto Sans CJK font, it will show up in some programs (like Word or Notepad), but in pre-Win8 systems, you will not be able to use it to create a System.Drawing.Font object. The first method will return a Font object, but it will not be what you requested; instead, you’ll be getting Microsoft Sans Serif.

The second method will barf at AddFontFile. How exactly? With a FileNotFoundException. Yup, you’re reading that right. Face. Palm.

Detour: GDI+ and OpenType

Now this section is not strictly relevant for how I went on with Zydeo, but I need the space for some ranting.

There are two standard ways to draw text using Windows functions. One is GDI, which is what happens if you use Windows.Forms.TextRenderer.DrawText in C#. The other is GDI+, which is what happens if you use System.Drawing.Graphics.DrawString. In effect, everything in System.Drawing is a wrapper around GDI+.

While the plus in the name suggests that GDI+ is the more advanced of the two, that’s not so obvious. As far as text rendering goes, GDI+ seems to be a building abandoned when it was just about halfway built. If you have nothing better to do, read up on the differences on The Art of Dev, here and here. In nearly all respects GDI+ appears to be inferior, but it can do one thing that GDI cannot that’s crucial in Zydeo: write text on top of an image or other non-uniform background.

Another thing GDI+ cannot cope with on pre-Win8 systems is OpenType fonts. Not like Microsoft tells you up front. This little detail is hidden in the same MSDN page that explains about the FileNotFound exception:

Windows Forms applications support TrueType fonts and have limited support for OpenType fonts. If you try to use a font that is not supported, such as an unsupported OpenType font or a Bitmap font, an exception will occur.

As it happens, Noto Sans CJK is an OpenType font. As it also happens, it is an OpenType font that no online font converter will be able to transform into a TrueType font for you. Even FontForge will either report errors or save a corrupt TTF file.

The truly frustrating part is that if you install Noto Sans CJK on Windows XP, the font shows up in Word and even Notepad. You just cannot use it in your own code, unless you are ready to switch from System.Drawing (GDI+) to TextRenderer (GDI). But that’s not an option for a different reason: it won’t paint over a gradient background, such as these buttons have:

Additionally, it’s only System.Drawing that gives you GDI+; the rest of Windows.Forms wraps up native Windows controls, which are all GDI and happy with OTF fonts for their part. But if you have a TextBox, which wraps a native edit control that can deal with OTF, using the TextBox class there’s no way to do that because the exposed property is a System.Drawing.Font, which will never, ever represent an OTF font… Face. Palm.

Bottom line

This post is already waaaay too long. What’s the deal in the end with Windows XP?

• When you run the installer on XP, it will download the Noto Sans fonts (only the Latin TTFs, not the large CJK OTF).
• Whatever is Segoe UI in later Windows versions shows up with Noto Sans on XP. That’s all the labels, and also the English in dictionary entries.
• Zydeo specifies “Microsoft Sans Serif” for the input field and the two buttons with the Chinese labels. Your mileage may vary, but this looks kinda OK for me on XP; better than it would on later Windows versions.
• At least with the on-demand font download I reduced the installer to 8MB from 18MB; that’s a benefit for everyone.

Is there anyone (I mean, anyone) out there that will use Zydeo on XP? I can’t wait to see.