Russian (original and up to date) version here

Mirror (Newmail)

Mirror (ROL)

Last update: 23-apr-2001. Konstantin Kazarnovsky

Forum/Guestbook

• Specifications and sources
• Character Sets, Fonts and FontSubstitutes
  • Character Sets diskussion
  • .FON-fonts
  • Font installation and character sets
  • FontSubstitutes diskussion
  • OEM-charset is a special case
  • Font name language suffixes
• Tables of the TrueType font file and their role in Windows
  • Cmap table
  • Name table
  • OS/2 table
  • PANOSE table
  • Recognition of character sets in the font
• Unicode and Postscript-fonts

• TrueType 1.0 rev. 1.66 Specification. The main document in a format Word 6/'95, containing the last version of the specification TrueType from Microsoft (at least for Microsofts products).
• OpenType 1.2 Specification. Developed jointly Adobe and Microsoft OpenType Specification in HTML-format HTML (attempt of the uniform approach to PostScript and TrueType fonts). Includes the extension TrueType - TrueType Open, accessible also as the separate specification:
• TrueType Open rev. 1.0 Specification. The document in a format Word 6/'95.
  
  
• Unicode Inc. The server of the Unicode consortium, holder of the of the same name trademark and developer of the of the same name standard of the general coding (curent version 3.0).
• Code Chart and Names List. Compact (as far as it is possible) variant of Unicode character charts on the same server, recently completed under the new version Unicode.
• Microsoft Typography. Here is a lot of info about fonts and typography, including significant part of references on this page.

• Character Sets.
  The concept of a character set of majority (but not in all) cases is identical to much more clear concept of a Codepage. Unicode-definitions of codepages, that supported in (international-european) Windows, are contained in resources GDI.EXE (for the 16-bit programs and for work with fonts through scripts), and also (for 32-bit Unicode-programs) in files of National Language Support files (NLS), about the role NLS and their structure see notes concerning MSIE 4. The table with the review of known character sets is indicated below.
  The codes of character sets (1-st and 2 table columns) are not a part neither standard Unicode, nor specifications TrueType, they represent only program-internal labels, defined in header files of the programming languages.
  Is seems in all probability that all codes of character sets have a mnemonic sense:
  

  CC	CyrilliC
  EE	Eastern European
  BA	BAltic
  A1, A2, A3	Additional 1, 2, 3
  B1, B2	Bidirectional 1, 2
  4D	character code of letter M(acintosh)
  DE	character code of letter Þ (THorn) - from THai

  
  The unique direct reference to codes of character sets I have found only in The specifications RTF in the description of the control word \fcharsetN:
  Specifies the character set of a font in the font table. Values for N are defined by Windows header files, and in the file RTFDEFS.H accompanying this document.
  In the file of the specification anything "accompanying" was not, however simbolical significances of these codes are determined, in particular, in the file WINGDI.H from a package Borland C ++ 5, see it's appropriate fragment wingdi_h.txt, including also simbolical label of a number of other performances of the font.
  In the last two table columns is indicated, whether there is in resources GDI.EXE Unicode table, defining given character set (given code page). The special case (in the different relations) represents charset Symbol, defining it the table is a continuous section of codes 0xF000-0xF0FF, by an obvious image it is not present at resources.
  Zero ("0") designates character sets, for which label and the code is present, but Unicode table is not present (it is necessary to consider, that for hieroglyphic languages the two-byte coding is always used and the concept of a code page loses a sense, if I here am not mistaken).
  

  Charsets, or character sets

  Code (hex)	Code (dec)	Name	Code Page	Font suffix	Win 3.1	Win 9x
  0x00	000	Western (Latin 1)	CP1252		+	+
  0x01	001	Unknown			0	0
  0x02	002	Symbol			+	+
  0x4D	077	Mac			- (?)	0
  0x80	128	Japanese			-	0
  0x81	129	Hangeul			-	0
  0x82	130	Hangeul (Johab)			-	0
  0x86	134	Chinese_GB2312			-	0
  0x88	136	Chinese_BIG5			-	0
  0xA1	161	Greek	CP1253	Greek	-	+
  0xA2	162	Turkish (Latin 5)	CP1254	Tur	+	+
  0xA3	163	Vietnamese	CP1258	Viet	-	-
  0xB1	177	Hebrew	CP1255	(Hebrew)	-	+
  0xB2	178	Arabic	CP1256	(Arabic)	-	+
  0xBA	186	Baltic	CP1257	Baltic	+	+
  0xCC	204	Cyrillic	CP1251	Cyr	+	+
  0xDE	222	Thai	CP874	Thai	-	-
  0xEE	238	Central (Eastern) European, (Latin 2)	CP1250	CE	+	+
  0xFF	255	OEM/DOS			0	0

   
   
• FON-Fonts.
  The fonts with such extension exist of two kinds - raster (bitmapped) and vectorial (scaled). The format of vectorial fonts is not known to me, them does not understand Borland Resource Workshop 4.5, in Windows 3.1* they are marked as fonts of a plotter and "all resolutions", and Windows'9x limits, similar, only last of these marks.
  Raster fonts with .fon extension actually (as well as writes Resource Workshop) represent library from several fonts for a number of sizes, for example, "8,10,12,14,18,24", and these numbers - some conditional significance describing a font size in items (the fixedwidth font Courier of a size 10 consists of signs by a size 8*13).
  Font header contain 2 fields of characteristics: Family (pale shadow from PANOSE characteristics in TrueType) and Char.Set.
  
  

  Values of a Family field (following Borland Resource Workshop)

  Value	Family Name	Description
  0		Not the alphabetic font
  1	Roman	The proportional font with serifs
  2	Swiss	The proportional sans-serif font
  3	Modern	The fixedwidth font
  4	Script	The italic or hand-written font
  5	Decorative	The decorative font

  
  The field Char.Set contains a code of a character set in the correspondence with the above-stated table of character sets. The fonts in Windows 3.1* contain in it a field 0, 2 or 255 (latter for fonts Terminal), irrespective of an actual code page of the font. In Windows ' 95 in it the field is underlined an actual code page, so that for compatibility with the programs under 3.1 it is required to insert appropriate lines (System,0=System,204 etc.) in WIN.INI into section FontSubstitutes.
  Generally speaking, the separate fonts of the file .FON can be marked as different character sets, but what will be Windows' behavior in this case, it is unknown.
  
  
• Installation of fonts and character sets.
  While installation of the font Windows determine it various characteristicss, in particular, character sets associated with the font (charsets). In Windows 3.* with TrueType font (and, most likely also with raster font .FON) the unique code of a character set (more often - 0, or western) is associated. In Windows'95 with TrueType font a number of character sets codes, or scripts, can be associated (more often among them is present 0, or western). The algorithm of such associations see lower. The result of font installation is named as the physical font.
  
  
• FontSubstitutes.
  Section in WIN.INI, appeared in Windows 3.1 for assuring compatibility with standard fonts in Windows 3.0 and in the complicated kind passed on into Windows'95 and (as section of the registry) in Windows NT. Each line in it looks like this
  FontLog=FontPhys Or (in Windows'9x)
  FontLog, CharsetLog=FontPhys, CharsetPhys  .
  Here at the left there is a name of the logical font, and on the right - name of the physical font;
  CharsetLog and CharsetPhys - codes of character sets in the correspondence with the table of character sets.
  Despite of apparent intuitive clearness of font substitution, the details of its algorithm are rather obscure and at least documented not fully. The most in-depth article was found in MSDN section of a site Microsoft: Windows Font Mapping (Ron Gery, June 8, 1992). (Though the article is old, but the reference to it is included just now).
  Recently has appeared a program (beta version)for Windows'9x/NT for convenient manipulations of items in [FontSubstitutes], so called "WGL Assistant". See sectionConverters of the font encoding and other tools.
  From some portion of an approximation, all works as follows:
  In Windows 3.1* are not specified (and do not work) the codes of character sets, but - by all - code 0 everywhere is meant, anyway it was impossible to me in international Windows 3.1*, where the delivered fonts with the suffix Cyr have a character set code 204, to force to work lines of a type
  Arial=Arial Cyr.
  In remaining part the general scheme can be extended on Windows 3.1*.
  
  1. If in a system there is a physical font with the same characteristics, as the logical font, indicated in a line (FontLog, CharsetLog), a line of a font substitution does not work.
     If the same logical font occurs in section FontSubstitutes twice and more, second and further entry also do not work.
  2. If in a system there is no physical font with the characterisitcs, indicated in a line (FontPhys, CharsetPhys), the substitution on the font on general substitution rules of the font (fontmapping) is made, when the required font in a system is absent. The weight factors of "similarity" are used, from which greatest significance, and exceeding all remaining taken together, is attributed just to code of a character set.
  3. If the indicated logical font occurs for the first time and in a system there is an indicated physical font, then is crated logical font FontLog, CharsetLog so, that for it's request in a program, this program receives the physical font, indicated in a line. Generally not only the name of the given font will not coincide with requested, but will not coincide and charset.
  4. If for the given physical font with the given CharsetPhys for the first time appears a logic font, referring on it, (seems that only with CharsetLog=CharsetPhys), the logic font occurs also in font enumeration of 16-bit programs and such program, as character map. Such way in Windows'95 ensures compatibility on national fonts with Windows 3.1*. So standard fonts
     Arial Cyr, Courier New Cyr etc. (and also Arial Creek etc.) - as the logic fonts - is created in Windows'95 by lines of a type:
     Arial Cyr,204=Arial,204
     Arial Greek,161=Arial,161 .
     Such fonts take an intermediate position between physical and pure logical, sometimes them name emulated, I shall use more likely for them a label "fonts-sections of the multilingual font".
     
  5. Lines of the following kind do not work, though could work:
     FontLog, CharsetLog=FontPhys, 0
     If CharsetLog is not equal 0 and
     FontLog,0=FontPhys,255  .
     The latter could be applied to use of the font in DOS-encoding, that do Windows itself in DOS-window, but - alas - this possibility does not work, however, more or less in the accordance with the decribed scheme, since during installation of the font (distinct from raster Terminal) the character set with a code 255 (script OEM/DOS) never occurs in the list as allowable.
     
     
• The suffixes in a title of the font.
  The standardized suffixes, listed in table character sets, of the national versions of fonts (not being like mandatory) execute nevertheless 2 functions:
  
  • One function obvious enough - so is ensured compatibility with the programs using standard names of national fonts, first of all it concerns to fonts such as Arial Cyr, Courier New Cyr and Times New Roman Cyr.
    
  • Other function not only is not obvious, but also undocumented: Word-95 during switching of the keyboard (and by that language) is automatically switched to the font of the same title appropriate to selected language. And the correspondence it is determined just sewed rigidly up in Word-95 by the allowable suffixes of the national versions of fonts: <none>, Cyr, CE, Baltic, Greek and Tur. It is coordinated to the set of languages supported in Word-95, which paneuropean version (including Russian) don't support Hebrew and Arabic, despite of support of appropriate code pages by the module GDI.
    It is interesting, that such switching works fully for the "native" switcher, but not for all alternate programs of switching - 2 of 4 tested programs called switching the font only on the second entered character, and the first at once entered character was appeared in the incorrect font. Probably such behaviour is caused whether an error, whether an not documented function.
    
    

• cmap table. This table determines encodings of the font and sets the correspondence between characters codes and their images (glyphs), is more point by indexes glyphs, which order is arbitrary.
  Generally speaking, the table cmap almost always consists of the several subtables, which determine the various encodings of the font. Each subtable is characterized by two numerical identifiers Platform ID (identifier of a platform) and Encoding ID (type of the encoding). Usually identifier of the table is noted as m.n. All really met TrueType in Windows, intended for european and middle East languages (about Chinese and that similar languages anything tell I can not), for a platform Microsoft contained equally one from the following two codings, which only and are recommended Microsoft for further use:
  

  Platform	The coding	Definition
  3		Microsoft
  	3.1	Microsoft UGL (Unicode Glyph List) - standard coding of text fonts in Windows
  	3.0	Microsoft unspecified (Symbol) encoding - standard encoding for non-text fonts in Windows.

  
  All known for me variants of identifiers are in the separate table of the codings TrueType, where the additional codings of a platform Microsoft are intended, most likely, for hieroglyphic languages (however, now there is also Unicode encoding of hieroglyphs).
  
  4 formats of the subtables cmap are described in the specification TrueType (Format 0, 2, 4 and 6), for want of it in the subtables for Windows (identifiers 3.0 and 3.1 are circumscribed, not confuse to the versions Windows:-) one should to be used only from these formats - Format 4. It is arranged rather smartly, but is logically equivalent simply to list allowable (two-byte) Unicodes and list of appropriate glyph indexes. The index 0 is intended for an image of an absent character (if those will be requested), in majority of fonts as an absent character the empty small square is used.
  The encoding 3.1 - Microsoft UGL is intended for majority (text) fonts in Windows, and the codes of characters in the cmap table, generally speaking, should correspond to the standard UNICODE. It is necessary to mark, that the specification does not limit neither number of characters in the font, nor the number of characters covered by the cmap table, bye maximal size of the single-byte encoding - by number 256 (and it is really 256-32=224), and in Windows'95 the large multilingual fonts with number of glyphs about 650 have appeared (and these glyphs are accessible for use through their Unicodes).
  On the other hand, the problems with so-called "non-Unicode" russian fonts are caused that Unicodes in these fonts appropriate to glyphs, representing the Russian letter, correspond for want of it to a latin code page CP1252, instead of cyrillic CP1251, though graphically to it and correspond. An apart from probablis semicompetence, the reason of emerging of such fonts was also impossibility of use in the USA-Canada versions Windows 3.1/3.11 correct cyrillic fonts - module GDI there supported a sole code page - CP1252. Besides down until appearance of strict Unicode systems or programs, such as Windows NT or Office - 97, the use of such fonts did not call any inconveniences.
  
  The encoding 3.0 - Microsoft Symbol Encoding is intended for non text fonts containing various design elements. Formally subtable cmap in such fonts also contains Unicodes of characters, but they are in so-called Unicode user area (Private Use Area), and really - in area U+F020 up to U+F0FF, being mapped by a natural image in single-byte codes 0x20... 0xFF.
  Above stated reason is invented at all not by me, but now I can not find a source in any way, where the format of the font in Symbol Encoding would be defined concretely. Similar, that it not the specification TrueType.
  Correct (or nearly so correct) the support of such fonts is carried out also for Unicode programs. So that in the certain period they could be considered most problemlos and the average generation of Russian fonts from Paragraph was such. However, the programs permitting to select for any purposes only text fonts, do not show fonts such as Symbol in the window of choice, and the Office-97 fails hyphenations and does not check up spelling for collected by them of the text.
  
  
• The table name. Table of names is contained a title of the font, family in which it enters, variation, Copyright and Trademark etc. (only 8 data types). These datas are specified for each encoding m.n of the font and are marked also by identifier of language Language ID, and languages can be a number for one coding. Lines of datas can be in the one-byte coding (for Macintosh, m=1 and ISO, m=2) and in UNICODE (for Apple Unicode, m=0 and Microsoft, m=3). From all these datas the title of the font in English is used only, and the other datas do not influence the interpretation of the font.
  
  
• The table OS/2. This table of font characteristics is contained 2 subtables, which relate to the encodings and can influence the interpretation of the font in a system.
  Within the specification TrueType 1.0 tables OS/2 have evolution from the version 0 to the version 1, and just at the expense of adding a mask of allowable code pages.
  The function Windows 95 font properties extension for the font with the table OS/2 of the version 0 in item Charset/Unicode in both windows (Supported Unicode Ranges and Supported Code Pages) writes: "Information not available in font".
  • version - 16-bit word containing a version number:
    0 (78 bytes) - up to TrueType v.1.0 rev.1.5
    1 (86 bytes) - up to TrueType v1.0 rev. 1.66
    2 (96 bytes) - under the specification OpenType v.1.*.
    (The fonts with the table OS/2 ver.2 yet did not come across to me.)
    
  • ulUnicodeRange - four 32-bit words containing a bit mask functional Unicode ranges in the font. A sense of this "functionality" Microsoft has appeared be unable to explain even in the specification TrueType. Besides this subtable, as well as specification as a whole, is guided by the standard Unicode of the version 1, whereas acting is the version 2. In this decoding of bits of the subtable ulUnicodeRange the attempt is made to arrange verbal labels from the specification TrueType and Unicode ranges from the standard Unicode 2.0, that gives some review of the standard Unicode generally.
    In the early specification TrueType, supposing the table OS/2 of the version 0, this subtable was present, but should be zero and was considered as a reserve for the future use.
    In multilingual fonts with a cyrillic Microsoft sets in this subtable bit 9 (0x00000200), though even in the full CP1251 there are representatives following Unicode ranges:
    

    Bit	Description	Unicode Range
    0	C0 Controls and Basic Latin	U+0000 to U+007F
    1	C1 Controls and Latin-1 Supplement	U+0080 to U+00FF
    9	Cyrillic	U+0400 to U+04FF
    35	Letterlike Symbols	U+2100 to U+214F

    
    However, this subtable nowhere is really used, though is deciphered during extended show of properties of the font.
    
    
  • ulCodePageRange - two 32-bit words containing a bit mask of code pages, covered by the font. The decoding of bits of the subtable ulCodePageRange is present, in particular, on a site at Microsoft.
    To cyrillic page CP1251 there corresponds(meets) bit 2 (0x00000004) in the first word ulCodePageRange.
    In the early specification TrueType, supposing the table OS/2 of the version 0, this subtable was absent.
    
    
• The table PANOSE.Developed by firm ElseWare (afterwards including in Hewlett-Packard) system of a classification of fonts realized as the 10-byte table and intended for definition of "similarity" of fonts (under the specification TrueType - only for not latin character sets). Each from bytes contains some value (not a bit mask). The following 2 field in the table are most important:
  • panose.bFamilyType - type of the font, it applicability, in the correspondence with the table:
    

    Value	Description
    0	Any
    1	No Fit
    2	Text and Display
    3	Script
    4	Decorative
    5	Pictorial (or Symbol)

    
    As has appeared, this field influences recognition of allowable character sets of the font.
    
    
  • panose.bProportion - proportion of the font in the correspondence with the table:
    

    Value	Description
    0	Any
    1	No Fit
    2	Old Style
    3	Modern
    4	Even Width
    5	Expanded
    6	Condensed
    7	Very Expanded
    8	Very Condensed
    9	Monospaced

    
    The value panose.bProportion=9 determines the font of fixed width. In the correspondence the choice of the font in the programs happens to this field, if they for any purposes need only monospaced font, irrespective of to what the font is "actually".
    However at very many old Russian fonts claiming for a fixed width, even this field of the table PANOSE, let alone other fields is not filled in.
  
  
• Recognition accessible of character sets of the font.
  For want of to installation of the font it "Unicodeness" and accessible the character sets are determined in Windows'95 approximately by the following not documented rules:
  1. The fonts with the table OS/2 ver.1 - allowable code pages (scripts) are determined by intersection of a set of allowable pages from a field ulCodepageRange (for script cyrillic bit 2 should be set) and set of character sets supported by GDI.
     The special case - support of a character set OEM/DOS (which generally defined values has no), used in DOS-window. Probably, in this case it is necessary to have in the mask of allowable codepages bit that corresponds to concrete language settings in Windows, for Russian settings it is CP866 (bit 49).
     
  2. (Multilingual) fonts with the table OS/2 ver.0 - allowable code pages (scripts) are determined (based on set of character sets supported by GDI) proceeding from actual availability of characters in the font, in particular for script cyrillic it is necessary and enough, that in the font the letter "£" (small cyrillic io, U+0451) is represented.
     (Recently I, have checked up it, at last, having made practically from one letter "£" the microfont onecyr.ttf, and also has found some documentary confirmation of this logic, see notes about Unicode in Postscript.)
     The special case - support of a character set OEM/DOS (which generally defined values has no), used in DOS-window. Most likely, such fonts are considered not supporting OEM/DOS-coding.
     
  3. The monolingual fonts from (inter)national ("localised") versions Win 3.1 - their sole code page is contained in higher byte of a field fsSelection, and it is a program-internal Microsoft-proprietary code of a character set (for script cyrillic - 204=0xCC). Such fonts will correctly be recognized as in the national versions Windows 3.1*, and in Windows'9x. However it is not only not documented, but also contradicts the specifications TrueType - as to the first issue rev. 1.0, and last revision rev. 1.66, where unequivocally it is required, that the unused bits of the fsSelection field should be equal to zero.
     I think, as such fonts are considered not supporting OEM/DOS-encoding.
     
  4. Monolingual fonts, in which the table cmap has the coding 3.1 - Microsoft UGL, and in the table PANOSE the type of the font panose.bFamilyType=5 (Pictorial) is indicated. Such fonts in Windows'95 by rather unexpected manner are treated as fonts with a character set Symbol and, for example, in Word-97 in the file are written Unicodes from range U+F020... U+F0FF, in the font at all absent.
     (Such russian and easterneuropean fonts are containing for example in multilingual products of Accentsoft.)

If the work with TruÅType by fonts (first of all in Windows) is based to taking characters on appropriate numerical value in Unicode, the work with Postscript fonts is based on names of characters in the font.
If it is necessary to take characters via Unicode (in Windows NT, in Word'97) so names of characters should be to deliver in the correspondence with Unicode values, that the manager of Postscript-fonts (Adobe Type Manager) should do. It is known that ATM/NT 4.0 can do such comparison, whereas ATM/95 4.0 - can't. The list of glyph names (Adobe Glyph List) and their Unicodes is contained in the documents from Adobe, in the main document Unicode and Glyph Names the algorithm of such comparison is defined and the references to the glyph lists are contained.

• It is interesting, that for want of solution of a particular problem - how to select the necessary Unicode value, if for some characters actually there are two possible values (for example, the Greek letter "mu" actually represented in two places: U+00B5 MICRO SIGN and U+03BC GREEK SMALL LETTER MU) - the important note about algorithm is made, on which the presence at the font of code pages 1250-1258 from Microsoft is determined. Affirms, that OpenType converter, and also ATM/NT and driver TrueType in GDI decide it on availability in the font of the following glyphs-markers:
  
  

  Glyph image	Glyph name	Unicode and Unicode name	Windows Code Page
  	ncaron	U+0148 LATIN SMALL LETTER N WITH CARON	1250: Windows Latin 2 (Central Europe)
  	afii10071	U+0451 CYRILLIC SMALL LETTER IO	1251: Windows Cyrillic (Slavic)
  	ecircumflex	U+00EA LATIN SMALL LETTER E WITH CIRCUMFLEX	1252: Windows Latin 1 (ANSI)
  	upsilondieresis	U+03CB GREEK SMALL LETTER UPSILON WITH DIALYTIKA	1253: Windows Greek
  	Idotaccent	U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE	1254: Windows Latin 5 (Turkish)
  	afii57664	U+05D0 HEBREW LETTER ALEF	1255: Windows Hebrew
  	afii57410	U+0622 ARABIC LETTER ALEF WITH MADDA ABOVE	1256: Windows Arabic
  	uogonek	U+0173 LATIN SMALL LETTER U WITH OGONEK	1257: Windows Baltic Rim
  	dong	U+20AB DONG SIGN	1258: Windows Vietnamese

  
  
• Quite recently Dmitry Yunov found a method to write character set into Type 1 PS-font under Windows'9x and ATM4 (is spoken, that it was by news to the experts from Adobe).
  A charset value (for a cyrillic 0xCC=204) should be written into PFM-file in byte with offset 0x55. It is possible also to add following lines at the end of the font INF-file:
  WindowsCharSet 204
  WindowsFirstChar 32
  WindowsLastChar 255
  Then during font installation ATM will create from a pair AFM + INF a correct PFM with a correct charset code.
  
  As a result in the system will appeare a font, which Windows designate as device (printer) font, with a sole charset (in our case with cyrillic).
  If we work with Russian or russian-tuned Windows'9x (main Codepage is CP1251), such fonts should correctly display in Office-97 on a screen correctly print on printer. Nevertheless Unicode character table for them is not shown correctly neither inBjondi Character Agent, nor in Word-97.
  As asserts Dmitry Yunov, in case of incongruity of main language in Windows and of font character set these fonts "will be on the screen in MSOffice-97 either with incorrect metric or with a lot of missing characters". Under Windows'98 I have not detected such effects in Word'97, but font with charset=2 (Symbol), in WordPad was displayed with incorrect metric.
  For editing fonts and INF-files it is possible to take advantage of the special program.

Return to the contents

Family   Search and infos   Eros and Tanatos   Languages, fonts and encodings Tropical fruits   Mailer and other lovely programs   Miscellaneous