Understanding Encodings, Fonts, Languages and Code Tables

This chapter describes the differences between encodings, fonts, languages, and code tables with respect to QNX Neutrino and Photon.

This chapter describes the following concepts:

• Terminology Definitions
• Unicode encoding in Photon
• Advanced Graphics
• General Notes
• Language Notes

Terminology Definitions

Each concept is described below:

Language

With respect to computers, language has a vague meaning. When someone asks, “Does your system support language X?”, you should say, “What do you mean? Input? Display? Unicode only? Translation of antiquated encodings?”. Some languages are complex, including, but not limited to, Thai, Hebrew, Arabic. Complex language can either require compositing of multiple glyphs to form a single character, or bi-directional (right-to-left) processing.

Code Table

In basic terms, a code table is a two column list that maps a numerical value to a glyph. The most widely used code table is Unicode (see http://www.unicode.org).

Encoding

Encoding values are “stored” from a code table. There are many different encoding types to choose from depending on your application. Here are some encodings for storing Unicode:

• UTF-8
• UTF-16 (UCS-2)
• UTF-32 (UCS-4)

Font

A font file is a binary file that contains glyphs, or “pictures”, of symbols representing the building blocks of a displayable character set. Depending on the language, multiple glyphs can comprise a single character. In Unicode encoded fonts, some languages, such as Chinese, Japanese, and Korean, have combining characters that are comprised of multiple glyphs.

Characters

A character can be one of the following:

• a single Unicode point glyph (for example, 0x00CB)
• a combined character (pre-composed glyphs, already assigned a single Unicode point, for example 0x6E90, ?)
• a composited character (composed of multiple glyphs, from multiple Unicode points)

Unicode encoding in Photon

Photon supports the Unicode 3.x (16-bits) code table through its font rendering subsystem. Range: 0x0000 to 0xFFFF.

Photon completely supports UTF-8 (preferred) and UTF-16 encodings. There is limited support for UTF-32 encoding.

Photon does not currently support complex language processing. For example, Photon does support Thai TIS-620 encoding, so that TIS-620 encoded data can be converted to and from Unicode, however it does not support the processing of this complex language for display and input.

The following lists specify UTF encoding support on a per-function basis. This information is also provided in the documentation for each function.

UTF-8, UTF-16 (Pg_TEXT_WIDECHAR)

• PgDrawText(), PgDrawTextCx()
• PgDrawTextv()
• PgDrawTextChars()
• PgDrawTextvCx(), PgDrawTextCharsCx()
• PgDrawTextArea(), PgDrawTextAreaCx()
• PfExtentFractTextCharPositions()
• PfExtentTextCharPositions(), PfExtentTextCharPositionsCx()

UTF-8

• PgExtentMultiText()
• PgExtentText()
• PfExtentText()
• PfExtentTextToRect()
• PfFractionalExtentText()
• PfExtentComponents(), PfExtentComponentsCx()
• PfTextWidthBytes()
• PfTextWidthChars()

UTF-8, UTF-16 (PF_WIDECHAR), UTF-32 (PF_WIDECHAR32)

• PfRender(), PfRenderCx()
• PfExtent(), PfExtentCx()
• PfExtent16dot16()

UTF-16

• PfExtentWideText()
• PfWideTextWidthBytes()
• PfWideTextWidthChars()

String representation

In this history of Unix, the size of wchar_t was changed from 16-bits to 32-bits. Therefore, when working in a Photon environment, you must use uint16_t for wide character strings. In general, there is no need to use wchar_t, except possibly when porting while creating a Photon application. Even in this situation, there is no true benefit of using a 32-bit wchar_t, because Photon currently only supports Unicode 3.x (16-bits). To avoid these sizing issues altogether, you should use multi-byte UTF-8 encoded strings.

Translation

Photon provides the PxTranslate*() family of functions in order to translate non-Unicode character set multi-byte strings to and from UTF-8, and Unicode-encoded UTF-16/UTF-32 to and from UTF-8. For more information, see Appendix: Unicode Multilingual Support.

Services

Photon uses a central font server called phfont. This server uses plugins to support the rendering of different types of font file formats. You can find more details on each plugin by reading the documentation, or by using the use utility on the plugins located in /lib/dll/font. A central server allows for the consolidation of resources, and the decoupling of vendor-specific API calls from client applications.

---

The font server phfont is not coupled to the Photon environment directly. Non-photon applications can use Pf*Cx() calls by linking against the libfont.* library.

---

Advanced Graphics

The Advanced Graphics API has no font-specific API calls. There are currently two options for Advanced Graphics applications:

1. Use phfont() and libfont.*() utilizing the Pf*Cx() API calls.
2. Use the Bitstream Font Fusion 2.4 library directly. This library provides a very low-level font rendering solution, which is used by several of the phfont rendering plugins. Header files for this library are installed at /usr/include/FontFusion/T2K.

General Notes

In general, Photon and Advanced Graphics can both utilize the same font files. The only exception is Photon's PHF file format, which is generated from widely available BDF files. If you must use a bitmap font, the utility bdftophf2 can be used to generate them.

The following font file formats are currently supported:

• Bitmap (.phf, converted from Unicode, ascending sorted, BDF files, legacy)
• TrueType (.ttf)
• Adobe Type 1 (.pfa)
• Adobe Type 2 (.ccf)
• Bitstream Stroke (.ffs)
• Bitstream T2K (.t2k)
• Bitstream Speedo (.spd, public encryption only)
• TrueType Collection (.ttc)
• Bitstream PFR (.pfr — legacy, not recommended for use in new products)

You should always use scalable font technology, unless you use a highly contained system that only requires specific point sizes. Once you start adding additional bitmap fonts for more point sizes, any benefits are quickly lost due to large file sizes. Due to the flexibility and speed of current scalable font technology, bitmap fonts are no longer recommended.

In Photon, fonts are installed under /usr/photon/font_repository. See the documentation for mkfontdir, phfont, and the chapter Appendix: Photon in Embedded Systems.

Documentation for the current shipping version of FontFusion is available at: $QNX_TARGET/usr/help/FontFusion/FF_Ref_Guide.pdf

Language Notes

Languages that use characters which are combined of glyphs that are already in the Unicode range (for example Chinese, Japanese, Korean) are currently supported for display and input. Input is supported by the input methods cpim(), vpim(), and kpim().

Languages that are complex, such as Thai, Arabic, and Hebrew (though single glyphs can be displayed) are not currently supported for display or input.