Windows code page

Windows code pages are sets of characters or code pages (known as character encodings in other operating systems) used in Microsoft Windows from the 1980s and 1990s. Windows code pages were gradually superseded when Unicode was implemented in Windows, although they are still supported both within Windows and other platforms.

There are two groups of code pages in Windows systems: OEM and ANSI code pages. Code pages in both of these groups are extended ASCII code pages.

ANSI code page

Windows-125x series
Alias(es) ANSI (misnomer)
Standard WHATWG Encoding Standard
Extends US-ASCII
Preceded by ISO 8859
Succeeded by Unicode
UTF-16 (in Win32 API)

ANSI code pages (officially called "Windows code pages"[1] after Microsoft accepted the former term being a misnomer[2]) are used for native non-Unicode (say, byte oriented) applications using a graphical user interface on Windows systems.

ANSI is a bit of a misnomer as the behavior does not exactly match the ANSI standards and other codepages can be selected, most recently UTF-8 Unicode.

ANSI Windows code pages, and especially the code page 1252, were so-called since they were purportedly based on drafts submitted or intended for ANSI. However, ANSI and ISO have not standardized any of these code pages. Instead they are either supersets of the standard sets such as those of ISO 8859 and the various national standards (like Windows-1252 vs. ISO-8859-1), major modifications of these (making them incompatible to various degrees, like Windows-1250 vs. ISO-8859-2) or having no parallel encoding (like Windows-1257 vs. ISO-8859-4; ISO-8859-13 was introduced much later).[2] About twelve of the typography and business characters from CP1252 at code points 0x80–0x9F (in ISO 8859 occupied by C1 control codes, which are useless in Windows) are present in many other ANSI/Windows code pages at the same codes. These code pages are labelled by Internet Assigned Numbers Authority (IANA) as "Windows-number".[3]

OEM code page

The OEM code pages (original equipment manufacturer) are used by Win32 console applications, and by virtual DOS, and can be considered a holdover from DOS and the original IBM PC architecture. A separate suite of code pages was implemented not only due to compatibility, but also because the fonts of VGA (and descendant) hardware suggest encoding of line drawing characters to be compatible with code page 437. Most OEM code pages share many code points, particularly for non-letter characters, with the second (non-ASCII) half of CP437.

A typical OEM code page, in its second half, does not resemble any ANSI/Windows code page even roughly. Nevertheless, two single-byte, fixed-width code pages (874 for Thai and 1258 for Vietnamese) and four multibyte CJK code pages (932, 936, 949, 950) are used as both OEM and ANSI code pages. Code page 1258 uses combining diacritics, as Vietnamese requires more than 128 letter-diacritic combinations. This is in contrast to VISCII, which replaces some of the C0 (i.e. ASCII) control codes.

History

Initially, computer systems and system programming languages did not make a distinction between characters and bytes. This led to much confusion subsequently. Microsoft software and systems previous to the Windows NT line are examples of this, using the OEM and ANSI code pages, which do not make the distinction.

Since the late 1990s, software and systems are increasingly adopting more direct encodings of Unicode, in particular UTF-8 and UTF-16; this trend has been improved by the widespread adoption of XML, which provides a more adequate mechanism for labelling the encoding used.[4] Recent Microsoft products and application program interfaces use Unicode internally, but many applications and APIs continue to use the default encoding of the computer's locale when reading and writing text data to files or standard output. Therefore, though Unicode is the accepted standard, there is still backwards compatibility with the older Windows code pages.

The euro sign was added relatively recently to ANSI and OEM code pages (1998 in the case of Code page 858) and therefore obsolete versions of Windows are unable to use it with code pages.

Since version 1803, Windows machines can be configured to allow UTF-8 as the "ANSI" and OEM codepage.[5]

List

This list is incomplete; you can help by expanding it.

The following Windows code pages exist:

IDNamesDescriptionTypeBaseEncodingStandardSupport DOS- based WindowsSupport Windows NT familySupport Windows CE familyComments
37CP037, IBM037IBM EBCDIC US-CanadaOtherEBCDIC derivation8-bit SBCSIBM CP037[6]?Yes
437CP437, IBM437IBM PC USOEMASCII derivation8-bit SBCSIBM CP437[7]1.00-4.90Yes
1250CP1250, Windows-1250Latin 2 / Central EuropeanANSIASCII derivation8-bit SBCSMicrosoft CP1250[8][9]?Yes
1251CP1251, Windows-1251CyrillicANSIASCII derivation8-bit SBCSMicrosoft CP1251[10][11]?Yes
1252CP1252, Windows-1252Latin 1 / Western EuropeanANSIASCII derivation8-bit SBCSMicrosoft CP1252[12][13]?Yesletter repertoire similar to CP850
1253CP1253, Windows-1253GreekANSIASCII derivation8-bit SBCSMicrosoft CP1253[14][15]?Yes
1254CP1254, Windows-1254TurkishANSIASCII derivation8-bit SBCSMicrosoft CP1254[16][17]?Yes
1255CP1255, Windows-1255HebrewANSIASCII derivation8-bit SBCSMicrosoft CP1255[18][19]?Yes
1256CP1256, Windows-1256ArabicANSIASCII derivation8-bit SBCSMicrosoft CP1256[20][21]?Yes
1257CP1257, Windows-1257BalticANSIASCII derivation8-bit SBCSMicrosoft CP1257[22][23]?Yes
1258CP1258, Windows-1258VietnameseOEM+ANSI?8-bit SBCSMicrosoft CP1258[24][25]?Yes

Problems arising from the use of code pages

Microsoft strongly recommends using Unicode in modern applications, but many applications or data files still depend on the legacy code pages.

  • Programs need to know what code page to use in order to display the contents of files correctly. If a program uses the wrong code page it may show text as mojibake.
  • The code page in use may differ between machines, so files created on one machine may be unreadable on another.
  • Data is often improperly tagged with the code page, or not tagged at all, making determination of the correct code page to read the data difficult.
  • These Microsoft code pages differ to various degrees from some of the standards and other vendors' implementations. This isn't a Microsoft issue per se, as it happens to all vendors, but the lack of consistency makes interoperability with other systems unreliable in some cases.
  • The use of code pages limits the set of characters that may be used.
  • Characters expressed in an unsupported code page may be converted to question marks (?) or other replacement characters, or to a simpler version (such as removing accents from a letter). In either case, the original character may be lost.

See also

  • AppLocale — a utility to run non-Unicode (code page-based) applications in a locale of the user's choice.

References

  1. ↑ Code Pages, MSDN
  2. 1 2 MSDN: Glossary of Terms
  3. ↑ IANA list of Character Sets
  4. ↑ http://www.w3.org/TR/xml11/#charencoding
  5. ↑ hylom (2017-11-14). "Windows 10のInsider PreviewでシステムロケールをUTF-8にするオプションが追加される" [The option to make UTF-8 the system locale added in Windows 10 Insider Preview]. スラド (in Japanese). Retrieved 2018-05-10.
  6. ↑ IBM. "SBCS code page information document - CPGID 00037". Retrieved 2014-07-04.
  7. ↑ IBM. "SBCS code page information document - CPGID 00437". Retrieved 2014-07-04.
  8. ↑ Microsoft. "Windows 1250". Retrieved 2014-07-06.
  9. ↑ IBM. "SBCS code page information document CPGID 01250". Retrieved 2014-07-06.
  10. ↑ Microsoft. "Windows 1251". Retrieved 2014-07-06.
  11. ↑ IBM. "SBCS code page information document CPGID 01251". Retrieved 2014-07-06.
  12. ↑ Microsoft. "Windows 1252". Retrieved 2014-07-06.
  13. ↑ IBM. "SBCS code page information document CPGID 01252". Retrieved 2014-07-06.
  14. ↑ Microsoft. "Windows 1253". Retrieved 2014-07-06.
  15. ↑ IBM. "SBCS code page information document CPGID 01253". Retrieved 2014-07-06.
  16. ↑ Microsoft. "Windows 1254". Retrieved 2014-07-06.
  17. ↑ IBM. "SBCS code page information document CPGID 01254". Retrieved 2014-07-06.
  18. ↑ Microsoft. "Windows 1255". Retrieved 2014-07-06.
  19. ↑ IBM. "SBCS code page information document CPGID 01255". Retrieved 2014-07-06.
  20. ↑ Microsoft. "Windows 1256". Retrieved 2014-07-06.
  21. ↑ IBM. "SBCS code page information document CPGID 01256". Retrieved 2014-07-06.
  22. ↑ Microsoft. "Windows 1257". Retrieved 2014-07-06.
  23. ↑ IBM. "SBCS code page information document CPGID 01257". Retrieved 2014-07-06.
  24. ↑ Microsoft. "Windows 1258". Retrieved 2014-07-06.
  25. ↑ IBM. "SBCS code page information document CPGID 01258". Retrieved 2014-07-06.
  26. ↑ Code page identifier list
  27. 1 2 3 4 5 "Code Page Identifiers". Microsoft Developer Network. Microsoft. 2014. Archived from the original on 2016-06-19. Retrieved 2016-06-19.
  28. 1 2 3 4 5 "Web Encodings - Internet Explorer - Encodings". WHATWG Wiki. 2012-10-23. Archived from the original on 2016-06-20. Retrieved 2016-06-20.
  29. ↑ Foller, Antonin (2014) [2011]. "Western European (IA5) encoding - Windows charsets". WUtils.com - Online web utility and help. Motobit Software. Archived from the original on 2016-06-20. Retrieved 2016-06-20.
  30. ↑ Foller, Antonin (2014) [2011]. "German (IA5) encoding - Windows charsets". WUtils.com - Online web utility and help. Motobit Software. Archived from the original on 2016-06-20. Retrieved 2016-06-20.
  31. ↑ Foller, Antonin (2014) [2011]. "Swedish (IA5) encoding - Windows charsets". WUtils.com - Online web utility and help. Motobit Software. Archived from the original on 2016-06-20. Retrieved 2016-06-20.
  32. ↑ Foller, Antonin (2014) [2011]. "Norwegian (IA5) encoding - Windows charsets". WUtils.com - Online web utility and help. Motobit Software. Archived from the original on 2016-06-20. Retrieved 2016-06-20.
  33. ↑ Foller, Antonin (2014) [2011]. "US-ASCII encoding - Windows charsets". WUtils.com - Online web utility and help. Motobit Software. Archived from the original on 2016-06-20. Retrieved 2016-06-20.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.