Editing Wide character

{{Short description|Data type}}
{{For|double-wide (CJK ideograph-sized) variations of ASCII characters|halfwidth and fullwidth forms}}
{{More citations needed|date=February 2011}}
{{Use dmy dates|date=June 2021}}
A '''wide character''' is a computer [[character (computing)|character]] [[datatype]] that generally has a size greater than the traditional [[8-bit]] character.  The increased datatype size allows for the use of larger coded [[Character encoding|character sets]].

==History==
During the 1960s, mainframe and mini-computer manufacturers began to standardize around the 8-bit [[byte]] as their smallest datatype.  The 7-bit [[ASCII]] character set became the industry standard method for encoding [[alphanumeric]] characters for [[teleprinter|teletype machines]] and [[computer terminal]]s. The extra bit was used for parity, to ensure the integrity of data storage and transmission.  As a result, the 8-bit byte became the [[de facto]] datatype for computer systems storing ASCII characters in memory.

Later, computer manufacturers began to make use of the spare bit to extend the ASCII character set beyond its limited set of [[English alphabet]] characters. [[Extended ASCII|8-bit extensions]] such as IBM code page 37, [[PETSCII]] and [[ISO/IEC 8859|ISO 8859]] became commonplace, offering terminal support for [[Greek alphabet|Greek]], [[Cyrillic script|Cyrillic]], and many others.  However, such extensions were still limited in that they were region specific and often could not be used in tandem.  Special conversion routines had to be used to convert from one character set to another, often resulting in destructive translation when no equivalent character existed in the target set.

In 1989, the [[International Organization for Standardization]] began work on the [[Universal Coded Character Set|Universal Character Set]] (UCS), a multilingual character set that could be encoded using either a 16-bit (2-byte) or 32-bit (4-byte) value.  These larger values required the use of a datatype larger than 8-bits to store the new character values in memory.  Thus the term wide character was used to differentiate them from traditional 8-bit character datatypes.

==Relation to UCS and Unicode==
A wide character refers to the size of the datatype in memory.  It does not state how each value in a character set is defined.  Those values are instead defined using character sets, with UCS and [[Unicode]] simply being two common character sets that encode more characters than an 8-bit wide numeric value (255 total) would allow.

==Relation to multibyte characters==
Just as earlier data transmission systems suffered from the lack of an [[8-bit clean]] data path, modern transmission systems often lack support for 16-bit or 32-bit data paths for character data.  This has led to character encoding systems such as [[UTF-8]] that can use [[variable-width encoding|multiple bytes]] to encode a value that is too large for a single 8-bit symbol.

The [[C (programming language)|C]] standard distinguishes between ''multibyte'' encodings of characters, which use a fixed or variable number of bytes to represent each character (primarily used in source code and external files), from ''wide characters'', which are [[Run time (program lifecycle phase)|run-time]] representations of characters in single objects (typically, greater than 8 bits).

==Size of a wide character==
Early adoption of [[UCS-2]] ("Unicode 1.0") led to common use of [[UTF-16]] in a number of platforms, most notably [[Microsoft Windows]], [[.NET]] and [[Java (software platform)|Java]]. In these systems, it is common to have a "wide character" ({{code|wchar_t}} in C/C++; {{code|char}} in Java) type of 16-bits. These types do not always map directly to one "character", as [[surrogate pairs]] are required to store the full range of Unicode (1996, Unicode 2.0).<ref>{{cite web |url=http://msdn.microsoft.com/en-us/goglobal/bb688113.aspx |title=Globalization Step-by-Step: Unicode Enabled |website=msdn.microsoft.com |url-status=dead |archive-url=https://web.archive.org/web/20090101025155/http://msdn.microsoft.com/en-us/goglobal/bb688113.aspx |archive-date=2009-01-01}}</ref><ref>{{cite web |title=String Class (System) |url=https://learn.microsoft.com/en-us/dotnet/api/system.string?view=net-7.0 |website=learn.microsoft.com |language=en-us}}</ref><ref>{{cite web |title=Primitive Data Types (The Java™ Tutorials > Learning the Java Language > Language Basics) |url=https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html |website=docs.oracle.com}}</ref>

[[Unix-like]] generally use a 32-bit {{code|wchar_t}} to fit the 21-bit Unicode code point, as C90 prescribed.<ref>{{cite web |title=Null-terminated wide strings <wctype.h> - cppreference.com |url=https://en.cppreference.com/w/c/string/wide |website=en.cppreference.com}}</ref>

The size of a wide character type does not dictate what kind of text encodings a system can process, as conversions are available. (Old conversion code commonly overlook surrogates, however.) The historical circumstances of their adoption does also decide what types of encoding they ''prefer''. A system influenced by Unicode 1.0, such as Windows, tends to mainly use "wide strings" made out of wide character units. Other systems such as the Unix-likes, however, tend to retain the 8-bit "narrow string" convention, using a multibyte encoding (almost universally UTF-8) to handle "wide" characters.<ref>{{cite web |title=UTF-8 Everywhere |url=http://utf8everywhere.org/ |quote=In the following years many systems have added support for Unicode and switched to the UCS-2 encoding. It was especially attractive for new technologies, such as the Qt framework (1992), Windows NT 3.1 (1993) and Java (1995).}}</ref>

==Programming specifics==

===C/C++===
The [[C Standard Library|C]] and [[C++ Standard Library|C++]] standard libraries include [[C string handling|a number of facilities]] for dealing with wide characters and strings composed of them. The wide characters are defined using datatype <code>wchar_t</code>, which in the original [[C90 (C version)|C90]] standard was defined as

: "an integral type whose range of values can represent distinct codes for all members of the largest extended character set specified among the supported locales" (ISO 9899:1990 §4.1.5)

Both C and [[C++]] introduced fixed-size character types <code>char16_t</code> and <code>char32_t</code> in the 2011 revisions of their respective standards to provide unambiguous representation of 16-bit and 32-bit [[Unicode]] transformation formats, leaving <code>wchar_t</code> implementation-defined. The ISO/IEC 10646:2003 [[Unicode]] standard 4.0 says that:

:"The width of <code>wchar_t</code> is compiler-specific and can be as small as 8 bits. Consequently, programs that need to be portable across any C or C++ compiler should not use <code>wchar_t</code> for storing Unicode text. The <code>wchar_t</code> type is intended for storing compiler-defined wide characters, which may be [[Unicode]] characters in some compilers."<ref>{{Cite book|url=https://www.worldcat.org/oclc/52257637|title=The Unicode standard|date=2003|publisher=Addison-Wesley|others=Aliprand, Joan., Unicode Consortium.|isbn=0-321-18578-1|edition=Version 4.0|location=Boston|pages=109|chapter=5.2 ANSI/ISO C wchar_t|oclc=52257637}}</ref>

===Python===
According to [[Python (programming language)|Python]] 2.7's documentation, the language sometimes uses <code>wchar_t</code> as the basis for its character type <code>Py_UNICODE</code>. It depends on whether <code>wchar_t</code> is "compatible with the chosen Python Unicode build variant" on that system.<ref>{{cite web |url=https://docs.python.org/2.7/c-api/unicode.html |title=Unicode Objects and Codecs — Python 2.7 documentation |website=docs.python.org |access-date=2009-12-19}}</ref> This distinction has been deprecated since Python 3.3, which introduced a flexibly-sized UCS1/2/4 storage for strings and formally aliased {{code|Py_UNICODE}} to <code>wchar_t</code>.<ref>{{cite web |url=https://docs.python.org/3.10/c-api/unicode.htm|title=Unicode Objects and Codecs — Python 3.10.10 documentation |website=docs.python.org |access-date=2023-02-18}}</ref> Since Python 3.12 use of <code>wchar_t</code>, i.e. the <code>Py_UNICODE</code> [[typedef]], for Python strings (wstr in implementation) has been dropped and still as before an "[[UTF-8]] representation is created on demand and cached in the Unicode object."<ref>{{Cite web |title=Unicode Objects and Codecs |url=https://docs.python.org/3.12/c-api/unicode.html |access-date=2023-09-09 |website=Python documentation}}</ref>

==References==
{{reflist}}

==External links==
{{Sister project links}}
* [https://unicode.org/versions/Unicode4.0.0/ch05.pdf The Unicode Standard, Version 4.0 - online edition]
* [https://www.java2s.com/Tutorial/C/0300__Wide-Character-String/WideCharacterFunctions.htm C Wide Character Functions @ Java2S]
* [https://www.java2s.com/Tutorial/Java/0120__Development/0240__Unicode.htm Java Unicode Functions @ Java2S]
* [https://www.freebsd.org/cgi/man.cgi?query=multibyte&apropos=0&sektion=0&format=html Multibyte (3) Man Page @ FreeBSD.org]
* [https://msdn.microsoft.com/en-us/library/z207t55f%28VS.100%29.aspx Multibyte and Wide Characters @ Microsoft Developer Network]
* [https://msdn.microsoft.com/en-us/library/dd317743%28VS.85%29.aspx Windows Character Sets @ Microsoft Developer Network]
* [https://msdn.microsoft.com/en-us/library/dd374087%28VS.85%29.aspx Unicode and Character Set Programming Reference @ Microsoft Developer Network]
* [https://www.openbsd.org/papers/eurobsdcon2016-utf8.pdf Keep multibyte character support simple @ EuroBSDCon, Beograd, September 25, 2016]

{{DEFAULTSORT:Wide Character}}
[[Category:Character encoding]]
[[Category:C (programming language)]]
[[Category:C++]]