Bush hid the facts

Bush hid the facts is a common name for a bug present in some versions of Microsoft Windows, which causes text encoded in ASCII to be interpreted as if it were UTF-16LE, resulting in mojibake. When the string "Bush hid the facts", without newline or quotes, was put in a new Notepad document and saved, closed, and reopened, the nonsensical Chinese characters "畂桳栠摩琠敨映捡獴" would appear instead.

While "Bush hid the facts" is the sentence most commonly presented on the Internet to induce the error, the bug can be triggered by many strings with letters and spaces in the same positions, for example "hhhh hhh hhh hhhhh".[1] Other sequences trigger the bug as well, including even the text "a ".

The bug occurs when the string is passed to the Win32 charset detection function IsTextUnicode. IsTextUnicode sees that the bytes match the UTF-16LE encoding of valid (if nonsensical) Chinese Unicode characters, concludes that the text is valid UTF-16LE Chinese and returns true, and the application then incorrectly interprets the text as UTF-16LE.[2]

The bug had existed since IsTextUnicode was introduced with Windows NT 3.5 in 1994, but was not discovered until early 2004.[3] Many text editors and tools exhibit this behavior on Windows because they use IsTextUnicode to determine the encoding of text files. As of Windows Vista, Notepad has been modified to use a different detection algorithm that does not exhibit the bug, but IsTextUnicode remains unchanged in the operating system, so any other tools that use the function are still affected.[4]

Workarounds

Several workarounds exist for this bug:

  • Editing the text to not be a pattern that triggers this bug will avoid it. For instance, adding a new line in the first 20 characters will work.
  • If the file is saved as "UTF-8" rather than "ANSI" the text loads correctly, because Notepad prepends a UTF-8 byte order mark, which is a pattern that does not trigger the bug. UTF-8 without the byte order mark would still trigger the bug, as this sequence is represented identically in UTF-8 as in ASCII.
  • The bug is also avoided by saving as "Unicode", which in Microsoft Windows means UTF-16LE. When loading this text IsTextUnicode should (and does) return true and the text is correct.
  • To retrieve the original text using Notepad, bring up the "Open a file" dialog box, select the file, select "ANSI" or "UTF-8" in the "Encoding" list box, and click Open. Under Windows 2000, Notepad lacks the "Encoding" list box. Notepad2 also lacks this. WordPad appears to load the text correctly without choosing the encoding, since it uses its own encoding detection.

References

  1. ↑ Christensen, Brett M. (November 2, 2009). "Bush Hid The Facts - Notepad Conspiracy Claim". Hoax Slayer.
  2. ↑ Chen, Raymond (March 24, 2007). "Some files come up strange in Notepad - The Old New Thing". blogs.msdn.com.
  3. ↑ Cumps, David (February 27, 2004). "Notepad bug? Encoding issue?". #region .Net Blog. Retrieved February 15, 2009.
  4. ↑ Kaplan, Michael S. (March 25, 2008). "Bush might've still hid the facts, but he can't hide them from Vista SP1/Server 2008 Notepad". Retrieved 13 April 2017.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.