Skip to content

Encoding and Character Codes

標準~31 min4 terms5 past questions

Goals of this unit

  • Explain how encoding and decoding relate, and work out the number of bits needed from how many kinds of things must be told apart.
  • Tell JIS Code and Shift_JIS Code apart by how they mix single-byte and double-byte characters.
  • Explain Unicode in terms of the range of characters it covers and the idea behind encoding schemes.
Terms at a glance (4)
TermIn one lineWhat it coversSize per character
符号化Mapping information to codes by fixed rulesAny information: text, sound, imagesDepends on the rules (n bits give 2^n patterns)
JISコードJapanese character code defined by Japanese Industrial StandardsJapanese characters and alphanumericsAlphanumerics 1 byte, kanji etc. 2 bytes
シフトJISコードCharacter code that shifts double-byte code numbers so they coexist with single-byte charactersJapanese characters and alphanumericsAlphanumerics and half-width kana 1 byte, kanji etc. 2 bytes
UnicodeCharacter code that gathers characters of many of the world's languages into one numbering systemCharacters of many of the world's languagesDepends on the encoding scheme (UTF-8 uses 1–4 bytes)

Terms

Encoding (符号化) means mapping information such as text, sound or images to codes such as bit strings according to fixed rules.

文字や情報

符号化(エンコード)

ビット列(0と1)

復号(デコード)

If sender and receiver share the mapping rules (the code system), the receiver can turn the code back into the original information, which is decoding; if the rules differ, the result can be garbled text and similar errors. A code of n bits can distinguish 2^n patterns, so the number of bits needed follows from how many kinds of things must be represented. Code length is either fixed-length, the same for every item, or variable-length, differing from item to item.

AnalogyIt is like assigning meanings in advance to a lighthouse's flashing patterns. Only ships holding the same table can read the meaning from the flashes.

How to rememberThe Japanese term 符号化 combines 符号 (a signal mark) and 化 (turning into); its reverse, 復号, uses 復 (restore): getting the original back from the code.

Exam points

  • Examn bits distinguish 2^n patterns, and the smallest n that covers the number of kinds to represent is the required number of bits (ビット数). For 10 kinds, for example, 3 bits give only 8 patterns, so 4 bits are needed.
  • ExamUsing more bits per sample (標本) raises the number of distinguishable levels, so changes in amplitude can be represented in finer steps.
  • ExamEven with a variable-length (可変長) code, an unseparated sequence can be read back uniquely from the start as long as no code is the leading part of another. If codes are assigned so that their 1 bits never overlap, a combination can be expressed by addition and a removal by subtraction.

3 past questions on this term 符号化

JIS Code (JISコード) is a character code defined by Japanese Industrial Standards (JIS); it uses 1 byte for alphanumerics and the like, and 2 bytes for kanji, hiragana and full-width kana.

Character set standard (example)Main charactersSize per character
JIS X 0201Alphanumerics, symbols, half-width kana1 byte (8 bits)
JIS X 0208Kanji, hiragana, full-width kana2 bytes (16 bits)

Characters represented in 1 byte are called single-byte codes and those in 2 bytes double-byte codes; the alphanumeric set is defined in JIS X 0201 and the set of kanji and other characters in JIS X 0208, for example. In the narrow sense, JIS Code refers to the format that inserts special switching codes whenever a string changes between single-byte and double-byte characters, and it has mainly been used for email.

AnalogyIt is like a shared table of character numbers issued nationwide. Devices using the same table point to the same character with the same number.

How to rememberJIS stands for Japanese Industrial Standards, the same family of standards that covers Japanese industrial products.

Exam points

  • ExamIn JIS character codes (文字コード), alphanumerics and the like take 1 byte (8 bits) per character, and kanji and the like take 2 bytes. JIS X 0201, for example, assigns 8-bit values to single-byte alphanumerics.
  • ExamThe size of uncompressed text data is found by multiplying each field's character count by its bytes per character, adding these up, and multiplying by the number of records. Double-byte characters take twice the space of the same number of single-byte characters.

1 past questions on this term JISコード

シフトJISコードShift_JIS Code

Open in glossary シフトJISコード

Shift_JIS Code (シフトJISコード) is a character code that moves the numbers of JIS double-byte characters such as kanji into ranges that do not overlap with single-byte characters.

The first byte of a double-byte character takes a value that single-byte characters do not use, so the value read shows whether a character is single-byte or double-byte. This makes documents mixing alphanumerics, half-width kana and kanji easy to handle, and it has been widely used for Japanese text on PCs such as Windows machines.

AnalogyPicture two kinds of goods on one shelf: if one kind's number tags are moved into a different color range, you can tell them apart by tag color alone, with no dividers.

How to rememberShift means to move over. The code numbers were shifted to a new place, hence Shift JIS.

Exam points

  • Mix-upShift_JIS Code (シフトJISコード) and Unicode differ in the range of characters they cover. Shift_JIS Code handles Japanese characters and alphanumerics, while Unicode handles characters of many of the world's languages in one system.

1 past questions on this term シフトJISコード

JIS Code vs. Shift_JIS Code

AspectJIS CodeShift_JIS Code
Key differenceHow single-byte and double-byte characters are mixedInserts special codes marking each switchGives the first byte of a double-byte character a value that single-byte characters do not use
Main useEmailJapanese text on PCs (Windows etc.)
Underlying character setJIS sets of alphanumerics, kanji, etc.The same JIS sets of alphanumerics, kanji, etc.

Unicode is a character code that gives each character of many of the world's languages its own number so that all of them can be handled in one system.

  1. Character (e.g. あ)
  2. Unicode code point (U+3042)
  3. UTF-8 byte sequence (3 bytes)

The method that turns the numbers assigned to characters into actual byte sequences is called an encoding scheme (符号化方式); examples are UTF-8 and UTF-16. UTF-8 is a variable-length scheme that uses 1 byte for alphanumerics and 3 bytes for most Japanese characters, and it is widely used on web pages.

AnalogyIt is like a library that catalogs books from every country under one running number. Any book can be found by its number in the same register.

How to rememberUni means one, as in uniform and unit: one system for the world's characters.

Exam points

  • ExamUnicode is a character code that unites the characters of many languages in one common numbering system, so multilingual text can share one document without switching character codes (文字コード) by language.
  • Mix-upUTF-8 is one of the schemes that express Unicode numbers as byte sequences; it is not the name of a character set. The same Unicode character has different byte sequences in UTF-8 and UTF-16.

1 past questions on this term Unicode

Shift_JIS Code vs. Unicode

AspectShift_JIS CodeUnicode
Key differenceCharacters coveredJapanese characters and alphanumericsCharacters of many of the world's languages
Bytes per character1 or 2Depends on the encoding scheme (UTF-8: 1–4)
Mixing multilingual charactersCannot represent many non-Japanese charactersCan mix them within one system

Key points

  • Encoding maps information to codes by rules; decoding with the same rules restores it, and mismatched rules cause garbled text.
  • n bits distinguish 2^n patterns, and the bits needed depend on how many kinds must be represented.
  • In JIS character codes, alphanumerics and the like take 1 byte and kanji and the like take 2 bytes; text data size is estimated from these byte counts and the number of characters.
  • Shift_JIS Code shifts double-byte code numbers into a range that does not overlap with single-byte characters, and has been widely used for Japanese text on PCs.
  • Unicode is a system of numbers for characters; encoding schemes such as UTF-8 turn those numbers into byte sequences.

Check questions

5 past questions from this unit

Start the check questions

All past questions in this topic