]> code.delx.au - gnu-emacs/blob - doc/emacs/mule.texi
* doc/emacs/glossary.texi (Glossary): Standardize on "text terminal" terminology.
[gnu-emacs] / doc / emacs / mule.texi
1 @c This is part of the Emacs manual.
2 @c Copyright (C) 1997, 1999-2012 Free Software Foundation, Inc.
3 @c See file emacs.texi for copying conditions.
4 @node International, Modes, Frames, Top
5 @chapter International Character Set Support
6 @c This node is referenced in the tutorial. When renaming or deleting
7 @c it, the tutorial needs to be adjusted. (TUTORIAL.de)
8 @cindex MULE
9 @cindex international scripts
10 @cindex multibyte characters
11 @cindex encoding of characters
12
13 @cindex Celtic
14 @cindex Chinese
15 @cindex Cyrillic
16 @cindex Czech
17 @cindex Devanagari
18 @cindex Hindi
19 @cindex Marathi
20 @cindex Ethiopic
21 @cindex German
22 @cindex Greek
23 @cindex Hebrew
24 @cindex IPA
25 @cindex Japanese
26 @cindex Korean
27 @cindex Lao
28 @cindex Latin
29 @cindex Polish
30 @cindex Romanian
31 @cindex Slovak
32 @cindex Slovenian
33 @cindex Thai
34 @cindex Tibetan
35 @cindex Turkish
36 @cindex Vietnamese
37 @cindex Dutch
38 @cindex Spanish
39 Emacs supports a wide variety of international character sets,
40 including European and Vietnamese variants of the Latin alphabet, as
41 well as Cyrillic, Devanagari (for Hindi and Marathi), Ethiopic, Greek,
42 Han (for Chinese and Japanese), Hangul (for Korean), Hebrew, IPA,
43 Kannada, Lao, Malayalam, Tamil, Thai, Tibetan, and Vietnamese scripts.
44 Emacs also supports various encodings of these characters that are used by
45 other internationalized software, such as word processors and mailers.
46
47 Emacs allows editing text with international characters by supporting
48 all the related activities:
49
50 @itemize @bullet
51 @item
52 You can visit files with non-@acronym{ASCII} characters, save non-@acronym{ASCII} text, and
53 pass non-@acronym{ASCII} text between Emacs and programs it invokes (such as
54 compilers, spell-checkers, and mailers). Setting your language
55 environment (@pxref{Language Environments}) takes care of setting up the
56 coding systems and other options for a specific language or culture.
57 Alternatively, you can specify how Emacs should encode or decode text
58 for each command; see @ref{Text Coding}.
59
60 @item
61 You can display non-@acronym{ASCII} characters encoded by the various
62 scripts. This works by using appropriate fonts on graphics displays
63 (@pxref{Defining Fontsets}), and by sending special codes to text
64 displays (@pxref{Terminal Coding}). If some characters are displayed
65 incorrectly, refer to @ref{Undisplayable Characters}, which describes
66 possible problems and explains how to solve them.
67
68 @item
69 Characters from scripts whose natural ordering of text is from right
70 to left are reordered for display (@pxref{Bidirectional Editing}).
71 These scripts include Arabic, Hebrew, Syriac, Thaana, and a few
72 others.
73
74 @item
75 You can insert non-@acronym{ASCII} characters or search for them. To do that,
76 you can specify an input method (@pxref{Select Input Method}) suitable
77 for your language, or use the default input method set up when you chose
78 your language environment. If
79 your keyboard can produce non-@acronym{ASCII} characters, you can select an
80 appropriate keyboard coding system (@pxref{Terminal Coding}), and Emacs
81 will accept those characters. Latin-1 characters can also be input by
82 using the @kbd{C-x 8} prefix, see @ref{Unibyte Mode}.
83
84 With the X Window System, your locale should be set to an appropriate
85 value to make sure Emacs interprets keyboard input correctly; see
86 @ref{Language Environments, locales}.
87 @end itemize
88
89 The rest of this chapter describes these issues in detail.
90
91 @menu
92 * International Chars:: Basic concepts of multibyte characters.
93 * Disabling Multibyte:: Controlling whether to use multibyte characters.
94 * Language Environments:: Setting things up for the language you use.
95 * Input Methods:: Entering text characters not on your keyboard.
96 * Select Input Method:: Specifying your choice of input methods.
97 * Coding Systems:: Character set conversion when you read and
98 write files, and so on.
99 * Recognize Coding:: How Emacs figures out which conversion to use.
100 * Specify Coding:: Specifying a file's coding system explicitly.
101 * Output Coding:: Choosing coding systems for output.
102 * Text Coding:: Choosing conversion to use for file text.
103 * Communication Coding:: Coding systems for interprocess communication.
104 * File Name Coding:: Coding systems for file @emph{names}.
105 * Terminal Coding:: Specifying coding systems for converting
106 terminal input and output.
107 * Fontsets:: Fontsets are collections of fonts
108 that cover the whole spectrum of characters.
109 * Defining Fontsets:: Defining a new fontset.
110 * Modifying Fontsets:: Modifying an existing fontset.
111 * Undisplayable Characters:: When characters don't display.
112 * Unibyte Mode:: You can pick one European character set
113 to use without multibyte characters.
114 * Charsets:: How Emacs groups its internal character codes.
115 * Bidirectional Editing:: Support for right-to-left scripts.
116 @end menu
117
118 @node International Chars
119 @section Introduction to International Character Sets
120
121 The users of international character sets and scripts have
122 established many more-or-less standard coding systems for storing
123 files. These coding systems are typically @dfn{multibyte}, meaning
124 that sequences of two or more bytes are used to represent individual
125 non-@acronym{ASCII} characters.
126
127 @cindex Unicode
128 Internally, Emacs uses its own multibyte character encoding, which
129 is a superset of the @dfn{Unicode} standard. This internal encoding
130 allows characters from almost every known script to be intermixed in a
131 single buffer or string. Emacs translates between the multibyte
132 character encoding and various other coding systems when reading and
133 writing files, and when exchanging data with subprocesses.
134
135 @kindex C-h h
136 @findex view-hello-file
137 @cindex undisplayable characters
138 @cindex @samp{?} in display
139 The command @kbd{C-h h} (@code{view-hello-file}) displays the file
140 @file{etc/HELLO}, which illustrates various scripts by showing
141 how to say ``hello'' in many languages. If some characters can't be
142 displayed on your terminal, they appear as @samp{?} or as hollow boxes
143 (@pxref{Undisplayable Characters}).
144
145 Keyboards, even in the countries where these character sets are
146 used, generally don't have keys for all the characters in them. You
147 can insert characters that your keyboard does not support, using
148 @kbd{C-q} (@code{quoted-insert}) or @kbd{C-x 8 @key{RET}}
149 (@code{ucs-insert}). @xref{Inserting Text}. Emacs also supports
150 various @dfn{input methods}, typically one for each script or
151 language, which make it easier to type characters in the script.
152 @xref{Input Methods}.
153
154 @kindex C-x RET
155 The prefix key @kbd{C-x @key{RET}} is used for commands that pertain
156 to multibyte characters, coding systems, and input methods.
157
158 @kindex C-x =
159 @findex what-cursor-position
160 The command @kbd{C-x =} (@code{what-cursor-position}) shows
161 information about the character at point. In addition to the
162 character position, which was described in @ref{Position Info}, this
163 command displays how the character is encoded. For instance, it
164 displays the following line in the echo area for the character
165 @samp{c}:
166
167 @smallexample
168 Char: c (99, #o143, #x63) point=28062 of 36168 (78%) column=53
169 @end smallexample
170
171 The four values after @samp{Char:} describe the character that
172 follows point, first by showing it and then by giving its character
173 code in decimal, octal and hex. For a non-@acronym{ASCII} multibyte
174 character, these are followed by @samp{file} and the character's
175 representation, in hex, in the buffer's coding system, if that coding
176 system encodes the character safely and with a single byte
177 (@pxref{Coding Systems}). If the character's encoding is longer than
178 one byte, Emacs shows @samp{file ...}.
179
180 As a special case, if the character lies in the range 128 (0200
181 octal) through 159 (0237 octal), it stands for a ``raw'' byte that
182 does not correspond to any specific displayable character. Such a
183 ``character'' lies within the @code{eight-bit-control} character set,
184 and is displayed as an escaped octal character code. In this case,
185 @kbd{C-x =} shows @samp{part of display ...} instead of @samp{file}.
186
187 @cindex character set of character at point
188 @cindex font of character at point
189 @cindex text properties at point
190 @cindex face at point
191 With a prefix argument (@kbd{C-u C-x =}), this command displays a
192 detailed description of the character in a window:
193
194 @itemize @bullet
195 @item
196 The character set name, and the codes that identify the character
197 within that character set; @acronym{ASCII} characters are identified
198 as belonging to the @code{ascii} character set.
199
200 @item
201 The character's syntax and categories.
202
203 @item
204 The character's encodings, both internally in the buffer, and externally
205 if you were to save the file.
206
207 @item
208 What keys to type to input the character in the current input method
209 (if it supports the character).
210
211 @item
212 If you are running Emacs on a graphical display, the font name and
213 glyph code for the character. If you are running Emacs on a text
214 terminal, the code(s) sent to the terminal.
215
216 @item
217 The character's text properties (@pxref{Text Properties,,,
218 elisp, the Emacs Lisp Reference Manual}), including any non-default
219 faces used to display the character, and any overlays containing it
220 (@pxref{Overlays,,, elisp, the same manual}).
221 @end itemize
222
223 Here's an example showing the Latin-1 character A with grave accent,
224 in a buffer whose coding system is @code{utf-8-unix}:
225
226 @smallexample
227 position: 1 of 1 (0%), column: 0
228 character: @`A (displayed as @`A) (codepoint 192, #o300, #xc0)
229 preferred charset: unicode (Unicode (ISO10646))
230 code point in charset: 0xC0
231 syntax: w which means: word
232 category: .:Base, L:Left-to-right (strong),
233 j:Japanese, l:Latin, v:Viet
234 buffer code: #xC3 #x80
235 file code: not encodable by coding system undecided-unix
236 display: by this font (glyph code)
237 xft:-unknown-DejaVu Sans Mono-normal-normal-
238 normal-*-13-*-*-*-m-0-iso10646-1 (#x82)
239
240 Character code properties: customize what to show
241 name: LATIN CAPITAL LETTER A WITH GRAVE
242 old-name: LATIN CAPITAL LETTER A GRAVE
243 general-category: Lu (Letter, Uppercase)
244 decomposition: (65 768) ('A' '`')
245 @end smallexample
246
247 @c FIXME? Does this section even belong in the user manual?
248 @c Seems more appropriate to the lispref?
249 @node Disabling Multibyte
250 @section Disabling Multibyte Characters
251
252 By default, Emacs starts in multibyte mode: it stores the contents
253 of buffers and strings using an internal encoding that represents
254 non-@acronym{ASCII} characters using multi-byte sequences. Multibyte
255 mode allows you to use all the supported languages and scripts without
256 limitations.
257
258 @cindex turn multibyte support on or off
259 Under very special circumstances, you may want to disable multibyte
260 character support, for a specific buffer.
261 When multibyte characters are disabled in a buffer, we call
262 that @dfn{unibyte mode}. In unibyte mode, each character in the
263 buffer has a character code ranging from 0 through 255 (0377 octal); 0
264 through 127 (0177 octal) represent @acronym{ASCII} characters, and 128
265 (0200 octal) through 255 (0377 octal) represent non-@acronym{ASCII}
266 characters.
267
268 To edit a particular file in unibyte representation, visit it using
269 @code{find-file-literally}. @xref{Visiting}. You can convert a
270 multibyte buffer to unibyte by saving it to a file, killing the
271 buffer, and visiting the file again with @code{find-file-literally}.
272 Alternatively, you can use @kbd{C-x @key{RET} c}
273 (@code{universal-coding-system-argument}) and specify @samp{raw-text}
274 as the coding system with which to visit or save a file. @xref{Text
275 Coding}. Unlike @code{find-file-literally}, finding a file as
276 @samp{raw-text} doesn't disable format conversion, uncompression, or
277 auto mode selection.
278
279 @c Not a single file in Emacs uses this feature. Is it really worth
280 @c mentioning in the _user_ manual? Also, this duplicates somewhat
281 @c "Loading Non-ASCII" from the lispref.
282 @cindex Lisp files, and multibyte operation
283 @cindex multibyte operation, and Lisp files
284 @cindex unibyte operation, and Lisp files
285 @cindex init file, and non-@acronym{ASCII} characters
286 Emacs normally loads Lisp files as multibyte.
287 This includes the Emacs initialization
288 file, @file{.emacs}, and the initialization files of packages
289 such as Gnus. However, you can specify unibyte loading for a
290 particular Lisp file, by adding an entry @samp{unibyte: t} in a file
291 local variables section (@pxref{File Variables}). Then that file is
292 always loaded as unibyte text. Note that this does not represent a
293 real @code{unibyte} variable, rather it just acts as an indicator
294 to Emacs in the same way as @code{coding} does (@pxref{Specify Coding}).
295 @ignore
296 @c I don't see the point of this statement:
297 The motivation for these conventions is that it is more reliable to
298 always load any particular Lisp file in the same way.
299 @end ignore
300 Note also that this feature only applies to @emph{loading} Lisp files
301 for evaluation, not to visiting them for editing. You can also load a
302 Lisp file as unibyte, on any one occasion, by typing @kbd{C-x
303 @key{RET} c raw-text @key{RET}} immediately before loading it.
304
305 @c See http://debbugs.gnu.org/11226 for lack of unibyte tooltip.
306 @vindex enable-multibyte-characters
307 The buffer-local variable @code{enable-multibyte-characters} is
308 non-@code{nil} in multibyte buffers, and @code{nil} in unibyte ones.
309 The mode line also indicates whether a buffer is multibyte or not.
310 @xref{Mode Line}. With a graphical display, in a multibyte buffer,
311 the portion of the mode line that indicates the character set has a
312 tooltip that (amongst other things) says that the buffer is multibyte.
313 In a unibyte buffer, the character set indicator is absent. Thus, in
314 a unibyte buffer (when using a graphical display) there is normally
315 nothing before the indication of the visited file's end-of-line
316 convention (colon, backslash, etc.), unless you are using an input
317 method.
318
319 @findex toggle-enable-multibyte-characters
320 You can turn off multibyte support in a specific buffer by invoking the
321 command @code{toggle-enable-multibyte-characters} in that buffer.
322
323 @node Language Environments
324 @section Language Environments
325 @cindex language environments
326
327 All supported character sets are supported in Emacs buffers whenever
328 multibyte characters are enabled; there is no need to select a
329 particular language in order to display its characters.
330 However, it is important to select a @dfn{language
331 environment} in order to set various defaults. Roughly speaking, the
332 language environment represents a choice of preferred script rather
333 than a choice of language.
334
335 The language environment controls which coding systems to recognize
336 when reading text (@pxref{Recognize Coding}). This applies to files,
337 incoming mail, and any other text you read into Emacs. It may also
338 specify the default coding system to use when you create a file. Each
339 language environment also specifies a default input method.
340
341 @findex set-language-environment
342 @vindex current-language-environment
343 To select a language environment, customize
344 @code{current-language-environment} or use the command @kbd{M-x
345 set-language-environment}. It makes no difference which buffer is
346 current when you use this command, because the effects apply globally
347 to the Emacs session. The supported language environments
348 (see the variable @code{language-info-alist}) include:
349
350 @cindex Euro sign
351 @cindex UTF-8
352 @quotation
353 ASCII, Belarusian, Bengali, Brazilian Portuguese, Bulgarian,
354 Chinese-BIG5, Chinese-CNS, Chinese-EUC-TW, Chinese-GB, Chinese-GBK,
355 Chinese-GB18030, Croatian, Cyrillic-ALT, Cyrillic-ISO, Cyrillic-KOI8,
356 Czech, Devanagari, Dutch, English, Esperanto, Ethiopic, French,
357 Georgian, German, Greek, Gujarati, Hebrew, IPA, Italian, Japanese,
358 Kannada, Khmer, Korean, Lao, Latin-1, Latin-2, Latin-3, Latin-4,
359 Latin-5, Latin-6, Latin-7, Latin-8 (Celtic), Latin-9 (updated Latin-1
360 with the Euro sign), Latvian, Lithuanian, Malayalam, Oriya, Polish,
361 Punjabi, Romanian, Russian, Sinhala, Slovak, Slovenian, Spanish,
362 Swedish, TaiViet, Tajik, Tamil, Telugu, Thai, Tibetan, Turkish, UTF-8
363 (for a setup which prefers Unicode characters and files encoded in
364 UTF-8), Ukrainian, Vietnamese, Welsh, and Windows-1255 (for a setup
365 which prefers Cyrillic characters and files encoded in Windows-1255).
366 @end quotation
367
368 @cindex fonts for various scripts
369 @cindex Intlfonts package, installation
370 To display the script(s) used by your language environment on a
371 graphical display, you need to have a suitable font. If some of the
372 characters appear as empty boxes or hex codes, you should install
373 extra fonts. Your operating system may have optional fonts that
374 you can install; or you can install the
375 GNU Intlfonts package, which includes fonts for most supported
376 scripts.@footnote{If you run Emacs on X, you may need to inform the X
377 server about the location of the newly installed fonts with
378 commands such as:
379 @c FIXME? I feel like this may be out of date.
380 @c Eg the intlfonts tarfile is ~ 10 years old.
381
382 @example
383 xset fp+ /usr/local/share/emacs/fonts
384 xset fp rehash
385 @end example
386 }
387 @xref{Fontsets}, for more details about setting up your fonts.
388
389 @findex set-locale-environment
390 @vindex locale-language-names
391 @vindex locale-charset-language-names
392 @cindex locales
393 Some operating systems let you specify the character-set locale you
394 are using by setting the locale environment variables @env{LC_ALL},
395 @env{LC_CTYPE}, or @env{LANG}. (If more than one of these is
396 set, the first one that is nonempty specifies your locale for this
397 purpose.) During startup, Emacs looks up your character-set locale's
398 name in the system locale alias table, matches its canonical name
399 against entries in the value of the variables
400 @code{locale-charset-language-names} and @code{locale-language-names}
401 (the former overrides the latter),
402 and selects the corresponding language environment if a match is found.
403 It also adjusts the display
404 table and terminal coding system, the locale coding system, the
405 preferred coding system as needed for the locale, and---last but not
406 least---the way Emacs decodes non-@acronym{ASCII} characters sent by your keyboard.
407
408 @c This seems unlikely, doesn't it?
409 If you modify the @env{LC_ALL}, @env{LC_CTYPE}, or @env{LANG}
410 environment variables while running Emacs (by using @kbd{M-x setenv}),
411 you may want to invoke the @code{set-locale-environment}
412 function afterwards to readjust the language environment from the new
413 locale.
414
415 @vindex locale-preferred-coding-systems
416 The @code{set-locale-environment} function normally uses the preferred
417 coding system established by the language environment to decode system
418 messages. But if your locale matches an entry in the variable
419 @code{locale-preferred-coding-systems}, Emacs uses the corresponding
420 coding system instead. For example, if the locale @samp{ja_JP.PCK}
421 matches @code{japanese-shift-jis} in
422 @code{locale-preferred-coding-systems}, Emacs uses that encoding even
423 though it might normally use @code{japanese-iso-8bit}.
424
425 You can override the language environment chosen at startup with
426 explicit use of the command @code{set-language-environment}, or with
427 customization of @code{current-language-environment} in your init
428 file.
429
430 @kindex C-h L
431 @findex describe-language-environment
432 To display information about the effects of a certain language
433 environment @var{lang-env}, use the command @kbd{C-h L @var{lang-env}
434 @key{RET}} (@code{describe-language-environment}). This tells you
435 which languages this language environment is useful for, and lists the
436 character sets, coding systems, and input methods that go with it. It
437 also shows some sample text to illustrate scripts used in this
438 language environment. If you give an empty input for @var{lang-env},
439 this command describes the chosen language environment.
440 @anchor{Describe Language Environment}
441
442 @vindex set-language-environment-hook
443 You can customize any language environment with the normal hook
444 @code{set-language-environment-hook}. The command
445 @code{set-language-environment} runs that hook after setting up the new
446 language environment. The hook functions can test for a specific
447 language environment by checking the variable
448 @code{current-language-environment}. This hook is where you should
449 put non-default settings for specific language environments, such as
450 coding systems for keyboard input and terminal output, the default
451 input method, etc.
452
453 @vindex exit-language-environment-hook
454 Before it starts to set up the new language environment,
455 @code{set-language-environment} first runs the hook
456 @code{exit-language-environment-hook}. This hook is useful for undoing
457 customizations that were made with @code{set-language-environment-hook}.
458 For instance, if you set up a special key binding in a specific language
459 environment using @code{set-language-environment-hook}, you should set
460 up @code{exit-language-environment-hook} to restore the normal binding
461 for that key.
462
463 @node Input Methods
464 @section Input Methods
465
466 @cindex input methods
467 An @dfn{input method} is a kind of character conversion designed
468 specifically for interactive input. In Emacs, typically each language
469 has its own input method; sometimes several languages that use the same
470 characters can share one input method. A few languages support several
471 input methods.
472
473 The simplest kind of input method works by mapping @acronym{ASCII} letters
474 into another alphabet; this allows you to use one other alphabet
475 instead of @acronym{ASCII}. The Greek and Russian input methods
476 work this way.
477
478 A more powerful technique is composition: converting sequences of
479 characters into one letter. Many European input methods use composition
480 to produce a single non-@acronym{ASCII} letter from a sequence that consists of a
481 letter followed by accent characters (or vice versa). For example, some
482 methods convert the sequence @kbd{o ^} into a single accented letter.
483 These input methods have no special commands of their own; all they do
484 is compose sequences of printing characters.
485
486 The input methods for syllabic scripts typically use mapping followed
487 by composition. The input methods for Thai and Korean work this way.
488 First, letters are mapped into symbols for particular sounds or tone
489 marks; then, sequences of these that make up a whole syllable are
490 mapped into one syllable sign.
491
492 Chinese and Japanese require more complex methods. In Chinese input
493 methods, first you enter the phonetic spelling of a Chinese word (in
494 input method @code{chinese-py}, among others), or a sequence of
495 portions of the character (input methods @code{chinese-4corner} and
496 @code{chinese-sw}, and others). One input sequence typically
497 corresponds to many possible Chinese characters. You select the one
498 you mean using keys such as @kbd{C-f}, @kbd{C-b}, @kbd{C-n},
499 @kbd{C-p} (or the arrow keys), and digits, which have special meanings
500 in this situation.
501
502 The possible characters are conceptually arranged in several rows,
503 with each row holding up to 10 alternatives. Normally, Emacs displays
504 just one row at a time, in the echo area; @code{(@var{i}/@var{j})}
505 appears at the beginning, to indicate that this is the @var{i}th row
506 out of a total of @var{j} rows. Type @kbd{C-n} or @kbd{C-p} to
507 display the next row or the previous row.
508
509 Type @kbd{C-f} and @kbd{C-b} to move forward and backward among
510 the alternatives in the current row. As you do this, Emacs highlights
511 the current alternative with a special color; type @code{C-@key{SPC}}
512 to select the current alternative and use it as input. The
513 alternatives in the row are also numbered; the number appears before
514 the alternative. Typing a number selects the associated alternative
515 of the current row and uses it as input.
516
517 @key{TAB} in these Chinese input methods displays a buffer showing
518 all the possible characters at once; then clicking @kbd{Mouse-2} on
519 one of them selects that alternative. The keys @kbd{C-f}, @kbd{C-b},
520 @kbd{C-n}, @kbd{C-p}, and digits continue to work as usual, but they
521 do the highlighting in the buffer showing the possible characters,
522 rather than in the echo area.
523
524 In Japanese input methods, first you input a whole word using
525 phonetic spelling; then, after the word is in the buffer, Emacs
526 converts it into one or more characters using a large dictionary. One
527 phonetic spelling corresponds to a number of different Japanese words;
528 to select one of them, use @kbd{C-n} and @kbd{C-p} to cycle through
529 the alternatives.
530
531 Sometimes it is useful to cut off input method processing so that the
532 characters you have just entered will not combine with subsequent
533 characters. For example, in input method @code{latin-1-postfix}, the
534 sequence @kbd{o ^} combines to form an @samp{o} with an accent. What if
535 you want to enter them as separate characters?
536
537 One way is to type the accent twice; this is a special feature for
538 entering the separate letter and accent. For example, @kbd{o ^ ^} gives
539 you the two characters @samp{o^}. Another way is to type another letter
540 after the @kbd{o}---something that won't combine with that---and
541 immediately delete it. For example, you could type @kbd{o o @key{DEL}
542 ^} to get separate @samp{o} and @samp{^}.
543
544 Another method, more general but not quite as easy to type, is to use
545 @kbd{C-\ C-\} between two characters to stop them from combining. This
546 is the command @kbd{C-\} (@code{toggle-input-method}) used twice.
547 @ifnottex
548 @xref{Select Input Method}.
549 @end ifnottex
550
551 @cindex incremental search, input method interference
552 @kbd{C-\ C-\} is especially useful inside an incremental search,
553 because it stops waiting for more characters to combine, and starts
554 searching for what you have already entered.
555
556 To find out how to input the character after point using the current
557 input method, type @kbd{C-u C-x =}. @xref{Position Info}.
558
559 @vindex input-method-verbose-flag
560 @vindex input-method-highlight-flag
561 The variables @code{input-method-highlight-flag} and
562 @code{input-method-verbose-flag} control how input methods explain
563 what is happening. If @code{input-method-highlight-flag} is
564 non-@code{nil}, the partial sequence is highlighted in the buffer (for
565 most input methods---some disable this feature). If
566 @code{input-method-verbose-flag} is non-@code{nil}, the list of
567 possible characters to type next is displayed in the echo area (but
568 not when you are in the minibuffer).
569
570 Another facility for typing characters not on your keyboard is by
571 using @kbd{C-x 8 @key{RET}} (@code{ucs-insert}) to insert a single
572 character based on its Unicode name or code-point; see @ref{Inserting
573 Text}.
574
575 @node Select Input Method
576 @section Selecting an Input Method
577
578 @table @kbd
579 @item C-\
580 Enable or disable use of the selected input method (@code{toggle-input-method}).
581
582 @item C-x @key{RET} C-\ @var{method} @key{RET}
583 Select a new input method for the current buffer (@code{set-input-method}).
584
585 @item C-h I @var{method} @key{RET}
586 @itemx C-h C-\ @var{method} @key{RET}
587 @findex describe-input-method
588 @kindex C-h I
589 @kindex C-h C-\
590 Describe the input method @var{method} (@code{describe-input-method}).
591 By default, it describes the current input method (if any). This
592 description should give you the full details of how to use any
593 particular input method.
594
595 @item M-x list-input-methods
596 Display a list of all the supported input methods.
597 @end table
598
599 @findex set-input-method
600 @vindex current-input-method
601 @kindex C-x RET C-\
602 To choose an input method for the current buffer, use @kbd{C-x
603 @key{RET} C-\} (@code{set-input-method}). This command reads the
604 input method name from the minibuffer; the name normally starts with the
605 language environment that it is meant to be used with. The variable
606 @code{current-input-method} records which input method is selected.
607
608 @findex toggle-input-method
609 @kindex C-\
610 Input methods use various sequences of @acronym{ASCII} characters to
611 stand for non-@acronym{ASCII} characters. Sometimes it is useful to
612 turn off the input method temporarily. To do this, type @kbd{C-\}
613 (@code{toggle-input-method}). To reenable the input method, type
614 @kbd{C-\} again.
615
616 If you type @kbd{C-\} and you have not yet selected an input method,
617 it prompts you to specify one. This has the same effect as using
618 @kbd{C-x @key{RET} C-\} to specify an input method.
619
620 When invoked with a numeric argument, as in @kbd{C-u C-\},
621 @code{toggle-input-method} always prompts you for an input method,
622 suggesting the most recently selected one as the default.
623
624 @vindex default-input-method
625 Selecting a language environment specifies a default input method for
626 use in various buffers. When you have a default input method, you can
627 select it in the current buffer by typing @kbd{C-\}. The variable
628 @code{default-input-method} specifies the default input method
629 (@code{nil} means there is none).
630
631 In some language environments, which support several different input
632 methods, you might want to use an input method different from the
633 default chosen by @code{set-language-environment}. You can instruct
634 Emacs to select a different default input method for a certain
635 language environment, if you wish, by using
636 @code{set-language-environment-hook} (@pxref{Language Environments,
637 set-language-environment-hook}). For example:
638
639 @lisp
640 (defun my-chinese-setup ()
641 "Set up my private Chinese environment."
642 (if (equal current-language-environment "Chinese-GB")
643 (setq default-input-method "chinese-tonepy")))
644 (add-hook 'set-language-environment-hook 'my-chinese-setup)
645 @end lisp
646
647 @noindent
648 This sets the default input method to be @code{chinese-tonepy}
649 whenever you choose a Chinese-GB language environment.
650
651 You can instruct Emacs to activate a certain input method
652 automatically. For example:
653
654 @lisp
655 (add-hook 'text-mode-hook
656 (lambda () (set-input-method "german-prefix")))
657 @end lisp
658
659 @noindent
660 This automatically activates the input method ``german-prefix'' in
661 Text mode.
662
663 @findex quail-set-keyboard-layout
664 Some input methods for alphabetic scripts work by (in effect)
665 remapping the keyboard to emulate various keyboard layouts commonly used
666 for those scripts. How to do this remapping properly depends on your
667 actual keyboard layout. To specify which layout your keyboard has, use
668 the command @kbd{M-x quail-set-keyboard-layout}.
669
670 @findex quail-show-key
671 You can use the command @kbd{M-x quail-show-key} to show what key (or
672 key sequence) to type in order to input the character following point,
673 using the selected keyboard layout. The command @kbd{C-u C-x =} also
674 shows that information, in addition to other information about the
675 character.
676
677 @findex list-input-methods
678 @kbd{M-x list-input-methods} displays a list of all the supported
679 input methods. The list gives information about each input method,
680 including the string that stands for it in the mode line.
681
682 @node Coding Systems
683 @section Coding Systems
684 @cindex coding systems
685
686 Users of various languages have established many more-or-less standard
687 coding systems for representing them. Emacs does not use these coding
688 systems internally; instead, it converts from various coding systems to
689 its own system when reading data, and converts the internal coding
690 system to other coding systems when writing data. Conversion is
691 possible in reading or writing files, in sending or receiving from the
692 terminal, and in exchanging data with subprocesses.
693
694 Emacs assigns a name to each coding system. Most coding systems are
695 used for one language, and the name of the coding system starts with
696 the language name. Some coding systems are used for several
697 languages; their names usually start with @samp{iso}. There are also
698 special coding systems, such as @code{no-conversion}, @code{raw-text},
699 and @code{emacs-internal}.
700
701 @cindex international files from DOS/Windows systems
702 A special class of coding systems, collectively known as
703 @dfn{codepages}, is designed to support text encoded by MS-Windows and
704 MS-DOS software. The names of these coding systems are
705 @code{cp@var{nnnn}}, where @var{nnnn} is a 3- or 4-digit number of the
706 codepage. You can use these encodings just like any other coding
707 system; for example, to visit a file encoded in codepage 850, type
708 @kbd{C-x @key{RET} c cp850 @key{RET} C-x C-f @var{filename}
709 @key{RET}}.
710
711 In addition to converting various representations of non-@acronym{ASCII}
712 characters, a coding system can perform end-of-line conversion. Emacs
713 handles three different conventions for how to separate lines in a file:
714 newline (``unix''), carriage-return linefeed (``dos''), and just
715 carriage-return (``mac'').
716
717 @table @kbd
718 @item C-h C @var{coding} @key{RET}
719 Describe coding system @var{coding} (@code{describe-coding-system}).
720
721 @item C-h C @key{RET}
722 Describe the coding systems currently in use.
723
724 @item M-x list-coding-systems
725 Display a list of all the supported coding systems.
726 @end table
727
728 @kindex C-h C
729 @findex describe-coding-system
730 The command @kbd{C-h C} (@code{describe-coding-system}) displays
731 information about particular coding systems, including the end-of-line
732 conversion specified by those coding systems. You can specify a coding
733 system name as the argument; alternatively, with an empty argument, it
734 describes the coding systems currently selected for various purposes,
735 both in the current buffer and as the defaults, and the priority list
736 for recognizing coding systems (@pxref{Recognize Coding}).
737
738 @findex list-coding-systems
739 To display a list of all the supported coding systems, type @kbd{M-x
740 list-coding-systems}. The list gives information about each coding
741 system, including the letter that stands for it in the mode line
742 (@pxref{Mode Line}).
743
744 @cindex end-of-line conversion
745 @cindex line endings
746 @cindex MS-DOS end-of-line conversion
747 @cindex Macintosh end-of-line conversion
748 Each of the coding systems that appear in this list---except for
749 @code{no-conversion}, which means no conversion of any kind---specifies
750 how and whether to convert printing characters, but leaves the choice of
751 end-of-line conversion to be decided based on the contents of each file.
752 For example, if the file appears to use the sequence carriage-return
753 linefeed to separate lines, DOS end-of-line conversion will be used.
754
755 Each of the listed coding systems has three variants, which specify
756 exactly what to do for end-of-line conversion:
757
758 @table @code
759 @item @dots{}-unix
760 Don't do any end-of-line conversion; assume the file uses
761 newline to separate lines. (This is the convention normally used
762 on Unix and GNU systems, and Mac OS X.)
763
764 @item @dots{}-dos
765 Assume the file uses carriage-return linefeed to separate lines, and do
766 the appropriate conversion. (This is the convention normally used on
767 Microsoft systems.@footnote{It is also specified for MIME @samp{text/*}
768 bodies and in other network transport contexts. It is different
769 from the SGML reference syntax record-start/record-end format, which
770 Emacs doesn't support directly.})
771
772 @item @dots{}-mac
773 Assume the file uses carriage-return to separate lines, and do the
774 appropriate conversion. (This was the convention used on the
775 Macintosh system prior to OS X.)
776 @end table
777
778 These variant coding systems are omitted from the
779 @code{list-coding-systems} display for brevity, since they are entirely
780 predictable. For example, the coding system @code{iso-latin-1} has
781 variants @code{iso-latin-1-unix}, @code{iso-latin-1-dos} and
782 @code{iso-latin-1-mac}.
783
784 @cindex @code{undecided}, coding system
785 The coding systems @code{unix}, @code{dos}, and @code{mac} are
786 aliases for @code{undecided-unix}, @code{undecided-dos}, and
787 @code{undecided-mac}, respectively. These coding systems specify only
788 the end-of-line conversion, and leave the character code conversion to
789 be deduced from the text itself.
790
791 @cindex @code{raw-text}, coding system
792 The coding system @code{raw-text} is good for a file which is mainly
793 @acronym{ASCII} text, but may contain byte values above 127 that are
794 not meant to encode non-@acronym{ASCII} characters. With
795 @code{raw-text}, Emacs copies those byte values unchanged, and sets
796 @code{enable-multibyte-characters} to @code{nil} in the current buffer
797 so that they will be interpreted properly. @code{raw-text} handles
798 end-of-line conversion in the usual way, based on the data
799 encountered, and has the usual three variants to specify the kind of
800 end-of-line conversion to use.
801
802 @cindex @code{no-conversion}, coding system
803 In contrast, the coding system @code{no-conversion} specifies no
804 character code conversion at all---none for non-@acronym{ASCII} byte values and
805 none for end of line. This is useful for reading or writing binary
806 files, tar files, and other files that must be examined verbatim. It,
807 too, sets @code{enable-multibyte-characters} to @code{nil}.
808
809 The easiest way to edit a file with no conversion of any kind is with
810 the @kbd{M-x find-file-literally} command. This uses
811 @code{no-conversion}, and also suppresses other Emacs features that
812 might convert the file contents before you see them. @xref{Visiting}.
813
814 @cindex @code{emacs-internal}, coding system
815 The coding system @code{emacs-internal} (or @code{utf-8-emacs},
816 which is equivalent) means that the file contains non-@acronym{ASCII}
817 characters stored with the internal Emacs encoding. This coding
818 system handles end-of-line conversion based on the data encountered,
819 and has the usual three variants to specify the kind of end-of-line
820 conversion.
821
822 @node Recognize Coding
823 @section Recognizing Coding Systems
824
825 Whenever Emacs reads a given piece of text, it tries to recognize
826 which coding system to use. This applies to files being read, output
827 from subprocesses, text from X selections, etc. Emacs can select the
828 right coding system automatically most of the time---once you have
829 specified your preferences.
830
831 Some coding systems can be recognized or distinguished by which byte
832 sequences appear in the data. However, there are coding systems that
833 cannot be distinguished, not even potentially. For example, there is no
834 way to distinguish between Latin-1 and Latin-2; they use the same byte
835 values with different meanings.
836
837 Emacs handles this situation by means of a priority list of coding
838 systems. Whenever Emacs reads a file, if you do not specify the coding
839 system to use, Emacs checks the data against each coding system,
840 starting with the first in priority and working down the list, until it
841 finds a coding system that fits the data. Then it converts the file
842 contents assuming that they are represented in this coding system.
843
844 The priority list of coding systems depends on the selected language
845 environment (@pxref{Language Environments}). For example, if you use
846 French, you probably want Emacs to prefer Latin-1 to Latin-2; if you use
847 Czech, you probably want Latin-2 to be preferred. This is one of the
848 reasons to specify a language environment.
849
850 @findex prefer-coding-system
851 However, you can alter the coding system priority list in detail
852 with the command @kbd{M-x prefer-coding-system}. This command reads
853 the name of a coding system from the minibuffer, and adds it to the
854 front of the priority list, so that it is preferred to all others. If
855 you use this command several times, each use adds one element to the
856 front of the priority list.
857
858 If you use a coding system that specifies the end-of-line conversion
859 type, such as @code{iso-8859-1-dos}, what this means is that Emacs
860 should attempt to recognize @code{iso-8859-1} with priority, and should
861 use DOS end-of-line conversion when it does recognize @code{iso-8859-1}.
862
863 @vindex file-coding-system-alist
864 Sometimes a file name indicates which coding system to use for the
865 file. The variable @code{file-coding-system-alist} specifies this
866 correspondence. There is a special function
867 @code{modify-coding-system-alist} for adding elements to this list. For
868 example, to read and write all @samp{.txt} files using the coding system
869 @code{chinese-iso-8bit}, you can execute this Lisp expression:
870
871 @smallexample
872 (modify-coding-system-alist 'file "\\.txt\\'" 'chinese-iso-8bit)
873 @end smallexample
874
875 @noindent
876 The first argument should be @code{file}, the second argument should be
877 a regular expression that determines which files this applies to, and
878 the third argument says which coding system to use for these files.
879
880 @vindex inhibit-eol-conversion
881 @cindex DOS-style end-of-line display
882 Emacs recognizes which kind of end-of-line conversion to use based on
883 the contents of the file: if it sees only carriage-returns, or only
884 carriage-return linefeed sequences, then it chooses the end-of-line
885 conversion accordingly. You can inhibit the automatic use of
886 end-of-line conversion by setting the variable @code{inhibit-eol-conversion}
887 to non-@code{nil}. If you do that, DOS-style files will be displayed
888 with the @samp{^M} characters visible in the buffer; some people
889 prefer this to the more subtle @samp{(DOS)} end-of-line type
890 indication near the left edge of the mode line (@pxref{Mode Line,
891 eol-mnemonic}).
892
893 @vindex inhibit-iso-escape-detection
894 @cindex escape sequences in files
895 By default, the automatic detection of coding system is sensitive to
896 escape sequences. If Emacs sees a sequence of characters that begin
897 with an escape character, and the sequence is valid as an ISO-2022
898 code, that tells Emacs to use one of the ISO-2022 encodings to decode
899 the file.
900
901 However, there may be cases that you want to read escape sequences
902 in a file as is. In such a case, you can set the variable
903 @code{inhibit-iso-escape-detection} to non-@code{nil}. Then the code
904 detection ignores any escape sequences, and never uses an ISO-2022
905 encoding. The result is that all escape sequences become visible in
906 the buffer.
907
908 The default value of @code{inhibit-iso-escape-detection} is
909 @code{nil}. We recommend that you not change it permanently, only for
910 one specific operation. That's because some Emacs Lisp source files
911 in the Emacs distribution contain non-@acronym{ASCII} characters encoded in the
912 coding system @code{iso-2022-7bit}, and they won't be
913 decoded correctly when you visit those files if you suppress the
914 escape sequence detection.
915 @c I count a grand total of 3 such files, so is the above really true?
916
917 @vindex auto-coding-alist
918 @vindex auto-coding-regexp-alist
919 The variables @code{auto-coding-alist} and
920 @code{auto-coding-regexp-alist} are
921 the strongest way to specify the coding system for certain patterns of
922 file names, or for files containing certain patterns, respectively.
923 These variables even override @samp{-*-coding:-*-} tags in the file
924 itself (@pxref{Specify Coding}). For example, Emacs
925 uses @code{auto-coding-alist} for tar and archive files, to prevent it
926 from being confused by a @samp{-*-coding:-*-} tag in a member of the
927 archive and thinking it applies to the archive file as a whole.
928 @ignore
929 @c This describes old-style BABYL files, which are no longer relevant.
930 Likewise, Emacs uses @code{auto-coding-regexp-alist} to ensure that
931 RMAIL files, whose names in general don't match any particular
932 pattern, are decoded correctly.
933 @end ignore
934
935 @vindex auto-coding-functions
936 Another way to specify a coding system is with the variable
937 @code{auto-coding-functions}. For example, one of the builtin
938 @code{auto-coding-functions} detects the encoding for XML files.
939 Unlike the previous two, this variable does not override any
940 @samp{-*-coding:-*-} tag.
941
942 @c FIXME? This seems somewhat out of place. Move to the Rmail section?
943 @vindex rmail-decode-mime-charset
944 @vindex rmail-file-coding-system
945 When you get new mail in Rmail, each message is translated
946 automatically from the coding system it is written in, as if it were a
947 separate file. This uses the priority list of coding systems that you
948 have specified. If a MIME message specifies a character set, Rmail
949 obeys that specification. For reading and saving Rmail files
950 themselves, Emacs uses the coding system specified by the variable
951 @code{rmail-file-coding-system}. The default value is @code{nil},
952 which means that Rmail files are not translated (they are read and
953 written in the Emacs internal character code).
954
955 @node Specify Coding
956 @section Specifying a File's Coding System
957
958 If Emacs recognizes the encoding of a file incorrectly, you can
959 reread the file using the correct coding system with @kbd{C-x
960 @key{RET} r} (@code{revert-buffer-with-coding-system}). This command
961 prompts for the coding system to use. To see what coding system Emacs
962 actually used to decode the file, look at the coding system mnemonic
963 letter near the left edge of the mode line (@pxref{Mode Line}), or
964 type @kbd{C-h C} (@code{describe-coding-system}).
965
966 @vindex coding
967 You can specify the coding system for a particular file in the file
968 itself, using the @w{@samp{-*-@dots{}-*-}} construct at the beginning,
969 or a local variables list at the end (@pxref{File Variables}). You do
970 this by defining a value for the ``variable'' named @code{coding}.
971 Emacs does not really have a variable @code{coding}; instead of
972 setting a variable, this uses the specified coding system for the
973 file. For example, @samp{-*-mode: C; coding: latin-1;-*-} specifies
974 use of the Latin-1 coding system, as well as C mode. When you specify
975 the coding explicitly in the file, that overrides
976 @code{file-coding-system-alist}.
977
978 @node Output Coding
979 @section Choosing Coding Systems for Output
980
981 @vindex buffer-file-coding-system
982 Once Emacs has chosen a coding system for a buffer, it stores that
983 coding system in @code{buffer-file-coding-system}. That makes it the
984 default for operations that write from this buffer into a file, such
985 as @code{save-buffer} and @code{write-region}. You can specify a
986 different coding system for further file output from the buffer using
987 @code{set-buffer-file-coding-system} (@pxref{Text Coding}).
988
989 You can insert any character Emacs supports into any Emacs buffer,
990 but most coding systems can only handle a subset of these characters.
991 Therefore, it's possible that the characters you insert cannot be
992 encoded with the coding system that will be used to save the buffer.
993 For example, you could visit a text file in Polish, encoded in
994 @code{iso-8859-2}, and add some Russian words to it. When you save
995 that buffer, Emacs cannot use the current value of
996 @code{buffer-file-coding-system}, because the characters you added
997 cannot be encoded by that coding system.
998
999 When that happens, Emacs tries the most-preferred coding system (set
1000 by @kbd{M-x prefer-coding-system} or @kbd{M-x
1001 set-language-environment}). If that coding system can safely encode
1002 all of the characters in the buffer, Emacs uses it, and stores its
1003 value in @code{buffer-file-coding-system}. Otherwise, Emacs displays
1004 a list of coding systems suitable for encoding the buffer's contents,
1005 and asks you to choose one of those coding systems.
1006
1007 If you insert the unsuitable characters in a mail message, Emacs
1008 behaves a bit differently. It additionally checks whether the
1009 @c What determines this?
1010 most-preferred coding system is recommended for use in MIME messages;
1011 if not, Emacs tells you that the most-preferred coding system is not
1012 recommended and prompts you for another coding system. This is so you
1013 won't inadvertently send a message encoded in a way that your
1014 recipient's mail software will have difficulty decoding. (You can
1015 still use an unsuitable coding system if you type its name in response
1016 to the question.)
1017
1018 @c It seems that select-message-coding-system does this.
1019 @c Both sendmail.el and smptmail.el call it; i.e. smtpmail.el still
1020 @c obeys sendmail-coding-system.
1021 @vindex sendmail-coding-system
1022 When you send a mail message (@pxref{Sending Mail}),
1023 Emacs has four different ways to determine the coding system to use
1024 for encoding the message text. It tries the buffer's own value of
1025 @code{buffer-file-coding-system}, if that is non-@code{nil}.
1026 Otherwise, it uses the value of @code{sendmail-coding-system}, if that
1027 is non-@code{nil}. The third way is to use the default coding system
1028 for new files, which is controlled by your choice of language
1029 @c i.e., default-sendmail-coding-system
1030 environment, if that is non-@code{nil}. If all of these three values
1031 are @code{nil}, Emacs encodes outgoing mail using the Latin-1 coding
1032 system.
1033 @c FIXME? Where does the Latin-1 default come in?
1034
1035 @node Text Coding
1036 @section Specifying a Coding System for File Text
1037
1038 In cases where Emacs does not automatically choose the right coding
1039 system for a file's contents, you can use these commands to specify
1040 one:
1041
1042 @table @kbd
1043 @item C-x @key{RET} f @var{coding} @key{RET}
1044 Use coding system @var{coding} to save or revisit the file in
1045 the current buffer (@code{set-buffer-file-coding-system}).
1046
1047 @item C-x @key{RET} c @var{coding} @key{RET}
1048 Specify coding system @var{coding} for the immediately following
1049 command (@code{universal-coding-system-argument}).
1050
1051 @item C-x @key{RET} r @var{coding} @key{RET}
1052 Revisit the current file using the coding system @var{coding}
1053 (@code{revert-buffer-with-coding-system}).
1054
1055 @item M-x recode-region @key{RET} @var{right} @key{RET} @var{wrong} @key{RET}
1056 Convert a region that was decoded using coding system @var{wrong},
1057 decoding it using coding system @var{right} instead.
1058 @end table
1059
1060 @kindex C-x RET f
1061 @findex set-buffer-file-coding-system
1062 The command @kbd{C-x @key{RET} f}
1063 (@code{set-buffer-file-coding-system}) sets the file coding system for
1064 the current buffer---in other words, it says which coding system to
1065 use when saving or reverting the visited file. You specify which
1066 coding system using the minibuffer. If you specify a coding system
1067 that cannot handle all of the characters in the buffer, Emacs warns
1068 you about the troublesome characters when you actually save the
1069 buffer.
1070
1071 @cindex specify end-of-line conversion
1072 You can also use this command to specify the end-of-line conversion
1073 (@pxref{Coding Systems, end-of-line conversion}) for encoding the
1074 current buffer. For example, @kbd{C-x @key{RET} f dos @key{RET}} will
1075 cause Emacs to save the current buffer's text with DOS-style
1076 carriage-return linefeed line endings.
1077
1078 @kindex C-x RET c
1079 @findex universal-coding-system-argument
1080 Another way to specify the coding system for a file is when you visit
1081 the file. First use the command @kbd{C-x @key{RET} c}
1082 (@code{universal-coding-system-argument}); this command uses the
1083 minibuffer to read a coding system name. After you exit the minibuffer,
1084 the specified coding system is used for @emph{the immediately following
1085 command}.
1086
1087 So if the immediately following command is @kbd{C-x C-f}, for example,
1088 it reads the file using that coding system (and records the coding
1089 system for when you later save the file). Or if the immediately following
1090 command is @kbd{C-x C-w}, it writes the file using that coding system.
1091 When you specify the coding system for saving in this way, instead
1092 of with @kbd{C-x @key{RET} f}, there is no warning if the buffer
1093 contains characters that the coding system cannot handle.
1094
1095 Other file commands affected by a specified coding system include
1096 @kbd{C-x i} and @kbd{C-x C-v}, as well as the other-window variants
1097 of @kbd{C-x C-f}. @kbd{C-x @key{RET} c} also affects commands that
1098 start subprocesses, including @kbd{M-x shell} (@pxref{Shell}). If the
1099 immediately following command does not use the coding system, then
1100 @kbd{C-x @key{RET} c} ultimately has no effect.
1101
1102 An easy way to visit a file with no conversion is with the @kbd{M-x
1103 find-file-literally} command. @xref{Visiting}.
1104
1105 The default value of the variable @code{buffer-file-coding-system}
1106 specifies the choice of coding system to use when you create a new file.
1107 It applies when you find a new file, and when you create a buffer and
1108 then save it in a file. Selecting a language environment typically sets
1109 this variable to a good choice of default coding system for that language
1110 environment.
1111
1112 @kindex C-x RET r
1113 @findex revert-buffer-with-coding-system
1114 If you visit a file with a wrong coding system, you can correct this
1115 with @kbd{C-x @key{RET} r} (@code{revert-buffer-with-coding-system}).
1116 This visits the current file again, using a coding system you specify.
1117
1118 @findex recode-region
1119 If a piece of text has already been inserted into a buffer using the
1120 wrong coding system, you can redo the decoding of it using @kbd{M-x
1121 recode-region}. This prompts you for the proper coding system, then
1122 for the wrong coding system that was actually used, and does the
1123 conversion. It first encodes the region using the wrong coding system,
1124 then decodes it again using the proper coding system.
1125
1126 @node Communication Coding
1127 @section Coding Systems for Interprocess Communication
1128
1129 This section explains how to specify coding systems for use
1130 in communication with other processes.
1131
1132 @table @kbd
1133 @item C-x @key{RET} x @var{coding} @key{RET}
1134 Use coding system @var{coding} for transferring selections to and from
1135 other window-based applications (@code{set-selection-coding-system}).
1136
1137 @item C-x @key{RET} X @var{coding} @key{RET}
1138 Use coding system @var{coding} for transferring @emph{one}
1139 selection---the next one---to or from another window-based application
1140 (@code{set-next-selection-coding-system}).
1141
1142 @item C-x @key{RET} p @var{input-coding} @key{RET} @var{output-coding} @key{RET}
1143 Use coding systems @var{input-coding} and @var{output-coding} for
1144 subprocess input and output in the current buffer
1145 (@code{set-buffer-process-coding-system}).
1146 @end table
1147
1148 @kindex C-x RET x
1149 @kindex C-x RET X
1150 @findex set-selection-coding-system
1151 @findex set-next-selection-coding-system
1152 The command @kbd{C-x @key{RET} x} (@code{set-selection-coding-system})
1153 specifies the coding system for sending selected text to other windowing
1154 applications, and for receiving the text of selections made in other
1155 applications. This command applies to all subsequent selections, until
1156 you override it by using the command again. The command @kbd{C-x
1157 @key{RET} X} (@code{set-next-selection-coding-system}) specifies the
1158 coding system for the next selection made in Emacs or read by Emacs.
1159
1160 @vindex x-select-request-type
1161 The variable @code{x-select-request-type} specifies the data type to
1162 request from the X Window System for receiving text selections from
1163 other applications. If the value is @code{nil} (the default), Emacs
1164 tries @code{UTF8_STRING} and @code{COMPOUND_TEXT}, in this order, and
1165 uses various heuristics to choose the more appropriate of the two
1166 results; if none of these succeed, Emacs falls back on @code{STRING}.
1167 If the value of @code{x-select-request-type} is one of the symbols
1168 @code{COMPOUND_TEXT}, @code{UTF8_STRING}, @code{STRING}, or
1169 @code{TEXT}, Emacs uses only that request type. If the value is a
1170 list of some of these symbols, Emacs tries only the request types in
1171 the list, in order, until one of them succeeds, or until the list is
1172 exhausted.
1173
1174 @kindex C-x RET p
1175 @findex set-buffer-process-coding-system
1176 The command @kbd{C-x @key{RET} p} (@code{set-buffer-process-coding-system})
1177 specifies the coding system for input and output to a subprocess. This
1178 command applies to the current buffer; normally, each subprocess has its
1179 own buffer, and thus you can use this command to specify translation to
1180 and from a particular subprocess by giving the command in the
1181 corresponding buffer.
1182
1183 You can also use @kbd{C-x @key{RET} c}
1184 (@code{universal-coding-system-argument}) just before the command that
1185 runs or starts a subprocess, to specify the coding system for
1186 communicating with that subprocess. @xref{Text Coding}.
1187
1188 The default for translation of process input and output depends on the
1189 current language environment.
1190
1191 @vindex locale-coding-system
1192 @cindex decoding non-@acronym{ASCII} keyboard input on X
1193 The variable @code{locale-coding-system} specifies a coding system
1194 to use when encoding and decoding system strings such as system error
1195 messages and @code{format-time-string} formats and time stamps. That
1196 coding system is also used for decoding non-@acronym{ASCII} keyboard
1197 input on the X Window System. You should choose a coding system that is compatible
1198 with the underlying system's text representation, which is normally
1199 specified by one of the environment variables @env{LC_ALL},
1200 @env{LC_CTYPE}, and @env{LANG}. (The first one, in the order
1201 specified above, whose value is nonempty is the one that determines
1202 the text representation.)
1203
1204 @node File Name Coding
1205 @section Coding Systems for File Names
1206
1207 @table @kbd
1208 @item C-x @key{RET} F @var{coding} @key{RET}
1209 Use coding system @var{coding} for encoding and decoding file
1210 names (@code{set-file-name-coding-system}).
1211 @end table
1212
1213 @findex set-file-name-coding-system
1214 @kindex C-x @key{RET} F
1215 @cindex file names with non-@acronym{ASCII} characters
1216 The command @kbd{C-x @key{RET} F} (@code{set-file-name-coding-system})
1217 specifies a coding system to use for encoding file @emph{names}. It
1218 has no effect on reading and writing the @emph{contents} of files.
1219
1220 @vindex file-name-coding-system
1221 In fact, all this command does is set the value of the variable
1222 @code{file-name-coding-system}. If you set the variable to a coding
1223 system name (as a Lisp symbol or a string), Emacs encodes file names
1224 using that coding system for all file operations. This makes it
1225 possible to use non-@acronym{ASCII} characters in file names---or, at
1226 least, those non-@acronym{ASCII} characters that the specified coding
1227 system can encode.
1228
1229 If @code{file-name-coding-system} is @code{nil}, Emacs uses a
1230 default coding system determined by the selected language environment,
1231 and stored in the @code{default-file-name-coding-system} variable.
1232 @c FIXME? Is this correct? What is the "default language environment"?
1233 In the default language environment, non-@acronym{ASCII} characters in
1234 file names are not encoded specially; they appear in the file system
1235 using the internal Emacs representation.
1236
1237 @strong{Warning:} if you change @code{file-name-coding-system} (or the
1238 language environment) in the middle of an Emacs session, problems can
1239 result if you have already visited files whose names were encoded using
1240 the earlier coding system and cannot be encoded (or are encoded
1241 differently) under the new coding system. If you try to save one of
1242 these buffers under the visited file name, saving may use the wrong file
1243 name, or it may encounter an error. If such a problem happens, use @kbd{C-x
1244 C-w} to specify a new file name for that buffer.
1245
1246 @findex recode-file-name
1247 If a mistake occurs when encoding a file name, use the command
1248 @kbd{M-x recode-file-name} to change the file name's coding
1249 system. This prompts for an existing file name, its old coding
1250 system, and the coding system to which you wish to convert.
1251
1252 @node Terminal Coding
1253 @section Coding Systems for Terminal I/O
1254
1255 @table @kbd
1256 @item C-x @key{RET} t @var{coding} @key{RET}
1257 Use coding system @var{coding} for terminal output
1258 (@code{set-terminal-coding-system}).
1259
1260 @item C-x @key{RET} k @var{coding} @key{RET}
1261 Use coding system @var{coding} for keyboard input
1262 (@code{set-keyboard-coding-system}).
1263 @end table
1264
1265 @kindex C-x RET t
1266 @findex set-terminal-coding-system
1267 The command @kbd{C-x @key{RET} t} (@code{set-terminal-coding-system})
1268 specifies the coding system for terminal output. If you specify a
1269 character code for terminal output, all characters output to the
1270 terminal are translated into that coding system.
1271
1272 This feature is useful for certain character-only terminals built to
1273 support specific languages or character sets---for example, European
1274 terminals that support one of the ISO Latin character sets. You need to
1275 specify the terminal coding system when using multibyte text, so that
1276 Emacs knows which characters the terminal can actually handle.
1277
1278 By default, output to the terminal is not translated at all, unless
1279 Emacs can deduce the proper coding system from your terminal type or
1280 your locale specification (@pxref{Language Environments}).
1281
1282 @kindex C-x RET k
1283 @findex set-keyboard-coding-system
1284 @vindex keyboard-coding-system
1285 The command @kbd{C-x @key{RET} k} (@code{set-keyboard-coding-system}),
1286 or the variable @code{keyboard-coding-system}, specifies the coding
1287 system for keyboard input. Character-code translation of keyboard
1288 input is useful for terminals with keys that send non-@acronym{ASCII}
1289 graphic characters---for example, some terminals designed for ISO
1290 Latin-1 or subsets of it.
1291
1292 By default, keyboard input is translated based on your system locale
1293 setting. If your terminal does not really support the encoding
1294 implied by your locale (for example, if you find it inserts a
1295 non-@acronym{ASCII} character if you type @kbd{M-i}), you will need to set
1296 @code{keyboard-coding-system} to @code{nil} to turn off encoding.
1297 You can do this by putting
1298
1299 @lisp
1300 (set-keyboard-coding-system nil)
1301 @end lisp
1302
1303 @noindent
1304 in your init file.
1305
1306 There is a similarity between using a coding system translation for
1307 keyboard input, and using an input method: both define sequences of
1308 keyboard input that translate into single characters. However, input
1309 methods are designed to be convenient for interactive use by humans, and
1310 the sequences that are translated are typically sequences of @acronym{ASCII}
1311 printing characters. Coding systems typically translate sequences of
1312 non-graphic characters.
1313
1314 @node Fontsets
1315 @section Fontsets
1316 @cindex fontsets
1317
1318 A font typically defines shapes for a single alphabet or script.
1319 Therefore, displaying the entire range of scripts that Emacs supports
1320 requires a collection of many fonts. In Emacs, such a collection is
1321 called a @dfn{fontset}. A fontset is defined by a list of font specs,
1322 each assigned to handle a range of character codes, and may fall back
1323 on another fontset for characters which are not covered by the fonts
1324 it specifies.
1325
1326 Each fontset has a name, like a font. However, while fonts are
1327 stored in the system and the available font names are defined by the
1328 system, fontsets are defined within Emacs itself. Once you have
1329 defined a fontset, you can use it within Emacs by specifying its name,
1330 anywhere that you could use a single font. Of course, Emacs fontsets
1331 can use only the fonts that the system supports; if certain characters
1332 appear on the screen as hollow boxes, this means that the fontset in
1333 use for them has no font for those characters.@footnote{The Emacs
1334 installation instructions have information on additional font
1335 support.}
1336
1337 Emacs creates three fontsets automatically: the @dfn{standard
1338 fontset}, the @dfn{startup fontset} and the @dfn{default fontset}.
1339 The default fontset is most likely to have fonts for a wide variety of
1340 non-@acronym{ASCII} characters and is the default fallback for the
1341 other two fontsets, and if you set a default font rather than fontset.
1342 However it does not specify font family names, so results can be
1343 somewhat random if you use it directly. You can specify use of a
1344 specific fontset with the @samp{-fn} option. For example,
1345
1346 @example
1347 emacs -fn fontset-standard
1348 @end example
1349
1350 @noindent
1351 You can also specify a fontset with the @samp{Font} resource (@pxref{X
1352 Resources}).
1353
1354 If no fontset is specified for use, then Emacs uses an
1355 @acronym{ASCII} font, with @samp{fontset-default} as a fallback for
1356 characters the font does not cover. The standard fontset is only used if
1357 explicitly requested, despite its name.
1358
1359 A fontset does not necessarily specify a font for every character
1360 code. If a fontset specifies no font for a certain character, or if
1361 it specifies a font that does not exist on your system, then it cannot
1362 display that character properly. It will display that character as a
1363 hex code or thin space or an empty box instead. (@xref{Text Display, ,
1364 glyphless characters}, for details.)
1365
1366 @node Defining Fontsets
1367 @section Defining fontsets
1368
1369 @vindex standard-fontset-spec
1370 @vindex w32-standard-fontset-spec
1371 @vindex ns-standard-fontset-spec
1372 @cindex standard fontset
1373 When running on X, Emacs creates a standard fontset automatically according to the value
1374 of @code{standard-fontset-spec}. This fontset's name is
1375
1376 @example
1377 -*-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-standard
1378 @end example
1379
1380 @noindent
1381 or just @samp{fontset-standard} for short.
1382
1383 On GNUstep and Mac, fontset-standard is created using the value of
1384 @code{ns-standard-fontset-spec}, and on Windows it is
1385 created using the value of @code{w32-standard-fontset-spec}.
1386
1387 Bold, italic, and bold-italic variants of the standard fontset are
1388 created automatically. Their names have @samp{bold} instead of
1389 @samp{medium}, or @samp{i} instead of @samp{r}, or both.
1390
1391 @cindex startup fontset
1392 Emacs generates a fontset automatically, based on any default
1393 @acronym{ASCII} font that you specify with the @samp{Font} resource or
1394 the @samp{-fn} argument, or the default font that Emacs found when it
1395 started. This is the @dfn{startup fontset} and its name is
1396 @code{fontset-startup}. It does this by replacing the
1397 @var{charset_registry} field with @samp{fontset}, and replacing
1398 @var{charset_encoding} field with @samp{startup}, then using the
1399 resulting string to specify a fontset.
1400
1401 For instance, if you start Emacs this way,
1402
1403 @example
1404 emacs -fn "*courier-medium-r-normal--14-140-*-iso8859-1"
1405 @end example
1406
1407 @noindent
1408 Emacs generates the following fontset and uses it for the initial X
1409 window frame:
1410
1411 @example
1412 -*-courier-medium-r-normal-*-14-140-*-*-*-*-fontset-startup
1413 @end example
1414
1415 The startup fontset will use the font that you specify or a variant
1416 with a different registry and encoding for all the characters which
1417 are supported by that font, and fallback on @samp{fontset-default} for
1418 other characters.
1419
1420 With the X resource @samp{Emacs.Font}, you can specify a fontset name
1421 just like an actual font name. But be careful not to specify a fontset
1422 name in a wildcard resource like @samp{Emacs*Font}---that wildcard
1423 specification matches various other resources, such as for menus, and
1424 menus cannot handle fontsets.
1425
1426 You can specify additional fontsets using X resources named
1427 @samp{Fontset-@var{n}}, where @var{n} is an integer starting from 0.
1428 The resource value should have this form:
1429
1430 @smallexample
1431 @var{fontpattern}, @r{[}@var{charset}:@var{font}@r{]@dots{}}
1432 @end smallexample
1433
1434 @noindent
1435 @var{fontpattern} should have the form of a standard X font name, except
1436 for the last two fields. They should have the form
1437 @samp{fontset-@var{alias}}.
1438
1439 The fontset has two names, one long and one short. The long name is
1440 @var{fontpattern}. The short name is @samp{fontset-@var{alias}}. You
1441 can refer to the fontset by either name.
1442
1443 The construct @samp{@var{charset}:@var{font}} specifies which font to
1444 use (in this fontset) for one particular character set. Here,
1445 @var{charset} is the name of a character set, and @var{font} is the
1446 font to use for that character set. You can use this construct any
1447 number of times in defining one fontset.
1448
1449 For the other character sets, Emacs chooses a font based on
1450 @var{fontpattern}. It replaces @samp{fontset-@var{alias}} with values
1451 that describe the character set. For the @acronym{ASCII} character font,
1452 @samp{fontset-@var{alias}} is replaced with @samp{ISO8859-1}.
1453
1454 In addition, when several consecutive fields are wildcards, Emacs
1455 collapses them into a single wildcard. This is to prevent use of
1456 auto-scaled fonts. Fonts made by scaling larger fonts are not usable
1457 for editing, and scaling a smaller font is not useful because it is
1458 better to use the smaller font in its own size, which is what Emacs
1459 does.
1460
1461 Thus if @var{fontpattern} is this,
1462
1463 @example
1464 -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24
1465 @end example
1466
1467 @noindent
1468 the font specification for @acronym{ASCII} characters would be this:
1469
1470 @example
1471 -*-fixed-medium-r-normal-*-24-*-ISO8859-1
1472 @end example
1473
1474 @noindent
1475 and the font specification for Chinese GB2312 characters would be this:
1476
1477 @example
1478 -*-fixed-medium-r-normal-*-24-*-gb2312*-*
1479 @end example
1480
1481 You may not have any Chinese font matching the above font
1482 specification. Most X distributions include only Chinese fonts that
1483 have @samp{song ti} or @samp{fangsong ti} in @var{family} field. In
1484 such a case, @samp{Fontset-@var{n}} can be specified as below:
1485
1486 @smallexample
1487 Emacs.Fontset-0: -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24,\
1488 chinese-gb2312:-*-*-medium-r-normal-*-24-*-gb2312*-*
1489 @end smallexample
1490
1491 @noindent
1492 Then, the font specifications for all but Chinese GB2312 characters have
1493 @samp{fixed} in the @var{family} field, and the font specification for
1494 Chinese GB2312 characters has a wild card @samp{*} in the @var{family}
1495 field.
1496
1497 @findex create-fontset-from-fontset-spec
1498 The function that processes the fontset resource value to create the
1499 fontset is called @code{create-fontset-from-fontset-spec}. You can also
1500 call this function explicitly to create a fontset.
1501
1502 @xref{Fonts}, for more information about font naming.
1503
1504 @node Modifying Fontsets
1505 @section Modifying Fontsets
1506 @cindex fontsets, modifying
1507 @findex set-fontset-font
1508
1509 Fontsets do not always have to be created from scratch. If only
1510 minor changes are required it may be easier to modify an existing
1511 fontset. Modifying @samp{fontset-default} will also affect other
1512 fontsets that use it as a fallback, so can be an effective way of
1513 fixing problems with the fonts that Emacs chooses for a particular
1514 script.
1515
1516 Fontsets can be modified using the function @code{set-fontset-font},
1517 specifying a character, a charset, a script, or a range of characters
1518 to modify the font for, and a font-spec for the font to be used. Some
1519 examples are:
1520
1521 @example
1522 ;; Use Liberation Mono for latin-3 charset.
1523 (set-fontset-font "fontset-default" 'iso-8859-3
1524 "Liberation Mono")
1525
1526 ;; Prefer a big5 font for han characters
1527 (set-fontset-font "fontset-default"
1528 'han (font-spec :registry "big5")
1529 nil 'prepend)
1530
1531 ;; Use DejaVu Sans Mono as a fallback in fontset-startup
1532 ;; before resorting to fontset-default.
1533 (set-fontset-font "fontset-startup" nil "DejaVu Sans Mono"
1534 nil 'append)
1535
1536 ;; Use MyPrivateFont for the Unicode private use area.
1537 (set-fontset-font "fontset-default" '(#xe000 . #xf8ff)
1538 "MyPrivateFont")
1539
1540 @end example
1541
1542
1543 @node Undisplayable Characters
1544 @section Undisplayable Characters
1545
1546 There may be a some non-@acronym{ASCII} characters that your
1547 terminal cannot display. Most text terminals support just a single
1548 character set (use the variable @code{default-terminal-coding-system}
1549 (@pxref{Terminal Coding}) to tell Emacs which one); characters which
1550 can't be encoded in that coding system are displayed as @samp{?} by
1551 default.
1552
1553 Graphical displays can display a broader range of characters, but
1554 you may not have fonts installed for all of them; characters that have
1555 no font appear as a hollow box.
1556
1557 If you use Latin-1 characters but your terminal can't display
1558 Latin-1, you can arrange to display mnemonic @acronym{ASCII} sequences
1559 instead, e.g.@: @samp{"o} for o-umlaut. Load the library
1560 @file{iso-ascii} to do this.
1561
1562 @vindex latin1-display
1563 If your terminal can display Latin-1, you can display characters
1564 from other European character sets using a mixture of equivalent
1565 Latin-1 characters and @acronym{ASCII} mnemonics. Customize the variable
1566 @code{latin1-display} to enable this. The mnemonic @acronym{ASCII}
1567 sequences mostly correspond to those of the prefix input methods.
1568
1569 @node Unibyte Mode
1570 @section Unibyte Editing Mode
1571
1572 @cindex European character sets
1573 @cindex accented characters
1574 @cindex ISO Latin character sets
1575 @cindex Unibyte operation
1576 The ISO 8859 Latin-@var{n} character sets define character codes in
1577 the range 0240 to 0377 octal (160 to 255 decimal) to handle the
1578 accented letters and punctuation needed by various European languages
1579 (and some non-European ones). Note that Emacs considers bytes with
1580 codes in this range as raw bytes, not as characters, even in a unibyte
1581 session, i.e.@: if you disable multibyte characters. However, Emacs
1582 can still handle these character codes as if they belonged to
1583 @emph{one} of the single-byte character sets at a time. To specify
1584 @emph{which} of these codes to use, invoke @kbd{M-x
1585 set-language-environment} and specify a suitable language environment
1586 such as @samp{Latin-@var{n}}.
1587
1588 For more information about unibyte operation, see @ref{Disabling
1589 Multibyte}. Note particularly that you probably want to ensure that
1590 your initialization files are read as unibyte if they contain
1591 non-@acronym{ASCII} characters.
1592
1593 @vindex unibyte-display-via-language-environment
1594 Emacs can also display bytes in the range 160 to 255 as readable
1595 characters, provided the terminal or font in use supports them. This
1596 works automatically. On a graphical display, Emacs can also display
1597 single-byte characters through fontsets, in effect by displaying the
1598 equivalent multibyte characters according to the current language
1599 environment. To request this, set the variable
1600 @code{unibyte-display-via-language-environment} to a non-@code{nil}
1601 value. Note that setting this only affects how these bytes are
1602 displayed, but does not change the fundamental fact that Emacs treats
1603 them as raw bytes, not as characters.
1604
1605 @cindex @code{iso-ascii} library
1606 If your terminal does not support display of the Latin-1 character
1607 set, Emacs can display these characters as @acronym{ASCII} sequences which at
1608 least give you a clear idea of what the characters are. To do this,
1609 load the library @code{iso-ascii}. Similar libraries for other
1610 Latin-@var{n} character sets could be implemented, but we don't have
1611 them yet.
1612
1613 @findex standard-display-8bit
1614 @cindex 8-bit display
1615 Normally non-ISO-8859 characters (decimal codes between 128 and 159
1616 inclusive) are displayed as octal escapes. You can change this for
1617 non-standard ``extended'' versions of ISO-8859 character sets by using the
1618 function @code{standard-display-8bit} in the @code{disp-table} library.
1619
1620 There are two ways to input single-byte non-@acronym{ASCII}
1621 characters:
1622
1623 @itemize @bullet
1624 @cindex 8-bit input
1625 @item
1626 You can use an input method for the selected language environment.
1627 @xref{Input Methods}. When you use an input method in a unibyte buffer,
1628 the non-@acronym{ASCII} character you specify with it is converted to unibyte.
1629
1630 @item
1631 If your keyboard can generate character codes 128 (decimal) and up,
1632 representing non-@acronym{ASCII} characters, you can type those character codes
1633 directly.
1634
1635 On a graphical display, you should not need to do anything special to
1636 use these keys; they should simply work. On a text terminal, you
1637 should use the command @code{M-x set-keyboard-coding-system} or the
1638 variable @code{keyboard-coding-system} to specify which coding system
1639 your keyboard uses (@pxref{Terminal Coding}). Enabling this feature
1640 will probably require you to use @kbd{ESC} to type Meta characters;
1641 however, on a console terminal or in @code{xterm}, you can arrange for
1642 Meta to be converted to @kbd{ESC} and still be able type 8-bit
1643 characters present directly on the keyboard or using @kbd{Compose} or
1644 @kbd{AltGr} keys. @xref{User Input}.
1645
1646 @kindex C-x 8
1647 @cindex @code{iso-transl} library
1648 @cindex compose character
1649 @cindex dead character
1650 @item
1651 For Latin-1 only, you can use the key @kbd{C-x 8} as a ``compose
1652 character'' prefix for entry of non-@acronym{ASCII} Latin-1 printing
1653 characters. @kbd{C-x 8} is good for insertion (in the minibuffer as
1654 well as other buffers), for searching, and in any other context where
1655 a key sequence is allowed.
1656
1657 @kbd{C-x 8} works by loading the @code{iso-transl} library. Once that
1658 library is loaded, the @key{ALT} modifier key, if the keyboard has
1659 one, serves the same purpose as @kbd{C-x 8}: use @key{ALT} together
1660 with an accent character to modify the following letter. In addition,
1661 if the keyboard has keys for the Latin-1 ``dead accent characters'',
1662 they too are defined to compose with the following character, once
1663 @code{iso-transl} is loaded.
1664
1665 Use @kbd{C-x 8 C-h} to list all the available @kbd{C-x 8} translations.
1666 @end itemize
1667
1668 @node Charsets
1669 @section Charsets
1670 @cindex charsets
1671
1672 In Emacs, @dfn{charset} is short for ``character set''. Emacs
1673 supports most popular charsets (such as @code{ascii},
1674 @code{iso-8859-1}, @code{cp1250}, @code{big5}, and @code{unicode}), in
1675 addition to some charsets of its own (such as @code{emacs},
1676 @code{unicode-bmp}, and @code{eight-bit}). All supported characters
1677 belong to one or more charsets.
1678
1679 Emacs normally ``does the right thing'' with respect to charsets, so
1680 that you don't have to worry about them. However, it is sometimes
1681 helpful to know some of the underlying details about charsets.
1682
1683 One example is font selection (@pxref{Fonts}). Each language
1684 environment (@pxref{Language Environments}) defines a ``priority
1685 list'' for the various charsets. When searching for a font, Emacs
1686 initially attempts to find one that can display the highest-priority
1687 charsets. For instance, in the Japanese language environment, the
1688 charset @code{japanese-jisx0208} has the highest priority, so Emacs
1689 tries to use a font whose @code{registry} property is
1690 @samp{JISX0208.1983-0}.
1691
1692 @findex list-charset-chars
1693 @cindex characters in a certain charset
1694 @findex describe-character-set
1695 There are two commands that can be used to obtain information about
1696 charsets. The command @kbd{M-x list-charset-chars} prompts for a
1697 charset name, and displays all the characters in that character set.
1698 The command @kbd{M-x describe-character-set} prompts for a charset
1699 name, and displays information about that charset, including its
1700 internal representation within Emacs.
1701
1702 @findex list-character-sets
1703 @kbd{M-x list-character-sets} displays a list of all supported
1704 charsets. The list gives the names of charsets and additional
1705 information to identity each charset (see
1706 @url{http://www.itscj.ipsj.or.jp/ISO-IR/} for details). In this list,
1707 charsets are divided into two categories: @dfn{normal charsets} are
1708 listed first, followed by @dfn{supplementary charsets}. A
1709 supplementary charset is one that is used to define another charset
1710 (as a parent or a subset), or to provide backward-compatibility for
1711 older Emacs versions.
1712
1713 To find out which charset a character in the buffer belongs to, put
1714 point before it and type @kbd{C-u C-x =} (@pxref{International
1715 Chars}).
1716
1717 @node Bidirectional Editing
1718 @section Bidirectional Editing
1719 @cindex bidirectional editing
1720 @cindex right-to-left text
1721
1722 Emacs supports editing text written in scripts, such as Arabic and
1723 Hebrew, whose natural ordering of horizontal text for display is from
1724 right to left. However, digits and Latin text embedded in these
1725 scripts are still displayed left to right. It is also not uncommon to
1726 have small portions of text in Arabic or Hebrew embedded in otherwise
1727 Latin document, e.g., as comments and strings in a program source
1728 file. For these reasons, text that uses these scripts is actually
1729 @dfn{bidirectional}: a mixture of runs of left-to-right and
1730 right-to-left characters.
1731
1732 This section describes the facilities and options provided by Emacs
1733 for editing bidirectional text.
1734
1735 @cindex logical order
1736 @cindex visual order
1737 Emacs stores right-to-left and bidirectional text in the so-called
1738 @dfn{logical} (or @dfn{reading}) order: the buffer or string position
1739 of the first character you read precedes that of the next character.
1740 Reordering of bidirectional text into the @dfn{visual} order happens
1741 at display time. As result, character positions no longer increase
1742 monotonically with their positions on display. Emacs implements the
1743 Unicode Bidirectional Algorithm described in the Unicode Standard
1744 Annex #9, for reordering of bidirectional text for display.
1745
1746 @vindex bidi-display-reordering
1747 The buffer-local variable @code{bidi-display-reordering} controls
1748 whether text in the buffer is reordered for display. If its value is
1749 non-@code{nil}, Emacs reorders characters that have right-to-left
1750 directionality when they are displayed. The default value is
1751 @code{t}.
1752
1753 Each paragraph of bidirectional text can have its own @dfn{base
1754 direction}, either right-to-left or left-to-right. (Paragraph
1755 boundaries are empty lines, i.e.@: lines consisting entirely of
1756 whitespace characters.) Text in left-to-right paragraphs begins at
1757 the left margin of the window and is truncated or continued when it
1758 reaches the right margin. By contrast, text in right-to-left
1759 paragraphs begins at the right margin and is continued or truncated at
1760 the left margin.
1761
1762 @vindex bidi-paragraph-direction
1763 Emacs determines the base direction of each paragraph dynamically,
1764 based on the text at the beginning of the paragraph. However,
1765 sometimes a buffer may need to force a certain base direction for its
1766 paragraphs. The variable @code{bidi-paragraph-direction}, if
1767 non-@code{nil}, disables the dynamic determination of the base
1768 direction, and instead forces all paragraphs in the buffer to have the
1769 direction specified by its buffer-local value. The value can be either
1770 @code{right-to-left} or @code{left-to-right}. Any other value is
1771 interpreted as @code{nil}.
1772
1773 @cindex LRM
1774 @cindex RLM
1775 Alternatively, you can control the base direction of a paragraph by
1776 inserting special formatting characters in front of the paragraph.
1777 The special character @code{RIGHT-TO-LEFT MARK}, or @sc{rlm}, forces
1778 the right-to-left direction on the following paragraph, while
1779 @code{LEFT-TO-RIGHT MARK}, or @sc{lrm} forces the left-to-right
1780 direction. (You can use @kbd{C-x 8 RET} to insert these characters.)
1781 In a GUI session, the @sc{lrm} and @sc{rlm} characters display as very
1782 thin blank characters; on text terminals they display as blanks.
1783
1784 Because characters are reordered for display, Emacs commands that
1785 operate in the logical order or on stretches of buffer positions may
1786 produce unusual effects. For example, @kbd{C-f} and @kbd{C-b}
1787 commands move point in the logical order, so the cursor will sometimes
1788 jump when point traverses reordered bidirectional text. Similarly, a
1789 highlighted region covering a contiguous range of character positions
1790 may look discontinuous if the region spans reordered text. This is
1791 normal and similar to behavior of other programs that support
1792 bidirectional text.