]> code.delx.au - gnu-emacs/blob - doc/emacs/mule.texi
Merge changes from emacs-24; up to 2012-04-26T02:03:19Z!ueno@unixuser.org
[gnu-emacs] / doc / emacs / mule.texi
1 @c This is part of the Emacs manual.
2 @c Copyright (C) 1997, 1999-2012 Free Software Foundation, Inc.
3 @c See file emacs.texi for copying conditions.
4 @node International
5 @chapter International Character Set Support
6 @c This node is referenced in the tutorial. When renaming or deleting
7 @c it, the tutorial needs to be adjusted. (TUTORIAL.de)
8 @cindex MULE
9 @cindex international scripts
10 @cindex multibyte characters
11 @cindex encoding of characters
12
13 @cindex Celtic
14 @cindex Chinese
15 @cindex Cyrillic
16 @cindex Czech
17 @cindex Devanagari
18 @cindex Hindi
19 @cindex Marathi
20 @cindex Ethiopic
21 @cindex German
22 @cindex Greek
23 @cindex Hebrew
24 @cindex IPA
25 @cindex Japanese
26 @cindex Korean
27 @cindex Lao
28 @cindex Latin
29 @cindex Polish
30 @cindex Romanian
31 @cindex Slovak
32 @cindex Slovenian
33 @cindex Thai
34 @cindex Tibetan
35 @cindex Turkish
36 @cindex Vietnamese
37 @cindex Dutch
38 @cindex Spanish
39 Emacs supports a wide variety of international character sets,
40 including European and Vietnamese variants of the Latin alphabet, as
41 well as Cyrillic, Devanagari (for Hindi and Marathi), Ethiopic, Greek,
42 Han (for Chinese and Japanese), Hangul (for Korean), Hebrew, IPA,
43 Kannada, Lao, Malayalam, Tamil, Thai, Tibetan, and Vietnamese scripts.
44 Emacs also supports various encodings of these characters that are used by
45 other internationalized software, such as word processors and mailers.
46
47 Emacs allows editing text with international characters by supporting
48 all the related activities:
49
50 @itemize @bullet
51 @item
52 You can visit files with non-@acronym{ASCII} characters, save non-@acronym{ASCII} text, and
53 pass non-@acronym{ASCII} text between Emacs and programs it invokes (such as
54 compilers, spell-checkers, and mailers). Setting your language
55 environment (@pxref{Language Environments}) takes care of setting up the
56 coding systems and other options for a specific language or culture.
57 Alternatively, you can specify how Emacs should encode or decode text
58 for each command; see @ref{Text Coding}.
59
60 @item
61 You can display non-@acronym{ASCII} characters encoded by the various
62 scripts. This works by using appropriate fonts on graphics displays
63 (@pxref{Defining Fontsets}), and by sending special codes to text
64 displays (@pxref{Terminal Coding}). If some characters are displayed
65 incorrectly, refer to @ref{Undisplayable Characters}, which describes
66 possible problems and explains how to solve them.
67
68 @item
69 Characters from scripts whose natural ordering of text is from right
70 to left are reordered for display (@pxref{Bidirectional Editing}).
71 These scripts include Arabic, Hebrew, Syriac, Thaana, and a few
72 others.
73
74 @item
75 You can insert non-@acronym{ASCII} characters or search for them. To do that,
76 you can specify an input method (@pxref{Select Input Method}) suitable
77 for your language, or use the default input method set up when you chose
78 your language environment. If
79 your keyboard can produce non-@acronym{ASCII} characters, you can select an
80 appropriate keyboard coding system (@pxref{Terminal Coding}), and Emacs
81 will accept those characters. Latin-1 characters can also be input by
82 using the @kbd{C-x 8} prefix, see @ref{Unibyte Mode}.
83
84 With the X Window System, your locale should be set to an appropriate
85 value to make sure Emacs interprets keyboard input correctly; see
86 @ref{Language Environments, locales}.
87 @end itemize
88
89 The rest of this chapter describes these issues in detail.
90
91 @menu
92 * International Chars:: Basic concepts of multibyte characters.
93 * Disabling Multibyte:: Controlling whether to use multibyte characters.
94 * Language Environments:: Setting things up for the language you use.
95 * Input Methods:: Entering text characters not on your keyboard.
96 * Select Input Method:: Specifying your choice of input methods.
97 * Coding Systems:: Character set conversion when you read and
98 write files, and so on.
99 * Recognize Coding:: How Emacs figures out which conversion to use.
100 * Specify Coding:: Specifying a file's coding system explicitly.
101 * Output Coding:: Choosing coding systems for output.
102 * Text Coding:: Choosing conversion to use for file text.
103 * Communication Coding:: Coding systems for interprocess communication.
104 * File Name Coding:: Coding systems for file @emph{names}.
105 * Terminal Coding:: Specifying coding systems for converting
106 terminal input and output.
107 * Fontsets:: Fontsets are collections of fonts
108 that cover the whole spectrum of characters.
109 * Defining Fontsets:: Defining a new fontset.
110 * Modifying Fontsets:: Modifying an existing fontset.
111 * Undisplayable Characters:: When characters don't display.
112 * Unibyte Mode:: You can pick one European character set
113 to use without multibyte characters.
114 * Charsets:: How Emacs groups its internal character codes.
115 * Bidirectional Editing:: Support for right-to-left scripts.
116 @end menu
117
118 @node International Chars
119 @section Introduction to International Character Sets
120
121 The users of international character sets and scripts have
122 established many more-or-less standard coding systems for storing
123 files. These coding systems are typically @dfn{multibyte}, meaning
124 that sequences of two or more bytes are used to represent individual
125 non-@acronym{ASCII} characters.
126
127 @cindex Unicode
128 Internally, Emacs uses its own multibyte character encoding, which
129 is a superset of the @dfn{Unicode} standard. This internal encoding
130 allows characters from almost every known script to be intermixed in a
131 single buffer or string. Emacs translates between the multibyte
132 character encoding and various other coding systems when reading and
133 writing files, and when exchanging data with subprocesses.
134
135 @kindex C-h h
136 @findex view-hello-file
137 @cindex undisplayable characters
138 @cindex @samp{?} in display
139 The command @kbd{C-h h} (@code{view-hello-file}) displays the file
140 @file{etc/HELLO}, which illustrates various scripts by showing
141 how to say ``hello'' in many languages. If some characters can't be
142 displayed on your terminal, they appear as @samp{?} or as hollow boxes
143 (@pxref{Undisplayable Characters}).
144
145 Keyboards, even in the countries where these character sets are
146 used, generally don't have keys for all the characters in them. You
147 can insert characters that your keyboard does not support, using
148 @kbd{C-q} (@code{quoted-insert}) or @kbd{C-x 8 @key{RET}}
149 (@code{ucs-insert}). @xref{Inserting Text}. Emacs also supports
150 various @dfn{input methods}, typically one for each script or
151 language, which make it easier to type characters in the script.
152 @xref{Input Methods}.
153
154 @kindex C-x RET
155 The prefix key @kbd{C-x @key{RET}} is used for commands that pertain
156 to multibyte characters, coding systems, and input methods.
157
158 @kindex C-x =
159 @findex what-cursor-position
160 The command @kbd{C-x =} (@code{what-cursor-position}) shows
161 information about the character at point. In addition to the
162 character position, which was described in @ref{Position Info}, this
163 command displays how the character is encoded. For instance, it
164 displays the following line in the echo area for the character
165 @samp{c}:
166
167 @smallexample
168 Char: c (99, #o143, #x63) point=28062 of 36168 (78%) column=53
169 @end smallexample
170
171 The four values after @samp{Char:} describe the character that
172 follows point, first by showing it and then by giving its character
173 code in decimal, octal and hex. For a non-@acronym{ASCII} multibyte
174 character, these are followed by @samp{file} and the character's
175 representation, in hex, in the buffer's coding system, if that coding
176 system encodes the character safely and with a single byte
177 (@pxref{Coding Systems}). If the character's encoding is longer than
178 one byte, Emacs shows @samp{file ...}.
179
180 As a special case, if the character lies in the range 128 (0200
181 octal) through 159 (0237 octal), it stands for a ``raw'' byte that
182 does not correspond to any specific displayable character. Such a
183 ``character'' lies within the @code{eight-bit-control} character set,
184 and is displayed as an escaped octal character code. In this case,
185 @kbd{C-x =} shows @samp{part of display ...} instead of @samp{file}.
186
187 @cindex character set of character at point
188 @cindex font of character at point
189 @cindex text properties at point
190 @cindex face at point
191 With a prefix argument (@kbd{C-u C-x =}), this command displays a
192 detailed description of the character in a window:
193
194 @itemize @bullet
195 @item
196 The character set name, and the codes that identify the character
197 within that character set; @acronym{ASCII} characters are identified
198 as belonging to the @code{ascii} character set.
199
200 @item
201 The character's syntax and categories.
202
203 @item
204 The character's encodings, both internally in the buffer, and externally
205 if you were to save the file.
206
207 @item
208 What keys to type to input the character in the current input method
209 (if it supports the character).
210
211 @item
212 If you are running Emacs on a graphical display, the font name and
213 glyph code for the character. If you are running Emacs on a text
214 terminal, the code(s) sent to the terminal.
215
216 @item
217 The character's text properties (@pxref{Text Properties,,,
218 elisp, the Emacs Lisp Reference Manual}), including any non-default
219 faces used to display the character, and any overlays containing it
220 (@pxref{Overlays,,, elisp, the same manual}).
221 @end itemize
222
223 Here's an example showing the Latin-1 character A with grave accent,
224 in a buffer whose coding system is @code{utf-8-unix}:
225
226 @smallexample
227 position: 1 of 1 (0%), column: 0
228 character: @`A (displayed as @`A) (codepoint 192, #o300, #xc0)
229 preferred charset: unicode (Unicode (ISO10646))
230 code point in charset: 0xC0
231 syntax: w which means: word
232 category: .:Base, L:Left-to-right (strong),
233 j:Japanese, l:Latin, v:Viet
234 buffer code: #xC3 #x80
235 file code: not encodable by coding system undecided-unix
236 display: by this font (glyph code)
237 xft:-unknown-DejaVu Sans Mono-normal-normal-
238 normal-*-13-*-*-*-m-0-iso10646-1 (#x82)
239
240 Character code properties: customize what to show
241 name: LATIN CAPITAL LETTER A WITH GRAVE
242 old-name: LATIN CAPITAL LETTER A GRAVE
243 general-category: Lu (Letter, Uppercase)
244 decomposition: (65 768) ('A' '`')
245 @end smallexample
246
247 @c FIXME? Does this section even belong in the user manual?
248 @c Seems more appropriate to the lispref?
249 @node Disabling Multibyte
250 @section Disabling Multibyte Characters
251
252 By default, Emacs starts in multibyte mode: it stores the contents
253 of buffers and strings using an internal encoding that represents
254 non-@acronym{ASCII} characters using multi-byte sequences. Multibyte
255 mode allows you to use all the supported languages and scripts without
256 limitations.
257
258 @cindex turn multibyte support on or off
259 Under very special circumstances, you may want to disable multibyte
260 character support, for a specific buffer.
261 When multibyte characters are disabled in a buffer, we call
262 that @dfn{unibyte mode}. In unibyte mode, each character in the
263 buffer has a character code ranging from 0 through 255 (0377 octal); 0
264 through 127 (0177 octal) represent @acronym{ASCII} characters, and 128
265 (0200 octal) through 255 (0377 octal) represent non-@acronym{ASCII}
266 characters.
267
268 To edit a particular file in unibyte representation, visit it using
269 @code{find-file-literally}. @xref{Visiting}. You can convert a
270 multibyte buffer to unibyte by saving it to a file, killing the
271 buffer, and visiting the file again with @code{find-file-literally}.
272 Alternatively, you can use @kbd{C-x @key{RET} c}
273 (@code{universal-coding-system-argument}) and specify @samp{raw-text}
274 as the coding system with which to visit or save a file. @xref{Text
275 Coding}. Unlike @code{find-file-literally}, finding a file as
276 @samp{raw-text} doesn't disable format conversion, uncompression, or
277 auto mode selection.
278
279 @c Not a single file in Emacs uses this feature. Is it really worth
280 @c mentioning in the _user_ manual? Also, this duplicates somewhat
281 @c "Loading Non-ASCII" from the lispref.
282 @cindex Lisp files, and multibyte operation
283 @cindex multibyte operation, and Lisp files
284 @cindex unibyte operation, and Lisp files
285 @cindex init file, and non-@acronym{ASCII} characters
286 Emacs normally loads Lisp files as multibyte.
287 This includes the Emacs initialization
288 file, @file{.emacs}, and the initialization files of packages
289 such as Gnus. However, you can specify unibyte loading for a
290 particular Lisp file, by adding an entry @samp{coding: raw-text} in a file
291 local variables section. @xref{Specify Coding}.
292 Then that file is always loaded as unibyte text.
293 @ignore
294 @c I don't see the point of this statement:
295 The motivation for these conventions is that it is more reliable to
296 always load any particular Lisp file in the same way.
297 @end ignore
298 You can also load a Lisp file as unibyte, on any one occasion, by
299 typing @kbd{C-x @key{RET} c raw-text @key{RET}} immediately before
300 loading it.
301
302 @c See http://debbugs.gnu.org/11226 for lack of unibyte tooltip.
303 @vindex enable-multibyte-characters
304 The buffer-local variable @code{enable-multibyte-characters} is
305 non-@code{nil} in multibyte buffers, and @code{nil} in unibyte ones.
306 The mode line also indicates whether a buffer is multibyte or not.
307 @xref{Mode Line}. With a graphical display, in a multibyte buffer,
308 the portion of the mode line that indicates the character set has a
309 tooltip that (amongst other things) says that the buffer is multibyte.
310 In a unibyte buffer, the character set indicator is absent. Thus, in
311 a unibyte buffer (when using a graphical display) there is normally
312 nothing before the indication of the visited file's end-of-line
313 convention (colon, backslash, etc.), unless you are using an input
314 method.
315
316 @findex toggle-enable-multibyte-characters
317 You can turn off multibyte support in a specific buffer by invoking the
318 command @code{toggle-enable-multibyte-characters} in that buffer.
319
320 @node Language Environments
321 @section Language Environments
322 @cindex language environments
323
324 All supported character sets are supported in Emacs buffers whenever
325 multibyte characters are enabled; there is no need to select a
326 particular language in order to display its characters.
327 However, it is important to select a @dfn{language
328 environment} in order to set various defaults. Roughly speaking, the
329 language environment represents a choice of preferred script rather
330 than a choice of language.
331
332 The language environment controls which coding systems to recognize
333 when reading text (@pxref{Recognize Coding}). This applies to files,
334 incoming mail, and any other text you read into Emacs. It may also
335 specify the default coding system to use when you create a file. Each
336 language environment also specifies a default input method.
337
338 @findex set-language-environment
339 @vindex current-language-environment
340 To select a language environment, customize
341 @code{current-language-environment} or use the command @kbd{M-x
342 set-language-environment}. It makes no difference which buffer is
343 current when you use this command, because the effects apply globally
344 to the Emacs session. The supported language environments
345 (see the variable @code{language-info-alist}) include:
346
347 @cindex Euro sign
348 @cindex UTF-8
349 @quotation
350 ASCII, Belarusian, Bengali, Brazilian Portuguese, Bulgarian, Cham,
351 Chinese-BIG5, Chinese-CNS, Chinese-EUC-TW, Chinese-GB, Chinese-GBK,
352 Chinese-GB18030, Croatian, Cyrillic-ALT, Cyrillic-ISO, Cyrillic-KOI8,
353 Czech, Devanagari, Dutch, English, Esperanto, Ethiopic, French,
354 Georgian, German, Greek, Gujarati, Hebrew, IPA, Italian, Japanese,
355 Kannada, Khmer, Korean, Lao, Latin-1, Latin-2, Latin-3, Latin-4,
356 Latin-5, Latin-6, Latin-7, Latin-8 (Celtic), Latin-9 (updated Latin-1
357 with the Euro sign), Latvian, Lithuanian, Malayalam, Oriya, Polish,
358 Punjabi, Romanian, Russian, Sinhala, Slovak, Slovenian, Spanish,
359 Swedish, TaiViet, Tajik, Tamil, Telugu, Thai, Tibetan, Turkish, UTF-8
360 (for a setup which prefers Unicode characters and files encoded in
361 UTF-8), Ukrainian, Vietnamese, Welsh, and Windows-1255 (for a setup
362 which prefers Cyrillic characters and files encoded in Windows-1255).
363 @end quotation
364
365 To display the script(s) used by your language environment on a
366 graphical display, you need to have suitable fonts.
367 @xref{Fontsets}, for more details about setting up your fonts.
368
369 @findex set-locale-environment
370 @vindex locale-language-names
371 @vindex locale-charset-language-names
372 @cindex locales
373 Some operating systems let you specify the character-set locale you
374 are using by setting the locale environment variables @env{LC_ALL},
375 @env{LC_CTYPE}, or @env{LANG}. (If more than one of these is
376 set, the first one that is nonempty specifies your locale for this
377 purpose.) During startup, Emacs looks up your character-set locale's
378 name in the system locale alias table, matches its canonical name
379 against entries in the value of the variables
380 @code{locale-charset-language-names} and @code{locale-language-names}
381 (the former overrides the latter),
382 and selects the corresponding language environment if a match is found.
383 It also adjusts the display
384 table and terminal coding system, the locale coding system, the
385 preferred coding system as needed for the locale, and---last but not
386 least---the way Emacs decodes non-@acronym{ASCII} characters sent by your keyboard.
387
388 @c This seems unlikely, doesn't it?
389 If you modify the @env{LC_ALL}, @env{LC_CTYPE}, or @env{LANG}
390 environment variables while running Emacs (by using @kbd{M-x setenv}),
391 you may want to invoke the @code{set-locale-environment}
392 function afterwards to readjust the language environment from the new
393 locale.
394
395 @vindex locale-preferred-coding-systems
396 The @code{set-locale-environment} function normally uses the preferred
397 coding system established by the language environment to decode system
398 messages. But if your locale matches an entry in the variable
399 @code{locale-preferred-coding-systems}, Emacs uses the corresponding
400 coding system instead. For example, if the locale @samp{ja_JP.PCK}
401 matches @code{japanese-shift-jis} in
402 @code{locale-preferred-coding-systems}, Emacs uses that encoding even
403 though it might normally use @code{japanese-iso-8bit}.
404
405 You can override the language environment chosen at startup with
406 explicit use of the command @code{set-language-environment}, or with
407 customization of @code{current-language-environment} in your init
408 file.
409
410 @kindex C-h L
411 @findex describe-language-environment
412 To display information about the effects of a certain language
413 environment @var{lang-env}, use the command @kbd{C-h L @var{lang-env}
414 @key{RET}} (@code{describe-language-environment}). This tells you
415 which languages this language environment is useful for, and lists the
416 character sets, coding systems, and input methods that go with it. It
417 also shows some sample text to illustrate scripts used in this
418 language environment. If you give an empty input for @var{lang-env},
419 this command describes the chosen language environment.
420 @anchor{Describe Language Environment}
421
422 @vindex set-language-environment-hook
423 You can customize any language environment with the normal hook
424 @code{set-language-environment-hook}. The command
425 @code{set-language-environment} runs that hook after setting up the new
426 language environment. The hook functions can test for a specific
427 language environment by checking the variable
428 @code{current-language-environment}. This hook is where you should
429 put non-default settings for specific language environments, such as
430 coding systems for keyboard input and terminal output, the default
431 input method, etc.
432
433 @vindex exit-language-environment-hook
434 Before it starts to set up the new language environment,
435 @code{set-language-environment} first runs the hook
436 @code{exit-language-environment-hook}. This hook is useful for undoing
437 customizations that were made with @code{set-language-environment-hook}.
438 For instance, if you set up a special key binding in a specific language
439 environment using @code{set-language-environment-hook}, you should set
440 up @code{exit-language-environment-hook} to restore the normal binding
441 for that key.
442
443 @node Input Methods
444 @section Input Methods
445
446 @cindex input methods
447 An @dfn{input method} is a kind of character conversion designed
448 specifically for interactive input. In Emacs, typically each language
449 has its own input method; sometimes several languages that use the same
450 characters can share one input method. A few languages support several
451 input methods.
452
453 The simplest kind of input method works by mapping @acronym{ASCII} letters
454 into another alphabet; this allows you to use one other alphabet
455 instead of @acronym{ASCII}. The Greek and Russian input methods
456 work this way.
457
458 A more powerful technique is composition: converting sequences of
459 characters into one letter. Many European input methods use composition
460 to produce a single non-@acronym{ASCII} letter from a sequence that consists of a
461 letter followed by accent characters (or vice versa). For example, some
462 methods convert the sequence @kbd{o ^} into a single accented letter.
463 These input methods have no special commands of their own; all they do
464 is compose sequences of printing characters.
465
466 The input methods for syllabic scripts typically use mapping followed
467 by composition. The input methods for Thai and Korean work this way.
468 First, letters are mapped into symbols for particular sounds or tone
469 marks; then, sequences of these that make up a whole syllable are
470 mapped into one syllable sign.
471
472 Chinese and Japanese require more complex methods. In Chinese input
473 methods, first you enter the phonetic spelling of a Chinese word (in
474 input method @code{chinese-py}, among others), or a sequence of
475 portions of the character (input methods @code{chinese-4corner} and
476 @code{chinese-sw}, and others). One input sequence typically
477 corresponds to many possible Chinese characters. You select the one
478 you mean using keys such as @kbd{C-f}, @kbd{C-b}, @kbd{C-n},
479 @kbd{C-p} (or the arrow keys), and digits, which have special meanings
480 in this situation.
481
482 The possible characters are conceptually arranged in several rows,
483 with each row holding up to 10 alternatives. Normally, Emacs displays
484 just one row at a time, in the echo area; @code{(@var{i}/@var{j})}
485 appears at the beginning, to indicate that this is the @var{i}th row
486 out of a total of @var{j} rows. Type @kbd{C-n} or @kbd{C-p} to
487 display the next row or the previous row.
488
489 Type @kbd{C-f} and @kbd{C-b} to move forward and backward among
490 the alternatives in the current row. As you do this, Emacs highlights
491 the current alternative with a special color; type @code{C-@key{SPC}}
492 to select the current alternative and use it as input. The
493 alternatives in the row are also numbered; the number appears before
494 the alternative. Typing a number selects the associated alternative
495 of the current row and uses it as input.
496
497 @key{TAB} in these Chinese input methods displays a buffer showing
498 all the possible characters at once; then clicking @kbd{Mouse-2} on
499 one of them selects that alternative. The keys @kbd{C-f}, @kbd{C-b},
500 @kbd{C-n}, @kbd{C-p}, and digits continue to work as usual, but they
501 do the highlighting in the buffer showing the possible characters,
502 rather than in the echo area.
503
504 In Japanese input methods, first you input a whole word using
505 phonetic spelling; then, after the word is in the buffer, Emacs
506 converts it into one or more characters using a large dictionary. One
507 phonetic spelling corresponds to a number of different Japanese words;
508 to select one of them, use @kbd{C-n} and @kbd{C-p} to cycle through
509 the alternatives.
510
511 Sometimes it is useful to cut off input method processing so that the
512 characters you have just entered will not combine with subsequent
513 characters. For example, in input method @code{latin-1-postfix}, the
514 sequence @kbd{o ^} combines to form an @samp{o} with an accent. What if
515 you want to enter them as separate characters?
516
517 One way is to type the accent twice; this is a special feature for
518 entering the separate letter and accent. For example, @kbd{o ^ ^} gives
519 you the two characters @samp{o^}. Another way is to type another letter
520 after the @kbd{o}---something that won't combine with that---and
521 immediately delete it. For example, you could type @kbd{o o @key{DEL}
522 ^} to get separate @samp{o} and @samp{^}.
523
524 Another method, more general but not quite as easy to type, is to use
525 @kbd{C-\ C-\} between two characters to stop them from combining. This
526 is the command @kbd{C-\} (@code{toggle-input-method}) used twice.
527 @ifnottex
528 @xref{Select Input Method}.
529 @end ifnottex
530
531 @cindex incremental search, input method interference
532 @kbd{C-\ C-\} is especially useful inside an incremental search,
533 because it stops waiting for more characters to combine, and starts
534 searching for what you have already entered.
535
536 To find out how to input the character after point using the current
537 input method, type @kbd{C-u C-x =}. @xref{Position Info}.
538
539 @vindex input-method-verbose-flag
540 @vindex input-method-highlight-flag
541 The variables @code{input-method-highlight-flag} and
542 @code{input-method-verbose-flag} control how input methods explain
543 what is happening. If @code{input-method-highlight-flag} is
544 non-@code{nil}, the partial sequence is highlighted in the buffer (for
545 most input methods---some disable this feature). If
546 @code{input-method-verbose-flag} is non-@code{nil}, the list of
547 possible characters to type next is displayed in the echo area (but
548 not when you are in the minibuffer).
549
550 Another facility for typing characters not on your keyboard is by
551 using @kbd{C-x 8 @key{RET}} (@code{ucs-insert}) to insert a single
552 character based on its Unicode name or code-point; see @ref{Inserting
553 Text}.
554
555 @node Select Input Method
556 @section Selecting an Input Method
557
558 @table @kbd
559 @item C-\
560 Enable or disable use of the selected input method (@code{toggle-input-method}).
561
562 @item C-x @key{RET} C-\ @var{method} @key{RET}
563 Select a new input method for the current buffer (@code{set-input-method}).
564
565 @item C-h I @var{method} @key{RET}
566 @itemx C-h C-\ @var{method} @key{RET}
567 @findex describe-input-method
568 @kindex C-h I
569 @kindex C-h C-\
570 Describe the input method @var{method} (@code{describe-input-method}).
571 By default, it describes the current input method (if any). This
572 description should give you the full details of how to use any
573 particular input method.
574
575 @item M-x list-input-methods
576 Display a list of all the supported input methods.
577 @end table
578
579 @findex set-input-method
580 @vindex current-input-method
581 @kindex C-x RET C-\
582 To choose an input method for the current buffer, use @kbd{C-x
583 @key{RET} C-\} (@code{set-input-method}). This command reads the
584 input method name from the minibuffer; the name normally starts with the
585 language environment that it is meant to be used with. The variable
586 @code{current-input-method} records which input method is selected.
587
588 @findex toggle-input-method
589 @kindex C-\
590 Input methods use various sequences of @acronym{ASCII} characters to
591 stand for non-@acronym{ASCII} characters. Sometimes it is useful to
592 turn off the input method temporarily. To do this, type @kbd{C-\}
593 (@code{toggle-input-method}). To reenable the input method, type
594 @kbd{C-\} again.
595
596 If you type @kbd{C-\} and you have not yet selected an input method,
597 it prompts you to specify one. This has the same effect as using
598 @kbd{C-x @key{RET} C-\} to specify an input method.
599
600 When invoked with a numeric argument, as in @kbd{C-u C-\},
601 @code{toggle-input-method} always prompts you for an input method,
602 suggesting the most recently selected one as the default.
603
604 @vindex default-input-method
605 Selecting a language environment specifies a default input method for
606 use in various buffers. When you have a default input method, you can
607 select it in the current buffer by typing @kbd{C-\}. The variable
608 @code{default-input-method} specifies the default input method
609 (@code{nil} means there is none).
610
611 In some language environments, which support several different input
612 methods, you might want to use an input method different from the
613 default chosen by @code{set-language-environment}. You can instruct
614 Emacs to select a different default input method for a certain
615 language environment, if you wish, by using
616 @code{set-language-environment-hook} (@pxref{Language Environments,
617 set-language-environment-hook}). For example:
618
619 @lisp
620 (defun my-chinese-setup ()
621 "Set up my private Chinese environment."
622 (if (equal current-language-environment "Chinese-GB")
623 (setq default-input-method "chinese-tonepy")))
624 (add-hook 'set-language-environment-hook 'my-chinese-setup)
625 @end lisp
626
627 @noindent
628 This sets the default input method to be @code{chinese-tonepy}
629 whenever you choose a Chinese-GB language environment.
630
631 You can instruct Emacs to activate a certain input method
632 automatically. For example:
633
634 @lisp
635 (add-hook 'text-mode-hook
636 (lambda () (set-input-method "german-prefix")))
637 @end lisp
638
639 @noindent
640 This automatically activates the input method ``german-prefix'' in
641 Text mode.
642
643 @findex quail-set-keyboard-layout
644 Some input methods for alphabetic scripts work by (in effect)
645 remapping the keyboard to emulate various keyboard layouts commonly used
646 for those scripts. How to do this remapping properly depends on your
647 actual keyboard layout. To specify which layout your keyboard has, use
648 the command @kbd{M-x quail-set-keyboard-layout}.
649
650 @findex quail-show-key
651 You can use the command @kbd{M-x quail-show-key} to show what key (or
652 key sequence) to type in order to input the character following point,
653 using the selected keyboard layout. The command @kbd{C-u C-x =} also
654 shows that information, in addition to other information about the
655 character.
656
657 @findex list-input-methods
658 @kbd{M-x list-input-methods} displays a list of all the supported
659 input methods. The list gives information about each input method,
660 including the string that stands for it in the mode line.
661
662 @node Coding Systems
663 @section Coding Systems
664 @cindex coding systems
665
666 Users of various languages have established many more-or-less standard
667 coding systems for representing them. Emacs does not use these coding
668 systems internally; instead, it converts from various coding systems to
669 its own system when reading data, and converts the internal coding
670 system to other coding systems when writing data. Conversion is
671 possible in reading or writing files, in sending or receiving from the
672 terminal, and in exchanging data with subprocesses.
673
674 Emacs assigns a name to each coding system. Most coding systems are
675 used for one language, and the name of the coding system starts with
676 the language name. Some coding systems are used for several
677 languages; their names usually start with @samp{iso}. There are also
678 special coding systems, such as @code{no-conversion}, @code{raw-text},
679 and @code{emacs-internal}.
680
681 @cindex international files from DOS/Windows systems
682 A special class of coding systems, collectively known as
683 @dfn{codepages}, is designed to support text encoded by MS-Windows and
684 MS-DOS software. The names of these coding systems are
685 @code{cp@var{nnnn}}, where @var{nnnn} is a 3- or 4-digit number of the
686 codepage. You can use these encodings just like any other coding
687 system; for example, to visit a file encoded in codepage 850, type
688 @kbd{C-x @key{RET} c cp850 @key{RET} C-x C-f @var{filename}
689 @key{RET}}.
690
691 In addition to converting various representations of non-@acronym{ASCII}
692 characters, a coding system can perform end-of-line conversion. Emacs
693 handles three different conventions for how to separate lines in a file:
694 newline (``unix''), carriage-return linefeed (``dos''), and just
695 carriage-return (``mac'').
696
697 @table @kbd
698 @item C-h C @var{coding} @key{RET}
699 Describe coding system @var{coding} (@code{describe-coding-system}).
700
701 @item C-h C @key{RET}
702 Describe the coding systems currently in use.
703
704 @item M-x list-coding-systems
705 Display a list of all the supported coding systems.
706 @end table
707
708 @kindex C-h C
709 @findex describe-coding-system
710 The command @kbd{C-h C} (@code{describe-coding-system}) displays
711 information about particular coding systems, including the end-of-line
712 conversion specified by those coding systems. You can specify a coding
713 system name as the argument; alternatively, with an empty argument, it
714 describes the coding systems currently selected for various purposes,
715 both in the current buffer and as the defaults, and the priority list
716 for recognizing coding systems (@pxref{Recognize Coding}).
717
718 @findex list-coding-systems
719 To display a list of all the supported coding systems, type @kbd{M-x
720 list-coding-systems}. The list gives information about each coding
721 system, including the letter that stands for it in the mode line
722 (@pxref{Mode Line}).
723
724 @cindex end-of-line conversion
725 @cindex line endings
726 @cindex MS-DOS end-of-line conversion
727 @cindex Macintosh end-of-line conversion
728 Each of the coding systems that appear in this list---except for
729 @code{no-conversion}, which means no conversion of any kind---specifies
730 how and whether to convert printing characters, but leaves the choice of
731 end-of-line conversion to be decided based on the contents of each file.
732 For example, if the file appears to use the sequence carriage-return
733 linefeed to separate lines, DOS end-of-line conversion will be used.
734
735 Each of the listed coding systems has three variants, which specify
736 exactly what to do for end-of-line conversion:
737
738 @table @code
739 @item @dots{}-unix
740 Don't do any end-of-line conversion; assume the file uses
741 newline to separate lines. (This is the convention normally used
742 on Unix and GNU systems, and Mac OS X.)
743
744 @item @dots{}-dos
745 Assume the file uses carriage-return linefeed to separate lines, and do
746 the appropriate conversion. (This is the convention normally used on
747 Microsoft systems.@footnote{It is also specified for MIME @samp{text/*}
748 bodies and in other network transport contexts. It is different
749 from the SGML reference syntax record-start/record-end format, which
750 Emacs doesn't support directly.})
751
752 @item @dots{}-mac
753 Assume the file uses carriage-return to separate lines, and do the
754 appropriate conversion. (This was the convention used on the
755 Macintosh system prior to OS X.)
756 @end table
757
758 These variant coding systems are omitted from the
759 @code{list-coding-systems} display for brevity, since they are entirely
760 predictable. For example, the coding system @code{iso-latin-1} has
761 variants @code{iso-latin-1-unix}, @code{iso-latin-1-dos} and
762 @code{iso-latin-1-mac}.
763
764 @cindex @code{undecided}, coding system
765 The coding systems @code{unix}, @code{dos}, and @code{mac} are
766 aliases for @code{undecided-unix}, @code{undecided-dos}, and
767 @code{undecided-mac}, respectively. These coding systems specify only
768 the end-of-line conversion, and leave the character code conversion to
769 be deduced from the text itself.
770
771 @cindex @code{raw-text}, coding system
772 The coding system @code{raw-text} is good for a file which is mainly
773 @acronym{ASCII} text, but may contain byte values above 127 that are
774 not meant to encode non-@acronym{ASCII} characters. With
775 @code{raw-text}, Emacs copies those byte values unchanged, and sets
776 @code{enable-multibyte-characters} to @code{nil} in the current buffer
777 so that they will be interpreted properly. @code{raw-text} handles
778 end-of-line conversion in the usual way, based on the data
779 encountered, and has the usual three variants to specify the kind of
780 end-of-line conversion to use.
781
782 @cindex @code{no-conversion}, coding system
783 In contrast, the coding system @code{no-conversion} specifies no
784 character code conversion at all---none for non-@acronym{ASCII} byte values and
785 none for end of line. This is useful for reading or writing binary
786 files, tar files, and other files that must be examined verbatim. It,
787 too, sets @code{enable-multibyte-characters} to @code{nil}.
788
789 The easiest way to edit a file with no conversion of any kind is with
790 the @kbd{M-x find-file-literally} command. This uses
791 @code{no-conversion}, and also suppresses other Emacs features that
792 might convert the file contents before you see them. @xref{Visiting}.
793
794 @cindex @code{emacs-internal}, coding system
795 The coding system @code{emacs-internal} (or @code{utf-8-emacs},
796 which is equivalent) means that the file contains non-@acronym{ASCII}
797 characters stored with the internal Emacs encoding. This coding
798 system handles end-of-line conversion based on the data encountered,
799 and has the usual three variants to specify the kind of end-of-line
800 conversion.
801
802 @node Recognize Coding
803 @section Recognizing Coding Systems
804
805 Whenever Emacs reads a given piece of text, it tries to recognize
806 which coding system to use. This applies to files being read, output
807 from subprocesses, text from X selections, etc. Emacs can select the
808 right coding system automatically most of the time---once you have
809 specified your preferences.
810
811 Some coding systems can be recognized or distinguished by which byte
812 sequences appear in the data. However, there are coding systems that
813 cannot be distinguished, not even potentially. For example, there is no
814 way to distinguish between Latin-1 and Latin-2; they use the same byte
815 values with different meanings.
816
817 Emacs handles this situation by means of a priority list of coding
818 systems. Whenever Emacs reads a file, if you do not specify the coding
819 system to use, Emacs checks the data against each coding system,
820 starting with the first in priority and working down the list, until it
821 finds a coding system that fits the data. Then it converts the file
822 contents assuming that they are represented in this coding system.
823
824 The priority list of coding systems depends on the selected language
825 environment (@pxref{Language Environments}). For example, if you use
826 French, you probably want Emacs to prefer Latin-1 to Latin-2; if you use
827 Czech, you probably want Latin-2 to be preferred. This is one of the
828 reasons to specify a language environment.
829
830 @findex prefer-coding-system
831 However, you can alter the coding system priority list in detail
832 with the command @kbd{M-x prefer-coding-system}. This command reads
833 the name of a coding system from the minibuffer, and adds it to the
834 front of the priority list, so that it is preferred to all others. If
835 you use this command several times, each use adds one element to the
836 front of the priority list.
837
838 If you use a coding system that specifies the end-of-line conversion
839 type, such as @code{iso-8859-1-dos}, what this means is that Emacs
840 should attempt to recognize @code{iso-8859-1} with priority, and should
841 use DOS end-of-line conversion when it does recognize @code{iso-8859-1}.
842
843 @vindex file-coding-system-alist
844 Sometimes a file name indicates which coding system to use for the
845 file. The variable @code{file-coding-system-alist} specifies this
846 correspondence. There is a special function
847 @code{modify-coding-system-alist} for adding elements to this list. For
848 example, to read and write all @samp{.txt} files using the coding system
849 @code{chinese-iso-8bit}, you can execute this Lisp expression:
850
851 @smallexample
852 (modify-coding-system-alist 'file "\\.txt\\'" 'chinese-iso-8bit)
853 @end smallexample
854
855 @noindent
856 The first argument should be @code{file}, the second argument should be
857 a regular expression that determines which files this applies to, and
858 the third argument says which coding system to use for these files.
859
860 @vindex inhibit-eol-conversion
861 @cindex DOS-style end-of-line display
862 Emacs recognizes which kind of end-of-line conversion to use based on
863 the contents of the file: if it sees only carriage-returns, or only
864 carriage-return linefeed sequences, then it chooses the end-of-line
865 conversion accordingly. You can inhibit the automatic use of
866 end-of-line conversion by setting the variable @code{inhibit-eol-conversion}
867 to non-@code{nil}. If you do that, DOS-style files will be displayed
868 with the @samp{^M} characters visible in the buffer; some people
869 prefer this to the more subtle @samp{(DOS)} end-of-line type
870 indication near the left edge of the mode line (@pxref{Mode Line,
871 eol-mnemonic}).
872
873 @vindex inhibit-iso-escape-detection
874 @cindex escape sequences in files
875 By default, the automatic detection of coding system is sensitive to
876 escape sequences. If Emacs sees a sequence of characters that begin
877 with an escape character, and the sequence is valid as an ISO-2022
878 code, that tells Emacs to use one of the ISO-2022 encodings to decode
879 the file.
880
881 However, there may be cases that you want to read escape sequences
882 in a file as is. In such a case, you can set the variable
883 @code{inhibit-iso-escape-detection} to non-@code{nil}. Then the code
884 detection ignores any escape sequences, and never uses an ISO-2022
885 encoding. The result is that all escape sequences become visible in
886 the buffer.
887
888 The default value of @code{inhibit-iso-escape-detection} is
889 @code{nil}. We recommend that you not change it permanently, only for
890 one specific operation. That's because some Emacs Lisp source files
891 in the Emacs distribution contain non-@acronym{ASCII} characters encoded in the
892 coding system @code{iso-2022-7bit}, and they won't be
893 decoded correctly when you visit those files if you suppress the
894 escape sequence detection.
895 @c I count a grand total of 3 such files, so is the above really true?
896
897 @vindex auto-coding-alist
898 @vindex auto-coding-regexp-alist
899 The variables @code{auto-coding-alist} and
900 @code{auto-coding-regexp-alist} are
901 the strongest way to specify the coding system for certain patterns of
902 file names, or for files containing certain patterns, respectively.
903 These variables even override @samp{-*-coding:-*-} tags in the file
904 itself (@pxref{Specify Coding}). For example, Emacs
905 uses @code{auto-coding-alist} for tar and archive files, to prevent it
906 from being confused by a @samp{-*-coding:-*-} tag in a member of the
907 archive and thinking it applies to the archive file as a whole.
908 @ignore
909 @c This describes old-style BABYL files, which are no longer relevant.
910 Likewise, Emacs uses @code{auto-coding-regexp-alist} to ensure that
911 RMAIL files, whose names in general don't match any particular
912 pattern, are decoded correctly.
913 @end ignore
914
915 @vindex auto-coding-functions
916 Another way to specify a coding system is with the variable
917 @code{auto-coding-functions}. For example, one of the builtin
918 @code{auto-coding-functions} detects the encoding for XML files.
919 Unlike the previous two, this variable does not override any
920 @samp{-*-coding:-*-} tag.
921
922 @c FIXME? This seems somewhat out of place. Move to the Rmail section?
923 @vindex rmail-decode-mime-charset
924 @vindex rmail-file-coding-system
925 When you get new mail in Rmail, each message is translated
926 automatically from the coding system it is written in, as if it were a
927 separate file. This uses the priority list of coding systems that you
928 have specified. If a MIME message specifies a character set, Rmail
929 obeys that specification. For reading and saving Rmail files
930 themselves, Emacs uses the coding system specified by the variable
931 @code{rmail-file-coding-system}. The default value is @code{nil},
932 which means that Rmail files are not translated (they are read and
933 written in the Emacs internal character code).
934
935 @node Specify Coding
936 @section Specifying a File's Coding System
937
938 If Emacs recognizes the encoding of a file incorrectly, you can
939 reread the file using the correct coding system with @kbd{C-x
940 @key{RET} r} (@code{revert-buffer-with-coding-system}). This command
941 prompts for the coding system to use. To see what coding system Emacs
942 actually used to decode the file, look at the coding system mnemonic
943 letter near the left edge of the mode line (@pxref{Mode Line}), or
944 type @kbd{C-h C} (@code{describe-coding-system}).
945
946 @vindex coding
947 You can specify the coding system for a particular file in the file
948 itself, using the @w{@samp{-*-@dots{}-*-}} construct at the beginning,
949 or a local variables list at the end (@pxref{File Variables}). You do
950 this by defining a value for the ``variable'' named @code{coding}.
951 Emacs does not really have a variable @code{coding}; instead of
952 setting a variable, this uses the specified coding system for the
953 file. For example, @samp{-*-mode: C; coding: latin-1;-*-} specifies
954 use of the Latin-1 coding system, as well as C mode. When you specify
955 the coding explicitly in the file, that overrides
956 @code{file-coding-system-alist}.
957
958 @node Output Coding
959 @section Choosing Coding Systems for Output
960
961 @vindex buffer-file-coding-system
962 Once Emacs has chosen a coding system for a buffer, it stores that
963 coding system in @code{buffer-file-coding-system}. That makes it the
964 default for operations that write from this buffer into a file, such
965 as @code{save-buffer} and @code{write-region}. You can specify a
966 different coding system for further file output from the buffer using
967 @code{set-buffer-file-coding-system} (@pxref{Text Coding}).
968
969 You can insert any character Emacs supports into any Emacs buffer,
970 but most coding systems can only handle a subset of these characters.
971 Therefore, it's possible that the characters you insert cannot be
972 encoded with the coding system that will be used to save the buffer.
973 For example, you could visit a text file in Polish, encoded in
974 @code{iso-8859-2}, and add some Russian words to it. When you save
975 that buffer, Emacs cannot use the current value of
976 @code{buffer-file-coding-system}, because the characters you added
977 cannot be encoded by that coding system.
978
979 When that happens, Emacs tries the most-preferred coding system (set
980 by @kbd{M-x prefer-coding-system} or @kbd{M-x
981 set-language-environment}). If that coding system can safely encode
982 all of the characters in the buffer, Emacs uses it, and stores its
983 value in @code{buffer-file-coding-system}. Otherwise, Emacs displays
984 a list of coding systems suitable for encoding the buffer's contents,
985 and asks you to choose one of those coding systems.
986
987 If you insert the unsuitable characters in a mail message, Emacs
988 behaves a bit differently. It additionally checks whether the
989 @c What determines this?
990 most-preferred coding system is recommended for use in MIME messages;
991 if not, it informs you of this fact and prompts you for another coding
992 system. This is so you won't inadvertently send a message encoded in
993 a way that your recipient's mail software will have difficulty
994 decoding. (You can still use an unsuitable coding system if you enter
995 its name at the prompt.)
996
997 @c It seems that select-message-coding-system does this.
998 @c Both sendmail.el and smptmail.el call it; i.e. smtpmail.el still
999 @c obeys sendmail-coding-system.
1000 @vindex sendmail-coding-system
1001 When you send a mail message (@pxref{Sending Mail}),
1002 Emacs has four different ways to determine the coding system to use
1003 for encoding the message text. It tries the buffer's own value of
1004 @code{buffer-file-coding-system}, if that is non-@code{nil}.
1005 Otherwise, it uses the value of @code{sendmail-coding-system}, if that
1006 is non-@code{nil}. The third way is to use the default coding system
1007 for new files, which is controlled by your choice of language
1008 @c i.e., default-sendmail-coding-system
1009 environment, if that is non-@code{nil}. If all of these three values
1010 are @code{nil}, Emacs encodes outgoing mail using the Latin-1 coding
1011 system.
1012 @c FIXME? Where does the Latin-1 default come in?
1013
1014 @node Text Coding
1015 @section Specifying a Coding System for File Text
1016
1017 In cases where Emacs does not automatically choose the right coding
1018 system for a file's contents, you can use these commands to specify
1019 one:
1020
1021 @table @kbd
1022 @item C-x @key{RET} f @var{coding} @key{RET}
1023 Use coding system @var{coding} to save or revisit the file in
1024 the current buffer (@code{set-buffer-file-coding-system}).
1025
1026 @item C-x @key{RET} c @var{coding} @key{RET}
1027 Specify coding system @var{coding} for the immediately following
1028 command (@code{universal-coding-system-argument}).
1029
1030 @item C-x @key{RET} r @var{coding} @key{RET}
1031 Revisit the current file using the coding system @var{coding}
1032 (@code{revert-buffer-with-coding-system}).
1033
1034 @item M-x recode-region @key{RET} @var{right} @key{RET} @var{wrong} @key{RET}
1035 Convert a region that was decoded using coding system @var{wrong},
1036 decoding it using coding system @var{right} instead.
1037 @end table
1038
1039 @kindex C-x RET f
1040 @findex set-buffer-file-coding-system
1041 The command @kbd{C-x @key{RET} f}
1042 (@code{set-buffer-file-coding-system}) sets the file coding system for
1043 the current buffer---in other words, it says which coding system to
1044 use when saving or reverting the visited file. You specify which
1045 coding system using the minibuffer. If you specify a coding system
1046 that cannot handle all of the characters in the buffer, Emacs warns
1047 you about the troublesome characters when you actually save the
1048 buffer.
1049
1050 @cindex specify end-of-line conversion
1051 You can also use this command to specify the end-of-line conversion
1052 (@pxref{Coding Systems, end-of-line conversion}) for encoding the
1053 current buffer. For example, @kbd{C-x @key{RET} f dos @key{RET}} will
1054 cause Emacs to save the current buffer's text with DOS-style
1055 carriage-return linefeed line endings.
1056
1057 @kindex C-x RET c
1058 @findex universal-coding-system-argument
1059 Another way to specify the coding system for a file is when you visit
1060 the file. First use the command @kbd{C-x @key{RET} c}
1061 (@code{universal-coding-system-argument}); this command uses the
1062 minibuffer to read a coding system name. After you exit the minibuffer,
1063 the specified coding system is used for @emph{the immediately following
1064 command}.
1065
1066 So if the immediately following command is @kbd{C-x C-f}, for example,
1067 it reads the file using that coding system (and records the coding
1068 system for when you later save the file). Or if the immediately following
1069 command is @kbd{C-x C-w}, it writes the file using that coding system.
1070 When you specify the coding system for saving in this way, instead
1071 of with @kbd{C-x @key{RET} f}, there is no warning if the buffer
1072 contains characters that the coding system cannot handle.
1073
1074 Other file commands affected by a specified coding system include
1075 @kbd{C-x i} and @kbd{C-x C-v}, as well as the other-window variants
1076 of @kbd{C-x C-f}. @kbd{C-x @key{RET} c} also affects commands that
1077 start subprocesses, including @kbd{M-x shell} (@pxref{Shell}). If the
1078 immediately following command does not use the coding system, then
1079 @kbd{C-x @key{RET} c} ultimately has no effect.
1080
1081 An easy way to visit a file with no conversion is with the @kbd{M-x
1082 find-file-literally} command. @xref{Visiting}.
1083
1084 The default value of the variable @code{buffer-file-coding-system}
1085 specifies the choice of coding system to use when you create a new file.
1086 It applies when you find a new file, and when you create a buffer and
1087 then save it in a file. Selecting a language environment typically sets
1088 this variable to a good choice of default coding system for that language
1089 environment.
1090
1091 @kindex C-x RET r
1092 @findex revert-buffer-with-coding-system
1093 If you visit a file with a wrong coding system, you can correct this
1094 with @kbd{C-x @key{RET} r} (@code{revert-buffer-with-coding-system}).
1095 This visits the current file again, using a coding system you specify.
1096
1097 @findex recode-region
1098 If a piece of text has already been inserted into a buffer using the
1099 wrong coding system, you can redo the decoding of it using @kbd{M-x
1100 recode-region}. This prompts you for the proper coding system, then
1101 for the wrong coding system that was actually used, and does the
1102 conversion. It first encodes the region using the wrong coding system,
1103 then decodes it again using the proper coding system.
1104
1105 @node Communication Coding
1106 @section Coding Systems for Interprocess Communication
1107
1108 This section explains how to specify coding systems for use
1109 in communication with other processes.
1110
1111 @table @kbd
1112 @item C-x @key{RET} x @var{coding} @key{RET}
1113 Use coding system @var{coding} for transferring selections to and from
1114 other graphical applications (@code{set-selection-coding-system}).
1115
1116 @item C-x @key{RET} X @var{coding} @key{RET}
1117 Use coding system @var{coding} for transferring @emph{one}
1118 selection---the next one---to or from another graphical application
1119 (@code{set-next-selection-coding-system}).
1120
1121 @item C-x @key{RET} p @var{input-coding} @key{RET} @var{output-coding} @key{RET}
1122 Use coding systems @var{input-coding} and @var{output-coding} for
1123 subprocess input and output in the current buffer
1124 (@code{set-buffer-process-coding-system}).
1125 @end table
1126
1127 @kindex C-x RET x
1128 @kindex C-x RET X
1129 @findex set-selection-coding-system
1130 @findex set-next-selection-coding-system
1131 The command @kbd{C-x @key{RET} x} (@code{set-selection-coding-system})
1132 specifies the coding system for sending selected text to other windowing
1133 applications, and for receiving the text of selections made in other
1134 applications. This command applies to all subsequent selections, until
1135 you override it by using the command again. The command @kbd{C-x
1136 @key{RET} X} (@code{set-next-selection-coding-system}) specifies the
1137 coding system for the next selection made in Emacs or read by Emacs.
1138
1139 @vindex x-select-request-type
1140 The variable @code{x-select-request-type} specifies the data type to
1141 request from the X Window System for receiving text selections from
1142 other applications. If the value is @code{nil} (the default), Emacs
1143 tries @code{UTF8_STRING} and @code{COMPOUND_TEXT}, in this order, and
1144 uses various heuristics to choose the more appropriate of the two
1145 results; if none of these succeed, Emacs falls back on @code{STRING}.
1146 If the value of @code{x-select-request-type} is one of the symbols
1147 @code{COMPOUND_TEXT}, @code{UTF8_STRING}, @code{STRING}, or
1148 @code{TEXT}, Emacs uses only that request type. If the value is a
1149 list of some of these symbols, Emacs tries only the request types in
1150 the list, in order, until one of them succeeds, or until the list is
1151 exhausted.
1152
1153 @kindex C-x RET p
1154 @findex set-buffer-process-coding-system
1155 The command @kbd{C-x @key{RET} p} (@code{set-buffer-process-coding-system})
1156 specifies the coding system for input and output to a subprocess. This
1157 command applies to the current buffer; normally, each subprocess has its
1158 own buffer, and thus you can use this command to specify translation to
1159 and from a particular subprocess by giving the command in the
1160 corresponding buffer.
1161
1162 You can also use @kbd{C-x @key{RET} c}
1163 (@code{universal-coding-system-argument}) just before the command that
1164 runs or starts a subprocess, to specify the coding system for
1165 communicating with that subprocess. @xref{Text Coding}.
1166
1167 The default for translation of process input and output depends on the
1168 current language environment.
1169
1170 @vindex locale-coding-system
1171 @cindex decoding non-@acronym{ASCII} keyboard input on X
1172 The variable @code{locale-coding-system} specifies a coding system
1173 to use when encoding and decoding system strings such as system error
1174 messages and @code{format-time-string} formats and time stamps. That
1175 coding system is also used for decoding non-@acronym{ASCII} keyboard
1176 input on the X Window System. You should choose a coding system that is compatible
1177 with the underlying system's text representation, which is normally
1178 specified by one of the environment variables @env{LC_ALL},
1179 @env{LC_CTYPE}, and @env{LANG}. (The first one, in the order
1180 specified above, whose value is nonempty is the one that determines
1181 the text representation.)
1182
1183 @node File Name Coding
1184 @section Coding Systems for File Names
1185
1186 @table @kbd
1187 @item C-x @key{RET} F @var{coding} @key{RET}
1188 Use coding system @var{coding} for encoding and decoding file
1189 names (@code{set-file-name-coding-system}).
1190 @end table
1191
1192 @findex set-file-name-coding-system
1193 @kindex C-x @key{RET} F
1194 @cindex file names with non-@acronym{ASCII} characters
1195 The command @kbd{C-x @key{RET} F} (@code{set-file-name-coding-system})
1196 specifies a coding system to use for encoding file @emph{names}. It
1197 has no effect on reading and writing the @emph{contents} of files.
1198
1199 @vindex file-name-coding-system
1200 In fact, all this command does is set the value of the variable
1201 @code{file-name-coding-system}. If you set the variable to a coding
1202 system name (as a Lisp symbol or a string), Emacs encodes file names
1203 using that coding system for all file operations. This makes it
1204 possible to use non-@acronym{ASCII} characters in file names---or, at
1205 least, those non-@acronym{ASCII} characters that the specified coding
1206 system can encode.
1207
1208 If @code{file-name-coding-system} is @code{nil}, Emacs uses a
1209 default coding system determined by the selected language environment,
1210 and stored in the @code{default-file-name-coding-system} variable.
1211 @c FIXME? Is this correct? What is the "default language environment"?
1212 In the default language environment, non-@acronym{ASCII} characters in
1213 file names are not encoded specially; they appear in the file system
1214 using the internal Emacs representation.
1215
1216 @strong{Warning:} if you change @code{file-name-coding-system} (or the
1217 language environment) in the middle of an Emacs session, problems can
1218 result if you have already visited files whose names were encoded using
1219 the earlier coding system and cannot be encoded (or are encoded
1220 differently) under the new coding system. If you try to save one of
1221 these buffers under the visited file name, saving may use the wrong file
1222 name, or it may encounter an error. If such a problem happens, use @kbd{C-x
1223 C-w} to specify a new file name for that buffer.
1224
1225 @findex recode-file-name
1226 If a mistake occurs when encoding a file name, use the command
1227 @kbd{M-x recode-file-name} to change the file name's coding
1228 system. This prompts for an existing file name, its old coding
1229 system, and the coding system to which you wish to convert.
1230
1231 @node Terminal Coding
1232 @section Coding Systems for Terminal I/O
1233
1234 @table @kbd
1235 @item C-x @key{RET} t @var{coding} @key{RET}
1236 Use coding system @var{coding} for terminal output
1237 (@code{set-terminal-coding-system}).
1238
1239 @item C-x @key{RET} k @var{coding} @key{RET}
1240 Use coding system @var{coding} for keyboard input
1241 (@code{set-keyboard-coding-system}).
1242 @end table
1243
1244 @kindex C-x RET t
1245 @findex set-terminal-coding-system
1246 The command @kbd{C-x @key{RET} t} (@code{set-terminal-coding-system})
1247 specifies the coding system for terminal output. If you specify a
1248 character code for terminal output, all characters output to the
1249 terminal are translated into that coding system.
1250
1251 This feature is useful for certain character-only terminals built to
1252 support specific languages or character sets---for example, European
1253 terminals that support one of the ISO Latin character sets. You need to
1254 specify the terminal coding system when using multibyte text, so that
1255 Emacs knows which characters the terminal can actually handle.
1256
1257 By default, output to the terminal is not translated at all, unless
1258 Emacs can deduce the proper coding system from your terminal type or
1259 your locale specification (@pxref{Language Environments}).
1260
1261 @kindex C-x RET k
1262 @findex set-keyboard-coding-system
1263 @vindex keyboard-coding-system
1264 The command @kbd{C-x @key{RET} k} (@code{set-keyboard-coding-system}),
1265 or the variable @code{keyboard-coding-system}, specifies the coding
1266 system for keyboard input. Character-code translation of keyboard
1267 input is useful for terminals with keys that send non-@acronym{ASCII}
1268 graphic characters---for example, some terminals designed for ISO
1269 Latin-1 or subsets of it.
1270
1271 By default, keyboard input is translated based on your system locale
1272 setting. If your terminal does not really support the encoding
1273 implied by your locale (for example, if you find it inserts a
1274 non-@acronym{ASCII} character if you type @kbd{M-i}), you will need to set
1275 @code{keyboard-coding-system} to @code{nil} to turn off encoding.
1276 You can do this by putting
1277
1278 @lisp
1279 (set-keyboard-coding-system nil)
1280 @end lisp
1281
1282 @noindent
1283 in your init file.
1284
1285 There is a similarity between using a coding system translation for
1286 keyboard input, and using an input method: both define sequences of
1287 keyboard input that translate into single characters. However, input
1288 methods are designed to be convenient for interactive use by humans, and
1289 the sequences that are translated are typically sequences of @acronym{ASCII}
1290 printing characters. Coding systems typically translate sequences of
1291 non-graphic characters.
1292
1293 @node Fontsets
1294 @section Fontsets
1295 @cindex fontsets
1296
1297 A font typically defines shapes for a single alphabet or script.
1298 Therefore, displaying the entire range of scripts that Emacs supports
1299 requires a collection of many fonts. In Emacs, such a collection is
1300 called a @dfn{fontset}. A fontset is defined by a list of font specifications,
1301 each assigned to handle a range of character codes, and may fall back
1302 on another fontset for characters that are not covered by the fonts
1303 it specifies.
1304
1305 @cindex fonts for various scripts
1306 @cindex Intlfonts package, installation
1307 Each fontset has a name, like a font. However, while fonts are
1308 stored in the system and the available font names are defined by the
1309 system, fontsets are defined within Emacs itself. Once you have
1310 defined a fontset, you can use it within Emacs by specifying its name,
1311 anywhere that you could use a single font. Of course, Emacs fontsets
1312 can use only the fonts that the system supports. If some characters
1313 appear on the screen as empty boxes or hex codes, this means that the
1314 fontset in use for them has no font for those characters. In this
1315 case, or if the characters are shown, but not as well as you would
1316 like, you may need to install extra fonts. Your operating system may
1317 have optional fonts that you can install; or you can install the GNU
1318 Intlfonts package, which includes fonts for most supported
1319 scripts.@footnote{If you run Emacs on X, you may need to inform the X
1320 server about the location of the newly installed fonts with commands
1321 such as:
1322 @c FIXME? I feel like this may be out of date.
1323 @c Eg the intlfonts tarfile is ~ 10 years old.
1324
1325 @example
1326 xset fp+ /usr/local/share/emacs/fonts
1327 xset fp rehash
1328 @end example
1329 }
1330
1331 Emacs creates three fontsets automatically: the @dfn{standard
1332 fontset}, the @dfn{startup fontset} and the @dfn{default fontset}.
1333 @c FIXME? The doc of *standard*-fontset-spec says:
1334 @c "You have the biggest chance to display international characters
1335 @c with correct glyphs by using the *standard* fontset." (my emphasis)
1336 @c See http://lists.gnu.org/archive/html/emacs-devel/2012-04/msg00430.html
1337 The default fontset is most likely to have fonts for a wide variety of
1338 non-@acronym{ASCII} characters, and is the default fallback for the
1339 other two fontsets, and if you set a default font rather than fontset.
1340 However, it does not specify font family names, so results can be
1341 somewhat random if you use it directly. You can specify use of a
1342 particular fontset by starting Emacs with the @samp{-fn} option.
1343 For example,
1344
1345 @example
1346 emacs -fn fontset-standard
1347 @end example
1348
1349 @noindent
1350 You can also specify a fontset with the @samp{Font} resource (@pxref{X
1351 Resources}).
1352
1353 If no fontset is specified for use, then Emacs uses an
1354 @acronym{ASCII} font, with @samp{fontset-default} as a fallback for
1355 characters the font does not cover. The standard fontset is only used if
1356 explicitly requested, despite its name.
1357
1358 A fontset does not necessarily specify a font for every character
1359 code. If a fontset specifies no font for a certain character, or if
1360 it specifies a font that does not exist on your system, then it cannot
1361 display that character properly. It will display that character as a
1362 hex code or thin space or an empty box instead. (@xref{Text Display, ,
1363 glyphless characters}, for details.)
1364
1365 @node Defining Fontsets
1366 @section Defining fontsets
1367
1368 @vindex standard-fontset-spec
1369 @vindex w32-standard-fontset-spec
1370 @vindex ns-standard-fontset-spec
1371 @cindex standard fontset
1372 When running on X, Emacs creates a standard fontset automatically according to the value
1373 of @code{standard-fontset-spec}. This fontset's name is
1374
1375 @example
1376 -*-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-standard
1377 @end example
1378
1379 @noindent
1380 or just @samp{fontset-standard} for short.
1381
1382 On GNUstep and Mac OS X, the standard fontset is created using the value of
1383 @code{ns-standard-fontset-spec}, and on MS Windows it is
1384 created using the value of @code{w32-standard-fontset-spec}.
1385
1386 @c FIXME? How does one access these, or do anything with them?
1387 @c Does it matter?
1388 Bold, italic, and bold-italic variants of the standard fontset are
1389 created automatically. Their names have @samp{bold} instead of
1390 @samp{medium}, or @samp{i} instead of @samp{r}, or both.
1391
1392 @cindex startup fontset
1393 Emacs generates a fontset automatically, based on any default
1394 @acronym{ASCII} font that you specify with the @samp{Font} resource or
1395 the @samp{-fn} argument, or the default font that Emacs found when it
1396 started. This is the @dfn{startup fontset} and its name is
1397 @code{fontset-startup}. It does this by replacing the
1398 @var{charset_registry} field with @samp{fontset}, and replacing
1399 @var{charset_encoding} field with @samp{startup}, then using the
1400 resulting string to specify a fontset.
1401
1402 For instance, if you start Emacs with a font of this form,
1403
1404 @c FIXME? I think this is a little misleading, because you cannot (?)
1405 @c actually specify a font with wildcards, it has to be a complete spec.
1406 @c Also, an X font specification of this form hasn't (?) been
1407 @c mentioned before now, and is somewhat obsolete these days.
1408 @c People are more likely to use a form like
1409 @c emacs -fn "DejaVu Sans Mono-12"
1410 @c How does any of this apply in that case?
1411 @example
1412 emacs -fn "*courier-medium-r-normal--14-140-*-iso8859-1"
1413 @end example
1414
1415 @noindent
1416 Emacs generates the following fontset and uses it for the initial X
1417 window frame:
1418
1419 @example
1420 -*-courier-medium-r-normal-*-14-140-*-*-*-*-fontset-startup
1421 @end example
1422
1423 The startup fontset will use the font that you specify, or a variant
1424 with a different registry and encoding, for all the characters that
1425 are supported by that font, and fallback on @samp{fontset-default} for
1426 other characters.
1427
1428 With the X resource @samp{Emacs.Font}, you can specify a fontset name
1429 just like an actual font name. But be careful not to specify a fontset
1430 name in a wildcard resource like @samp{Emacs*Font}---that wildcard
1431 specification matches various other resources, such as for menus, and
1432 @c FIXME is this still true?
1433 menus cannot handle fontsets. @xref{X Resources}.
1434
1435 You can specify additional fontsets using X resources named
1436 @samp{Fontset-@var{n}}, where @var{n} is an integer starting from 0.
1437 The resource value should have this form:
1438
1439 @smallexample
1440 @var{fontpattern}, @r{[}@var{charset}:@var{font}@r{]@dots{}}
1441 @end smallexample
1442
1443 @noindent
1444 @var{fontpattern} should have the form of a standard X font name (see
1445 the previous fontset-startup example), except
1446 for the last two fields. They should have the form
1447 @samp{fontset-@var{alias}}.
1448
1449 The fontset has two names, one long and one short. The long name is
1450 @var{fontpattern}. The short name is @samp{fontset-@var{alias}}. You
1451 can refer to the fontset by either name.
1452
1453 The construct @samp{@var{charset}:@var{font}} specifies which font to
1454 use (in this fontset) for one particular character set. Here,
1455 @var{charset} is the name of a character set, and @var{font} is the
1456 font to use for that character set. You can use this construct any
1457 number of times in defining one fontset.
1458
1459 For the other character sets, Emacs chooses a font based on
1460 @var{fontpattern}. It replaces @samp{fontset-@var{alias}} with values
1461 that describe the character set. For the @acronym{ASCII} character font,
1462 @samp{fontset-@var{alias}} is replaced with @samp{ISO8859-1}.
1463
1464 In addition, when several consecutive fields are wildcards, Emacs
1465 collapses them into a single wildcard. This is to prevent use of
1466 auto-scaled fonts. Fonts made by scaling larger fonts are not usable
1467 for editing, and scaling a smaller font is not also useful, because it is
1468 better to use the smaller font in its own size, which is what Emacs
1469 does.
1470
1471 Thus if @var{fontpattern} is this,
1472
1473 @example
1474 -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24
1475 @end example
1476
1477 @noindent
1478 the font specification for @acronym{ASCII} characters would be this:
1479
1480 @example
1481 -*-fixed-medium-r-normal-*-24-*-ISO8859-1
1482 @end example
1483
1484 @noindent
1485 and the font specification for Chinese GB2312 characters would be this:
1486
1487 @example
1488 -*-fixed-medium-r-normal-*-24-*-gb2312*-*
1489 @end example
1490
1491 You may not have any Chinese font matching the above font
1492 specification. Most X distributions include only Chinese fonts that
1493 have @samp{song ti} or @samp{fangsong ti} in the @var{family} field. In
1494 such a case, @samp{Fontset-@var{n}} can be specified as:
1495
1496 @smallexample
1497 Emacs.Fontset-0: -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24,\
1498 chinese-gb2312:-*-*-medium-r-normal-*-24-*-gb2312*-*
1499 @end smallexample
1500
1501 @noindent
1502 Then, the font specifications for all but Chinese GB2312 characters have
1503 @samp{fixed} in the @var{family} field, and the font specification for
1504 Chinese GB2312 characters has a wild card @samp{*} in the @var{family}
1505 field.
1506
1507 @findex create-fontset-from-fontset-spec
1508 The function that processes the fontset resource value to create the
1509 fontset is called @code{create-fontset-from-fontset-spec}. You can also
1510 call this function explicitly to create a fontset.
1511
1512 @xref{Fonts}, for more information about font naming.
1513
1514 @node Modifying Fontsets
1515 @section Modifying Fontsets
1516 @cindex fontsets, modifying
1517 @findex set-fontset-font
1518
1519 Fontsets do not always have to be created from scratch. If only
1520 minor changes are required it may be easier to modify an existing
1521 fontset. Modifying @samp{fontset-default} will also affect other
1522 fontsets that use it as a fallback, so can be an effective way of
1523 fixing problems with the fonts that Emacs chooses for a particular
1524 script.
1525
1526 Fontsets can be modified using the function @code{set-fontset-font},
1527 specifying a character, a charset, a script, or a range of characters
1528 to modify the font for, and a font specification for the font to be
1529 used. Some examples are:
1530
1531 @example
1532 ;; Use Liberation Mono for latin-3 charset.
1533 (set-fontset-font "fontset-default" 'iso-8859-3
1534 "Liberation Mono")
1535
1536 ;; Prefer a big5 font for han characters
1537 (set-fontset-font "fontset-default"
1538 'han (font-spec :registry "big5")
1539 nil 'prepend)
1540
1541 ;; Use DejaVu Sans Mono as a fallback in fontset-startup
1542 ;; before resorting to fontset-default.
1543 (set-fontset-font "fontset-startup" nil "DejaVu Sans Mono"
1544 nil 'append)
1545
1546 ;; Use MyPrivateFont for the Unicode private use area.
1547 (set-fontset-font "fontset-default" '(#xe000 . #xf8ff)
1548 "MyPrivateFont")
1549
1550 @end example
1551
1552
1553 @node Undisplayable Characters
1554 @section Undisplayable Characters
1555
1556 There may be some non-@acronym{ASCII} characters that your
1557 terminal cannot display. Most text terminals support just a single
1558 character set (use the variable @code{default-terminal-coding-system}
1559 to tell Emacs which one, @ref{Terminal Coding}); characters that
1560 can't be encoded in that coding system are displayed as @samp{?} by
1561 default.
1562
1563 Graphical displays can display a broader range of characters, but
1564 you may not have fonts installed for all of them; characters that have
1565 no font appear as a hollow box.
1566
1567 If you use Latin-1 characters but your terminal can't display
1568 Latin-1, you can arrange to display mnemonic @acronym{ASCII} sequences
1569 instead, e.g.@: @samp{"o} for o-umlaut. Load the library
1570 @file{iso-ascii} to do this.
1571
1572 @vindex latin1-display
1573 If your terminal can display Latin-1, you can display characters
1574 from other European character sets using a mixture of equivalent
1575 Latin-1 characters and @acronym{ASCII} mnemonics. Customize the variable
1576 @code{latin1-display} to enable this. The mnemonic @acronym{ASCII}
1577 sequences mostly correspond to those of the prefix input methods.
1578
1579 @node Unibyte Mode
1580 @section Unibyte Editing Mode
1581
1582 @cindex European character sets
1583 @cindex accented characters
1584 @cindex ISO Latin character sets
1585 @cindex Unibyte operation
1586 The ISO 8859 Latin-@var{n} character sets define character codes in
1587 the range 0240 to 0377 octal (160 to 255 decimal) to handle the
1588 accented letters and punctuation needed by various European languages
1589 (and some non-European ones). Note that Emacs considers bytes with
1590 codes in this range as raw bytes, not as characters, even in a unibyte
1591 buffer, i.e.@: if you disable multibyte characters. However, Emacs
1592 can still handle these character codes as if they belonged to
1593 @emph{one} of the single-byte character sets at a time. To specify
1594 @emph{which} of these codes to use, invoke @kbd{M-x
1595 set-language-environment} and specify a suitable language environment
1596 such as @samp{Latin-@var{n}}.
1597
1598 For more information about unibyte operation, see
1599 @ref{Disabling Multibyte}.
1600
1601 @vindex unibyte-display-via-language-environment
1602 Emacs can also display bytes in the range 160 to 255 as readable
1603 characters, provided the terminal or font in use supports them. This
1604 works automatically. On a graphical display, Emacs can also display
1605 single-byte characters through fontsets, in effect by displaying the
1606 equivalent multibyte characters according to the current language
1607 environment. To request this, set the variable
1608 @code{unibyte-display-via-language-environment} to a non-@code{nil}
1609 value. Note that setting this only affects how these bytes are
1610 displayed, but does not change the fundamental fact that Emacs treats
1611 them as raw bytes, not as characters.
1612
1613 @cindex @code{iso-ascii} library
1614 If your terminal does not support display of the Latin-1 character
1615 set, Emacs can display these characters as @acronym{ASCII} sequences which at
1616 least give you a clear idea of what the characters are. To do this,
1617 load the library @code{iso-ascii}. Similar libraries for other
1618 Latin-@var{n} character sets could be implemented, but have not been
1619 so far.
1620
1621 @findex standard-display-8bit
1622 @cindex 8-bit display
1623 Normally non-ISO-8859 characters (decimal codes between 128 and 159
1624 inclusive) are displayed as octal escapes. You can change this for
1625 non-standard ``extended'' versions of ISO-8859 character sets by using the
1626 function @code{standard-display-8bit} in the @code{disp-table} library.
1627
1628 There are two ways to input single-byte non-@acronym{ASCII}
1629 characters:
1630
1631 @itemize @bullet
1632 @cindex 8-bit input
1633 @item
1634 You can use an input method for the selected language environment.
1635 @xref{Input Methods}. When you use an input method in a unibyte buffer,
1636 the non-@acronym{ASCII} character you specify with it is converted to unibyte.
1637
1638 @item
1639 If your keyboard can generate character codes 128 (decimal) and up,
1640 representing non-@acronym{ASCII} characters, you can type those character codes
1641 directly.
1642
1643 On a graphical display, you should not need to do anything special to
1644 use these keys; they should simply work. On a text terminal, you
1645 should use the command @code{M-x set-keyboard-coding-system} or customize the
1646 variable @code{keyboard-coding-system} to specify which coding system
1647 your keyboard uses (@pxref{Terminal Coding}). Enabling this feature
1648 will probably require you to use @kbd{ESC} to type Meta characters;
1649 however, on a console terminal or in @code{xterm}, you can arrange for
1650 Meta to be converted to @kbd{ESC} and still be able type 8-bit
1651 characters present directly on the keyboard or using @kbd{Compose} or
1652 @kbd{AltGr} keys. @xref{User Input}.
1653
1654 @kindex C-x 8
1655 @cindex @code{iso-transl} library
1656 @cindex compose character
1657 @cindex dead character
1658 @item
1659 For Latin-1 only, you can use the key @kbd{C-x 8} as a ``compose
1660 character'' prefix for entry of non-@acronym{ASCII} Latin-1 printing
1661 characters. @kbd{C-x 8} is good for insertion (in the minibuffer as
1662 well as other buffers), for searching, and in any other context where
1663 a key sequence is allowed.
1664
1665 @kbd{C-x 8} works by loading the @code{iso-transl} library. Once that
1666 library is loaded, the @key{ALT} modifier key, if the keyboard has
1667 one, serves the same purpose as @kbd{C-x 8}: use @key{ALT} together
1668 with an accent character to modify the following letter. In addition,
1669 if the keyboard has keys for the Latin-1 ``dead accent characters'',
1670 they too are defined to compose with the following character, once
1671 @code{iso-transl} is loaded.
1672
1673 Use @kbd{C-x 8 C-h} to list all the available @kbd{C-x 8} translations.
1674 @end itemize
1675
1676 @node Charsets
1677 @section Charsets
1678 @cindex charsets
1679
1680 In Emacs, @dfn{charset} is short for ``character set''. Emacs
1681 supports most popular charsets (such as @code{ascii},
1682 @code{iso-8859-1}, @code{cp1250}, @code{big5}, and @code{unicode}), in
1683 addition to some charsets of its own (such as @code{emacs},
1684 @code{unicode-bmp}, and @code{eight-bit}). All supported characters
1685 belong to one or more charsets.
1686
1687 Emacs normally ``does the right thing'' with respect to charsets, so
1688 that you don't have to worry about them. However, it is sometimes
1689 helpful to know some of the underlying details about charsets.
1690
1691 One example is font selection (@pxref{Fonts}). Each language
1692 environment (@pxref{Language Environments}) defines a ``priority
1693 list'' for the various charsets. When searching for a font, Emacs
1694 initially attempts to find one that can display the highest-priority
1695 charsets. For instance, in the Japanese language environment, the
1696 charset @code{japanese-jisx0208} has the highest priority, so Emacs
1697 tries to use a font whose @code{registry} property is
1698 @samp{JISX0208.1983-0}.
1699
1700 @findex list-charset-chars
1701 @cindex characters in a certain charset
1702 @findex describe-character-set
1703 There are two commands that can be used to obtain information about
1704 charsets. The command @kbd{M-x list-charset-chars} prompts for a
1705 charset name, and displays all the characters in that character set.
1706 The command @kbd{M-x describe-character-set} prompts for a charset
1707 name, and displays information about that charset, including its
1708 internal representation within Emacs.
1709
1710 @findex list-character-sets
1711 @kbd{M-x list-character-sets} displays a list of all supported
1712 charsets. The list gives the names of charsets and additional
1713 information to identity each charset; see the
1714 @url{http://www.itscj.ipsj.or.jp/ISO-IR/, International Register of
1715 Coded Character Sets} for more details. In this list,
1716 charsets are divided into two categories: @dfn{normal charsets} are
1717 listed first, followed by @dfn{supplementary charsets}. A
1718 supplementary charset is one that is used to define another charset
1719 (as a parent or a subset), or to provide backward-compatibility for
1720 older Emacs versions.
1721
1722 To find out which charset a character in the buffer belongs to, put
1723 point before it and type @kbd{C-u C-x =} (@pxref{International
1724 Chars}).
1725
1726 @node Bidirectional Editing
1727 @section Bidirectional Editing
1728 @cindex bidirectional editing
1729 @cindex right-to-left text
1730
1731 Emacs supports editing text written in scripts, such as Arabic and
1732 Hebrew, whose natural ordering of horizontal text for display is from
1733 right to left. However, digits and Latin text embedded in these
1734 scripts are still displayed left to right. It is also not uncommon to
1735 have small portions of text in Arabic or Hebrew embedded in an otherwise
1736 Latin document; e.g., as comments and strings in a program source
1737 file. For these reasons, text that uses these scripts is actually
1738 @dfn{bidirectional}: a mixture of runs of left-to-right and
1739 right-to-left characters.
1740
1741 This section describes the facilities and options provided by Emacs
1742 for editing bidirectional text.
1743
1744 @cindex logical order
1745 @cindex visual order
1746 Emacs stores right-to-left and bidirectional text in the so-called
1747 @dfn{logical} (or @dfn{reading}) order: the buffer or string position
1748 of the first character you read precedes that of the next character.
1749 Reordering of bidirectional text into the @dfn{visual} order happens
1750 at display time. As result, character positions no longer increase
1751 monotonically with their positions on display. Emacs implements the
1752 Unicode Bidirectional Algorithm described in the Unicode Standard
1753 Annex #9, for reordering of bidirectional text for display.
1754
1755 @vindex bidi-display-reordering
1756 The buffer-local variable @code{bidi-display-reordering} controls
1757 whether text in the buffer is reordered for display. If its value is
1758 non-@code{nil}, Emacs reorders characters that have right-to-left
1759 directionality when they are displayed. The default value is
1760 @code{t}.
1761
1762 @cindex base direction of paragraphs
1763 @cindex paragraph, base direction
1764 Each paragraph of bidirectional text can have its own @dfn{base
1765 direction}, either right-to-left or left-to-right. (Paragraph
1766 @c paragraph-separate etc have no influence on this?
1767 boundaries are empty lines, i.e.@: lines consisting entirely of
1768 whitespace characters.) Text in left-to-right paragraphs begins on
1769 the screen at the left margin of the window and is truncated or
1770 continued when it reaches the right margin. By contrast, text in
1771 right-to-left paragraphs is displayed starting at the right margin and
1772 is continued or truncated at the left margin.
1773
1774 @vindex bidi-paragraph-direction
1775 Emacs determines the base direction of each paragraph dynamically,
1776 based on the text at the beginning of the paragraph. However,
1777 sometimes a buffer may need to force a certain base direction for its
1778 paragraphs. The variable @code{bidi-paragraph-direction}, if
1779 non-@code{nil}, disables the dynamic determination of the base
1780 direction, and instead forces all paragraphs in the buffer to have the
1781 direction specified by its buffer-local value. The value can be either
1782 @code{right-to-left} or @code{left-to-right}. Any other value is
1783 interpreted as @code{nil}.
1784
1785 @cindex LRM
1786 @cindex RLM
1787 Alternatively, you can control the base direction of a paragraph by
1788 inserting special formatting characters in front of the paragraph.
1789 The special character @code{RIGHT-TO-LEFT MARK}, or @sc{rlm}, forces
1790 the right-to-left direction on the following paragraph, while
1791 @code{LEFT-TO-RIGHT MARK}, or @sc{lrm} forces the left-to-right
1792 direction. (You can use @kbd{C-x 8 RET} to insert these characters.)
1793 In a GUI session, the @sc{lrm} and @sc{rlm} characters display as very
1794 thin blank characters; on text terminals they display as blanks.
1795
1796 Because characters are reordered for display, Emacs commands that
1797 operate in the logical order or on stretches of buffer positions may
1798 produce unusual effects. For example, @kbd{C-f} and @kbd{C-b}
1799 commands move point in the logical order, so the cursor will sometimes
1800 jump when point traverses reordered bidirectional text. Similarly, a
1801 highlighted region covering a contiguous range of character positions
1802 may look discontinuous if the region spans reordered text. This is
1803 normal and similar to the behavior of other programs that support
1804 bidirectional text.