* Translation of Characters:: Translation tables are used for conversion.
* Coding Systems:: Coding systems are conversions for saving files.
* Input Methods:: Input methods allow users to enter various
- non-ASCII characters without speciak keyboards.
+ non-ASCII characters without special keyboards.
* Locales:: Interacting with the POSIX locale.
@end menu
0377); these values are @dfn{trailing codes}.
Some sequences of bytes are not valid in multibyte text: for example,
-a single isolated byte in the range 128 through 159 is not allowed.
-But character codes 128 through 159 can appear in multibyte text,
-represented as two-byte sequences. None of the character codes 128
-through 255 normally appear in ordinary multibyte text, but they do
+a single isolated byte in the range 128 through 159 is not allowed. But
+character codes 128 through 159 can appear in multibyte text,
+represented as two-byte sequences. All the character codes 128 through
+255 are possible (though slightly abnormal) in multibyte text; they
appear in multibyte buffers and strings when you do explicit encoding
and decoding (@pxref{Explicit Encoding}).
user that cannot be overridden automatically.
Converting unibyte text to multibyte text leaves @sc{ascii} characters
-unchanged, and likewise 128 through 159. It converts the non-@sc{ascii}
-codes 160 through 255 by adding the value @code{nonascii-insert-offset}
-to each character code. By setting this variable, you specify which
-character set the unibyte characters correspond to (@pxref{Character
-Sets}). For example, if @code{nonascii-insert-offset} is 2048, which is
-@code{(- (make-char 'latin-iso8859-1) 128)}, then the unibyte
-non-@sc{ascii} characters correspond to Latin 1. If it is 2688, which
-is @code{(- (make-char 'greek-iso8859-7) 128)}, then they correspond to
-Greek letters.
+unchanged, and likewise character codes 128 through 159. It converts
+the non-@sc{ascii} codes 160 through 255 by adding the value
+@code{nonascii-insert-offset} to each character code. By setting this
+variable, you specify which character set the unibyte characters
+correspond to (@pxref{Character Sets}). For example, if
+@code{nonascii-insert-offset} is 2048, which is @code{(- (make-char
+'latin-iso8859-1) 128)}, then the unibyte non-@sc{ascii} characters
+correspond to Latin 1. If it is 2688, which is @code{(- (make-char
+'greek-iso8859-7) 128)}, then they correspond to Greek letters.
Converting multibyte text to unibyte is simpler: it discards all but
the low 8 bits of each character code. If @code{nonascii-insert-offset}
This variable specifies the amount to add to a non-@sc{ascii} character
when converting unibyte text to multibyte. It also applies when
@code{self-insert-command} inserts a character in the unibyte
-non-@sc{ascii} range, 128 through 255. However, the function
-@code{insert-char} does not perform this conversion.
+non-@sc{ascii} range, 128 through 255. However, the functions
+@code{insert} and @code{insert-char} do not perform this conversion.
The right value to use to select character set @var{cs} is @code{(-
(make-char @var{cs}) 128)}. If the value of
This variable provides a more general alternative to
@code{nonascii-insert-offset}. You can use it to specify independently
how to translate each code in the range of 128 through 255 into a
-multibyte character. The value should be a vector, or @code{nil}.
+multibyte character. The value should be a char-table, or @code{nil}.
If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
@end defvar
@defun string-make-unibyte string
This function converts the text of @var{string} to unibyte
representation, if it isn't already, and returns the result. If
-@var{string} is a unibyte string, it is returned unchanged.
+@var{string} is a unibyte string, it is returned unchanged. Multibyte
+character codes are converted to unibyte according to
+@code{nonascii-translation-table} or, if that is @code{nil}, using
+@code{nonascii-insert-offset}. If the lookup in the translation table
+fails, this function takes just the low 8 bits of each character.
@end defun
@defun string-make-multibyte string
This function converts the text of @var{string} to multibyte
representation, if it isn't already, and returns the result. If
@var{string} is a multibyte string, it is returned unchanged.
+The function @code{unibyte-char-to-multibyte} is used to convert
+each unibyte character to a multibyte character.
@end defun
@node Selecting a Representation
sequence of bytes. As a consequence, it can change the contents viewed
as characters; a sequence of two bytes which is treated as one character
in multibyte representation will count as two characters in unibyte
-representation.
+representation. Character codes 128 through 159 are an exception. They
+are represented by one byte in a unibyte buffer, but when the buffer is
+set to multibyte, they are converted to two-byte sequences, and vice
+versa.
This function sets @code{enable-multibyte-characters} to record which
representation is in use. It also adjusts various data in the buffer
more characters than @var{string} has.
If @var{string} is already a unibyte string, then the value is
-@var{string} itself.
+@var{string} itself. Otherwise it is a newly created string, with no
+text properties. If @var{string} is multibyte, any characters it
+contains of charset @var{eight-bit-control} or @var{eight-bit-graphic}
+are converted to the corresponding single byte.
@end defun
@defun string-as-multibyte string
value may have fewer characters than @var{string} has.
If @var{string} is already a multibyte string, then the value is
-@var{string} itself.
+@var{string} itself. Otherwise it is a newly created string, with no
+text properties. If @var{string} is unibyte and contains any individual
+8-bit bytes (i.e.@: not part of a multibyte form), they are converted to
+the corresponding multibyte character of charset @var{eight-bit-control}
+or @var{eight-bit-graphic}.
@end defun
@node Character Codes
0 to 255---the values that can fit in one byte. The valid character
codes for multibyte representation range from 0 to 524287, but not all
values in that range are valid. The values 128 through 255 are not
-really proper in multibyte text, but they can occur if you do explicit
+entirely proper in multibyte text, but they can occur if you do explicit
encoding and decoding (@pxref{Explicit Encoding}). Some other character
codes cannot occur at all in multibyte text. Only the @sc{ascii} codes
-0 through 127 are truly legitimate in both representations.
+0 through 127 are completely legitimate in both representations.
-@defun char-valid-p charcode
+@defun char-valid-p charcode &optional genericp
This returns @code{t} if @var{charcode} is valid for either one of the two
text representations.
(char-valid-p 2248)
@result{} t
@end example
+
+If the optional argument @var{genericp} is non-nil, this function
+returns @code{t} if @var{charcode} is a generic character
+(@pxref{Splitting Characters}).
@end defun
@node Character Sets
characters, generally known as Big 5, is divided into two Emacs
character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
+ @sc{ascii} characters are in character set @code{ascii}. The
+non-@sc{ascii} characters 128 through 159 are in character set
+@code{eight-bit-control}, and codes 160 through 255 are in character set
+@code{eight-bit-graphic}.
+
@defun charsetp object
Returns @code{t} if @var{object} is a symbol that names a character set,
@code{nil} otherwise.
This function returns the charset property list of the character set
@var{charset}. Although @var{charset} is a symbol, this is not the same
as the property list of that symbol. Charset properties are used for
-special purposes within Emacs; for example, @code{x-charset-registry}
-helps determine which fonts to use (@pxref{Font Selection}).
+special purposes within Emacs; for example,
+@code{preferred-coding-system} helps determine which coding system to
+use to encode characters in a charset.
@end defun
@node Chars and Bytes
In multibyte representation, each character occupies one or more
bytes. Each character set has an @dfn{introduction sequence}, which is
normally one or two bytes long. (Exception: the @sc{ascii} character
-set has a zero-length introduction sequence.) The introduction sequence
-is the beginning of the byte sequence for any character in the character
-set. The rest of the character's bytes distinguish it from the other
-characters in the same character set. Depending on the character set,
-there are either one or two distinguishing bytes; the number of such
-bytes is called the @dfn{dimension} of the character set.
+set and the @sc{eight-bit-graphic} character set have a zero-length
+introduction sequence.) The introduction sequence is the beginning of
+the byte sequence for any character in the character set. The rest of
+the character's bytes distinguish it from the other characters in the
+same character set. Depending on the character set, there are either
+one or two distinguishing bytes; the number of such bytes is called the
+@dfn{dimension} of the character set.
@defun charset-dimension charset
This function returns the dimension of @var{charset}; at present, the
@result{} (latin-iso8859-1 72)
(split-char 65)
@result{} (ascii 65)
-@end example
-
-Unibyte non-@sc{ascii} characters are considered as part of
-the @code{ascii} character set:
-
-@example
-(split-char 192)
- @result{} (ascii 192)
+(split-char 128)
+ @result{} (eight-bit-control 128)
@end example
@end defun
-@defun make-char charset &rest byte-values
-This function returns the character in character set @var{charset}
-identified by @var{byte-values}. This is roughly the inverse of
-@code{split-char}. Normally, you should specify either one or two
-@var{byte-values}, according to the dimension of @var{charset}. For
-example,
+@defun make-char charset &optional code1 code2
+This function returns the character in character set @var{charset} whose
+position codes are @var{code1} and @var{code2}. This is roughly the
+inverse of @code{split-char}. Normally, you should specify either one
+or both of @var{code1} and @var{code2} according to the dimension of
+@var{charset}. For example,
@example
(make-char 'latin-iso8859-1 72)
@result{} 2176
(char-valid-p 2176)
@result{} nil
+(char-valid-p 2176 t)
+ @result{} t
(split-char 2176)
@result{} (latin-iso8859-1 0)
@end example
+The character sets @sc{ascii}, @sc{eight-bit-control}, and
+@sc{eight-bit-graphic} don't have corresponding generic characters. If
+@var{charset} is one of them and you don't supply @var{code1},
+@code{make-char} returns the character code corresponding to the
+smallest code in @var{charset}.
+
@node Scanning Charsets
@section Scanning for Character Sets
is non-@code{nil}, then each character in the region is translated
through this table, and the value returned describes the translated
characters instead of the characters actually in the buffer.
-
-When a buffer contains non-@sc{ascii} characters, codes 128 through 255,
-they are assigned the character set @code{unknown}. @xref{Explicit
-Encoding}.
@end defun
@defun find-charset-string string &optional translation
@defun make-translation-table &rest translations
This function returns a translation table based on the argument
-@var{translations}. Each element of
-@var{translations} should be a list of the form @code{(@var{from}
-. @var{to})}; this says to translate the character @var{from} into
-@var{to}.
+@var{translations}. Each element of @var{translations} should be a
+list of elements of the form @code{(@var{from} . @var{to})}; this says
+to translate the character @var{from} into @var{to}.
+
+The arguments and the forms in each argument are processed in order,
+and if a previous form already translates @var{to} to some other
+character, say @var{to-alt}, @var{from} is also translated to
+@var{to-alt}.
You can also map one whole character set into another character set with
the same dimension. To do this, you specify a generic character (which
@defvar buffer-file-coding-system
This variable records the coding system that was used for visiting the
current buffer. It is used for saving the buffer, and for writing part
-of the buffer with @code{write-region}. When those operations ask the
-user to specify a different coding system,
-@code{buffer-file-coding-system} is updated to the coding system
-specified.
-
-However, @code{buffer-file-coding-system} does not affect sending text
+of the buffer with @code{write-region}. If the text to be written
+cannot be safely encoded using the coding system specified by this
+variable, these operations select an alternative encoding by calling
+the function @code{select-safe-coding-system} (@pxref{User-Chosen
+Coding Systems}). If selecting a different encoding requires to ask
+the user to specify a coding system, @code{buffer-file-coding-system}
+is updated to the newly selected coding system.
+
+@code{buffer-file-coding-system} does @emph{not} affect sending text
to a subprocess.
@end defvar
@defvar save-buffer-coding-system
-This variable specifies the coding system for saving the buffer---but it
-is not used for @code{write-region}.
+This variable specifies the coding system for saving the buffer (by
+overriding @code{buffer-file-coding-system}). Note that it is not used
+for @code{write-region}.
When a command to save the buffer starts out to use
-@code{save-buffer-coding-system}, and that coding system cannot handle
+@code{buffer-file-coding-system} (or @code{save-buffer-coding-system}),
+and that coding system cannot handle
the actual text in the buffer, the command asks the user to choose
-another coding system. After that happens, the command also updates
-@code{save-buffer-coding-system} to represent the coding system that the
-user specified.
+another coding system (by calling @code{select-safe-coding-system}).
+After that happens, the command also updates
+@code{buffer-file-coding-system} to represent the coding system that
+the user specified.
@end defvar
@defvar last-coding-system-used
@defun coding-system-list &optional base-only
This function returns a list of all coding system names (symbols). If
@var{base-only} is non-@code{nil}, the value includes only the
-base coding systems. Otherwise, it includes variant coding systems as well.
+base coding systems. Otherwise, it includes alias and variant coding
+systems as well.
@end defun
@defun coding-system-p object
@node User-Chosen Coding Systems
@subsection User-Chosen Coding Systems
-@defun select-safe-coding-system from to &optional preferred-coding-system
-This function selects a coding system for encoding the text between
-@var{from} and @var{to}, asking the user to choose if necessary.
-
-The optional argument @var{preferred-coding-system} specifies a coding
-system to try first. If that one can handle the text in the specified
-region, then it is used. If this argument is omitted, the current
-buffer's value of @code{buffer-file-coding-system} is tried first.
-
-If the region contains some multibyte characters that the preferred
-coding system cannot encode, this function asks the user to choose from
-a list of coding systems which can encode the text, and returns the
-user's choice.
-
-One other kludgy feature: if @var{from} is a string, the string is the
-target text, and @var{to} is ignored.
+@cindex select safe coding system
+@defun select-safe-coding-system from to &optional default-coding-system accept-default-p
+This function selects a coding system for encoding specified text,
+asking the user to choose if necessary. Normally the specified text
+is the text in the current buffer between @var{from} and @var{to},
+defaulting to the whole buffer if they are @code{nil}. If @var{from}
+is a string, the string specifies the text to encode, and @var{to} is
+ignored.
+
+If @var{default-coding-system} is non-@code{nil}, that is the first
+coding system to try; if that can handle the text,
+@code{select-safe-coding-system} returns that coding system. It can
+also be a list of coding systems; then the function tries each of them
+one by one. After trying all of them, it next tries the user's most
+preferred coding system (@pxref{Recognize Coding,
+prefer-coding-system, the description of @code{prefer-coding-system},
+emacs, GNU Emacs Manual}), and after that the current buffer's value
+of @code{buffer-file-coding-system} (if it is not @code{undecided}).
+
+If one of those coding systems can safely encode all the specified
+text, @code{select-safe-coding-system} chooses it and returns it.
+Otherwise, it asks the user to choose from a list of coding systems
+which can encode all the text, and returns the user's choice.
+
+The optional argument @var{accept-default-p}, if non-@code{nil},
+should be a function to determine whether the coding system selected
+without user interaction is acceptable. If this function returns
+@code{nil}, the silently selected coding system is rejected, and the
+user is asked to select a coding system from a list of possible
+candidates.
+
+@vindex select-safe-coding-system-accept-default-p
+If the variable @code{select-safe-coding-system-accept-default-p} is
+non-@code{nil}, its value overrides the value of
+@var{accept-default-p}.
@end defun
Here are two functions you can use to let the user specify a coding
@code{coding-system-for-read} and @code{coding-system-for-write}
(@pxref{Specifying Coding Systems}).
+@defvar auto-coding-regexp-alist
+This variable is an alist of text patterns and corresponding coding
+systems. Each element has the form @code{(@var{regexp}
+. @var{coding-system})}; a file whose first few kilobytes match
+@var{regexp} is decoded with @var{coding-system} when its contents are
+read into a buffer. The settings in this alist take priority over
+@code{coding:} tags in the files and the contents of
+@code{file-coding-system-alist} (see below). The default value is set
+so that Emacs automatically recognizes mail files in Babyl format and
+reads them with no code conversions.
+@end defvar
+
@defvar file-coding-system-alist
This variable is an alist that specifies the coding systems to use for
reading and writing particular files. Each element has the form
@defvar locale-coding-system
@tindex locale-coding-system
+@cindex keyboard input decoding on X
This variable specifies the coding system to use for decoding system
-error messages, for encoding the format argument to
-@code{format-time-string}, and for decoding the return value of
-@code{format-time-string}.
+error messages and---on X Window system only---keyboard input, for
+encoding the format argument to @code{format-time-string}, and for
+decoding the return value of @code{format-time-string}.
@end defvar
@defvar system-messages-locale