@c -*-texinfo-*-
@c This is part of the GNU Emacs Lisp Reference Manual.
-@c Copyright (C) 1998 Free Software Foundation, Inc.
+@c Copyright (C) 1998, 1999 Free Software Foundation, Inc.
@c See the file elisp.texi for copying conditions.
@setfilename ../info/characters
@node Non-ASCII Characters, Searching and Matching, Text, Top
-@chapter Non-ASCII Characters
+@chapter Non-@sc{ascii} Characters
@cindex multibyte characters
-@cindex non-ASCII characters
+@cindex non-@sc{ascii} characters
This chapter covers the special issues relating to non-@sc{ascii}
characters and how they are stored in strings and buffers.
@menu
-* Text Representations::
-* Converting Representations::
-* Selecting a Representation::
-* Character Codes::
-* Character Sets::
-* Chars and Bytes::
-* Splitting Characters::
-* Scanning Charsets::
-* Translation of Characters::
-* Coding Systems::
-* Input Methods::
+* Text Representations:: Unibyte and multibyte representations
+* Converting Representations:: Converting unibyte to multibyte and vice versa.
+* Selecting a Representation:: Treating a byte sequence as unibyte or multi.
+* Character Codes:: How unibyte and multibyte relate to
+ codes of individual characters.
+* Character Sets:: The space of possible characters codes
+ is divided into various character sets.
+* Chars and Bytes:: More information about multibyte encodings.
+* Splitting Characters:: Converting a character to its byte sequence.
+* Scanning Charsets:: Which character sets are used in a buffer?
+* Translation of Characters:: Translation tables are used for conversion.
+* Coding Systems:: Coding systems are conversions for saving files.
+* Input Methods:: Input methods allow users to enter various
+ non-ASCII characters without special keyboards.
+* Locales:: Interacting with the POSIX locale.
@end menu
@node Text Representations
character are always in the range 160 through 255 (octal 0240 through
0377); these values are @dfn{trailing codes}.
- Some sequences of bytes do not form meaningful multibyte characters:
-for example, a single isolated byte in the range 128 through 255 is
-never meaningful. Such byte sequences are not entirely valid, and never
-appear in proper multibyte text (since that consists of a sequence of
-@emph{characters}); but they can appear as part of ``raw bytes''
-(@pxref{Explicit Encoding}).
+ Some sequences of bytes are not valid in multibyte text: for example,
+a single isolated byte in the range 128 through 159 is not allowed. But
+character codes 128 through 159 can appear in multibyte text,
+represented as two-byte sequences. All the character codes 128 through
+255 are possible (though slightly abnormal) in multibyte text; they
+appear in multibyte buffers and strings when you do explicit encoding
+and decoding (@pxref{Explicit Encoding}).
In a buffer, the buffer-local value of the variable
@code{enable-multibyte-characters} specifies the representation used.
when the string is constructed.
@defvar enable-multibyte-characters
-@tindex enable-multibyte-characters
This variable specifies the current buffer's text representation.
If it is non-@code{nil}, the buffer contains multibyte text; otherwise,
it contains unibyte text.
@end defvar
@defvar default-enable-multibyte-characters
-@tindex default-enable-multibyte-characters
This variable's value is entirely equivalent to @code{(default-value
'enable-multibyte-characters)}, and setting this variable changes that
default value. Setting the local binding of
@end defun
@defun multibyte-string-p string
-@tindex multibyte-string-p
Return @code{t} if @var{string} is a multibyte string.
@end defun
user that cannot be overridden automatically.
Converting unibyte text to multibyte text leaves @sc{ascii} characters
-unchanged, and likewise 128 through 159. It converts the non-@sc{ascii}
-codes 160 through 255 by adding the value @code{nonascii-insert-offset}
-to each character code. By setting this variable, you specify which
-character set the unibyte characters correspond to (@pxref{Character
-Sets}). For example, if @code{nonascii-insert-offset} is 2048, which is
-@code{(- (make-char 'latin-iso8859-1) 128)}, then the unibyte
-non-@sc{ascii} characters correspond to Latin 1. If it is 2688, which
-is @code{(- (make-char 'greek-iso8859-7) 128)}, then they correspond to
-Greek letters.
+unchanged, and likewise character codes 128 through 159. It converts
+the non-@sc{ascii} codes 160 through 255 by adding the value
+@code{nonascii-insert-offset} to each character code. By setting this
+variable, you specify which character set the unibyte characters
+correspond to (@pxref{Character Sets}). For example, if
+@code{nonascii-insert-offset} is 2048, which is @code{(- (make-char
+'latin-iso8859-1) 128)}, then the unibyte non-@sc{ascii} characters
+correspond to Latin 1. If it is 2688, which is @code{(- (make-char
+'greek-iso8859-7) 128)}, then they correspond to Greek letters.
Converting multibyte text to unibyte is simpler: it discards all but
the low 8 bits of each character code. If @code{nonascii-insert-offset}
text.
@defvar nonascii-insert-offset
-@tindex nonascii-insert-offset
This variable specifies the amount to add to a non-@sc{ascii} character
when converting unibyte text to multibyte. It also applies when
@code{self-insert-command} inserts a character in the unibyte
-non-@sc{ascii} range, 128 through 255. However, the function
-@code{insert-char} does not perform this conversion.
+non-@sc{ascii} range, 128 through 255. However, the functions
+@code{insert} and @code{insert-char} do not perform this conversion.
The right value to use to select character set @var{cs} is @code{(-
(make-char @var{cs}) 128)}. If the value of
@end defvar
@defvar nonascii-translation-table
-@tindex nonascii-translation-table
This variable provides a more general alternative to
@code{nonascii-insert-offset}. You can use it to specify independently
how to translate each code in the range of 128 through 255 into a
-multibyte character. The value should be a vector, or @code{nil}.
+multibyte character. The value should be a char-table, or @code{nil}.
If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
@end defvar
@defun string-make-unibyte string
-@tindex string-make-unibyte
This function converts the text of @var{string} to unibyte
representation, if it isn't already, and returns the result. If
-@var{string} is a unibyte string, it is returned unchanged.
+@var{string} is a unibyte string, it is returned unchanged. Multibyte
+character codes are converted to unibyte according to
+@code{nonascii-translation-table} or, if that is @code{nil}, using
+@code{nonascii-insert-offset}. If the lookup in the translation table
+fails, this function takes just the low 8 bits of each character.
@end defun
@defun string-make-multibyte string
-@tindex string-make-multibyte
This function converts the text of @var{string} to multibyte
representation, if it isn't already, and returns the result. If
@var{string} is a multibyte string, it is returned unchanged.
+The function @code{unibyte-char-to-multibyte} is used to convert
+each unibyte character to a multibyte character.
@end defun
@node Selecting a Representation
multibyte when it was unibyte, or vice versa.
@defun set-buffer-multibyte multibyte
-@tindex set-buffer-multibyte
Set the representation type of the current buffer. If @var{multibyte}
is non-@code{nil}, the buffer becomes multibyte. If @var{multibyte}
is @code{nil}, the buffer becomes unibyte.
sequence of bytes. As a consequence, it can change the contents viewed
as characters; a sequence of two bytes which is treated as one character
in multibyte representation will count as two characters in unibyte
-representation.
+representation. Character codes 128 through 159 are an exception. They
+are represented by one byte in a unibyte buffer, but when the buffer is
+set to multibyte, they are converted to two-byte sequences, and vice
+versa.
This function sets @code{enable-multibyte-characters} to record which
representation is in use. It also adjusts various data in the buffer
@end defun
@defun string-as-unibyte string
-@tindex string-as-unibyte
This function returns a string with the same bytes as @var{string} but
treating each byte as a character. This means that the value may have
more characters than @var{string} has.
If @var{string} is already a unibyte string, then the value is
-@var{string} itself.
+@var{string} itself. Otherwise it is a newly created string, with no
+text properties. If @var{string} is multibyte, any characters it
+contains of charset @var{eight-bit-control} or @var{eight-bit-graphic}
+are converted to the corresponding single byte.
@end defun
@defun string-as-multibyte string
-@tindex string-as-multibyte
This function returns a string with the same bytes as @var{string} but
treating each multibyte sequence as one character. This means that the
value may have fewer characters than @var{string} has.
If @var{string} is already a multibyte string, then the value is
-@var{string} itself.
+@var{string} itself. Otherwise it is a newly created string, with no
+text properties. If @var{string} is unibyte and contains any individual
+8-bit bytes (i.e.@: not part of a multibyte form), they are converted to
+the corresponding multibyte character of charset @var{eight-bit-control}
+or @var{eight-bit-graphic}.
@end defun
@node Character Codes
codes. The valid character codes for unibyte representation range from
0 to 255---the values that can fit in one byte. The valid character
codes for multibyte representation range from 0 to 524287, but not all
-values in that range are valid. In particular, the values 128 through
-255 are not legitimate in multibyte text (though they can occur in ``raw
-bytes''; @pxref{Explicit Encoding}). Only the @sc{ascii} codes 0
-through 127 are fully legitimate in both representations.
+values in that range are valid. The values 128 through 255 are not
+entirely proper in multibyte text, but they can occur if you do explicit
+encoding and decoding (@pxref{Explicit Encoding}). Some other character
+codes cannot occur at all in multibyte text. Only the @sc{ascii} codes
+0 through 127 are completely legitimate in both representations.
-@defun char-valid-p charcode
+@defun char-valid-p charcode &optional genericp
This returns @code{t} if @var{charcode} is valid for either one of the two
text representations.
(char-valid-p 2248)
@result{} t
@end example
+
+If the optional argument @var{genericp} is non-nil, this function
+returns @code{t} if @var{charcode} is a generic character
+(@pxref{Splitting Characters}).
@end defun
@node Character Sets
characters, generally known as Big 5, is divided into two Emacs
character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
+ @sc{ascii} characters are in character set @code{ascii}. The
+non-@sc{ascii} characters 128 through 159 are in character set
+@code{eight-bit-control}, and codes 160 through 255 are in character set
+@code{eight-bit-graphic}.
+
@defun charsetp object
-@tindex charsetp
Returns @code{t} if @var{object} is a symbol that names a character set,
@code{nil} otherwise.
@end defun
@defun charset-list
-@tindex charset-list
This function returns a list of all defined character set names.
@end defun
@defun char-charset character
-@tindex char-charset
This function returns the name of the character set that @var{character}
belongs to.
@end defun
This function returns the charset property list of the character set
@var{charset}. Although @var{charset} is a symbol, this is not the same
as the property list of that symbol. Charset properties are used for
-special purposes within Emacs; for example, @code{x-charset-registry}
-helps determine which fonts to use (@pxref{Font Selection}).
+special purposes within Emacs; for example,
+@code{preferred-coding-system} helps determine which coding system to
+use to encode characters in a charset.
@end defun
@node Chars and Bytes
In multibyte representation, each character occupies one or more
bytes. Each character set has an @dfn{introduction sequence}, which is
normally one or two bytes long. (Exception: the @sc{ascii} character
-set has a zero-length introduction sequence.) The introduction sequence
-is the beginning of the byte sequence for any character in the character
-set. The rest of the character's bytes distinguish it from the other
-characters in the same character set. Depending on the character set,
-there are either one or two distinguishing bytes; the number of such
-bytes is called the @dfn{dimension} of the character set.
+set and the @sc{eight-bit-graphic} character set have a zero-length
+introduction sequence.) The introduction sequence is the beginning of
+the byte sequence for any character in the character set. The rest of
+the character's bytes distinguish it from the other characters in the
+same character set. Depending on the character set, there are either
+one or two distinguishing bytes; the number of such bytes is called the
+@dfn{dimension} of the character set.
@defun charset-dimension charset
-@tindex charset-dimension
This function returns the dimension of @var{charset}; at present, the
dimension is always 1 or 2.
@end defun
because Emacs translates automatically when necessary.
@defun split-char character
-@tindex split-char
Return a list containing the name of the character set of
@var{character}, followed by one or two byte values (integers) which
identify @var{character} within that character set. The number of byte
@result{} (latin-iso8859-1 72)
(split-char 65)
@result{} (ascii 65)
-@end example
-
-Unibyte non-@sc{ascii} characters are considered as part of
-the @code{ascii} character set:
-
-@example
-(split-char 192)
- @result{} (ascii 192)
+(split-char 128)
+ @result{} (eight-bit-control 128)
@end example
@end defun
-@defun make-char charset &rest byte-values
-@tindex make-char
-This function returns the character in character set @var{charset}
-identified by @var{byte-values}. This is roughly the inverse of
-@code{split-char}. Normally, you should specify either one or two
-@var{byte-values}, according to the dimension of @var{charset}. For
-example,
+@defun make-char charset &optional code1 code2
+This function returns the character in character set @var{charset} whose
+position codes are @var{code1} and @var{code2}. This is roughly the
+inverse of @code{split-char}. Normally, you should specify either one
+or both of @var{code1} and @var{code2} according to the dimension of
+@var{charset}. For example,
@example
(make-char 'latin-iso8859-1 72)
@result{} 2176
(char-valid-p 2176)
@result{} nil
+(char-valid-p 2176 t)
+ @result{} t
(split-char 2176)
@result{} (latin-iso8859-1 0)
@end example
+The character sets @sc{ascii}, @sc{eight-bit-control}, and
+@sc{eight-bit-graphic} don't have corresponding generic characters. If
+@var{charset} is one of them and you don't supply @var{code1},
+@code{make-char} returns the character code corresponding to the
+smallest code in @var{charset}.
+
@node Scanning Charsets
@section Scanning for Character Sets
of the text in question.
@defun find-charset-region beg end &optional translation
-@tindex find-charset-region
This function returns a list of the character sets that appear in the
current buffer between positions @var{beg} and @var{end}.
is non-@code{nil}, then each character in the region is translated
through this table, and the value returned describes the translated
characters instead of the characters actually in the buffer.
-
-In two peculiar cases, the value includes the symbol @code{unknown}:
-
-@itemize @bullet
-@item
-When a unibyte buffer contains non-@sc{ascii} characters.
-
-@item
-When a multibyte buffer contains invalid byte-sequences (raw bytes).
-@xref{Explicit Encoding}.
-@end itemize
@end defun
@defun find-charset-string string &optional translation
-@tindex find-charset-string
This function returns a list of the character sets that appear in the
string @var{string}. It is just like @code{find-charset-region}, except
that it applies to the contents of @var{string} instead of part of the
@defun make-translation-table &rest translations
This function returns a translation table based on the argument
-@var{translations}. Each element of
-@var{translations} should be a list of the form @code{(@var{from}
-. @var{to})}; this says to translate the character @var{from} into
-@var{to}.
+@var{translations}. Each element of @var{translations} should be a
+list of elements of the form @code{(@var{from} . @var{to})}; this says
+to translate the character @var{from} into @var{to}.
+
+The arguments and the forms in each argument are processed in order,
+and if a previous form already translates @var{to} to some other
+character, say @var{to-alt}, @var{from} is also translated to
+@var{to-alt}.
You can also map one whole character set into another character set with
the same dimension. To do this, you specify a generic character (which
documented here.
@menu
-* Coding System Basics::
-* Encoding and I/O::
-* Lisp and Coding Systems::
-* User-Chosen Coding Systems::
-* Default Coding Systems::
-* Specifying Coding Systems::
-* Explicit Encoding::
-* Terminal I/O Encoding::
-* MS-DOS File Types::
+* Coding System Basics:: Basic concepts.
+* Encoding and I/O:: How file I/O functions handle coding systems.
+* Lisp and Coding Systems:: Functions to operate on coding system names.
+* User-Chosen Coding Systems:: Asking the user to choose a coding system.
+* Default Coding Systems:: Controlling the default choices.
+* Specifying Coding Systems:: Requesting a particular coding system
+ for a single file operation.
+* Explicit Encoding:: Encoding or decoding text without doing I/O.
+* Terminal I/O Encoding:: Use of encoding for terminal I/O.
+* MS-DOS File Types:: How DOS "text" and "binary" files
+ relate to coding systems.
@end menu
@node Coding System Basics
that the result is multibyte data.
@defun coding-system-get coding-system property
-@tindex coding-system-get
This function returns the specified property of the coding system
@var{coding-system}. Most coding system properties exist for internal
purposes, but one that you might find useful is @code{mime-charset}.
you will want to find out afterwards which coding system was chosen.
@defvar buffer-file-coding-system
-@tindex buffer-file-coding-system
This variable records the coding system that was used for visiting the
current buffer. It is used for saving the buffer, and for writing part
-of the buffer with @code{write-region}. When those operations ask the
-user to specify a different coding system,
-@code{buffer-file-coding-system} is updated to the coding system
-specified.
-
-However, @code{buffer-file-coding-system} does not affect sending text
+of the buffer with @code{write-region}. If the text to be written
+cannot be safely encoded using the coding system specified by this
+variable, these operations select an alternative encoding by calling
+the function @code{select-safe-coding-system} (@pxref{User-Chosen
+Coding Systems}). If selecting a different encoding requires to ask
+the user to specify a coding system, @code{buffer-file-coding-system}
+is updated to the newly selected coding system.
+
+@code{buffer-file-coding-system} does @emph{not} affect sending text
to a subprocess.
@end defvar
@defvar save-buffer-coding-system
-@tindex save-buffer-coding-system
-This variable specifies the coding system for saving the buffer---but it
-is not used for @code{write-region}.
+This variable specifies the coding system for saving the buffer (by
+overriding @code{buffer-file-coding-system}). Note that it is not used
+for @code{write-region}.
When a command to save the buffer starts out to use
-@code{save-buffer-coding-system}, and that coding system cannot handle
+@code{buffer-file-coding-system} (or @code{save-buffer-coding-system}),
+and that coding system cannot handle
the actual text in the buffer, the command asks the user to choose
-another coding system. After that happens, the command also updates
-@code{save-buffer-coding-system} to represent the coding system that the
-user specified.
+another coding system (by calling @code{select-safe-coding-system}).
+After that happens, the command also updates
+@code{buffer-file-coding-system} to represent the coding system that
+the user specified.
@end defvar
@defvar last-coding-system-used
-@tindex last-coding-system-used
I/O operations for files and subprocesses set this variable to the
coding system name that was used. The explicit encoding and decoding
functions (@pxref{Explicit Encoding}) set it too.
Here are the Lisp facilities for working with coding systems:
@defun coding-system-list &optional base-only
-@tindex coding-system-list
This function returns a list of all coding system names (symbols). If
@var{base-only} is non-@code{nil}, the value includes only the
-base coding systems. Otherwise, it includes variant coding systems as well.
+base coding systems. Otherwise, it includes alias and variant coding
+systems as well.
@end defun
@defun coding-system-p object
-@tindex coding-system-p
This function returns @code{t} if @var{object} is a coding system
name.
@end defun
@defun check-coding-system coding-system
-@tindex check-coding-system
This function checks the validity of @var{coding-system}.
If that is valid, it returns @var{coding-system}.
Otherwise it signals an error with condition @code{coding-system-error}.
@end defun
@defun coding-system-change-eol-conversion coding-system eol-type
-@tindex coding-system-change-eol-conversion
This function returns a coding system which is like @var{coding-system}
except for its eol conversion, which is specified by @code{eol-type}.
@var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
@end defun
@defun coding-system-change-text-conversion eol-coding text-coding
-@tindex coding-system-change-text-conversion
This function returns a coding system which uses the end-of-line
conversion of @var{eol-coding}, and the text conversion of
@var{text-coding}. If @var{text-coding} is @code{nil}, it returns
@end defun
@defun find-coding-systems-region from to
-@tindex find-coding-systems-region
This function returns a list of coding systems that could be used to
encode a text between @var{from} and @var{to}. All coding systems in
the list can safely encode any multibyte characters in that portion of
@end defun
@defun find-coding-systems-string string
-@tindex find-coding-systems-string
This function returns a list of coding systems that could be used to
encode the text of @var{string}. All coding systems in the list can
safely encode any multibyte characters in @var{string}. If the text
@end defun
@defun find-coding-systems-for-charsets charsets
-@tindex find-coding-systems-for-charsets
This function returns a list of coding systems that could be used to
encode all the character sets in the list @var{charsets}.
@end defun
@defun detect-coding-region start end &optional highest
-@tindex detect-coding-region
This function chooses a plausible coding system for decoding the text
-from @var{start} to @var{end}. This text should be ``raw bytes''
+from @var{start} to @var{end}. This text should be a byte sequence
(@pxref{Explicit Encoding}).
Normally this function returns a list of coding systems that could
@end defun
@defun detect-coding-string string highest
-@tindex detect-coding-string
This function is like @code{detect-coding-region} except that it
operates on the contents of @var{string} instead of bytes in the buffer.
@end defun
@node User-Chosen Coding Systems
@subsection User-Chosen Coding Systems
-@tindex select-safe-coding-system
-@defun select-safe-coding-system from to &optional preferred-coding-system
-This function selects a coding system for encoding the text between
-@var{from} and @var{to}, asking the user to choose if necessary.
-
-The optional argument @var{preferred-coding-system} specifies a coding
-system to try first. If that one can handle the text in the specified
-region, then it is used. If this argument is omitted, the current
-buffer's value of @code{buffer-file-coding-system} is tried first.
-
-If the region contains some multibyte characters that the preferred
-coding system cannot encode, this function asks the user to choose from
-a list of coding systems which can encode the text, and returns the
-user's choice.
-
-One other kludgy feature: if @var{from} is a string, the string is the
-target text, and @var{to} is ignored.
+@cindex select safe coding system
+@defun select-safe-coding-system from to &optional default-coding-system accept-default-p
+This function selects a coding system for encoding specified text,
+asking the user to choose if necessary. Normally the specified text
+is the text in the current buffer between @var{from} and @var{to},
+defaulting to the whole buffer if they are @code{nil}. If @var{from}
+is a string, the string specifies the text to encode, and @var{to} is
+ignored.
+
+If @var{default-coding-system} is non-@code{nil}, that is the first
+coding system to try; if that can handle the text,
+@code{select-safe-coding-system} returns that coding system. It can
+also be a list of coding systems; then the function tries each of them
+one by one. After trying all of them, it next tries the user's most
+preferred coding system (@pxref{Recognize Coding,
+prefer-coding-system, the description of @code{prefer-coding-system},
+emacs, GNU Emacs Manual}), and after that the current buffer's value
+of @code{buffer-file-coding-system} (if it is not @code{undecided}).
+
+If one of those coding systems can safely encode all the specified
+text, @code{select-safe-coding-system} chooses it and returns it.
+Otherwise, it asks the user to choose from a list of coding systems
+which can encode all the text, and returns the user's choice.
+
+The optional argument @var{accept-default-p}, if non-@code{nil},
+should be a function to determine whether the coding system selected
+without user interaction is acceptable. If this function returns
+@code{nil}, the silently selected coding system is rejected, and the
+user is asked to select a coding system from a list of possible
+candidates.
+
+@vindex select-safe-coding-system-accept-default-p
+If the variable @code{select-safe-coding-system-accept-default-p} is
+non-@code{nil}, its value overrides the value of
+@var{accept-default-p}.
@end defun
Here are two functions you can use to let the user specify a coding
system, with completion. @xref{Completion}.
@defun read-coding-system prompt &optional default
-@tindex read-coding-system
This function reads a coding system using the minibuffer, prompting with
string @var{prompt}, and returns the coding system name as a symbol. If
the user enters null input, @var{default} specifies which coding system
@end defun
@defun read-non-nil-coding-system prompt
-@tindex read-non-nil-coding-system
This function reads a coding system using the minibuffer, prompting with
string @var{prompt}, and returns the coding system name as a symbol. If
the user tries to enter null input, it asks the user to try again.
@code{coding-system-for-read} and @code{coding-system-for-write}
(@pxref{Specifying Coding Systems}).
+@defvar auto-coding-regexp-alist
+This variable is an alist of text patterns and corresponding coding
+systems. Each element has the form @code{(@var{regexp}
+. @var{coding-system})}; a file whose first few kilobytes match
+@var{regexp} is decoded with @var{coding-system} when its contents are
+read into a buffer. The settings in this alist take priority over
+@code{coding:} tags in the files and the contents of
+@code{file-coding-system-alist} (see below). The default value is set
+so that Emacs automatically recognizes mail files in Babyl format and
+reads them with no code conversions.
+@end defvar
+
@defvar file-coding-system-alist
-@tindex file-coding-system-alist
This variable is an alist that specifies the coding systems to use for
reading and writing particular files. Each element has the form
@code{(@var{pattern} . @var{coding})}, where @var{pattern} is a regular
@end defvar
@defvar process-coding-system-alist
-@tindex process-coding-system-alist
This variable is an alist specifying which coding systems to use for a
subprocess, depending on which program is running in the subprocess. It
works like @code{file-coding-system-alist}, except that @var{pattern} is
rather than @code{undecided} or @code{latin-1}.
@defvar network-coding-system-alist
-@tindex network-coding-system-alist
This variable is an alist that specifies the coding system to use for
network streams. It works much like @code{file-coding-system-alist},
with the difference that the @var{pattern} in an element may be either a
@end defvar
@defvar default-process-coding-system
-@tindex default-process-coding-system
This variable specifies the coding systems to use for subprocess (and
network stream) input and output, when nothing else specifies what to
do.
@end defvar
@defun find-operation-coding-system operation &rest arguments
-@tindex find-operation-coding-system
This function returns the coding system to use (by default) for
performing @var{operation} with @var{arguments}. The value has this
form:
@code{coding-system-for-write}.
@defvar coding-system-for-read
-@tindex coding-system-for-read
If this variable is non-@code{nil}, it specifies the coding system to
use for reading a file, or for input from a synchronous subprocess.
@end defvar
@defvar coding-system-for-write
-@tindex coding-system-for-write
This works much like @code{coding-system-for-read}, except that it
applies to output rather than input. It affects writing to files,
as well as sending output to subprocesses and net connections.
@end defvar
@defvar inhibit-eol-conversion
-@tindex inhibit-eol-conversion
When this variable is non-@code{nil}, no end-of-line conversion is done,
no matter which coding system is specified. This applies to all the
Emacs I/O and subprocess primitives, and to the explicit encoding and
You can also explicitly encode and decode text using the functions
in this section.
-@cindex raw bytes
The result of encoding, and the input to decoding, are not ordinary
-text. They are ``raw bytes''---bytes that represent text in the same
-way that an external file would. When a buffer contains raw bytes, it
-is most natural to mark that buffer as using unibyte representation,
-using @code{set-buffer-multibyte} (@pxref{Selecting a Representation}),
-but this is not required. If the buffer's contents are only temporarily
-raw, leave the buffer multibyte, which will be correct after you decode
-them.
-
- The usual way to get raw bytes in a buffer, for explicit decoding, is
-to read them from a file with @code{insert-file-contents-literally}
-(@pxref{Reading from Files}) or specify a non-@code{nil} @var{rawfile}
-argument when visiting a file with @code{find-file-noselect}.
-
- The usual way to use the raw bytes that result from explicitly
-encoding text is to copy them to a file or process---for example, to
-write them with @code{write-region} (@pxref{Writing to Files}), and
-suppress encoding for that @code{write-region} call by binding
-@code{coding-system-for-write} to @code{no-conversion}.
-
- Raw bytes typically contain stray individual bytes with values in the
-range 128 through 255, that are legitimate only as part of multibyte
-sequences. Even if the buffer is multibyte, Emacs treats each such
-individual byte as a character and uses the byte value as its character
-code. In this way, character codes 128 through 255 can be found in a
-multibyte buffer, even though they are not legitimate multibyte
-character codes.
-
- Raw bytes sometimes contain overlong byte-sequences that look like a
-proper multibyte character plus extra superfluous trailing codes. For
-most purposes, Emacs treats such a sequence in a buffer or string as a
-single character, and if you look at its character code, you get the
-value that corresponds to the multibyte character
-sequence---disregarding the extra trailing codes. This is not quite
-clean, but raw bytes are used only in limited ways, so as a practical
-matter it is not worth the trouble to treat this case differently.
-
- When a multibyte buffer contains illegitimate byte sequences,
-sometimes insertion or deletion can cause them to coalesce into a
-legitimate multibyte character. For example, suppose the buffer
-contains the sequence 129 68 192, 68 being the character @samp{D}. If
-you delete the @samp{D}, the bytes 129 and 192 become adjacent, and thus
-become one multibyte character (Latin-1 A with grave accent). Point
-moves to one side or the other of the character, since it cannot be
-within a character. Don't be alarmed by this.
-
- Some really peculiar situations prevent proper coalescence. For
-example, if you narrow the buffer so that the accessible portion begins
-just before the @samp{D}, then delete the @samp{D}, the two surrounding
-bytes cannot coalesce because one of them is outside the accessible
-portion of the buffer. In this case, the deletion cannot be done, so
-@code{delete-region} signals an error.
+text. They logically consist of a series of byte values; that is, a
+series of characters whose codes are in the range 0 through 255. In a
+multibyte buffer or string, character codes 128 through 159 are
+represented by multibyte sequences, but this is invisible to Lisp
+programs.
+
+ The usual way to read a file into a buffer as a sequence of bytes, so
+you can decode the contents explicitly, is with
+@code{insert-file-contents-literally} (@pxref{Reading from Files});
+alternatively, specify a non-@code{nil} @var{rawfile} argument when
+visiting a file with @code{find-file-noselect}. These methods result in
+a unibyte buffer.
+
+ The usual way to use the byte sequence that results from explicitly
+encoding text is to copy it to a file or process---for example, to write
+it with @code{write-region} (@pxref{Writing to Files}), and suppress
+encoding by binding @code{coding-system-for-write} to
+@code{no-conversion}.
Here are the functions to perform explicit encoding or decoding. The
-decoding functions produce ``raw bytes''; the encoding functions are
-meant to operate on ``raw bytes''. All of these functions discard text
-properties.
+decoding functions produce sequences of bytes; the encoding functions
+are meant to operate on sequences of bytes. All of these functions
+discard text properties.
@defun encode-coding-region start end coding-system
-@tindex encode-coding-region
This function encodes the text from @var{start} to @var{end} according
to coding system @var{coding-system}. The encoded text replaces the
-original text in the buffer. The result of encoding is ``raw bytes,''
-but the buffer remains multibyte if it was multibyte before.
+original text in the buffer. The result of encoding is logically a
+sequence of bytes, but the buffer remains multibyte if it was multibyte
+before.
@end defun
@defun encode-coding-string string coding-system
-@tindex encode-coding-string
This function encodes the text in @var{string} according to coding
system @var{coding-system}. It returns a new string containing the
-encoded text. The result of encoding is a unibyte string of ``raw bytes.''
+encoded text. The result of encoding is a unibyte string.
@end defun
@defun decode-coding-region start end coding-system
-@tindex decode-coding-region
This function decodes the text from @var{start} to @var{end} according
to coding system @var{coding-system}. The decoded text replaces the
original text in the buffer. To make explicit decoding useful, the text
-before decoding ought to be ``raw bytes.''
+before decoding ought to be a sequence of byte values, but both
+multibyte and unibyte buffers are acceptable.
@end defun
@defun decode-coding-string string coding-system
-@tindex decode-coding-string
This function decodes the text in @var{string} according to coding
system @var{coding-system}. It returns a new string containing the
decoded text. To make explicit decoding useful, the contents of
-@var{string} ought to be ``raw bytes.''
+@var{string} ought to be a sequence of byte values, but a multibyte
+string is acceptable.
@end defun
@node Terminal I/O Encoding
terminal.
@defun keyboard-coding-system
-@tindex keyboard-coding-system
This function returns the coding system that is in use for decoding
keyboard input---or @code{nil} if no coding system is to be used.
@end defun
@defun set-keyboard-coding-system coding-system
-@tindex set-keyboard-coding-system
This function specifies @var{coding-system} as the coding system to
use for decoding keyboard input. If @var{coding-system} is @code{nil},
that means do not decode keyboard input.
@end defun
@defun terminal-coding-system
-@tindex terminal-coding-system
This function returns the coding system that is in use for encoding
terminal output---or @code{nil} for no encoding.
@end defun
@defun set-terminal-coding-system coding-system
-@tindex set-terminal-coding-system
This function specifies @var{coding-system} as the coding system to use
for encoding terminal output. If @var{coding-system} is @code{nil},
that means do not encode terminal output.
On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
end-of-line conversion for a file by looking at the file's name. This
-feature classifies fils as @dfn{text files} and @dfn{binary files}. By
+feature classifies files as @dfn{text files} and @dfn{binary files}. By
``binary file'' we mean a file of literal byte values that are not
necessarily meant to be characters; Emacs does no end-of-line conversion
and no character code conversion for them. On the other hand, the bytes
Each input method has a name, which is currently a string;
in the future, symbols may also be usable as input method names.
-@tindex current-input-method
@defvar current-input-method
This variable holds the name of the input method now active in the
current buffer. (It automatically becomes local in each buffer when set
buffer now.
@end defvar
-@tindex default-input-method
@defvar default-input-method
This variable holds the default input method for commands that choose an
input method. Unlike @code{current-input-method}, this variable is
normally global.
@end defvar
-@tindex set-input-method
@defun set-input-method input-method
This function activates input method @var{input-method} for the current
buffer. It also sets @code{default-input-method} to @var{input-method}.
method for the current buffer.
@end defun
-@tindex read-input-method-name
@defun read-input-method-name prompt &optional default inhibit-null
This function reads an input method name with the minibuffer, prompting
with @var{prompt}. If @var{default} is non-@code{nil}, that is returned
The returned value is a string.
@end defun
-@tindex input-method-alist
@defvar input-method-alist
This variable defines all the supported input methods.
Each element defines one input method, and should have the form:
environment this input method is recommended for. (That serves only for
documentation purposes.)
-@var{title} is a string to display in the mode line while this method is
-active. @var{description} is a string describing this method and what
-it is good for.
-
@var{activate-func} is a function to call to activate this method. The
@var{args}, if any, are passed as arguments to @var{activate-func}. All
told, the arguments to @var{activate-func} are @var{input-method} and
the @var{args}.
+
+@var{title} is a string to display in the mode line while this method is
+active. @var{description} is a string describing this method and what
+it is good for.
@end defvar
The fundamental interface to input methods is through the
variable @code{input-method-function}. @xref{Reading One Event}.
+
+@node Locales
+@section Locales
+@cindex locale
+
+ POSIX defines a concept of ``locales'' which control which language
+to use in language-related features. These Emacs variables control
+how Emacs interacts with these features.
+
+@defvar locale-coding-system
+@tindex locale-coding-system
+@cindex keyboard input decoding on X
+This variable specifies the coding system to use for decoding system
+error messages and---on X Window system only---keyboard input, for
+encoding the format argument to @code{format-time-string}, and for
+decoding the return value of @code{format-time-string}.
+@end defvar
+
+@defvar system-messages-locale
+@tindex system-messages-locale
+This variable specifies the locale to use for generating system error
+messages. Changing the locale can cause messages to come out in a
+different language or in a different orthography. If the variable is
+@code{nil}, the locale is specified by environment variables in the
+usual POSIX fashion.
+@end defvar
+
+@defvar system-time-locale
+@tindex system-time-locale
+This variable specifies the locale to use for formatting time values.
+Changing the locale can cause messages to appear according to the
+conventions of a different language. If the variable is @code{nil}, the
+locale is specified by environment variables in the usual POSIX fashion.
+@end defvar
+