1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 1995, 1997, 1998 Electrotechnical Laboratory, JAPAN.
3 Licensed to the Free Software Foundation.
5 This file is part of GNU Emacs.
7 GNU Emacs is free software; you can redistribute it and/or modify
8 it under the terms of the GNU General Public License as published by
9 the Free Software Foundation; either version 2, or (at your option)
12 GNU Emacs is distributed in the hope that it will be useful,
13 but WITHOUT ANY WARRANTY; without even the implied warranty of
14 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 GNU General Public License for more details.
17 You should have received a copy of the GNU General Public License
18 along with GNU Emacs; see the file COPYING. If not, write to
19 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
20 Boston, MA 02111-1307, USA. */
22 /*** TABLE OF CONTENTS ***
26 2. Emacs' internal format (emacs-mule) handlers
28 4. Shift-JIS and BIG5 handlers
30 6. End-of-line handlers
31 7. C library functions
32 8. Emacs Lisp library functions
37 /*** 0. General comments ***/
40 /*** GENERAL NOTE on CODING SYSTEM ***
42 Coding system is an encoding mechanism of one or more character
43 sets. Here's a list of coding systems which Emacs can handle. When
44 we say "decode", it means converting some other coding system to
45 Emacs' internal format (emacs-internal), and when we say "encode",
46 it means converting the coding system emacs-mule to some other
49 0. Emacs' internal format (emacs-mule)
51 Emacs itself holds a multi-lingual character in a buffer and a string
52 in a special format. Details are described in section 2.
56 The most famous coding system for multiple character sets. X's
57 Compound Text, various EUCs (Extended Unix Code), and coding
58 systems used in Internet communication such as ISO-2022-JP are
59 all variants of ISO2022. Details are described in section 3.
61 2. SJIS (or Shift-JIS or MS-Kanji-Code)
63 A coding system to encode character sets: ASCII, JISX0201, and
64 JISX0208. Widely used for PC's in Japan. Details are described in
69 A coding system to encode character sets: ASCII and Big5. Widely
70 used by Chinese (mainly in Taiwan and Hong Kong). Details are
71 described in section 4. In this file, when we write "BIG5"
72 (all uppercase), we mean the coding system, and when we write
73 "Big5" (capitalized), we mean the character set.
77 A coding system for a text containing random 8-bit code. Emacs does
78 no code conversion on such a text except for end-of-line format.
82 If a user wants to read/write a text encoded in a coding system not
83 listed above, he can supply a decoder and an encoder for it in CCL
84 (Code Conversion Language) programs. Emacs executes the CCL program
85 while reading/writing.
87 Emacs represents a coding system by a Lisp symbol that has a property
88 `coding-system'. But, before actually using the coding system, the
89 information about it is set in a structure of type `struct
90 coding_system' for rapid processing. See section 6 for more details.
94 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
96 How end-of-line of a text is encoded depends on a system. For
97 instance, Unix's format is just one byte of `line-feed' code,
98 whereas DOS's format is two-byte sequence of `carriage-return' and
99 `line-feed' codes. MacOS's format is usually one byte of
102 Since text characters encoding and end-of-line encoding are
103 independent, any coding system described above can take
104 any format of end-of-line. So, Emacs has information of format of
105 end-of-line in each coding-system. See section 6 for more details.
109 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
111 These functions check if a text between SRC and SRC_END is encoded
112 in the coding system category XXX. Each returns an integer value in
113 which appropriate flag bits for the category XXX is set. The flag
114 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
115 template of these functions. */
118 detect_coding_emacs_mule (src
, src_end
)
119 unsigned char *src
, *src_end
;
125 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
127 These functions decode SRC_BYTES length of unibyte text at SOURCE
128 encoded in CODING to Emacs' internal format. The resulting
129 multibyte text goes to a place pointed to by DESTINATION, the length
130 of which should not exceed DST_BYTES.
132 These functions set the information of original and decoded texts in
133 the members produced, produced_char, consumed, and consumed_char of
134 the structure *CODING. They also set the member result to one of
135 CODING_FINISH_XXX indicating how the decoding finished.
137 DST_BYTES zero means that source area and destination area are
138 overlapped, which means that we can produce a decoded text until it
139 reaches at the head of not-yet-decoded source text.
141 Below is a template of these functions. */
144 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
145 struct coding_system
*coding
;
146 unsigned char *source
, *destination
;
147 int src_bytes
, dst_bytes
;
153 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
155 These functions encode SRC_BYTES length text at SOURCE of Emacs'
156 internal multibyte format to CODING. The resulting unibyte text
157 goes to a place pointed to by DESTINATION, the length of which
158 should not exceed DST_BYTES.
160 These functions set the information of original and encoded texts in
161 the members produced, produced_char, consumed, and consumed_char of
162 the structure *CODING. They also set the member result to one of
163 CODING_FINISH_XXX indicating how the encoding finished.
165 DST_BYTES zero means that source area and destination area are
166 overlapped, which means that we can produce a encoded text until it
167 reaches at the head of not-yet-encoded source text.
169 Below is a template of these functions. */
172 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
173 struct coding_system
*coding
;
174 unsigned char *source
, *destination
;
175 int src_bytes
, dst_bytes
;
181 /*** COMMONLY USED MACROS ***/
183 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
184 get one, two, and three bytes from the source text respectively.
185 If there are not enough bytes in the source, they jump to
186 `label_end_of_loop'. The caller should set variables `coding',
187 `src' and `src_end' to appropriate pointer in advance. These
188 macros are called from decoding routines `decode_coding_XXX', thus
189 it is assumed that the source text is unibyte. */
191 #define ONE_MORE_BYTE(c1) \
193 if (src >= src_end) \
195 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
196 goto label_end_of_loop; \
201 #define TWO_MORE_BYTES(c1, c2) \
203 if (src + 1 >= src_end) \
205 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
206 goto label_end_of_loop; \
213 /* Set C to the next character at the source text pointed by `src'.
214 If there are not enough characters in the source, jump to
215 `label_end_of_loop'. The caller should set variables `coding'
216 `src', `src_end', and `translation_table' to appropriate pointers
217 in advance. This macro is used in encoding routines
218 `encode_coding_XXX', thus it assumes that the source text is in
219 multibyte form except for 8-bit characters. 8-bit characters are
220 in multibyte form if coding->src_multibyte is nonzero, else they
221 are represented by a single byte. */
223 #define ONE_MORE_CHAR(c) \
225 int len = src_end - src; \
229 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
230 goto label_end_of_loop; \
232 if (coding->src_multibyte \
233 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
234 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
236 c = *src, bytes = 1; \
237 if (!NILP (translation_table)) \
238 c = translate_char (translation_table, c, 0, 0, 0); \
243 /* Produce a multibyte form of characater C to `dst'. Jump to
244 `label_end_of_loop' if there's not enough space at `dst'.
246 If we are now in the middle of composition sequence, the decoded
247 character may be ALTCHAR (for the current composition). In that
248 case, the character goes to coding->cmp_data->data instead of
251 This macro is used in decoding routines. */
253 #define EMIT_CHAR(c) \
255 if (! COMPOSING_P (coding) \
256 || coding->composing == COMPOSITION_RELATIVE \
257 || coding->composing == COMPOSITION_WITH_RULE) \
259 int bytes = CHAR_BYTES (c); \
260 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
262 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
263 goto label_end_of_loop; \
265 dst += CHAR_STRING (c, dst); \
266 coding->produced_char++; \
269 if (COMPOSING_P (coding) \
270 && coding->composing != COMPOSITION_RELATIVE) \
272 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
273 coding->composition_rule_follows \
274 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
279 #define EMIT_ONE_BYTE(c) \
281 if (dst >= (dst_bytes ? dst_end : src)) \
283 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
284 goto label_end_of_loop; \
289 #define EMIT_TWO_BYTES(c1, c2) \
291 if (dst + 2 > (dst_bytes ? dst_end : src)) \
293 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
294 goto label_end_of_loop; \
296 *dst++ = c1, *dst++ = c2; \
299 #define EMIT_BYTES(from, to) \
301 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
303 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
304 goto label_end_of_loop; \
311 /*** 1. Preamble ***/
324 #include "composite.h"
329 #else /* not emacs */
333 #endif /* not emacs */
335 Lisp_Object Qcoding_system
, Qeol_type
;
336 Lisp_Object Qbuffer_file_coding_system
;
337 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
338 Lisp_Object Qno_conversion
, Qundecided
;
339 Lisp_Object Qcoding_system_history
;
340 Lisp_Object Qsafe_chars
;
341 Lisp_Object Qvalid_codes
;
343 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
344 Lisp_Object Qcall_process
, Qcall_process_region
, Qprocess_argument
;
345 Lisp_Object Qstart_process
, Qopen_network_stream
;
346 Lisp_Object Qtarget_idx
;
348 Lisp_Object Vselect_safe_coding_system_function
;
350 /* Mnemonic string for each format of end-of-line. */
351 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
352 /* Mnemonic string to indicate format of end-of-line is not yet
354 Lisp_Object eol_mnemonic_undecided
;
356 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
357 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
362 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
364 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
366 /* Coding system emacs-mule and raw-text are for converting only
367 end-of-line format. */
368 Lisp_Object Qemacs_mule
, Qraw_text
;
370 /* Coding-systems are handed between Emacs Lisp programs and C internal
371 routines by the following three variables. */
372 /* Coding-system for reading files and receiving data from process. */
373 Lisp_Object Vcoding_system_for_read
;
374 /* Coding-system for writing files and sending data to process. */
375 Lisp_Object Vcoding_system_for_write
;
376 /* Coding-system actually used in the latest I/O. */
377 Lisp_Object Vlast_coding_system_used
;
379 /* A vector of length 256 which contains information about special
380 Latin codes (especially for dealing with Microsoft codes). */
381 Lisp_Object Vlatin_extra_code_table
;
383 /* Flag to inhibit code conversion of end-of-line format. */
384 int inhibit_eol_conversion
;
386 /* Flag to inhibit ISO2022 escape sequence detection. */
387 int inhibit_iso_escape_detection
;
389 /* Flag to make buffer-file-coding-system inherit from process-coding. */
390 int inherit_process_coding_system
;
392 /* Coding system to be used to encode text for terminal display. */
393 struct coding_system terminal_coding
;
395 /* Coding system to be used to encode text for terminal display when
396 terminal coding system is nil. */
397 struct coding_system safe_terminal_coding
;
399 /* Coding system of what is sent from terminal keyboard. */
400 struct coding_system keyboard_coding
;
402 /* Default coding system to be used to write a file. */
403 struct coding_system default_buffer_file_coding
;
405 Lisp_Object Vfile_coding_system_alist
;
406 Lisp_Object Vprocess_coding_system_alist
;
407 Lisp_Object Vnetwork_coding_system_alist
;
409 Lisp_Object Vlocale_coding_system
;
413 Lisp_Object Qcoding_category
, Qcoding_category_index
;
415 /* List of symbols `coding-category-xxx' ordered by priority. */
416 Lisp_Object Vcoding_category_list
;
418 /* Table of coding categories (Lisp symbols). */
419 Lisp_Object Vcoding_category_table
;
421 /* Table of names of symbol for each coding-category. */
422 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
423 "coding-category-emacs-mule",
424 "coding-category-sjis",
425 "coding-category-iso-7",
426 "coding-category-iso-7-tight",
427 "coding-category-iso-8-1",
428 "coding-category-iso-8-2",
429 "coding-category-iso-7-else",
430 "coding-category-iso-8-else",
431 "coding-category-ccl",
432 "coding-category-big5",
433 "coding-category-utf-8",
434 "coding-category-utf-16-be",
435 "coding-category-utf-16-le",
436 "coding-category-raw-text",
437 "coding-category-binary"
440 /* Table of pointers to coding systems corresponding to each coding
442 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
444 /* Table of coding category masks. Nth element is a mask for a coding
445 cateogry of which priority is Nth. */
447 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
449 /* Flag to tell if we look up translation table on character code
451 Lisp_Object Venable_character_translation
;
452 /* Standard translation table to look up on decoding (reading). */
453 Lisp_Object Vstandard_translation_table_for_decode
;
454 /* Standard translation table to look up on encoding (writing). */
455 Lisp_Object Vstandard_translation_table_for_encode
;
457 Lisp_Object Qtranslation_table
;
458 Lisp_Object Qtranslation_table_id
;
459 Lisp_Object Qtranslation_table_for_decode
;
460 Lisp_Object Qtranslation_table_for_encode
;
462 /* Alist of charsets vs revision number. */
463 Lisp_Object Vcharset_revision_alist
;
465 /* Default coding systems used for process I/O. */
466 Lisp_Object Vdefault_process_coding_system
;
468 /* Global flag to tell that we can't call post-read-conversion and
469 pre-write-conversion functions. Usually the value is zero, but it
470 is set to 1 temporarily while such functions are running. This is
471 to avoid infinite recursive call. */
472 static int inhibit_pre_post_conversion
;
474 /* Char-table containing safe coding systems of each character. */
475 Lisp_Object Vchar_coding_system_table
;
476 Lisp_Object Qchar_coding_system
;
478 /* Return `safe-chars' property of coding system CODING. Don't check
479 validity of CODING. */
482 coding_safe_chars (coding
)
483 struct coding_system
*coding
;
485 Lisp_Object coding_spec
, plist
, safe_chars
;
487 coding_spec
= Fget (coding
->symbol
, Qcoding_system
);
488 plist
= XVECTOR (coding_spec
)->contents
[3];
489 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
490 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
493 #define CODING_SAFE_CHAR_P(safe_chars, c) \
494 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
497 /*** 2. Emacs internal format (emacs-mule) handlers ***/
499 /* Emacs' internal format for encoding multiple character sets is a
500 kind of multi-byte encoding, i.e. characters are encoded by
501 variable-length sequences of one-byte codes.
503 ASCII characters and control characters (e.g. `tab', `newline') are
504 represented by one-byte sequences which are their ASCII codes, in
505 the range 0x00 through 0x7F.
507 8-bit characters of the range 0x80..0x9F are represented by
508 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
511 8-bit characters of the range 0xA0..0xFF are represented by
512 one-byte sequences which are their 8-bit code.
514 The other characters are represented by a sequence of `base
515 leading-code', optional `extended leading-code', and one or two
516 `position-code's. The length of the sequence is determined by the
517 base leading-code. Leading-code takes the range 0x80 through 0x9F,
518 whereas extended leading-code and position-code take the range 0xA0
519 through 0xFF. See `charset.h' for more details about leading-code
522 --- CODE RANGE of Emacs' internal format ---
526 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
527 eight-bit-graphic 0xA0..0xBF
528 ELSE 0x81..0x9F + [0xA0..0xFF]+
529 ---------------------------------------------
533 enum emacs_code_class_type emacs_code_class
[256];
535 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
536 Check if a text is encoded in Emacs' internal format. If it is,
537 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
540 detect_coding_emacs_mule (src
, src_end
)
541 unsigned char *src
, *src_end
;
545 /* Dummy for ONE_MORE_BYTE. */
546 struct coding_system dummy_coding
;
547 struct coding_system
*coding
= &dummy_coding
;
568 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
571 else if (c
>= 0x80 && c
< 0xA0)
574 /* Old leading code for a composite character. */
578 unsigned char *src_base
= src
- 1;
581 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
584 src
= src_base
+ bytes
;
589 return CODING_CATEGORY_MASK_EMACS_MULE
;
593 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
596 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
597 struct coding_system
*coding
;
598 unsigned char *source
, *destination
;
599 int src_bytes
, dst_bytes
;
601 unsigned char *src
= source
;
602 unsigned char *src_end
= source
+ src_bytes
;
603 unsigned char *dst
= destination
;
604 unsigned char *dst_end
= destination
+ dst_bytes
;
605 /* SRC_BASE remembers the start position in source in each loop.
606 The loop will be exited when there's not enough source code, or
607 when there's not enough destination area to produce a
609 unsigned char *src_base
;
611 coding
->produced_char
= 0;
612 while ((src_base
= src
) < src_end
)
614 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
], *p
;
617 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
))
624 bytes
= CHAR_STRING (*src
, tmp
);
628 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
630 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
633 while (bytes
--) *dst
++ = *p
++;
634 coding
->produced_char
++;
636 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
637 coding
->produced
= dst
- destination
;
640 #define encode_coding_emacs_mule(coding, source, destination, src_bytes, dst_bytes) \
641 encode_eol (coding, source, destination, src_bytes, dst_bytes)
645 /*** 3. ISO2022 handlers ***/
647 /* The following note describes the coding system ISO2022 briefly.
648 Since the intention of this note is to help understand the
649 functions in this file, some parts are NOT ACCURATE or OVERLY
650 SIMPLIFIED. For thorough understanding, please refer to the
651 original document of ISO2022.
653 ISO2022 provides many mechanisms to encode several character sets
654 in 7-bit and 8-bit environments. For 7-bite environments, all text
655 is encoded using bytes less than 128. This may make the encoded
656 text a little bit longer, but the text passes more easily through
657 several gateways, some of which strip off MSB (Most Signigant Bit).
659 There are two kinds of character sets: control character set and
660 graphic character set. The former contains control characters such
661 as `newline' and `escape' to provide control functions (control
662 functions are also provided by escape sequences). The latter
663 contains graphic characters such as 'A' and '-'. Emacs recognizes
664 two control character sets and many graphic character sets.
666 Graphic character sets are classified into one of the following
667 four classes, according to the number of bytes (DIMENSION) and
668 number of characters in one dimension (CHARS) of the set:
674 In addition, each character set is assigned an identification tag,
675 unique for each set, called "final character" (denoted as <F>
676 hereafter). The <F> of each character set is decided by ECMA(*)
677 when it is registered in ISO. The code range of <F> is 0x30..0x7F
678 (0x30..0x3F are for private use only).
680 Note (*): ECMA = European Computer Manufacturers Association
682 Here are examples of graphic character set [NAME(<F>)]:
683 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
684 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
685 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
686 o DIMENSION2_CHARS96 -- none for the moment
688 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
689 C0 [0x00..0x1F] -- control character plane 0
690 GL [0x20..0x7F] -- graphic character plane 0
691 C1 [0x80..0x9F] -- control character plane 1
692 GR [0xA0..0xFF] -- graphic character plane 1
694 A control character set is directly designated and invoked to C0 or
695 C1 by an escape sequence. The most common case is that:
696 - ISO646's control character set is designated/invoked to C0, and
697 - ISO6429's control character set is designated/invoked to C1,
698 and usually these designations/invocations are omitted in encoded
699 text. In a 7-bit environment, only C0 can be used, and a control
700 character for C1 is encoded by an appropriate escape sequence to
701 fit into the environment. All control characters for C1 are
702 defined to have corresponding escape sequences.
704 A graphic character set is at first designated to one of four
705 graphic registers (G0 through G3), then these graphic registers are
706 invoked to GL or GR. These designations and invocations can be
707 done independently. The most common case is that G0 is invoked to
708 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
709 these invocations and designations are omitted in encoded text.
710 In a 7-bit environment, only GL can be used.
712 When a graphic character set of CHARS94 is invoked to GL, codes
713 0x20 and 0x7F of the GL area work as control characters SPACE and
714 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
717 There are two ways of invocation: locking-shift and single-shift.
718 With locking-shift, the invocation lasts until the next different
719 invocation, whereas with single-shift, the invocation affects the
720 following character only and doesn't affect the locking-shift
721 state. Invocations are done by the following control characters or
724 ----------------------------------------------------------------------
725 abbrev function cntrl escape seq description
726 ----------------------------------------------------------------------
727 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
728 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
729 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
730 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
731 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
732 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
733 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
734 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
735 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
736 ----------------------------------------------------------------------
737 (*) These are not used by any known coding system.
739 Control characters for these functions are defined by macros
740 ISO_CODE_XXX in `coding.h'.
742 Designations are done by the following escape sequences:
743 ----------------------------------------------------------------------
744 escape sequence description
745 ----------------------------------------------------------------------
746 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
747 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
748 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
749 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
750 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
751 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
752 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
753 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
754 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
755 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
756 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
757 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
758 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
759 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
760 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
761 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
762 ----------------------------------------------------------------------
764 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
765 of dimension 1, chars 94, and final character <F>, etc...
767 Note (*): Although these designations are not allowed in ISO2022,
768 Emacs accepts them on decoding, and produces them on encoding
769 CHARS96 character sets in a coding system which is characterized as
770 7-bit environment, non-locking-shift, and non-single-shift.
772 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
773 '(' can be omitted. We refer to this as "short-form" hereafter.
775 Now you may notice that there are a lot of ways for encoding the
776 same multilingual text in ISO2022. Actually, there exist many
777 coding systems such as Compound Text (used in X11's inter client
778 communication, ISO-2022-JP (used in Japanese internet), ISO-2022-KR
779 (used in Korean internet), EUC (Extended UNIX Code, used in Asian
780 localized platforms), and all of these are variants of ISO2022.
782 In addition to the above, Emacs handles two more kinds of escape
783 sequences: ISO6429's direction specification and Emacs' private
784 sequence for specifying character composition.
786 ISO6429's direction specification takes the following form:
787 o CSI ']' -- end of the current direction
788 o CSI '0' ']' -- end of the current direction
789 o CSI '1' ']' -- start of left-to-right text
790 o CSI '2' ']' -- start of right-to-left text
791 The control character CSI (0x9B: control sequence introducer) is
792 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
794 Character composition specification takes the following form:
795 o ESC '0' -- start relative composition
796 o ESC '1' -- end composition
797 o ESC '2' -- start rule-base composition (*)
798 o ESC '3' -- start relative composition with alternate chars (**)
799 o ESC '4' -- start rule-base composition with alternate chars (**)
800 Since these are not standard escape sequences of any ISO standard,
801 the use of them for these meaning is restricted to Emacs only.
803 (*) This form is used only in Emacs 20.5 and the older versions,
804 but the newer versions can safely decode it.
805 (**) This form is used only in Emacs 21.1 and the newer versions,
806 and the older versions can't decode it.
808 Here's a list of examples usages of these composition escape
809 sequences (categorized by `enum composition_method').
811 COMPOSITION_RELATIVE:
812 ESC 0 CHAR [ CHAR ] ESC 1
813 COMPOSITOIN_WITH_RULE:
814 ESC 2 CHAR [ RULE CHAR ] ESC 1
815 COMPOSITION_WITH_ALTCHARS:
816 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
817 COMPOSITION_WITH_RULE_ALTCHARS:
818 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
820 enum iso_code_class_type iso_code_class
[256];
822 #define CHARSET_OK(idx, charset, c) \
823 (coding_system_table[idx] \
824 && (charset == CHARSET_ASCII \
825 || (safe_chars = coding_safe_chars (coding_system_table[idx]), \
826 CODING_SAFE_CHAR_P (safe_chars, c))) \
827 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
829 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
831 #define SHIFT_OUT_OK(idx) \
832 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
834 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
835 Check if a text is encoded in ISO2022. If it is, returns an
836 integer in which appropriate flag bits any of:
837 CODING_CATEGORY_MASK_ISO_7
838 CODING_CATEGORY_MASK_ISO_7_TIGHT
839 CODING_CATEGORY_MASK_ISO_8_1
840 CODING_CATEGORY_MASK_ISO_8_2
841 CODING_CATEGORY_MASK_ISO_7_ELSE
842 CODING_CATEGORY_MASK_ISO_8_ELSE
843 are set. If a code which should never appear in ISO2022 is found,
847 detect_coding_iso2022 (src
, src_end
)
848 unsigned char *src
, *src_end
;
850 int mask
= CODING_CATEGORY_MASK_ISO
;
852 int reg
[4], shift_out
= 0, single_shifting
= 0;
853 int c
, c1
, i
, charset
;
854 /* Dummy for ONE_MORE_BYTE. */
855 struct coding_system dummy_coding
;
856 struct coding_system
*coding
= &dummy_coding
;
857 Lisp_Object safe_chars
;
859 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
860 while (mask
&& src
< src_end
)
866 if (inhibit_iso_escape_detection
)
870 if (c
>= '(' && c
<= '/')
872 /* Designation sequence for a charset of dimension 1. */
874 if (c1
< ' ' || c1
>= 0x80
875 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
876 /* Invalid designation sequence. Just ignore. */
878 reg
[(c
- '(') % 4] = charset
;
882 /* Designation sequence for a charset of dimension 2. */
884 if (c
>= '@' && c
<= 'B')
885 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
886 reg
[0] = charset
= iso_charset_table
[1][0][c
];
887 else if (c
>= '(' && c
<= '/')
890 if (c1
< ' ' || c1
>= 0x80
891 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
892 /* Invalid designation sequence. Just ignore. */
894 reg
[(c
- '(') % 4] = charset
;
897 /* Invalid designation sequence. Just ignore. */
900 else if (c
== 'N' || c
== 'O')
902 /* ESC <Fe> for SS2 or SS3. */
903 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
906 else if (c
>= '0' && c
<= '4')
908 /* ESC <Fp> for start/end composition. */
909 mask_found
|= CODING_CATEGORY_MASK_ISO
;
913 /* Invalid escape sequence. Just ignore. */
916 /* We found a valid designation sequence for CHARSET. */
917 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
918 c
= MAKE_CHAR (charset
, 0, 0);
919 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
920 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
922 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
923 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
924 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
926 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
927 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
928 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
930 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
931 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
932 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
934 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
938 if (inhibit_iso_escape_detection
)
943 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
944 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
946 /* Locking shift out. */
947 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
948 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
953 if (inhibit_iso_escape_detection
)
958 /* Locking shift in. */
959 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
960 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
969 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
971 if (inhibit_iso_escape_detection
)
973 if (c
!= ISO_CODE_CSI
)
975 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
976 & CODING_FLAG_ISO_SINGLE_SHIFT
)
977 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
978 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
979 & CODING_FLAG_ISO_SINGLE_SHIFT
)
980 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
983 if (VECTORP (Vlatin_extra_code_table
)
984 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
986 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
987 & CODING_FLAG_ISO_LATIN_EXTRA
)
988 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
989 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
990 & CODING_FLAG_ISO_LATIN_EXTRA
)
991 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
994 mask_found
|= newmask
;
1001 single_shifting
= 0;
1006 single_shifting
= 0;
1007 if (VECTORP (Vlatin_extra_code_table
)
1008 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1012 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1013 & CODING_FLAG_ISO_LATIN_EXTRA
)
1014 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1015 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1016 & CODING_FLAG_ISO_LATIN_EXTRA
)
1017 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1019 mask_found
|= newmask
;
1026 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1027 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1028 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1029 /* Check the length of succeeding codes of the range
1030 0xA0..0FF. If the byte length is odd, we exclude
1031 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1032 when we are not single shifting. */
1033 if (!single_shifting
1034 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1037 while (src
< src_end
)
1045 if (i
& 1 && src
< src_end
)
1046 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1048 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1055 return (mask
& mask_found
);
1058 /* Decode a character of which charset is CHARSET, the 1st position
1059 code is C1, the 2nd position code is C2, and return the decoded
1060 character code. If the variable `translation_table' is non-nil,
1061 returned the translated code. */
1063 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1064 (NILP (translation_table) \
1065 ? MAKE_CHAR (charset, c1, c2) \
1066 : translate_char (translation_table, -1, charset, c1, c2))
1068 /* Set designation state into CODING. */
1069 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1073 if (final_char < '0' || final_char >= 128) \
1074 goto label_invalid_code; \
1075 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1076 make_number (chars), \
1077 make_number (final_char)); \
1078 c = MAKE_CHAR (charset, 0, 0); \
1080 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1081 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1083 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1085 && charset == CHARSET_ASCII) \
1087 /* We should insert this designation sequence as is so \
1088 that it is surely written back to a file. */ \
1089 coding->spec.iso2022.last_invalid_designation_register = -1; \
1090 goto label_invalid_code; \
1092 coding->spec.iso2022.last_invalid_designation_register = -1; \
1093 if ((coding->mode & CODING_MODE_DIRECTION) \
1094 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1095 charset = CHARSET_REVERSE_CHARSET (charset); \
1096 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1100 coding->spec.iso2022.last_invalid_designation_register = reg; \
1101 goto label_invalid_code; \
1105 /* Allocate a memory block for storing information about compositions.
1106 The block is chained to the already allocated blocks. */
1109 coding_allocate_composition_data (coding
, char_offset
)
1110 struct coding_system
*coding
;
1113 struct composition_data
*cmp_data
1114 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1116 cmp_data
->char_offset
= char_offset
;
1118 cmp_data
->prev
= coding
->cmp_data
;
1119 cmp_data
->next
= NULL
;
1120 if (coding
->cmp_data
)
1121 coding
->cmp_data
->next
= cmp_data
;
1122 coding
->cmp_data
= cmp_data
;
1123 coding
->cmp_data_start
= 0;
1126 /* Record the starting position START and METHOD of one composition. */
1128 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
1130 struct composition_data *cmp_data = coding->cmp_data; \
1131 int *data = cmp_data->data + cmp_data->used; \
1132 coding->cmp_data_start = cmp_data->used; \
1134 data[1] = cmp_data->char_offset + start; \
1135 data[3] = (int) method; \
1136 cmp_data->used += 4; \
1139 /* Record the ending position END of the current composition. */
1141 #define CODING_ADD_COMPOSITION_END(coding, end) \
1143 struct composition_data *cmp_data = coding->cmp_data; \
1144 int *data = cmp_data->data + coding->cmp_data_start; \
1145 data[0] = cmp_data->used - coding->cmp_data_start; \
1146 data[2] = cmp_data->char_offset + end; \
1149 /* Record one COMPONENT (alternate character or composition rule). */
1151 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
1152 (coding->cmp_data->data[coding->cmp_data->used++] = component)
1154 /* Handle compositoin start sequence ESC 0, ESC 2, ESC 3, or ESC 4. */
1156 #define DECODE_COMPOSITION_START(c1) \
1158 if (coding->composing == COMPOSITION_DISABLED) \
1160 *dst++ = ISO_CODE_ESC; \
1161 *dst++ = c1 & 0x7f; \
1162 coding->produced_char += 2; \
1164 else if (!COMPOSING_P (coding)) \
1166 /* This is surely the start of a composition. We must be sure \
1167 that coding->cmp_data has enough space to store the \
1168 information about the composition. If not, terminate the \
1169 current decoding loop, allocate one more memory block for \
1170 coding->cmp_data in the calller, then start the decoding \
1171 loop again. We can't allocate memory here directly because \
1172 it may cause buffer/string relocation. */ \
1173 if (!coding->cmp_data \
1174 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1175 >= COMPOSITION_DATA_SIZE)) \
1177 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1178 goto label_end_of_loop; \
1180 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1181 : c1 == '2' ? COMPOSITION_WITH_RULE \
1182 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1183 : COMPOSITION_WITH_RULE_ALTCHARS); \
1184 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1185 coding->composing); \
1186 coding->composition_rule_follows = 0; \
1190 /* We are already handling a composition. If the method is \
1191 the following two, the codes following the current escape \
1192 sequence are actual characters stored in a buffer. */ \
1193 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1194 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1196 coding->composing = COMPOSITION_RELATIVE; \
1197 coding->composition_rule_follows = 0; \
1202 /* Handle compositoin end sequence ESC 1. */
1204 #define DECODE_COMPOSITION_END(c1) \
1206 if (coding->composing == COMPOSITION_DISABLED) \
1208 *dst++ = ISO_CODE_ESC; \
1210 coding->produced_char += 2; \
1214 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1215 coding->composing = COMPOSITION_NO; \
1219 /* Decode a composition rule from the byte C1 (and maybe one more byte
1220 from SRC) and store one encoded composition rule in
1221 coding->cmp_data. */
1223 #define DECODE_COMPOSITION_RULE(c1) \
1227 if (c1 < 81) /* old format (before ver.21) */ \
1229 int gref = (c1) / 9; \
1230 int nref = (c1) % 9; \
1231 if (gref == 4) gref = 10; \
1232 if (nref == 4) nref = 10; \
1233 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1235 else if (c1 < 93) /* new format (after ver.21) */ \
1237 ONE_MORE_BYTE (c2); \
1238 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1240 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1241 coding->composition_rule_follows = 0; \
1245 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1248 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1249 struct coding_system
*coding
;
1250 unsigned char *source
, *destination
;
1251 int src_bytes
, dst_bytes
;
1253 unsigned char *src
= source
;
1254 unsigned char *src_end
= source
+ src_bytes
;
1255 unsigned char *dst
= destination
;
1256 unsigned char *dst_end
= destination
+ dst_bytes
;
1257 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1258 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1259 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1260 /* SRC_BASE remembers the start position in source in each loop.
1261 The loop will be exited when there's not enough source code
1262 (within macro ONE_MORE_BYTE), or when there's not enough
1263 destination area to produce a character (within macro
1265 unsigned char *src_base
;
1267 Lisp_Object translation_table
;
1268 Lisp_Object safe_chars
;
1270 safe_chars
= coding_safe_chars (coding
);
1272 if (NILP (Venable_character_translation
))
1273 translation_table
= Qnil
;
1276 translation_table
= coding
->translation_table_for_decode
;
1277 if (NILP (translation_table
))
1278 translation_table
= Vstandard_translation_table_for_decode
;
1281 coding
->result
= CODING_FINISH_NORMAL
;
1290 /* We produce no character or one character. */
1291 switch (iso_code_class
[c1
])
1293 case ISO_0x20_or_0x7F
:
1294 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1296 DECODE_COMPOSITION_RULE (c1
);
1299 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1301 /* This is SPACE or DEL. */
1302 charset
= CHARSET_ASCII
;
1305 /* This is a graphic character, we fall down ... */
1307 case ISO_graphic_plane_0
:
1308 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1310 DECODE_COMPOSITION_RULE (c1
);
1316 case ISO_0xA0_or_0xFF
:
1317 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1318 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1319 goto label_invalid_code
;
1320 /* This is a graphic character, we fall down ... */
1322 case ISO_graphic_plane_1
:
1324 goto label_invalid_code
;
1329 if (COMPOSING_P (coding
))
1330 DECODE_COMPOSITION_END ('1');
1332 /* All ISO2022 control characters in this class have the
1333 same representation in Emacs internal format. */
1335 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1336 && (coding
->eol_type
== CODING_EOL_CR
1337 || coding
->eol_type
== CODING_EOL_CRLF
))
1339 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1340 goto label_end_of_loop
;
1342 charset
= CHARSET_ASCII
;
1346 if (COMPOSING_P (coding
))
1347 DECODE_COMPOSITION_END ('1');
1348 goto label_invalid_code
;
1350 case ISO_carriage_return
:
1351 if (COMPOSING_P (coding
))
1352 DECODE_COMPOSITION_END ('1');
1354 if (coding
->eol_type
== CODING_EOL_CR
)
1356 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1359 if (c1
!= ISO_CODE_LF
)
1361 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1363 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1364 goto label_end_of_loop
;
1370 charset
= CHARSET_ASCII
;
1374 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1375 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1376 goto label_invalid_code
;
1377 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1378 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1382 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1383 goto label_invalid_code
;
1384 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1385 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1388 case ISO_single_shift_2_7
:
1389 case ISO_single_shift_2
:
1390 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1391 goto label_invalid_code
;
1392 /* SS2 is handled as an escape sequence of ESC 'N' */
1394 goto label_escape_sequence
;
1396 case ISO_single_shift_3
:
1397 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1398 goto label_invalid_code
;
1399 /* SS2 is handled as an escape sequence of ESC 'O' */
1401 goto label_escape_sequence
;
1403 case ISO_control_sequence_introducer
:
1404 /* CSI is handled as an escape sequence of ESC '[' ... */
1406 goto label_escape_sequence
;
1410 label_escape_sequence
:
1411 /* Escape sequences handled by Emacs are invocation,
1412 designation, direction specification, and character
1413 composition specification. */
1416 case '&': /* revision of following character set */
1418 if (!(c1
>= '@' && c1
<= '~'))
1419 goto label_invalid_code
;
1421 if (c1
!= ISO_CODE_ESC
)
1422 goto label_invalid_code
;
1424 goto label_escape_sequence
;
1426 case '$': /* designation of 2-byte character set */
1427 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1428 goto label_invalid_code
;
1430 if (c1
>= '@' && c1
<= 'B')
1431 { /* designation of JISX0208.1978, GB2312.1980,
1433 DECODE_DESIGNATION (0, 2, 94, c1
);
1435 else if (c1
>= 0x28 && c1
<= 0x2B)
1436 { /* designation of DIMENSION2_CHARS94 character set */
1438 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
1440 else if (c1
>= 0x2C && c1
<= 0x2F)
1441 { /* designation of DIMENSION2_CHARS96 character set */
1443 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
1446 goto label_invalid_code
;
1447 /* We must update these variables now. */
1448 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1449 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1452 case 'n': /* invocation of locking-shift-2 */
1453 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1454 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
1455 goto label_invalid_code
;
1456 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
1457 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1460 case 'o': /* invocation of locking-shift-3 */
1461 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1462 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
1463 goto label_invalid_code
;
1464 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
1465 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1468 case 'N': /* invocation of single-shift-2 */
1469 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1470 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
1471 goto label_invalid_code
;
1472 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
1474 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
1475 goto label_invalid_code
;
1478 case 'O': /* invocation of single-shift-3 */
1479 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1480 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
1481 goto label_invalid_code
;
1482 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
1484 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
1485 goto label_invalid_code
;
1488 case '0': case '2': case '3': case '4': /* start composition */
1489 DECODE_COMPOSITION_START (c1
);
1492 case '1': /* end composition */
1493 DECODE_COMPOSITION_END (c1
);
1496 case '[': /* specification of direction */
1497 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
1498 goto label_invalid_code
;
1499 /* For the moment, nested direction is not supported.
1500 So, `coding->mode & CODING_MODE_DIRECTION' zero means
1501 left-to-right, and nozero means right-to-left. */
1505 case ']': /* end of the current direction */
1506 coding
->mode
&= ~CODING_MODE_DIRECTION
;
1508 case '0': /* end of the current direction */
1509 case '1': /* start of left-to-right direction */
1512 coding
->mode
&= ~CODING_MODE_DIRECTION
;
1514 goto label_invalid_code
;
1517 case '2': /* start of right-to-left direction */
1520 coding
->mode
|= CODING_MODE_DIRECTION
;
1522 goto label_invalid_code
;
1526 goto label_invalid_code
;
1531 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1532 goto label_invalid_code
;
1533 if (c1
>= 0x28 && c1
<= 0x2B)
1534 { /* designation of DIMENSION1_CHARS94 character set */
1536 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
1538 else if (c1
>= 0x2C && c1
<= 0x2F)
1539 { /* designation of DIMENSION1_CHARS96 character set */
1541 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
1544 goto label_invalid_code
;
1545 /* We must update these variables now. */
1546 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1547 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1552 /* Now we know CHARSET and 1st position code C1 of a character.
1553 Produce a multibyte sequence for that character while getting
1554 2nd position code C2 if necessary. */
1555 if (CHARSET_DIMENSION (charset
) == 2)
1558 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
1559 /* C2 is not in a valid range. */
1560 goto label_invalid_code
;
1562 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
1568 if (COMPOSING_P (coding
))
1569 DECODE_COMPOSITION_END ('1');
1576 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1577 coding
->produced
= dst
- destination
;
1582 /* ISO2022 encoding stuff. */
1585 It is not enough to say just "ISO2022" on encoding, we have to
1586 specify more details. In Emacs, each coding system of ISO2022
1587 variant has the following specifications:
1588 1. Initial designation to G0 thru G3.
1589 2. Allows short-form designation?
1590 3. ASCII should be designated to G0 before control characters?
1591 4. ASCII should be designated to G0 at end of line?
1592 5. 7-bit environment or 8-bit environment?
1593 6. Use locking-shift?
1594 7. Use Single-shift?
1595 And the following two are only for Japanese:
1596 8. Use ASCII in place of JIS0201-1976-Roman?
1597 9. Use JISX0208-1983 in place of JISX0208-1978?
1598 These specifications are encoded in `coding->flags' as flag bits
1599 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
1603 /* Produce codes (escape sequence) for designating CHARSET to graphic
1604 register REG at DST, and increment DST. If <final-char> of CHARSET is
1605 '@', 'A', or 'B' and the coding system CODING allows, produce
1606 designation sequence of short-form. */
1608 #define ENCODE_DESIGNATION(charset, reg, coding) \
1610 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
1611 char *intermediate_char_94 = "()*+"; \
1612 char *intermediate_char_96 = ",-./"; \
1613 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
1615 if (revision < 255) \
1617 *dst++ = ISO_CODE_ESC; \
1619 *dst++ = '@' + revision; \
1621 *dst++ = ISO_CODE_ESC; \
1622 if (CHARSET_DIMENSION (charset) == 1) \
1624 if (CHARSET_CHARS (charset) == 94) \
1625 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
1627 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
1632 if (CHARSET_CHARS (charset) == 94) \
1634 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
1636 || final_char < '@' || final_char > 'B') \
1637 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
1640 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
1642 *dst++ = final_char; \
1643 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1646 /* The following two macros produce codes (control character or escape
1647 sequence) for ISO2022 single-shift functions (single-shift-2 and
1650 #define ENCODE_SINGLE_SHIFT_2 \
1652 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1653 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
1655 *dst++ = ISO_CODE_SS2; \
1656 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
1659 #define ENCODE_SINGLE_SHIFT_3 \
1661 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1662 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
1664 *dst++ = ISO_CODE_SS3; \
1665 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
1668 /* The following four macros produce codes (control character or
1669 escape sequence) for ISO2022 locking-shift functions (shift-in,
1670 shift-out, locking-shift-2, and locking-shift-3). */
1672 #define ENCODE_SHIFT_IN \
1674 *dst++ = ISO_CODE_SI; \
1675 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
1678 #define ENCODE_SHIFT_OUT \
1680 *dst++ = ISO_CODE_SO; \
1681 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
1684 #define ENCODE_LOCKING_SHIFT_2 \
1686 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
1687 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
1690 #define ENCODE_LOCKING_SHIFT_3 \
1692 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
1693 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
1696 /* Produce codes for a DIMENSION1 character whose character set is
1697 CHARSET and whose position-code is C1. Designation and invocation
1698 sequences are also produced in advance if necessary. */
1700 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
1702 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
1704 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1705 *dst++ = c1 & 0x7F; \
1707 *dst++ = c1 | 0x80; \
1708 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
1711 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
1713 *dst++ = c1 & 0x7F; \
1716 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
1718 *dst++ = c1 | 0x80; \
1722 /* Since CHARSET is not yet invoked to any graphic planes, we \
1723 must invoke it, or, at first, designate it to some graphic \
1724 register. Then repeat the loop to actually produce the \
1726 dst = encode_invocation_designation (charset, coding, dst); \
1729 /* Produce codes for a DIMENSION2 character whose character set is
1730 CHARSET and whose position-codes are C1 and C2. Designation and
1731 invocation codes are also produced in advance if necessary. */
1733 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
1735 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
1737 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1738 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
1740 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
1741 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
1744 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
1746 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
1749 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
1751 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
1755 /* Since CHARSET is not yet invoked to any graphic planes, we \
1756 must invoke it, or, at first, designate it to some graphic \
1757 register. Then repeat the loop to actually produce the \
1759 dst = encode_invocation_designation (charset, coding, dst); \
1762 #define ENCODE_ISO_CHARACTER(c) \
1764 int charset, c1, c2; \
1766 SPLIT_CHAR (c, charset, c1, c2); \
1767 if (CHARSET_DEFINED_P (charset)) \
1769 if (CHARSET_DIMENSION (charset) == 1) \
1771 if (charset == CHARSET_ASCII \
1772 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
1773 charset = charset_latin_jisx0201; \
1774 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
1778 if (charset == charset_jisx0208 \
1779 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
1780 charset = charset_jisx0208_1978; \
1781 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
1793 /* Instead of encoding character C, produce one or two `?'s. */
1795 #define ENCODE_UNSAFE_CHARACTER(c) \
1797 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \
1798 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
1799 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \
1803 /* Produce designation and invocation codes at a place pointed by DST
1804 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
1808 encode_invocation_designation (charset
, coding
, dst
)
1810 struct coding_system
*coding
;
1813 int reg
; /* graphic register number */
1815 /* At first, check designations. */
1816 for (reg
= 0; reg
< 4; reg
++)
1817 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
1822 /* CHARSET is not yet designated to any graphic registers. */
1823 /* At first check the requested designation. */
1824 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
1825 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
1826 /* Since CHARSET requests no special designation, designate it
1827 to graphic register 0. */
1830 ENCODE_DESIGNATION (charset
, reg
, coding
);
1833 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
1834 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
1836 /* Since the graphic register REG is not invoked to any graphic
1837 planes, invoke it to graphic plane 0. */
1840 case 0: /* graphic register 0 */
1844 case 1: /* graphic register 1 */
1848 case 2: /* graphic register 2 */
1849 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1850 ENCODE_SINGLE_SHIFT_2
;
1852 ENCODE_LOCKING_SHIFT_2
;
1855 case 3: /* graphic register 3 */
1856 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1857 ENCODE_SINGLE_SHIFT_3
;
1859 ENCODE_LOCKING_SHIFT_3
;
1867 /* Produce 2-byte codes for encoded composition rule RULE. */
1869 #define ENCODE_COMPOSITION_RULE(rule) \
1872 COMPOSITION_DECODE_RULE (rule, gref, nref); \
1873 *dst++ = 32 + 81 + gref; \
1874 *dst++ = 32 + nref; \
1877 /* Produce codes for indicating the start of a composition sequence
1878 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
1879 which specify information about the composition. See the comment
1880 in coding.h for the format of DATA. */
1882 #define ENCODE_COMPOSITION_START(coding, data) \
1884 coding->composing = data[3]; \
1885 *dst++ = ISO_CODE_ESC; \
1886 if (coding->composing == COMPOSITION_RELATIVE) \
1890 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
1892 coding->cmp_data_index = coding->cmp_data_start + 4; \
1893 coding->composition_rule_follows = 0; \
1897 /* Produce codes for indicating the end of the current composition. */
1899 #define ENCODE_COMPOSITION_END(coding, data) \
1901 *dst++ = ISO_CODE_ESC; \
1903 coding->cmp_data_start += data[0]; \
1904 coding->composing = COMPOSITION_NO; \
1905 if (coding->cmp_data_start == coding->cmp_data->used \
1906 && coding->cmp_data->next) \
1908 coding->cmp_data = coding->cmp_data->next; \
1909 coding->cmp_data_start = 0; \
1913 /* Produce composition start sequence ESC 0. Here, this sequence
1914 doesn't mean the start of a new composition but means that we have
1915 just produced components (alternate chars and composition rules) of
1916 the composition and the actual text follows in SRC. */
1918 #define ENCODE_COMPOSITION_FAKE_START(coding) \
1920 *dst++ = ISO_CODE_ESC; \
1922 coding->composing = COMPOSITION_RELATIVE; \
1925 /* The following three macros produce codes for indicating direction
1927 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
1929 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
1930 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
1932 *dst++ = ISO_CODE_CSI; \
1935 #define ENCODE_DIRECTION_R2L \
1936 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
1938 #define ENCODE_DIRECTION_L2R \
1939 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
1941 /* Produce codes for designation and invocation to reset the graphic
1942 planes and registers to initial state. */
1943 #define ENCODE_RESET_PLANE_AND_REGISTER \
1946 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
1948 for (reg = 0; reg < 4; reg++) \
1949 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
1950 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
1951 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
1952 ENCODE_DESIGNATION \
1953 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
1956 /* Produce designation sequences of charsets in the line started from
1957 SRC to a place pointed by DST, and return updated DST.
1959 If the current block ends before any end-of-line, we may fail to
1960 find all the necessary designations. */
1962 static unsigned char *
1963 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
1964 struct coding_system
*coding
;
1965 Lisp_Object translation_table
;
1966 unsigned char *src
, *src_end
, *dst
;
1968 int charset
, c
, found
= 0, reg
;
1969 /* Table of charsets to be designated to each graphic register. */
1972 for (reg
= 0; reg
< 4; reg
++)
1981 charset
= CHAR_CHARSET (c
);
1982 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
1983 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
1993 for (reg
= 0; reg
< 4; reg
++)
1995 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
1996 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2002 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2005 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2006 struct coding_system
*coding
;
2007 unsigned char *source
, *destination
;
2008 int src_bytes
, dst_bytes
;
2010 unsigned char *src
= source
;
2011 unsigned char *src_end
= source
+ src_bytes
;
2012 unsigned char *dst
= destination
;
2013 unsigned char *dst_end
= destination
+ dst_bytes
;
2014 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2015 from DST_END to assure overflow checking is necessary only at the
2017 unsigned char *adjusted_dst_end
= dst_end
- 19;
2018 /* SRC_BASE remembers the start position in source in each loop.
2019 The loop will be exited when there's not enough source text to
2020 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2021 there's not enough destination area to produce encoded codes
2022 (within macro EMIT_BYTES). */
2023 unsigned char *src_base
;
2025 Lisp_Object translation_table
;
2026 Lisp_Object safe_chars
;
2028 safe_chars
= coding_safe_chars (coding
);
2030 if (NILP (Venable_character_translation
))
2031 translation_table
= Qnil
;
2034 translation_table
= coding
->translation_table_for_encode
;
2035 if (NILP (translation_table
))
2036 translation_table
= Vstandard_translation_table_for_encode
;
2039 coding
->consumed_char
= 0;
2045 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2047 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2051 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2052 && CODING_SPEC_ISO_BOL (coding
))
2054 /* We have to produce designation sequences if any now. */
2055 dst
= encode_designation_at_bol (coding
, translation_table
,
2057 CODING_SPEC_ISO_BOL (coding
) = 0;
2060 /* Check composition start and end. */
2061 if (coding
->composing
!= COMPOSITION_DISABLED
2062 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2064 struct composition_data
*cmp_data
= coding
->cmp_data
;
2065 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2066 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2068 if (coding
->composing
== COMPOSITION_RELATIVE
)
2070 if (this_pos
== data
[2])
2072 ENCODE_COMPOSITION_END (coding
, data
);
2073 cmp_data
= coding
->cmp_data
;
2074 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2077 else if (COMPOSING_P (coding
))
2079 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2080 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2081 /* We have consumed components of the composition.
2082 What follows in SRC is the compositions's base
2084 ENCODE_COMPOSITION_FAKE_START (coding
);
2087 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2088 if (coding
->composition_rule_follows
)
2090 ENCODE_COMPOSITION_RULE (c
);
2091 coding
->composition_rule_follows
= 0;
2095 if (coding
->flags
& CODING_FLAG_ISO_SAFE
2096 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2097 ENCODE_UNSAFE_CHARACTER (c
);
2099 ENCODE_ISO_CHARACTER (c
);
2100 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2101 coding
->composition_rule_follows
= 1;
2106 if (!COMPOSING_P (coding
))
2108 if (this_pos
== data
[1])
2110 ENCODE_COMPOSITION_START (coding
, data
);
2118 /* Now encode the character C. */
2119 if (c
< 0x20 || c
== 0x7F)
2123 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2125 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2126 ENCODE_RESET_PLANE_AND_REGISTER
;
2130 /* fall down to treat '\r' as '\n' ... */
2135 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2136 ENCODE_RESET_PLANE_AND_REGISTER
;
2137 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2138 bcopy (coding
->spec
.iso2022
.initial_designation
,
2139 coding
->spec
.iso2022
.current_designation
,
2140 sizeof coding
->spec
.iso2022
.initial_designation
);
2141 if (coding
->eol_type
== CODING_EOL_LF
2142 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2143 *dst
++ = ISO_CODE_LF
;
2144 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2145 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2147 *dst
++ = ISO_CODE_CR
;
2148 CODING_SPEC_ISO_BOL (coding
) = 1;
2152 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2153 ENCODE_RESET_PLANE_AND_REGISTER
;
2157 else if (ASCII_BYTE_P (c
))
2158 ENCODE_ISO_CHARACTER (c
);
2159 else if (SINGLE_BYTE_CHAR_P (c
))
2164 else if (coding
->flags
& CODING_FLAG_ISO_SAFE
2165 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2166 ENCODE_UNSAFE_CHARACTER (c
);
2168 ENCODE_ISO_CHARACTER (c
);
2170 coding
->consumed_char
++;
2174 coding
->consumed
= src_base
- source
;
2175 coding
->produced
= coding
->produced_char
= dst
- destination
;
2179 /*** 4. SJIS and BIG5 handlers ***/
2181 /* Although SJIS and BIG5 are not ISO's coding system, they are used
2182 quite widely. So, for the moment, Emacs supports them in the bare
2183 C code. But, in the future, they may be supported only by CCL. */
2185 /* SJIS is a coding system encoding three character sets: ASCII, right
2186 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2187 as is. A character of charset katakana-jisx0201 is encoded by
2188 "position-code + 0x80". A character of charset japanese-jisx0208
2189 is encoded in 2-byte but two position-codes are divided and shifted
2190 so that it fit in the range below.
2192 --- CODE RANGE of SJIS ---
2193 (character set) (range)
2195 KATAKANA-JISX0201 0xA0 .. 0xDF
2196 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2197 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2198 -------------------------------
2202 /* BIG5 is a coding system encoding two character sets: ASCII and
2203 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2204 character set and is encoded in two-byte.
2206 --- CODE RANGE of BIG5 ---
2207 (character set) (range)
2209 Big5 (1st byte) 0xA1 .. 0xFE
2210 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2211 --------------------------
2213 Since the number of characters in Big5 is larger than maximum
2214 characters in Emacs' charset (96x96), it can't be handled as one
2215 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2216 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2217 contains frequently used characters and the latter contains less
2218 frequently used characters. */
2220 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2221 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2222 C1 and C2 are the 1st and 2nd position-codes of of Emacs' internal
2223 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2225 /* Number of Big5 characters which have the same code in 1st byte. */
2226 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2228 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2231 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2233 charset = charset_big5_1; \
2236 charset = charset_big5_2; \
2237 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2239 c1 = temp / (0xFF - 0xA1) + 0x21; \
2240 c2 = temp % (0xFF - 0xA1) + 0x21; \
2243 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2245 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2246 if (charset == charset_big5_2) \
2247 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2248 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2249 b2 = temp % BIG5_SAME_ROW; \
2250 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2253 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2254 Check if a text is encoded in SJIS. If it is, return
2255 CODING_CATEGORY_MASK_SJIS, else return 0. */
2258 detect_coding_sjis (src
, src_end
)
2259 unsigned char *src
, *src_end
;
2262 /* Dummy for ONE_MORE_BYTE. */
2263 struct coding_system dummy_coding
;
2264 struct coding_system
*coding
= &dummy_coding
;
2269 if ((c
>= 0x80 && c
< 0xA0) || c
>= 0xE0)
2277 return CODING_CATEGORY_MASK_SJIS
;
2280 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2281 Check if a text is encoded in BIG5. If it is, return
2282 CODING_CATEGORY_MASK_BIG5, else return 0. */
2285 detect_coding_big5 (src
, src_end
)
2286 unsigned char *src
, *src_end
;
2289 /* Dummy for ONE_MORE_BYTE. */
2290 struct coding_system dummy_coding
;
2291 struct coding_system
*coding
= &dummy_coding
;
2299 if (c
< 0x40 || (c
>= 0x7F && c
<= 0xA0))
2304 return CODING_CATEGORY_MASK_BIG5
;
2307 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2308 Check if a text is encoded in UTF-8. If it is, return
2309 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2311 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2312 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2313 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2314 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2315 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2316 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2317 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2320 detect_coding_utf_8 (src
, src_end
)
2321 unsigned char *src
, *src_end
;
2324 int seq_maybe_bytes
;
2325 /* Dummy for ONE_MORE_BYTE. */
2326 struct coding_system dummy_coding
;
2327 struct coding_system
*coding
= &dummy_coding
;
2332 if (UTF_8_1_OCTET_P (c
))
2334 else if (UTF_8_2_OCTET_LEADING_P (c
))
2335 seq_maybe_bytes
= 1;
2336 else if (UTF_8_3_OCTET_LEADING_P (c
))
2337 seq_maybe_bytes
= 2;
2338 else if (UTF_8_4_OCTET_LEADING_P (c
))
2339 seq_maybe_bytes
= 3;
2340 else if (UTF_8_5_OCTET_LEADING_P (c
))
2341 seq_maybe_bytes
= 4;
2342 else if (UTF_8_6_OCTET_LEADING_P (c
))
2343 seq_maybe_bytes
= 5;
2350 if (!UTF_8_EXTRA_OCTET_P (c
))
2354 while (seq_maybe_bytes
> 0);
2358 return CODING_CATEGORY_MASK_UTF_8
;
2361 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2362 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
2363 Little Endian (otherwise). If it is, return
2364 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
2367 #define UTF_16_INVALID_P(val) \
2368 (((val) == 0xFFFE) \
2369 || ((val) == 0xFFFF))
2371 #define UTF_16_HIGH_SURROGATE_P(val) \
2372 (((val) & 0xD800) == 0xD800)
2374 #define UTF_16_LOW_SURROGATE_P(val) \
2375 (((val) & 0xDC00) == 0xDC00)
2378 detect_coding_utf_16 (src
, src_end
)
2379 unsigned char *src
, *src_end
;
2381 unsigned char c1
, c2
;
2382 /* Dummy for TWO_MORE_BYTES. */
2383 struct coding_system dummy_coding
;
2384 struct coding_system
*coding
= &dummy_coding
;
2386 TWO_MORE_BYTES (c1
, c2
);
2388 if ((c1
== 0xFF) && (c2
== 0xFE))
2389 return CODING_CATEGORY_MASK_UTF_16_LE
;
2390 else if ((c1
== 0xFE) && (c2
== 0xFF))
2391 return CODING_CATEGORY_MASK_UTF_16_BE
;
2397 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
2398 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
2401 decode_coding_sjis_big5 (coding
, source
, destination
,
2402 src_bytes
, dst_bytes
, sjis_p
)
2403 struct coding_system
*coding
;
2404 unsigned char *source
, *destination
;
2405 int src_bytes
, dst_bytes
;
2408 unsigned char *src
= source
;
2409 unsigned char *src_end
= source
+ src_bytes
;
2410 unsigned char *dst
= destination
;
2411 unsigned char *dst_end
= destination
+ dst_bytes
;
2412 /* SRC_BASE remembers the start position in source in each loop.
2413 The loop will be exited when there's not enough source code
2414 (within macro ONE_MORE_BYTE), or when there's not enough
2415 destination area to produce a character (within macro
2417 unsigned char *src_base
;
2418 Lisp_Object translation_table
;
2420 if (NILP (Venable_character_translation
))
2421 translation_table
= Qnil
;
2424 translation_table
= coding
->translation_table_for_decode
;
2425 if (NILP (translation_table
))
2426 translation_table
= Vstandard_translation_table_for_decode
;
2429 coding
->produced_char
= 0;
2432 int c
, charset
, c1
, c2
;
2439 charset
= CHARSET_ASCII
;
2444 if (coding
->eol_type
== CODING_EOL_CRLF
)
2449 else if (coding
->mode
2450 & CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2452 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2453 goto label_end_of_loop
;
2456 /* To process C2 again, SRC is subtracted by 1. */
2459 else if (coding
->eol_type
== CODING_EOL_CR
)
2463 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2464 && (coding
->eol_type
== CODING_EOL_CR
2465 || coding
->eol_type
== CODING_EOL_CRLF
))
2467 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2468 goto label_end_of_loop
;
2477 goto label_invalid_code
;
2478 if (c1
< 0xA0 || c1
>= 0xE0)
2480 /* SJIS -> JISX0208 */
2482 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
2483 goto label_invalid_code
;
2484 DECODE_SJIS (c1
, c2
, c1
, c2
);
2485 charset
= charset_jisx0208
;
2488 /* SJIS -> JISX0201-Kana */
2489 charset
= charset_katakana_jisx0201
;
2494 if (c1
< 0xA1 || c1
> 0xFE)
2495 goto label_invalid_code
;
2497 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
2498 goto label_invalid_code
;
2499 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
2503 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2515 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2516 coding
->produced
= dst
- destination
;
2520 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
2521 This function can encode charsets `ascii', `katakana-jisx0201',
2522 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
2523 are sure that all these charsets are registered as official charset
2524 (i.e. do not have extended leading-codes). Characters of other
2525 charsets are produced without any encoding. If SJIS_P is 1, encode
2526 SJIS text, else encode BIG5 text. */
2529 encode_coding_sjis_big5 (coding
, source
, destination
,
2530 src_bytes
, dst_bytes
, sjis_p
)
2531 struct coding_system
*coding
;
2532 unsigned char *source
, *destination
;
2533 int src_bytes
, dst_bytes
;
2536 unsigned char *src
= source
;
2537 unsigned char *src_end
= source
+ src_bytes
;
2538 unsigned char *dst
= destination
;
2539 unsigned char *dst_end
= destination
+ dst_bytes
;
2540 /* SRC_BASE remembers the start position in source in each loop.
2541 The loop will be exited when there's not enough source text to
2542 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2543 there's not enough destination area to produce encoded codes
2544 (within macro EMIT_BYTES). */
2545 unsigned char *src_base
;
2546 Lisp_Object translation_table
;
2548 if (NILP (Venable_character_translation
))
2549 translation_table
= Qnil
;
2552 translation_table
= coding
->translation_table_for_decode
;
2553 if (NILP (translation_table
))
2554 translation_table
= Vstandard_translation_table_for_decode
;
2559 int c
, charset
, c1
, c2
;
2564 /* Now encode the character C. */
2565 if (SINGLE_BYTE_CHAR_P (c
))
2570 if (!coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
)
2577 if (coding
->eol_type
== CODING_EOL_CRLF
)
2579 EMIT_TWO_BYTES ('\r', c
);
2582 else if (coding
->eol_type
== CODING_EOL_CR
)
2590 SPLIT_CHAR (c
, charset
, c1
, c2
);
2593 if (charset
== charset_jisx0208
2594 || charset
== charset_jisx0208_1978
)
2596 ENCODE_SJIS (c1
, c2
, c1
, c2
);
2597 EMIT_TWO_BYTES (c1
, c2
);
2599 else if (charset
== charset_latin_jisx0201
)
2602 /* There's no way other than producing the internal
2604 EMIT_BYTES (src_base
, src
);
2608 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
2610 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
2611 EMIT_TWO_BYTES (c1
, c2
);
2614 /* There's no way other than producing the internal
2616 EMIT_BYTES (src_base
, src
);
2619 coding
->consumed_char
++;
2623 coding
->consumed
= src_base
- source
;
2624 coding
->produced
= coding
->produced_char
= dst
- destination
;
2628 /*** 5. CCL handlers ***/
2630 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2631 Check if a text is encoded in a coding system of which
2632 encoder/decoder are written in CCL program. If it is, return
2633 CODING_CATEGORY_MASK_CCL, else return 0. */
2636 detect_coding_ccl (src
, src_end
)
2637 unsigned char *src
, *src_end
;
2639 unsigned char *valid
;
2641 /* Dummy for ONE_MORE_BYTE. */
2642 struct coding_system dummy_coding
;
2643 struct coding_system
*coding
= &dummy_coding
;
2645 /* No coding system is assigned to coding-category-ccl. */
2646 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
2649 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
2657 return CODING_CATEGORY_MASK_CCL
;
2661 /*** 6. End-of-line handlers ***/
2663 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
2666 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
2667 struct coding_system
*coding
;
2668 unsigned char *source
, *destination
;
2669 int src_bytes
, dst_bytes
;
2671 unsigned char *src
= source
;
2672 unsigned char *dst
= destination
;
2673 unsigned char *src_end
= src
+ src_bytes
;
2674 unsigned char *dst_end
= dst
+ dst_bytes
;
2675 Lisp_Object translation_table
;
2676 /* SRC_BASE remembers the start position in source in each loop.
2677 The loop will be exited when there's not enough source code
2678 (within macro ONE_MORE_BYTE), or when there's not enough
2679 destination area to produce a character (within macro
2681 unsigned char *src_base
;
2684 translation_table
= Qnil
;
2685 switch (coding
->eol_type
)
2687 case CODING_EOL_CRLF
:
2697 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2699 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2700 goto label_end_of_loop
;
2707 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
2709 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2710 goto label_end_of_loop
;
2723 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2725 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2726 goto label_end_of_loop
;
2735 default: /* no need for EOL handling */
2745 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2746 coding
->produced
= dst
- destination
;
2750 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
2751 format of end-of-line according to `coding->eol_type'. It also
2752 convert multibyte form 8-bit characers to unibyte if
2753 CODING->src_multibyte is nonzero. If `coding->mode &
2754 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
2755 also means end-of-line. */
2758 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
2759 struct coding_system
*coding
;
2760 unsigned char *source
, *destination
;
2761 int src_bytes
, dst_bytes
;
2763 unsigned char *src
= source
;
2764 unsigned char *dst
= destination
;
2765 unsigned char *src_end
= src
+ src_bytes
;
2766 unsigned char *dst_end
= dst
+ dst_bytes
;
2767 Lisp_Object translation_table
;
2768 /* SRC_BASE remembers the start position in source in each loop.
2769 The loop will be exited when there's not enough source text to
2770 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2771 there's not enough destination area to produce encoded codes
2772 (within macro EMIT_BYTES). */
2773 unsigned char *src_base
;
2775 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
2777 translation_table
= Qnil
;
2778 if (coding
->src_multibyte
2779 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
2783 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
2786 if (coding
->eol_type
== CODING_EOL_CRLF
)
2788 while (src
< src_end
)
2794 else if (c
== '\n' || (c
== '\r' && selective_display
))
2795 EMIT_TWO_BYTES ('\r', '\n');
2805 if (src_bytes
<= dst_bytes
)
2807 safe_bcopy (src
, dst
, src_bytes
);
2813 if (coding
->src_multibyte
2814 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
2816 safe_bcopy (src
, dst
, dst_bytes
);
2817 src_base
= src
+ dst_bytes
;
2818 dst
= destination
+ dst_bytes
;
2819 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2821 if (coding
->eol_type
== CODING_EOL_CR
)
2823 for (src
= destination
; src
< dst
; src
++)
2824 if (*src
== '\n') *src
= '\r';
2826 else if (selective_display
)
2828 for (src
= destination
; src
< dst
; src
++)
2829 if (*src
== '\r') *src
= '\n';
2832 if (coding
->src_multibyte
)
2833 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
2835 coding
->consumed
= src_base
- source
;
2836 coding
->produced
= dst
- destination
;
2840 /*** 7. C library functions ***/
2842 /* In Emacs Lisp, coding system is represented by a Lisp symbol which
2843 has a property `coding-system'. The value of this property is a
2844 vector of length 5 (called as coding-vector). Among elements of
2845 this vector, the first (element[0]) and the fifth (element[4])
2846 carry important information for decoding/encoding. Before
2847 decoding/encoding, this information should be set in fields of a
2848 structure of type `coding_system'.
2850 A value of property `coding-system' can be a symbol of another
2851 subsidiary coding-system. In that case, Emacs gets coding-vector
2854 `element[0]' contains information to be set in `coding->type'. The
2855 value and its meaning is as follows:
2857 0 -- coding_type_emacs_mule
2858 1 -- coding_type_sjis
2859 2 -- coding_type_iso2022
2860 3 -- coding_type_big5
2861 4 -- coding_type_ccl encoder/decoder written in CCL
2862 nil -- coding_type_no_conversion
2863 t -- coding_type_undecided (automatic conversion on decoding,
2864 no-conversion on encoding)
2866 `element[4]' contains information to be set in `coding->flags' and
2867 `coding->spec'. The meaning varies by `coding->type'.
2869 If `coding->type' is `coding_type_iso2022', element[4] is a vector
2870 of length 32 (of which the first 13 sub-elements are used now).
2871 Meanings of these sub-elements are:
2873 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
2874 If the value is an integer of valid charset, the charset is
2875 assumed to be designated to graphic register N initially.
2877 If the value is minus, it is a minus value of charset which
2878 reserves graphic register N, which means that the charset is
2879 not designated initially but should be designated to graphic
2880 register N just before encoding a character in that charset.
2882 If the value is nil, graphic register N is never used on
2885 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
2886 Each value takes t or nil. See the section ISO2022 of
2887 `coding.h' for more information.
2889 If `coding->type' is `coding_type_big5', element[4] is t to denote
2890 BIG5-ETen or nil to denote BIG5-HKU.
2892 If `coding->type' takes the other value, element[4] is ignored.
2894 Emacs Lisp's coding system also carries information about format of
2895 end-of-line in a value of property `eol-type'. If the value is
2896 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
2897 means CODING_EOL_CR. If it is not integer, it should be a vector
2898 of subsidiary coding systems of which property `eol-type' has one
2903 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
2904 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
2905 is setup so that no conversion is necessary and return -1, else
2909 setup_coding_system (coding_system
, coding
)
2910 Lisp_Object coding_system
;
2911 struct coding_system
*coding
;
2913 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
2917 /* Initialize some fields required for all kinds of coding systems. */
2918 coding
->symbol
= coding_system
;
2919 coding
->common_flags
= 0;
2921 coding
->heading_ascii
= -1;
2922 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
2923 coding
->composing
= COMPOSITION_DISABLED
;
2924 coding
->cmp_data
= NULL
;
2926 if (NILP (coding_system
))
2927 goto label_invalid_coding_system
;
2929 coding_spec
= Fget (coding_system
, Qcoding_system
);
2931 if (!VECTORP (coding_spec
)
2932 || XVECTOR (coding_spec
)->size
!= 5
2933 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
2934 goto label_invalid_coding_system
;
2936 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
2937 if (VECTORP (eol_type
))
2939 coding
->eol_type
= CODING_EOL_UNDECIDED
;
2940 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
2942 else if (XFASTINT (eol_type
) == 1)
2944 coding
->eol_type
= CODING_EOL_CRLF
;
2945 coding
->common_flags
2946 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
2948 else if (XFASTINT (eol_type
) == 2)
2950 coding
->eol_type
= CODING_EOL_CR
;
2951 coding
->common_flags
2952 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
2955 coding
->eol_type
= CODING_EOL_LF
;
2957 coding_type
= XVECTOR (coding_spec
)->contents
[0];
2958 /* Try short cut. */
2959 if (SYMBOLP (coding_type
))
2961 if (EQ (coding_type
, Qt
))
2963 coding
->type
= coding_type_undecided
;
2964 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
2967 coding
->type
= coding_type_no_conversion
;
2971 /* Get values of coding system properties:
2972 `post-read-conversion', `pre-write-conversion',
2973 `translation-table-for-decode', `translation-table-for-encode'. */
2974 plist
= XVECTOR (coding_spec
)->contents
[3];
2975 /* Pre & post conversion functions should be disabled if
2976 inhibit_eol_conversion is nozero. This is the case that a code
2977 conversion function is called while those functions are running. */
2978 if (! inhibit_pre_post_conversion
)
2980 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
2981 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
2983 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
2985 val
= Fget (val
, Qtranslation_table_for_decode
);
2986 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
2987 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
2989 val
= Fget (val
, Qtranslation_table_for_encode
);
2990 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
2991 val
= Fplist_get (plist
, Qcoding_category
);
2994 val
= Fget (val
, Qcoding_category_index
);
2996 coding
->category_idx
= XINT (val
);
2998 goto label_invalid_coding_system
;
3001 goto label_invalid_coding_system
;
3003 /* If the coding system has non-nil `composition' property, enable
3004 composition handling. */
3005 val
= Fplist_get (plist
, Qcomposition
);
3007 coding
->composing
= COMPOSITION_NO
;
3009 switch (XFASTINT (coding_type
))
3012 coding
->type
= coding_type_emacs_mule
;
3013 if (!NILP (coding
->post_read_conversion
))
3014 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3015 if (!NILP (coding
->pre_write_conversion
))
3016 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3020 coding
->type
= coding_type_sjis
;
3021 coding
->common_flags
3022 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3026 coding
->type
= coding_type_iso2022
;
3027 coding
->common_flags
3028 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3030 Lisp_Object val
, temp
;
3032 int i
, charset
, reg_bits
= 0;
3034 val
= XVECTOR (coding_spec
)->contents
[4];
3036 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3037 goto label_invalid_coding_system
;
3039 flags
= XVECTOR (val
)->contents
;
3041 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3042 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3043 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3044 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3045 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3046 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3047 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3048 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3049 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3050 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3051 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3052 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3053 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3056 /* Invoke graphic register 0 to plane 0. */
3057 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3058 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3059 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3060 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3061 /* Not single shifting at first. */
3062 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3063 /* Beginning of buffer should also be regarded as bol. */
3064 CODING_SPEC_ISO_BOL (coding
) = 1;
3066 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3067 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3068 val
= Vcharset_revision_alist
;
3071 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3073 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3074 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3075 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3079 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3080 FLAGS[REG] can be one of below:
3081 integer CHARSET: CHARSET occupies register I,
3082 t: designate nothing to REG initially, but can be used
3084 list of integer, nil, or t: designate the first
3085 element (if integer) to REG initially, the remaining
3086 elements (if integer) is designated to REG on request,
3087 if an element is t, REG can be used by any charsets,
3088 nil: REG is never used. */
3089 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3090 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3091 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3092 for (i
= 0; i
< 4; i
++)
3094 if (INTEGERP (flags
[i
])
3095 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
))
3096 || (charset
= get_charset_id (flags
[i
])) >= 0)
3098 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3099 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3101 else if (EQ (flags
[i
], Qt
))
3103 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3105 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3107 else if (CONSP (flags
[i
]))
3112 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3113 if (INTEGERP (XCAR (tail
))
3114 && (charset
= XINT (XCAR (tail
)),
3115 CHARSET_VALID_P (charset
))
3116 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3118 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3119 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3122 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3124 while (CONSP (tail
))
3126 if (INTEGERP (XCAR (tail
))
3127 && (charset
= XINT (XCAR (tail
)),
3128 CHARSET_VALID_P (charset
))
3129 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3130 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3132 else if (EQ (XCAR (tail
), Qt
))
3138 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3140 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3141 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3144 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3146 /* REG 1 can be used only by locking shift in 7-bit env. */
3147 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3149 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3150 /* Without any shifting, only REG 0 and 1 can be used. */
3155 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3157 if (CHARSET_VALID_P (charset
)
3158 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3159 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3161 /* There exist some default graphic registers to be
3164 /* We had better avoid designating a charset of
3165 CHARS96 to REG 0 as far as possible. */
3166 if (CHARSET_CHARS (charset
) == 96)
3167 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3169 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3171 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3173 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3177 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3178 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3182 coding
->type
= coding_type_big5
;
3183 coding
->common_flags
3184 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3186 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3187 ? CODING_FLAG_BIG5_HKU
3188 : CODING_FLAG_BIG5_ETEN
);
3192 coding
->type
= coding_type_ccl
;
3193 coding
->common_flags
3194 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3196 val
= XVECTOR (coding_spec
)->contents
[4];
3198 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3200 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3202 goto label_invalid_coding_system
;
3204 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3205 val
= Fplist_get (plist
, Qvalid_codes
);
3210 for (; CONSP (val
); val
= XCDR (val
))
3214 && XINT (this) >= 0 && XINT (this) < 256)
3215 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3216 else if (CONSP (this)
3217 && INTEGERP (XCAR (this))
3218 && INTEGERP (XCDR (this)))
3220 int start
= XINT (XCAR (this));
3221 int end
= XINT (XCDR (this));
3223 if (start
>= 0 && start
<= end
&& end
< 256)
3224 while (start
<= end
)
3225 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3230 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3231 coding
->spec
.ccl
.cr_carryover
= 0;
3235 coding
->type
= coding_type_raw_text
;
3239 goto label_invalid_coding_system
;
3243 label_invalid_coding_system
:
3244 coding
->type
= coding_type_no_conversion
;
3245 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3246 coding
->common_flags
= 0;
3247 coding
->eol_type
= CODING_EOL_LF
;
3248 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3252 /* Free memory blocks allocated for storing composition information. */
3255 coding_free_composition_data (coding
)
3256 struct coding_system
*coding
;
3258 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3262 /* Memory blocks are chained. At first, rewind to the first, then,
3263 free blocks one by one. */
3264 while (cmp_data
->prev
)
3265 cmp_data
= cmp_data
->prev
;
3268 next
= cmp_data
->next
;
3272 coding
->cmp_data
= NULL
;
3275 /* Set `char_offset' member of all memory blocks pointed by
3276 coding->cmp_data to POS. */
3279 coding_adjust_composition_offset (coding
, pos
)
3280 struct coding_system
*coding
;
3283 struct composition_data
*cmp_data
;
3285 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3286 cmp_data
->char_offset
= pos
;
3289 /* Setup raw-text or one of its subsidiaries in the structure
3290 coding_system CODING according to the already setup value eol_type
3291 in CODING. CODING should be setup for some coding system in
3295 setup_raw_text_coding_system (coding
)
3296 struct coding_system
*coding
;
3298 if (coding
->type
!= coding_type_raw_text
)
3300 coding
->symbol
= Qraw_text
;
3301 coding
->type
= coding_type_raw_text
;
3302 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3304 Lisp_Object subsidiaries
;
3305 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3307 if (VECTORP (subsidiaries
)
3308 && XVECTOR (subsidiaries
)->size
== 3)
3310 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3312 setup_coding_system (coding
->symbol
, coding
);
3317 /* Emacs has a mechanism to automatically detect a coding system if it
3318 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3319 it's impossible to distinguish some coding systems accurately
3320 because they use the same range of codes. So, at first, coding
3321 systems are categorized into 7, those are:
3323 o coding-category-emacs-mule
3325 The category for a coding system which has the same code range
3326 as Emacs' internal format. Assigned the coding-system (Lisp
3327 symbol) `emacs-mule' by default.
3329 o coding-category-sjis
3331 The category for a coding system which has the same code range
3332 as SJIS. Assigned the coding-system (Lisp
3333 symbol) `japanese-shift-jis' by default.
3335 o coding-category-iso-7
3337 The category for a coding system which has the same code range
3338 as ISO2022 of 7-bit environment. This doesn't use any locking
3339 shift and single shift functions. This can encode/decode all
3340 charsets. Assigned the coding-system (Lisp symbol)
3341 `iso-2022-7bit' by default.
3343 o coding-category-iso-7-tight
3345 Same as coding-category-iso-7 except that this can
3346 encode/decode only the specified charsets.
3348 o coding-category-iso-8-1
3350 The category for a coding system which has the same code range
3351 as ISO2022 of 8-bit environment and graphic plane 1 used only
3352 for DIMENSION1 charset. This doesn't use any locking shift
3353 and single shift functions. Assigned the coding-system (Lisp
3354 symbol) `iso-latin-1' by default.
3356 o coding-category-iso-8-2
3358 The category for a coding system which has the same code range
3359 as ISO2022 of 8-bit environment and graphic plane 1 used only
3360 for DIMENSION2 charset. This doesn't use any locking shift
3361 and single shift functions. Assigned the coding-system (Lisp
3362 symbol) `japanese-iso-8bit' by default.
3364 o coding-category-iso-7-else
3366 The category for a coding system which has the same code range
3367 as ISO2022 of 7-bit environemnt but uses locking shift or
3368 single shift functions. Assigned the coding-system (Lisp
3369 symbol) `iso-2022-7bit-lock' by default.
3371 o coding-category-iso-8-else
3373 The category for a coding system which has the same code range
3374 as ISO2022 of 8-bit environemnt but uses locking shift or
3375 single shift functions. Assigned the coding-system (Lisp
3376 symbol) `iso-2022-8bit-ss2' by default.
3378 o coding-category-big5
3380 The category for a coding system which has the same code range
3381 as BIG5. Assigned the coding-system (Lisp symbol)
3382 `cn-big5' by default.
3384 o coding-category-utf-8
3386 The category for a coding system which has the same code range
3387 as UTF-8 (cf. RFC2279). Assigned the coding-system (Lisp
3388 symbol) `utf-8' by default.
3390 o coding-category-utf-16-be
3392 The category for a coding system in which a text has an
3393 Unicode signature (cf. Unicode Standard) in the order of BIG
3394 endian at the head. Assigned the coding-system (Lisp symbol)
3395 `utf-16-be' by default.
3397 o coding-category-utf-16-le
3399 The category for a coding system in which a text has an
3400 Unicode signature (cf. Unicode Standard) in the order of
3401 LITTLE endian at the head. Assigned the coding-system (Lisp
3402 symbol) `utf-16-le' by default.
3404 o coding-category-ccl
3406 The category for a coding system of which encoder/decoder is
3407 written in CCL programs. The default value is nil, i.e., no
3408 coding system is assigned.
3410 o coding-category-binary
3412 The category for a coding system not categorized in any of the
3413 above. Assigned the coding-system (Lisp symbol)
3414 `no-conversion' by default.
3416 Each of them is a Lisp symbol and the value is an actual
3417 `coding-system's (this is also a Lisp symbol) assigned by a user.
3418 What Emacs does actually is to detect a category of coding system.
3419 Then, it uses a `coding-system' assigned to it. If Emacs can't
3420 decide only one possible category, it selects a category of the
3421 highest priority. Priorities of categories are also specified by a
3422 user in a Lisp variable `coding-category-list'.
3427 int ascii_skip_code
[256];
3429 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
3430 If it detects possible coding systems, return an integer in which
3431 appropriate flag bits are set. Flag bits are defined by macros
3432 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
3433 it should point the table `coding_priorities'. In that case, only
3434 the flag bit for a coding system of the highest priority is set in
3437 How many ASCII characters are at the head is returned as *SKIP. */
3440 detect_coding_mask (source
, src_bytes
, priorities
, skip
)
3441 unsigned char *source
;
3442 int src_bytes
, *priorities
, *skip
;
3444 register unsigned char c
;
3445 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
3446 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
3449 /* At first, skip all ASCII characters and control characters except
3450 for three ISO2022 specific control characters. */
3451 ascii_skip_code
[ISO_CODE_SO
] = 0;
3452 ascii_skip_code
[ISO_CODE_SI
] = 0;
3453 ascii_skip_code
[ISO_CODE_ESC
] = 0;
3455 label_loop_detect_coding
:
3456 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
3457 *skip
= src
- source
;
3460 /* We found nothing other than ASCII. There's nothing to do. */
3464 /* The text seems to be encoded in some multilingual coding system.
3465 Now, try to find in which coding system the text is encoded. */
3468 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
3469 /* C is an ISO2022 specific control code of C0. */
3470 mask
= detect_coding_iso2022 (src
, src_end
);
3473 /* No valid ISO2022 code follows C. Try again. */
3475 if (c
== ISO_CODE_ESC
)
3476 ascii_skip_code
[ISO_CODE_ESC
] = 1;
3478 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
3479 goto label_loop_detect_coding
;
3483 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
3485 if (mask
& priorities
[i
])
3486 return priorities
[i
];
3488 return CODING_CATEGORY_MASK_RAW_TEXT
;
3497 /* C is the first byte of SJIS character code,
3498 or a leading-code of Emacs' internal format (emacs-mule),
3499 or the first byte of UTF-16. */
3500 try = (CODING_CATEGORY_MASK_SJIS
3501 | CODING_CATEGORY_MASK_EMACS_MULE
3502 | CODING_CATEGORY_MASK_UTF_16_BE
3503 | CODING_CATEGORY_MASK_UTF_16_LE
);
3505 /* Or, if C is a special latin extra code,
3506 or is an ISO2022 specific control code of C1 (SS2 or SS3),
3507 or is an ISO2022 control-sequence-introducer (CSI),
3508 we should also consider the possibility of ISO2022 codings. */
3509 if ((VECTORP (Vlatin_extra_code_table
)
3510 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
3511 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
3512 || (c
== ISO_CODE_CSI
3515 || ((*src
== '0' || *src
== '1' || *src
== '2')
3516 && src
+ 1 < src_end
3517 && src
[1] == ']')))))
3518 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
3519 | CODING_CATEGORY_MASK_ISO_8BIT
);
3522 /* C is a character of ISO2022 in graphic plane right,
3523 or a SJIS's 1-byte character code (i.e. JISX0201),
3524 or the first byte of BIG5's 2-byte code,
3525 or the first byte of UTF-8/16. */
3526 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
3527 | CODING_CATEGORY_MASK_ISO_8BIT
3528 | CODING_CATEGORY_MASK_SJIS
3529 | CODING_CATEGORY_MASK_BIG5
3530 | CODING_CATEGORY_MASK_UTF_8
3531 | CODING_CATEGORY_MASK_UTF_16_BE
3532 | CODING_CATEGORY_MASK_UTF_16_LE
);
3534 /* Or, we may have to consider the possibility of CCL. */
3535 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
3536 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
3537 ->spec
.ccl
.valid_codes
)[c
])
3538 try |= CODING_CATEGORY_MASK_CCL
;
3541 utf16_examined_p
= iso2022_examined_p
= 0;
3544 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
3546 if (!iso2022_examined_p
3547 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
3549 mask
|= detect_coding_iso2022 (src
, src_end
);
3550 iso2022_examined_p
= 1;
3552 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
3553 mask
|= detect_coding_sjis (src
, src_end
);
3554 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
3555 mask
|= detect_coding_utf_8 (src
, src_end
);
3556 else if (!utf16_examined_p
3557 && (priorities
[i
] & try &
3558 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
3560 mask
|= detect_coding_utf_16 (src
, src_end
);
3561 utf16_examined_p
= 1;
3563 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
3564 mask
|= detect_coding_big5 (src
, src_end
);
3565 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
3566 mask
|= detect_coding_emacs_mule (src
, src_end
);
3567 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
3568 mask
|= detect_coding_ccl (src
, src_end
);
3569 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
3570 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
3571 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
3572 mask
|= CODING_CATEGORY_MASK_BINARY
;
3573 if (mask
& priorities
[i
])
3574 return priorities
[i
];
3576 return CODING_CATEGORY_MASK_RAW_TEXT
;
3578 if (try & CODING_CATEGORY_MASK_ISO
)
3579 mask
|= detect_coding_iso2022 (src
, src_end
);
3580 if (try & CODING_CATEGORY_MASK_SJIS
)
3581 mask
|= detect_coding_sjis (src
, src_end
);
3582 if (try & CODING_CATEGORY_MASK_BIG5
)
3583 mask
|= detect_coding_big5 (src
, src_end
);
3584 if (try & CODING_CATEGORY_MASK_UTF_8
)
3585 mask
|= detect_coding_utf_8 (src
, src_end
);
3586 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
3587 mask
|= detect_coding_utf_16 (src
, src_end
);
3588 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
3589 mask
|= detect_coding_emacs_mule (src
, src_end
);
3590 if (try & CODING_CATEGORY_MASK_CCL
)
3591 mask
|= detect_coding_ccl (src
, src_end
);
3593 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
3596 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
3597 The information of the detected coding system is set in CODING. */
3600 detect_coding (coding
, src
, src_bytes
)
3601 struct coding_system
*coding
;
3609 val
= Vcoding_category_list
;
3610 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
);
3611 coding
->heading_ascii
= skip
;
3615 /* We found a single coding system of the highest priority in MASK. */
3617 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
3619 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
3621 val
= XSYMBOL (XVECTOR (Vcoding_category_table
)->contents
[idx
])->value
;
3623 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3627 tmp
= Fget (val
, Qeol_type
);
3629 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
3632 /* Setup this new coding system while preserving some slots. */
3634 int src_multibyte
= coding
->src_multibyte
;
3635 int dst_multibyte
= coding
->dst_multibyte
;
3637 setup_coding_system (val
, coding
);
3638 coding
->src_multibyte
= src_multibyte
;
3639 coding
->dst_multibyte
= dst_multibyte
;
3640 coding
->heading_ascii
= skip
;
3644 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
3645 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
3646 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
3648 How many non-eol characters are at the head is returned as *SKIP. */
3650 #define MAX_EOL_CHECK_COUNT 3
3653 detect_eol_type (source
, src_bytes
, skip
)
3654 unsigned char *source
;
3655 int src_bytes
, *skip
;
3657 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
3659 int total
= 0; /* How many end-of-lines are found so far. */
3660 int eol_type
= CODING_EOL_UNDECIDED
;
3665 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
3668 if (c
== '\n' || c
== '\r')
3671 *skip
= src
- 1 - source
;
3674 this_eol_type
= CODING_EOL_LF
;
3675 else if (src
>= src_end
|| *src
!= '\n')
3676 this_eol_type
= CODING_EOL_CR
;
3678 this_eol_type
= CODING_EOL_CRLF
, src
++;
3680 if (eol_type
== CODING_EOL_UNDECIDED
)
3681 /* This is the first end-of-line. */
3682 eol_type
= this_eol_type
;
3683 else if (eol_type
!= this_eol_type
)
3685 /* The found type is different from what found before. */
3686 eol_type
= CODING_EOL_INCONSISTENT
;
3693 *skip
= src_end
- source
;
3697 /* Like detect_eol_type, but detect EOL type in 2-octet
3698 big-endian/little-endian format for coding systems utf-16-be and
3702 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
3703 unsigned char *source
;
3704 int src_bytes
, *skip
;
3706 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
3707 unsigned int c1
, c2
;
3708 int total
= 0; /* How many end-of-lines are found so far. */
3709 int eol_type
= CODING_EOL_UNDECIDED
;
3720 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
3722 c1
= (src
[msb
] << 8) | (src
[lsb
]);
3725 if (c1
== '\n' || c1
== '\r')
3728 *skip
= src
- 2 - source
;
3732 this_eol_type
= CODING_EOL_LF
;
3736 if ((src
+ 1) >= src_end
)
3738 this_eol_type
= CODING_EOL_CR
;
3742 c2
= (src
[msb
] << 8) | (src
[lsb
]);
3744 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
3746 this_eol_type
= CODING_EOL_CR
;
3750 if (eol_type
== CODING_EOL_UNDECIDED
)
3751 /* This is the first end-of-line. */
3752 eol_type
= this_eol_type
;
3753 else if (eol_type
!= this_eol_type
)
3755 /* The found type is different from what found before. */
3756 eol_type
= CODING_EOL_INCONSISTENT
;
3763 *skip
= src_end
- source
;
3767 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
3768 is encoded. If it detects an appropriate format of end-of-line, it
3769 sets the information in *CODING. */
3772 detect_eol (coding
, src
, src_bytes
)
3773 struct coding_system
*coding
;
3781 switch (coding
->category_idx
)
3783 case CODING_CATEGORY_IDX_UTF_16_BE
:
3784 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
3786 case CODING_CATEGORY_IDX_UTF_16_LE
:
3787 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
3790 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
3794 if (coding
->heading_ascii
> skip
)
3795 coding
->heading_ascii
= skip
;
3797 skip
= coding
->heading_ascii
;
3799 if (eol_type
== CODING_EOL_UNDECIDED
)
3801 if (eol_type
== CODING_EOL_INCONSISTENT
)
3804 /* This code is suppressed until we find a better way to
3805 distinguish raw text file and binary file. */
3807 /* If we have already detected that the coding is raw-text, the
3808 coding should actually be no-conversion. */
3809 if (coding
->type
== coding_type_raw_text
)
3811 setup_coding_system (Qno_conversion
, coding
);
3814 /* Else, let's decode only text code anyway. */
3816 eol_type
= CODING_EOL_LF
;
3819 val
= Fget (coding
->symbol
, Qeol_type
);
3820 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
3822 int src_multibyte
= coding
->src_multibyte
;
3823 int dst_multibyte
= coding
->dst_multibyte
;
3825 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
3826 coding
->src_multibyte
= src_multibyte
;
3827 coding
->dst_multibyte
= dst_multibyte
;
3828 coding
->heading_ascii
= skip
;
3832 #define CONVERSION_BUFFER_EXTRA_ROOM 256
3834 #define DECODING_BUFFER_MAG(coding) \
3835 (coding->type == coding_type_iso2022 \
3837 : (coding->type == coding_type_ccl \
3838 ? coding->spec.ccl.decoder.buf_magnification \
3841 /* Return maximum size (bytes) of a buffer enough for decoding
3842 SRC_BYTES of text encoded in CODING. */
3845 decoding_buffer_size (coding
, src_bytes
)
3846 struct coding_system
*coding
;
3849 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
3850 + CONVERSION_BUFFER_EXTRA_ROOM
);
3853 /* Return maximum size (bytes) of a buffer enough for encoding
3854 SRC_BYTES of text to CODING. */
3857 encoding_buffer_size (coding
, src_bytes
)
3858 struct coding_system
*coding
;
3863 if (coding
->type
== coding_type_ccl
)
3864 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
3865 else if (CODING_REQUIRE_ENCODING (coding
))
3870 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
3873 #ifndef MINIMUM_CONVERSION_BUFFER_SIZE
3874 #define MINIMUM_CONVERSION_BUFFER_SIZE 1024
3877 char *conversion_buffer
;
3878 int conversion_buffer_size
;
3880 /* Return a pointer to a SIZE bytes of buffer to be used for encoding
3881 or decoding. Sufficient memory is allocated automatically. If we
3882 run out of memory, return NULL. */
3885 get_conversion_buffer (size
)
3888 if (size
> conversion_buffer_size
)
3891 int real_size
= conversion_buffer_size
* 2;
3893 while (real_size
< size
) real_size
*= 2;
3894 buf
= (char *) xmalloc (real_size
);
3895 xfree (conversion_buffer
);
3896 conversion_buffer
= buf
;
3897 conversion_buffer_size
= real_size
;
3899 return conversion_buffer
;
3903 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
3904 struct coding_system
*coding
;
3905 unsigned char *source
, *destination
;
3906 int src_bytes
, dst_bytes
, encodep
;
3908 struct ccl_program
*ccl
3909 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
3912 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
3914 ccl
->eol_type
= coding
->eol_type
;
3915 coding
->produced
= ccl_driver (ccl
, source
, destination
,
3916 src_bytes
, dst_bytes
, &(coding
->consumed
));
3918 coding
->produced_char
= coding
->produced
;
3922 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
3923 coding
->produced
= str_as_multibyte (destination
, bytes
,
3925 &(coding
->produced_char
));
3928 switch (ccl
->status
)
3930 case CCL_STAT_SUSPEND_BY_SRC
:
3931 result
= CODING_FINISH_INSUFFICIENT_SRC
;
3933 case CCL_STAT_SUSPEND_BY_DST
:
3934 result
= CODING_FINISH_INSUFFICIENT_DST
;
3937 case CCL_STAT_INVALID_CMD
:
3938 result
= CODING_FINISH_INTERRUPT
;
3941 result
= CODING_FINISH_NORMAL
;
3947 /* Decode EOL format of the text at PTR of BYTES length destructively
3948 according to CODING->eol_type. This is called after the CCL
3949 program produced a decoded text at PTR. If we do CRLF->LF
3950 conversion, update CODING->produced and CODING->produced_char. */
3953 decode_eol_post_ccl (coding
, ptr
, bytes
)
3954 struct coding_system
*coding
;
3958 Lisp_Object val
, saved_coding_symbol
;
3959 unsigned char *pend
= ptr
+ bytes
;
3962 /* Remember the current coding system symbol. We set it back when
3963 an inconsistent EOL is found so that `last-coding-system-used' is
3964 set to the coding system that doesn't specify EOL conversion. */
3965 saved_coding_symbol
= coding
->symbol
;
3967 coding
->spec
.ccl
.cr_carryover
= 0;
3968 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
3970 /* Here, to avoid the call of setup_coding_system, we directly
3971 call detect_eol_type. */
3972 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
3973 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
3974 coding
->eol_type
= CODING_EOL_LF
;
3975 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3977 val
= Fget (coding
->symbol
, Qeol_type
);
3978 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
3979 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
3981 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
3984 if (coding
->eol_type
== CODING_EOL_LF
3985 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
3987 /* We have nothing to do. */
3990 else if (coding
->eol_type
== CODING_EOL_CRLF
)
3992 unsigned char *pstart
= ptr
, *p
= ptr
;
3994 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
3995 && *(pend
- 1) == '\r')
3997 /* If the last character is CR, we can't handle it here
3998 because LF will be in the not-yet-decoded source text.
3999 Recorded that the CR is not yet processed. */
4000 coding
->spec
.ccl
.cr_carryover
= 1;
4002 coding
->produced_char
--;
4009 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4016 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4017 goto undo_eol_conversion
;
4021 else if (*ptr
== '\n'
4022 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4023 goto undo_eol_conversion
;
4028 undo_eol_conversion
:
4029 /* We have faced with inconsistent EOL format at PTR.
4030 Convert all LFs before PTR back to CRLFs. */
4031 for (p
--, ptr
--; p
>= pstart
; p
--)
4034 *ptr
-- = '\n', *ptr
-- = '\r';
4038 /* If carryover is recorded, cancel it because we don't
4039 convert CRLF anymore. */
4040 if (coding
->spec
.ccl
.cr_carryover
)
4042 coding
->spec
.ccl
.cr_carryover
= 0;
4044 coding
->produced_char
++;
4048 coding
->eol_type
= CODING_EOL_LF
;
4049 coding
->symbol
= saved_coding_symbol
;
4053 /* As each two-byte sequence CRLF was converted to LF, (PEND
4054 - P) is the number of deleted characters. */
4055 coding
->produced
-= pend
- p
;
4056 coding
->produced_char
-= pend
- p
;
4059 else /* i.e. coding->eol_type == CODING_EOL_CR */
4061 unsigned char *p
= ptr
;
4063 for (; ptr
< pend
; ptr
++)
4067 else if (*ptr
== '\n'
4068 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4070 for (; p
< ptr
; p
++)
4076 coding
->eol_type
= CODING_EOL_LF
;
4077 coding
->symbol
= saved_coding_symbol
;
4083 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4084 decoding, it may detect coding system and format of end-of-line if
4085 those are not yet decided. The source should be unibyte, the
4086 result is multibyte if CODING->dst_multibyte is nonzero, else
4090 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4091 struct coding_system
*coding
;
4092 unsigned char *source
, *destination
;
4093 int src_bytes
, dst_bytes
;
4095 if (coding
->type
== coding_type_undecided
)
4096 detect_coding (coding
, source
, src_bytes
);
4098 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4099 && coding
->type
!= coding_type_ccl
)
4100 detect_eol (coding
, source
, src_bytes
);
4102 coding
->produced
= coding
->produced_char
= 0;
4103 coding
->consumed
= coding
->consumed_char
= 0;
4105 coding
->result
= CODING_FINISH_NORMAL
;
4107 switch (coding
->type
)
4109 case coding_type_sjis
:
4110 decode_coding_sjis_big5 (coding
, source
, destination
,
4111 src_bytes
, dst_bytes
, 1);
4114 case coding_type_iso2022
:
4115 decode_coding_iso2022 (coding
, source
, destination
,
4116 src_bytes
, dst_bytes
);
4119 case coding_type_big5
:
4120 decode_coding_sjis_big5 (coding
, source
, destination
,
4121 src_bytes
, dst_bytes
, 0);
4124 case coding_type_emacs_mule
:
4125 decode_coding_emacs_mule (coding
, source
, destination
,
4126 src_bytes
, dst_bytes
);
4129 case coding_type_ccl
:
4130 if (coding
->spec
.ccl
.cr_carryover
)
4132 /* Set the CR which is not processed by the previous call of
4133 decode_eol_post_ccl in DESTINATION. */
4134 *destination
= '\r';
4136 coding
->produced_char
++;
4139 ccl_coding_driver (coding
, source
,
4140 destination
+ coding
->spec
.ccl
.cr_carryover
,
4141 src_bytes
, dst_bytes
, 0);
4142 if (coding
->eol_type
!= CODING_EOL_LF
)
4143 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4147 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4150 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4151 && coding
->consumed
== src_bytes
)
4152 coding
->result
= CODING_FINISH_NORMAL
;
4154 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4155 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4157 unsigned char *src
= source
+ coding
->consumed
;
4158 unsigned char *dst
= destination
+ coding
->produced
;
4160 src_bytes
-= coding
->consumed
;
4162 if (COMPOSING_P (coding
))
4163 DECODE_COMPOSITION_END ('1');
4167 dst
+= CHAR_STRING (c
, dst
);
4168 coding
->produced_char
++;
4170 coding
->consumed
= coding
->consumed_char
= src
- source
;
4171 coding
->produced
= dst
- destination
;
4174 if (!coding
->dst_multibyte
)
4176 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4177 coding
->produced_char
= coding
->produced
;
4180 return coding
->result
;
4183 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4184 multibyteness of the source is CODING->src_multibyte, the
4185 multibyteness of the result is always unibyte. */
4188 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4189 struct coding_system
*coding
;
4190 unsigned char *source
, *destination
;
4191 int src_bytes
, dst_bytes
;
4193 coding
->produced
= coding
->produced_char
= 0;
4194 coding
->consumed
= coding
->consumed_char
= 0;
4196 coding
->result
= CODING_FINISH_NORMAL
;
4198 switch (coding
->type
)
4200 case coding_type_sjis
:
4201 encode_coding_sjis_big5 (coding
, source
, destination
,
4202 src_bytes
, dst_bytes
, 1);
4205 case coding_type_iso2022
:
4206 encode_coding_iso2022 (coding
, source
, destination
,
4207 src_bytes
, dst_bytes
);
4210 case coding_type_big5
:
4211 encode_coding_sjis_big5 (coding
, source
, destination
,
4212 src_bytes
, dst_bytes
, 0);
4215 case coding_type_emacs_mule
:
4216 encode_coding_emacs_mule (coding
, source
, destination
,
4217 src_bytes
, dst_bytes
);
4220 case coding_type_ccl
:
4221 ccl_coding_driver (coding
, source
, destination
,
4222 src_bytes
, dst_bytes
, 1);
4226 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4229 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4230 && coding
->consumed
== src_bytes
)
4231 coding
->result
= CODING_FINISH_NORMAL
;
4233 if (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4235 unsigned char *src
= source
+ coding
->consumed
;
4236 unsigned char *src_end
= src
+ src_bytes
;
4237 unsigned char *dst
= destination
+ coding
->produced
;
4239 if (coding
->type
== coding_type_iso2022
)
4240 ENCODE_RESET_PLANE_AND_REGISTER
;
4241 if (COMPOSING_P (coding
))
4242 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
4243 if (coding
->consumed
< src_bytes
)
4245 int len
= src_bytes
- coding
->consumed
;
4247 BCOPY_SHORT (source
+ coding
->consumed
, dst
, len
);
4248 if (coding
->src_multibyte
)
4249 len
= str_as_unibyte (dst
, len
);
4251 coding
->consumed
= src_bytes
;
4253 coding
->produced
= coding
->produced_char
= dst
- destination
;
4256 return coding
->result
;
4259 /* Scan text in the region between *BEG and *END (byte positions),
4260 skip characters which we don't have to decode by coding system
4261 CODING at the head and tail, then set *BEG and *END to the region
4262 of the text we actually have to convert. The caller should move
4263 the gap out of the region in advance if the region is from a
4266 If STR is not NULL, *BEG and *END are indices into STR. */
4269 shrink_decoding_region (beg
, end
, coding
, str
)
4271 struct coding_system
*coding
;
4274 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
4276 Lisp_Object translation_table
;
4278 if (coding
->type
== coding_type_ccl
4279 || coding
->type
== coding_type_undecided
4280 || coding
->eol_type
!= CODING_EOL_LF
4281 || !NILP (coding
->post_read_conversion
)
4282 || coding
->composing
!= COMPOSITION_DISABLED
)
4284 /* We can't skip any data. */
4287 if (coding
->type
== coding_type_no_conversion
4288 || coding
->type
== coding_type_raw_text
4289 || coding
->type
== coding_type_emacs_mule
)
4291 /* We need no conversion, but don't have to skip any data here.
4292 Decoding routine handles them effectively anyway. */
4296 translation_table
= coding
->translation_table_for_decode
;
4297 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
4298 translation_table
= Vstandard_translation_table_for_decode
;
4299 if (CHAR_TABLE_P (translation_table
))
4302 for (i
= 0; i
< 128; i
++)
4303 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
4306 /* Some ASCII character should be translated. We give up
4311 if (coding
->heading_ascii
>= 0)
4312 /* Detection routine has already found how much we can skip at the
4314 *beg
+= coding
->heading_ascii
;
4318 begp_orig
= begp
= str
+ *beg
;
4319 endp_orig
= endp
= str
+ *end
;
4323 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
4324 endp_orig
= endp
= begp
+ *end
- *beg
;
4327 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
4328 || coding
->eol_type
== CODING_EOL_CRLF
);
4330 switch (coding
->type
)
4332 case coding_type_sjis
:
4333 case coding_type_big5
:
4334 /* We can skip all ASCII characters at the head. */
4335 if (coding
->heading_ascii
< 0)
4338 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
4340 while (begp
< endp
&& *begp
< 0x80) begp
++;
4342 /* We can skip all ASCII characters at the tail except for the
4343 second byte of SJIS or BIG5 code. */
4345 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
4347 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
4348 /* Do not consider LF as ascii if preceded by CR, since that
4349 confuses eol decoding. */
4350 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
4352 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
4356 case coding_type_iso2022
:
4357 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
4358 /* We can't skip any data. */
4360 if (coding
->heading_ascii
< 0)
4362 /* We can skip all ASCII characters at the head except for a
4363 few control codes. */
4364 while (begp
< endp
&& (c
= *begp
) < 0x80
4365 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
4366 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
4367 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
4370 switch (coding
->category_idx
)
4372 case CODING_CATEGORY_IDX_ISO_8_1
:
4373 case CODING_CATEGORY_IDX_ISO_8_2
:
4374 /* We can skip all ASCII characters at the tail. */
4376 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
4378 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
4379 /* Do not consider LF as ascii if preceded by CR, since that
4380 confuses eol decoding. */
4381 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
4385 case CODING_CATEGORY_IDX_ISO_7
:
4386 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
4388 /* We can skip all charactes at the tail except for 8-bit
4389 codes and ESC and the following 2-byte at the tail. */
4390 unsigned char *eight_bit
= NULL
;
4394 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
4396 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
4401 && (c
= endp
[-1]) != ISO_CODE_ESC
)
4403 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
4406 /* Do not consider LF as ascii if preceded by CR, since that
4407 confuses eol decoding. */
4408 if (begp
< endp
&& endp
< endp_orig
4409 && endp
[-1] == '\r' && endp
[0] == '\n')
4411 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
4413 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
4414 /* This is an ASCII designation sequence. We can
4415 surely skip the tail. But, if we have
4416 encountered an 8-bit code, skip only the codes
4418 endp
= eight_bit
? eight_bit
: endp
+ 2;
4420 /* Hmmm, we can't skip the tail. */
4432 *beg
+= begp
- begp_orig
;
4433 *end
+= endp
- endp_orig
;
4437 /* Like shrink_decoding_region but for encoding. */
4440 shrink_encoding_region (beg
, end
, coding
, str
)
4442 struct coding_system
*coding
;
4445 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
4447 Lisp_Object translation_table
;
4449 if (coding
->type
== coding_type_ccl
4450 || coding
->eol_type
== CODING_EOL_CRLF
4451 || coding
->eol_type
== CODING_EOL_CR
4452 || coding
->cmp_data
&& coding
->cmp_data
->used
> 0)
4454 /* We can't skip any data. */
4457 if (coding
->type
== coding_type_no_conversion
4458 || coding
->type
== coding_type_raw_text
4459 || coding
->type
== coding_type_emacs_mule
4460 || coding
->type
== coding_type_undecided
)
4462 /* We need no conversion, but don't have to skip any data here.
4463 Encoding routine handles them effectively anyway. */
4467 translation_table
= coding
->translation_table_for_encode
;
4468 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
4469 translation_table
= Vstandard_translation_table_for_encode
;
4470 if (CHAR_TABLE_P (translation_table
))
4473 for (i
= 0; i
< 128; i
++)
4474 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
4477 /* Some ASCII character should be tranlsated. We give up
4484 begp_orig
= begp
= str
+ *beg
;
4485 endp_orig
= endp
= str
+ *end
;
4489 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
4490 endp_orig
= endp
= begp
+ *end
- *beg
;
4493 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
4494 || coding
->eol_type
== CODING_EOL_CRLF
);
4496 /* Here, we don't have to check coding->pre_write_conversion because
4497 the caller is expected to have handled it already. */
4498 switch (coding
->type
)
4500 case coding_type_iso2022
:
4501 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
4502 /* We can't skip any data. */
4504 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
4506 unsigned char *bol
= begp
;
4507 while (begp
< endp
&& *begp
< 0x80)
4510 if (begp
[-1] == '\n')
4514 goto label_skip_tail
;
4518 case coding_type_sjis
:
4519 case coding_type_big5
:
4520 /* We can skip all ASCII characters at the head and tail. */
4522 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
4524 while (begp
< endp
&& *begp
< 0x80) begp
++;
4527 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
4529 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
4536 *beg
+= begp
- begp_orig
;
4537 *end
+= endp
- endp_orig
;
4541 /* As shrinking conversion region requires some overhead, we don't try
4542 shrinking if the length of conversion region is less than this
4544 static int shrink_conversion_region_threshhold
= 1024;
4546 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
4548 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
4550 if (encodep) shrink_encoding_region (beg, end, coding, str); \
4551 else shrink_decoding_region (beg, end, coding, str); \
4556 code_convert_region_unwind (dummy
)
4559 inhibit_pre_post_conversion
= 0;
4563 /* Store information about all compositions in the range FROM and TO
4564 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
4565 buffer or a string, defaults to the current buffer. */
4568 coding_save_composition (coding
, from
, to
, obj
)
4569 struct coding_system
*coding
;
4576 if (coding
->composing
== COMPOSITION_DISABLED
)
4578 if (!coding
->cmp_data
)
4579 coding_allocate_composition_data (coding
, from
);
4580 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
4584 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
4587 coding
->composing
= COMPOSITION_NO
;
4590 if (COMPOSITION_VALID_P (start
, end
, prop
))
4592 enum composition_method method
= COMPOSITION_METHOD (prop
);
4593 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
4594 >= COMPOSITION_DATA_SIZE
)
4595 coding_allocate_composition_data (coding
, from
);
4596 /* For relative composition, we remember start and end
4597 positions, for the other compositions, we also remember
4599 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
4600 if (method
!= COMPOSITION_RELATIVE
)
4602 /* We must store a*/
4603 Lisp_Object val
, ch
;
4605 val
= COMPOSITION_COMPONENTS (prop
);
4609 ch
= XCAR (val
), val
= XCDR (val
);
4610 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
4612 else if (VECTORP (val
) || STRINGP (val
))
4614 int len
= (VECTORP (val
)
4615 ? XVECTOR (val
)->size
: XSTRING (val
)->size
);
4617 for (i
= 0; i
< len
; i
++)
4620 ? Faref (val
, make_number (i
))
4621 : XVECTOR (val
)->contents
[i
]);
4622 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
4625 else /* INTEGERP (val) */
4626 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
4628 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
4633 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
4636 /* Make coding->cmp_data point to the first memory block. */
4637 while (coding
->cmp_data
->prev
)
4638 coding
->cmp_data
= coding
->cmp_data
->prev
;
4639 coding
->cmp_data_start
= 0;
4642 /* Reflect the saved information about compositions to OBJ.
4643 CODING->cmp_data points to a memory block for the informaiton. OBJ
4644 is a buffer or a string, defaults to the current buffer. */
4647 coding_restore_composition (coding
, obj
)
4648 struct coding_system
*coding
;
4651 struct composition_data
*cmp_data
= coding
->cmp_data
;
4656 while (cmp_data
->prev
)
4657 cmp_data
= cmp_data
->prev
;
4663 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
4664 i
+= cmp_data
->data
[i
])
4666 int *data
= cmp_data
->data
+ i
;
4667 enum composition_method method
= (enum composition_method
) data
[3];
4668 Lisp_Object components
;
4670 if (method
== COMPOSITION_RELATIVE
)
4674 int len
= data
[0] - 4, j
;
4675 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
4677 for (j
= 0; j
< len
; j
++)
4678 args
[j
] = make_number (data
[4 + j
]);
4679 components
= (method
== COMPOSITION_WITH_ALTCHARS
4680 ? Fstring (len
, args
) : Fvector (len
, args
));
4682 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
4684 cmp_data
= cmp_data
->next
;
4688 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
4689 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
4690 coding system CODING, and return the status code of code conversion
4691 (currently, this value has no meaning).
4693 How many characters (and bytes) are converted to how many
4694 characters (and bytes) are recorded in members of the structure
4697 If REPLACE is nonzero, we do various things as if the original text
4698 is deleted and a new text is inserted. See the comments in
4699 replace_range (insdel.c) to know what we are doing.
4701 If REPLACE is zero, it is assumed that the source text is unibyte.
4702 Otherwize, it is assumed that the source text is multibyte. */
4705 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
4706 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
4707 struct coding_system
*coding
;
4709 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
4710 int require
, inserted
, inserted_byte
;
4711 int head_skip
, tail_skip
, total_skip
= 0;
4712 Lisp_Object saved_coding_symbol
;
4714 unsigned char *src
, *dst
;
4715 Lisp_Object deletion
;
4716 int orig_point
= PT
, orig_len
= len
;
4718 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
4720 coding
->src_multibyte
= replace
&& multibyte_p
;
4721 coding
->dst_multibyte
= multibyte_p
;
4724 saved_coding_symbol
= Qnil
;
4726 if (from
< PT
&& PT
< to
)
4728 TEMP_SET_PT_BOTH (from
, from_byte
);
4734 int saved_from
= from
;
4735 int saved_inhibit_modification_hooks
;
4737 prepare_to_modify_buffer (from
, to
, &from
);
4738 if (saved_from
!= from
)
4741 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
4742 len_byte
= to_byte
- from_byte
;
4745 /* The code conversion routine can not preserve text properties
4746 for now. So, we must remove all text properties in the
4747 region. Here, we must suppress all modification hooks. */
4748 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
4749 inhibit_modification_hooks
= 1;
4750 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
4751 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
4754 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
4756 /* We must detect encoding of text and eol format. */
4758 if (from
< GPT
&& to
> GPT
)
4759 move_gap_both (from
, from_byte
);
4760 if (coding
->type
== coding_type_undecided
)
4762 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
4763 if (coding
->type
== coding_type_undecided
)
4764 /* It seems that the text contains only ASCII, but we
4765 should not left it undecided because the deeper
4766 decoding routine (decode_coding) tries to detect the
4767 encodings again in vain. */
4768 coding
->type
= coding_type_emacs_mule
;
4770 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4771 && coding
->type
!= coding_type_ccl
)
4773 saved_coding_symbol
= coding
->symbol
;
4774 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
4775 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4776 coding
->eol_type
= CODING_EOL_LF
;
4777 /* We had better recover the original eol format if we
4778 encounter an inconsitent eol format while decoding. */
4779 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4783 /* Now we convert the text. */
4785 /* For encoding, we must process pre-write-conversion in advance. */
4786 if (! inhibit_pre_post_conversion
4788 && SYMBOLP (coding
->pre_write_conversion
)
4789 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
4791 /* The function in pre-write-conversion may put a new text in a
4793 struct buffer
*prev
= current_buffer
;
4795 int count
= specpdl_ptr
- specpdl
;
4797 record_unwind_protect (code_convert_region_unwind
, Qnil
);
4798 /* We should not call any more pre-write/post-read-conversion
4799 functions while this pre-write-conversion is running. */
4800 inhibit_pre_post_conversion
= 1;
4801 call2 (coding
->pre_write_conversion
,
4802 make_number (from
), make_number (to
));
4803 inhibit_pre_post_conversion
= 0;
4804 /* Discard the unwind protect. */
4807 if (current_buffer
!= prev
)
4810 new = Fcurrent_buffer ();
4811 set_buffer_internal_1 (prev
);
4812 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
4813 TEMP_SET_PT_BOTH (from
, from_byte
);
4814 insert_from_buffer (XBUFFER (new), 1, len
, 0);
4816 if (orig_point
>= to
)
4817 orig_point
+= len
- orig_len
;
4818 else if (orig_point
> from
)
4822 from_byte
= CHAR_TO_BYTE (from
);
4823 to_byte
= CHAR_TO_BYTE (to
);
4824 len_byte
= to_byte
- from_byte
;
4825 TEMP_SET_PT_BOTH (from
, from_byte
);
4830 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
4832 if (coding
->composing
!= COMPOSITION_DISABLED
)
4835 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
4837 coding_allocate_composition_data (coding
, from
);
4840 /* Try to skip the heading and tailing ASCIIs. */
4841 if (coding
->type
!= coding_type_ccl
)
4843 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
4845 if (from
< GPT
&& GPT
< to
)
4846 move_gap_both (from
, from_byte
);
4847 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
4848 if (from_byte
== to_byte
4849 && (encodep
|| NILP (coding
->post_read_conversion
))
4850 && ! CODING_REQUIRE_FLUSHING (coding
))
4852 coding
->produced
= len_byte
;
4853 coding
->produced_char
= len
;
4855 /* We must record and adjust for this new text now. */
4856 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
4860 head_skip
= from_byte
- from_byte_orig
;
4861 tail_skip
= to_byte_orig
- to_byte
;
4862 total_skip
= head_skip
+ tail_skip
;
4865 len
-= total_skip
; len_byte
-= total_skip
;
4868 /* For converion, we must put the gap before the text in addition to
4869 making the gap larger for efficient decoding. The required gap
4870 size starts from 2000 which is the magic number used in make_gap.
4871 But, after one batch of conversion, it will be incremented if we
4872 find that it is not enough . */
4875 if (GAP_SIZE
< require
)
4876 make_gap (require
- GAP_SIZE
);
4877 move_gap_both (from
, from_byte
);
4879 inserted
= inserted_byte
= 0;
4881 GAP_SIZE
+= len_byte
;
4884 ZV_BYTE
-= len_byte
;
4887 if (GPT
- BEG
< BEG_UNCHANGED
)
4888 BEG_UNCHANGED
= GPT
- BEG
;
4889 if (Z
- GPT
< END_UNCHANGED
)
4890 END_UNCHANGED
= Z
- GPT
;
4892 if (!encodep
&& coding
->src_multibyte
)
4894 /* Decoding routines expects that the source text is unibyte.
4895 We must convert 8-bit characters of multibyte form to
4897 int len_byte_orig
= len_byte
;
4898 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
4899 if (len_byte
< len_byte_orig
)
4900 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
4902 coding
->src_multibyte
= 0;
4909 /* The buffer memory is now:
4910 +--------+converted-text+---------+-------original-text-------+---+
4911 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
4912 |<---------------------- GAP ----------------------->| */
4913 src
= GAP_END_ADDR
- len_byte
;
4914 dst
= GPT_ADDR
+ inserted_byte
;
4917 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
4919 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
4921 /* The buffer memory is now:
4922 +--------+-------converted-text----+--+------original-text----+---+
4923 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
4924 |<---------------------- GAP ----------------------->| */
4926 inserted
+= coding
->produced_char
;
4927 inserted_byte
+= coding
->produced
;
4928 len_byte
-= coding
->consumed
;
4930 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
4932 coding_allocate_composition_data (coding
, from
+ inserted
);
4936 src
+= coding
->consumed
;
4937 dst
+= coding
->produced
;
4939 if (result
== CODING_FINISH_NORMAL
)
4944 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
4946 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
4947 Lisp_Object eol_type
;
4949 /* Encode LFs back to the original eol format (CR or CRLF). */
4950 if (coding
->eol_type
== CODING_EOL_CR
)
4952 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
4958 while (p
< pend
) if (*p
++ == '\n') count
++;
4959 if (src
- dst
< count
)
4961 /* We don't have sufficient room for encoding LFs
4962 back to CRLF. We must record converted and
4963 not-yet-converted text back to the buffer
4964 content, enlarge the gap, then record them out of
4965 the buffer contents again. */
4966 int add
= len_byte
+ inserted_byte
;
4969 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
4970 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
4971 make_gap (count
- GAP_SIZE
);
4973 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
4974 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
4975 /* Don't forget to update SRC, DST, and PEND. */
4976 src
= GAP_END_ADDR
- len_byte
;
4977 dst
= GPT_ADDR
+ inserted_byte
;
4981 inserted_byte
+= count
;
4982 coding
->produced
+= count
;
4983 p
= dst
= pend
+ count
;
4987 if (*p
== '\n') count
--, *--p
= '\r';
4991 /* Suppress eol-format conversion in the further conversion. */
4992 coding
->eol_type
= CODING_EOL_LF
;
4994 /* Set the coding system symbol to that for Unix-like EOL. */
4995 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
4996 if (VECTORP (eol_type
)
4997 && XVECTOR (eol_type
)->size
== 3
4998 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
4999 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5001 coding
->symbol
= saved_coding_symbol
;
5007 if (coding
->type
!= coding_type_ccl
5008 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5010 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5013 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5015 /* The source text ends in invalid codes. Let's just
5016 make them valid buffer contents, and finish conversion. */
5017 inserted
+= len_byte
;
5018 inserted_byte
+= len_byte
;
5023 if (result
== CODING_FINISH_INTERRUPT
)
5025 /* The conversion procedure was interrupted by a user. */
5028 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5029 if (coding
->consumed
< 1)
5031 /* It's quite strange to require more memory without
5032 consuming any bytes. Perhaps CCL program bug. */
5037 /* We have just done the first batch of conversion which was
5038 stoped because of insufficient gap. Let's reconsider the
5039 required gap size (i.e. SRT - DST) now.
5041 We have converted ORIG bytes (== coding->consumed) into
5042 NEW bytes (coding->produced). To convert the remaining
5043 LEN bytes, we may need REQUIRE bytes of gap, where:
5044 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5045 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5046 Here, we are sure that NEW >= ORIG. */
5047 float ratio
= coding
->produced
- coding
->consumed
;
5048 ratio
/= coding
->consumed
;
5049 require
= len_byte
* ratio
;
5052 if ((src
- dst
) < (require
+ 2000))
5054 /* See the comment above the previous call of make_gap. */
5055 int add
= len_byte
+ inserted_byte
;
5058 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5059 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5060 make_gap (require
+ 2000);
5062 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5063 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5066 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5068 if (encodep
&& coding
->dst_multibyte
)
5070 /* The output is unibyte. We must convert 8-bit characters to
5072 if (inserted_byte
* 2 > GAP_SIZE
)
5074 GAP_SIZE
-= inserted_byte
;
5075 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5076 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5077 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5078 make_gap (inserted_byte
- GAP_SIZE
);
5079 GAP_SIZE
+= inserted_byte
;
5080 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5081 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5082 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5084 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5087 /* If we have shrinked the conversion area, adjust it now. */
5091 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5092 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5093 GAP_SIZE
+= total_skip
;
5094 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5095 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5096 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5097 from
-= head_skip
; from_byte
-= head_skip
;
5098 to
+= tail_skip
; to_byte
+= tail_skip
;
5102 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5103 inserted
= Z
- prev_Z
;
5105 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5106 coding_restore_composition (coding
, Fcurrent_buffer ());
5107 coding_free_composition_data (coding
);
5109 if (! inhibit_pre_post_conversion
5110 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5113 int count
= specpdl_ptr
- specpdl
;
5116 TEMP_SET_PT_BOTH (from
, from_byte
);
5118 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5119 /* We should not call any more pre-write/post-read-conversion
5120 functions while this post-read-conversion is running. */
5121 inhibit_pre_post_conversion
= 1;
5122 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
5123 inhibit_pre_post_conversion
= 0;
5124 /* Discard the unwind protect. */
5126 CHECK_NUMBER (val
, 0);
5127 inserted
+= Z
- prev_Z
;
5130 if (orig_point
>= from
)
5132 if (orig_point
>= from
+ orig_len
)
5133 orig_point
+= inserted
- orig_len
;
5136 TEMP_SET_PT (orig_point
);
5141 signal_after_change (from
, to
- from
, inserted
);
5142 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
5146 coding
->consumed
= to_byte
- from_byte
;
5147 coding
->consumed_char
= to
- from
;
5148 coding
->produced
= inserted_byte
;
5149 coding
->produced_char
= inserted
;
5156 run_pre_post_conversion_on_str (str
, coding
, encodep
)
5158 struct coding_system
*coding
;
5161 int count
= specpdl_ptr
- specpdl
;
5162 struct gcpro gcpro1
;
5163 struct buffer
*prev
= current_buffer
;
5164 int multibyte
= STRING_MULTIBYTE (str
);
5166 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
5167 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5169 temp_output_buffer_setup (" *code-converting-work*");
5170 set_buffer_internal (XBUFFER (Vstandard_output
));
5171 /* We must insert the contents of STR as is without
5172 unibyte<->multibyte conversion. For that, we adjust the
5173 multibyteness of the working buffer to that of STR. */
5175 current_buffer
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
5176 insert_from_string (str
, 0, 0,
5177 XSTRING (str
)->size
, STRING_BYTES (XSTRING (str
)), 0);
5179 inhibit_pre_post_conversion
= 1;
5181 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
5184 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
5185 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
5187 inhibit_pre_post_conversion
= 0;
5188 str
= make_buffer_string (BEG
, Z
, 1);
5189 return unbind_to (count
, str
);
5193 decode_coding_string (str
, coding
, nocopy
)
5195 struct coding_system
*coding
;
5200 int from
, to
, to_byte
;
5201 struct gcpro gcpro1
;
5202 Lisp_Object saved_coding_symbol
;
5204 int require_decoding
;
5207 to
= XSTRING (str
)->size
;
5208 to_byte
= STRING_BYTES (XSTRING (str
));
5210 saved_coding_symbol
= Qnil
;
5211 if (CODING_REQUIRE_DETECTION (coding
))
5213 /* See the comments in code_convert_region. */
5214 if (coding
->type
== coding_type_undecided
)
5216 detect_coding (coding
, XSTRING (str
)->data
, to_byte
);
5217 if (coding
->type
== coding_type_undecided
)
5218 coding
->type
= coding_type_emacs_mule
;
5220 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5221 && coding
->type
!= coding_type_ccl
)
5223 saved_coding_symbol
= coding
->symbol
;
5224 detect_eol (coding
, XSTRING (str
)->data
, to_byte
);
5225 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5226 coding
->eol_type
= CODING_EOL_LF
;
5227 /* We had better recover the original eol format if we
5228 encounter an inconsitent eol format while decoding. */
5229 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5233 require_decoding
= CODING_REQUIRE_DECODING (coding
);
5235 if (STRING_MULTIBYTE (str
))
5237 /* Decoding routines expect the source text to be unibyte. */
5238 str
= Fstring_as_unibyte (str
);
5239 to_byte
= STRING_BYTES (XSTRING (str
));
5242 coding
->src_multibyte
= 0;
5243 coding
->dst_multibyte
= (coding
->type
!= coding_type_no_conversion
5244 && coding
->type
!= coding_type_raw_text
);
5246 /* Try to skip the heading and tailing ASCIIs. */
5247 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
5249 int from_orig
= from
;
5251 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, XSTRING (str
)->data
,
5253 if (from
== to_byte
)
5254 require_decoding
= 0;
5257 if (!require_decoding
)
5259 coding
->consumed
= STRING_BYTES (XSTRING (str
));
5260 coding
->consumed_char
= XSTRING (str
)->size
;
5261 if (coding
->dst_multibyte
)
5263 str
= Fstring_as_multibyte (str
);
5266 coding
->produced
= STRING_BYTES (XSTRING (str
));
5267 coding
->produced_char
= XSTRING (str
)->size
;
5268 return (nocopy
? str
: Fcopy_sequence (str
));
5271 if (coding
->composing
!= COMPOSITION_DISABLED
)
5272 coding_allocate_composition_data (coding
, from
);
5274 len
= decoding_buffer_size (coding
, to_byte
- from
);
5275 len
+= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5277 buf
= get_conversion_buffer (len
);
5281 bcopy (XSTRING (str
)->data
, buf
, from
);
5282 result
= decode_coding (coding
, XSTRING (str
)->data
+ from
,
5283 buf
+ from
, to_byte
- from
, len
);
5284 if (result
== CODING_FINISH_INCONSISTENT_EOL
)
5286 /* We simply try to decode the whole string again but without
5287 eol-conversion this time. */
5288 coding
->eol_type
= CODING_EOL_LF
;
5289 coding
->symbol
= saved_coding_symbol
;
5290 coding_free_composition_data (coding
);
5291 return decode_coding_string (str
, coding
, nocopy
);
5294 bcopy (XSTRING (str
)->data
+ to_byte
, buf
+ from
+ coding
->produced
,
5295 STRING_BYTES (XSTRING (str
)) - to_byte
);
5297 len
= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5298 if (coding
->dst_multibyte
)
5299 str
= make_multibyte_string (buf
, len
+ coding
->produced_char
,
5300 len
+ coding
->produced
);
5302 str
= make_unibyte_string (buf
, len
+ coding
->produced
);
5304 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
5305 coding_restore_composition (coding
, str
);
5306 coding_free_composition_data (coding
);
5308 if (SYMBOLP (coding
->post_read_conversion
)
5309 && !NILP (Ffboundp (coding
->post_read_conversion
)))
5310 str
= run_pre_post_conversion_on_str (str
, coding
, 0);
5316 encode_coding_string (str
, coding
, nocopy
)
5318 struct coding_system
*coding
;
5323 int from
, to
, to_byte
;
5324 struct gcpro gcpro1
;
5325 Lisp_Object saved_coding_symbol
;
5328 if (SYMBOLP (coding
->pre_write_conversion
)
5329 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
5330 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
5333 to
= XSTRING (str
)->size
;
5334 to_byte
= STRING_BYTES (XSTRING (str
));
5336 saved_coding_symbol
= Qnil
;
5337 if (! CODING_REQUIRE_ENCODING (coding
))
5339 if (STRING_MULTIBYTE (str
))
5341 str
= Fstring_as_unibyte (str
);
5344 return (nocopy
? str
: Fcopy_sequence (str
));
5347 /* Encoding routines determine the multibyteness of the source text
5348 by coding->src_multibyte. */
5349 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
5350 coding
->dst_multibyte
= 0;
5352 if (coding
->composing
!= COMPOSITION_DISABLED
)
5353 coding_save_composition (coding
, from
, to
, str
);
5355 /* Try to skip the heading and tailing ASCIIs. */
5356 if (coding
->type
!= coding_type_ccl
)
5358 int from_orig
= from
;
5360 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, XSTRING (str
)->data
,
5362 if (from
== to_byte
)
5363 return (nocopy
? str
: Fcopy_sequence (str
));
5366 len
= encoding_buffer_size (coding
, to_byte
- from
);
5367 len
+= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5369 buf
= get_conversion_buffer (len
);
5373 bcopy (XSTRING (str
)->data
, buf
, from
);
5374 result
= encode_coding (coding
, XSTRING (str
)->data
+ from
,
5375 buf
+ from
, to_byte
- from
, len
);
5376 bcopy (XSTRING (str
)->data
+ to_byte
, buf
+ from
+ coding
->produced
,
5377 STRING_BYTES (XSTRING (str
)) - to_byte
);
5379 len
= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5380 str
= make_unibyte_string (buf
, len
+ coding
->produced
);
5381 coding_free_composition_data (coding
);
5388 /*** 8. Emacs Lisp library functions ***/
5390 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
5391 "Return t if OBJECT is nil or a coding-system.\n\
5392 See the documentation of `make-coding-system' for information\n\
5393 about coding-system objects.")
5401 /* Get coding-spec vector for OBJ. */
5402 obj
= Fget (obj
, Qcoding_system
);
5403 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
5407 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
5408 Sread_non_nil_coding_system
, 1, 1, 0,
5409 "Read a coding system from the minibuffer, prompting with string PROMPT.")
5416 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
5417 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
5419 while (XSTRING (val
)->size
== 0);
5420 return (Fintern (val
, Qnil
));
5423 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
5424 "Read a coding system from the minibuffer, prompting with string PROMPT.\n\
5425 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.")
5426 (prompt
, default_coding_system
)
5427 Lisp_Object prompt
, default_coding_system
;
5430 if (SYMBOLP (default_coding_system
))
5431 XSETSTRING (default_coding_system
, XSYMBOL (default_coding_system
)->name
);
5432 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
5433 Qt
, Qnil
, Qcoding_system_history
,
5434 default_coding_system
, Qnil
);
5435 return (XSTRING (val
)->size
== 0 ? Qnil
: Fintern (val
, Qnil
));
5438 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
5440 "Check validity of CODING-SYSTEM.\n\
5441 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.\n\
5442 It is valid if it is a symbol with a non-nil `coding-system' property.\n\
5443 The value of property should be a vector of length 5.")
5445 Lisp_Object coding_system
;
5447 CHECK_SYMBOL (coding_system
, 0);
5448 if (!NILP (Fcoding_system_p (coding_system
)))
5449 return coding_system
;
5451 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
5455 detect_coding_system (src
, src_bytes
, highest
)
5457 int src_bytes
, highest
;
5459 int coding_mask
, eol_type
;
5460 Lisp_Object val
, tmp
;
5463 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
);
5464 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
5465 if (eol_type
== CODING_EOL_INCONSISTENT
)
5466 eol_type
= CODING_EOL_UNDECIDED
;
5471 if (eol_type
!= CODING_EOL_UNDECIDED
)
5474 val2
= Fget (Qundecided
, Qeol_type
);
5476 val
= XVECTOR (val2
)->contents
[eol_type
];
5478 return (highest
? val
: Fcons (val
, Qnil
));
5481 /* At first, gather possible coding systems in VAL. */
5483 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
5485 Lisp_Object category_val
, category_index
;
5487 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
5488 category_val
= Fsymbol_value (XCAR (tmp
));
5489 if (!NILP (category_val
)
5490 && NATNUMP (category_index
)
5491 && (coding_mask
& (1 << XFASTINT (category_index
))))
5493 val
= Fcons (category_val
, val
);
5499 val
= Fnreverse (val
);
5501 /* Then, replace the elements with subsidiary coding systems. */
5502 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
5504 if (eol_type
!= CODING_EOL_UNDECIDED
5505 && eol_type
!= CODING_EOL_INCONSISTENT
)
5508 eol
= Fget (XCAR (tmp
), Qeol_type
);
5510 XCAR (tmp
) = XVECTOR (eol
)->contents
[eol_type
];
5513 return (highest
? XCAR (val
) : val
);
5516 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
5518 "Detect coding system of the text in the region between START and END.\n\
5519 Return a list of possible coding systems ordered by priority.\n\
5521 If only ASCII characters are found, it returns a list of single element\n\
5522 `undecided' or its subsidiary coding system according to a detected\n\
5523 end-of-line format.\n\
5525 If optional argument HIGHEST is non-nil, return the coding system of\n\
5527 (start
, end
, highest
)
5528 Lisp_Object start
, end
, highest
;
5531 int from_byte
, to_byte
;
5533 CHECK_NUMBER_COERCE_MARKER (start
, 0);
5534 CHECK_NUMBER_COERCE_MARKER (end
, 1);
5536 validate_region (&start
, &end
);
5537 from
= XINT (start
), to
= XINT (end
);
5538 from_byte
= CHAR_TO_BYTE (from
);
5539 to_byte
= CHAR_TO_BYTE (to
);
5541 if (from
< GPT
&& to
>= GPT
)
5542 move_gap_both (to
, to_byte
);
5544 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
5545 to_byte
- from_byte
,
5549 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
5551 "Detect coding system of the text in STRING.\n\
5552 Return a list of possible coding systems ordered by priority.\n\
5554 If only ASCII characters are found, it returns a list of single element\n\
5555 `undecided' or its subsidiary coding system according to a detected\n\
5556 end-of-line format.\n\
5558 If optional argument HIGHEST is non-nil, return the coding system of\n\
5561 Lisp_Object string
, highest
;
5563 CHECK_STRING (string
, 0);
5565 return detect_coding_system (XSTRING (string
)->data
,
5566 STRING_BYTES (XSTRING (string
)),
5570 /* Return an intersection of lists L1 and L2. */
5573 intersection (l1
, l2
)
5578 for (val
= Qnil
; CONSP (l1
); l1
= XCDR (l1
))
5580 if (!NILP (Fmemq (XCAR (l1
), l2
)))
5581 val
= Fcons (XCAR (l1
), val
);
5587 /* Subroutine for Fsafe_coding_systems_region_internal.
5589 Return a list of coding systems that safely encode the multibyte
5590 text between P and PEND. SAFE_CODINGS, if non-nil, is a list of
5591 possible coding systems. If it is nil, it means that we have not
5592 yet found any coding systems.
5594 WORK_TABLE is a copy of the char-table Vchar_coding_system_table. An
5595 element of WORK_TABLE is set to t once the element is looked up.
5597 If a non-ASCII single byte char is found, set
5598 *single_byte_char_found to 1. */
5601 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
5602 unsigned char *p
, *pend
;
5603 Lisp_Object safe_codings
, work_table
;
5604 int *single_byte_char_found
;
5611 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
5613 if (ASCII_BYTE_P (c
))
5614 /* We can ignore ASCII characters here. */
5616 if (SINGLE_BYTE_CHAR_P (c
))
5617 *single_byte_char_found
= 1;
5618 if (NILP (safe_codings
))
5620 /* Check the safe coding systems for C. */
5621 val
= char_table_ref_and_index (work_table
, c
, &idx
);
5623 /* This element was already checked. Ignore it. */
5625 /* Remember that we checked this element. */
5626 CHAR_TABLE_SET (work_table
, make_number (idx
), Qt
);
5628 /* If there are some safe coding systems for C and we have
5629 already found the other set of coding systems for the
5630 different characters, get the intersection of them. */
5631 if (!EQ (safe_codings
, Qt
) && !NILP (val
))
5632 val
= intersection (safe_codings
, val
);
5635 return safe_codings
;
5639 /* Return a list of coding systems that safely encode the text between
5640 START and END. If the text contains only ASCII or is unibyte,
5643 DEFUN ("find-coding-systems-region-internal",
5644 Ffind_coding_systems_region_internal
,
5645 Sfind_coding_systems_region_internal
, 2, 2, 0,
5646 "Internal use only.")
5648 Lisp_Object start
, end
;
5650 Lisp_Object work_table
, safe_codings
;
5651 int non_ascii_p
= 0;
5652 int single_byte_char_found
= 0;
5653 unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
5654 Lisp_Object args
[2];
5656 if (STRINGP (start
))
5658 if (!STRING_MULTIBYTE (start
))
5660 p1
= XSTRING (start
)->data
, p1end
= p1
+ STRING_BYTES (XSTRING (start
));
5662 if (XSTRING (start
)->size
!= STRING_BYTES (XSTRING (start
)))
5669 CHECK_NUMBER_COERCE_MARKER (start
, 0);
5670 CHECK_NUMBER_COERCE_MARKER (end
, 1);
5671 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
5672 args_out_of_range (start
, end
);
5673 if (NILP (current_buffer
->enable_multibyte_characters
))
5675 from
= CHAR_TO_BYTE (XINT (start
));
5676 to
= CHAR_TO_BYTE (XINT (end
));
5677 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
5678 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
5682 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
5683 if (XINT (end
) - XINT (start
) != to
- from
)
5689 /* We are sure that the text contains no multibyte character.
5690 Check if it contains eight-bit-graphic. */
5692 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
5695 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
5701 /* The text contains non-ASCII characters. */
5702 work_table
= Fcopy_sequence (Vchar_coding_system_table
);
5703 safe_codings
= find_safe_codings (p1
, p1end
, Qt
, work_table
,
5704 &single_byte_char_found
);
5706 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
5707 &single_byte_char_found
);
5709 if (!single_byte_char_found
)
5711 /* Append generic coding systems. */
5712 Lisp_Object args
[2];
5713 args
[0] = safe_codings
;
5714 args
[1] = Fchar_table_extra_slot (Vchar_coding_system_table
,
5716 safe_codings
= Fappend (2, args
);
5719 safe_codings
= Fcons (Qraw_text
, Fcons (Qemacs_mule
, safe_codings
));
5720 return safe_codings
;
5725 code_convert_region1 (start
, end
, coding_system
, encodep
)
5726 Lisp_Object start
, end
, coding_system
;
5729 struct coding_system coding
;
5732 CHECK_NUMBER_COERCE_MARKER (start
, 0);
5733 CHECK_NUMBER_COERCE_MARKER (end
, 1);
5734 CHECK_SYMBOL (coding_system
, 2);
5736 validate_region (&start
, &end
);
5737 from
= XFASTINT (start
);
5738 to
= XFASTINT (end
);
5740 if (NILP (coding_system
))
5741 return make_number (to
- from
);
5743 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
5744 error ("Invalid coding system: %s", XSYMBOL (coding_system
)->name
->data
);
5746 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
5747 coding
.src_multibyte
= coding
.dst_multibyte
5748 = !NILP (current_buffer
->enable_multibyte_characters
);
5749 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
5750 &coding
, encodep
, 1);
5751 Vlast_coding_system_used
= coding
.symbol
;
5752 return make_number (coding
.produced_char
);
5755 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
5756 3, 3, "r\nzCoding system: ",
5757 "Decode the current region by specified coding system.\n\
5758 When called from a program, takes three arguments:\n\
5759 START, END, and CODING-SYSTEM. START and END are buffer positions.\n\
5760 This function sets `last-coding-system-used' to the precise coding system\n\
5761 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5762 not fully specified.)\n\
5763 It returns the length of the decoded text.")
5764 (start
, end
, coding_system
)
5765 Lisp_Object start
, end
, coding_system
;
5767 return code_convert_region1 (start
, end
, coding_system
, 0);
5770 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
5771 3, 3, "r\nzCoding system: ",
5772 "Encode the current region by specified coding system.\n\
5773 When called from a program, takes three arguments:\n\
5774 START, END, and CODING-SYSTEM. START and END are buffer positions.\n\
5775 This function sets `last-coding-system-used' to the precise coding system\n\
5776 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5777 not fully specified.)\n\
5778 It returns the length of the encoded text.")
5779 (start
, end
, coding_system
)
5780 Lisp_Object start
, end
, coding_system
;
5782 return code_convert_region1 (start
, end
, coding_system
, 1);
5786 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
5787 Lisp_Object string
, coding_system
, nocopy
;
5790 struct coding_system coding
;
5792 CHECK_STRING (string
, 0);
5793 CHECK_SYMBOL (coding_system
, 1);
5795 if (NILP (coding_system
))
5796 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
5798 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
5799 error ("Invalid coding system: %s", XSYMBOL (coding_system
)->name
->data
);
5801 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
5803 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
5804 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
5805 Vlast_coding_system_used
= coding
.symbol
;
5810 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
5812 "Decode STRING which is encoded in CODING-SYSTEM, and return the result.\n\
5813 Optional arg NOCOPY non-nil means it is ok to return STRING itself\n\
5814 if the decoding operation is trivial.\n\
5815 This function sets `last-coding-system-used' to the precise coding system\n\
5816 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5817 not fully specified.)")
5818 (string
, coding_system
, nocopy
)
5819 Lisp_Object string
, coding_system
, nocopy
;
5821 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
5824 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
5826 "Encode STRING to CODING-SYSTEM, and return the result.\n\
5827 Optional arg NOCOPY non-nil means it is ok to return STRING itself\n\
5828 if the encoding operation is trivial.\n\
5829 This function sets `last-coding-system-used' to the precise coding system\n\
5830 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5831 not fully specified.)")
5832 (string
, coding_system
, nocopy
)
5833 Lisp_Object string
, coding_system
, nocopy
;
5835 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
5838 /* Encode or decode STRING according to CODING_SYSTEM.
5839 Do not set Vlast_coding_system_used.
5841 This function is called only from macros DECODE_FILE and
5842 ENCODE_FILE, thus we ignore character composition. */
5845 code_convert_string_norecord (string
, coding_system
, encodep
)
5846 Lisp_Object string
, coding_system
;
5849 struct coding_system coding
;
5851 CHECK_STRING (string
, 0);
5852 CHECK_SYMBOL (coding_system
, 1);
5854 if (NILP (coding_system
))
5857 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
5858 error ("Invalid coding system: %s", XSYMBOL (coding_system
)->name
->data
);
5860 coding
.composing
= COMPOSITION_DISABLED
;
5861 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
5863 ? encode_coding_string (string
, &coding
, 1)
5864 : decode_coding_string (string
, &coding
, 1));
5867 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
5868 "Decode a Japanese character which has CODE in shift_jis encoding.\n\
5869 Return the corresponding character.")
5873 unsigned char c1
, c2
, s1
, s2
;
5876 CHECK_NUMBER (code
, 0);
5877 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
5881 XSETFASTINT (val
, s2
);
5882 else if (s2
>= 0xA0 || s2
<= 0xDF)
5883 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
5885 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
5889 if ((s1
< 0x80 || s1
> 0x9F && s1
< 0xE0 || s1
> 0xEF)
5890 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
5891 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
5892 DECODE_SJIS (s1
, s2
, c1
, c2
);
5893 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
5898 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
5899 "Encode a Japanese character CHAR to shift_jis encoding.\n\
5900 Return the corresponding code in SJIS.")
5904 int charset
, c1
, c2
, s1
, s2
;
5907 CHECK_NUMBER (ch
, 0);
5908 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
5909 if (charset
== CHARSET_ASCII
)
5913 else if (charset
== charset_jisx0208
5914 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
5916 ENCODE_SJIS (c1
, c2
, s1
, s2
);
5917 XSETFASTINT (val
, (s1
<< 8) | s2
);
5919 else if (charset
== charset_katakana_jisx0201
5920 && c1
> 0x20 && c2
< 0xE0)
5922 XSETFASTINT (val
, c1
| 0x80);
5925 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
5929 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
5930 "Decode a Big5 character which has CODE in BIG5 coding system.\n\
5931 Return the corresponding character.")
5936 unsigned char b1
, b2
, c1
, c2
;
5939 CHECK_NUMBER (code
, 0);
5940 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
5944 error ("Invalid BIG5 code: %x", XFASTINT (code
));
5949 if ((b1
< 0xA1 || b1
> 0xFE)
5950 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
5951 error ("Invalid BIG5 code: %x", XFASTINT (code
));
5952 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
5953 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
5958 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
5959 "Encode the Big5 character CHAR to BIG5 coding system.\n\
5960 Return the corresponding character code in Big5.")
5964 int charset
, c1
, c2
, b1
, b2
;
5967 CHECK_NUMBER (ch
, 0);
5968 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
5969 if (charset
== CHARSET_ASCII
)
5973 else if ((charset
== charset_big5_1
5974 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
5975 || (charset
== charset_big5_2
5976 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
5978 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
5979 XSETFASTINT (val
, (b1
<< 8) | b2
);
5982 error ("Can't encode to Big5: %d", XFASTINT (ch
));
5986 DEFUN ("set-terminal-coding-system-internal",
5987 Fset_terminal_coding_system_internal
,
5988 Sset_terminal_coding_system_internal
, 1, 1, 0, "")
5990 Lisp_Object coding_system
;
5992 CHECK_SYMBOL (coding_system
, 0);
5993 setup_coding_system (Fcheck_coding_system (coding_system
), &terminal_coding
);
5994 /* We had better not send unsafe characters to terminal. */
5995 terminal_coding
.flags
|= CODING_FLAG_ISO_SAFE
;
5996 /* Characer composition should be disabled. */
5997 terminal_coding
.composing
= COMPOSITION_DISABLED
;
5998 terminal_coding
.src_multibyte
= 1;
5999 terminal_coding
.dst_multibyte
= 0;
6003 DEFUN ("set-safe-terminal-coding-system-internal",
6004 Fset_safe_terminal_coding_system_internal
,
6005 Sset_safe_terminal_coding_system_internal
, 1, 1, 0, "")
6007 Lisp_Object coding_system
;
6009 CHECK_SYMBOL (coding_system
, 0);
6010 setup_coding_system (Fcheck_coding_system (coding_system
),
6011 &safe_terminal_coding
);
6012 /* Characer composition should be disabled. */
6013 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
6014 safe_terminal_coding
.src_multibyte
= 1;
6015 safe_terminal_coding
.dst_multibyte
= 0;
6019 DEFUN ("terminal-coding-system",
6020 Fterminal_coding_system
, Sterminal_coding_system
, 0, 0, 0,
6021 "Return coding system specified for terminal output.")
6024 return terminal_coding
.symbol
;
6027 DEFUN ("set-keyboard-coding-system-internal",
6028 Fset_keyboard_coding_system_internal
,
6029 Sset_keyboard_coding_system_internal
, 1, 1, 0, "")
6031 Lisp_Object coding_system
;
6033 CHECK_SYMBOL (coding_system
, 0);
6034 setup_coding_system (Fcheck_coding_system (coding_system
), &keyboard_coding
);
6035 /* Characer composition should be disabled. */
6036 keyboard_coding
.composing
= COMPOSITION_DISABLED
;
6040 DEFUN ("keyboard-coding-system",
6041 Fkeyboard_coding_system
, Skeyboard_coding_system
, 0, 0, 0,
6042 "Return coding system specified for decoding keyboard input.")
6045 return keyboard_coding
.symbol
;
6049 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
6050 Sfind_operation_coding_system
, 1, MANY
, 0,
6051 "Choose a coding system for an operation based on the target name.\n\
6052 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).\n\
6053 DECODING-SYSTEM is the coding system to use for decoding\n\
6054 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system\n\
6055 for encoding (in case OPERATION does encoding).\n\
6057 The first argument OPERATION specifies an I/O primitive:\n\
6058 For file I/O, `insert-file-contents' or `write-region'.\n\
6059 For process I/O, `call-process', `call-process-region', or `start-process'.\n\
6060 For network I/O, `open-network-stream'.\n\
6062 The remaining arguments should be the same arguments that were passed\n\
6063 to the primitive. Depending on which primitive, one of those arguments\n\
6064 is selected as the TARGET. For example, if OPERATION does file I/O,\n\
6065 whichever argument specifies the file name is TARGET.\n\
6067 TARGET has a meaning which depends on OPERATION:\n\
6068 For file I/O, TARGET is a file name.\n\
6069 For process I/O, TARGET is a process name.\n\
6070 For network I/O, TARGET is a service name or a port number\n\
6072 This function looks up what specified for TARGET in,\n\
6073 `file-coding-system-alist', `process-coding-system-alist',\n\
6074 or `network-coding-system-alist' depending on OPERATION.\n\
6075 They may specify a coding system, a cons of coding systems,\n\
6076 or a function symbol to call.\n\
6077 In the last case, we call the function with one argument,\n\
6078 which is a list of all the arguments given to this function.")
6083 Lisp_Object operation
, target_idx
, target
, val
;
6084 register Lisp_Object chain
;
6087 error ("Too few arguments");
6088 operation
= args
[0];
6089 if (!SYMBOLP (operation
)
6090 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
6091 error ("Invalid first arguement");
6092 if (nargs
< 1 + XINT (target_idx
))
6093 error ("Too few arguments for operation: %s",
6094 XSYMBOL (operation
)->name
->data
);
6095 target
= args
[XINT (target_idx
) + 1];
6096 if (!(STRINGP (target
)
6097 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
6098 error ("Invalid %dth argument", XINT (target_idx
) + 1);
6100 chain
= ((EQ (operation
, Qinsert_file_contents
)
6101 || EQ (operation
, Qwrite_region
))
6102 ? Vfile_coding_system_alist
6103 : (EQ (operation
, Qopen_network_stream
)
6104 ? Vnetwork_coding_system_alist
6105 : Vprocess_coding_system_alist
));
6109 for (; CONSP (chain
); chain
= XCDR (chain
))
6115 && ((STRINGP (target
)
6116 && STRINGP (XCAR (elt
))
6117 && fast_string_match (XCAR (elt
), target
) >= 0)
6118 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
6121 /* Here, if VAL is both a valid coding system and a valid
6122 function symbol, we return VAL as a coding system. */
6125 if (! SYMBOLP (val
))
6127 if (! NILP (Fcoding_system_p (val
)))
6128 return Fcons (val
, val
);
6129 if (! NILP (Ffboundp (val
)))
6131 val
= call1 (val
, Flist (nargs
, args
));
6134 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
6135 return Fcons (val
, val
);
6143 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
6144 Supdate_coding_systems_internal
, 0, 0, 0,
6145 "Update internal database for ISO2022 and CCL based coding systems.\n\
6146 When values of any coding categories are changed, you must\n\
6147 call this function")
6152 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
6156 val
= XSYMBOL (XVECTOR (Vcoding_category_table
)->contents
[i
])->value
;
6159 if (! coding_system_table
[i
])
6160 coding_system_table
[i
] = ((struct coding_system
*)
6161 xmalloc (sizeof (struct coding_system
)));
6162 setup_coding_system (val
, coding_system_table
[i
]);
6164 else if (coding_system_table
[i
])
6166 xfree (coding_system_table
[i
]);
6167 coding_system_table
[i
] = NULL
;
6174 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
6175 Sset_coding_priority_internal
, 0, 0, 0,
6176 "Update internal database for the current value of `coding-category-list'.\n\
6177 This function is internal use only.")
6183 val
= Vcoding_category_list
;
6185 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
6187 if (! SYMBOLP (XCAR (val
)))
6189 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
6190 if (idx
>= CODING_CATEGORY_IDX_MAX
)
6192 coding_priorities
[i
++] = (1 << idx
);
6195 /* If coding-category-list is valid and contains all coding
6196 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
6197 the following code saves Emacs from crashing. */
6198 while (i
< CODING_CATEGORY_IDX_MAX
)
6199 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
6207 /*** 9. Post-amble ***/
6212 conversion_buffer
= (char *) xmalloc (MINIMUM_CONVERSION_BUFFER_SIZE
);
6220 /* Emacs' internal format specific initialize routine. */
6221 for (i
= 0; i
<= 0x20; i
++)
6222 emacs_code_class
[i
] = EMACS_control_code
;
6223 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
6224 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
6225 for (i
= 0x21 ; i
< 0x7F; i
++)
6226 emacs_code_class
[i
] = EMACS_ascii_code
;
6227 emacs_code_class
[0x7F] = EMACS_control_code
;
6228 for (i
= 0x80; i
< 0xFF; i
++)
6229 emacs_code_class
[i
] = EMACS_invalid_code
;
6230 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
6231 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
6232 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
6233 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
6235 /* ISO2022 specific initialize routine. */
6236 for (i
= 0; i
< 0x20; i
++)
6237 iso_code_class
[i
] = ISO_control_0
;
6238 for (i
= 0x21; i
< 0x7F; i
++)
6239 iso_code_class
[i
] = ISO_graphic_plane_0
;
6240 for (i
= 0x80; i
< 0xA0; i
++)
6241 iso_code_class
[i
] = ISO_control_1
;
6242 for (i
= 0xA1; i
< 0xFF; i
++)
6243 iso_code_class
[i
] = ISO_graphic_plane_1
;
6244 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
6245 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
6246 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
6247 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
6248 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
6249 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
6250 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
6251 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
6252 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
6253 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
6255 conversion_buffer_size
= MINIMUM_CONVERSION_BUFFER_SIZE
;
6257 setup_coding_system (Qnil
, &keyboard_coding
);
6258 setup_coding_system (Qnil
, &terminal_coding
);
6259 setup_coding_system (Qnil
, &safe_terminal_coding
);
6260 setup_coding_system (Qnil
, &default_buffer_file_coding
);
6262 bzero (coding_system_table
, sizeof coding_system_table
);
6264 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
6265 for (i
= 0; i
< 128; i
++)
6266 ascii_skip_code
[i
] = 1;
6268 #if defined (MSDOS) || defined (WINDOWSNT)
6269 system_eol_type
= CODING_EOL_CRLF
;
6271 system_eol_type
= CODING_EOL_LF
;
6274 inhibit_pre_post_conversion
= 0;
6282 Qtarget_idx
= intern ("target-idx");
6283 staticpro (&Qtarget_idx
);
6285 Qcoding_system_history
= intern ("coding-system-history");
6286 staticpro (&Qcoding_system_history
);
6287 Fset (Qcoding_system_history
, Qnil
);
6289 /* Target FILENAME is the first argument. */
6290 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
6291 /* Target FILENAME is the third argument. */
6292 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
6294 Qcall_process
= intern ("call-process");
6295 staticpro (&Qcall_process
);
6296 /* Target PROGRAM is the first argument. */
6297 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
6299 Qcall_process_region
= intern ("call-process-region");
6300 staticpro (&Qcall_process_region
);
6301 /* Target PROGRAM is the third argument. */
6302 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
6304 Qstart_process
= intern ("start-process");
6305 staticpro (&Qstart_process
);
6306 /* Target PROGRAM is the third argument. */
6307 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
6309 Qopen_network_stream
= intern ("open-network-stream");
6310 staticpro (&Qopen_network_stream
);
6311 /* Target SERVICE is the fourth argument. */
6312 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
6314 Qcoding_system
= intern ("coding-system");
6315 staticpro (&Qcoding_system
);
6317 Qeol_type
= intern ("eol-type");
6318 staticpro (&Qeol_type
);
6320 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
6321 staticpro (&Qbuffer_file_coding_system
);
6323 Qpost_read_conversion
= intern ("post-read-conversion");
6324 staticpro (&Qpost_read_conversion
);
6326 Qpre_write_conversion
= intern ("pre-write-conversion");
6327 staticpro (&Qpre_write_conversion
);
6329 Qno_conversion
= intern ("no-conversion");
6330 staticpro (&Qno_conversion
);
6332 Qundecided
= intern ("undecided");
6333 staticpro (&Qundecided
);
6335 Qcoding_system_p
= intern ("coding-system-p");
6336 staticpro (&Qcoding_system_p
);
6338 Qcoding_system_error
= intern ("coding-system-error");
6339 staticpro (&Qcoding_system_error
);
6341 Fput (Qcoding_system_error
, Qerror_conditions
,
6342 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
6343 Fput (Qcoding_system_error
, Qerror_message
,
6344 build_string ("Invalid coding system"));
6346 Qcoding_category
= intern ("coding-category");
6347 staticpro (&Qcoding_category
);
6348 Qcoding_category_index
= intern ("coding-category-index");
6349 staticpro (&Qcoding_category_index
);
6351 Vcoding_category_table
6352 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
6353 staticpro (&Vcoding_category_table
);
6356 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
6358 XVECTOR (Vcoding_category_table
)->contents
[i
]
6359 = intern (coding_category_name
[i
]);
6360 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
6361 Qcoding_category_index
, make_number (i
));
6365 Qtranslation_table
= intern ("translation-table");
6366 staticpro (&Qtranslation_table
);
6367 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (1));
6369 Qtranslation_table_id
= intern ("translation-table-id");
6370 staticpro (&Qtranslation_table_id
);
6372 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
6373 staticpro (&Qtranslation_table_for_decode
);
6375 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
6376 staticpro (&Qtranslation_table_for_encode
);
6378 Qsafe_chars
= intern ("safe-chars");
6379 staticpro (&Qsafe_chars
);
6381 Qchar_coding_system
= intern ("char-coding-system");
6382 staticpro (&Qchar_coding_system
);
6384 /* Intern this now in case it isn't already done.
6385 Setting this variable twice is harmless.
6386 But don't staticpro it here--that is done in alloc.c. */
6387 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
6388 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
6389 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (1));
6391 Qvalid_codes
= intern ("valid-codes");
6392 staticpro (&Qvalid_codes
);
6394 Qemacs_mule
= intern ("emacs-mule");
6395 staticpro (&Qemacs_mule
);
6397 Qraw_text
= intern ("raw-text");
6398 staticpro (&Qraw_text
);
6400 defsubr (&Scoding_system_p
);
6401 defsubr (&Sread_coding_system
);
6402 defsubr (&Sread_non_nil_coding_system
);
6403 defsubr (&Scheck_coding_system
);
6404 defsubr (&Sdetect_coding_region
);
6405 defsubr (&Sdetect_coding_string
);
6406 defsubr (&Sfind_coding_systems_region_internal
);
6407 defsubr (&Sdecode_coding_region
);
6408 defsubr (&Sencode_coding_region
);
6409 defsubr (&Sdecode_coding_string
);
6410 defsubr (&Sencode_coding_string
);
6411 defsubr (&Sdecode_sjis_char
);
6412 defsubr (&Sencode_sjis_char
);
6413 defsubr (&Sdecode_big5_char
);
6414 defsubr (&Sencode_big5_char
);
6415 defsubr (&Sset_terminal_coding_system_internal
);
6416 defsubr (&Sset_safe_terminal_coding_system_internal
);
6417 defsubr (&Sterminal_coding_system
);
6418 defsubr (&Sset_keyboard_coding_system_internal
);
6419 defsubr (&Skeyboard_coding_system
);
6420 defsubr (&Sfind_operation_coding_system
);
6421 defsubr (&Supdate_coding_systems_internal
);
6422 defsubr (&Sset_coding_priority_internal
);
6424 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
6425 "List of coding systems.\n\
6427 Do not alter the value of this variable manually. This variable should be\n\
6428 updated by the functions `make-coding-system' and\n\
6429 `define-coding-system-alias'.");
6430 Vcoding_system_list
= Qnil
;
6432 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
6433 "Alist of coding system names.\n\
6434 Each element is one element list of coding system name.\n\
6435 This variable is given to `completing-read' as TABLE argument.\n\
6437 Do not alter the value of this variable manually. This variable should be\n\
6438 updated by the functions `make-coding-system' and\n\
6439 `define-coding-system-alias'.");
6440 Vcoding_system_alist
= Qnil
;
6442 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
6443 "List of coding-categories (symbols) ordered by priority.");
6447 Vcoding_category_list
= Qnil
;
6448 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
6449 Vcoding_category_list
6450 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
6451 Vcoding_category_list
);
6454 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
6455 "Specify the coding system for read operations.\n\
6456 It is useful to bind this variable with `let', but do not set it globally.\n\
6457 If the value is a coding system, it is used for decoding on read operation.\n\
6458 If not, an appropriate element is used from one of the coding system alists:\n\
6459 There are three such tables, `file-coding-system-alist',\n\
6460 `process-coding-system-alist', and `network-coding-system-alist'.");
6461 Vcoding_system_for_read
= Qnil
;
6463 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
6464 "Specify the coding system for write operations.\n\
6465 Programs bind this variable with `let', but you should not set it globally.\n\
6466 If the value is a coding system, it is used for encoding of output,\n\
6467 when writing it to a file and when sending it to a file or subprocess.\n\
6469 If this does not specify a coding system, an appropriate element\n\
6470 is used from one of the coding system alists:\n\
6471 There are three such tables, `file-coding-system-alist',\n\
6472 `process-coding-system-alist', and `network-coding-system-alist'.\n\
6473 For output to files, if the above procedure does not specify a coding system,\n\
6474 the value of `buffer-file-coding-system' is used.");
6475 Vcoding_system_for_write
= Qnil
;
6477 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
6478 "Coding system used in the latest file or process I/O.");
6479 Vlast_coding_system_used
= Qnil
;
6481 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
6482 "*Non-nil means always inhibit code conversion of end-of-line format.\n\
6483 See info node `Coding Systems' and info node `Text and Binary' concerning\n\
6485 inhibit_eol_conversion
= 0;
6487 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
6488 "Non-nil means process buffer inherits coding system of process output.\n\
6489 Bind it to t if the process output is to be treated as if it were a file\n\
6490 read from some filesystem.");
6491 inherit_process_coding_system
= 0;
6493 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
6494 "Alist to decide a coding system to use for a file I/O operation.\n\
6495 The format is ((PATTERN . VAL) ...),\n\
6496 where PATTERN is a regular expression matching a file name,\n\
6497 VAL is a coding system, a cons of coding systems, or a function symbol.\n\
6498 If VAL is a coding system, it is used for both decoding and encoding\n\
6499 the file contents.\n\
6500 If VAL is a cons of coding systems, the car part is used for decoding,\n\
6501 and the cdr part is used for encoding.\n\
6502 If VAL is a function symbol, the function must return a coding system\n\
6503 or a cons of coding systems which are used as above.\n\
6505 See also the function `find-operation-coding-system'\n\
6506 and the variable `auto-coding-alist'.");
6507 Vfile_coding_system_alist
= Qnil
;
6509 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
6510 "Alist to decide a coding system to use for a process I/O operation.\n\
6511 The format is ((PATTERN . VAL) ...),\n\
6512 where PATTERN is a regular expression matching a program name,\n\
6513 VAL is a coding system, a cons of coding systems, or a function symbol.\n\
6514 If VAL is a coding system, it is used for both decoding what received\n\
6515 from the program and encoding what sent to the program.\n\
6516 If VAL is a cons of coding systems, the car part is used for decoding,\n\
6517 and the cdr part is used for encoding.\n\
6518 If VAL is a function symbol, the function must return a coding system\n\
6519 or a cons of coding systems which are used as above.\n\
6521 See also the function `find-operation-coding-system'.");
6522 Vprocess_coding_system_alist
= Qnil
;
6524 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
6525 "Alist to decide a coding system to use for a network I/O operation.\n\
6526 The format is ((PATTERN . VAL) ...),\n\
6527 where PATTERN is a regular expression matching a network service name\n\
6528 or is a port number to connect to,\n\
6529 VAL is a coding system, a cons of coding systems, or a function symbol.\n\
6530 If VAL is a coding system, it is used for both decoding what received\n\
6531 from the network stream and encoding what sent to the network stream.\n\
6532 If VAL is a cons of coding systems, the car part is used for decoding,\n\
6533 and the cdr part is used for encoding.\n\
6534 If VAL is a function symbol, the function must return a coding system\n\
6535 or a cons of coding systems which are used as above.\n\
6537 See also the function `find-operation-coding-system'.");
6538 Vnetwork_coding_system_alist
= Qnil
;
6540 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
6541 "Coding system to use with system messages.");
6542 Vlocale_coding_system
= Qnil
;
6544 /* The eol mnemonics are reset in startup.el system-dependently. */
6545 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
6546 "*String displayed in mode line for UNIX-like (LF) end-of-line format.");
6547 eol_mnemonic_unix
= build_string (":");
6549 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
6550 "*String displayed in mode line for DOS-like (CRLF) end-of-line format.");
6551 eol_mnemonic_dos
= build_string ("\\");
6553 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
6554 "*String displayed in mode line for MAC-like (CR) end-of-line format.");
6555 eol_mnemonic_mac
= build_string ("/");
6557 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
6558 "*String displayed in mode line when end-of-line format is not yet determined.");
6559 eol_mnemonic_undecided
= build_string (":");
6561 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
6562 "*Non-nil enables character translation while encoding and decoding.");
6563 Venable_character_translation
= Qt
;
6565 DEFVAR_LISP ("standard-translation-table-for-decode",
6566 &Vstandard_translation_table_for_decode
,
6567 "Table for translating characters while decoding.");
6568 Vstandard_translation_table_for_decode
= Qnil
;
6570 DEFVAR_LISP ("standard-translation-table-for-encode",
6571 &Vstandard_translation_table_for_encode
,
6572 "Table for translationg characters while encoding.");
6573 Vstandard_translation_table_for_encode
= Qnil
;
6575 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
6576 "Alist of charsets vs revision numbers.\n\
6577 While encoding, if a charset (car part of an element) is found,\n\
6578 designate it with the escape sequence identifing revision (cdr part of the element).");
6579 Vcharset_revision_alist
= Qnil
;
6581 DEFVAR_LISP ("default-process-coding-system",
6582 &Vdefault_process_coding_system
,
6583 "Cons of coding systems used for process I/O by default.\n\
6584 The car part is used for decoding a process output,\n\
6585 the cdr part is used for encoding a text to be sent to a process.");
6586 Vdefault_process_coding_system
= Qnil
;
6588 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
6589 "Table of extra Latin codes in the range 128..159 (inclusive).\n\
6590 This is a vector of length 256.\n\
6591 If Nth element is non-nil, the existence of code N in a file\n\
6592 \(or output of subprocess) doesn't prevent it to be detected as\n\
6593 a coding system of ISO 2022 variant which has a flag\n\
6594 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file\n\
6595 or reading output of a subprocess.\n\
6596 Only 128th through 159th elements has a meaning.");
6597 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
6599 DEFVAR_LISP ("select-safe-coding-system-function",
6600 &Vselect_safe_coding_system_function
,
6601 "Function to call to select safe coding system for encoding a text.\n\
6603 If set, this function is called to force a user to select a proper\n\
6604 coding system which can encode the text in the case that a default\n\
6605 coding system used in each operation can't encode the text.\n\
6607 The default value is `select-safe-coding-system' (which see).");
6608 Vselect_safe_coding_system_function
= Qnil
;
6610 DEFVAR_LISP ("char-coding-system-table", &Vchar_coding_system_table
,
6611 "Char-table containing safe coding systems of each characters.\n\
6612 Each element doesn't include such generic coding systems that can\n\
6613 encode any characters. They are in the first extra slot.");
6614 Vchar_coding_system_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6616 DEFVAR_BOOL ("inhibit-iso-escape-detection",
6617 &inhibit_iso_escape_detection
,
6618 "If non-nil, Emacs ignores ISO2022's escape sequence on code detection.\n\
6620 By default, on reading a file, Emacs tries to detect how the text is\n\
6621 encoded. This code detection is sensitive to escape sequences. If\n\
6622 the sequence is valid as ISO2022, the code is determined as one of\n\
6623 the ISO2022 encodings, and the file is decoded by the corresponding\n\
6624 coding system (e.g. `iso-2022-7bit').\n\
6626 However, there may be a case that you want to read escape sequences in\n\
6627 a file as is. In such a case, you can set this variable to non-nil.\n\
6628 Then, as the code detection ignores any escape sequences, no file is\n\
6629 detected as encoded in some ISO2022 encoding. The result is that all\n\
6630 escape sequences become visible in a buffer.\n\
6632 The default value is nil, and it is strongly recommended not to change\n\
6633 it. That is because many Emacs Lisp source files that contain\n\
6634 non-ASCII characters are encoded by the coding system `iso-2022-7bit'\n\
6635 in Emacs's distribution, and they won't be decoded correctly on\n\
6636 reading if you suppress escape sequence detection.\n\
6638 The other way to read escape sequences in a file without decoding is\n\
6639 to explicitly specify some coding system that doesn't use ISO2022's\n\
6640 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument].");
6641 inhibit_iso_escape_detection
= 0;
6645 emacs_strerror (error_number
)
6650 synchronize_system_messages_locale ();
6651 str
= strerror (error_number
);
6653 if (! NILP (Vlocale_coding_system
))
6655 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
6656 Vlocale_coding_system
,
6658 str
= (char *) XSTRING (dec
)->data
;