1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 1995, 1997, 1998, 2002 Electrotechnical Laboratory, JAPAN.
3 Licensed to the Free Software Foundation.
4 Copyright (C) 2001 Free Software Foundation, Inc.
6 This file is part of GNU Emacs.
8 GNU Emacs is free software; you can redistribute it and/or modify
9 it under the terms of the GNU General Public License as published by
10 the Free Software Foundation; either version 2, or (at your option)
13 GNU Emacs is distributed in the hope that it will be useful,
14 but WITHOUT ANY WARRANTY; without even the implied warranty of
15 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 GNU General Public License for more details.
18 You should have received a copy of the GNU General Public License
19 along with GNU Emacs; see the file COPYING. If not, write to
20 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
21 Boston, MA 02111-1307, USA. */
23 /*** TABLE OF CONTENTS ***
27 2. Emacs' internal format (emacs-mule) handlers
29 4. Shift-JIS and BIG5 handlers
31 6. End-of-line handlers
32 7. C library functions
33 8. Emacs Lisp library functions
38 /*** 0. General comments ***/
41 /*** GENERAL NOTE on CODING SYSTEMS ***
43 A coding system is an encoding mechanism for one or more character
44 sets. Here's a list of coding systems which Emacs can handle. When
45 we say "decode", it means converting some other coding system to
46 Emacs' internal format (emacs-mule), and when we say "encode",
47 it means converting the coding system emacs-mule to some other
50 0. Emacs' internal format (emacs-mule)
52 Emacs itself holds a multi-lingual character in buffers and strings
53 in a special format. Details are described in section 2.
57 The most famous coding system for multiple character sets. X's
58 Compound Text, various EUCs (Extended Unix Code), and coding
59 systems used in Internet communication such as ISO-2022-JP are
60 all variants of ISO2022. Details are described in section 3.
62 2. SJIS (or Shift-JIS or MS-Kanji-Code)
64 A coding system to encode character sets: ASCII, JISX0201, and
65 JISX0208. Widely used for PC's in Japan. Details are described in
70 A coding system to encode the character sets ASCII and Big5. Widely
71 used for Chinese (mainly in Taiwan and Hong Kong). Details are
72 described in section 4. In this file, when we write "BIG5"
73 (all uppercase), we mean the coding system, and when we write
74 "Big5" (capitalized), we mean the character set.
78 A coding system for text containing random 8-bit code. Emacs does
79 no code conversion on such text except for end-of-line format.
83 If a user wants to read/write text encoded in a coding system not
84 listed above, he can supply a decoder and an encoder for it as CCL
85 (Code Conversion Language) programs. Emacs executes the CCL program
86 while reading/writing.
88 Emacs represents a coding system by a Lisp symbol that has a property
89 `coding-system'. But, before actually using the coding system, the
90 information about it is set in a structure of type `struct
91 coding_system' for rapid processing. See section 6 for more details.
95 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
97 How end-of-line of text is encoded depends on the operating system.
98 For instance, Unix's format is just one byte of `line-feed' code,
99 whereas DOS's format is two-byte sequence of `carriage-return' and
100 `line-feed' codes. MacOS's format is usually one byte of
103 Since text character encoding and end-of-line encoding are
104 independent, any coding system described above can have any
105 end-of-line format. So Emacs has information about end-of-line
106 format in each coding-system. See section 6 for more details.
110 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
112 These functions check if a text between SRC and SRC_END is encoded
113 in the coding system category XXX. Each returns an integer value in
114 which appropriate flag bits for the category XXX are set. The flag
115 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
116 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
117 of the range 0x80..0x9F are in multibyte form. */
120 detect_coding_emacs_mule (src
, src_end
, multibytep
)
121 unsigned char *src
, *src_end
;
128 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
130 These functions decode SRC_BYTES length of unibyte text at SOURCE
131 encoded in CODING to Emacs' internal format. The resulting
132 multibyte text goes to a place pointed to by DESTINATION, the length
133 of which should not exceed DST_BYTES.
135 These functions set the information about original and decoded texts
136 in the members `produced', `produced_char', `consumed', and
137 `consumed_char' of the structure *CODING. They also set the member
138 `result' to one of CODING_FINISH_XXX indicating how the decoding
141 DST_BYTES zero means that the source area and destination area are
142 overlapped, which means that we can produce a decoded text until it
143 reaches the head of the not-yet-decoded source text.
145 Below is a template for these functions. */
148 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
149 struct coding_system
*coding
;
150 unsigned char *source
, *destination
;
151 int src_bytes
, dst_bytes
;
157 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
159 These functions encode SRC_BYTES length text at SOURCE from Emacs'
160 internal multibyte format to CODING. The resulting unibyte text
161 goes to a place pointed to by DESTINATION, the length of which
162 should not exceed DST_BYTES.
164 These functions set the information about original and encoded texts
165 in the members `produced', `produced_char', `consumed', and
166 `consumed_char' of the structure *CODING. They also set the member
167 `result' to one of CODING_FINISH_XXX indicating how the encoding
170 DST_BYTES zero means that the source area and destination area are
171 overlapped, which means that we can produce encoded text until it
172 reaches at the head of the not-yet-encoded source text.
174 Below is a template for these functions. */
177 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
178 struct coding_system
*coding
;
179 unsigned char *source
, *destination
;
180 int src_bytes
, dst_bytes
;
186 /*** COMMONLY USED MACROS ***/
188 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
189 get one, two, and three bytes from the source text respectively.
190 If there are not enough bytes in the source, they jump to
191 `label_end_of_loop'. The caller should set variables `coding',
192 `src' and `src_end' to appropriate pointer in advance. These
193 macros are called from decoding routines `decode_coding_XXX', thus
194 it is assumed that the source text is unibyte. */
196 #define ONE_MORE_BYTE(c1) \
198 if (src >= src_end) \
200 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
201 goto label_end_of_loop; \
206 #define TWO_MORE_BYTES(c1, c2) \
208 if (src + 1 >= src_end) \
210 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
211 goto label_end_of_loop; \
218 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
219 form if MULTIBYTEP is nonzero. */
221 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
223 if (src >= src_end) \
225 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
226 goto label_end_of_loop; \
229 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
230 c1 = *src++ - 0x20; \
233 /* Set C to the next character at the source text pointed by `src'.
234 If there are not enough characters in the source, jump to
235 `label_end_of_loop'. The caller should set variables `coding'
236 `src', `src_end', and `translation_table' to appropriate pointers
237 in advance. This macro is used in encoding routines
238 `encode_coding_XXX', thus it assumes that the source text is in
239 multibyte form except for 8-bit characters. 8-bit characters are
240 in multibyte form if coding->src_multibyte is nonzero, else they
241 are represented by a single byte. */
243 #define ONE_MORE_CHAR(c) \
245 int len = src_end - src; \
249 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
250 goto label_end_of_loop; \
252 if (coding->src_multibyte \
253 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
254 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
256 c = *src, bytes = 1; \
257 if (!NILP (translation_table)) \
258 c = translate_char (translation_table, c, -1, 0, 0); \
263 /* Produce a multibyte form of character C to `dst'. Jump to
264 `label_end_of_loop' if there's not enough space at `dst'.
266 If we are now in the middle of a composition sequence, the decoded
267 character may be ALTCHAR (for the current composition). In that
268 case, the character goes to coding->cmp_data->data instead of
271 This macro is used in decoding routines. */
273 #define EMIT_CHAR(c) \
275 if (! COMPOSING_P (coding) \
276 || coding->composing == COMPOSITION_RELATIVE \
277 || coding->composing == COMPOSITION_WITH_RULE) \
279 int bytes = CHAR_BYTES (c); \
280 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
282 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
283 goto label_end_of_loop; \
285 dst += CHAR_STRING (c, dst); \
286 coding->produced_char++; \
289 if (COMPOSING_P (coding) \
290 && coding->composing != COMPOSITION_RELATIVE) \
292 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
293 coding->composition_rule_follows \
294 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
299 #define EMIT_ONE_BYTE(c) \
301 if (dst >= (dst_bytes ? dst_end : src)) \
303 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
304 goto label_end_of_loop; \
309 #define EMIT_TWO_BYTES(c1, c2) \
311 if (dst + 2 > (dst_bytes ? dst_end : src)) \
313 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
314 goto label_end_of_loop; \
316 *dst++ = c1, *dst++ = c2; \
319 #define EMIT_BYTES(from, to) \
321 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
323 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
324 goto label_end_of_loop; \
331 /*** 1. Preamble ***/
344 #include "composite.h"
349 #else /* not emacs */
353 #endif /* not emacs */
355 Lisp_Object Qcoding_system
, Qeol_type
;
356 Lisp_Object Qbuffer_file_coding_system
;
357 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
358 Lisp_Object Qno_conversion
, Qundecided
;
359 Lisp_Object Qcoding_system_history
;
360 Lisp_Object Qsafe_chars
;
361 Lisp_Object Qvalid_codes
;
363 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
364 Lisp_Object Qcall_process
, Qcall_process_region
, Qprocess_argument
;
365 Lisp_Object Qstart_process
, Qopen_network_stream
;
366 Lisp_Object Qtarget_idx
;
368 Lisp_Object Vselect_safe_coding_system_function
;
370 /* Mnemonic string for each format of end-of-line. */
371 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
372 /* Mnemonic string to indicate format of end-of-line is not yet
374 Lisp_Object eol_mnemonic_undecided
;
376 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
377 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
382 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
384 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
386 /* Coding system emacs-mule and raw-text are for converting only
387 end-of-line format. */
388 Lisp_Object Qemacs_mule
, Qraw_text
;
390 /* Coding-systems are handed between Emacs Lisp programs and C internal
391 routines by the following three variables. */
392 /* Coding-system for reading files and receiving data from process. */
393 Lisp_Object Vcoding_system_for_read
;
394 /* Coding-system for writing files and sending data to process. */
395 Lisp_Object Vcoding_system_for_write
;
396 /* Coding-system actually used in the latest I/O. */
397 Lisp_Object Vlast_coding_system_used
;
399 /* A vector of length 256 which contains information about special
400 Latin codes (especially for dealing with Microsoft codes). */
401 Lisp_Object Vlatin_extra_code_table
;
403 /* Flag to inhibit code conversion of end-of-line format. */
404 int inhibit_eol_conversion
;
406 /* Flag to inhibit ISO2022 escape sequence detection. */
407 int inhibit_iso_escape_detection
;
409 /* Flag to make buffer-file-coding-system inherit from process-coding. */
410 int inherit_process_coding_system
;
412 /* Coding system to be used to encode text for terminal display. */
413 struct coding_system terminal_coding
;
415 /* Coding system to be used to encode text for terminal display when
416 terminal coding system is nil. */
417 struct coding_system safe_terminal_coding
;
419 /* Coding system of what is sent from terminal keyboard. */
420 struct coding_system keyboard_coding
;
422 /* Default coding system to be used to write a file. */
423 struct coding_system default_buffer_file_coding
;
425 Lisp_Object Vfile_coding_system_alist
;
426 Lisp_Object Vprocess_coding_system_alist
;
427 Lisp_Object Vnetwork_coding_system_alist
;
429 Lisp_Object Vlocale_coding_system
;
433 Lisp_Object Qcoding_category
, Qcoding_category_index
;
435 /* List of symbols `coding-category-xxx' ordered by priority. */
436 Lisp_Object Vcoding_category_list
;
438 /* Table of coding categories (Lisp symbols). */
439 Lisp_Object Vcoding_category_table
;
441 /* Table of names of symbol for each coding-category. */
442 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
443 "coding-category-emacs-mule",
444 "coding-category-sjis",
445 "coding-category-iso-7",
446 "coding-category-iso-7-tight",
447 "coding-category-iso-8-1",
448 "coding-category-iso-8-2",
449 "coding-category-iso-7-else",
450 "coding-category-iso-8-else",
451 "coding-category-ccl",
452 "coding-category-big5",
453 "coding-category-utf-8",
454 "coding-category-utf-16-be",
455 "coding-category-utf-16-le",
456 "coding-category-raw-text",
457 "coding-category-binary"
460 /* Table of pointers to coding systems corresponding to each coding
462 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
464 /* Table of coding category masks. Nth element is a mask for a coding
465 category of which priority is Nth. */
467 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
469 /* Flag to tell if we look up translation table on character code
471 Lisp_Object Venable_character_translation
;
472 /* Standard translation table to look up on decoding (reading). */
473 Lisp_Object Vstandard_translation_table_for_decode
;
474 /* Standard translation table to look up on encoding (writing). */
475 Lisp_Object Vstandard_translation_table_for_encode
;
477 Lisp_Object Qtranslation_table
;
478 Lisp_Object Qtranslation_table_id
;
479 Lisp_Object Qtranslation_table_for_decode
;
480 Lisp_Object Qtranslation_table_for_encode
;
482 /* Alist of charsets vs revision number. */
483 Lisp_Object Vcharset_revision_alist
;
485 /* Default coding systems used for process I/O. */
486 Lisp_Object Vdefault_process_coding_system
;
488 /* Global flag to tell that we can't call post-read-conversion and
489 pre-write-conversion functions. Usually the value is zero, but it
490 is set to 1 temporarily while such functions are running. This is
491 to avoid infinite recursive call. */
492 static int inhibit_pre_post_conversion
;
494 /* Char-table containing safe coding systems of each character. */
495 Lisp_Object Vchar_coding_system_table
;
496 Lisp_Object Qchar_coding_system
;
498 /* Return `safe-chars' property of coding system CODING. Don't check
499 validity of CODING. */
502 coding_safe_chars (coding
)
503 struct coding_system
*coding
;
505 Lisp_Object coding_spec
, plist
, safe_chars
;
507 coding_spec
= Fget (coding
->symbol
, Qcoding_system
);
508 plist
= XVECTOR (coding_spec
)->contents
[3];
509 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
510 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
513 #define CODING_SAFE_CHAR_P(safe_chars, c) \
514 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
517 /*** 2. Emacs internal format (emacs-mule) handlers ***/
519 /* Emacs' internal format for representation of multiple character
520 sets is a kind of multi-byte encoding, i.e. characters are
521 represented by variable-length sequences of one-byte codes.
523 ASCII characters and control characters (e.g. `tab', `newline') are
524 represented by one-byte sequences which are their ASCII codes, in
525 the range 0x00 through 0x7F.
527 8-bit characters of the range 0x80..0x9F are represented by
528 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
531 8-bit characters of the range 0xA0..0xFF are represented by
532 one-byte sequences which are their 8-bit code.
534 The other characters are represented by a sequence of `base
535 leading-code', optional `extended leading-code', and one or two
536 `position-code's. The length of the sequence is determined by the
537 base leading-code. Leading-code takes the range 0x81 through 0x9D,
538 whereas extended leading-code and position-code take the range 0xA0
539 through 0xFF. See `charset.h' for more details about leading-code
542 --- CODE RANGE of Emacs' internal format ---
546 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
547 eight-bit-graphic 0xA0..0xBF
548 ELSE 0x81..0x9D + [0xA0..0xFF]+
549 ---------------------------------------------
551 As this is the internal character representation, the format is
552 usually not used externally (i.e. in a file or in a data sent to a
553 process). But, it is possible to have a text externally in this
554 format (i.e. by encoding by the coding system `emacs-mule').
556 In that case, a sequence of one-byte codes has a slightly different
559 Firstly, all characters in eight-bit-control are represented by
560 one-byte sequences which are their 8-bit code.
562 Next, character composition data are represented by the byte
563 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
565 METHOD is 0xF0 plus one of composition method (enum
568 BYTES is 0xA0 plus the byte length of these composition data,
570 CHARS is 0xA0 plus the number of characters composed by these
573 COMPONENTs are characters of multibyte form or composition
574 rules encoded by two-byte of ASCII codes.
576 In addition, for backward compatibility, the following formats are
577 also recognized as composition data on decoding.
580 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
583 MSEQ is a multibyte form but in these special format:
584 ASCII: 0xA0 ASCII_CODE+0x80,
585 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
586 RULE is a one byte code of the range 0xA0..0xF0 that
587 represents a composition rule.
590 enum emacs_code_class_type emacs_code_class
[256];
592 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
593 Check if a text is encoded in Emacs' internal format. If it is,
594 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
597 detect_coding_emacs_mule (src
, src_end
, multibytep
)
598 unsigned char *src
, *src_end
;
603 /* Dummy for ONE_MORE_BYTE. */
604 struct coding_system dummy_coding
;
605 struct coding_system
*coding
= &dummy_coding
;
609 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
617 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
626 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
629 else if (c
>= 0x80 && c
< 0xA0)
632 /* Old leading code for a composite character. */
636 unsigned char *src_base
= src
- 1;
639 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
642 src
= src_base
+ bytes
;
647 return CODING_CATEGORY_MASK_EMACS_MULE
;
651 /* Record the starting position START and METHOD of one composition. */
653 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
655 struct composition_data *cmp_data = coding->cmp_data; \
656 int *data = cmp_data->data + cmp_data->used; \
657 coding->cmp_data_start = cmp_data->used; \
659 data[1] = cmp_data->char_offset + start; \
660 data[3] = (int) method; \
661 cmp_data->used += 4; \
664 /* Record the ending position END of the current composition. */
666 #define CODING_ADD_COMPOSITION_END(coding, end) \
668 struct composition_data *cmp_data = coding->cmp_data; \
669 int *data = cmp_data->data + coding->cmp_data_start; \
670 data[0] = cmp_data->used - coding->cmp_data_start; \
671 data[2] = cmp_data->char_offset + end; \
674 /* Record one COMPONENT (alternate character or composition rule). */
676 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
677 (coding->cmp_data->data[coding->cmp_data->used++] = component)
680 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
681 is not less than SRC_END, return -1 without incrementing Src. */
683 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
686 /* Decode a character represented as a component of composition
687 sequence of Emacs 20 style at SRC. Set C to that character, store
688 its multibyte form sequence at P, and set P to the end of that
689 sequence. If no valid character is found, set C to -1. */
691 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
695 c = SAFE_ONE_MORE_BYTE (); \
698 if (CHAR_HEAD_P (c)) \
700 else if (c == 0xA0) \
702 c = SAFE_ONE_MORE_BYTE (); \
711 else if (BASE_LEADING_CODE_P (c - 0x20)) \
713 unsigned char *p0 = p; \
717 bytes = BYTES_BY_CHAR_HEAD (c); \
720 c = SAFE_ONE_MORE_BYTE (); \
725 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes)) \
726 c = STRING_CHAR (p0, bytes); \
735 /* Decode a composition rule represented as a component of composition
736 sequence of Emacs 20 style at SRC. Set C to the rule. If not
737 valid rule is found, set C to -1. */
739 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
741 c = SAFE_ONE_MORE_BYTE (); \
743 if (c < 0 || c >= 81) \
747 gref = c / 9, nref = c % 9; \
748 c = COMPOSITION_ENCODE_RULE (gref, nref); \
753 /* Decode composition sequence encoded by `emacs-mule' at the source
754 pointed by SRC. SRC_END is the end of source. Store information
755 of the composition in CODING->cmp_data.
757 For backward compatibility, decode also a composition sequence of
758 Emacs 20 style. In that case, the composition sequence contains
759 characters that should be extracted into a buffer or string. Store
760 those characters at *DESTINATION in multibyte form.
762 If we encounter an invalid byte sequence, return 0.
763 If we encounter an insufficient source or destination, or
764 insufficient space in CODING->cmp_data, return 1.
765 Otherwise, return consumed bytes in the source.
769 decode_composition_emacs_mule (coding
, src
, src_end
,
770 destination
, dst_end
, dst_bytes
)
771 struct coding_system
*coding
;
772 unsigned char *src
, *src_end
, **destination
, *dst_end
;
775 unsigned char *dst
= *destination
;
776 int method
, data_len
, nchars
;
777 unsigned char *src_base
= src
++;
778 /* Store components of composition. */
779 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
781 /* Store multibyte form of characters to be composed. This is for
782 Emacs 20 style composition sequence. */
783 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
784 unsigned char *bufp
= buf
;
785 int c
, i
, gref
, nref
;
787 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
788 >= COMPOSITION_DATA_SIZE
)
790 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
795 if (c
- 0xF0 >= COMPOSITION_RELATIVE
796 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
801 with_rule
= (method
== COMPOSITION_WITH_RULE
802 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
806 || src_base
+ data_len
> src_end
)
812 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
814 /* If it is longer than this, it can't be valid. */
815 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
818 if (ncomponent
% 2 && with_rule
)
820 ONE_MORE_BYTE (gref
);
822 ONE_MORE_BYTE (nref
);
824 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
829 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
))
830 c
= STRING_CHAR (src
, bytes
);
835 component
[ncomponent
] = c
;
840 /* This may be an old Emacs 20 style format. See the comment at
841 the section 2 of this file. */
842 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
844 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
845 goto label_end_of_loop
;
851 method
= COMPOSITION_RELATIVE
;
852 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
854 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
857 component
[ncomponent
++] = c
;
865 method
= COMPOSITION_WITH_RULE
;
867 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
872 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
874 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
877 component
[ncomponent
++] = c
;
878 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
881 component
[ncomponent
++] = c
;
885 nchars
= (ncomponent
+ 1) / 2;
891 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
893 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
894 for (i
= 0; i
< ncomponent
; i
++)
895 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
896 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
899 unsigned char *p
= buf
;
900 EMIT_BYTES (p
, bufp
);
901 *destination
+= bufp
- buf
;
902 coding
->produced_char
+= nchars
;
904 return (src
- src_base
);
910 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
913 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
914 struct coding_system
*coding
;
915 unsigned char *source
, *destination
;
916 int src_bytes
, dst_bytes
;
918 unsigned char *src
= source
;
919 unsigned char *src_end
= source
+ src_bytes
;
920 unsigned char *dst
= destination
;
921 unsigned char *dst_end
= destination
+ dst_bytes
;
922 /* SRC_BASE remembers the start position in source in each loop.
923 The loop will be exited when there's not enough source code, or
924 when there's not enough destination area to produce a
926 unsigned char *src_base
;
928 coding
->produced_char
= 0;
929 while ((src_base
= src
) < src_end
)
931 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
], *p
;
938 if (coding
->eol_type
== CODING_EOL_CR
)
940 else if (coding
->eol_type
== CODING_EOL_CRLF
)
945 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
947 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
948 goto label_end_of_loop
;
955 coding
->produced_char
++;
958 else if (*src
== '\n')
960 if ((coding
->eol_type
== CODING_EOL_CR
961 || coding
->eol_type
== CODING_EOL_CRLF
)
962 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
964 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
965 goto label_end_of_loop
;
968 coding
->produced_char
++;
971 else if (*src
== 0x80)
973 /* Start of composition data. */
974 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
978 goto label_end_of_loop
;
979 else if (consumed
> 0)
984 bytes
= CHAR_STRING (*src
, tmp
);
988 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
))
995 bytes
= CHAR_STRING (*src
, tmp
);
999 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1001 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1004 while (bytes
--) *dst
++ = *p
++;
1005 coding
->produced_char
++;
1008 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1009 coding
->produced
= dst
- destination
;
1013 /* Encode composition data stored at DATA into a special byte sequence
1014 starting by 0x80. Update CODING->cmp_data_start and maybe
1015 CODING->cmp_data for the next call. */
1017 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1019 unsigned char buf[1024], *p0 = buf, *p; \
1020 int len = data[0]; \
1024 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1025 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1027 if (data[3] == COMPOSITION_WITH_RULE \
1028 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1030 p += CHAR_STRING (data[4], p); \
1031 for (i = 5; i < len; i += 2) \
1034 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1035 *p++ = 0x20 + gref; \
1036 *p++ = 0x20 + nref; \
1037 p += CHAR_STRING (data[i + 1], p); \
1042 for (i = 4; i < len; i++) \
1043 p += CHAR_STRING (data[i], p); \
1045 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1047 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1049 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1050 goto label_end_of_loop; \
1054 coding->cmp_data_start += data[0]; \
1055 if (coding->cmp_data_start == coding->cmp_data->used \
1056 && coding->cmp_data->next) \
1058 coding->cmp_data = coding->cmp_data->next; \
1059 coding->cmp_data_start = 0; \
1064 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1065 unsigned char *, int, int));
1068 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1069 struct coding_system
*coding
;
1070 unsigned char *source
, *destination
;
1071 int src_bytes
, dst_bytes
;
1073 unsigned char *src
= source
;
1074 unsigned char *src_end
= source
+ src_bytes
;
1075 unsigned char *dst
= destination
;
1076 unsigned char *dst_end
= destination
+ dst_bytes
;
1077 unsigned char *src_base
;
1082 Lisp_Object translation_table
;
1084 translation_table
= Qnil
;
1086 /* Optimization for the case that there's no composition. */
1087 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1089 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1093 char_offset
= coding
->cmp_data
->char_offset
;
1094 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1099 /* If SRC starts a composition, encode the information about the
1100 composition in advance. */
1101 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1102 && char_offset
+ coding
->consumed_char
== data
[1])
1104 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1105 char_offset
= coding
->cmp_data
->char_offset
;
1106 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1110 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1111 || coding
->eol_type
== CODING_EOL_CR
))
1113 if (coding
->eol_type
== CODING_EOL_CRLF
)
1114 EMIT_TWO_BYTES ('\r', c
);
1116 EMIT_ONE_BYTE ('\r');
1118 else if (SINGLE_BYTE_CHAR_P (c
))
1121 EMIT_BYTES (src_base
, src
);
1122 coding
->consumed_char
++;
1125 coding
->consumed
= src_base
- source
;
1126 coding
->produced
= coding
->produced_char
= dst
- destination
;
1131 /*** 3. ISO2022 handlers ***/
1133 /* The following note describes the coding system ISO2022 briefly.
1134 Since the intention of this note is to help understand the
1135 functions in this file, some parts are NOT ACCURATE or are OVERLY
1136 SIMPLIFIED. For thorough understanding, please refer to the
1137 original document of ISO2022. This is equivalent to the standard
1138 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1140 ISO2022 provides many mechanisms to encode several character sets
1141 in 7-bit and 8-bit environments. For 7-bit environments, all text
1142 is encoded using bytes less than 128. This may make the encoded
1143 text a little bit longer, but the text passes more easily through
1144 several types of gateway, some of which strip off the MSB (Most
1147 There are two kinds of character sets: control character sets and
1148 graphic character sets. The former contain control characters such
1149 as `newline' and `escape' to provide control functions (control
1150 functions are also provided by escape sequences). The latter
1151 contain graphic characters such as 'A' and '-'. Emacs recognizes
1152 two control character sets and many graphic character sets.
1154 Graphic character sets are classified into one of the following
1155 four classes, according to the number of bytes (DIMENSION) and
1156 number of characters in one dimension (CHARS) of the set:
1157 - DIMENSION1_CHARS94
1158 - DIMENSION1_CHARS96
1159 - DIMENSION2_CHARS94
1160 - DIMENSION2_CHARS96
1162 In addition, each character set is assigned an identification tag,
1163 unique for each set, called the "final character" (denoted as <F>
1164 hereafter). The <F> of each character set is decided by ECMA(*)
1165 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1166 (0x30..0x3F are for private use only).
1168 Note (*): ECMA = European Computer Manufacturers Association
1170 Here are examples of graphic character sets [NAME(<F>)]:
1171 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1172 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1173 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1174 o DIMENSION2_CHARS96 -- none for the moment
1176 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1177 C0 [0x00..0x1F] -- control character plane 0
1178 GL [0x20..0x7F] -- graphic character plane 0
1179 C1 [0x80..0x9F] -- control character plane 1
1180 GR [0xA0..0xFF] -- graphic character plane 1
1182 A control character set is directly designated and invoked to C0 or
1183 C1 by an escape sequence. The most common case is that:
1184 - ISO646's control character set is designated/invoked to C0, and
1185 - ISO6429's control character set is designated/invoked to C1,
1186 and usually these designations/invocations are omitted in encoded
1187 text. In a 7-bit environment, only C0 can be used, and a control
1188 character for C1 is encoded by an appropriate escape sequence to
1189 fit into the environment. All control characters for C1 are
1190 defined to have corresponding escape sequences.
1192 A graphic character set is at first designated to one of four
1193 graphic registers (G0 through G3), then these graphic registers are
1194 invoked to GL or GR. These designations and invocations can be
1195 done independently. The most common case is that G0 is invoked to
1196 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1197 these invocations and designations are omitted in encoded text.
1198 In a 7-bit environment, only GL can be used.
1200 When a graphic character set of CHARS94 is invoked to GL, codes
1201 0x20 and 0x7F of the GL area work as control characters SPACE and
1202 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1205 There are two ways of invocation: locking-shift and single-shift.
1206 With locking-shift, the invocation lasts until the next different
1207 invocation, whereas with single-shift, the invocation affects the
1208 following character only and doesn't affect the locking-shift
1209 state. Invocations are done by the following control characters or
1212 ----------------------------------------------------------------------
1213 abbrev function cntrl escape seq description
1214 ----------------------------------------------------------------------
1215 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1216 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1217 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1218 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1219 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1220 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1221 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1222 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1223 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1224 ----------------------------------------------------------------------
1225 (*) These are not used by any known coding system.
1227 Control characters for these functions are defined by macros
1228 ISO_CODE_XXX in `coding.h'.
1230 Designations are done by the following escape sequences:
1231 ----------------------------------------------------------------------
1232 escape sequence description
1233 ----------------------------------------------------------------------
1234 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1235 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1236 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1237 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1238 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1239 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1240 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1241 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1242 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1243 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1244 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1245 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1246 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1247 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1248 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1249 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1250 ----------------------------------------------------------------------
1252 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1253 of dimension 1, chars 94, and final character <F>, etc...
1255 Note (*): Although these designations are not allowed in ISO2022,
1256 Emacs accepts them on decoding, and produces them on encoding
1257 CHARS96 character sets in a coding system which is characterized as
1258 7-bit environment, non-locking-shift, and non-single-shift.
1260 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1261 '(' can be omitted. We refer to this as "short-form" hereafter.
1263 Now you may notice that there are a lot of ways of encoding the
1264 same multilingual text in ISO2022. Actually, there exist many
1265 coding systems such as Compound Text (used in X11's inter client
1266 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1267 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1268 localized platforms), and all of these are variants of ISO2022.
1270 In addition to the above, Emacs handles two more kinds of escape
1271 sequences: ISO6429's direction specification and Emacs' private
1272 sequence for specifying character composition.
1274 ISO6429's direction specification takes the following form:
1275 o CSI ']' -- end of the current direction
1276 o CSI '0' ']' -- end of the current direction
1277 o CSI '1' ']' -- start of left-to-right text
1278 o CSI '2' ']' -- start of right-to-left text
1279 The control character CSI (0x9B: control sequence introducer) is
1280 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1282 Character composition specification takes the following form:
1283 o ESC '0' -- start relative composition
1284 o ESC '1' -- end composition
1285 o ESC '2' -- start rule-base composition (*)
1286 o ESC '3' -- start relative composition with alternate chars (**)
1287 o ESC '4' -- start rule-base composition with alternate chars (**)
1288 Since these are not standard escape sequences of any ISO standard,
1289 the use of them with these meanings is restricted to Emacs only.
1291 (*) This form is used only in Emacs 20.5 and older versions,
1292 but the newer versions can safely decode it.
1293 (**) This form is used only in Emacs 21.1 and newer versions,
1294 and the older versions can't decode it.
1296 Here's a list of example usages of these composition escape
1297 sequences (categorized by `enum composition_method').
1299 COMPOSITION_RELATIVE:
1300 ESC 0 CHAR [ CHAR ] ESC 1
1301 COMPOSITION_WITH_RULE:
1302 ESC 2 CHAR [ RULE CHAR ] ESC 1
1303 COMPOSITION_WITH_ALTCHARS:
1304 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1305 COMPOSITION_WITH_RULE_ALTCHARS:
1306 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1308 enum iso_code_class_type iso_code_class
[256];
1310 #define CHARSET_OK(idx, charset, c) \
1311 (coding_system_table[idx] \
1312 && (charset == CHARSET_ASCII \
1313 || (safe_chars = coding_safe_chars (coding_system_table[idx]), \
1314 CODING_SAFE_CHAR_P (safe_chars, c))) \
1315 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1317 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1319 #define SHIFT_OUT_OK(idx) \
1320 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1322 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1323 Check if a text is encoded in ISO2022. If it is, return an
1324 integer in which appropriate flag bits any of:
1325 CODING_CATEGORY_MASK_ISO_7
1326 CODING_CATEGORY_MASK_ISO_7_TIGHT
1327 CODING_CATEGORY_MASK_ISO_8_1
1328 CODING_CATEGORY_MASK_ISO_8_2
1329 CODING_CATEGORY_MASK_ISO_7_ELSE
1330 CODING_CATEGORY_MASK_ISO_8_ELSE
1331 are set. If a code which should never appear in ISO2022 is found,
1335 detect_coding_iso2022 (src
, src_end
, multibytep
)
1336 unsigned char *src
, *src_end
;
1339 int mask
= CODING_CATEGORY_MASK_ISO
;
1341 int reg
[4], shift_out
= 0, single_shifting
= 0;
1343 /* Dummy for ONE_MORE_BYTE. */
1344 struct coding_system dummy_coding
;
1345 struct coding_system
*coding
= &dummy_coding
;
1346 Lisp_Object safe_chars
;
1348 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1349 while (mask
&& src
< src_end
)
1351 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1355 if (inhibit_iso_escape_detection
)
1357 single_shifting
= 0;
1358 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1359 if (c
>= '(' && c
<= '/')
1361 /* Designation sequence for a charset of dimension 1. */
1362 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1363 if (c1
< ' ' || c1
>= 0x80
1364 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1365 /* Invalid designation sequence. Just ignore. */
1367 reg
[(c
- '(') % 4] = charset
;
1371 /* Designation sequence for a charset of dimension 2. */
1372 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1373 if (c
>= '@' && c
<= 'B')
1374 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1375 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1376 else if (c
>= '(' && c
<= '/')
1378 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1379 if (c1
< ' ' || c1
>= 0x80
1380 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1381 /* Invalid designation sequence. Just ignore. */
1383 reg
[(c
- '(') % 4] = charset
;
1386 /* Invalid designation sequence. Just ignore. */
1389 else if (c
== 'N' || c
== 'O')
1391 /* ESC <Fe> for SS2 or SS3. */
1392 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1395 else if (c
>= '0' && c
<= '4')
1397 /* ESC <Fp> for start/end composition. */
1398 mask_found
|= CODING_CATEGORY_MASK_ISO
;
1402 /* Invalid escape sequence. Just ignore. */
1405 /* We found a valid designation sequence for CHARSET. */
1406 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1407 c
= MAKE_CHAR (charset
, 0, 0);
1408 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1409 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1411 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1412 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1413 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1415 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1416 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1417 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1419 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1420 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1421 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1423 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1427 if (inhibit_iso_escape_detection
)
1429 single_shifting
= 0;
1432 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1433 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1435 /* Locking shift out. */
1436 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1437 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1442 if (inhibit_iso_escape_detection
)
1444 single_shifting
= 0;
1447 /* Locking shift in. */
1448 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1449 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1454 single_shifting
= 0;
1458 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1460 if (inhibit_iso_escape_detection
)
1462 if (c
!= ISO_CODE_CSI
)
1464 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1465 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1466 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1467 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1468 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1469 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1470 single_shifting
= 1;
1472 if (VECTORP (Vlatin_extra_code_table
)
1473 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1475 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1476 & CODING_FLAG_ISO_LATIN_EXTRA
)
1477 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1478 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1479 & CODING_FLAG_ISO_LATIN_EXTRA
)
1480 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1483 mask_found
|= newmask
;
1490 single_shifting
= 0;
1495 single_shifting
= 0;
1496 if (VECTORP (Vlatin_extra_code_table
)
1497 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1501 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1502 & CODING_FLAG_ISO_LATIN_EXTRA
)
1503 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1504 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1505 & CODING_FLAG_ISO_LATIN_EXTRA
)
1506 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1508 mask_found
|= newmask
;
1515 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1516 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1517 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1518 /* Check the length of succeeding codes of the range
1519 0xA0..0FF. If the byte length is odd, we exclude
1520 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1521 when we are not single shifting. */
1522 if (!single_shifting
1523 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1526 while (src
< src_end
)
1528 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1534 if (i
& 1 && src
< src_end
)
1535 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1537 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1544 return (mask
& mask_found
);
1547 /* Decode a character of which charset is CHARSET, the 1st position
1548 code is C1, the 2nd position code is C2, and return the decoded
1549 character code. If the variable `translation_table' is non-nil,
1550 returned the translated code. */
1552 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1553 (NILP (translation_table) \
1554 ? MAKE_CHAR (charset, c1, c2) \
1555 : translate_char (translation_table, -1, charset, c1, c2))
1557 /* Set designation state into CODING. */
1558 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1562 if (final_char < '0' || final_char >= 128) \
1563 goto label_invalid_code; \
1564 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1565 make_number (chars), \
1566 make_number (final_char)); \
1567 c = MAKE_CHAR (charset, 0, 0); \
1569 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1570 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1572 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1574 && charset == CHARSET_ASCII) \
1576 /* We should insert this designation sequence as is so \
1577 that it is surely written back to a file. */ \
1578 coding->spec.iso2022.last_invalid_designation_register = -1; \
1579 goto label_invalid_code; \
1581 coding->spec.iso2022.last_invalid_designation_register = -1; \
1582 if ((coding->mode & CODING_MODE_DIRECTION) \
1583 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1584 charset = CHARSET_REVERSE_CHARSET (charset); \
1585 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1589 coding->spec.iso2022.last_invalid_designation_register = reg; \
1590 goto label_invalid_code; \
1594 /* Allocate a memory block for storing information about compositions.
1595 The block is chained to the already allocated blocks. */
1598 coding_allocate_composition_data (coding
, char_offset
)
1599 struct coding_system
*coding
;
1602 struct composition_data
*cmp_data
1603 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1605 cmp_data
->char_offset
= char_offset
;
1607 cmp_data
->prev
= coding
->cmp_data
;
1608 cmp_data
->next
= NULL
;
1609 if (coding
->cmp_data
)
1610 coding
->cmp_data
->next
= cmp_data
;
1611 coding
->cmp_data
= cmp_data
;
1612 coding
->cmp_data_start
= 0;
1615 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1616 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1617 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1618 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1619 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1622 #define DECODE_COMPOSITION_START(c1) \
1624 if (coding->composing == COMPOSITION_DISABLED) \
1626 *dst++ = ISO_CODE_ESC; \
1627 *dst++ = c1 & 0x7f; \
1628 coding->produced_char += 2; \
1630 else if (!COMPOSING_P (coding)) \
1632 /* This is surely the start of a composition. We must be sure \
1633 that coding->cmp_data has enough space to store the \
1634 information about the composition. If not, terminate the \
1635 current decoding loop, allocate one more memory block for \
1636 coding->cmp_data in the caller, then start the decoding \
1637 loop again. We can't allocate memory here directly because \
1638 it may cause buffer/string relocation. */ \
1639 if (!coding->cmp_data \
1640 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1641 >= COMPOSITION_DATA_SIZE)) \
1643 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1644 goto label_end_of_loop; \
1646 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1647 : c1 == '2' ? COMPOSITION_WITH_RULE \
1648 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1649 : COMPOSITION_WITH_RULE_ALTCHARS); \
1650 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1651 coding->composing); \
1652 coding->composition_rule_follows = 0; \
1656 /* We are already handling a composition. If the method is \
1657 the following two, the codes following the current escape \
1658 sequence are actual characters stored in a buffer. */ \
1659 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1660 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1662 coding->composing = COMPOSITION_RELATIVE; \
1663 coding->composition_rule_follows = 0; \
1668 /* Handle composition end sequence ESC 1. */
1670 #define DECODE_COMPOSITION_END(c1) \
1672 if (! COMPOSING_P (coding)) \
1674 *dst++ = ISO_CODE_ESC; \
1676 coding->produced_char += 2; \
1680 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1681 coding->composing = COMPOSITION_NO; \
1685 /* Decode a composition rule from the byte C1 (and maybe one more byte
1686 from SRC) and store one encoded composition rule in
1687 coding->cmp_data. */
1689 #define DECODE_COMPOSITION_RULE(c1) \
1693 if (c1 < 81) /* old format (before ver.21) */ \
1695 int gref = (c1) / 9; \
1696 int nref = (c1) % 9; \
1697 if (gref == 4) gref = 10; \
1698 if (nref == 4) nref = 10; \
1699 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1701 else if (c1 < 93) /* new format (after ver.21) */ \
1703 ONE_MORE_BYTE (c2); \
1704 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1706 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1707 coding->composition_rule_follows = 0; \
1711 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1714 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1715 struct coding_system
*coding
;
1716 unsigned char *source
, *destination
;
1717 int src_bytes
, dst_bytes
;
1719 unsigned char *src
= source
;
1720 unsigned char *src_end
= source
+ src_bytes
;
1721 unsigned char *dst
= destination
;
1722 unsigned char *dst_end
= destination
+ dst_bytes
;
1723 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1724 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1725 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1726 /* SRC_BASE remembers the start position in source in each loop.
1727 The loop will be exited when there's not enough source code
1728 (within macro ONE_MORE_BYTE), or when there's not enough
1729 destination area to produce a character (within macro
1731 unsigned char *src_base
;
1733 Lisp_Object translation_table
;
1734 Lisp_Object safe_chars
;
1736 safe_chars
= coding_safe_chars (coding
);
1738 if (NILP (Venable_character_translation
))
1739 translation_table
= Qnil
;
1742 translation_table
= coding
->translation_table_for_decode
;
1743 if (NILP (translation_table
))
1744 translation_table
= Vstandard_translation_table_for_decode
;
1747 coding
->result
= CODING_FINISH_NORMAL
;
1756 /* We produce no character or one character. */
1757 switch (iso_code_class
[c1
])
1759 case ISO_0x20_or_0x7F
:
1760 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1762 DECODE_COMPOSITION_RULE (c1
);
1765 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1767 /* This is SPACE or DEL. */
1768 charset
= CHARSET_ASCII
;
1771 /* This is a graphic character, we fall down ... */
1773 case ISO_graphic_plane_0
:
1774 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1776 DECODE_COMPOSITION_RULE (c1
);
1782 case ISO_0xA0_or_0xFF
:
1783 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1784 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1785 goto label_invalid_code
;
1786 /* This is a graphic character, we fall down ... */
1788 case ISO_graphic_plane_1
:
1790 goto label_invalid_code
;
1795 if (COMPOSING_P (coding
))
1796 DECODE_COMPOSITION_END ('1');
1798 /* All ISO2022 control characters in this class have the
1799 same representation in Emacs internal format. */
1801 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1802 && (coding
->eol_type
== CODING_EOL_CR
1803 || coding
->eol_type
== CODING_EOL_CRLF
))
1805 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1806 goto label_end_of_loop
;
1808 charset
= CHARSET_ASCII
;
1812 if (COMPOSING_P (coding
))
1813 DECODE_COMPOSITION_END ('1');
1814 goto label_invalid_code
;
1816 case ISO_carriage_return
:
1817 if (COMPOSING_P (coding
))
1818 DECODE_COMPOSITION_END ('1');
1820 if (coding
->eol_type
== CODING_EOL_CR
)
1822 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1825 if (c1
!= ISO_CODE_LF
)
1827 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1829 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1830 goto label_end_of_loop
;
1836 charset
= CHARSET_ASCII
;
1840 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1841 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1842 goto label_invalid_code
;
1843 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1844 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1848 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1849 goto label_invalid_code
;
1850 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1851 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1854 case ISO_single_shift_2_7
:
1855 case ISO_single_shift_2
:
1856 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1857 goto label_invalid_code
;
1858 /* SS2 is handled as an escape sequence of ESC 'N' */
1860 goto label_escape_sequence
;
1862 case ISO_single_shift_3
:
1863 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1864 goto label_invalid_code
;
1865 /* SS2 is handled as an escape sequence of ESC 'O' */
1867 goto label_escape_sequence
;
1869 case ISO_control_sequence_introducer
:
1870 /* CSI is handled as an escape sequence of ESC '[' ... */
1872 goto label_escape_sequence
;
1876 label_escape_sequence
:
1877 /* Escape sequences handled by Emacs are invocation,
1878 designation, direction specification, and character
1879 composition specification. */
1882 case '&': /* revision of following character set */
1884 if (!(c1
>= '@' && c1
<= '~'))
1885 goto label_invalid_code
;
1887 if (c1
!= ISO_CODE_ESC
)
1888 goto label_invalid_code
;
1890 goto label_escape_sequence
;
1892 case '$': /* designation of 2-byte character set */
1893 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1894 goto label_invalid_code
;
1896 if (c1
>= '@' && c1
<= 'B')
1897 { /* designation of JISX0208.1978, GB2312.1980,
1899 DECODE_DESIGNATION (0, 2, 94, c1
);
1901 else if (c1
>= 0x28 && c1
<= 0x2B)
1902 { /* designation of DIMENSION2_CHARS94 character set */
1904 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
1906 else if (c1
>= 0x2C && c1
<= 0x2F)
1907 { /* designation of DIMENSION2_CHARS96 character set */
1909 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
1912 goto label_invalid_code
;
1913 /* We must update these variables now. */
1914 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1915 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1918 case 'n': /* invocation of locking-shift-2 */
1919 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1920 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
1921 goto label_invalid_code
;
1922 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
1923 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1926 case 'o': /* invocation of locking-shift-3 */
1927 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1928 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
1929 goto label_invalid_code
;
1930 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
1931 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1934 case 'N': /* invocation of single-shift-2 */
1935 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1936 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
1937 goto label_invalid_code
;
1938 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
1940 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
1941 goto label_invalid_code
;
1944 case 'O': /* invocation of single-shift-3 */
1945 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1946 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
1947 goto label_invalid_code
;
1948 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
1950 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
1951 goto label_invalid_code
;
1954 case '0': case '2': case '3': case '4': /* start composition */
1955 DECODE_COMPOSITION_START (c1
);
1958 case '1': /* end composition */
1959 DECODE_COMPOSITION_END (c1
);
1962 case '[': /* specification of direction */
1963 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
1964 goto label_invalid_code
;
1965 /* For the moment, nested direction is not supported.
1966 So, `coding->mode & CODING_MODE_DIRECTION' zero means
1967 left-to-right, and nonzero means right-to-left. */
1971 case ']': /* end of the current direction */
1972 coding
->mode
&= ~CODING_MODE_DIRECTION
;
1974 case '0': /* end of the current direction */
1975 case '1': /* start of left-to-right direction */
1978 coding
->mode
&= ~CODING_MODE_DIRECTION
;
1980 goto label_invalid_code
;
1983 case '2': /* start of right-to-left direction */
1986 coding
->mode
|= CODING_MODE_DIRECTION
;
1988 goto label_invalid_code
;
1992 goto label_invalid_code
;
1997 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1998 goto label_invalid_code
;
1999 if (c1
>= 0x28 && c1
<= 0x2B)
2000 { /* designation of DIMENSION1_CHARS94 character set */
2002 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2004 else if (c1
>= 0x2C && c1
<= 0x2F)
2005 { /* designation of DIMENSION1_CHARS96 character set */
2007 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2010 goto label_invalid_code
;
2011 /* We must update these variables now. */
2012 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2013 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2018 /* Now we know CHARSET and 1st position code C1 of a character.
2019 Produce a multibyte sequence for that character while getting
2020 2nd position code C2 if necessary. */
2021 if (CHARSET_DIMENSION (charset
) == 2)
2024 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2025 /* C2 is not in a valid range. */
2026 goto label_invalid_code
;
2028 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2034 if (COMPOSING_P (coding
))
2035 DECODE_COMPOSITION_END ('1');
2042 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2043 coding
->produced
= dst
- destination
;
2048 /* ISO2022 encoding stuff. */
2051 It is not enough to say just "ISO2022" on encoding, we have to
2052 specify more details. In Emacs, each ISO2022 coding system
2053 variant has the following specifications:
2054 1. Initial designation to G0 through G3.
2055 2. Allows short-form designation?
2056 3. ASCII should be designated to G0 before control characters?
2057 4. ASCII should be designated to G0 at end of line?
2058 5. 7-bit environment or 8-bit environment?
2059 6. Use locking-shift?
2060 7. Use Single-shift?
2061 And the following two are only for Japanese:
2062 8. Use ASCII in place of JIS0201-1976-Roman?
2063 9. Use JISX0208-1983 in place of JISX0208-1978?
2064 These specifications are encoded in `coding->flags' as flag bits
2065 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2069 /* Produce codes (escape sequence) for designating CHARSET to graphic
2070 register REG at DST, and increment DST. If <final-char> of CHARSET is
2071 '@', 'A', or 'B' and the coding system CODING allows, produce
2072 designation sequence of short-form. */
2074 #define ENCODE_DESIGNATION(charset, reg, coding) \
2076 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2077 char *intermediate_char_94 = "()*+"; \
2078 char *intermediate_char_96 = ",-./"; \
2079 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2081 if (revision < 255) \
2083 *dst++ = ISO_CODE_ESC; \
2085 *dst++ = '@' + revision; \
2087 *dst++ = ISO_CODE_ESC; \
2088 if (CHARSET_DIMENSION (charset) == 1) \
2090 if (CHARSET_CHARS (charset) == 94) \
2091 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2093 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2098 if (CHARSET_CHARS (charset) == 94) \
2100 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2102 || final_char < '@' || final_char > 'B') \
2103 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2106 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2108 *dst++ = final_char; \
2109 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2112 /* The following two macros produce codes (control character or escape
2113 sequence) for ISO2022 single-shift functions (single-shift-2 and
2116 #define ENCODE_SINGLE_SHIFT_2 \
2118 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2119 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2121 *dst++ = ISO_CODE_SS2; \
2122 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2125 #define ENCODE_SINGLE_SHIFT_3 \
2127 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2128 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2130 *dst++ = ISO_CODE_SS3; \
2131 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2134 /* The following four macros produce codes (control character or
2135 escape sequence) for ISO2022 locking-shift functions (shift-in,
2136 shift-out, locking-shift-2, and locking-shift-3). */
2138 #define ENCODE_SHIFT_IN \
2140 *dst++ = ISO_CODE_SI; \
2141 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2144 #define ENCODE_SHIFT_OUT \
2146 *dst++ = ISO_CODE_SO; \
2147 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2150 #define ENCODE_LOCKING_SHIFT_2 \
2152 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2153 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2156 #define ENCODE_LOCKING_SHIFT_3 \
2158 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2159 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2162 /* Produce codes for a DIMENSION1 character whose character set is
2163 CHARSET and whose position-code is C1. Designation and invocation
2164 sequences are also produced in advance if necessary. */
2166 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2168 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2170 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2171 *dst++ = c1 & 0x7F; \
2173 *dst++ = c1 | 0x80; \
2174 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2177 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2179 *dst++ = c1 & 0x7F; \
2182 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2184 *dst++ = c1 | 0x80; \
2188 /* Since CHARSET is not yet invoked to any graphic planes, we \
2189 must invoke it, or, at first, designate it to some graphic \
2190 register. Then repeat the loop to actually produce the \
2192 dst = encode_invocation_designation (charset, coding, dst); \
2195 /* Produce codes for a DIMENSION2 character whose character set is
2196 CHARSET and whose position-codes are C1 and C2. Designation and
2197 invocation codes are also produced in advance if necessary. */
2199 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2201 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2203 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2204 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2206 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2207 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2210 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2212 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2215 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2217 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2221 /* Since CHARSET is not yet invoked to any graphic planes, we \
2222 must invoke it, or, at first, designate it to some graphic \
2223 register. Then repeat the loop to actually produce the \
2225 dst = encode_invocation_designation (charset, coding, dst); \
2228 #define ENCODE_ISO_CHARACTER(c) \
2230 int charset, c1, c2; \
2232 SPLIT_CHAR (c, charset, c1, c2); \
2233 if (CHARSET_DEFINED_P (charset)) \
2235 if (CHARSET_DIMENSION (charset) == 1) \
2237 if (charset == CHARSET_ASCII \
2238 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2239 charset = charset_latin_jisx0201; \
2240 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2244 if (charset == charset_jisx0208 \
2245 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2246 charset = charset_jisx0208_1978; \
2247 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2259 /* Instead of encoding character C, produce one or two `?'s. */
2261 #define ENCODE_UNSAFE_CHARACTER(c) \
2263 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \
2264 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2265 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \
2269 /* Produce designation and invocation codes at a place pointed by DST
2270 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2274 encode_invocation_designation (charset
, coding
, dst
)
2276 struct coding_system
*coding
;
2279 int reg
; /* graphic register number */
2281 /* At first, check designations. */
2282 for (reg
= 0; reg
< 4; reg
++)
2283 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2288 /* CHARSET is not yet designated to any graphic registers. */
2289 /* At first check the requested designation. */
2290 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2291 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2292 /* Since CHARSET requests no special designation, designate it
2293 to graphic register 0. */
2296 ENCODE_DESIGNATION (charset
, reg
, coding
);
2299 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2300 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2302 /* Since the graphic register REG is not invoked to any graphic
2303 planes, invoke it to graphic plane 0. */
2306 case 0: /* graphic register 0 */
2310 case 1: /* graphic register 1 */
2314 case 2: /* graphic register 2 */
2315 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2316 ENCODE_SINGLE_SHIFT_2
;
2318 ENCODE_LOCKING_SHIFT_2
;
2321 case 3: /* graphic register 3 */
2322 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2323 ENCODE_SINGLE_SHIFT_3
;
2325 ENCODE_LOCKING_SHIFT_3
;
2333 /* Produce 2-byte codes for encoded composition rule RULE. */
2335 #define ENCODE_COMPOSITION_RULE(rule) \
2338 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2339 *dst++ = 32 + 81 + gref; \
2340 *dst++ = 32 + nref; \
2343 /* Produce codes for indicating the start of a composition sequence
2344 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2345 which specify information about the composition. See the comment
2346 in coding.h for the format of DATA. */
2348 #define ENCODE_COMPOSITION_START(coding, data) \
2350 coding->composing = data[3]; \
2351 *dst++ = ISO_CODE_ESC; \
2352 if (coding->composing == COMPOSITION_RELATIVE) \
2356 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2358 coding->cmp_data_index = coding->cmp_data_start + 4; \
2359 coding->composition_rule_follows = 0; \
2363 /* Produce codes for indicating the end of the current composition. */
2365 #define ENCODE_COMPOSITION_END(coding, data) \
2367 *dst++ = ISO_CODE_ESC; \
2369 coding->cmp_data_start += data[0]; \
2370 coding->composing = COMPOSITION_NO; \
2371 if (coding->cmp_data_start == coding->cmp_data->used \
2372 && coding->cmp_data->next) \
2374 coding->cmp_data = coding->cmp_data->next; \
2375 coding->cmp_data_start = 0; \
2379 /* Produce composition start sequence ESC 0. Here, this sequence
2380 doesn't mean the start of a new composition but means that we have
2381 just produced components (alternate chars and composition rules) of
2382 the composition and the actual text follows in SRC. */
2384 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2386 *dst++ = ISO_CODE_ESC; \
2388 coding->composing = COMPOSITION_RELATIVE; \
2391 /* The following three macros produce codes for indicating direction
2393 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2395 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2396 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2398 *dst++ = ISO_CODE_CSI; \
2401 #define ENCODE_DIRECTION_R2L \
2402 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2404 #define ENCODE_DIRECTION_L2R \
2405 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2407 /* Produce codes for designation and invocation to reset the graphic
2408 planes and registers to initial state. */
2409 #define ENCODE_RESET_PLANE_AND_REGISTER \
2412 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2414 for (reg = 0; reg < 4; reg++) \
2415 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2416 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2417 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2418 ENCODE_DESIGNATION \
2419 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2422 /* Produce designation sequences of charsets in the line started from
2423 SRC to a place pointed by DST, and return updated DST.
2425 If the current block ends before any end-of-line, we may fail to
2426 find all the necessary designations. */
2428 static unsigned char *
2429 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2430 struct coding_system
*coding
;
2431 Lisp_Object translation_table
;
2432 unsigned char *src
, *src_end
, *dst
;
2434 int charset
, c
, found
= 0, reg
;
2435 /* Table of charsets to be designated to each graphic register. */
2438 for (reg
= 0; reg
< 4; reg
++)
2447 charset
= CHAR_CHARSET (c
);
2448 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2449 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2459 for (reg
= 0; reg
< 4; reg
++)
2461 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2462 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2468 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2471 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2472 struct coding_system
*coding
;
2473 unsigned char *source
, *destination
;
2474 int src_bytes
, dst_bytes
;
2476 unsigned char *src
= source
;
2477 unsigned char *src_end
= source
+ src_bytes
;
2478 unsigned char *dst
= destination
;
2479 unsigned char *dst_end
= destination
+ dst_bytes
;
2480 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2481 from DST_END to assure overflow checking is necessary only at the
2483 unsigned char *adjusted_dst_end
= dst_end
- 19;
2484 /* SRC_BASE remembers the start position in source in each loop.
2485 The loop will be exited when there's not enough source text to
2486 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2487 there's not enough destination area to produce encoded codes
2488 (within macro EMIT_BYTES). */
2489 unsigned char *src_base
;
2491 Lisp_Object translation_table
;
2492 Lisp_Object safe_chars
;
2494 safe_chars
= coding_safe_chars (coding
);
2496 if (NILP (Venable_character_translation
))
2497 translation_table
= Qnil
;
2500 translation_table
= coding
->translation_table_for_encode
;
2501 if (NILP (translation_table
))
2502 translation_table
= Vstandard_translation_table_for_encode
;
2505 coding
->consumed_char
= 0;
2511 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2513 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2517 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2518 && CODING_SPEC_ISO_BOL (coding
))
2520 /* We have to produce designation sequences if any now. */
2521 dst
= encode_designation_at_bol (coding
, translation_table
,
2523 CODING_SPEC_ISO_BOL (coding
) = 0;
2526 /* Check composition start and end. */
2527 if (coding
->composing
!= COMPOSITION_DISABLED
2528 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2530 struct composition_data
*cmp_data
= coding
->cmp_data
;
2531 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2532 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2534 if (coding
->composing
== COMPOSITION_RELATIVE
)
2536 if (this_pos
== data
[2])
2538 ENCODE_COMPOSITION_END (coding
, data
);
2539 cmp_data
= coding
->cmp_data
;
2540 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2543 else if (COMPOSING_P (coding
))
2545 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2546 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2547 /* We have consumed components of the composition.
2548 What follows in SRC is the composition's base
2550 ENCODE_COMPOSITION_FAKE_START (coding
);
2553 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2554 if (coding
->composition_rule_follows
)
2556 ENCODE_COMPOSITION_RULE (c
);
2557 coding
->composition_rule_follows
= 0;
2561 if (coding
->flags
& CODING_FLAG_ISO_SAFE
2562 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2563 ENCODE_UNSAFE_CHARACTER (c
);
2565 ENCODE_ISO_CHARACTER (c
);
2566 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2567 coding
->composition_rule_follows
= 1;
2572 if (!COMPOSING_P (coding
))
2574 if (this_pos
== data
[1])
2576 ENCODE_COMPOSITION_START (coding
, data
);
2584 /* Now encode the character C. */
2585 if (c
< 0x20 || c
== 0x7F)
2589 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2591 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2592 ENCODE_RESET_PLANE_AND_REGISTER
;
2596 /* fall down to treat '\r' as '\n' ... */
2601 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2602 ENCODE_RESET_PLANE_AND_REGISTER
;
2603 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2604 bcopy (coding
->spec
.iso2022
.initial_designation
,
2605 coding
->spec
.iso2022
.current_designation
,
2606 sizeof coding
->spec
.iso2022
.initial_designation
);
2607 if (coding
->eol_type
== CODING_EOL_LF
2608 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2609 *dst
++ = ISO_CODE_LF
;
2610 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2611 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2613 *dst
++ = ISO_CODE_CR
;
2614 CODING_SPEC_ISO_BOL (coding
) = 1;
2618 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2619 ENCODE_RESET_PLANE_AND_REGISTER
;
2623 else if (ASCII_BYTE_P (c
))
2624 ENCODE_ISO_CHARACTER (c
);
2625 else if (SINGLE_BYTE_CHAR_P (c
))
2630 else if (coding
->flags
& CODING_FLAG_ISO_SAFE
2631 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2632 ENCODE_UNSAFE_CHARACTER (c
);
2634 ENCODE_ISO_CHARACTER (c
);
2636 coding
->consumed_char
++;
2640 coding
->consumed
= src_base
- source
;
2641 coding
->produced
= coding
->produced_char
= dst
- destination
;
2645 /*** 4. SJIS and BIG5 handlers ***/
2647 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2648 quite widely. So, for the moment, Emacs supports them in the bare
2649 C code. But, in the future, they may be supported only by CCL. */
2651 /* SJIS is a coding system encoding three character sets: ASCII, right
2652 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2653 as is. A character of charset katakana-jisx0201 is encoded by
2654 "position-code + 0x80". A character of charset japanese-jisx0208
2655 is encoded in 2-byte but two position-codes are divided and shifted
2656 so that it fits in the range below.
2658 --- CODE RANGE of SJIS ---
2659 (character set) (range)
2661 KATAKANA-JISX0201 0xA1 .. 0xDF
2662 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2663 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2664 -------------------------------
2668 /* BIG5 is a coding system encoding two character sets: ASCII and
2669 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2670 character set and is encoded in two bytes.
2672 --- CODE RANGE of BIG5 ---
2673 (character set) (range)
2675 Big5 (1st byte) 0xA1 .. 0xFE
2676 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2677 --------------------------
2679 Since the number of characters in Big5 is larger than maximum
2680 characters in Emacs' charset (96x96), it can't be handled as one
2681 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2682 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2683 contains frequently used characters and the latter contains less
2684 frequently used characters. */
2686 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2687 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2688 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2689 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2691 /* Number of Big5 characters which have the same code in 1st byte. */
2692 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2694 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2697 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2699 charset = charset_big5_1; \
2702 charset = charset_big5_2; \
2703 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2705 c1 = temp / (0xFF - 0xA1) + 0x21; \
2706 c2 = temp % (0xFF - 0xA1) + 0x21; \
2709 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2711 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2712 if (charset == charset_big5_2) \
2713 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2714 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2715 b2 = temp % BIG5_SAME_ROW; \
2716 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2719 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2720 Check if a text is encoded in SJIS. If it is, return
2721 CODING_CATEGORY_MASK_SJIS, else return 0. */
2724 detect_coding_sjis (src
, src_end
, multibytep
)
2725 unsigned char *src
, *src_end
;
2729 /* Dummy for ONE_MORE_BYTE. */
2730 struct coding_system dummy_coding
;
2731 struct coding_system
*coding
= &dummy_coding
;
2735 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2738 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2740 if (c
<= 0x9F || c
>= 0xE0)
2742 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2743 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2748 return CODING_CATEGORY_MASK_SJIS
;
2751 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2752 Check if a text is encoded in BIG5. If it is, return
2753 CODING_CATEGORY_MASK_BIG5, else return 0. */
2756 detect_coding_big5 (src
, src_end
, multibytep
)
2757 unsigned char *src
, *src_end
;
2761 /* Dummy for ONE_MORE_BYTE. */
2762 struct coding_system dummy_coding
;
2763 struct coding_system
*coding
= &dummy_coding
;
2767 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2770 if (c
< 0xA1 || c
> 0xFE)
2772 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2773 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2777 return CODING_CATEGORY_MASK_BIG5
;
2780 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2781 Check if a text is encoded in UTF-8. If it is, return
2782 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2784 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2785 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2786 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2787 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2788 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2789 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2790 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2793 detect_coding_utf_8 (src
, src_end
, multibytep
)
2794 unsigned char *src
, *src_end
;
2798 int seq_maybe_bytes
;
2799 /* Dummy for ONE_MORE_BYTE. */
2800 struct coding_system dummy_coding
;
2801 struct coding_system
*coding
= &dummy_coding
;
2805 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2806 if (UTF_8_1_OCTET_P (c
))
2808 else if (UTF_8_2_OCTET_LEADING_P (c
))
2809 seq_maybe_bytes
= 1;
2810 else if (UTF_8_3_OCTET_LEADING_P (c
))
2811 seq_maybe_bytes
= 2;
2812 else if (UTF_8_4_OCTET_LEADING_P (c
))
2813 seq_maybe_bytes
= 3;
2814 else if (UTF_8_5_OCTET_LEADING_P (c
))
2815 seq_maybe_bytes
= 4;
2816 else if (UTF_8_6_OCTET_LEADING_P (c
))
2817 seq_maybe_bytes
= 5;
2823 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2824 if (!UTF_8_EXTRA_OCTET_P (c
))
2828 while (seq_maybe_bytes
> 0);
2832 return CODING_CATEGORY_MASK_UTF_8
;
2835 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2836 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
2837 Little Endian (otherwise). If it is, return
2838 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
2841 #define UTF_16_INVALID_P(val) \
2842 (((val) == 0xFFFE) \
2843 || ((val) == 0xFFFF))
2845 #define UTF_16_HIGH_SURROGATE_P(val) \
2846 (((val) & 0xD800) == 0xD800)
2848 #define UTF_16_LOW_SURROGATE_P(val) \
2849 (((val) & 0xDC00) == 0xDC00)
2852 detect_coding_utf_16 (src
, src_end
, multibytep
)
2853 unsigned char *src
, *src_end
;
2856 unsigned char c1
, c2
;
2857 /* Dummy for TWO_MORE_BYTES. */
2858 struct coding_system dummy_coding
;
2859 struct coding_system
*coding
= &dummy_coding
;
2861 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
2862 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
);
2864 if ((c1
== 0xFF) && (c2
== 0xFE))
2865 return CODING_CATEGORY_MASK_UTF_16_LE
;
2866 else if ((c1
== 0xFE) && (c2
== 0xFF))
2867 return CODING_CATEGORY_MASK_UTF_16_BE
;
2873 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
2874 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
2877 decode_coding_sjis_big5 (coding
, source
, destination
,
2878 src_bytes
, dst_bytes
, sjis_p
)
2879 struct coding_system
*coding
;
2880 unsigned char *source
, *destination
;
2881 int src_bytes
, dst_bytes
;
2884 unsigned char *src
= source
;
2885 unsigned char *src_end
= source
+ src_bytes
;
2886 unsigned char *dst
= destination
;
2887 unsigned char *dst_end
= destination
+ dst_bytes
;
2888 /* SRC_BASE remembers the start position in source in each loop.
2889 The loop will be exited when there's not enough source code
2890 (within macro ONE_MORE_BYTE), or when there's not enough
2891 destination area to produce a character (within macro
2893 unsigned char *src_base
;
2894 Lisp_Object translation_table
;
2896 if (NILP (Venable_character_translation
))
2897 translation_table
= Qnil
;
2900 translation_table
= coding
->translation_table_for_decode
;
2901 if (NILP (translation_table
))
2902 translation_table
= Vstandard_translation_table_for_decode
;
2905 coding
->produced_char
= 0;
2908 int c
, charset
, c1
, c2
;
2915 charset
= CHARSET_ASCII
;
2920 if (coding
->eol_type
== CODING_EOL_CRLF
)
2925 else if (coding
->mode
2926 & CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2928 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2929 goto label_end_of_loop
;
2932 /* To process C2 again, SRC is subtracted by 1. */
2935 else if (coding
->eol_type
== CODING_EOL_CR
)
2939 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2940 && (coding
->eol_type
== CODING_EOL_CR
2941 || coding
->eol_type
== CODING_EOL_CRLF
))
2943 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2944 goto label_end_of_loop
;
2952 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
2953 goto label_invalid_code
;
2954 if (c1
<= 0x9F || c1
>= 0xE0)
2956 /* SJIS -> JISX0208 */
2958 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
2959 goto label_invalid_code
;
2960 DECODE_SJIS (c1
, c2
, c1
, c2
);
2961 charset
= charset_jisx0208
;
2964 /* SJIS -> JISX0201-Kana */
2965 charset
= charset_katakana_jisx0201
;
2970 if (c1
< 0xA0 || c1
> 0xFE)
2971 goto label_invalid_code
;
2973 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
2974 goto label_invalid_code
;
2975 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
2979 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2991 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2992 coding
->produced
= dst
- destination
;
2996 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
2997 This function can encode charsets `ascii', `katakana-jisx0201',
2998 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
2999 are sure that all these charsets are registered as official charset
3000 (i.e. do not have extended leading-codes). Characters of other
3001 charsets are produced without any encoding. If SJIS_P is 1, encode
3002 SJIS text, else encode BIG5 text. */
3005 encode_coding_sjis_big5 (coding
, source
, destination
,
3006 src_bytes
, dst_bytes
, sjis_p
)
3007 struct coding_system
*coding
;
3008 unsigned char *source
, *destination
;
3009 int src_bytes
, dst_bytes
;
3012 unsigned char *src
= source
;
3013 unsigned char *src_end
= source
+ src_bytes
;
3014 unsigned char *dst
= destination
;
3015 unsigned char *dst_end
= destination
+ dst_bytes
;
3016 /* SRC_BASE remembers the start position in source in each loop.
3017 The loop will be exited when there's not enough source text to
3018 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3019 there's not enough destination area to produce encoded codes
3020 (within macro EMIT_BYTES). */
3021 unsigned char *src_base
;
3022 Lisp_Object translation_table
;
3024 if (NILP (Venable_character_translation
))
3025 translation_table
= Qnil
;
3028 translation_table
= coding
->translation_table_for_encode
;
3029 if (NILP (translation_table
))
3030 translation_table
= Vstandard_translation_table_for_encode
;
3035 int c
, charset
, c1
, c2
;
3040 /* Now encode the character C. */
3041 if (SINGLE_BYTE_CHAR_P (c
))
3046 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3053 if (coding
->eol_type
== CODING_EOL_CRLF
)
3055 EMIT_TWO_BYTES ('\r', c
);
3058 else if (coding
->eol_type
== CODING_EOL_CR
)
3066 SPLIT_CHAR (c
, charset
, c1
, c2
);
3069 if (charset
== charset_jisx0208
3070 || charset
== charset_jisx0208_1978
)
3072 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3073 EMIT_TWO_BYTES (c1
, c2
);
3075 else if (charset
== charset_katakana_jisx0201
)
3076 EMIT_ONE_BYTE (c1
| 0x80);
3077 else if (charset
== charset_latin_jisx0201
)
3080 /* There's no way other than producing the internal
3082 EMIT_BYTES (src_base
, src
);
3086 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3088 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3089 EMIT_TWO_BYTES (c1
, c2
);
3092 /* There's no way other than producing the internal
3094 EMIT_BYTES (src_base
, src
);
3097 coding
->consumed_char
++;
3101 coding
->consumed
= src_base
- source
;
3102 coding
->produced
= coding
->produced_char
= dst
- destination
;
3106 /*** 5. CCL handlers ***/
3108 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3109 Check if a text is encoded in a coding system of which
3110 encoder/decoder are written in CCL program. If it is, return
3111 CODING_CATEGORY_MASK_CCL, else return 0. */
3114 detect_coding_ccl (src
, src_end
, multibytep
)
3115 unsigned char *src
, *src_end
;
3118 unsigned char *valid
;
3120 /* Dummy for ONE_MORE_BYTE. */
3121 struct coding_system dummy_coding
;
3122 struct coding_system
*coding
= &dummy_coding
;
3124 /* No coding system is assigned to coding-category-ccl. */
3125 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3128 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3131 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3136 return CODING_CATEGORY_MASK_CCL
;
3140 /*** 6. End-of-line handlers ***/
3142 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3145 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3146 struct coding_system
*coding
;
3147 unsigned char *source
, *destination
;
3148 int src_bytes
, dst_bytes
;
3150 unsigned char *src
= source
;
3151 unsigned char *dst
= destination
;
3152 unsigned char *src_end
= src
+ src_bytes
;
3153 unsigned char *dst_end
= dst
+ dst_bytes
;
3154 Lisp_Object translation_table
;
3155 /* SRC_BASE remembers the start position in source in each loop.
3156 The loop will be exited when there's not enough source code
3157 (within macro ONE_MORE_BYTE), or when there's not enough
3158 destination area to produce a character (within macro
3160 unsigned char *src_base
;
3163 translation_table
= Qnil
;
3164 switch (coding
->eol_type
)
3166 case CODING_EOL_CRLF
:
3176 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3178 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3179 goto label_end_of_loop
;
3186 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3188 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3189 goto label_end_of_loop
;
3202 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3204 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3205 goto label_end_of_loop
;
3214 default: /* no need for EOL handling */
3224 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3225 coding
->produced
= dst
- destination
;
3229 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3230 format of end-of-line according to `coding->eol_type'. It also
3231 convert multibyte form 8-bit characters to unibyte if
3232 CODING->src_multibyte is nonzero. If `coding->mode &
3233 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3234 also means end-of-line. */
3237 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3238 struct coding_system
*coding
;
3239 const unsigned char *source
;
3240 unsigned char *destination
;
3241 int src_bytes
, dst_bytes
;
3243 const unsigned char *src
= source
;
3244 unsigned char *dst
= destination
;
3245 const unsigned char *src_end
= src
+ src_bytes
;
3246 unsigned char *dst_end
= dst
+ dst_bytes
;
3247 Lisp_Object translation_table
;
3248 /* SRC_BASE remembers the start position in source in each loop.
3249 The loop will be exited when there's not enough source text to
3250 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3251 there's not enough destination area to produce encoded codes
3252 (within macro EMIT_BYTES). */
3253 const unsigned char *src_base
;
3256 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3258 translation_table
= Qnil
;
3259 if (coding
->src_multibyte
3260 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3264 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3267 if (coding
->eol_type
== CODING_EOL_CRLF
)
3269 while (src
< src_end
)
3275 else if (c
== '\n' || (c
== '\r' && selective_display
))
3276 EMIT_TWO_BYTES ('\r', '\n');
3286 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3288 safe_bcopy (src
, dst
, src_bytes
);
3294 if (coding
->src_multibyte
3295 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3297 safe_bcopy (src
, dst
, dst_bytes
);
3298 src_base
= src
+ dst_bytes
;
3299 dst
= destination
+ dst_bytes
;
3300 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3302 if (coding
->eol_type
== CODING_EOL_CR
)
3304 for (tmp
= destination
; tmp
< dst
; tmp
++)
3305 if (*tmp
== '\n') *tmp
= '\r';
3307 else if (selective_display
)
3309 for (tmp
= destination
; tmp
< dst
; tmp
++)
3310 if (*tmp
== '\r') *tmp
= '\n';
3313 if (coding
->src_multibyte
)
3314 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3316 coding
->consumed
= src_base
- source
;
3317 coding
->produced
= dst
- destination
;
3318 coding
->produced_char
= coding
->produced
;
3322 /*** 7. C library functions ***/
3324 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3325 has a property `coding-system'. The value of this property is a
3326 vector of length 5 (called the coding-vector). Among elements of
3327 this vector, the first (element[0]) and the fifth (element[4])
3328 carry important information for decoding/encoding. Before
3329 decoding/encoding, this information should be set in fields of a
3330 structure of type `coding_system'.
3332 The value of the property `coding-system' can be a symbol of another
3333 subsidiary coding-system. In that case, Emacs gets coding-vector
3336 `element[0]' contains information to be set in `coding->type'. The
3337 value and its meaning is as follows:
3339 0 -- coding_type_emacs_mule
3340 1 -- coding_type_sjis
3341 2 -- coding_type_iso2022
3342 3 -- coding_type_big5
3343 4 -- coding_type_ccl encoder/decoder written in CCL
3344 nil -- coding_type_no_conversion
3345 t -- coding_type_undecided (automatic conversion on decoding,
3346 no-conversion on encoding)
3348 `element[4]' contains information to be set in `coding->flags' and
3349 `coding->spec'. The meaning varies by `coding->type'.
3351 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3352 of length 32 (of which the first 13 sub-elements are used now).
3353 Meanings of these sub-elements are:
3355 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3356 If the value is an integer of valid charset, the charset is
3357 assumed to be designated to graphic register N initially.
3359 If the value is minus, it is a minus value of charset which
3360 reserves graphic register N, which means that the charset is
3361 not designated initially but should be designated to graphic
3362 register N just before encoding a character in that charset.
3364 If the value is nil, graphic register N is never used on
3367 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3368 Each value takes t or nil. See the section ISO2022 of
3369 `coding.h' for more information.
3371 If `coding->type' is `coding_type_big5', element[4] is t to denote
3372 BIG5-ETen or nil to denote BIG5-HKU.
3374 If `coding->type' takes the other value, element[4] is ignored.
3376 Emacs Lisp's coding systems also carry information about format of
3377 end-of-line in a value of property `eol-type'. If the value is
3378 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3379 means CODING_EOL_CR. If it is not integer, it should be a vector
3380 of subsidiary coding systems of which property `eol-type' has one
3381 of the above values.
3385 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3386 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3387 is setup so that no conversion is necessary and return -1, else
3391 setup_coding_system (coding_system
, coding
)
3392 Lisp_Object coding_system
;
3393 struct coding_system
*coding
;
3395 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3398 /* At first, zero clear all members. */
3399 bzero (coding
, sizeof (struct coding_system
));
3401 /* Initialize some fields required for all kinds of coding systems. */
3402 coding
->symbol
= coding_system
;
3403 coding
->heading_ascii
= -1;
3404 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3405 coding
->composing
= COMPOSITION_DISABLED
;
3406 coding
->cmp_data
= NULL
;
3408 if (NILP (coding_system
))
3409 goto label_invalid_coding_system
;
3411 coding_spec
= Fget (coding_system
, Qcoding_system
);
3413 if (!VECTORP (coding_spec
)
3414 || XVECTOR (coding_spec
)->size
!= 5
3415 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3416 goto label_invalid_coding_system
;
3418 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3419 if (VECTORP (eol_type
))
3421 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3422 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3424 else if (XFASTINT (eol_type
) == 1)
3426 coding
->eol_type
= CODING_EOL_CRLF
;
3427 coding
->common_flags
3428 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3430 else if (XFASTINT (eol_type
) == 2)
3432 coding
->eol_type
= CODING_EOL_CR
;
3433 coding
->common_flags
3434 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3437 coding
->eol_type
= CODING_EOL_LF
;
3439 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3440 /* Try short cut. */
3441 if (SYMBOLP (coding_type
))
3443 if (EQ (coding_type
, Qt
))
3445 coding
->type
= coding_type_undecided
;
3446 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3449 coding
->type
= coding_type_no_conversion
;
3450 /* Initialize this member. Any thing other than
3451 CODING_CATEGORY_IDX_UTF_16_BE and
3452 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3453 special treatment in detect_eol. */
3454 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3459 /* Get values of coding system properties:
3460 `post-read-conversion', `pre-write-conversion',
3461 `translation-table-for-decode', `translation-table-for-encode'. */
3462 plist
= XVECTOR (coding_spec
)->contents
[3];
3463 /* Pre & post conversion functions should be disabled if
3464 inhibit_eol_conversion is nonzero. This is the case that a code
3465 conversion function is called while those functions are running. */
3466 if (! inhibit_pre_post_conversion
)
3468 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3469 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3471 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3473 val
= Fget (val
, Qtranslation_table_for_decode
);
3474 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3475 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3477 val
= Fget (val
, Qtranslation_table_for_encode
);
3478 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3479 val
= Fplist_get (plist
, Qcoding_category
);
3482 val
= Fget (val
, Qcoding_category_index
);
3484 coding
->category_idx
= XINT (val
);
3486 goto label_invalid_coding_system
;
3489 goto label_invalid_coding_system
;
3491 /* If the coding system has non-nil `composition' property, enable
3492 composition handling. */
3493 val
= Fplist_get (plist
, Qcomposition
);
3495 coding
->composing
= COMPOSITION_NO
;
3497 switch (XFASTINT (coding_type
))
3500 coding
->type
= coding_type_emacs_mule
;
3501 coding
->common_flags
3502 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3503 coding
->composing
= COMPOSITION_NO
;
3504 if (!NILP (coding
->post_read_conversion
))
3505 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3506 if (!NILP (coding
->pre_write_conversion
))
3507 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3511 coding
->type
= coding_type_sjis
;
3512 coding
->common_flags
3513 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3517 coding
->type
= coding_type_iso2022
;
3518 coding
->common_flags
3519 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3521 Lisp_Object val
, temp
;
3523 int i
, charset
, reg_bits
= 0;
3525 val
= XVECTOR (coding_spec
)->contents
[4];
3527 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3528 goto label_invalid_coding_system
;
3530 flags
= XVECTOR (val
)->contents
;
3532 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3533 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3534 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3535 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3536 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3537 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3538 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3539 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3540 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3541 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3542 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3543 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3544 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3547 /* Invoke graphic register 0 to plane 0. */
3548 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3549 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3550 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3551 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3552 /* Not single shifting at first. */
3553 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3554 /* Beginning of buffer should also be regarded as bol. */
3555 CODING_SPEC_ISO_BOL (coding
) = 1;
3557 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3558 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3559 val
= Vcharset_revision_alist
;
3562 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3564 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3565 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3566 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3570 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3571 FLAGS[REG] can be one of below:
3572 integer CHARSET: CHARSET occupies register I,
3573 t: designate nothing to REG initially, but can be used
3575 list of integer, nil, or t: designate the first
3576 element (if integer) to REG initially, the remaining
3577 elements (if integer) is designated to REG on request,
3578 if an element is t, REG can be used by any charsets,
3579 nil: REG is never used. */
3580 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3581 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3582 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3583 for (i
= 0; i
< 4; i
++)
3585 if ((INTEGERP (flags
[i
])
3586 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3587 || (charset
= get_charset_id (flags
[i
])) >= 0)
3589 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3590 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3592 else if (EQ (flags
[i
], Qt
))
3594 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3596 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3598 else if (CONSP (flags
[i
]))
3603 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3604 if ((INTEGERP (XCAR (tail
))
3605 && (charset
= XINT (XCAR (tail
)),
3606 CHARSET_VALID_P (charset
)))
3607 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3609 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3610 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3613 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3615 while (CONSP (tail
))
3617 if ((INTEGERP (XCAR (tail
))
3618 && (charset
= XINT (XCAR (tail
)),
3619 CHARSET_VALID_P (charset
)))
3620 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3621 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3623 else if (EQ (XCAR (tail
), Qt
))
3629 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3631 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3632 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3635 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3637 /* REG 1 can be used only by locking shift in 7-bit env. */
3638 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3640 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3641 /* Without any shifting, only REG 0 and 1 can be used. */
3646 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3648 if (CHARSET_DEFINED_P (charset
)
3649 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3650 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3652 /* There exist some default graphic registers to be
3655 /* We had better avoid designating a charset of
3656 CHARS96 to REG 0 as far as possible. */
3657 if (CHARSET_CHARS (charset
) == 96)
3658 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3660 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3662 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3664 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3668 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3669 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3673 coding
->type
= coding_type_big5
;
3674 coding
->common_flags
3675 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3677 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3678 ? CODING_FLAG_BIG5_HKU
3679 : CODING_FLAG_BIG5_ETEN
);
3683 coding
->type
= coding_type_ccl
;
3684 coding
->common_flags
3685 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3687 val
= XVECTOR (coding_spec
)->contents
[4];
3689 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3691 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3693 goto label_invalid_coding_system
;
3695 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3696 val
= Fplist_get (plist
, Qvalid_codes
);
3701 for (; CONSP (val
); val
= XCDR (val
))
3705 && XINT (this) >= 0 && XINT (this) < 256)
3706 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3707 else if (CONSP (this)
3708 && INTEGERP (XCAR (this))
3709 && INTEGERP (XCDR (this)))
3711 int start
= XINT (XCAR (this));
3712 int end
= XINT (XCDR (this));
3714 if (start
>= 0 && start
<= end
&& end
< 256)
3715 while (start
<= end
)
3716 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3721 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3722 coding
->spec
.ccl
.cr_carryover
= 0;
3723 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3727 coding
->type
= coding_type_raw_text
;
3731 goto label_invalid_coding_system
;
3735 label_invalid_coding_system
:
3736 coding
->type
= coding_type_no_conversion
;
3737 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3738 coding
->common_flags
= 0;
3739 coding
->eol_type
= CODING_EOL_LF
;
3740 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3744 /* Free memory blocks allocated for storing composition information. */
3747 coding_free_composition_data (coding
)
3748 struct coding_system
*coding
;
3750 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3754 /* Memory blocks are chained. At first, rewind to the first, then,
3755 free blocks one by one. */
3756 while (cmp_data
->prev
)
3757 cmp_data
= cmp_data
->prev
;
3760 next
= cmp_data
->next
;
3764 coding
->cmp_data
= NULL
;
3767 /* Set `char_offset' member of all memory blocks pointed by
3768 coding->cmp_data to POS. */
3771 coding_adjust_composition_offset (coding
, pos
)
3772 struct coding_system
*coding
;
3775 struct composition_data
*cmp_data
;
3777 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3778 cmp_data
->char_offset
= pos
;
3781 /* Setup raw-text or one of its subsidiaries in the structure
3782 coding_system CODING according to the already setup value eol_type
3783 in CODING. CODING should be setup for some coding system in
3787 setup_raw_text_coding_system (coding
)
3788 struct coding_system
*coding
;
3790 if (coding
->type
!= coding_type_raw_text
)
3792 coding
->symbol
= Qraw_text
;
3793 coding
->type
= coding_type_raw_text
;
3794 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3796 Lisp_Object subsidiaries
;
3797 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3799 if (VECTORP (subsidiaries
)
3800 && XVECTOR (subsidiaries
)->size
== 3)
3802 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3804 setup_coding_system (coding
->symbol
, coding
);
3809 /* Emacs has a mechanism to automatically detect a coding system if it
3810 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3811 it's impossible to distinguish some coding systems accurately
3812 because they use the same range of codes. So, at first, coding
3813 systems are categorized into 7, those are:
3815 o coding-category-emacs-mule
3817 The category for a coding system which has the same code range
3818 as Emacs' internal format. Assigned the coding-system (Lisp
3819 symbol) `emacs-mule' by default.
3821 o coding-category-sjis
3823 The category for a coding system which has the same code range
3824 as SJIS. Assigned the coding-system (Lisp
3825 symbol) `japanese-shift-jis' by default.
3827 o coding-category-iso-7
3829 The category for a coding system which has the same code range
3830 as ISO2022 of 7-bit environment. This doesn't use any locking
3831 shift and single shift functions. This can encode/decode all
3832 charsets. Assigned the coding-system (Lisp symbol)
3833 `iso-2022-7bit' by default.
3835 o coding-category-iso-7-tight
3837 Same as coding-category-iso-7 except that this can
3838 encode/decode only the specified charsets.
3840 o coding-category-iso-8-1
3842 The category for a coding system which has the same code range
3843 as ISO2022 of 8-bit environment and graphic plane 1 used only
3844 for DIMENSION1 charset. This doesn't use any locking shift
3845 and single shift functions. Assigned the coding-system (Lisp
3846 symbol) `iso-latin-1' by default.
3848 o coding-category-iso-8-2
3850 The category for a coding system which has the same code range
3851 as ISO2022 of 8-bit environment and graphic plane 1 used only
3852 for DIMENSION2 charset. This doesn't use any locking shift
3853 and single shift functions. Assigned the coding-system (Lisp
3854 symbol) `japanese-iso-8bit' by default.
3856 o coding-category-iso-7-else
3858 The category for a coding system which has the same code range
3859 as ISO2022 of 7-bit environment but uses locking shift or
3860 single shift functions. Assigned the coding-system (Lisp
3861 symbol) `iso-2022-7bit-lock' by default.
3863 o coding-category-iso-8-else
3865 The category for a coding system which has the same code range
3866 as ISO2022 of 8-bit environment but uses locking shift or
3867 single shift functions. Assigned the coding-system (Lisp
3868 symbol) `iso-2022-8bit-ss2' by default.
3870 o coding-category-big5
3872 The category for a coding system which has the same code range
3873 as BIG5. Assigned the coding-system (Lisp symbol)
3874 `cn-big5' by default.
3876 o coding-category-utf-8
3878 The category for a coding system which has the same code range
3879 as UTF-8 (cf. RFC2279). Assigned the coding-system (Lisp
3880 symbol) `utf-8' by default.
3882 o coding-category-utf-16-be
3884 The category for a coding system in which a text has an
3885 Unicode signature (cf. Unicode Standard) in the order of BIG
3886 endian at the head. Assigned the coding-system (Lisp symbol)
3887 `utf-16-be' by default.
3889 o coding-category-utf-16-le
3891 The category for a coding system in which a text has an
3892 Unicode signature (cf. Unicode Standard) in the order of
3893 LITTLE endian at the head. Assigned the coding-system (Lisp
3894 symbol) `utf-16-le' by default.
3896 o coding-category-ccl
3898 The category for a coding system of which encoder/decoder is
3899 written in CCL programs. The default value is nil, i.e., no
3900 coding system is assigned.
3902 o coding-category-binary
3904 The category for a coding system not categorized in any of the
3905 above. Assigned the coding-system (Lisp symbol)
3906 `no-conversion' by default.
3908 Each of them is a Lisp symbol and the value is an actual
3909 `coding-system' (this is also a Lisp symbol) assigned by a user.
3910 What Emacs does actually is to detect a category of coding system.
3911 Then, it uses a `coding-system' assigned to it. If Emacs can't
3912 decide a single possible category, it selects a category of the
3913 highest priority. Priorities of categories are also specified by a
3914 user in a Lisp variable `coding-category-list'.
3919 int ascii_skip_code
[256];
3921 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
3922 If it detects possible coding systems, return an integer in which
3923 appropriate flag bits are set. Flag bits are defined by macros
3924 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
3925 it should point the table `coding_priorities'. In that case, only
3926 the flag bit for a coding system of the highest priority is set in
3927 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
3928 range 0x80..0x9F are in multibyte form.
3930 How many ASCII characters are at the head is returned as *SKIP. */
3933 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
3934 unsigned char *source
;
3935 int src_bytes
, *priorities
, *skip
;
3938 register unsigned char c
;
3939 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
3940 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
3943 /* At first, skip all ASCII characters and control characters except
3944 for three ISO2022 specific control characters. */
3945 ascii_skip_code
[ISO_CODE_SO
] = 0;
3946 ascii_skip_code
[ISO_CODE_SI
] = 0;
3947 ascii_skip_code
[ISO_CODE_ESC
] = 0;
3949 label_loop_detect_coding
:
3950 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
3951 *skip
= src
- source
;
3954 /* We found nothing other than ASCII. There's nothing to do. */
3958 /* The text seems to be encoded in some multilingual coding system.
3959 Now, try to find in which coding system the text is encoded. */
3962 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
3963 /* C is an ISO2022 specific control code of C0. */
3964 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
);
3967 /* No valid ISO2022 code follows C. Try again. */
3969 if (c
== ISO_CODE_ESC
)
3970 ascii_skip_code
[ISO_CODE_ESC
] = 1;
3972 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
3973 goto label_loop_detect_coding
;
3977 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
3979 if (mask
& priorities
[i
])
3980 return priorities
[i
];
3982 return CODING_CATEGORY_MASK_RAW_TEXT
;
3989 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
3994 /* C is the first byte of SJIS character code,
3995 or a leading-code of Emacs' internal format (emacs-mule),
3996 or the first byte of UTF-16. */
3997 try = (CODING_CATEGORY_MASK_SJIS
3998 | CODING_CATEGORY_MASK_EMACS_MULE
3999 | CODING_CATEGORY_MASK_UTF_16_BE
4000 | CODING_CATEGORY_MASK_UTF_16_LE
);
4002 /* Or, if C is a special latin extra code,
4003 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4004 or is an ISO2022 control-sequence-introducer (CSI),
4005 we should also consider the possibility of ISO2022 codings. */
4006 if ((VECTORP (Vlatin_extra_code_table
)
4007 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4008 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4009 || (c
== ISO_CODE_CSI
4012 || ((*src
== '0' || *src
== '1' || *src
== '2')
4013 && src
+ 1 < src_end
4014 && src
[1] == ']')))))
4015 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4016 | CODING_CATEGORY_MASK_ISO_8BIT
);
4019 /* C is a character of ISO2022 in graphic plane right,
4020 or a SJIS's 1-byte character code (i.e. JISX0201),
4021 or the first byte of BIG5's 2-byte code,
4022 or the first byte of UTF-8/16. */
4023 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4024 | CODING_CATEGORY_MASK_ISO_8BIT
4025 | CODING_CATEGORY_MASK_SJIS
4026 | CODING_CATEGORY_MASK_BIG5
4027 | CODING_CATEGORY_MASK_UTF_8
4028 | CODING_CATEGORY_MASK_UTF_16_BE
4029 | CODING_CATEGORY_MASK_UTF_16_LE
);
4031 /* Or, we may have to consider the possibility of CCL. */
4032 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4033 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4034 ->spec
.ccl
.valid_codes
)[c
])
4035 try |= CODING_CATEGORY_MASK_CCL
;
4038 utf16_examined_p
= iso2022_examined_p
= 0;
4041 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4043 if (!iso2022_examined_p
4044 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4046 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4047 iso2022_examined_p
= 1;
4049 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4050 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4051 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4052 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4053 else if (!utf16_examined_p
4054 && (priorities
[i
] & try &
4055 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4057 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4058 utf16_examined_p
= 1;
4060 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4061 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4062 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4063 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4064 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4065 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4066 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4067 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4068 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4069 mask
|= CODING_CATEGORY_MASK_BINARY
;
4070 if (mask
& priorities
[i
])
4071 return priorities
[i
];
4073 return CODING_CATEGORY_MASK_RAW_TEXT
;
4075 if (try & CODING_CATEGORY_MASK_ISO
)
4076 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4077 if (try & CODING_CATEGORY_MASK_SJIS
)
4078 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4079 if (try & CODING_CATEGORY_MASK_BIG5
)
4080 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4081 if (try & CODING_CATEGORY_MASK_UTF_8
)
4082 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4083 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4084 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4085 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4086 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4087 if (try & CODING_CATEGORY_MASK_CCL
)
4088 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4090 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4093 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4094 The information of the detected coding system is set in CODING. */
4097 detect_coding (coding
, src
, src_bytes
)
4098 struct coding_system
*coding
;
4099 const unsigned char *src
;
4106 val
= Vcoding_category_list
;
4107 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4108 coding
->src_multibyte
);
4109 coding
->heading_ascii
= skip
;
4113 /* We found a single coding system of the highest priority in MASK. */
4115 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4117 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4119 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4121 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4125 tmp
= Fget (val
, Qeol_type
);
4127 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4130 /* Setup this new coding system while preserving some slots. */
4132 int src_multibyte
= coding
->src_multibyte
;
4133 int dst_multibyte
= coding
->dst_multibyte
;
4135 setup_coding_system (val
, coding
);
4136 coding
->src_multibyte
= src_multibyte
;
4137 coding
->dst_multibyte
= dst_multibyte
;
4138 coding
->heading_ascii
= skip
;
4142 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4143 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4144 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4146 How many non-eol characters are at the head is returned as *SKIP. */
4148 #define MAX_EOL_CHECK_COUNT 3
4151 detect_eol_type (source
, src_bytes
, skip
)
4152 unsigned char *source
;
4153 int src_bytes
, *skip
;
4155 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4157 int total
= 0; /* How many end-of-lines are found so far. */
4158 int eol_type
= CODING_EOL_UNDECIDED
;
4163 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4166 if (c
== '\n' || c
== '\r')
4169 *skip
= src
- 1 - source
;
4172 this_eol_type
= CODING_EOL_LF
;
4173 else if (src
>= src_end
|| *src
!= '\n')
4174 this_eol_type
= CODING_EOL_CR
;
4176 this_eol_type
= CODING_EOL_CRLF
, src
++;
4178 if (eol_type
== CODING_EOL_UNDECIDED
)
4179 /* This is the first end-of-line. */
4180 eol_type
= this_eol_type
;
4181 else if (eol_type
!= this_eol_type
)
4183 /* The found type is different from what found before. */
4184 eol_type
= CODING_EOL_INCONSISTENT
;
4191 *skip
= src_end
- source
;
4195 /* Like detect_eol_type, but detect EOL type in 2-octet
4196 big-endian/little-endian format for coding systems utf-16-be and
4200 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4201 unsigned char *source
;
4202 int src_bytes
, *skip
, big_endian_p
;
4204 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4205 unsigned int c1
, c2
;
4206 int total
= 0; /* How many end-of-lines are found so far. */
4207 int eol_type
= CODING_EOL_UNDECIDED
;
4218 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4220 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4223 if (c1
== '\n' || c1
== '\r')
4226 *skip
= src
- 2 - source
;
4230 this_eol_type
= CODING_EOL_LF
;
4234 if ((src
+ 1) >= src_end
)
4236 this_eol_type
= CODING_EOL_CR
;
4240 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4242 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4244 this_eol_type
= CODING_EOL_CR
;
4248 if (eol_type
== CODING_EOL_UNDECIDED
)
4249 /* This is the first end-of-line. */
4250 eol_type
= this_eol_type
;
4251 else if (eol_type
!= this_eol_type
)
4253 /* The found type is different from what found before. */
4254 eol_type
= CODING_EOL_INCONSISTENT
;
4261 *skip
= src_end
- source
;
4265 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4266 is encoded. If it detects an appropriate format of end-of-line, it
4267 sets the information in *CODING. */
4270 detect_eol (coding
, src
, src_bytes
)
4271 struct coding_system
*coding
;
4272 const unsigned char *src
;
4279 switch (coding
->category_idx
)
4281 case CODING_CATEGORY_IDX_UTF_16_BE
:
4282 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4284 case CODING_CATEGORY_IDX_UTF_16_LE
:
4285 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4288 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4292 if (coding
->heading_ascii
> skip
)
4293 coding
->heading_ascii
= skip
;
4295 skip
= coding
->heading_ascii
;
4297 if (eol_type
== CODING_EOL_UNDECIDED
)
4299 if (eol_type
== CODING_EOL_INCONSISTENT
)
4302 /* This code is suppressed until we find a better way to
4303 distinguish raw text file and binary file. */
4305 /* If we have already detected that the coding is raw-text, the
4306 coding should actually be no-conversion. */
4307 if (coding
->type
== coding_type_raw_text
)
4309 setup_coding_system (Qno_conversion
, coding
);
4312 /* Else, let's decode only text code anyway. */
4314 eol_type
= CODING_EOL_LF
;
4317 val
= Fget (coding
->symbol
, Qeol_type
);
4318 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4320 int src_multibyte
= coding
->src_multibyte
;
4321 int dst_multibyte
= coding
->dst_multibyte
;
4322 struct composition_data
*cmp_data
= coding
->cmp_data
;
4324 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4325 coding
->src_multibyte
= src_multibyte
;
4326 coding
->dst_multibyte
= dst_multibyte
;
4327 coding
->heading_ascii
= skip
;
4328 coding
->cmp_data
= cmp_data
;
4332 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4334 #define DECODING_BUFFER_MAG(coding) \
4335 (coding->type == coding_type_iso2022 \
4337 : (coding->type == coding_type_ccl \
4338 ? coding->spec.ccl.decoder.buf_magnification \
4341 /* Return maximum size (bytes) of a buffer enough for decoding
4342 SRC_BYTES of text encoded in CODING. */
4345 decoding_buffer_size (coding
, src_bytes
)
4346 struct coding_system
*coding
;
4349 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4350 + CONVERSION_BUFFER_EXTRA_ROOM
);
4353 /* Return maximum size (bytes) of a buffer enough for encoding
4354 SRC_BYTES of text to CODING. */
4357 encoding_buffer_size (coding
, src_bytes
)
4358 struct coding_system
*coding
;
4363 if (coding
->type
== coding_type_ccl
)
4364 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4365 else if (CODING_REQUIRE_ENCODING (coding
))
4370 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4373 /* Working buffer for code conversion. */
4374 struct conversion_buffer
4376 int size
; /* size of data. */
4377 int on_stack
; /* 1 if allocated by alloca. */
4378 unsigned char *data
;
4381 /* Don't use alloca for allocating memory space larger than this, lest
4382 we overflow their stack. */
4383 #define MAX_ALLOCA 16*1024
4385 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4386 #define allocate_conversion_buffer(buf, len) \
4388 if (len < MAX_ALLOCA) \
4390 buf.data = (unsigned char *) alloca (len); \
4395 buf.data = (unsigned char *) xmalloc (len); \
4401 /* Double the allocated memory for *BUF. */
4403 extend_conversion_buffer (buf
)
4404 struct conversion_buffer
*buf
;
4408 unsigned char *save
= buf
->data
;
4409 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4410 bcopy (save
, buf
->data
, buf
->size
);
4415 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4420 /* Free the allocated memory for BUF if it is not on stack. */
4422 free_conversion_buffer (buf
)
4423 struct conversion_buffer
*buf
;
4430 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4431 struct coding_system
*coding
;
4432 unsigned char *source
, *destination
;
4433 int src_bytes
, dst_bytes
, encodep
;
4435 struct ccl_program
*ccl
4436 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4437 unsigned char *dst
= destination
;
4439 ccl
->suppress_error
= coding
->suppress_error
;
4440 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4443 /* On encoding, EOL format is converted within ccl_driver. For
4444 that, setup proper information in the structure CCL. */
4445 ccl
->eol_type
= coding
->eol_type
;
4446 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4447 ccl
->eol_type
= CODING_EOL_LF
;
4448 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4450 ccl
->multibyte
= coding
->src_multibyte
;
4451 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4453 /* Move carryover bytes to DESTINATION. */
4454 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4457 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4459 dst_bytes
-= dst
- destination
;
4462 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4463 &(coding
->consumed
))
4464 + dst
- destination
);
4468 coding
->produced_char
= coding
->produced
;
4469 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4471 else if (!ccl
->eight_bit_control
)
4473 /* The produced bytes forms a valid multibyte sequence. */
4474 coding
->produced_char
4475 = multibyte_chars_in_text (destination
, coding
->produced
);
4476 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4480 /* On decoding, the destination should always multibyte. But,
4481 CCL program might have been generated an invalid multibyte
4482 sequence. Here we make such a sequence valid as
4485 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4487 if ((coding
->consumed
< src_bytes
4488 || !ccl
->last_block
)
4489 && coding
->produced
>= 1
4490 && destination
[coding
->produced
- 1] >= 0x80)
4492 /* We should not convert the tailing 8-bit codes to
4493 multibyte form even if they doesn't form a valid
4494 multibyte sequence. They may form a valid sequence in
4498 if (destination
[coding
->produced
- 1] < 0xA0)
4500 else if (coding
->produced
>= 2)
4502 if (destination
[coding
->produced
- 2] >= 0x80)
4504 if (destination
[coding
->produced
- 2] < 0xA0)
4506 else if (coding
->produced
>= 3
4507 && destination
[coding
->produced
- 3] >= 0x80
4508 && destination
[coding
->produced
- 3] < 0xA0)
4514 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4515 coding
->spec
.ccl
.eight_bit_carryover
,
4517 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4518 coding
->produced
-= carryover
;
4521 coding
->produced
= str_as_multibyte (destination
, bytes
,
4523 &(coding
->produced_char
));
4526 switch (ccl
->status
)
4528 case CCL_STAT_SUSPEND_BY_SRC
:
4529 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4531 case CCL_STAT_SUSPEND_BY_DST
:
4532 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4535 case CCL_STAT_INVALID_CMD
:
4536 coding
->result
= CODING_FINISH_INTERRUPT
;
4539 coding
->result
= CODING_FINISH_NORMAL
;
4542 return coding
->result
;
4545 /* Decode EOL format of the text at PTR of BYTES length destructively
4546 according to CODING->eol_type. This is called after the CCL
4547 program produced a decoded text at PTR. If we do CRLF->LF
4548 conversion, update CODING->produced and CODING->produced_char. */
4551 decode_eol_post_ccl (coding
, ptr
, bytes
)
4552 struct coding_system
*coding
;
4556 Lisp_Object val
, saved_coding_symbol
;
4557 unsigned char *pend
= ptr
+ bytes
;
4560 /* Remember the current coding system symbol. We set it back when
4561 an inconsistent EOL is found so that `last-coding-system-used' is
4562 set to the coding system that doesn't specify EOL conversion. */
4563 saved_coding_symbol
= coding
->symbol
;
4565 coding
->spec
.ccl
.cr_carryover
= 0;
4566 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4568 /* Here, to avoid the call of setup_coding_system, we directly
4569 call detect_eol_type. */
4570 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4571 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4572 coding
->eol_type
= CODING_EOL_LF
;
4573 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4575 val
= Fget (coding
->symbol
, Qeol_type
);
4576 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4577 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4579 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4582 if (coding
->eol_type
== CODING_EOL_LF
4583 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4585 /* We have nothing to do. */
4588 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4590 unsigned char *pstart
= ptr
, *p
= ptr
;
4592 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4593 && *(pend
- 1) == '\r')
4595 /* If the last character is CR, we can't handle it here
4596 because LF will be in the not-yet-decoded source text.
4597 Record that the CR is not yet processed. */
4598 coding
->spec
.ccl
.cr_carryover
= 1;
4600 coding
->produced_char
--;
4607 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4614 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4615 goto undo_eol_conversion
;
4619 else if (*ptr
== '\n'
4620 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4621 goto undo_eol_conversion
;
4626 undo_eol_conversion
:
4627 /* We have faced with inconsistent EOL format at PTR.
4628 Convert all LFs before PTR back to CRLFs. */
4629 for (p
--, ptr
--; p
>= pstart
; p
--)
4632 *ptr
-- = '\n', *ptr
-- = '\r';
4636 /* If carryover is recorded, cancel it because we don't
4637 convert CRLF anymore. */
4638 if (coding
->spec
.ccl
.cr_carryover
)
4640 coding
->spec
.ccl
.cr_carryover
= 0;
4642 coding
->produced_char
++;
4646 coding
->eol_type
= CODING_EOL_LF
;
4647 coding
->symbol
= saved_coding_symbol
;
4651 /* As each two-byte sequence CRLF was converted to LF, (PEND
4652 - P) is the number of deleted characters. */
4653 coding
->produced
-= pend
- p
;
4654 coding
->produced_char
-= pend
- p
;
4657 else /* i.e. coding->eol_type == CODING_EOL_CR */
4659 unsigned char *p
= ptr
;
4661 for (; ptr
< pend
; ptr
++)
4665 else if (*ptr
== '\n'
4666 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4668 for (; p
< ptr
; p
++)
4674 coding
->eol_type
= CODING_EOL_LF
;
4675 coding
->symbol
= saved_coding_symbol
;
4681 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4682 decoding, it may detect coding system and format of end-of-line if
4683 those are not yet decided. The source should be unibyte, the
4684 result is multibyte if CODING->dst_multibyte is nonzero, else
4688 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4689 struct coding_system
*coding
;
4690 const unsigned char *source
;
4691 unsigned char *destination
;
4692 int src_bytes
, dst_bytes
;
4696 if (coding
->type
== coding_type_undecided
)
4697 detect_coding (coding
, source
, src_bytes
);
4699 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4700 && coding
->type
!= coding_type_ccl
)
4702 detect_eol (coding
, source
, src_bytes
);
4703 /* We had better recover the original eol format if we
4704 encounter an inconsistent eol format while decoding. */
4705 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4708 coding
->produced
= coding
->produced_char
= 0;
4709 coding
->consumed
= coding
->consumed_char
= 0;
4711 coding
->result
= CODING_FINISH_NORMAL
;
4713 switch (coding
->type
)
4715 case coding_type_sjis
:
4716 decode_coding_sjis_big5 (coding
, source
, destination
,
4717 src_bytes
, dst_bytes
, 1);
4720 case coding_type_iso2022
:
4721 decode_coding_iso2022 (coding
, source
, destination
,
4722 src_bytes
, dst_bytes
);
4725 case coding_type_big5
:
4726 decode_coding_sjis_big5 (coding
, source
, destination
,
4727 src_bytes
, dst_bytes
, 0);
4730 case coding_type_emacs_mule
:
4731 decode_coding_emacs_mule (coding
, source
, destination
,
4732 src_bytes
, dst_bytes
);
4735 case coding_type_ccl
:
4736 if (coding
->spec
.ccl
.cr_carryover
)
4738 /* Put the CR which was not processed by the previous call
4739 of decode_eol_post_ccl in DESTINATION. It will be
4740 decoded together with the following LF by the call to
4741 decode_eol_post_ccl below. */
4742 *destination
= '\r';
4744 coding
->produced_char
++;
4746 extra
= coding
->spec
.ccl
.cr_carryover
;
4748 ccl_coding_driver (coding
, source
, destination
+ extra
,
4749 src_bytes
, dst_bytes
, 0);
4750 if (coding
->eol_type
!= CODING_EOL_LF
)
4752 coding
->produced
+= extra
;
4753 coding
->produced_char
+= extra
;
4754 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4759 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4762 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4763 && coding
->mode
& CODING_MODE_LAST_BLOCK
4764 && coding
->consumed
== src_bytes
)
4765 coding
->result
= CODING_FINISH_NORMAL
;
4767 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4768 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4770 const unsigned char *src
= source
+ coding
->consumed
;
4771 unsigned char *dst
= destination
+ coding
->produced
;
4773 src_bytes
-= coding
->consumed
;
4775 if (COMPOSING_P (coding
))
4776 DECODE_COMPOSITION_END ('1');
4780 dst
+= CHAR_STRING (c
, dst
);
4781 coding
->produced_char
++;
4783 coding
->consumed
= coding
->consumed_char
= src
- source
;
4784 coding
->produced
= dst
- destination
;
4785 coding
->result
= CODING_FINISH_NORMAL
;
4788 if (!coding
->dst_multibyte
)
4790 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4791 coding
->produced_char
= coding
->produced
;
4794 return coding
->result
;
4797 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4798 multibyteness of the source is CODING->src_multibyte, the
4799 multibyteness of the result is always unibyte. */
4802 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4803 struct coding_system
*coding
;
4804 const unsigned char *source
;
4805 unsigned char *destination
;
4806 int src_bytes
, dst_bytes
;
4808 coding
->produced
= coding
->produced_char
= 0;
4809 coding
->consumed
= coding
->consumed_char
= 0;
4811 coding
->result
= CODING_FINISH_NORMAL
;
4813 switch (coding
->type
)
4815 case coding_type_sjis
:
4816 encode_coding_sjis_big5 (coding
, source
, destination
,
4817 src_bytes
, dst_bytes
, 1);
4820 case coding_type_iso2022
:
4821 encode_coding_iso2022 (coding
, source
, destination
,
4822 src_bytes
, dst_bytes
);
4825 case coding_type_big5
:
4826 encode_coding_sjis_big5 (coding
, source
, destination
,
4827 src_bytes
, dst_bytes
, 0);
4830 case coding_type_emacs_mule
:
4831 encode_coding_emacs_mule (coding
, source
, destination
,
4832 src_bytes
, dst_bytes
);
4835 case coding_type_ccl
:
4836 ccl_coding_driver (coding
, source
, destination
,
4837 src_bytes
, dst_bytes
, 1);
4841 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4844 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4845 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4847 const unsigned char *src
= source
+ coding
->consumed
;
4848 unsigned char *dst
= destination
+ coding
->produced
;
4850 if (coding
->type
== coding_type_iso2022
)
4851 ENCODE_RESET_PLANE_AND_REGISTER
;
4852 if (COMPOSING_P (coding
))
4853 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
4854 if (coding
->consumed
< src_bytes
)
4856 int len
= src_bytes
- coding
->consumed
;
4858 BCOPY_SHORT (src
, dst
, len
);
4859 if (coding
->src_multibyte
)
4860 len
= str_as_unibyte (dst
, len
);
4862 coding
->consumed
= src_bytes
;
4864 coding
->produced
= coding
->produced_char
= dst
- destination
;
4865 coding
->result
= CODING_FINISH_NORMAL
;
4868 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4869 && coding
->consumed
== src_bytes
)
4870 coding
->result
= CODING_FINISH_NORMAL
;
4872 return coding
->result
;
4875 /* Scan text in the region between *BEG and *END (byte positions),
4876 skip characters which we don't have to decode by coding system
4877 CODING at the head and tail, then set *BEG and *END to the region
4878 of the text we actually have to convert. The caller should move
4879 the gap out of the region in advance if the region is from a
4882 If STR is not NULL, *BEG and *END are indices into STR. */
4885 shrink_decoding_region (beg
, end
, coding
, str
)
4887 struct coding_system
*coding
;
4890 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
4892 Lisp_Object translation_table
;
4894 if (coding
->type
== coding_type_ccl
4895 || coding
->type
== coding_type_undecided
4896 || coding
->eol_type
!= CODING_EOL_LF
4897 || !NILP (coding
->post_read_conversion
)
4898 || coding
->composing
!= COMPOSITION_DISABLED
)
4900 /* We can't skip any data. */
4903 if (coding
->type
== coding_type_no_conversion
4904 || coding
->type
== coding_type_raw_text
4905 || coding
->type
== coding_type_emacs_mule
)
4907 /* We need no conversion, but don't have to skip any data here.
4908 Decoding routine handles them effectively anyway. */
4912 translation_table
= coding
->translation_table_for_decode
;
4913 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
4914 translation_table
= Vstandard_translation_table_for_decode
;
4915 if (CHAR_TABLE_P (translation_table
))
4918 for (i
= 0; i
< 128; i
++)
4919 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
4922 /* Some ASCII character should be translated. We give up
4927 if (coding
->heading_ascii
>= 0)
4928 /* Detection routine has already found how much we can skip at the
4930 *beg
+= coding
->heading_ascii
;
4934 begp_orig
= begp
= str
+ *beg
;
4935 endp_orig
= endp
= str
+ *end
;
4939 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
4940 endp_orig
= endp
= begp
+ *end
- *beg
;
4943 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
4944 || coding
->eol_type
== CODING_EOL_CRLF
);
4946 switch (coding
->type
)
4948 case coding_type_sjis
:
4949 case coding_type_big5
:
4950 /* We can skip all ASCII characters at the head. */
4951 if (coding
->heading_ascii
< 0)
4954 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
4956 while (begp
< endp
&& *begp
< 0x80) begp
++;
4958 /* We can skip all ASCII characters at the tail except for the
4959 second byte of SJIS or BIG5 code. */
4961 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
4963 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
4964 /* Do not consider LF as ascii if preceded by CR, since that
4965 confuses eol decoding. */
4966 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
4968 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
4972 case coding_type_iso2022
:
4973 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
4974 /* We can't skip any data. */
4976 if (coding
->heading_ascii
< 0)
4978 /* We can skip all ASCII characters at the head except for a
4979 few control codes. */
4980 while (begp
< endp
&& (c
= *begp
) < 0x80
4981 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
4982 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
4983 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
4986 switch (coding
->category_idx
)
4988 case CODING_CATEGORY_IDX_ISO_8_1
:
4989 case CODING_CATEGORY_IDX_ISO_8_2
:
4990 /* We can skip all ASCII characters at the tail. */
4992 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
4994 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
4995 /* Do not consider LF as ascii if preceded by CR, since that
4996 confuses eol decoding. */
4997 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5001 case CODING_CATEGORY_IDX_ISO_7
:
5002 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5004 /* We can skip all characters at the tail except for 8-bit
5005 codes and ESC and the following 2-byte at the tail. */
5006 unsigned char *eight_bit
= NULL
;
5010 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5012 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5017 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5019 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5022 /* Do not consider LF as ascii if preceded by CR, since that
5023 confuses eol decoding. */
5024 if (begp
< endp
&& endp
< endp_orig
5025 && endp
[-1] == '\r' && endp
[0] == '\n')
5027 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5029 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5030 /* This is an ASCII designation sequence. We can
5031 surely skip the tail. But, if we have
5032 encountered an 8-bit code, skip only the codes
5034 endp
= eight_bit
? eight_bit
: endp
+ 2;
5036 /* Hmmm, we can't skip the tail. */
5048 *beg
+= begp
- begp_orig
;
5049 *end
+= endp
- endp_orig
;
5053 /* Like shrink_decoding_region but for encoding. */
5056 shrink_encoding_region (beg
, end
, coding
, str
)
5058 struct coding_system
*coding
;
5061 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5063 Lisp_Object translation_table
;
5065 if (coding
->type
== coding_type_ccl
5066 || coding
->eol_type
== CODING_EOL_CRLF
5067 || coding
->eol_type
== CODING_EOL_CR
5068 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5070 /* We can't skip any data. */
5073 if (coding
->type
== coding_type_no_conversion
5074 || coding
->type
== coding_type_raw_text
5075 || coding
->type
== coding_type_emacs_mule
5076 || coding
->type
== coding_type_undecided
)
5078 /* We need no conversion, but don't have to skip any data here.
5079 Encoding routine handles them effectively anyway. */
5083 translation_table
= coding
->translation_table_for_encode
;
5084 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5085 translation_table
= Vstandard_translation_table_for_encode
;
5086 if (CHAR_TABLE_P (translation_table
))
5089 for (i
= 0; i
< 128; i
++)
5090 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5093 /* Some ASCII character should be translated. We give up
5100 begp_orig
= begp
= str
+ *beg
;
5101 endp_orig
= endp
= str
+ *end
;
5105 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5106 endp_orig
= endp
= begp
+ *end
- *beg
;
5109 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5110 || coding
->eol_type
== CODING_EOL_CRLF
);
5112 /* Here, we don't have to check coding->pre_write_conversion because
5113 the caller is expected to have handled it already. */
5114 switch (coding
->type
)
5116 case coding_type_iso2022
:
5117 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5118 /* We can't skip any data. */
5120 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5122 unsigned char *bol
= begp
;
5123 while (begp
< endp
&& *begp
< 0x80)
5126 if (begp
[-1] == '\n')
5130 goto label_skip_tail
;
5134 case coding_type_sjis
:
5135 case coding_type_big5
:
5136 /* We can skip all ASCII characters at the head and tail. */
5138 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5140 while (begp
< endp
&& *begp
< 0x80) begp
++;
5143 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5145 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5152 *beg
+= begp
- begp_orig
;
5153 *end
+= endp
- endp_orig
;
5157 /* As shrinking conversion region requires some overhead, we don't try
5158 shrinking if the length of conversion region is less than this
5160 static int shrink_conversion_region_threshhold
= 1024;
5162 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5164 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5166 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5167 else shrink_decoding_region (beg, end, coding, str); \
5172 code_convert_region_unwind (dummy
)
5175 inhibit_pre_post_conversion
= 0;
5179 /* Store information about all compositions in the range FROM and TO
5180 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5181 buffer or a string, defaults to the current buffer. */
5184 coding_save_composition (coding
, from
, to
, obj
)
5185 struct coding_system
*coding
;
5192 if (coding
->composing
== COMPOSITION_DISABLED
)
5194 if (!coding
->cmp_data
)
5195 coding_allocate_composition_data (coding
, from
);
5196 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5200 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5203 coding
->composing
= COMPOSITION_NO
;
5206 if (COMPOSITION_VALID_P (start
, end
, prop
))
5208 enum composition_method method
= COMPOSITION_METHOD (prop
);
5209 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5210 >= COMPOSITION_DATA_SIZE
)
5211 coding_allocate_composition_data (coding
, from
);
5212 /* For relative composition, we remember start and end
5213 positions, for the other compositions, we also remember
5215 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5216 if (method
!= COMPOSITION_RELATIVE
)
5218 /* We must store a*/
5219 Lisp_Object val
, ch
;
5221 val
= COMPOSITION_COMPONENTS (prop
);
5225 ch
= XCAR (val
), val
= XCDR (val
);
5226 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5228 else if (VECTORP (val
) || STRINGP (val
))
5230 int len
= (VECTORP (val
)
5231 ? XVECTOR (val
)->size
: SCHARS (val
));
5233 for (i
= 0; i
< len
; i
++)
5236 ? Faref (val
, make_number (i
))
5237 : XVECTOR (val
)->contents
[i
]);
5238 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5241 else /* INTEGERP (val) */
5242 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5244 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5249 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5252 /* Make coding->cmp_data point to the first memory block. */
5253 while (coding
->cmp_data
->prev
)
5254 coding
->cmp_data
= coding
->cmp_data
->prev
;
5255 coding
->cmp_data_start
= 0;
5258 /* Reflect the saved information about compositions to OBJ.
5259 CODING->cmp_data points to a memory block for the information. OBJ
5260 is a buffer or a string, defaults to the current buffer. */
5263 coding_restore_composition (coding
, obj
)
5264 struct coding_system
*coding
;
5267 struct composition_data
*cmp_data
= coding
->cmp_data
;
5272 while (cmp_data
->prev
)
5273 cmp_data
= cmp_data
->prev
;
5279 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5280 i
+= cmp_data
->data
[i
])
5282 int *data
= cmp_data
->data
+ i
;
5283 enum composition_method method
= (enum composition_method
) data
[3];
5284 Lisp_Object components
;
5286 if (method
== COMPOSITION_RELATIVE
)
5290 int len
= data
[0] - 4, j
;
5291 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5293 for (j
= 0; j
< len
; j
++)
5294 args
[j
] = make_number (data
[4 + j
]);
5295 components
= (method
== COMPOSITION_WITH_ALTCHARS
5296 ? Fstring (len
, args
) : Fvector (len
, args
));
5298 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5300 cmp_data
= cmp_data
->next
;
5304 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5305 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5306 coding system CODING, and return the status code of code conversion
5307 (currently, this value has no meaning).
5309 How many characters (and bytes) are converted to how many
5310 characters (and bytes) are recorded in members of the structure
5313 If REPLACE is nonzero, we do various things as if the original text
5314 is deleted and a new text is inserted. See the comments in
5315 replace_range (insdel.c) to know what we are doing.
5317 If REPLACE is zero, it is assumed that the source text is unibyte.
5318 Otherwise, it is assumed that the source text is multibyte. */
5321 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5322 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5323 struct coding_system
*coding
;
5325 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5326 int nchars_del
= 0, nbytes_del
= 0;
5327 int require
, inserted
, inserted_byte
;
5328 int head_skip
, tail_skip
, total_skip
= 0;
5329 Lisp_Object saved_coding_symbol
;
5331 unsigned char *src
, *dst
;
5332 Lisp_Object deletion
;
5333 int orig_point
= PT
, orig_len
= len
;
5335 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5338 saved_coding_symbol
= coding
->symbol
;
5340 if (from
< PT
&& PT
< to
)
5342 TEMP_SET_PT_BOTH (from
, from_byte
);
5348 int saved_from
= from
;
5349 int saved_inhibit_modification_hooks
;
5351 prepare_to_modify_buffer (from
, to
, &from
);
5352 if (saved_from
!= from
)
5355 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5356 len_byte
= to_byte
- from_byte
;
5359 /* The code conversion routine can not preserve text properties
5360 for now. So, we must remove all text properties in the
5361 region. Here, we must suppress all modification hooks. */
5362 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5363 inhibit_modification_hooks
= 1;
5364 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5365 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5368 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5370 /* We must detect encoding of text and eol format. */
5372 if (from
< GPT
&& to
> GPT
)
5373 move_gap_both (from
, from_byte
);
5374 if (coding
->type
== coding_type_undecided
)
5376 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5377 if (coding
->type
== coding_type_undecided
)
5379 /* It seems that the text contains only ASCII, but we
5380 should not leave it undecided because the deeper
5381 decoding routine (decode_coding) tries to detect the
5382 encodings again in vain. */
5383 coding
->type
= coding_type_emacs_mule
;
5384 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5385 /* As emacs-mule decoder will handle composition, we
5386 need this setting to allocate coding->cmp_data
5388 coding
->composing
= COMPOSITION_NO
;
5391 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5392 && coding
->type
!= coding_type_ccl
)
5394 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5395 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5396 coding
->eol_type
= CODING_EOL_LF
;
5397 /* We had better recover the original eol format if we
5398 encounter an inconsistent eol format while decoding. */
5399 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5403 /* Now we convert the text. */
5405 /* For encoding, we must process pre-write-conversion in advance. */
5406 if (! inhibit_pre_post_conversion
5408 && SYMBOLP (coding
->pre_write_conversion
)
5409 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5411 /* The function in pre-write-conversion may put a new text in a
5413 struct buffer
*prev
= current_buffer
;
5416 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5417 /* We should not call any more pre-write/post-read-conversion
5418 functions while this pre-write-conversion is running. */
5419 inhibit_pre_post_conversion
= 1;
5420 call2 (coding
->pre_write_conversion
,
5421 make_number (from
), make_number (to
));
5422 inhibit_pre_post_conversion
= 0;
5423 /* Discard the unwind protect. */
5426 if (current_buffer
!= prev
)
5429 new = Fcurrent_buffer ();
5430 set_buffer_internal_1 (prev
);
5431 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5432 TEMP_SET_PT_BOTH (from
, from_byte
);
5433 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5435 if (orig_point
>= to
)
5436 orig_point
+= len
- orig_len
;
5437 else if (orig_point
> from
)
5441 from_byte
= CHAR_TO_BYTE (from
);
5442 to_byte
= CHAR_TO_BYTE (to
);
5443 len_byte
= to_byte
- from_byte
;
5444 TEMP_SET_PT_BOTH (from
, from_byte
);
5450 if (! EQ (current_buffer
->undo_list
, Qt
))
5451 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5454 nchars_del
= to
- from
;
5455 nbytes_del
= to_byte
- from_byte
;
5459 if (coding
->composing
!= COMPOSITION_DISABLED
)
5462 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5464 coding_allocate_composition_data (coding
, from
);
5467 /* Try to skip the heading and tailing ASCIIs. */
5468 if (coding
->type
!= coding_type_ccl
)
5470 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5472 if (from
< GPT
&& GPT
< to
)
5473 move_gap_both (from
, from_byte
);
5474 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5475 if (from_byte
== to_byte
5476 && (encodep
|| NILP (coding
->post_read_conversion
))
5477 && ! CODING_REQUIRE_FLUSHING (coding
))
5479 coding
->produced
= len_byte
;
5480 coding
->produced_char
= len
;
5482 /* We must record and adjust for this new text now. */
5483 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5487 head_skip
= from_byte
- from_byte_orig
;
5488 tail_skip
= to_byte_orig
- to_byte
;
5489 total_skip
= head_skip
+ tail_skip
;
5492 len
-= total_skip
; len_byte
-= total_skip
;
5495 /* For conversion, we must put the gap before the text in addition to
5496 making the gap larger for efficient decoding. The required gap
5497 size starts from 2000 which is the magic number used in make_gap.
5498 But, after one batch of conversion, it will be incremented if we
5499 find that it is not enough . */
5502 if (GAP_SIZE
< require
)
5503 make_gap (require
- GAP_SIZE
);
5504 move_gap_both (from
, from_byte
);
5506 inserted
= inserted_byte
= 0;
5508 GAP_SIZE
+= len_byte
;
5511 ZV_BYTE
-= len_byte
;
5514 if (GPT
- BEG
< BEG_UNCHANGED
)
5515 BEG_UNCHANGED
= GPT
- BEG
;
5516 if (Z
- GPT
< END_UNCHANGED
)
5517 END_UNCHANGED
= Z
- GPT
;
5519 if (!encodep
&& coding
->src_multibyte
)
5521 /* Decoding routines expects that the source text is unibyte.
5522 We must convert 8-bit characters of multibyte form to
5524 int len_byte_orig
= len_byte
;
5525 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5526 if (len_byte
< len_byte_orig
)
5527 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5529 coding
->src_multibyte
= 0;
5536 /* The buffer memory is now:
5537 +--------+converted-text+---------+-------original-text-------+---+
5538 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5539 |<---------------------- GAP ----------------------->| */
5540 src
= GAP_END_ADDR
- len_byte
;
5541 dst
= GPT_ADDR
+ inserted_byte
;
5544 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5547 if (coding
->composing
!= COMPOSITION_DISABLED
)
5548 coding
->cmp_data
->char_offset
= from
+ inserted
;
5549 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5552 /* The buffer memory is now:
5553 +--------+-------converted-text----+--+------original-text----+---+
5554 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5555 |<---------------------- GAP ----------------------->| */
5557 inserted
+= coding
->produced_char
;
5558 inserted_byte
+= coding
->produced
;
5559 len_byte
-= coding
->consumed
;
5561 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5563 coding_allocate_composition_data (coding
, from
+ inserted
);
5567 src
+= coding
->consumed
;
5568 dst
+= coding
->produced
;
5570 if (result
== CODING_FINISH_NORMAL
)
5575 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5577 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5578 Lisp_Object eol_type
;
5580 /* Encode LFs back to the original eol format (CR or CRLF). */
5581 if (coding
->eol_type
== CODING_EOL_CR
)
5583 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5589 while (p
< pend
) if (*p
++ == '\n') count
++;
5590 if (src
- dst
< count
)
5592 /* We don't have sufficient room for encoding LFs
5593 back to CRLF. We must record converted and
5594 not-yet-converted text back to the buffer
5595 content, enlarge the gap, then record them out of
5596 the buffer contents again. */
5597 int add
= len_byte
+ inserted_byte
;
5600 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5601 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5602 make_gap (count
- GAP_SIZE
);
5604 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5605 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5606 /* Don't forget to update SRC, DST, and PEND. */
5607 src
= GAP_END_ADDR
- len_byte
;
5608 dst
= GPT_ADDR
+ inserted_byte
;
5612 inserted_byte
+= count
;
5613 coding
->produced
+= count
;
5614 p
= dst
= pend
+ count
;
5618 if (*p
== '\n') count
--, *--p
= '\r';
5622 /* Suppress eol-format conversion in the further conversion. */
5623 coding
->eol_type
= CODING_EOL_LF
;
5625 /* Set the coding system symbol to that for Unix-like EOL. */
5626 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5627 if (VECTORP (eol_type
)
5628 && XVECTOR (eol_type
)->size
== 3
5629 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5630 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5632 coding
->symbol
= saved_coding_symbol
;
5638 if (coding
->type
!= coding_type_ccl
5639 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5641 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5644 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5646 /* The source text ends in invalid codes. Let's just
5647 make them valid buffer contents, and finish conversion. */
5650 unsigned char *start
= dst
;
5652 inserted
+= len_byte
;
5656 dst
+= CHAR_STRING (c
, dst
);
5659 inserted_byte
+= dst
- start
;
5663 inserted
+= len_byte
;
5664 inserted_byte
+= len_byte
;
5670 if (result
== CODING_FINISH_INTERRUPT
)
5672 /* The conversion procedure was interrupted by a user. */
5675 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5676 if (coding
->consumed
< 1)
5678 /* It's quite strange to require more memory without
5679 consuming any bytes. Perhaps CCL program bug. */
5684 /* We have just done the first batch of conversion which was
5685 stopped because of insufficient gap. Let's reconsider the
5686 required gap size (i.e. SRT - DST) now.
5688 We have converted ORIG bytes (== coding->consumed) into
5689 NEW bytes (coding->produced). To convert the remaining
5690 LEN bytes, we may need REQUIRE bytes of gap, where:
5691 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5692 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5693 Here, we are sure that NEW >= ORIG. */
5694 float ratio
= coding
->produced
- coding
->consumed
;
5695 ratio
/= coding
->consumed
;
5696 require
= len_byte
* ratio
;
5699 if ((src
- dst
) < (require
+ 2000))
5701 /* See the comment above the previous call of make_gap. */
5702 int add
= len_byte
+ inserted_byte
;
5705 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5706 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5707 make_gap (require
+ 2000);
5709 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5710 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5713 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5715 if (encodep
&& coding
->dst_multibyte
)
5717 /* The output is unibyte. We must convert 8-bit characters to
5719 if (inserted_byte
* 2 > GAP_SIZE
)
5721 GAP_SIZE
-= inserted_byte
;
5722 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5723 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5724 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5725 make_gap (inserted_byte
- GAP_SIZE
);
5726 GAP_SIZE
+= inserted_byte
;
5727 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5728 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5729 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5731 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5734 /* If we shrank the conversion area, adjust it now. */
5738 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5739 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5740 GAP_SIZE
+= total_skip
;
5741 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5742 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5743 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5744 from
-= head_skip
; from_byte
-= head_skip
;
5745 to
+= tail_skip
; to_byte
+= tail_skip
;
5749 if (! EQ (current_buffer
->undo_list
, Qt
))
5750 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5752 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
5753 inserted
, inserted_byte
);
5754 inserted
= Z
- prev_Z
;
5756 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5757 coding_restore_composition (coding
, Fcurrent_buffer ());
5758 coding_free_composition_data (coding
);
5760 if (! inhibit_pre_post_conversion
5761 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5766 TEMP_SET_PT_BOTH (from
, from_byte
);
5768 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5769 /* We should not call any more pre-write/post-read-conversion
5770 functions while this post-read-conversion is running. */
5771 inhibit_pre_post_conversion
= 1;
5772 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
5773 inhibit_pre_post_conversion
= 0;
5774 /* Discard the unwind protect. */
5777 inserted
+= Z
- prev_Z
;
5780 if (orig_point
>= from
)
5782 if (orig_point
>= from
+ orig_len
)
5783 orig_point
+= inserted
- orig_len
;
5786 TEMP_SET_PT (orig_point
);
5791 signal_after_change (from
, to
- from
, inserted
);
5792 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
5796 coding
->consumed
= to_byte
- from_byte
;
5797 coding
->consumed_char
= to
- from
;
5798 coding
->produced
= inserted_byte
;
5799 coding
->produced_char
= inserted
;
5806 run_pre_post_conversion_on_str (str
, coding
, encodep
)
5808 struct coding_system
*coding
;
5811 int count
= SPECPDL_INDEX ();
5812 struct gcpro gcpro1
;
5813 int multibyte
= STRING_MULTIBYTE (str
);
5817 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
5818 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5821 buffer
= Fget_buffer_create (build_string (" *code-converting-work*"));
5822 buf
= XBUFFER (buffer
);
5824 buf
->directory
= current_buffer
->directory
;
5825 buf
->read_only
= Qnil
;
5826 buf
->filename
= Qnil
;
5827 buf
->undo_list
= Qt
;
5828 buf
->overlays_before
= Qnil
;
5829 buf
->overlays_after
= Qnil
;
5831 set_buffer_internal (buf
);
5832 /* We must insert the contents of STR as is without
5833 unibyte<->multibyte conversion. For that, we adjust the
5834 multibyteness of the working buffer to that of STR. */
5836 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
5838 insert_from_string (str
, 0, 0,
5839 SCHARS (str
), SBYTES (str
), 0);
5841 inhibit_pre_post_conversion
= 1;
5843 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
5846 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
5847 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
5849 inhibit_pre_post_conversion
= 0;
5850 str
= make_buffer_string (BEG
, Z
, 1);
5851 return unbind_to (count
, str
);
5855 decode_coding_string (str
, coding
, nocopy
)
5857 struct coding_system
*coding
;
5861 struct conversion_buffer buf
;
5863 Lisp_Object saved_coding_symbol
;
5865 int require_decoding
;
5866 int shrinked_bytes
= 0;
5868 int consumed
, consumed_char
, produced
, produced_char
;
5871 to_byte
= SBYTES (str
);
5873 saved_coding_symbol
= coding
->symbol
;
5874 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
5875 coding
->dst_multibyte
= 1;
5876 if (CODING_REQUIRE_DETECTION (coding
))
5878 /* See the comments in code_convert_region. */
5879 if (coding
->type
== coding_type_undecided
)
5881 detect_coding (coding
, SDATA (str
), to_byte
);
5882 if (coding
->type
== coding_type_undecided
)
5884 coding
->type
= coding_type_emacs_mule
;
5885 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5886 /* As emacs-mule decoder will handle composition, we
5887 need this setting to allocate coding->cmp_data
5889 coding
->composing
= COMPOSITION_NO
;
5892 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5893 && coding
->type
!= coding_type_ccl
)
5895 saved_coding_symbol
= coding
->symbol
;
5896 detect_eol (coding
, SDATA (str
), to_byte
);
5897 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5898 coding
->eol_type
= CODING_EOL_LF
;
5899 /* We had better recover the original eol format if we
5900 encounter an inconsistent eol format while decoding. */
5901 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5905 if (coding
->type
== coding_type_no_conversion
5906 || coding
->type
== coding_type_raw_text
)
5907 coding
->dst_multibyte
= 0;
5909 require_decoding
= CODING_REQUIRE_DECODING (coding
);
5911 if (STRING_MULTIBYTE (str
))
5913 /* Decoding routines expect the source text to be unibyte. */
5914 str
= Fstring_as_unibyte (str
);
5915 to_byte
= SBYTES (str
);
5917 coding
->src_multibyte
= 0;
5920 /* Try to skip the heading and tailing ASCIIs. */
5921 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
5923 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
5925 if (from
== to_byte
)
5926 require_decoding
= 0;
5927 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
5930 if (!require_decoding
)
5932 coding
->consumed
= SBYTES (str
);
5933 coding
->consumed_char
= SCHARS (str
);
5934 if (coding
->dst_multibyte
)
5936 str
= Fstring_as_multibyte (str
);
5939 coding
->produced
= SBYTES (str
);
5940 coding
->produced_char
= SCHARS (str
);
5941 return (nocopy
? str
: Fcopy_sequence (str
));
5944 if (coding
->composing
!= COMPOSITION_DISABLED
)
5945 coding_allocate_composition_data (coding
, from
);
5946 len
= decoding_buffer_size (coding
, to_byte
- from
);
5947 allocate_conversion_buffer (buf
, len
);
5949 consumed
= consumed_char
= produced
= produced_char
= 0;
5952 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
5953 buf
.data
+ produced
, to_byte
- from
- consumed
,
5954 buf
.size
- produced
);
5955 consumed
+= coding
->consumed
;
5956 consumed_char
+= coding
->consumed_char
;
5957 produced
+= coding
->produced
;
5958 produced_char
+= coding
->produced_char
;
5959 if (result
== CODING_FINISH_NORMAL
5960 || (result
== CODING_FINISH_INSUFFICIENT_SRC
5961 && coding
->consumed
== 0))
5963 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5964 coding_allocate_composition_data (coding
, from
+ produced_char
);
5965 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
5966 extend_conversion_buffer (&buf
);
5967 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
5969 Lisp_Object eol_type
;
5971 /* Recover the original EOL format. */
5972 if (coding
->eol_type
== CODING_EOL_CR
)
5975 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
5976 if (*p
== '\n') *p
= '\r';
5978 else if (coding
->eol_type
== CODING_EOL_CRLF
)
5981 unsigned char *p0
, *p1
;
5982 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
5983 if (*p0
== '\n') num_eol
++;
5984 if (produced
+ num_eol
>= buf
.size
)
5985 extend_conversion_buffer (&buf
);
5986 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
5989 if (*p0
== '\n') *--p1
= '\r';
5991 produced
+= num_eol
;
5992 produced_char
+= num_eol
;
5994 /* Suppress eol-format conversion in the further conversion. */
5995 coding
->eol_type
= CODING_EOL_LF
;
5997 /* Set the coding system symbol to that for Unix-like EOL. */
5998 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5999 if (VECTORP (eol_type
)
6000 && XVECTOR (eol_type
)->size
== 3
6001 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6002 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6004 coding
->symbol
= saved_coding_symbol
;
6010 coding
->consumed
= consumed
;
6011 coding
->consumed_char
= consumed_char
;
6012 coding
->produced
= produced
;
6013 coding
->produced_char
= produced_char
;
6015 if (coding
->dst_multibyte
)
6016 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6017 produced
+ shrinked_bytes
);
6019 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6021 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6022 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6023 if (shrinked_bytes
> from
)
6024 STRING_COPYIN (newstr
, from
+ produced
,
6025 SDATA (str
) + to_byte
,
6026 shrinked_bytes
- from
);
6027 free_conversion_buffer (&buf
);
6029 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6030 coding_restore_composition (coding
, newstr
);
6031 coding_free_composition_data (coding
);
6033 if (SYMBOLP (coding
->post_read_conversion
)
6034 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6035 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6041 encode_coding_string (str
, coding
, nocopy
)
6043 struct coding_system
*coding
;
6047 struct conversion_buffer buf
;
6048 int from
, to
, to_byte
;
6050 int shrinked_bytes
= 0;
6052 int consumed
, consumed_char
, produced
, produced_char
;
6054 if (SYMBOLP (coding
->pre_write_conversion
)
6055 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6056 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6060 to_byte
= SBYTES (str
);
6062 /* Encoding routines determine the multibyteness of the source text
6063 by coding->src_multibyte. */
6064 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6065 coding
->dst_multibyte
= 0;
6066 if (! CODING_REQUIRE_ENCODING (coding
))
6068 coding
->consumed
= SBYTES (str
);
6069 coding
->consumed_char
= SCHARS (str
);
6070 if (STRING_MULTIBYTE (str
))
6072 str
= Fstring_as_unibyte (str
);
6075 coding
->produced
= SBYTES (str
);
6076 coding
->produced_char
= SCHARS (str
);
6077 return (nocopy
? str
: Fcopy_sequence (str
));
6080 if (coding
->composing
!= COMPOSITION_DISABLED
)
6081 coding_save_composition (coding
, from
, to
, str
);
6083 /* Try to skip the heading and tailing ASCIIs. */
6084 if (coding
->type
!= coding_type_ccl
)
6086 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6088 if (from
== to_byte
)
6089 return (nocopy
? str
: Fcopy_sequence (str
));
6090 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6093 len
= encoding_buffer_size (coding
, to_byte
- from
);
6094 allocate_conversion_buffer (buf
, len
);
6096 consumed
= consumed_char
= produced
= produced_char
= 0;
6099 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6100 buf
.data
+ produced
, to_byte
- from
- consumed
,
6101 buf
.size
- produced
);
6102 consumed
+= coding
->consumed
;
6103 consumed_char
+= coding
->consumed_char
;
6104 produced
+= coding
->produced
;
6105 produced_char
+= coding
->produced_char
;
6106 if (result
== CODING_FINISH_NORMAL
6107 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6108 && coding
->consumed
== 0))
6110 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6111 extend_conversion_buffer (&buf
);
6114 coding
->consumed
= consumed
;
6115 coding
->consumed_char
= consumed_char
;
6116 coding
->produced
= produced
;
6117 coding
->produced_char
= produced_char
;
6119 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6121 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6122 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6123 if (shrinked_bytes
> from
)
6124 STRING_COPYIN (newstr
, from
+ produced
,
6125 SDATA (str
) + to_byte
,
6126 shrinked_bytes
- from
);
6128 free_conversion_buffer (&buf
);
6129 coding_free_composition_data (coding
);
6136 /*** 8. Emacs Lisp library functions ***/
6138 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6139 doc
: /* Return t if OBJECT is nil or a coding-system.
6140 See the documentation of `make-coding-system' for information
6141 about coding-system objects. */)
6149 /* Get coding-spec vector for OBJ. */
6150 obj
= Fget (obj
, Qcoding_system
);
6151 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6155 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6156 Sread_non_nil_coding_system
, 1, 1, 0,
6157 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6164 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6165 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6167 while (SCHARS (val
) == 0);
6168 return (Fintern (val
, Qnil
));
6171 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6172 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6173 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6174 (prompt
, default_coding_system
)
6175 Lisp_Object prompt
, default_coding_system
;
6178 if (SYMBOLP (default_coding_system
))
6179 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6180 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6181 Qt
, Qnil
, Qcoding_system_history
,
6182 default_coding_system
, Qnil
);
6183 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6186 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6188 doc
: /* Check validity of CODING-SYSTEM.
6189 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6190 It is valid if it is a symbol with a non-nil `coding-system' property.
6191 The value of property should be a vector of length 5. */)
6193 Lisp_Object coding_system
;
6195 CHECK_SYMBOL (coding_system
);
6196 if (!NILP (Fcoding_system_p (coding_system
)))
6197 return coding_system
;
6199 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
6203 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6204 const unsigned char *src
;
6205 int src_bytes
, highest
;
6208 int coding_mask
, eol_type
;
6209 Lisp_Object val
, tmp
;
6212 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6213 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6214 if (eol_type
== CODING_EOL_INCONSISTENT
)
6215 eol_type
= CODING_EOL_UNDECIDED
;
6220 if (eol_type
!= CODING_EOL_UNDECIDED
)
6223 val2
= Fget (Qundecided
, Qeol_type
);
6225 val
= XVECTOR (val2
)->contents
[eol_type
];
6227 return (highest
? val
: Fcons (val
, Qnil
));
6230 /* At first, gather possible coding systems in VAL. */
6232 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6234 Lisp_Object category_val
, category_index
;
6236 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6237 category_val
= Fsymbol_value (XCAR (tmp
));
6238 if (!NILP (category_val
)
6239 && NATNUMP (category_index
)
6240 && (coding_mask
& (1 << XFASTINT (category_index
))))
6242 val
= Fcons (category_val
, val
);
6248 val
= Fnreverse (val
);
6250 /* Then, replace the elements with subsidiary coding systems. */
6251 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6253 if (eol_type
!= CODING_EOL_UNDECIDED
6254 && eol_type
!= CODING_EOL_INCONSISTENT
)
6257 eol
= Fget (XCAR (tmp
), Qeol_type
);
6259 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6262 return (highest
? XCAR (val
) : val
);
6265 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6267 doc
: /* Detect coding system of the text in the region between START and END.
6268 Return a list of possible coding systems ordered by priority.
6270 If only ASCII characters are found, it returns a list of single element
6271 `undecided' or its subsidiary coding system according to a detected
6274 If optional argument HIGHEST is non-nil, return the coding system of
6275 highest priority. */)
6276 (start
, end
, highest
)
6277 Lisp_Object start
, end
, highest
;
6280 int from_byte
, to_byte
;
6281 int include_anchor_byte
= 0;
6283 CHECK_NUMBER_COERCE_MARKER (start
);
6284 CHECK_NUMBER_COERCE_MARKER (end
);
6286 validate_region (&start
, &end
);
6287 from
= XINT (start
), to
= XINT (end
);
6288 from_byte
= CHAR_TO_BYTE (from
);
6289 to_byte
= CHAR_TO_BYTE (to
);
6291 if (from
< GPT
&& to
>= GPT
)
6292 move_gap_both (to
, to_byte
);
6293 /* If we an anchor byte `\0' follows the region, we include it in
6294 the detecting source. Then code detectors can handle the tailing
6295 byte sequence more accurately.
6297 Fix me: This is not an perfect solution. It is better that we
6298 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6300 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6301 include_anchor_byte
= 1;
6302 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6303 to_byte
- from_byte
+ include_anchor_byte
,
6305 !NILP (current_buffer
6306 ->enable_multibyte_characters
));
6309 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6311 doc
: /* Detect coding system of the text in STRING.
6312 Return a list of possible coding systems ordered by priority.
6314 If only ASCII characters are found, it returns a list of single element
6315 `undecided' or its subsidiary coding system according to a detected
6318 If optional argument HIGHEST is non-nil, return the coding system of
6319 highest priority. */)
6321 Lisp_Object string
, highest
;
6323 CHECK_STRING (string
);
6325 return detect_coding_system (SDATA (string
),
6326 /* "+ 1" is to include the anchor byte
6327 `\0'. With this, code detectors can
6328 handle the tailing bytes more
6330 SBYTES (string
) + 1,
6332 STRING_MULTIBYTE (string
));
6335 /* Return an intersection of lists L1 and L2. */
6338 intersection (l1
, l2
)
6341 Lisp_Object val
= Fcons (Qnil
, Qnil
), tail
;
6343 for (tail
= val
; CONSP (l1
); l1
= XCDR (l1
))
6345 if (!NILP (Fmemq (XCAR (l1
), l2
)))
6347 XSETCDR (tail
, Fcons (XCAR (l1
), Qnil
));
6355 /* Subroutine for Fsafe_coding_systems_region_internal.
6357 Return a list of coding systems that safely encode the multibyte
6358 text between P and PEND. SAFE_CODINGS, if non-nil, is a list of
6359 possible coding systems. If it is nil, it means that we have not
6360 yet found any coding systems.
6362 WORK_TABLE is a copy of the char-table Vchar_coding_system_table. An
6363 element of WORK_TABLE is set to t once the element is looked up.
6365 If a non-ASCII single byte char is found, set
6366 *single_byte_char_found to 1. */
6369 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6370 unsigned char *p
, *pend
;
6371 Lisp_Object safe_codings
, work_table
;
6372 int *single_byte_char_found
;
6379 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6381 if (ASCII_BYTE_P (c
))
6382 /* We can ignore ASCII characters here. */
6384 if (SINGLE_BYTE_CHAR_P (c
))
6385 *single_byte_char_found
= 1;
6386 if (NILP (safe_codings
))
6388 /* Check the safe coding systems for C. */
6389 val
= char_table_ref_and_index (work_table
, c
, &idx
);
6391 /* This element was already checked. Ignore it. */
6393 /* Remember that we checked this element. */
6394 CHAR_TABLE_SET (work_table
, make_number (idx
), Qt
);
6396 /* If there are some safe coding systems for C and we have
6397 already found the other set of coding systems for the
6398 different characters, get the intersection of them. */
6399 if (!EQ (safe_codings
, Qt
) && !NILP (val
))
6400 val
= intersection (safe_codings
, val
);
6403 return safe_codings
;
6407 /* Return a list of coding systems that safely encode the text between
6408 START and END. If the text contains only ASCII or is unibyte,
6411 DEFUN ("find-coding-systems-region-internal",
6412 Ffind_coding_systems_region_internal
,
6413 Sfind_coding_systems_region_internal
, 2, 2, 0,
6414 doc
: /* Internal use only. */)
6416 Lisp_Object start
, end
;
6418 Lisp_Object work_table
, safe_codings
;
6419 int non_ascii_p
= 0;
6420 int single_byte_char_found
= 0;
6421 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6423 if (STRINGP (start
))
6425 if (!STRING_MULTIBYTE (start
))
6427 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6429 if (SCHARS (start
) != SBYTES (start
))
6436 CHECK_NUMBER_COERCE_MARKER (start
);
6437 CHECK_NUMBER_COERCE_MARKER (end
);
6438 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6439 args_out_of_range (start
, end
);
6440 if (NILP (current_buffer
->enable_multibyte_characters
))
6442 from
= CHAR_TO_BYTE (XINT (start
));
6443 to
= CHAR_TO_BYTE (XINT (end
));
6444 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6445 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6449 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6450 if (XINT (end
) - XINT (start
) != to
- from
)
6456 /* We are sure that the text contains no multibyte character.
6457 Check if it contains eight-bit-graphic. */
6459 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
6462 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
6468 /* The text contains non-ASCII characters. */
6469 work_table
= Fcopy_sequence (Vchar_coding_system_table
);
6470 safe_codings
= find_safe_codings (p1
, p1end
, Qt
, work_table
,
6471 &single_byte_char_found
);
6473 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
6474 &single_byte_char_found
);
6476 if (EQ (safe_codings
, Qt
))
6477 ; /* Nothing to be done. */
6478 else if (!single_byte_char_found
)
6480 /* Append generic coding systems. */
6481 Lisp_Object args
[2];
6482 args
[0] = safe_codings
;
6483 args
[1] = Fchar_table_extra_slot (Vchar_coding_system_table
,
6485 safe_codings
= Fappend (2, args
);
6488 safe_codings
= Fcons (Qraw_text
,
6490 Fcons (Qno_conversion
, safe_codings
)));
6491 return safe_codings
;
6496 code_convert_region1 (start
, end
, coding_system
, encodep
)
6497 Lisp_Object start
, end
, coding_system
;
6500 struct coding_system coding
;
6503 CHECK_NUMBER_COERCE_MARKER (start
);
6504 CHECK_NUMBER_COERCE_MARKER (end
);
6505 CHECK_SYMBOL (coding_system
);
6507 validate_region (&start
, &end
);
6508 from
= XFASTINT (start
);
6509 to
= XFASTINT (end
);
6511 if (NILP (coding_system
))
6512 return make_number (to
- from
);
6514 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
6515 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
6517 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
6518 coding
.src_multibyte
= coding
.dst_multibyte
6519 = !NILP (current_buffer
->enable_multibyte_characters
);
6520 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
6521 &coding
, encodep
, 1);
6522 Vlast_coding_system_used
= coding
.symbol
;
6523 return make_number (coding
.produced_char
);
6526 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
6527 3, 3, "r\nzCoding system: ",
6528 doc
: /* Decode the current region from the specified coding system.
6529 When called from a program, takes three arguments:
6530 START, END, and CODING-SYSTEM. START and END are buffer positions.
6531 This function sets `last-coding-system-used' to the precise coding system
6532 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6533 not fully specified.)
6534 It returns the length of the decoded text. */)
6535 (start
, end
, coding_system
)
6536 Lisp_Object start
, end
, coding_system
;
6538 return code_convert_region1 (start
, end
, coding_system
, 0);
6541 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
6542 3, 3, "r\nzCoding system: ",
6543 doc
: /* Encode the current region into the specified coding system.
6544 When called from a program, takes three arguments:
6545 START, END, and CODING-SYSTEM. START and END are buffer positions.
6546 This function sets `last-coding-system-used' to the precise coding system
6547 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6548 not fully specified.)
6549 It returns the length of the encoded text. */)
6550 (start
, end
, coding_system
)
6551 Lisp_Object start
, end
, coding_system
;
6553 return code_convert_region1 (start
, end
, coding_system
, 1);
6557 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
6558 Lisp_Object string
, coding_system
, nocopy
;
6561 struct coding_system coding
;
6563 CHECK_STRING (string
);
6564 CHECK_SYMBOL (coding_system
);
6566 if (NILP (coding_system
))
6567 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
6569 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
6570 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
6572 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
6574 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
6575 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
6576 Vlast_coding_system_used
= coding
.symbol
;
6581 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
6583 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
6584 Optional arg NOCOPY non-nil means it is OK to return STRING itself
6585 if the decoding operation is trivial.
6586 This function sets `last-coding-system-used' to the precise coding system
6587 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6588 not fully specified.) */)
6589 (string
, coding_system
, nocopy
)
6590 Lisp_Object string
, coding_system
, nocopy
;
6592 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
6595 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
6597 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
6598 Optional arg NOCOPY non-nil means it is OK to return STRING itself
6599 if the encoding operation is trivial.
6600 This function sets `last-coding-system-used' to the precise coding system
6601 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6602 not fully specified.) */)
6603 (string
, coding_system
, nocopy
)
6604 Lisp_Object string
, coding_system
, nocopy
;
6606 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
6609 /* Encode or decode STRING according to CODING_SYSTEM.
6610 Do not set Vlast_coding_system_used.
6612 This function is called only from macros DECODE_FILE and
6613 ENCODE_FILE, thus we ignore character composition. */
6616 code_convert_string_norecord (string
, coding_system
, encodep
)
6617 Lisp_Object string
, coding_system
;
6620 struct coding_system coding
;
6622 CHECK_STRING (string
);
6623 CHECK_SYMBOL (coding_system
);
6625 if (NILP (coding_system
))
6628 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
6629 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
6631 coding
.composing
= COMPOSITION_DISABLED
;
6632 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
6634 ? encode_coding_string (string
, &coding
, 1)
6635 : decode_coding_string (string
, &coding
, 1));
6638 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
6639 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
6640 Return the corresponding character. */)
6644 unsigned char c1
, c2
, s1
, s2
;
6647 CHECK_NUMBER (code
);
6648 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
6652 XSETFASTINT (val
, s2
);
6653 else if (s2
>= 0xA0 || s2
<= 0xDF)
6654 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
6656 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
6660 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
6661 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
6662 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
6663 DECODE_SJIS (s1
, s2
, c1
, c2
);
6664 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
6669 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
6670 doc
: /* Encode a Japanese character CHAR to shift_jis encoding.
6671 Return the corresponding code in SJIS. */)
6675 int charset
, c1
, c2
, s1
, s2
;
6679 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
6680 if (charset
== CHARSET_ASCII
)
6684 else if (charset
== charset_jisx0208
6685 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
6687 ENCODE_SJIS (c1
, c2
, s1
, s2
);
6688 XSETFASTINT (val
, (s1
<< 8) | s2
);
6690 else if (charset
== charset_katakana_jisx0201
6691 && c1
> 0x20 && c2
< 0xE0)
6693 XSETFASTINT (val
, c1
| 0x80);
6696 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
6700 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
6701 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
6702 Return the corresponding character. */)
6707 unsigned char b1
, b2
, c1
, c2
;
6710 CHECK_NUMBER (code
);
6711 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
6715 error ("Invalid BIG5 code: %x", XFASTINT (code
));
6720 if ((b1
< 0xA1 || b1
> 0xFE)
6721 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
6722 error ("Invalid BIG5 code: %x", XFASTINT (code
));
6723 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
6724 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
6729 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
6730 doc
: /* Encode the Big5 character CHAR to BIG5 coding system.
6731 Return the corresponding character code in Big5. */)
6735 int charset
, c1
, c2
, b1
, b2
;
6739 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
6740 if (charset
== CHARSET_ASCII
)
6744 else if ((charset
== charset_big5_1
6745 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
6746 || (charset
== charset_big5_2
6747 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
6749 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
6750 XSETFASTINT (val
, (b1
<< 8) | b2
);
6753 error ("Can't encode to Big5: %d", XFASTINT (ch
));
6757 DEFUN ("set-terminal-coding-system-internal",
6758 Fset_terminal_coding_system_internal
,
6759 Sset_terminal_coding_system_internal
, 1, 1, 0,
6760 doc
: /* Internal use only. */)
6762 Lisp_Object coding_system
;
6764 CHECK_SYMBOL (coding_system
);
6765 setup_coding_system (Fcheck_coding_system (coding_system
), &terminal_coding
);
6766 /* We had better not send unsafe characters to terminal. */
6767 terminal_coding
.flags
|= CODING_FLAG_ISO_SAFE
;
6768 /* Character composition should be disabled. */
6769 terminal_coding
.composing
= COMPOSITION_DISABLED
;
6770 /* Error notification should be suppressed. */
6771 terminal_coding
.suppress_error
= 1;
6772 terminal_coding
.src_multibyte
= 1;
6773 terminal_coding
.dst_multibyte
= 0;
6777 DEFUN ("set-safe-terminal-coding-system-internal",
6778 Fset_safe_terminal_coding_system_internal
,
6779 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
6780 doc
: /* Internal use only. */)
6782 Lisp_Object coding_system
;
6784 CHECK_SYMBOL (coding_system
);
6785 setup_coding_system (Fcheck_coding_system (coding_system
),
6786 &safe_terminal_coding
);
6787 /* Character composition should be disabled. */
6788 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
6789 /* Error notification should be suppressed. */
6790 terminal_coding
.suppress_error
= 1;
6791 safe_terminal_coding
.src_multibyte
= 1;
6792 safe_terminal_coding
.dst_multibyte
= 0;
6796 DEFUN ("terminal-coding-system",
6797 Fterminal_coding_system
, Sterminal_coding_system
, 0, 0, 0,
6798 doc
: /* Return coding system specified for terminal output. */)
6801 return terminal_coding
.symbol
;
6804 DEFUN ("set-keyboard-coding-system-internal",
6805 Fset_keyboard_coding_system_internal
,
6806 Sset_keyboard_coding_system_internal
, 1, 1, 0,
6807 doc
: /* Internal use only. */)
6809 Lisp_Object coding_system
;
6811 CHECK_SYMBOL (coding_system
);
6812 setup_coding_system (Fcheck_coding_system (coding_system
), &keyboard_coding
);
6813 /* Character composition should be disabled. */
6814 keyboard_coding
.composing
= COMPOSITION_DISABLED
;
6818 DEFUN ("keyboard-coding-system",
6819 Fkeyboard_coding_system
, Skeyboard_coding_system
, 0, 0, 0,
6820 doc
: /* Return coding system specified for decoding keyboard input. */)
6823 return keyboard_coding
.symbol
;
6827 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
6828 Sfind_operation_coding_system
, 1, MANY
, 0,
6829 doc
: /* Choose a coding system for an operation based on the target name.
6830 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
6831 DECODING-SYSTEM is the coding system to use for decoding
6832 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
6833 for encoding (in case OPERATION does encoding).
6835 The first argument OPERATION specifies an I/O primitive:
6836 For file I/O, `insert-file-contents' or `write-region'.
6837 For process I/O, `call-process', `call-process-region', or `start-process'.
6838 For network I/O, `open-network-stream'.
6840 The remaining arguments should be the same arguments that were passed
6841 to the primitive. Depending on which primitive, one of those arguments
6842 is selected as the TARGET. For example, if OPERATION does file I/O,
6843 whichever argument specifies the file name is TARGET.
6845 TARGET has a meaning which depends on OPERATION:
6846 For file I/O, TARGET is a file name.
6847 For process I/O, TARGET is a process name.
6848 For network I/O, TARGET is a service name or a port number
6850 This function looks up what specified for TARGET in,
6851 `file-coding-system-alist', `process-coding-system-alist',
6852 or `network-coding-system-alist' depending on OPERATION.
6853 They may specify a coding system, a cons of coding systems,
6854 or a function symbol to call.
6855 In the last case, we call the function with one argument,
6856 which is a list of all the arguments given to this function.
6858 usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
6863 Lisp_Object operation
, target_idx
, target
, val
;
6864 register Lisp_Object chain
;
6867 error ("Too few arguments");
6868 operation
= args
[0];
6869 if (!SYMBOLP (operation
)
6870 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
6871 error ("Invalid first argument");
6872 if (nargs
< 1 + XINT (target_idx
))
6873 error ("Too few arguments for operation: %s",
6874 SDATA (SYMBOL_NAME (operation
)));
6875 target
= args
[XINT (target_idx
) + 1];
6876 if (!(STRINGP (target
)
6877 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
6878 error ("Invalid argument %d", XINT (target_idx
) + 1);
6880 chain
= ((EQ (operation
, Qinsert_file_contents
)
6881 || EQ (operation
, Qwrite_region
))
6882 ? Vfile_coding_system_alist
6883 : (EQ (operation
, Qopen_network_stream
)
6884 ? Vnetwork_coding_system_alist
6885 : Vprocess_coding_system_alist
));
6889 for (; CONSP (chain
); chain
= XCDR (chain
))
6895 && ((STRINGP (target
)
6896 && STRINGP (XCAR (elt
))
6897 && fast_string_match (XCAR (elt
), target
) >= 0)
6898 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
6901 /* Here, if VAL is both a valid coding system and a valid
6902 function symbol, we return VAL as a coding system. */
6905 if (! SYMBOLP (val
))
6907 if (! NILP (Fcoding_system_p (val
)))
6908 return Fcons (val
, val
);
6909 if (! NILP (Ffboundp (val
)))
6911 val
= call1 (val
, Flist (nargs
, args
));
6914 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
6915 return Fcons (val
, val
);
6923 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
6924 Supdate_coding_systems_internal
, 0, 0, 0,
6925 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
6926 When values of any coding categories are changed, you must
6927 call this function. */)
6932 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
6936 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[i
]);
6939 if (! coding_system_table
[i
])
6940 coding_system_table
[i
] = ((struct coding_system
*)
6941 xmalloc (sizeof (struct coding_system
)));
6942 setup_coding_system (val
, coding_system_table
[i
]);
6944 else if (coding_system_table
[i
])
6946 xfree (coding_system_table
[i
]);
6947 coding_system_table
[i
] = NULL
;
6954 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
6955 Sset_coding_priority_internal
, 0, 0, 0,
6956 doc
: /* Update internal database for the current value of `coding-category-list'.
6957 This function is internal use only. */)
6963 val
= Vcoding_category_list
;
6965 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
6967 if (! SYMBOLP (XCAR (val
)))
6969 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
6970 if (idx
>= CODING_CATEGORY_IDX_MAX
)
6972 coding_priorities
[i
++] = (1 << idx
);
6975 /* If coding-category-list is valid and contains all coding
6976 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
6977 the following code saves Emacs from crashing. */
6978 while (i
< CODING_CATEGORY_IDX_MAX
)
6979 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
6987 /*** 9. Post-amble ***/
6994 /* Emacs' internal format specific initialize routine. */
6995 for (i
= 0; i
<= 0x20; i
++)
6996 emacs_code_class
[i
] = EMACS_control_code
;
6997 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
6998 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
6999 for (i
= 0x21 ; i
< 0x7F; i
++)
7000 emacs_code_class
[i
] = EMACS_ascii_code
;
7001 emacs_code_class
[0x7F] = EMACS_control_code
;
7002 for (i
= 0x80; i
< 0xFF; i
++)
7003 emacs_code_class
[i
] = EMACS_invalid_code
;
7004 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7005 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7006 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7007 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7009 /* ISO2022 specific initialize routine. */
7010 for (i
= 0; i
< 0x20; i
++)
7011 iso_code_class
[i
] = ISO_control_0
;
7012 for (i
= 0x21; i
< 0x7F; i
++)
7013 iso_code_class
[i
] = ISO_graphic_plane_0
;
7014 for (i
= 0x80; i
< 0xA0; i
++)
7015 iso_code_class
[i
] = ISO_control_1
;
7016 for (i
= 0xA1; i
< 0xFF; i
++)
7017 iso_code_class
[i
] = ISO_graphic_plane_1
;
7018 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7019 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7020 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7021 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7022 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7023 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7024 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7025 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7026 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7027 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7029 setup_coding_system (Qnil
, &keyboard_coding
);
7030 setup_coding_system (Qnil
, &terminal_coding
);
7031 setup_coding_system (Qnil
, &safe_terminal_coding
);
7032 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7034 bzero (coding_system_table
, sizeof coding_system_table
);
7036 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7037 for (i
= 0; i
< 128; i
++)
7038 ascii_skip_code
[i
] = 1;
7040 #if defined (MSDOS) || defined (WINDOWSNT)
7041 system_eol_type
= CODING_EOL_CRLF
;
7043 system_eol_type
= CODING_EOL_LF
;
7046 inhibit_pre_post_conversion
= 0;
7054 Qtarget_idx
= intern ("target-idx");
7055 staticpro (&Qtarget_idx
);
7057 Qcoding_system_history
= intern ("coding-system-history");
7058 staticpro (&Qcoding_system_history
);
7059 Fset (Qcoding_system_history
, Qnil
);
7061 /* Target FILENAME is the first argument. */
7062 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7063 /* Target FILENAME is the third argument. */
7064 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7066 Qcall_process
= intern ("call-process");
7067 staticpro (&Qcall_process
);
7068 /* Target PROGRAM is the first argument. */
7069 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7071 Qcall_process_region
= intern ("call-process-region");
7072 staticpro (&Qcall_process_region
);
7073 /* Target PROGRAM is the third argument. */
7074 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7076 Qstart_process
= intern ("start-process");
7077 staticpro (&Qstart_process
);
7078 /* Target PROGRAM is the third argument. */
7079 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7081 Qopen_network_stream
= intern ("open-network-stream");
7082 staticpro (&Qopen_network_stream
);
7083 /* Target SERVICE is the fourth argument. */
7084 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7086 Qcoding_system
= intern ("coding-system");
7087 staticpro (&Qcoding_system
);
7089 Qeol_type
= intern ("eol-type");
7090 staticpro (&Qeol_type
);
7092 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7093 staticpro (&Qbuffer_file_coding_system
);
7095 Qpost_read_conversion
= intern ("post-read-conversion");
7096 staticpro (&Qpost_read_conversion
);
7098 Qpre_write_conversion
= intern ("pre-write-conversion");
7099 staticpro (&Qpre_write_conversion
);
7101 Qno_conversion
= intern ("no-conversion");
7102 staticpro (&Qno_conversion
);
7104 Qundecided
= intern ("undecided");
7105 staticpro (&Qundecided
);
7107 Qcoding_system_p
= intern ("coding-system-p");
7108 staticpro (&Qcoding_system_p
);
7110 Qcoding_system_error
= intern ("coding-system-error");
7111 staticpro (&Qcoding_system_error
);
7113 Fput (Qcoding_system_error
, Qerror_conditions
,
7114 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7115 Fput (Qcoding_system_error
, Qerror_message
,
7116 build_string ("Invalid coding system"));
7118 Qcoding_category
= intern ("coding-category");
7119 staticpro (&Qcoding_category
);
7120 Qcoding_category_index
= intern ("coding-category-index");
7121 staticpro (&Qcoding_category_index
);
7123 Vcoding_category_table
7124 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7125 staticpro (&Vcoding_category_table
);
7128 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7130 XVECTOR (Vcoding_category_table
)->contents
[i
]
7131 = intern (coding_category_name
[i
]);
7132 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7133 Qcoding_category_index
, make_number (i
));
7137 Qtranslation_table
= intern ("translation-table");
7138 staticpro (&Qtranslation_table
);
7139 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (1));
7141 Qtranslation_table_id
= intern ("translation-table-id");
7142 staticpro (&Qtranslation_table_id
);
7144 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7145 staticpro (&Qtranslation_table_for_decode
);
7147 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7148 staticpro (&Qtranslation_table_for_encode
);
7150 Qsafe_chars
= intern ("safe-chars");
7151 staticpro (&Qsafe_chars
);
7153 Qchar_coding_system
= intern ("char-coding-system");
7154 staticpro (&Qchar_coding_system
);
7156 /* Intern this now in case it isn't already done.
7157 Setting this variable twice is harmless.
7158 But don't staticpro it here--that is done in alloc.c. */
7159 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7160 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7161 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (2));
7163 Qvalid_codes
= intern ("valid-codes");
7164 staticpro (&Qvalid_codes
);
7166 Qemacs_mule
= intern ("emacs-mule");
7167 staticpro (&Qemacs_mule
);
7169 Qraw_text
= intern ("raw-text");
7170 staticpro (&Qraw_text
);
7172 defsubr (&Scoding_system_p
);
7173 defsubr (&Sread_coding_system
);
7174 defsubr (&Sread_non_nil_coding_system
);
7175 defsubr (&Scheck_coding_system
);
7176 defsubr (&Sdetect_coding_region
);
7177 defsubr (&Sdetect_coding_string
);
7178 defsubr (&Sfind_coding_systems_region_internal
);
7179 defsubr (&Sdecode_coding_region
);
7180 defsubr (&Sencode_coding_region
);
7181 defsubr (&Sdecode_coding_string
);
7182 defsubr (&Sencode_coding_string
);
7183 defsubr (&Sdecode_sjis_char
);
7184 defsubr (&Sencode_sjis_char
);
7185 defsubr (&Sdecode_big5_char
);
7186 defsubr (&Sencode_big5_char
);
7187 defsubr (&Sset_terminal_coding_system_internal
);
7188 defsubr (&Sset_safe_terminal_coding_system_internal
);
7189 defsubr (&Sterminal_coding_system
);
7190 defsubr (&Sset_keyboard_coding_system_internal
);
7191 defsubr (&Skeyboard_coding_system
);
7192 defsubr (&Sfind_operation_coding_system
);
7193 defsubr (&Supdate_coding_systems_internal
);
7194 defsubr (&Sset_coding_priority_internal
);
7196 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7197 doc
: /* List of coding systems.
7199 Do not alter the value of this variable manually. This variable should be
7200 updated by the functions `make-coding-system' and
7201 `define-coding-system-alias'. */);
7202 Vcoding_system_list
= Qnil
;
7204 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7205 doc
: /* Alist of coding system names.
7206 Each element is one element list of coding system name.
7207 This variable is given to `completing-read' as TABLE argument.
7209 Do not alter the value of this variable manually. This variable should be
7210 updated by the functions `make-coding-system' and
7211 `define-coding-system-alias'. */);
7212 Vcoding_system_alist
= Qnil
;
7214 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7215 doc
: /* List of coding-categories (symbols) ordered by priority.
7217 On detecting a coding system, Emacs tries code detection algorithms
7218 associated with each coding-category one by one in this order. When
7219 one algorithm agrees with a byte sequence of source text, the coding
7220 system bound to the corresponding coding-category is selected. */);
7224 Vcoding_category_list
= Qnil
;
7225 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7226 Vcoding_category_list
7227 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7228 Vcoding_category_list
);
7231 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
7232 doc
: /* Specify the coding system for read operations.
7233 It is useful to bind this variable with `let', but do not set it globally.
7234 If the value is a coding system, it is used for decoding on read operation.
7235 If not, an appropriate element is used from one of the coding system alists:
7236 There are three such tables, `file-coding-system-alist',
7237 `process-coding-system-alist', and `network-coding-system-alist'. */);
7238 Vcoding_system_for_read
= Qnil
;
7240 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
7241 doc
: /* Specify the coding system for write operations.
7242 Programs bind this variable with `let', but you should not set it globally.
7243 If the value is a coding system, it is used for encoding of output,
7244 when writing it to a file and when sending it to a file or subprocess.
7246 If this does not specify a coding system, an appropriate element
7247 is used from one of the coding system alists:
7248 There are three such tables, `file-coding-system-alist',
7249 `process-coding-system-alist', and `network-coding-system-alist'.
7250 For output to files, if the above procedure does not specify a coding system,
7251 the value of `buffer-file-coding-system' is used. */);
7252 Vcoding_system_for_write
= Qnil
;
7254 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
7255 doc
: /* Coding system used in the latest file or process I/O. */);
7256 Vlast_coding_system_used
= Qnil
;
7258 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
7259 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
7260 See info node `Coding Systems' and info node `Text and Binary' concerning
7261 such conversion. */);
7262 inhibit_eol_conversion
= 0;
7264 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
7265 doc
: /* Non-nil means process buffer inherits coding system of process output.
7266 Bind it to t if the process output is to be treated as if it were a file
7267 read from some filesystem. */);
7268 inherit_process_coding_system
= 0;
7270 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
7271 doc
: /* Alist to decide a coding system to use for a file I/O operation.
7272 The format is ((PATTERN . VAL) ...),
7273 where PATTERN is a regular expression matching a file name,
7274 VAL is a coding system, a cons of coding systems, or a function symbol.
7275 If VAL is a coding system, it is used for both decoding and encoding
7277 If VAL is a cons of coding systems, the car part is used for decoding,
7278 and the cdr part is used for encoding.
7279 If VAL is a function symbol, the function must return a coding system
7280 or a cons of coding systems which are used as above. The function gets
7281 the arguments with which `find-operation-coding-system' was called.
7283 See also the function `find-operation-coding-system'
7284 and the variable `auto-coding-alist'. */);
7285 Vfile_coding_system_alist
= Qnil
;
7287 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
7288 doc
: /* Alist to decide a coding system to use for a process I/O operation.
7289 The format is ((PATTERN . VAL) ...),
7290 where PATTERN is a regular expression matching a program name,
7291 VAL is a coding system, a cons of coding systems, or a function symbol.
7292 If VAL is a coding system, it is used for both decoding what received
7293 from the program and encoding what sent to the program.
7294 If VAL is a cons of coding systems, the car part is used for decoding,
7295 and the cdr part is used for encoding.
7296 If VAL is a function symbol, the function must return a coding system
7297 or a cons of coding systems which are used as above.
7299 See also the function `find-operation-coding-system'. */);
7300 Vprocess_coding_system_alist
= Qnil
;
7302 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
7303 doc
: /* Alist to decide a coding system to use for a network I/O operation.
7304 The format is ((PATTERN . VAL) ...),
7305 where PATTERN is a regular expression matching a network service name
7306 or is a port number to connect to,
7307 VAL is a coding system, a cons of coding systems, or a function symbol.
7308 If VAL is a coding system, it is used for both decoding what received
7309 from the network stream and encoding what sent to the network stream.
7310 If VAL is a cons of coding systems, the car part is used for decoding,
7311 and the cdr part is used for encoding.
7312 If VAL is a function symbol, the function must return a coding system
7313 or a cons of coding systems which are used as above.
7315 See also the function `find-operation-coding-system'. */);
7316 Vnetwork_coding_system_alist
= Qnil
;
7318 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
7319 doc
: /* Coding system to use with system messages.
7320 Also used for decoding keyboard input on X Window system. */);
7321 Vlocale_coding_system
= Qnil
;
7323 /* The eol mnemonics are reset in startup.el system-dependently. */
7324 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
7325 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7326 eol_mnemonic_unix
= build_string (":");
7328 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
7329 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7330 eol_mnemonic_dos
= build_string ("\\");
7332 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
7333 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
7334 eol_mnemonic_mac
= build_string ("/");
7336 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
7337 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
7338 eol_mnemonic_undecided
= build_string (":");
7340 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
7341 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
7342 Venable_character_translation
= Qt
;
7344 DEFVAR_LISP ("standard-translation-table-for-decode",
7345 &Vstandard_translation_table_for_decode
,
7346 doc
: /* Table for translating characters while decoding. */);
7347 Vstandard_translation_table_for_decode
= Qnil
;
7349 DEFVAR_LISP ("standard-translation-table-for-encode",
7350 &Vstandard_translation_table_for_encode
,
7351 doc
: /* Table for translating characters while encoding. */);
7352 Vstandard_translation_table_for_encode
= Qnil
;
7354 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
7355 doc
: /* Alist of charsets vs revision numbers.
7356 While encoding, if a charset (car part of an element) is found,
7357 designate it with the escape sequence identifying revision (cdr part of the element). */);
7358 Vcharset_revision_alist
= Qnil
;
7360 DEFVAR_LISP ("default-process-coding-system",
7361 &Vdefault_process_coding_system
,
7362 doc
: /* Cons of coding systems used for process I/O by default.
7363 The car part is used for decoding a process output,
7364 the cdr part is used for encoding a text to be sent to a process. */);
7365 Vdefault_process_coding_system
= Qnil
;
7367 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
7368 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
7369 This is a vector of length 256.
7370 If Nth element is non-nil, the existence of code N in a file
7371 \(or output of subprocess) doesn't prevent it to be detected as
7372 a coding system of ISO 2022 variant which has a flag
7373 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
7374 or reading output of a subprocess.
7375 Only 128th through 159th elements has a meaning. */);
7376 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
7378 DEFVAR_LISP ("select-safe-coding-system-function",
7379 &Vselect_safe_coding_system_function
,
7380 doc
: /* Function to call to select safe coding system for encoding a text.
7382 If set, this function is called to force a user to select a proper
7383 coding system which can encode the text in the case that a default
7384 coding system used in each operation can't encode the text.
7386 The default value is `select-safe-coding-system' (which see). */);
7387 Vselect_safe_coding_system_function
= Qnil
;
7389 DEFVAR_LISP ("char-coding-system-table", &Vchar_coding_system_table
,
7390 doc
: /* Char-table containing safe coding systems of each characters.
7391 Each element doesn't include such generic coding systems that can
7392 encode any characters. They are in the first extra slot. */);
7393 Vchar_coding_system_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
7395 DEFVAR_BOOL ("inhibit-iso-escape-detection",
7396 &inhibit_iso_escape_detection
,
7397 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
7399 By default, on reading a file, Emacs tries to detect how the text is
7400 encoded. This code detection is sensitive to escape sequences. If
7401 the sequence is valid as ISO2022, the code is determined as one of
7402 the ISO2022 encodings, and the file is decoded by the corresponding
7403 coding system (e.g. `iso-2022-7bit').
7405 However, there may be a case that you want to read escape sequences in
7406 a file as is. In such a case, you can set this variable to non-nil.
7407 Then, as the code detection ignores any escape sequences, no file is
7408 detected as encoded in some ISO2022 encoding. The result is that all
7409 escape sequences become visible in a buffer.
7411 The default value is nil, and it is strongly recommended not to change
7412 it. That is because many Emacs Lisp source files that contain
7413 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
7414 in Emacs's distribution, and they won't be decoded correctly on
7415 reading if you suppress escape sequence detection.
7417 The other way to read escape sequences in a file without decoding is
7418 to explicitly specify some coding system that doesn't use ISO2022's
7419 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
7420 inhibit_iso_escape_detection
= 0;
7424 emacs_strerror (error_number
)
7429 synchronize_system_messages_locale ();
7430 str
= strerror (error_number
);
7432 if (! NILP (Vlocale_coding_system
))
7434 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
7435 Vlocale_coding_system
,
7437 str
= (char *) SDATA (dec
);