@c -*-texinfo-*-
@c This is part of the GNU Emacs Lisp Reference Manual.
-@c Copyright (C) 1990-1995, 1998-1999, 2001-2011
-@c Free Software Foundation, Inc.
+@c Copyright (C) 1990-1995, 1998-1999, 2001-2015 Free Software
+@c Foundation, Inc.
@c See the file elisp.texi for copying conditions.
-@setfilename ../../info/syntax
-@node Syntax Tables, Abbrevs, Searching and Matching, Top
+@node Syntax Tables
@chapter Syntax Tables
@cindex parsing buffer text
@cindex syntax table
@cindex text parsing
- A @dfn{syntax table} specifies the syntactic textual function of each
-character. This information is used by the @dfn{parsing functions}, the
-complex movement commands, and others to determine where words, symbols,
-and other syntactic constructs begin and end. The current syntax table
-controls the meaning of the word motion functions (@pxref{Word Motion})
-and the list motion functions (@pxref{List Motion}), as well as the
-functions in this chapter.
+ A @dfn{syntax table} specifies the syntactic role of each character
+in a buffer. It can be used to determine where words, symbols, and
+other syntactic constructs begin and end. This information is used by
+many Emacs facilities, including Font Lock mode (@pxref{Font Lock
+Mode}) and the various complex movement commands (@pxref{Motion}).
@menu
* Basics: Syntax Basics. Basic concepts of syntax tables.
-* Desc: Syntax Descriptors. How characters are classified.
+* Syntax Descriptors:: How characters are classified.
* Syntax Table Functions:: How to create, examine and alter syntax tables.
* Syntax Properties:: Overriding syntax with text properties.
* Motion and Syntax:: Moving over characters with certain syntaxes.
* Parsing Expressions:: Parsing balanced expressions
using the syntax table.
-* Standard Syntax Tables:: Syntax tables used by various major modes.
* Syntax Table Internals:: How syntax table information is stored.
* Categories:: Another way of classifying character syntax.
@end menu
@node Syntax Basics
@section Syntax Table Concepts
-@ifnottex
- A @dfn{syntax table} provides Emacs with the information that
-determines the syntactic use of each character in a buffer. This
-information is used by the parsing commands, the complex movement
-commands, and others to determine where words, symbols, and other
-syntactic constructs begin and end. The current syntax table controls
-the meaning of the word motion functions (@pxref{Word Motion}) and the
-list motion functions (@pxref{List Motion}) as well as the functions in
-this chapter.
-@end ifnottex
+ A syntax table is a data structure which can be used to look up the
+@dfn{syntax class} and other syntactic properties of each character.
+Syntax tables are used by Lisp programs for scanning and moving across
+text.
- A syntax table is a char-table (@pxref{Char-Tables}). The element at
-index @var{c} describes the character with code @var{c}. The element's
-value should be a list that encodes the syntax of the character in
-question.
+ Internally, a syntax table is a char-table (@pxref{Char-Tables}).
+The element at index @var{c} describes the character with code
+@var{c}; its value is a cons cell which specifies the syntax of the
+character in question. @xref{Syntax Table Internals}, for details.
+However, instead of using @code{aset} and @code{aref} to modify and
+inspect syntax table contents, you should usually use the higher-level
+functions @code{char-syntax} and @code{modify-syntax-entry}, which are
+described in @ref{Syntax Table Functions}.
- Syntax tables are used only for moving across text, not for the Emacs
-Lisp reader. Emacs Lisp uses built-in syntactic rules when reading Lisp
-expressions, and these rules cannot be changed. (Some Lisp systems
-provide ways to redefine the read syntax, but we decided to leave this
-feature out of Emacs Lisp for simplicity.)
+@defun syntax-table-p object
+This function returns @code{t} if @var{object} is a syntax table.
+@end defun
Each buffer has its own major mode, and each major mode has its own
-idea of the syntactic class of various characters. For example, in Lisp
+idea of the syntax class of various characters. For example, in Lisp
mode, the character @samp{;} begins a comment, but in C mode, it
-terminates a statement. To support these variations, Emacs makes the
-choice of syntax table local to each buffer. Typically, each major
-mode has its own syntax table and installs that table in each buffer
-that uses that mode. Changing this table alters the syntax in all
-those buffers as well as in any buffers subsequently put in that mode.
-Occasionally several similar modes share one syntax table.
+terminates a statement. To support these variations, the syntax table
+is local to each buffer. Typically, each major mode has its own
+syntax table, which it installs in all buffers that use that mode.
+For example, the variable @code{emacs-lisp-mode-syntax-table} holds
+the syntax table used by Emacs Lisp mode, and
+@code{c-mode-syntax-table} holds the syntax table used by C mode.
+Changing a major mode's syntax table alters the syntax in all of that
+mode's buffers, as well as in any buffers subsequently put in that
+mode. Occasionally, several similar modes share one syntax table.
@xref{Example Major Modes}, for an example of how to set up a syntax
table.
-A syntax table can inherit the data for some characters from the
-standard syntax table, while specifying other characters itself. The
-``inherit'' syntax class means ``inherit this character's syntax from
-the standard syntax table.'' Just changing the standard syntax for a
-character affects all syntax tables that inherit from it.
+@cindex standard syntax table
+@cindex inheritance, syntax table
+ A syntax table can @dfn{inherit} from another syntax table, which is
+called its @dfn{parent syntax table}. A syntax table can leave the
+syntax class of some characters unspecified, by giving them the
+``inherit'' syntax class; such a character then acquires the syntax
+class specified by the parent syntax table (@pxref{Syntax Class
+Table}). Emacs defines a @dfn{standard syntax table}, which is the
+default parent syntax table, and is also the syntax table used by
+Fundamental mode.
-@defun syntax-table-p object
-This function returns @code{t} if @var{object} is a syntax table.
+@defun standard-syntax-table
+This function returns the standard syntax table, which is the syntax
+table used in Fundamental mode.
@end defun
+ Syntax tables are not used by the Emacs Lisp reader, which has its
+own built-in syntactic rules which cannot be changed. (Some Lisp
+systems provide ways to redefine the read syntax, but we decided to
+leave this feature out of Emacs Lisp for simplicity.)
+
@node Syntax Descriptors
@section Syntax Descriptors
@cindex syntax class
- This section describes the syntax classes and flags that denote the
-syntax of a character, and how they are represented as a @dfn{syntax
-descriptor}, which is a Lisp string that you pass to
-@code{modify-syntax-entry} to specify the syntax you want.
-
- The syntax table specifies a syntax class for each character. There
+ The @dfn{syntax class} of a character describes its syntactic role.
+Each syntax table specifies the syntax class of each character. There
is no necessary relationship between the class of a character in one
syntax table and its class in any other table.
- Each class is designated by a mnemonic character, which serves as the
-name of the class when you need to specify a class. Usually the
-designator character is one that is often assigned that class; however,
-its meaning as a designator is unvarying and independent of what syntax
-that character currently has. Thus, @samp{\} as a designator character
-always gives ``escape character'' syntax, regardless of what syntax
-@samp{\} currently has.
+ Each syntax class is designated by a mnemonic character, which
+serves as the name of the class when you need to specify a class.
+Usually, this designator character is one that is often assigned that
+class; however, its meaning as a designator is unvarying and
+independent of what syntax that character currently has. Thus,
+@samp{\} as a designator character always means ``escape character''
+syntax, regardless of whether the @samp{\} character actually has that
+syntax in the current syntax table.
+@ifnottex
+@xref{Syntax Class Table}, for a list of syntax classes and their
+designator characters.
+@end ifnottex
@cindex syntax descriptor
- A syntax descriptor is a Lisp string that specifies a syntax class, a
-matching character (used only for the parenthesis classes) and flags.
-The first character is the designator for a syntax class. The second
-character is the character to match; if it is unused, put a space there.
-Then come the characters for any desired flags. If no matching
-character or flags are needed, one character is sufficient.
+ A @dfn{syntax descriptor} is a Lisp string that describes the syntax
+class and other syntactic properties of a character. When you want to
+modify the syntax of a character, that is done by calling the function
+@code{modify-syntax-entry} and passing a syntax descriptor as one of
+its arguments (@pxref{Syntax Table Functions}).
+
+ The first character in a syntax descriptor must be a syntax class
+designator character. The second character, if present, specifies a
+matching character (e.g., in Lisp, the matching character for
+@samp{(} is @samp{)}); a space specifies that there is no matching
+character. Then come characters specifying additional syntax
+properties (@pxref{Syntax Flags}).
+
+ If no matching character or flags are needed, only one character
+(specifying the syntax class) is sufficient.
For example, the syntax descriptor for the character @samp{*} in C
mode is @code{". 23"} (i.e., punctuation, matching character slot
punctuation, matching character slot unused, first character of a
comment-starter, second character of a comment-ender).
+ Emacs also defines @dfn{raw syntax descriptors}, which are used to
+describe syntax classes at a lower level. @xref{Syntax Table
+Internals}.
+
@menu
* Syntax Class Table:: Table of syntax classes.
* Syntax Flags:: Additional flags each character can have.
@node Syntax Class Table
@subsection Table of Syntax Classes
+@cindex syntax class table
- Here is a table of syntax classes, the characters that stand for them,
-their meanings, and examples of their use.
-
-@deffn {Syntax class} @w{whitespace character}
-@dfn{Whitespace characters} (designated by @w{@samp{@ }} or @samp{-})
-separate symbols and words from each other. Typically, whitespace
-characters have no other syntactic significance, and multiple whitespace
-characters are syntactically equivalent to a single one. Space, tab,
-newline and formfeed are classified as whitespace in almost all major
-modes.
-@end deffn
+ Here is a table of syntax classes, the characters that designate
+them, their meanings, and examples of their use.
-@deffn {Syntax class} @w{word constituent}
-@dfn{Word constituents} (designated by @samp{w}) are parts of words in
-human languages, and are typically used in variable and command names
-in programs. All upper- and lower-case letters, and the digits, are
-typically word constituents.
-@end deffn
-
-@deffn {Syntax class} @w{symbol constituent}
-@dfn{Symbol constituents} (designated by @samp{_}) are the extra
-characters that are used in variable and command names along with word
-constituents. For example, the symbol constituents class is used in
-Lisp mode to indicate that certain characters may be part of symbol
-names even though they are not part of English words. These characters
-are @samp{$&*+-_<>}. In standard C, the only non-word-constituent
+@table @asis
+@item Whitespace characters: @samp{@ } or @samp{-}
+Characters that separate symbols and words from each other.
+Typically, whitespace characters have no other syntactic significance,
+and multiple whitespace characters are syntactically equivalent to a
+single one. Space, tab, and formfeed are classified as whitespace in
+almost all major modes.
+
+This syntax class can be designated by either @w{@samp{@ }} or
+@samp{-}. Both designators are equivalent.
+
+@item Word constituents: @samp{w}
+Parts of words in human languages. These are typically used in
+variable and command names in programs. All upper- and lower-case
+letters, and the digits, are typically word constituents.
+
+@item Symbol constituents: @samp{_}
+Extra characters used in variable and command names along with word
+constituents. Examples include the characters @samp{$&*+-_<>} in Lisp
+mode, which may be part of a symbol name even though they are not part
+of English words. In standard C, the only non-word-constituent
character that is valid in symbols is underscore (@samp{_}).
-@end deffn
-@deffn {Syntax class} @w{punctuation character}
-@dfn{Punctuation characters} (designated by @samp{.}) are those
-characters that are used as punctuation in English, or are used in some
-way in a programming language to separate symbols from one another.
-Some programming language modes, such as Emacs Lisp mode, have no
-characters in this class since the few characters that are not symbol or
-word constituents all have other uses. Other programming language modes,
-such as C mode, use punctuation syntax for operators.
-@end deffn
-
-@deffn {Syntax class} @w{open parenthesis character}
-@deffnx {Syntax class} @w{close parenthesis character}
-@cindex parenthesis syntax
-Open and close @dfn{parenthesis characters} are characters used in
-dissimilar pairs to surround sentences or expressions. Such a grouping
-is begun with an open parenthesis character and terminated with a close.
-Each open parenthesis character matches a particular close parenthesis
-character, and vice versa. Normally, Emacs indicates momentarily the
-matching open parenthesis when you insert a close parenthesis.
-@xref{Blinking}.
-
-The class of open parentheses is designated by @samp{(}, and that of
-close parentheses by @samp{)}.
-
-In English text, and in C code, the parenthesis pairs are @samp{()},
-@samp{[]}, and @samp{@{@}}. In Emacs Lisp, the delimiters for lists and
-vectors (@samp{()} and @samp{[]}) are classified as parenthesis
-characters.
-@end deffn
-
-@deffn {Syntax class} @w{string quote}
-@dfn{String quote characters} (designated by @samp{"}) are used in
-many languages, including Lisp and C, to delimit string constants. The
-same string quote character appears at the beginning and the end of a
-string. Such quoted strings do not nest.
+@item Punctuation characters: @samp{.}
+Characters used as punctuation in a human language, or used in a
+programming language to separate symbols from one another. Some
+programming language modes, such as Emacs Lisp mode, have no
+characters in this class since the few characters that are not symbol
+or word constituents all have other uses. Other programming language
+modes, such as C mode, use punctuation syntax for operators.
+
+@item Open parenthesis characters: @samp{(}
+@itemx Close parenthesis characters: @samp{)}
+Characters used in dissimilar pairs to surround sentences or
+expressions. Such a grouping is begun with an open parenthesis
+character and terminated with a close. Each open parenthesis
+character matches a particular close parenthesis character, and vice
+versa. Normally, Emacs indicates momentarily the matching open
+parenthesis when you insert a close parenthesis. @xref{Blinking}.
+
+In human languages, and in C code, the parenthesis pairs are
+@samp{()}, @samp{[]}, and @samp{@{@}}. In Emacs Lisp, the delimiters
+for lists and vectors (@samp{()} and @samp{[]}) are classified as
+parenthesis characters.
+
+@item String quotes: @samp{"}
+Characters used to delimit string constants. The same string quote
+character appears at the beginning and the end of a string. Such
+quoted strings do not nest.
The parsing facilities of Emacs consider a string as a single token.
The usual syntactic meanings of the characters in the string are
double-quote for strings, and single-quote (@samp{'}) for character
constants.
-English text has no string quote characters because English is not a
-programming language. Although quotation marks are used in English,
-we do not want them to turn off the usual syntactic properties of
-other characters in the quotation.
-@end deffn
+Human text has no string quote characters. We do not want quotation
+marks to turn off the usual syntactic properties of other characters
+in the quotation.
-@deffn {Syntax class} @w{escape-syntax character}
-An @dfn{escape character} (designated by @samp{\}) starts an escape
-sequence such as is used in C string and character constants. The
-character @samp{\} belongs to this class in both C and Lisp. (In C, it
-is used thus only inside strings, but it turns out to cause no trouble
-to treat it this way throughout C code.)
+@item Escape-syntax characters: @samp{\}
+Characters that start an escape sequence, such as is used in string
+and character constants. The character @samp{\} belongs to this class
+in both C and Lisp. (In C, it is used thus only inside strings, but
+it turns out to cause no trouble to treat it this way throughout C
+code.)
Characters in this class count as part of words if
@code{words-include-escapes} is non-@code{nil}. @xref{Word Motion}.
-@end deffn
-@deffn {Syntax class} @w{character quote}
-A @dfn{character quote character} (designated by @samp{/}) quotes the
-following character so that it loses its normal syntactic meaning. This
-differs from an escape character in that only the character immediately
-following is ever affected.
+@item Character quotes: @samp{/}
+Characters used to quote the following character so that it loses its
+normal syntactic meaning. This differs from an escape character in
+that only the character immediately following is ever affected.
Characters in this class count as part of words if
@code{words-include-escapes} is non-@code{nil}. @xref{Word Motion}.
This class is used for backslash in @TeX{} mode.
-@end deffn
-@deffn {Syntax class} @w{paired delimiter}
-@dfn{Paired delimiter characters} (designated by @samp{$}) are like
-string quote characters except that the syntactic properties of the
-characters between the delimiters are not suppressed. Only @TeX{} mode
-uses a paired delimiter presently---the @samp{$} that both enters and
-leaves math mode.
-@end deffn
-
-@deffn {Syntax class} @w{expression prefix}
-An @dfn{expression prefix operator} (designated by @samp{'}) is used for
-syntactic operators that are considered as part of an expression if they
-appear next to one. In Lisp modes, these characters include the
-apostrophe, @samp{'} (used for quoting), the comma, @samp{,} (used in
-macros), and @samp{#} (used in the read syntax for certain data types).
-@end deffn
-
-@deffn {Syntax class} @w{comment starter}
-@deffnx {Syntax class} @w{comment ender}
+@item Paired delimiters: @samp{$}
+Similar to string quote characters, except that the syntactic
+properties of the characters between the delimiters are not
+suppressed. Only @TeX{} mode uses a paired delimiter presently---the
+@samp{$} that both enters and leaves math mode.
+
+@item Expression prefixes: @samp{'}
+Characters used for syntactic operators that are considered as part of
+an expression if they appear next to one. In Lisp modes, these
+characters include the apostrophe, @samp{'} (used for quoting), the
+comma, @samp{,} (used in macros), and @samp{#} (used in the read
+syntax for certain data types).
+
+@item Comment starters: @samp{<}
+@itemx Comment enders: @samp{>}
@cindex comment syntax
-The @dfn{comment starter} and @dfn{comment ender} characters are used in
-various languages to delimit comments. These classes are designated
-by @samp{<} and @samp{>}, respectively.
+Characters used in various languages to delimit comments. Human text
+has no comment characters. In Lisp, the semicolon (@samp{;}) starts a
+comment and a newline or formfeed ends one.
-English text has no comment characters. In Lisp, the semicolon
-(@samp{;}) starts a comment and a newline or formfeed ends one.
-@end deffn
+@item Inherit standard syntax: @samp{@@}
+This syntax class does not specify a particular syntax. It says to
+look in the standard syntax table to find the syntax of this
+character.
-@deffn {Syntax class} @w{inherit standard syntax}
-This syntax class does not specify a particular syntax. It says to look
-in the standard syntax table to find the syntax of this character. The
-designator for this syntax class is @samp{@@}.
-@end deffn
-
-@deffn {Syntax class} @w{generic comment delimiter}
-A @dfn{generic comment delimiter} (designated by @samp{!}) starts
-or ends a special kind of comment. @emph{Any} generic comment delimiter
-matches @emph{any} generic comment delimiter, but they cannot match
-a comment starter or comment ender; generic comment delimiters can only
-match each other.
+@item Generic comment delimiters: @samp{!}
+Characters that start or end a special kind of comment. @emph{Any}
+generic comment delimiter matches @emph{any} generic comment
+delimiter, but they cannot match a comment starter or comment ender;
+generic comment delimiters can only match each other.
This syntax class is primarily meant for use with the
-@code{syntax-table} text property (@pxref{Syntax Properties}). You can
-mark any range of characters as forming a comment, by giving the first
-and last characters of the range @code{syntax-table} properties
+@code{syntax-table} text property (@pxref{Syntax Properties}). You
+can mark any range of characters as forming a comment, by giving the
+first and last characters of the range @code{syntax-table} properties
identifying them as generic comment delimiters.
-@end deffn
-@deffn {Syntax class} @w{generic string delimiter}
-A @dfn{generic string delimiter} (designated by @samp{|}) starts or ends
-a string. This class differs from the string quote class in that @emph{any}
-generic string delimiter can match any other generic string delimiter; but
-they do not match ordinary string quote characters.
+@item Generic string delimiters: @samp{|}
+Characters that start or end a string. This class differs from the
+string quote class in that @emph{any} generic string delimiter can
+match any other generic string delimiter; but they do not match
+ordinary string quote characters.
This syntax class is primarily meant for use with the
-@code{syntax-table} text property (@pxref{Syntax Properties}). You can
-mark any range of characters as forming a string constant, by giving the
-first and last characters of the range @code{syntax-table} properties
-identifying them as generic string delimiters.
-@end deffn
+@code{syntax-table} text property (@pxref{Syntax Properties}). You
+can mark any range of characters as forming a string constant, by
+giving the first and last characters of the range @code{syntax-table}
+properties identifying them as generic string delimiters.
+@end table
@node Syntax Flags
@subsection Syntax Flags
comment delimiter, @samp{n} on either character makes it
nestable.
+@cindex comment style
Emacs supports several comment styles simultaneously in any one syntax
table. A comment style is a set of flags @samp{b}, @samp{c}, and
@samp{n}, so there can be up to 8 different comment styles.
@end table
@item
-@c Emacs 19 feature
@samp{p} identifies an additional ``prefix character'' for Lisp syntax.
These characters are treated as whitespace when they appear between
expressions. When they appear within an expression, they are handled
altering syntax tables.
@defun make-syntax-table &optional table
-This function creates a new syntax table, with all values initialized
-to @code{nil}. If @var{table} is non-@code{nil}, it becomes the
-parent of the new syntax table, otherwise the standard syntax table is
-the parent. Like all char-tables, a syntax table inherits from its
-parent. Thus the original syntax of all characters in the returned
-syntax table is determined by the parent. @xref{Char-Tables}.
-
-Most major mode syntax tables are created in this way.
+This function creates a new syntax table. If @var{table} is
+non-@code{nil}, the parent of the new syntax table is @var{table};
+otherwise, the parent is the standard syntax table.
+
+In the new syntax table, all characters are initially given the
+``inherit'' (@samp{@@}) syntax class, i.e., their syntax is inherited
+from the parent table (@pxref{Syntax Class Table}).
@end defun
@defun copy-syntax-table &optional table
This function constructs a copy of @var{table} and returns it. If
-@var{table} is not supplied (or is @code{nil}), it returns a copy of the
-standard syntax table. Otherwise, an error is signaled if @var{table} is
-not a syntax table.
+@var{table} is omitted or @code{nil}, it returns a copy of the
+standard syntax table. Otherwise, an error is signaled if @var{table}
+is not a syntax table.
@end defun
@deffn Command modify-syntax-entry char syntax-descriptor &optional table
+@cindex syntax entry, setting
This function sets the syntax entry for @var{char} according to
-@var{syntax-descriptor}. @var{char} can be a character, or a cons
+@var{syntax-descriptor}. @var{char} must be a character, or a cons
cell of the form @code{(@var{min} . @var{max})}; in the latter case,
the function sets the syntax entries for all characters in the range
between @var{min} and @var{max}, inclusive.
The syntax is changed only for @var{table}, which defaults to the
-current buffer's syntax table, and not in any other syntax table. The
-argument @var{syntax-descriptor} specifies the desired syntax; this is
-a string beginning with a class designator character, and optionally
-containing a matching character and flags as well. @xref{Syntax
-Descriptors}.
+current buffer's syntax table, and not in any other syntax table.
+
+The argument @var{syntax-descriptor} is a syntax descriptor, i.e., a
+string whose first character is a syntax class designator and whose
+second and subsequent characters optionally specify a matching
+character and syntax flags. @xref{Syntax Descriptors}. An error is
+signaled if @var{syntax-descriptor} is not a valid syntax descriptor.
This function always returns @code{nil}. The old syntax information in
the table for this character is discarded.
-An error is signaled if the first character of the syntax descriptor is not
-one of the seventeen syntax class designator characters. An error is also
-signaled if @var{char} is not a character.
-
@example
@group
@exdent @r{Examples:}
@defun char-syntax character
This function returns the syntax class of @var{character}, represented
-by its mnemonic designator character. This returns @emph{only} the
-class, not any matching parenthesis or flags.
-
-An error is signaled if @var{char} is not a character.
+by its designator character (@pxref{Syntax Class Table}). This
+returns @emph{only} the class, not its matching character or syntax
+flags.
-The following examples apply to C mode. The first example shows that
-the syntax class of space is whitespace (represented by a space). The
-second example shows that the syntax of @samp{/} is punctuation. This
-does not show the fact that it is also part of comment-start and -end
-sequences. The third example shows that open parenthesis is in the class
-of open parentheses. This does not show the fact that it has a matching
-character, @samp{)}.
+The following examples apply to C mode. (We use @code{string} to make
+it easier to see the character returned by @code{char-syntax}.)
@example
@group
+;; Space characters have whitespace syntax class.
(string (char-syntax ?\s))
@result{} " "
@end group
@group
+;; Forward slash characters have punctuation syntax.
+;; Note that this @code{char-syntax} call does not reveal
+;; that it is also part of comment-start and -end sequences.
(string (char-syntax ?/))
@result{} "."
@end group
@group
+;; Open parenthesis characters have open parenthesis syntax.
+;; Note that this @code{char-syntax} call does not reveal that
+;; it has a matching character, @samp{)}.
(string (char-syntax ?\())
@result{} "("
@end group
@end example
-We use @code{string} to make it easier to see the character returned by
-@code{char-syntax}.
@end defun
@defun set-syntax-table table
the current buffer.
@end defun
-@defmac with-syntax-table @var{table} @var{body}@dots{}
+@deffn Command describe-syntax &optional buffer
+This command displays the contents of the syntax table of
+@var{buffer} (by default, the current buffer) in a help buffer.
+@end deffn
+
+@defmac with-syntax-table table body@dots{}
This macro executes @var{body} using @var{table} as the current syntax
table. It returns the value of the last form in @var{body}, after
restoring the old current syntax table.
@kindex syntax-table @r{(text property)}
When the syntax table is not flexible enough to specify the syntax of
-a language, you can use @code{syntax-table} text properties to
-override the syntax table for specific character occurrences in the
-buffer. @xref{Text Properties}. You can use Font Lock mode to set
-@code{syntax-table} text properties. @xref{Setting Syntax
-Properties}.
+a language, you can override the syntax table for specific character
+occurrences in the buffer, by applying a @code{syntax-table} text
+property. @xref{Text Properties}, for how to apply text properties.
-The valid values of @code{syntax-table} text property are:
+ The valid values of @code{syntax-table} text property are:
@table @asis
@item @var{syntax-table}
If the property value is a syntax table, that table is used instead of
-the current buffer's syntax table to determine the syntax for this
-occurrence of the character.
+the current buffer's syntax table to determine the syntax for the
+underlying text character.
@item @code{(@var{syntax-code} . @var{matching-char})}
-A cons cell of this format specifies the syntax for this
-occurrence of the character. (@pxref{Syntax Table Internals})
+A cons cell of this format is a raw syntax descriptor (@pxref{Syntax
+Table Internals}), which directly specifies a syntax class for the
+underlying text character.
@item @code{nil}
If the property is @code{nil}, the character's syntax is determined from
@end table
@defvar parse-sexp-lookup-properties
-If this is non-@code{nil}, the syntax scanning functions pay attention
-to syntax text properties. Otherwise they use only the current syntax
-table.
+If this is non-@code{nil}, the syntax scanning functions, like
+@code{forward-sexp}, pay attention to syntax text properties.
+Otherwise they use only the current syntax table.
+@end defvar
+
+@defvar syntax-propertize-function
+This variable, if non-@code{nil}, should store a function for applying
+@code{syntax-table} properties to a specified stretch of text. It is
+intended to be used by major modes to install a function which applies
+@code{syntax-table} properties in some mode-appropriate way.
+
+The function is called by @code{syntax-ppss} (@pxref{Position Parse}),
+and by Font Lock mode during syntactic fontification (@pxref{Syntactic
+Font Lock}). It is called with two arguments, @var{start} and
+@var{end}, which are the starting and ending positions of the text on
+which it should act. It is allowed to call @code{syntax-ppss} on any
+position before @var{end}. However, it should not call
+@code{syntax-ppss-flush-cache}; so, it is not allowed to call
+@code{syntax-ppss} on some position and later modify the buffer at an
+earlier position.
+@end defvar
+
+@defvar syntax-propertize-extend-region-functions
+This abnormal hook is run by the syntax parsing code prior to calling
+@code{syntax-propertize-function}. Its role is to help locate safe
+starting and ending buffer positions for passing to
+@code{syntax-propertize-function}. For example, a major mode can add
+a function to this hook to identify multi-line syntactic constructs,
+and ensure that the boundaries do not fall in the middle of one.
+
+Each function in this hook should accept two arguments, @var{start}
+and @var{end}. It should return either a cons cell of two adjusted
+buffer positions, @code{(@var{new-start} . @var{new-end})}, or
+@code{nil} if no adjustment is necessary. The hook functions are run
+in turn, repeatedly, until they all return @code{nil}.
@end defvar
@node Motion and Syntax
@section Motion and Syntax
+@cindex moving across syntax classes
+@cindex skipping characters of certain syntax
This section describes functions for moving across characters that
have certain syntax classes.
@node Parsing Expressions
@section Parsing Expressions
+@cindex parsing expressions
+@cindex scanning expressions
This section describes functions for parsing and scanning balanced
-expressions, also known as @dfn{sexps}. Basically, a sexp is either a
-balanced parenthetical grouping, a string, or a symbol name (a
-sequence of characters whose syntax is either word constituent or
-symbol constituent). However, characters whose syntax is expression
-prefix are treated as part of the sexp if they appear next to it.
+expressions. We will refer to such expressions as @dfn{sexps},
+following the terminology of Lisp, even though these functions can act
+on languages other than Lisp. Basically, a sexp is either a balanced
+parenthetical grouping, a string, or a ``symbol'' (i.e., a sequence
+of characters whose syntax is either word constituent or symbol
+constituent). However, characters in the expression prefix syntax
+class (@pxref{Syntax Class Table}) are treated as part of the sexp if
+they appear next to it.
The syntax table controls the interpretation of characters, so these
functions can be used for Lisp expressions when in Lisp mode and for C
A character's syntax controls how it changes the state of the
parser, rather than describing the state itself. For example, a
string delimiter character toggles the parser state between
-``in-string'' and ``in-code,'' but the syntax of characters does not
+``in-string'' and ``in-code'', but the syntax of characters does not
directly say whether they are inside a string. For example (note that
15 is the syntax code for generic string delimiters),
@node Motion via Parsing
@subsection Motion Commands Based on Parsing
+@cindex motion based on parsing
This section describes simple point-motion functions that operate
based on parsing expressions.
@defun scan-lists from count depth
-This function scans forward @var{count} balanced parenthetical groupings
-from position @var{from}. It returns the position where the scan stops.
-If @var{count} is negative, the scan moves backwards.
-
-If @var{depth} is nonzero, parenthesis depth counting begins from that
-value. The only candidates for stopping are places where the depth in
-parentheses becomes zero; @code{scan-lists} counts @var{count} such
-places and then stops. Thus, a positive value for @var{depth} means go
-out @var{depth} levels of parenthesis.
+This function scans forward @var{count} balanced parenthetical
+groupings from position @var{from}. It returns the position where the
+scan stops. If @var{count} is negative, the scan moves backwards.
+
+If @var{depth} is nonzero, treat the starting position as being
+@var{depth} parentheses deep. The scanner moves forward or backward
+through the buffer until the depth changes to zero @var{count} times.
+Hence, a positive value for @var{depth} has the effect of moving out
+@var{depth} levels of parenthesis from the starting position, while a
+negative @var{depth} has the effect of moving deeper by @var{-depth}
+levels of parenthesis.
Scanning ignores comments if @code{parse-sexp-ignore-comments} is
non-@code{nil}.
-If the scan reaches the beginning or end of the buffer (or its
-accessible portion), and the depth is not zero, an error is signaled.
-If the depth is zero but the count is not used up, @code{nil} is
-returned.
+If the scan reaches the beginning or end of the accessible part of the
+buffer before it has scanned over @var{count} parenthetical groupings,
+the return value is @code{nil} if the depth at that point is zero; if
+the depth is non-zero, a @code{scan-error} error is signaled.
@end defun
@defun scan-sexps from count
This function cannot tell whether the ``comments'' it traverses are
embedded within a string. If they look like comments, it treats them
as comments.
-@end defun
To move forward over all comments and whitespace following point, use
-@code{(forward-comment (buffer-size))}. @code{(buffer-size)} is a good
-argument to use, because the number of comments in the buffer cannot
-exceed that many.
+@code{(forward-comment (buffer-size))}. @code{(buffer-size)} is a
+good argument to use, because the number of comments in the buffer
+cannot exceed that many.
+@end defun
@node Position Parse
@subsection Finding the Parse State for a Position
+@cindex parse state for a position
For syntactic analysis, such as in indentation, often the useful
thing is to compute the syntactic state corresponding to a given buffer
position. This function does that conveniently.
@defun syntax-ppss &optional pos
-This function returns the parser state (see next section) that the
-parser would reach at position @var{pos} starting from the beginning
-of the buffer. This is equivalent to @code{(parse-partial-sexp
-(point-min) @var{pos})}, except that @code{syntax-ppss} uses a cache
-to speed up the computation. Due to this optimization, the 2nd value
-(previous complete subexpression) and 6th value (minimum parenthesis
-depth) of the returned parser state are not meaningful.
-@end defun
-
- @code{syntax-ppss} automatically hooks itself to
-@code{before-change-functions} to keep its cache consistent. But
-updating can fail if @code{syntax-ppss} is called while
+This function returns the parser state that the parser would reach at
+position @var{pos} starting from the beginning of the buffer.
+@iftex
+See the next section for
+@end iftex
+@ifnottex
+@xref{Parser State},
+@end ifnottex
+for a description of the parser state.
+
+The return value is the same as if you call the low-level parsing
+function @code{parse-partial-sexp} to parse from the beginning of the
+buffer to @var{pos} (@pxref{Low-Level Parsing}). However,
+@code{syntax-ppss} uses a cache to speed up the computation. Due to
+this optimization, the second value (previous complete subexpression)
+and sixth value (minimum parenthesis depth) in the returned parser
+state are not meaningful.
+
+This function has a side effect: it adds a buffer-local entry to
+@code{before-change-functions} (@pxref{Change Hooks}) for
+@code{syntax-ppss-flush-cache} (see below). This entry keeps the
+cache consistent as the buffer is modified. However, the cache might
+not be updated if @code{syntax-ppss} is called while
@code{before-change-functions} is temporarily let-bound, or if the
-buffer is modified without obeying the hook, such as when using
-@code{inhibit-modification-hooks}. For this reason, it is sometimes
-necessary to flush the cache manually.
+buffer is modified without running the hook, such as when using
+@code{inhibit-modification-hooks}. In those cases, it is necessary to
+call @code{syntax-ppss-flush-cache} explicitly.
+@end defun
@defun syntax-ppss-flush-cache beg &rest ignored-args
This function flushes the cache used by @code{syntax-ppss}, starting
@subsection Parser State
@cindex parser state
- A @dfn{parser state} is a list of ten elements describing the final
-state of parsing text syntactically as part of an expression. The
-parsing functions in the following sections return a parser state as
-the value, and in some cases accept one as an argument also, so that
-you can resume parsing after it stops. Here are the meanings of the
-elements of the parser state:
+ A @dfn{parser state} is a list of ten elements describing the state
+of the syntactic parser, after it parses the text between a specified
+starting point and a specified end point in the buffer. Parsing
+functions such as @code{syntax-ppss}
+@ifnottex
+(@pxref{Position Parse})
+@end ifnottex
+return a parser state as the value. Some parsing functions accept a
+parser state as an argument, for resuming parsing.
+
+ Here are the meanings of the elements of the parser state:
@enumerate 0
@item
The depth in parentheses, counting from 0. @strong{Warning:} this can
be negative if there are more close parens than open parens between
-the start of the defun and point.
+the parser's starting point and end point.
@item
@cindex innermost containing parentheses
@item
@cindex inside comment
-@code{t} if inside a comment (of either style),
-or the comment nesting level if inside a kind of comment
-that can be nested.
+@code{t} if inside a non-nestable comment (of any comment style;
+@pxref{Syntax Flags}); or the comment nesting level if inside a
+comment that can be nested.
@item
@cindex quote character
-@code{t} if point is just after a quote character.
+@code{t} if the end point is just after a quote character.
@item
The minimum parenthesis depth encountered during this scan.
@item
-What kind of comment is active: @code{nil} for a comment of style
-``a'' or when not inside a comment, @code{t} for a comment of style
-``b,'' and @code{syntax-table} for a comment that should be ended by a
-generic comment delimiter character.
+What kind of comment is active: @code{nil} if not in a comment or in a
+comment of style @samp{a}; 1 for a comment of style @samp{b}; 2 for a
+comment of style @samp{c}; and @code{syntax-table} for a comment that
+should be ended by a generic comment delimiter character.
@item
The string or comment start position. While inside a comment, this is
Elements 1, 2, and 6 are ignored in a state which you pass as an
argument to continue parsing, and elements 8 and 9 are used only in
-trivial cases. Those elements serve primarily to convey information
-to the Lisp program which does the parsing.
+trivial cases. Those elements are mainly used internally by the
+parser code.
One additional piece of useful information is available from a
parser state using this function:
arrived at a top level position.
@end defun
- We have provided this access function rather than document how the
-data is represented in the state, because we plan to change the
-representation in the future.
-
@node Low-Level Parsing
@subsection Low-Level Parsing
@var{start}, not scanning past @var{limit}. It stops at position
@var{limit} or when certain criteria described below are met, and sets
point to the location where parsing stops. It returns a parser state
+@ifinfo
+(@pxref{Parser State})
+@end ifinfo
describing the status of the parse at the point where it stops.
@cindex parenthesis depth
@node Control Parsing
@subsection Parameters to Control Parsing
+@cindex parsing, control parameters
@defvar multibyte-syntax-as-symbol
If this variable is non-@code{nil}, @code{scan-sexps} treats all
You can use @code{forward-comment} to move forward or backward over
one comment or several comments.
-@node Standard Syntax Tables
-@section Some Standard Syntax Tables
-
- Most of the major modes in Emacs have their own syntax tables. Here
-are several of them:
-
-@defun standard-syntax-table
-This function returns the standard syntax table, which is the syntax
-table used in Fundamental mode.
-@end defun
-
-@defvar text-mode-syntax-table
-The value of this variable is the syntax table used in Text mode.
-@end defvar
-
-@defvar c-mode-syntax-table
-The value of this variable is the syntax table for C-mode buffers.
-@end defvar
-
-@defvar emacs-lisp-mode-syntax-table
-The value of this variable is the syntax table used in Emacs Lisp mode
-by editing commands. (It has no effect on the Lisp @code{read}
-function.)
-@end defvar
-
@node Syntax Table Internals
@section Syntax Table Internals
@cindex syntax table internals
- Lisp programs don't usually work with the elements directly; the
-Lisp-level syntax table functions usually work with syntax descriptors
-(@pxref{Syntax Descriptors}). Nonetheless, here we document the
-internal format. This format is used mostly when manipulating
-syntax properties.
-
- Each element of a syntax table is a cons cell of the form
-@code{(@var{syntax-code} . @var{matching-char})}. The @sc{car},
-@var{syntax-code}, is an integer that encodes the syntax class, and any
-flags. The @sc{cdr}, @var{matching-char}, is non-@code{nil} if
-a character to match was specified.
-
- This table gives the value of @var{syntax-code} which corresponds
-to each syntactic type.
-
-@multitable @columnfractions .05 .3 .3 .31
+ Syntax tables are implemented as char-tables (@pxref{Char-Tables}),
+but most Lisp programs don't work directly with their elements.
+Syntax tables do not store syntax data as syntax descriptors
+(@pxref{Syntax Descriptors}); they use an internal format, which is
+documented in this section. This internal format can also be assigned
+as syntax properties (@pxref{Syntax Properties}).
+
+@cindex syntax code
+@cindex raw syntax descriptor
+ Each entry in a syntax table is a @dfn{raw syntax descriptor}: a
+cons cell of the form @code{(@var{syntax-code}
+. @var{matching-char})}. @var{syntax-code} is an integer which
+encodes the syntax class and syntax flags, according to the table
+below. @var{matching-char}, if non-@code{nil}, specifies a matching
+character (similar to the second character in a syntax descriptor).
+
+ Here are the syntax codes corresponding to the various syntax
+classes:
+
+@multitable @columnfractions .2 .3 .2 .3
@item
-@tab
-@i{Integer} @i{Class}
-@tab
-@i{Integer} @i{Class}
-@tab
-@i{Integer} @i{Class}
+@i{Code} @tab @i{Class} @tab @i{Code} @tab @i{Class}
@item
-@tab
-0 @ @ whitespace
-@tab
-5 @ @ close parenthesis
-@tab
-10 @ @ character quote
+0 @tab whitespace @tab 8 @tab paired delimiter
@item
-@tab
-1 @ @ punctuation
-@tab
-6 @ @ expression prefix
-@tab
-11 @ @ comment-start
+1 @tab punctuation @tab 9 @tab escape
@item
-@tab
-2 @ @ word
-@tab
-7 @ @ string quote
-@tab
-12 @ @ comment-end
+2 @tab word @tab 10 @tab character quote
@item
-@tab
-3 @ @ symbol
-@tab
-8 @ @ paired delimiter
-@tab
-13 @ @ inherit
+3 @tab symbol @tab 11 @tab comment-start
@item
-@tab
-4 @ @ open parenthesis
-@tab
-9 @ @ escape
-@tab
-14 @ @ generic comment
+4 @tab open parenthesis @tab 12 @tab comment-end
@item
-@tab
-15 @ generic string
+5 @tab close parenthesis @tab 13 @tab inherit
+@item
+6 @tab expression prefix @tab 14 @tab generic comment
+@item
+7 @tab string quote @tab 15 @tab generic string
@end multitable
- For example, the usual syntax value for @samp{(} is @code{(4 . 41)}.
-(41 is the character code for @samp{)}.)
+@noindent
+For example, in the standard syntax table, the entry for @samp{(} is
+@code{(4 . 41)}. 41 is the character code for @samp{)}.
- The flags are encoded in higher order bits, starting 16 bits from the
-least significant bit. This table gives the power of two which
+ Syntax flags are encoded in higher order bits, starting 16 bits from
+the least significant bit. This table gives the power of two which
corresponds to each syntax flag.
-@multitable @columnfractions .05 .3 .3 .3
+@multitable @columnfractions .15 .3 .15 .3
@item
-@tab
-@i{Prefix} @i{Flag}
-@tab
-@i{Prefix} @i{Flag}
-@tab
-@i{Prefix} @i{Flag}
+@i{Prefix} @tab @i{Flag} @tab @i{Prefix} @tab @i{Flag}
@item
-@tab
-@samp{1} @ @ @code{(lsh 1 16)}
-@tab
-@samp{4} @ @ @code{(lsh 1 19)}
-@tab
-@samp{b} @ @ @code{(lsh 1 21)}
+@samp{1} @tab @code{(lsh 1 16)} @tab @samp{p} @tab @code{(lsh 1 20)}
@item
-@tab
-@samp{2} @ @ @code{(lsh 1 17)}
-@tab
-@samp{p} @ @ @code{(lsh 1 20)}
-@tab
-@samp{n} @ @ @code{(lsh 1 22)}
+@samp{2} @tab @code{(lsh 1 17)} @tab @samp{b} @tab @code{(lsh 1 21)}
@item
-@tab
-@samp{3} @ @ @code{(lsh 1 18)}
+@samp{3} @tab @code{(lsh 1 18)} @tab @samp{n} @tab @code{(lsh 1 22)}
+@item
+@samp{4} @tab @code{(lsh 1 19)}
@end multitable
-@defun string-to-syntax @var{desc}
-This function returns the internal form corresponding to the syntax
-descriptor @var{desc}, a cons cell @code{(@var{syntax-code}
-. @var{matching-char})}.
+@defun string-to-syntax desc
+Given a syntax descriptor @var{desc} (a string), this function returns
+the corresponding raw syntax descriptor.
@end defun
@defun syntax-after pos
-This function returns the syntax code of the character in the buffer
-after position @var{pos}, taking account of syntax properties as well
-as the syntax table. If @var{pos} is outside the buffer's accessible
-portion (@pxref{Narrowing, accessible portion}), this function returns
-@code{nil}.
+This function returns the raw syntax descriptor for the character in
+the buffer after position @var{pos}, taking account of syntax
+properties as well as the syntax table. If @var{pos} is outside the
+buffer's accessible portion (@pxref{Narrowing, accessible portion}),
+the return value is @code{nil}.
@end defun
@defun syntax-class syntax
-This function returns the syntax class of the syntax code
-@var{syntax}. (It masks off the high 16 bits that hold the flags
-encoded in the syntax descriptor.) If @var{syntax} is @code{nil}, it
-returns @code{nil}; this is so evaluating the expression
+This function returns the syntax code for the raw syntax descriptor
+@var{syntax}. More precisely, it takes the raw syntax descriptor's
+@var{syntax-code} component, masks off the high 16 bits which record
+the syntax flags, and returns the resulting integer.
+
+If @var{syntax} is @code{nil}, the return value is returns @code{nil}.
+This is so that the expression
@example
(syntax-class (syntax-after pos))
@end example
@noindent
-where @code{pos} is outside the buffer's accessible portion, will
-yield @code{nil} without throwing errors or producing wrong syntax
-class codes.
+evaluates to @code{nil} if @code{pos} is outside the buffer's
+accessible portion, without throwing errors or returning an incorrect
+code.
@end defun
@node Categories
the range @w{@samp{ }} to @samp{~}. You specify the name of a category
when you define it with @code{define-category}.
+@cindex category set
The category table is actually a char-table (@pxref{Char-Tables}).
The element of the category table at index @var{c} is a @dfn{category
set}---a bool-vector---that indicates which categories character @var{c}