Syntax Tables
*************

A syntax table specifies the syntactic role of each character in a buffer.  It can be used to determine where words, symbols, and other syntactic constructs begin and end.  This information is used by many Emacs facilities, including Font Lock mode (see Font Lock Mode) and the various complex movement commands (see Motion).

Syntax Table Concepts
=====================

A syntax table is a data structure which can be used to look up the syntax class and other syntactic properties of each character. Syntax tables are used by Lisp programs for scanning and moving across text.

Internally, a syntax table is a char-table (see Char-Tables). The element at index C describes the character with code C; its value is a cons cell which specifies the syntax of the character in question.  See Syntax Table Internals, for details. However, instead of using aset and aref to modify and inspect syntax table contents, you should usually use the higher-level functions char-syntax and modify-syntax-entry, which are described in Syntax Table Functions.

 -- Function: syntax-table-p object
     This function returns t if OBJECT is a syntax table.

Each buffer has its own major mode, and each major mode has its own idea of the syntax class of various characters.  For example, in Lisp mode, the character ; begins a comment, but in C mode, it terminates a statement.  To support these variations, the syntax table is local to each buffer.  Typically, each major mode has its own syntax table, which it installs in all buffers that use that mode. For example, the variable emacs-lisp-mode-syntax-table holds the syntax table used by Emacs Lisp mode, and c-mode-syntax-table holds the syntax table used by C mode. Changing a major mode's syntax table alters the syntax in all of that mode's buffers, as well as in any buffers subsequently put in that mode.  Occasionally, several similar modes share one syntax table. See Example Major Modes, for an example of how to set up a syntax table.

A syntax table can inherit from another syntax table, which is called its parent syntax table.  A syntax table can leave the syntax class of some characters unspecified, by giving them the ``inherit'' syntax class; such a character then acquires the syntax class specified by the parent syntax table (see Syntax Class Table).  Emacs defines a standard syntax table, which is the default parent syntax table, and is also the syntax table used by Fundamental mode.

 -- Function: standard-syntax-table
     This function returns the standard syntax table, which is the syntax table used in Fundamental mode.

Syntax tables are not used by the Emacs Lisp reader, which has its own built-in syntactic rules which cannot be changed.  (Some Lisp systems provide ways to redefine the read syntax, but we decided to leave this feature out of Emacs Lisp for simplicity.)

Syntax Descriptors
==================

The syntax class of a character describes its syntactic role. Each syntax table specifies the syntax class of each character.  There is no necessary relationship between the class of a character in one syntax table and its class in any other table.

Each syntax class is designated by a mnemonic character, which serves as the name of the class when you need to specify a class. Usually, this designator character is one that is often assigned that class; however, its meaning as a designator is unvarying and independent of what syntax that character currently has.  Thus, \ as a designator character always stands for escape character syntax, regardless of whether the \ character actually has that syntax in the current syntax table. See Syntax Class Table, for a list of syntax classes and their designator characters.

A syntax descriptor is a Lisp string that describes the syntax class and other syntactic properties of a character.  When you want to modify the syntax of a character, that is done by calling the function modify-syntax-entry and passing a syntax descriptor as one of its arguments (see Syntax Table Functions).

The first character in a syntax descriptor must be a syntax class designator character.  The second character, if present, specifies a matching character (e.g., in Lisp, the matching character for ( is )); a space specifies that there is no matching character.  Then come characters specifying additional syntax properties (see Syntax Flags).

If no matching character or flags are needed, only one character (specifying the syntax class) is sufficient.

For example, the syntax descriptor for the character * in C mode is ". 23" (i.e., punctuation, matching character slot unused, second character of a comment-starter, first character of a comment-ender), and the entry for / is . 14 (i.e., punctuation, matching character slot unused, first character of a comment-starter, second character of a comment-ender).

Emacs also defines raw syntax descriptors, which are used to describe syntax classes at a lower level.  See Syntax Table Internals.

Table of Syntax Classes
-----------------------

Here is a table of syntax classes, the characters that designate them, their meanings, and examples of their use.

Whitespace characters:   or -
     Characters that separate symbols and words from each other. Typically, whitespace characters have no other syntactic significance, and multiple whitespace characters are syntactically equivalent to a single one.  Space, tab, and formfeed are classified as whitespace in almost all major modes.

     This syntax class can be designated by either   or -.  Both designators are equivalent.

Word constituents: w
     Parts of words in human languages.  These are typically used in variable and command names in programs.  All upper- and lower-case letters, and the digits, are typically word constituents.

Symbol constituents: _
     Extra characters used in variable and command names along with word constituents.  Examples include the characters $&*+-_<> in Lisp mode, which may be part of a symbol name even though they are not part of English words.  In standard C, the only non-word-constituent character that is valid in symbols is underscore (_).

Punctuation characters: .
     Characters used as punctuation in a human language, or used in a programming language to separate symbols from one another.  Some programming language modes, such as Emacs Lisp mode, have no characters in this class since the few characters that are not symbol or word constituents all have other uses.  Other programming language modes, such as C mode, use punctuation syntax for operators.

Open parenthesis characters: (
Close parenthesis characters: )
     Characters used in dissimilar pairs to surround sentences or expressions.  Such a grouping is begun with an open parenthesis character and terminated with a close.  Each open parenthesis character matches a particular close parenthesis character, and vice versa.  Normally, Emacs indicates momentarily the matching open parenthesis when you insert a close parenthesis.  See Blinking.

     In human languages, and in C code, the parenthesis pairs are (), [], and {}.  In Emacs Lisp, the delimiters for lists and vectors (() and []) are classified as parenthesis characters.

String quotes: "
     Characters used to delimit string constants.  The same string quote character appears at the beginning and the end of a string.  Such quoted strings do not nest.

     The parsing facilities of Emacs consider a string as a single token. The usual syntactic meanings of the characters in the string are suppressed.

     The Lisp modes have two string quote characters: double-quote (") and vertical bar (|).  | is not used in Emacs Lisp, but it is used in Common Lisp.  C also has two string quote characters: double-quote for strings, and apostrophe (') for character constants.

     Human text has no string quote characters.  We do not want quotation marks to turn off the usual syntactic properties of other characters in the quotation.

Escape-syntax characters: \
     Characters that start an escape sequence, such as is used in string and character constants.  The character \ belongs to this class in both C and Lisp.  (In C, it is used thus only inside strings, but it turns out to cause no trouble to treat it this way throughout C code.)

     Characters in this class count as part of words if words-include-escapes is non-nil.  See Word Motion.

Character quotes: /
     Characters used to quote the following character so that it loses its normal syntactic meaning.  This differs from an escape character in that only the character immediately following is ever affected.

     Characters in this class count as part of words if words-include-escapes is non-nil.  See Word Motion.

     This class is used for backslash in TeX mode.

Paired delimiters: $
     Similar to string quote characters, except that the syntactic properties of the characters between the delimiters are not suppressed.  Only TeX mode uses a paired delimiter presently---the $ that both enters and leaves math mode.

Expression prefixes: '
     Characters used for syntactic operators that are considered as part of an expression if they appear next to one.  In Lisp modes, these characters include the apostrophe, ' (used for quoting), the comma, , (used in macros), and # (used in the read syntax for certain data types).

Comment starters: <
Comment enders: >
     Characters used in various languages to delimit comments.  Human text has no comment characters.  In Lisp, the semicolon (;) starts a comment and a newline or formfeed ends one.

Inherit standard syntax: @
     This syntax class does not specify a particular syntax.  It says to look in the standard syntax table to find the syntax of this character.

Generic comment delimiters: !
     Characters that start or end a special kind of comment.  Any generic comment delimiter matches any generic comment delimiter, but they cannot match a comment starter or comment ender; generic comment delimiters can only match each other.

     This syntax class is primarily meant for use with the syntax-table text property (see Syntax Properties).  You can mark any range of characters as forming a comment, by giving the first and last characters of the range syntax-table properties identifying them as generic comment delimiters.

Generic string delimiters: |
     Characters that start or end a string.  This class differs from the string quote class in that any generic string delimiter can match any other generic string delimiter; but they do not match ordinary string quote characters.

     This syntax class is primarily meant for use with the syntax-table text property (see Syntax Properties).  You can mark any range of characters as forming a string constant, by giving the first and last characters of the range syntax-table properties identifying them as generic string delimiters.

Syntax Flags
------------

In addition to the classes, entries for characters in a syntax table can specify flags.  There are eight possible flags, represented by the characters 1, 2, 3, 4, b, c, n, and p.

All the flags except p are used to describe comment delimiters.  The digit flags are used for comment delimiters made up of 2 characters.  They indicate that a character can also be part of a comment sequence, in addition to the syntactic properties associated with its character class.  The flags are independent of the class and each other for the sake of characters such as * in C mode, which is a punctuation character, and the second character of a start-of-comment sequence (/*), and the first character of an end-of-comment sequence (*/).  The flags b, c, and n are used to qualify the corresponding comment delimiter.

Here is a table of the possible flags for a character C, and what they mean:

  * 1 means C is the start of a two-character comment-start sequence.

  * 2 means C is the second character of such a sequence.

  * 3 means C is the start of a two-character comment-end sequence.

  * 4 means C is the second character of such a sequence.

  * b means that C as a comment delimiter belongs to the alternative ``b'' comment style.  For a two-character comment starter, this flag is only significant on the second char, and for a 2-character comment ender it is only significant on the first char.

  * c means that C as a comment delimiter belongs to the alternative ``c'' comment style.  For a two-character comment delimiter, c on either character makes it of style ``c''.

  * n on a comment delimiter character specifies that this kind of comment can be nested.  Inside such a comment, only comments of the same style will be recognized.  For a two-character comment delimiter, n on either character makes it nestable.

     Emacs supports several comment styles simultaneously in any one syntax table.  A comment style is a set of flags b, c, and n, so there can be up to 8 different comment styles. Each comment delimiter has a style and only matches comment delimiters of the same style.  Thus if a comment starts with the comment-start sequence of style ``bn'', it will extend until the next matching comment-end sequence of style ``bn''.

     The appropriate comment syntax settings for C++ can be as follows:

     /
          124

     *
          23b

     newline
          >

     This defines four comment-delimiting sequences:

     /*
          This is a comment-start sequence for ``b'' style because the second character, *, has the b flag.

     //
          This is a comment-start sequence for ``a'' style because the second character, /, does not have the b flag.

     */
          This is a comment-end sequence for ``b'' style because the first character, *, has the b flag.

     newline
          This is a comment-end sequence for ``a'' style, because the newline character does not have the b flag.

  * p identifies an additional prefix character for Lisp syntax. These characters are treated as whitespace when they appear between expressions.  When they appear within an expression, they are handled according to their usual syntax classes.

     The function backward-prefix-chars moves back over these characters, as well as over characters whose primary syntax class is prefix (').  See Motion and Syntax.

Syntax Table Functions
======================

In this section we describe functions for creating, accessing and altering syntax tables.

 -- Function: make-syntax-table &optional table
     This function creates a new syntax table.  If TABLE is non-nil, the parent of the new syntax table is TABLE; otherwise, the parent is the standard syntax table.

     In the new syntax table, all characters are initially given the ``inherit'' (@) syntax class, i.e., their syntax is inherited from the parent table (see Syntax Class Table).

 -- Function: copy-syntax-table &optional table
     This function constructs a copy of TABLE and returns it.  If TABLE is omitted or nil, it returns a copy of the standard syntax table.  Otherwise, an error is signaled if TABLE is not a syntax table.

 -- Command: modify-syntax-entry char syntax-descriptor &optional table
     This function sets the syntax entry for CHAR according to SYNTAX-DESCRIPTOR.  CHAR must be a character, or a cons cell of the form (MIN . MAX); in the latter case, the function sets the syntax entries for all characters in the range between MIN and MAX, inclusive.

     The syntax is changed only for TABLE, which defaults to the current buffer's syntax table, and not in any other syntax table.

     The argument SYNTAX-DESCRIPTOR is a syntax descriptor, i.e., a string whose first character is a syntax class designator and whose second and subsequent characters optionally specify a matching character and syntax flags.  See Syntax Descriptors.  An error is signaled if SYNTAX-DESCRIPTOR is not a valid syntax descriptor.

     This function always returns nil.  The old syntax information in the table for this character is discarded.

          Examples:

          ;; Put the space character in class whitespace.
          (modify-syntax-entry ?\s " ")
               => nil

          ;; Make $ an open parenthesis character,
          ;;   with ^ as its matching close.
          (modify-syntax-entry ?$ "(^")
               => nil

          ;; Make ^ a close parenthesis character,
          ;;   with $ as its matching open.
          (modify-syntax-entry ?^ ")$")
               => nil

          ;; Make / a punctuation character,
          ;;   the first character of a start-comment sequence,
          ;;   and the second character of an end-comment sequence.
          ;;   This is used in C mode.
          (modify-syntax-entry ?/ ". 14")
               => nil

 -- Function: char-syntax character
     This function returns the syntax class of CHARACTER, represented by its designator character (see Syntax Class Table).  This returns only the class, not its matching character or syntax flags.

     The following examples apply to C mode.  (We use string to make it easier to see the character returned by char-syntax.)

          ;; Space characters have whitespace syntax class.
          (string (char-syntax ?\s))
               => " "

          ;; Forward slash characters have punctuation syntax.
          ;; Note that this char-syntax call does not reveal
          ;; that it is also part of comment-start and -end sequences.
          (string (char-syntax ?/))
               => "."

          ;; Open parenthesis characters have open parenthesis syntax.
          ;; Note that this char-syntax call does not reveal that
          ;; it has a matching character, ).
          (string (char-syntax ?\())
               => "("

 -- Function: set-syntax-table table
     This function makes TABLE the syntax table for the current buffer. It returns TABLE.

 -- Function: syntax-table
     This function returns the current syntax table, which is the table for the current buffer.

 -- Command: describe-syntax &optional buffer
     This command displays the contents of the syntax table of BUFFER (by default, the current buffer) in a help buffer.

 -- Macro: with-syntax-table table body...
     This macro executes BODY using TABLE as the current syntax table.  It returns the value of the last form in BODY, after restoring the old current syntax table.

     Since each buffer has its own current syntax table, we should make that more precise: with-syntax-table temporarily alters the current syntax table of whichever buffer is current at the time the macro execution starts.  Other buffers are not affected.

Syntax Properties
=================

When the syntax table is not flexible enough to specify the syntax of a language, you can override the syntax table for specific character occurrences in the buffer, by applying a syntax-table text property.  See Text Properties, for how to apply text properties.

The valid values of syntax-table text property are:

SYNTAX-TABLE
     If the property value is a syntax table, that table is used instead of the current buffer's syntax table to determine the syntax for the underlying text character.

(SYNTAX-CODE . MATCHING-CHAR)
     A cons cell of this format is a raw syntax descriptor (see Syntax Table Internals), which directly specifies a syntax class for the underlying text character.

nil
     If the property is nil, the character's syntax is determined from the current syntax table in the usual way.

 -- Variable: parse-sexp-lookup-properties
     If this is non-nil, the syntax scanning functions, like forward-sexp, pay attention to syntax text properties. Otherwise they use only the current syntax table.

 -- Variable: syntax-propertize-function
     This variable, if non-nil, should store a function for applying syntax-table properties to a specified stretch of text.  It is intended to be used by major modes to install a function which applies syntax-table properties in some mode-appropriate way.

     The function is called by syntax-ppss (see Position Parse), and by Font Lock mode during syntactic fontification (see Syntactic Font Lock).  It is called with two arguments, START and END, which are the starting and ending positions of the text on which it should act.  It is allowed to call syntax-ppss on any position before END.  However, it should not call syntax-ppss-flush-cache; so, it is not allowed to call syntax-ppss on some position and later modify the buffer at an earlier position.

 -- Variable: syntax-propertize-extend-region-functions
     This abnormal hook is run by the syntax parsing code prior to calling syntax-propertize-function.  Its role is to help locate safe starting and ending buffer positions for passing to syntax-propertize-function.  For example, a major mode can add a function to this hook to identify multi-line syntactic constructs, and ensure that the boundaries do not fall in the middle of one.

     Each function in this hook should accept two arguments, START and END.  It should return either a cons cell of two adjusted buffer positions, (NEW-START . NEW-END), or nil if no adjustment is necessary.  The hook functions are run in turn, repeatedly, until they all return nil.

Motion and Syntax
=================

This section describes functions for moving across characters that have certain syntax classes.

 -- Function: skip-syntax-forward syntaxes &optional limit
     This function moves point forward across characters having syntax classes mentioned in SYNTAXES (a string of syntax class characters).  It stops when it encounters the end of the buffer, or position LIMIT (if specified), or a character it is not supposed to skip.

     If SYNTAXES starts with ^, then the function skips characters whose syntax is not in SYNTAXES.

     The return value is the distance traveled, which is a nonnegative integer.

 -- Function: skip-syntax-backward syntaxes &optional limit
     This function moves point backward across characters whose syntax classes are mentioned in SYNTAXES.  It stops when it encounters the beginning of the buffer, or position LIMIT (if specified), or a character it is not supposed to skip.

     If SYNTAXES starts with ^, then the function skips characters whose syntax is not in SYNTAXES.

     The return value indicates the distance traveled.  It is an integer that is zero or less.

 -- Function: backward-prefix-chars
     This function moves point backward over any number of characters with expression prefix syntax.  This includes both characters in the expression prefix syntax class, and characters with the p flag.

Parsing Expressions
===================

This section describes functions for parsing and scanning balanced expressions.  We will refer to such expressions as sexps, following the terminology of Lisp, even though these functions can act on languages other than Lisp.  Basically, a sexp is either a balanced parenthetical grouping, a string, or a symbol (i.e., a sequence of characters whose syntax is either word constituent or symbol constituent).  However, characters in the expression prefix syntax class (see Syntax Class Table) are treated as part of the sexp if they appear next to it.

The syntax table controls the interpretation of characters, so these functions can be used for Lisp expressions when in Lisp mode and for C expressions when in C mode.  See List Motion, for convenient higher-level functions for moving over balanced expressions.

A character's syntax controls how it changes the state of the parser, rather than describing the state itself.  For example, a string delimiter character toggles the parser state between in-string and in-code, but the syntax of characters does not directly say whether they are inside a string.  For example (note that 15 is the syntax code for generic string delimiters),

     (put-text-property 1 9 'syntax-table '(15 . nil))

does not tell Emacs that the first eight chars of the current buffer are a string, but rather that they are all string delimiters.  As a result, Emacs treats them as four consecutive empty string constants.

Motion Commands Based on Parsing
--------------------------------

This section describes simple point-motion functions that operate based on parsing expressions.

 -- Function: scan-lists from count depth
     This function scans forward COUNT balanced parenthetical groupings from position FROM.  It returns the position where the scan stops.  If COUNT is negative, the scan moves backwards.

     If DEPTH is nonzero, treat the starting position as being DEPTH parentheses deep.  The scanner moves forward or backward through the buffer until the depth changes to zero COUNT times. Hence, a positive value for DEPTH has the effect of moving out DEPTH levels of parenthesis from the starting position, while a negative DEPTH has the effect of moving deeper by -DEPTH levels of parenthesis.

     Scanning ignores comments if parse-sexp-ignore-comments is non-nil.

     If the scan reaches the beginning or end of the accessible part of the buffer before it has scanned over COUNT parenthetical groupings, the return value is nil if the depth at that point is zero; if the depth is non-zero, a scan-error error is signaled.

 -- Function: scan-sexps from count
     This function scans forward COUNT sexps from position FROM. It returns the position where the scan stops.  If COUNT is negative, the scan moves backwards.

     Scanning ignores comments if parse-sexp-ignore-comments is non-nil.

     If the scan reaches the beginning or end of (the accessible part of) the buffer while in the middle of a parenthetical grouping, an error is signaled.  If it reaches the beginning or end between groupings but before count is used up, nil is returned.

 -- Function: forward-comment count
     This function moves point forward across COUNT complete comments (that is, including the starting delimiter and the terminating delimiter if any), plus any whitespace encountered on the way.  It moves backward if COUNT is negative.  If it encounters anything other than a comment or whitespace, it stops, leaving point at the place where it stopped.  This includes (for instance) finding the end of a comment when moving forward and expecting the beginning of one. The function also stops immediately after moving over the specified number of complete comments.  If COUNT comments are found as expected, with nothing except whitespace between them, it returns t; otherwise it returns nil.

     This function cannot tell whether the comments it traverses are embedded within a string.  If they look like comments, it treats them as comments.

     To move forward over all comments and whitespace following point, use (forward-comment (buffer-size)).  (buffer-size) is a good argument to use, because the number of comments in the buffer cannot exceed that many.

Finding the Parse State for a Position
--------------------------------------

For syntactic analysis, such as in indentation, often the useful thing is to compute the syntactic state corresponding to a given buffer position.  This function does that conveniently.

 -- Function: syntax-ppss &optional pos
     This function returns the parser state that the parser would reach at position POS starting from the beginning of the buffer. See Parser State, for a description of the parser state.

     The return value is the same as if you call the low-level parsing function parse-partial-sexp to parse from the beginning of the buffer to POS (see Low-Level Parsing).  However, syntax-ppss uses a cache to speed up the computation.  Due to this optimization, the second value (previous complete subexpression) and sixth value (minimum parenthesis depth) in the returned parser state are not meaningful.

     This function has a side effect: it adds a buffer-local entry to before-change-functions (see Change Hooks) for syntax-ppss-flush-cache (see below).  This entry keeps the cache consistent as the buffer is modified.  However, the cache might not be updated if syntax-ppss is called while before-change-functions is temporarily let-bound, or if the buffer is modified without running the hook, such as when using inhibit-modification-hooks.  In those cases, it is necessary to call syntax-ppss-flush-cache explicitly.

 -- Function: syntax-ppss-flush-cache beg &rest ignored-args
     This function flushes the cache used by syntax-ppss, starting at position BEG.  The remaining arguments, IGNORED-ARGS, are ignored; this function accepts them so that it can be directly used on hooks such as before-change-functions (see Change Hooks).

Parser State
------------

A parser state is a list of (currently) eleven elements describing the state of the syntactic parser, after it parses the text between a specified starting point and a specified end point in the buffer.  Parsing functions such as syntax-ppss (see Position Parse) return a parser state as the value.  Some parsing functions accept a parser state as an argument, for resuming parsing.

Here are the meanings of the elements of the parser state:

 0. The depth in parentheses, counting from 0.  Warning: this can be negative if there are more close parens than open parens between the parser's starting point and end point.

 1. The character position of the start of the innermost parenthetical grouping containing the stopping point; nil if none.

 2. The character position of the start of the last complete subexpression terminated; nil if none.

 3. Non-nil if inside a string.  More precisely, this is the character that will terminate the string, or t if a generic string delimiter character should terminate it.

 4. t if inside a non-nestable comment (of any comment style; see Syntax Flags); or the comment nesting level if inside a comment that can be nested.

 5. t if the end point is just after a quote character.

 6. The minimum parenthesis depth encountered during this scan.

 7. What kind of comment is active: nil if not in a comment or in a comment of style a; 1 for a comment of style b; 2 for a comment of style c; and syntax-table for a comment that should be ended by a generic comment delimiter character.

 8. The string or comment start position.  While inside a comment, this is the position where the comment began; while inside a string, this is the position where the string began.  When outside of strings and comments, this element is nil.

 9. The list of the positions of the currently open parentheses, starting with the outermost.

 10. When the last buffer position scanned was the (potential) first character of a two character construct (comment delimiter or escaped/char-quoted character pair), the SYNTAX-CODE (see Syntax Table Internals) of that position.  Otherwise nil.

Elements 1, 2, and 6 are ignored in a state which you pass as an argument to continue parsing.  Elements 9 and 10 are mainly used internally by the parser code.

One additional piece of useful information is available from a parser state using this function:

 -- Function: syntax-ppss-toplevel-pos state
     This function extracts, from parser state STATE, the last position scanned in the parse which was at top level in grammatical structure.  ``At top level'' means outside of any parentheses, comments, or strings.

     The value is nil if STATE represents a parse which has arrived at a top level position.

Low-Level Parsing
-----------------

The most basic way to use the expression parser is to tell it to start at a given position with a certain state, and parse up to a specified end position.

 -- Function: parse-partial-sexp start limit &optional target-depth stop-before state stop-comment
     This function parses a sexp in the current buffer starting at START, not scanning past LIMIT.  It stops at position LIMIT or when certain criteria described below are met, and sets point to the location where parsing stops.  It returns a parser state (see Parser State) describing the status of the parse at the point where it stops.

     If the third argument TARGET-DEPTH is non-nil, parsing stops if the depth in parentheses becomes equal to TARGET-DEPTH. The depth starts at 0, or at whatever is given in STATE.

     If the fourth argument STOP-BEFORE is non-nil, parsing stops when it comes to any character that starts a sexp.  If STOP-COMMENT is non-nil, parsing stops after the start of an unnested comment.  If STOP-COMMENT is the symbol syntax-table, parsing stops after the start of an unnested comment or a string, or after the end of an unnested comment or a string, whichever comes first.

     If STATE is nil, START is assumed to be at the top level of parenthesis structure, such as the beginning of a function definition.  Alternatively, you might wish to resume parsing in the middle of the structure.  To do this, you must provide a STATE argument that describes the initial status of parsing.  The value returned by a previous call to parse-partial-sexp will do nicely.

Parameters to Control Parsing
-----------------------------

 -- Variable: multibyte-syntax-as-symbol
     If this variable is non-nil, scan-sexps treats all non-ASCII characters as symbol constituents regardless of what the syntax table says about them.  (However, text properties can still override the syntax.)

 -- User Option: parse-sexp-ignore-comments
     If the value is non-nil, then comments are treated as whitespace by the functions in this section and by forward-sexp, scan-lists and scan-sexps.

The behavior of parse-partial-sexp is also affected by parse-sexp-lookup-properties (see Syntax Properties).

 -- Variable: comment-end-can-be-escaped
     If this buffer local variable is non-nil, a single character which usually terminates a comment doesn't do so when that character is escaped.  This is used in C and C++ Modes, where line comments starting with // can be continued onto the next line by escaping the newline with \.

You can use forward-comment to move forward or backward over one comment or several comments.

Syntax Table Internals
======================

Syntax tables are implemented as char-tables (see Char-Tables), but most Lisp programs don't work directly with their elements. Syntax tables do not store syntax data as syntax descriptors (see Syntax Descriptors); they use an internal format, which is documented in this section.  This internal format can also be assigned as syntax properties (see Syntax Properties).

Each entry in a syntax table is a raw syntax descriptor: a cons cell of the form (SYNTAX-CODE . MATCHING-CHAR).  SYNTAX-CODE is an integer which encodes the syntax class and syntax flags, according to the table below.  MATCHING-CHAR, if non-nil, specifies a matching character (similar to the second character in a syntax descriptor).

Here are the syntax codes corresponding to the various syntax classes:

Code   Class   Code   Class

0   whitespace   8   paired delimiter

1   punctuation   9   escape

2   word   10   character quote

3   symbol   11   comment-start

4   open parenthesis   12   comment-end

5   close parenthesis   13   inherit

6   expression prefix   14   generic comment

7   string quote   15   generic string

For example, in the standard syntax table, the entry for ( is (4 . 41).  41 is the character code for ).

Syntax flags are encoded in higher order bits, starting 16 bits from the least significant bit.  This table gives the power of two which corresponds to each syntax flag.

Prefix   Flag   Prefix   Flag

1   (lsh 1 16)   p   (lsh 1 20)

2   (lsh 1 17)   b   (lsh 1 21)

3   (lsh 1 18)   n   (lsh 1 22)

4   (lsh 1 19)

 -- Function: string-to-syntax desc
     Given a syntax descriptor DESC (a string), this function returns the corresponding raw syntax descriptor.

 -- Function: syntax-after pos
     This function returns the raw syntax descriptor for the character in the buffer after position POS, taking account of syntax properties as well as the syntax table.  If POS is outside the buffer's accessible portion (see accessible portion), the return value is nil.

 -- Function: syntax-class syntax
     This function returns the syntax code for the raw syntax descriptor SYNTAX.  More precisely, it takes the raw syntax descriptor's SYNTAX-CODE component, masks off the high 16 bits which record the syntax flags, and returns the resulting integer.

     If SYNTAX is nil, the return value is returns nil. This is so that the expression

          (syntax-class (syntax-after pos))

     evaluates to nil if pos is outside the buffer's accessible portion, without throwing errors or returning an incorrect code.

Categories
==========

Categories provide an alternate way of classifying characters syntactically.  You can define several categories as needed, then independently assign each character to one or more categories.  Unlike syntax classes, categories are not mutually exclusive; it is normal for one character to belong to several categories.

Each buffer has a category table which records which categories are defined and also which characters belong to each category.  Each category table defines its own categories, but normally these are initialized by copying from the standard categories table, so that the standard categories are available in all modes.

Each category has a name, which is an ASCII printing character in the range   to ~.  You specify the name of a category when you define it with define-category.

The category table is actually a char-table (see Char-Tables). The element of the category table at index C is a category set---a bool-vector---that indicates which categories character C belongs to.  In this category set, if the element at index CAT is t, that means category CAT is a member of the set, and that character C belongs to category CAT.

For the next three functions, the optional argument TABLE defaults to the current buffer's category table.

 -- Function: define-category char docstring &optional table
     This function defines a new category, with name CHAR and documentation DOCSTRING, for the category table TABLE.

     Here's an example of defining a new category for characters that have strong right-to-left directionality (see Bidirectional Display) and using it in a special category table.  To obtain the information about the directionality of characters, the example code uses the bidi-class Unicode property (see bidi-class).

          (defvar special-category-table-for-bidi
            ;;     Make an empty category-table.
            (let ((category-table (make-category-table))
                  ;; Create a char-table which gives the 'bidi-class' Unicode
                  ;; property for each character.
                  (uniprop-table (unicode-property-table-internal 'bidi-class)))
              (define-category ?R "Characters of bidi-class R, AL, or RLO"
                               category-table)
              ;; Modify the category entry of each character whose 'bidi-class'
              ;; Unicode property is R, AL, or RLO -- these have a
              ;; right-to-left directionality.
              (map-char-table
               #'(lambda (key val)
                   (if (memq val '(R AL RLO))
                       (modify-category-entry key ?R category-table)))
               uniprop-table)
              category-table))

 -- Function: category-docstring category &optional table
     This function returns the documentation string of category CATEGORY in category table TABLE.

          (category-docstring ?a)
               => "ASCII"
          (category-docstring ?l)
               => "Latin"

 -- Function: get-unused-category &optional table
     This function returns a category name (a character) which is not currently defined in TABLE.  If all possible categories are in use in TABLE, it returns nil.

 -- Function: category-table
     This function returns the current buffer's category table.

 -- Function: category-table-p object
     This function returns t if OBJECT is a category table, otherwise nil.

 -- Function: standard-category-table
     This function returns the standard category table.

 -- Function: copy-category-table &optional table
     This function constructs a copy of TABLE and returns it.  If TABLE is not supplied (or is nil), it returns a copy of the standard category table.  Otherwise, an error is signaled if TABLE is not a category table.

 -- Function: set-category-table table
     This function makes TABLE the category table for the current buffer.  It returns TABLE.

 -- Function: make-category-table
     This creates and returns an empty category table.  In an empty category table, no categories have been allocated, and no characters belong to any categories.

 -- Function: make-category-set categories
     This function returns a new category set---a bool-vector---whose initial contents are the categories listed in the string CATEGORIES.  The elements of CATEGORIES should be category names; the new category set has t for each of those categories, and nil for all other categories.

          (make-category-set "al")
               => #&128"\0\0\0\0\0\0\0\0\0\0\0\0\2\20\0\0"

 -- Function: char-category-set char
     This function returns the category set for character CHAR in the current buffer's category table.  This is the bool-vector which records which categories the character CHAR belongs to.  The function char-category-set does not allocate storage, because it returns the same bool-vector that exists in the category table.

          (char-category-set ?a)
               => #&128"\0\0\0\0\0\0\0\0\0\0\0\0\2\20\0\0"

 -- Function: category-set-mnemonics category-set
     This function converts the category set CATEGORY-SET into a string containing the characters that designate the categories that are members of the set.

          (category-set-mnemonics (char-category-set ?a))
               => "al"

 -- Function: modify-category-entry char category &optional table reset
     This function modifies the category set of CHAR in category table TABLE (which defaults to the current buffer's category table).  CHAR can be a character, or a cons cell of the form (MIN . MAX); in the latter case, the function modifies the category sets of all characters in the range between MIN and MAX, inclusive.

     Normally, it modifies a category set by adding CATEGORY to it. But if RESET is non-nil, then it deletes CATEGORY instead.

 -- Command: describe-categories &optional buffer-or-name
     This function describes the category specifications in the current category table.  It inserts the descriptions in a buffer, and then displays that buffer.  If BUFFER-OR-NAME is non-nil, it describes the category table of that buffer instead.