Version

Special Sequence Format

Topic Overview

Purpose

This topic explains the format of the special sequence sections in the EBNF file used to configure the grammar.

Required background

The following topics are prerequisites to understanding this topic:

Topic Purpose

This topic provides an overview of the Syntax Parsing Engine.

This topic provides an overview of the Syntax Parsing Engine’s Grammar.

This topic explains the grammar analysis performed by the Syntax Parsing Engine.

This topic gives an overview of the EBNF file format used to define a grammar.

Special Sequence Sections Overview

Special sequence sections summary

By using the EBNF format supported by the Syntax Parsing Engine you can supply a lot of descriptive information about grammar instance, lexer states, terminal symbols, and non-terminal symbols.

This information is specified using XML within special sequences of the EBNF file.

Special Sequence Section Types

Special sequence sections summary chart

Section Details

The grammar properties section defines the grammar, lexer states, and terminal symbols and is allowed only at the top of an EBNF file.

This section is used to set specific properties on the NonTerminalSymbol instance defined by the next rule.

Grammar Properties Section

Overview

The grammar properties section defines the grammar, lexer states, and terminal symbols and is allowed only at the top of an EBNF file. There can be only one grammar properties section.

Elements

The following tables list all elements allowed in the grammar properties section with their respective parent/child relations:

Grammar element:

Description: defines everything in a Grammar instance except the non-terminal symbols and syntax rule trees, which are done in the normal EBNF format.

Parent elements: (none)

Attribute Type * Description*

String

(optional) The value for the Grammar.EndOfStreamSymbolName property. If not specified, the value will be “EndOfStreamToken”.

String

(optional) The value for the Grammar.Name property. If not specified, the value will be null.

String

(optional) The value for the Grammar.NewLineSymbolName property. If not specified, the value will be “NewLineToken”

String

(optional) The name of the NonTerminalSymbol which should be set as the Grammar.StartSymbol value. If not specified, the first non-terminal symbol defined in the file will be used as the start symbol.

bool

(optional) The value for the Grammar.SupportsParsing property. If not specified, the value will be True.

SyntaxTreePruningMode

(optional) The value for the Grammar.SyntaxTreePruningMode property. If not specified, the value will be BasedOnChildrenAndName.

String

(optional) The value for the Grammar.UnrecognizedSymbolName property. If not specified, the value will be “UnrecognizedToken”.

String

(optional) The value for the Grammar.WhitespaceSymbolName property. If not specified, the value will be “WhitespaceToken”.

String

(optional) The value of the Grammar.WhitespacePattern property. If not specified, the value will be “[ \t]+”.

Child Element * Description*

Defines a TerminalSymbol and adds it to the default LexerState of the grammar (Grammar.LexerStates.DefaultLexerState).

TerminalSymbolReference

Adds an existing TerminalSymbol to the default LexerState of the grammar.

TerminalSymbol element:

Description: Creates a new TerminalSymbol instance and adds it to the owning lexer state.

Parent Elements: Grammar, LexerState

Attribute Type * Description*

(optional) The value indicating how the Value attribute should be interpreted. Possible values are Literal, LiteralIgnoreCase, and RegularExpression. If not specified, the value will be Literal.

bool

(optional) Indicates whether the new TerminalSymbol should be added to its owning lexer state as an exit symbol. See the Lexical Analysis Overview topic for more information on exit symbols.

bool

(optional) The value for the TerminalSymbol. IsLookaheadNegative property, which indicates whether the LookaheadPattern specifies a positive or negative lookahead. If not specified, the value will be False.

bool?

(optional) The value for the TerminalSymbol.IsSignificant property. If not specified, the value will be null.

bool

(optional) The value for the TerminalSymbol.IsStartOfErrorRecoveryPair property. If not specified, the value will be False.

String

(optional) The value for the TerminalSymbol. LanguageElementName property. If not specified, the value will be “Undefined”.

String

(optional) The name of the existing LexerState which should be set as the TerminalSymbol. LexerStateToEnter property. If not specified, the value will be null.

Note
Note

The lexer state with the specified name may be defined before or after the LexerStateToEnter attribute in the document.

String

(optional) The value for the TerminalSymbol.LookaheadPattern property, which is a regular expression that makes an assertion about the text that should or should not follow a token associated with this terminal symbol. If not specified, the value will be null.

String

(required) The value for the TerminalSymbol.Name property.

String

(optional) The value for the TerminalSymbol.Value property. If not specified, the value from the Name attribute will be used.

Child Element * Description*

LexerState

Defines a LexerState, adds it to the Grammar.LexerStates collection, and sets that LexerState instance as the TerminalSymbol.LexerStateToEnter value.

TerminalSymbolReference element

Description: Adds an existing TerminalSymbol instance to the owning lexer state.

Note
Note

The definition for the referenced TerminalSymbol may occur before or after this element in the document.

Parent Elements: Grammar, LexerState

Attribute Type * Description*

IsExitSymbol

bool

(optional) Indicates whether the new TerminalSymbol should be added to its owning lexer state as an exit symbol or not. See the Lexical Analysis Overview topic for more information on exit symbols.

String

(required) The name of the existing TerminalSymbol instance to add to the owning lexer state.

LexerState element

Description: Defines a new LexerState instance and adds it to the Grammar.LexerStates collection.

Parent Elements: TerminalSymbol

Attribute Type * Description*

String

(required) The name of the new LexerState instance.

Child Element * Description*

TerminalSymbol

Defines a TerminalSymbol and adds it to the parent LexerState.

TerminalSymbolReference

Adds an existing TerminalSymbol to the parent LexerState.

Example

The following is an example of a grammar properties section:

?
<Grammar Name="My Custom Grammar" NewLineSymbolName="LineBreakToken">
    <TerminalSymbolReference Name="LineBreakToken" />
    <TerminalSymbolReference Name="WhitespaceToken" />
    <TerminalSymbol Name="Word" Value="[\w]+" Comparison="RegularExpression" />
    <TerminalSymbol Name="Punctuation" Value="\W" Comparison="RegularExpression" />
    <TerminalSymbol Name="DoubleQuote" Value="&quot;" LanguageElementName="StringLiteral">
        <LexerState Name="StringLiteral">
            <TerminalSymbol Name="StringLiteralContent" Value="([^&quot;\\\r\n]|(\\[^\r\n]))+"
                Comparison="RegularExpression"LanguageElementName="StringLiteral" />
            <TerminalSymbolReference Name="LineBreakToken" IsExitSymbol="true" />
            <TerminalSymbolReference Name="DoubleQuote" IsExitSymbol="true" />
        </LexerState>
    </TerminalSymbol>
</Grammar>
?

Non-Terminal Symbol Attributes Sections

Overview

The non-terminal symbol attributes sections are allowed immediately before any non-terminal symbol definition. This section is used to set specific properties that will apply only to the next NonTerminalSymbol instance defined.

Elements

The following table lists all attributes for the NonTerminalSymbolOptions element:

Attribute Type * Description*

bool

(optional) The value for the NonTerminalSymbol.HasPriority property. If not specified, the value will be False.

bool

(optional) The value for the NonTerminalSymbol.IsError property. If not specified, the value will be False.

bool

(optional) The value for the NonTerminalSymbol.IsStartOfErrorRecoveryPair property. If not specified, the value will be False.

bool

(optional) The value for the NonTerminalSymbol.PreventPruning property. If not specified, the value will be False.

bool

(optional) The value for the NonTerminalSymbol.SuppressAmbiguityWarnings property. If not specified, the value will be False.

SuppressErrorRecoveryForSymbols

string

(optional) A comma separated list of the names of symbols which should be passed to the NonTerminalSymbol.SuppressErrorRecoveryForSymbol method. The named symbols can be defined before or after the NonTerminalSymbol.

Example

The following is an example of a non-terminal symbol definition along with its attributes section defined with XML between the “?” characters :

Document = {Sentence};
?<NonTerminalSymbolOptions SuppressErrorRecoveryForSymbols="Punctuation" />?
Sentence = {Word | StringLiteral}-, Punctuation;
StringLiteral = DoubleQuote, [StringLiteralContent], DoubleQuote;

Related Content

Topics

The following topics provide additional information related to this topic.

Topic Purpose

This topic explains the deviations from the ISO 14977 specification in the EBNF format used by the Syntax Parsing Engine.

This topic explains the process of creating a grammar from EBNF content.

This topic explains the process of creating EBNF content from a grammar.