Low-level APIs

Tomlyn provides public low-level building blocks for tooling, analyzers, formatters, and advanced scenarios. These APIs live in Tomlyn.Parsing and Tomlyn.Syntax.

If you want to map TOML to .NET objects, start with Serialization instead. These low-level APIs are for building custom tooling or when you need full control over parsing.

API	Purpose	Allocations
`TomlLexer`	Token stream (syntax highlighting, validation)	Minimal - struct-based iteration
`TomlParser`	Incremental semantic events (custom readers)	Minimal - no intermediate DOM
`SyntaxParser`	Full-fidelity syntax tree (round-tripping, tooling)	Allocates tree nodes

TomlLexer (tokenization)

TomlLexer produces a flat stream of tokens. Use it for syntax highlighting, quick validation, or building custom parsers.

using Tomlyn.Parsing;

var lexer = TomlLexer.Create("name = \"Ada\"", sourceName: "config.toml");
while (lexer.MoveNext())
{
    var token = lexer.Current;
    Console.WriteLine($"{token.Kind} = {token.StringValue} @ {lexer.CurrentSpan}");
}

Token kinds

Tokens are identified by TokenKind:

Category	Kinds
Structure	`OpenBracket`, `CloseBracket`, `OpenBracketDouble`, `CloseBracketDouble`, `OpenBrace`, `CloseBrace`, `Equal`, `Comma`, `Dot`
Strings	`String`, `StringMulti`, `StringLiteral`, `StringLiteralMulti`
Numbers	`Integer`, `IntegerHexa`, `IntegerOctal`, `IntegerBinary`, `Float`
Booleans	`True`, `False`
Date/time	`OffsetDateTimeByZ`, `OffsetDateTimeByNumber`, `LocalDateTime`, `LocalDate`, `LocalTime`
Special	`Infinite`, `PositiveInfinite`, `NegativeInfinite`, `Nan`, `PositiveNan`, `NegativeNan`
Trivia	`Whitespaces`, `NewLine`, `Comment`
Meta	`BasicKey`, `Invalid`, `Eof`

Lexer options

TomlLexerOptions controls decoding behavior:

var options = new TomlLexerOptions { DecodeScalars = true };
var lexer = TomlLexer.Create(toml, options, sourceName: "config.toml");

When DecodeScalars is true, escape sequences in strings are decoded during tokenization.

TomlParser (incremental parse events)

TomlParser produces a sequence of TomlParseEvent values via MoveNext() without building an intermediate DOM. This is ideal for building custom readers or streaming processors.

using Tomlyn.Parsing;

var parser = TomlParser.Create("[server]\nhost = \"localhost\"\nport = 8080");
while (parser.MoveNext())
{
    ref readonly var evt = ref parser.Current;
    switch (evt.Kind)
    {
        case TomlParseEventKind.PropertyName:
            Console.Write($"{evt.GetString()} = ");
            break;
        case TomlParseEventKind.String:
            Console.WriteLine($"\"{evt.GetString()}\"");
            break;
        case TomlParseEventKind.Integer:
            Console.WriteLine(evt.GetInt64());
            break;
        case TomlParseEventKind.StartTable:
            Console.WriteLine($"[table depth={parser.Depth}]");
            break;
    }
}

Parse events

Kind	Description
`StartDocument` / `EndDocument`	Document boundaries.
`StartTable` / `EndTable`	Table boundaries (including inline tables).
`StartArray` / `EndArray`	Array boundaries (including table arrays).
`PropertyName`	A TOML key - use `evt.GetString()` to read it.
`String`	A string value - use `evt.GetString()`.
`Integer`	An integer value - use `evt.GetInt64()`.
`Float`	A float value - use `evt.GetDouble()`.
`Boolean`	A boolean value - use `evt.GetBoolean()`.
`DateTime`	A date/time value - use `evt.GetTomlDateTime()`.

Parser options

TomlParserOptions controls parsing behavior:

var parserOptions = new TomlParserOptions
{
    Mode = TomlParserMode.Tolerant,   // collect errors instead of throwing
    DecodeScalars = true,             // decode escape sequences
    CaptureTrivia = true,             // include comments/whitespace in events
    EagerStringValues = false,        // defer string materialization
};

var parser = TomlParser.Create(toml, parserOptions);

Option	Default	Description
`Mode`	`Strict`	`Strict` throws on first error; `Tolerant` collects diagnostics.
`DecodeScalars`	`false`	Decode escape sequences in strings.
`CaptureTrivia`	`false`	Include trivia tokens (comments, whitespace).
`EagerStringValues`	`false`	Materialize string values eagerly.

Input sources

TomlParser.Create(...) accepts string and TextReader:

// From a file stream
using var reader = new StreamReader("config.toml");
var parser = TomlParser.Create(reader);

SyntaxParser (full-fidelity syntax tree)

SyntaxParser builds a complete DocumentSyntax tree that preserves every character from the original input - comments, whitespace, formatting. This is the right tool for formatters, analyzers, linters, and round-trip editing.

doc.ToString() reproduces the exact original text. Modify the tree nodes in-place and call ToString() again to get the updated TOML with all formatting and comments preserved.

using Tomlyn.Parsing;

var doc = SyntaxParser.Parse("# config\nname = \"Ada\"\nage = 37", sourceName: "config.toml");

if (doc.HasErrors)
{
    foreach (var diag in doc.Diagnostics)
        Console.WriteLine(diag);
}

// Round-trip: ToString() reproduces the exact original text
Console.WriteLine(doc.ToString());

Parse vs ParseStrict

Method	Behavior
`SyntaxParser.Parse(...)`	Tolerant - collects errors in `doc.Diagnostics`, always returns a tree.
`SyntaxParser.ParseStrict(...)`	Strict - throws `TomlException` on the first error.

Both accept string and TomlLexer inputs. Parse also accepts TextReader.

Syntax tree structure

The tree root is DocumentSyntax, which contains:

KeyValues - top-level key-value pairs (SyntaxList<KeyValueSyntax>)
Tables - table and table-array sections (SyntaxList<TableSyntaxBase>)
Diagnostics - parse errors/warnings (DiagnosticsBag)
HasErrors - quick check for errors

Key node types:

Node	Description
`KeyValueSyntax`	A `key = value` pair.
`KeySyntax`	The key part (may be dotted).
`StringValueSyntax`	A string value (basic, literal, multiline).
`IntegerValueSyntax`	An integer value.
`FloatValueSyntax`	A float value.
`BooleanValueSyntax`	A boolean value.
`DateTimeValueSyntax`	A date/time value.
`ArraySyntax`	An array `[...]`.
`InlineTableSyntax`	An inline table `{...}`.
`TableSyntax`	A `[table]` header section.
`TableArraySyntax`	A `[[table_array]]` header section.

Trivia

Every SyntaxNode can carry LeadingTrivia and TrailingTrivia - lists of SyntaxTrivia containing comments, whitespace, and newlines.

Tree traversal

Use the visitor pattern or enumerable extensions:

using Tomlyn.Syntax;

// Visitor pattern
var visitor = new MyVisitor();
visitor.Visit(doc);

// Enumerable extensions
foreach (var node in doc.Descendants())
{
    if (node is StringValueSyntax str)
        Console.WriteLine(str.Value);
}

foreach (var token in doc.Tokens(includeCommentsAndWhitespaces: true))
{
    // Process every token
}

To implement a visitor, inherit from SyntaxVisitor and override the Visit methods for the node types you care about:

using Tomlyn.Syntax;

public sealed class StringCollector : SyntaxVisitor
{
    public List<string> Strings { get; } = new();

    protected override void Visit(StringValueSyntax node)
    {
        if (node.Value is not null)
            Strings.Add(node.Value);
        base.Visit(node);
    }
}

var collector = new StringCollector();
collector.Visit(doc);

Diagnostics

DiagnosticsBag is a collection of DiagnosticMessage items:

foreach (var diag in doc.Diagnostics)
{
    // diag.Kind    - Error or Warning
    // diag.Span    - source location
    // diag.Message - description
    Console.WriteLine($"{diag.Kind}: {diag.Message} at {diag.Span}");
}