3 Racket Lexer

▼ |  | Syntax Color: Utilities

1 | Parenthesis Matching
2 | Lexer Contract and The Don’t-Stop Structure Type
3 | Racket Lexer
4 | Default Lexer
5 | Module Lexer
6 | Scribble Lexer
7 | Splay Tree for Tokenization
8 | Interface for Clients of Syntax Color Output
9 | Racket S-Expression Indentation and Navigation

On this page:

racket-lexer
racket-lexer*
racket-lexer/ status
racket-lexer*/ status
racket-nobar-lexer/ status
racket-nobar-lexer*/ status
current-lexeme->semantic-type-guess

top contents ← prev up next →

## 3 Racket Lexer🔗ℹ

(require syntax-color/racket-lexer)
| package: syntax-color-lib

> > procedure
> > (racket-lexer in) |  | → |  |
> > (or/c string? eof-object?)
> > symbol?
> > (or/c symbol? #f)
> > (or/c number? #f)
> > (or/c number? #f)
> in : input-port?

A lexer for Racket, including reader extensions (Reader Extension), built specifically for color:text<%>.

The racket-lexer function returns 5 values:

- Either a string containing the matching text or the eof object. Block comments and specials currently return an empty string. This may change in the future to other string or non-string data.

- A symbol in '(error comment sexp-comment white-space constant string no-color parenthesis hash-colon-keyword symbol eof other).

- A symbol in '(|(| |)| |[| |]| |{| |}|) or #f.

- A number representing the starting position of the match (or #f if eof).

- A number representing the ending position of the match (or #f if eof).

> procedure (racket-lexer* in offset mode)
> | → |  |
> (or/c string? eof-object?)
> (or/c symbol?
> (and/c (hash/c symbol? any/c) immutable?))
> (or/c symbol? #f)
> (or/c number? #f)
> (or/c number? #f)
> exact-nonnegative-integer?
> any/c
> in : input-port?
> offset : exact-nonnegative-integer?
> mode : any/c

Like racket-lexer, but uses the extended lexer protocol to track and report regions that are commented out with #;. It also uses current-lexeme->semantic-type-guess to potentially add 'semantic-type-guess to the second result.

Added in version 1.2 of package syntax-color-lib.
Changed in version 1.7: Added use of current-lexeme->semantic-type-guess.

> > procedure
> > (racket-lexer/status in) |  | → |  |
> > (or/c string? eof-object?)
> > symbol?
> > (or/c symbol? #f)
> > (or/c number? #f)
> > (or/c number? #f)
> > (or/c 'datum 'open 'close 'continue)
> in : input-port?

Like racket-lexer, but returns an extra value. The last return value indicates whether the consumed token should count as a datum, an opening parenthesis (or similar starting token to group other tokens), a closing parenthesis (or similar), or a prefix (such as whitespace) on a datum.

> procedure (racket-lexer*/status in offset mode)
> | → |  |
> (or/c string? eof-object?)
> (or/c symbol?
> (and/c (hash/c symbol? any/c) immutable?))
> (or/c symbol? #f)
> (or/c number? #f)
> (or/c number? #f)
> exact-nonnegative-integer?
> any/c
> (or/c 'datum 'open 'close 'continue)
> in : input-port?
> offset : exact-nonnegative-integer?
> mode : any/c

Like racket-lexer/status, but with comment tracking and 'semantic-type-guess addition like racket-lexer*.

Added in version 1.2 of package syntax-color-lib.
Changed in version 1.7: Added use of current-lexeme->semantic-type-guess.

> procedure (racket-nobar-lexer/status in)
> | → |  |
> (or/c string? eof-object?)
> symbol?
> (or/c symbol? #f)
> (or/c number? #f)
> (or/c number? #f)
> (or/c 'datum 'open 'close 'continue)
> in : input-port?

Like racket-lexer/status, except it treats | as a delimiter instead of quoting syntax for a symbol. This function is used by scribble-lexer.

> procedure (racket-nobar-lexer*/status in offset mode)
> | → |  |
> (or/c string? eof-object?)
> (or/c symbol?
> (and/c (hash/c symbol? any/c) immutable?))
> (or/c symbol? #f)
> (or/c number? #f)
> (or/c number? #f)
> exact-nonnegative-integer?
> any/c
> (or/c 'datum 'open 'close 'continue)
> in : input-port?
> offset : exact-nonnegative-integer?
> mode : any/c

Like racket-nobar-lexer/status, but with comment tracking and 'semantic-type-guess addition like racket-lexer*.

Added in version 1.2 of package syntax-color-lib.
Changed in version 1.7: Added use of current-lexeme->semantic-type-guess.

> parameter (current-lexeme->semantic-type-guess)
> → (string . -> . (or/c #f symbol?))
> (current-lexeme->semantic-type-guess proc) → void?
> proc : (string . -> . (or/c #f symbol?))

A parameter to determine when a 'semantic-type-guess attribute should be added to a token result of type 'symbol by racket-lexer* and similar functions. An attribute is added whenever the proc value of the parameter returns a symbol (instead of #f).

Added in version 1.7 of package syntax-color-lib.

top contents ← prev up next →