Module highlite

Source highlighter for programming or markup languages. Currently only few languages are supported, other languages may be added. The interface supports one language nested in another.


TokenClass = enum
  gtEof, gtNone, gtWhitespace, gtDecNumber, gtBinNumber, gtHexNumber, gtOctNumber,
  gtFloatNumber, gtIdentifier, gtKeyword, gtStringLit, gtLongStringLit, gtCharLit,
  gtEscapeSequence, gtOperator, gtPunctuation, gtComment, gtLongComment,
  gtRegularExpression, gtTagStart, gtTagEnd, gtKey, gtValue, gtRawData, gtAssembler,
  gtPreprocessor, gtDirective, gtCommand, gtRule, gtHyperlink, gtLabel, gtReference,
GeneralTokenizer = object of RootObj
  kind*: TokenClass
  start*, length*: int
  buf: cstring
  pos: int
  state: TokenClass
SourceLanguage = enum
  langNone, langNim, langCpp, langCsharp, langC, langJava, langYaml


sourceLanguageToStr: array[SourceLanguage, string] = ["none", "Nim", "C++", "C#", "C",
    "Java", "Yaml"]
tokenClassToStr: array[TokenClass, string] = ["Eof", "None", "Whitespace", "DecNumber",
    "BinNumber", "HexNumber", "OctNumber", "FloatNumber", "Identifier", "Keyword",
    "StringLit", "LongStringLit", "CharLit", "EscapeSequence", "Operator",
    "Punctuation", "Comment", "LongComment", "RegularExpression", "TagStart",
    "TagEnd", "Key", "Value", "RawData", "Assembler", "Preprocessor", "Directive",
    "Command", "Rule", "Hyperlink", "Label", "Reference", "Other"]


proc getSourceLanguage(name: string): SourceLanguage {...}{.raises: [], tags: [].}
Counts from ordinal value a up to b (inclusive) with the given step count. S, T may be any ordinal type, step may only be positive. Note: This fails to count to high(int) if T = int for efficiency reasons.
proc initGeneralTokenizer(g: var GeneralTokenizer; buf: cstring) {...}{.raises: [], tags: [].}
proc initGeneralTokenizer(g: var GeneralTokenizer; buf: string) {...}{.raises: [], tags: [].}
proc deinitGeneralTokenizer(g: var GeneralTokenizer) {...}{.raises: [], tags: [].}
proc getNextToken(g: var GeneralTokenizer; lang: SourceLanguage) {...}{.raises: [], tags: [].}

Raises AssertionError with msg if cond is false. Note that AssertionError is hidden from the effect system, so it doesn't produce {.raises: [AssertionError].}. This exception is only supposed to be caught by unit testing frameworks.

The compiler may not generate any code at all for assert if it is advised to do so through the -d:release or --assertions:off command line switches.

© 2006–2018 Andreas Rumpf
Licensed under the MIT License.