The first stage of compiling, in which a stream of raw source-code characters is grouped into tokens, such as keywords, identifiers, literals and punctuation, that the later parsing stage can work with.
Facts
Core PrincipleLexical analysis is the conversion of a stream of raw source-code text into semantically or syntactically meaningful lexical tokens for later processing. 1 Connections
Sources
1. Lexical analysis - Wikipedia
Lexical token and lexical tokenization section
Lexical tokenization is conversion of a text into (semantically or syntactically) meaningful lexical tokens
In Group: Compiler and Language Implementation Techniques, Lead
Lexical tokenization is conversion of a text into (semantically or syntactically) meaningful lexical tokens belonging to categories defined by a "lexer" program.
View the SourceReader Challenges (0)
No disputes yet. Spotted an error or a better source? Open the first one.
Sign in to dispute this or suggest a correction.