𝑙𝑒𝑡𝑡𝑒𝑟→[𝐴−𝑍𝑎−𝑧]
𝑑𝑖𝑔𝑖𝑡→[0−9]
𝑖𝑑→𝑙𝑒𝑡𝑡𝑒𝑟 (𝑙𝑒𝑡𝑡𝑒𝑟 | 𝑑𝑖𝑔𝑖𝑡)*
𝑛𝑢𝑚𝑏𝑒𝑟→𝑑𝑖𝑔𝑖𝑡+
𝑤𝑠→(𝑏𝑙𝑎𝑛𝑘 | 𝑡𝑎𝑏 | 𝑛𝑒𝑤𝑙𝑖𝑛𝑒)+
For the string given below,
x1 23 𝑚𝑚 78 𝑦 7𝑧 𝑧𝑧5 14𝐴 8𝐻 𝐴𝑎𝑌𝑐𝐷
the number of tokens (excluding 𝑤𝑠) that will be produced by the lexical analyzer is __________. (answer in integer)
Correct : 13
The correct answer is 13.
The lexical analyzer applies the maximal munch rule - it always tries to match the longest possible token. The token rules are: id must start with a letter (followed by letters/digits), number is one or more digits, and ws (whitespace) is excluded from the count.
Tokenizing each whitespace-separated segment:
x1 → starts with letter, followed by digit → id (1 token)
23 → digits only → number (1 token)
mm → letters only → id (1 token)
78 → digits only → number (1 token)
y → single letter → id (1 token)
7z → digit first → number: 7; then letter z cannot extend the number and cannot be skipped → id: z (2 tokens)
zz5 → starts with letter → id: zz5 (1 token)
14A → digit first → number: 14; then letter A → id: A (2 tokens)
8H → digit first → number: 8; then letter H → id: H (2 tokens)
AaYcD → starts with letter, all letters → id: AaYcD (1 token)
Total tokens (excluding ws) = 1+1+1+1+1+2+1+2+2+1 = 13.
Similar Questions
Total Unique Visitors