Currently all parsing is done in ast.c by checking if the text at the current location in the code is a known keyword/operator etc.. First tokenizing the input string should be faster as for small programs the lookahead function consumes a substantially part of the execution time.
