单令牌提前查询的性能损失是多少？

Question

When comparing Go and Scala end of statement detection I found out that the rules for Scala are richer, namely:

A line ending is treated as a semicolon unless one of the following conditions is true:

• The line in question ends in a word that would not be legal as the end of a statement, such as a period or an infix operator.
• The next line begins with a word that cannot start a statement.
• The line ends while inside parentheses (...) or brackets [...], because these cannot contain multiple statements anyway.

Quoted from Scala - The rules of semicolon inference.

Rule #1 is how the Go works as well. Rule #3 too. The only difference is rule #2 – it involves single lookahead, since there is one token involved ("word").

What kind of performance penalty is involved: 1% slower, 5%, 10%?

I would love to see a comment (not the question) why Go designers left out that rule – if not for performance, it makes language more reliable, for example in method chaining:

x = some_object.select(...)
               .sort(...)
               .reverse(...)
               .where(...)
               .single()

If I am not mistaken for Go it is an error (you can solve it in two possible ways – taking entire statement in braces or expression in parentheses, but it is manual tweaking), Scala will take it as it should.

Answer 1

The performance penalty is utterly negligible compared to everything else that the compiler has to do. The Scala-internals mailing list has the following exchange between Haoyi Li and Martin Odersky regarding a parboiled2 parser Haoyi wrote for Scala:

Haoyi Li: In terms of perf[ormance], it can parse everything in scala/scala, lift, scalaz, scalajs, playframework and shapeless in 15 seconds.... Does anyone know how much of the time in the compiler and macros is spent parsing? My impression is that the vast vast vast majority of the time is spent in the typechecker.

Odersky: Yes, parsing is pretty insignificant compared to the other tasks of a compiler ... That said, the [parser for the next-generation Scala compiler] (hand-written, 2100 lines including error reporting, accurate positions and tree construction) achieves several hundred thousand lines a second. So parboiled still has some way to go to beat that :-)

When we're talking about hundreds of thousands of lines of code parsed per second including rule #2, one can infer that speed is not the issue. Go compilation tends to clock in at around 20k lines per second, so even if Go parsing took zero time, and the entire time for Scala parsing was taken up by the one-line lookahead, it would be less than 10% of a penalty to the build process.

In reality it should be more like 0%. Lookahead is usually really cheap; you've already got a token stream, so you just look at the next one.