[libSyntax] Don't cache token nodes

[ahoppen]

It turns out that caching is actually more expensive than just creating new nodes. I assume this is because memory allocation has become fast through the bump allocator and the cache lookup requires computing a hash value which is rather expensive.

In terms of memory, I measured the memory used by the SyntaxArena when parsing Alamofire, which increases from 1.07MB to 1.24MB (15% increase), which should be negligible compared to the >100MB overall memory footprint of swift-frontend. I haven’t measured again, but the performance increase when creating a libSyntax tree was significant (something like 15-40%, I can measure again if you’d like more accurate numbers).

Also, out of curiosity, I measured how many cache hits we were actually seeing and 75% of all tokens could be served from the cache. This tells me that the majority of the SyntaxArena's memory is used by layout nodes.

[ahoppen]

@swift-ci Please smoke test

[akyrtzi]

IIRC I came to same conclusion on the SwiftSyntax side and removed the caching there as well.

I was thinking at the time we could reconsider caching as bulk creating plain token nodes in advance and re-using them process-wide, but perhaps it would complicate memory management a bit so not sure how much it is worth.

[ahoppen]

I was thinking at the time we could reconsider caching as bulk creating plain token nodes in advance and re-using them process-wide, but perhaps it would complicate memory management a bit so not sure how much it is worth.

As in, create tokens for the common keywords with common trivia (e.g. one trailing space) and have an efficient look-up structure for them?

[akyrtzi]

I was thinking at the time we could reconsider caching as bulk creating plain token nodes in advance and re-using them process-wide, but perhaps it would complicate memory management a bit so not sure how much it is worth.

As in, create tokens for the common keywords with common trivia (e.g. one trailing space) and have an efficient look-up structure for them?

Yes, exactly.

[ahoppen]

@swift-ci Please smoke test