From: Benjamin Woodruff <github@benjam.info>
Date: Wed, 8 May 2019 20:38:38 +0000 (-0400)
Subject: Move tokenizer config onto grammar, rename flag
X-Git-Url: https://git.madduck.net/etc/vim.git/commitdiff_plain/448885b256ca1741fda7c4ef17f80f750ea614c0?ds=sidebyside;hp=448885b256ca1741fda7c4ef17f80f750ea614c0

Move tokenizer config onto grammar, rename flag

Based on the feedback in
https://github.com/python/black/pull/845#issuecomment-490622711

- Remove TokenizerConfig, and add a field to Grammar instead.
- Pass the Grammar to the tokenizer.
- Rename `ASYNC_IS_RESERVED_KEYWORD` to `ASYNC_KEYWORDS` and
  `ASYNC_IS_VALID_IDENTIFIER` to `ASYNC_IDENTIFIERS`.
---