]> git.madduck.net Git - etc/vim.git/blob - blib2to3/pgen2/tokenize.pyi

madduck's git repository

Every one of the projects in this repository is available at the canonical URL git://git.madduck.net/madduck/pub/<projectpath> — see each project's metadata for the exact URL.

All patches and comments are welcome. Please squash your changes to logical commits before using git-format-patch and git-send-email to patches@git.madduck.net. If you'd read over the Git project's submission guidelines and adhered to them, I'd be especially grateful.

SSH access, as well as push access can be individually arranged.

If you use my repositories frequently, consider adding the following snippet to ~/.gitconfig and using the third clone URL listed for each project:

[url "git://git.madduck.net/madduck/"]
  insteadOf = madduck:

Add `explode_split` to documentation
[etc/vim.git] / blib2to3 / pgen2 / tokenize.pyi
1 # Stubs for lib2to3.pgen2.tokenize (Python 3.6)
2 # NOTE: Only elements from __all__ are present.
3
4 from typing import Callable, Iterable, Iterator, List, Text, Tuple
5 from blib2to3.pgen2.token import *  # noqa
6
7
8 _Coord = Tuple[int, int]
9 _TokenEater = Callable[[int, Text, _Coord, _Coord, Text], None]
10 _TokenInfo = Tuple[int, Text, _Coord, _Coord, Text]
11
12
13 class TokenError(Exception): ...
14 class StopTokenizing(Exception): ...
15
16 def tokenize(readline: Callable[[], Text], tokeneater: _TokenEater = ...) -> None: ...
17
18 class Untokenizer:
19     tokens: List[Text]
20     prev_row: int
21     prev_col: int
22     def __init__(self) -> None: ...
23     def add_whitespace(self, start: _Coord) -> None: ...
24     def untokenize(self, iterable: Iterable[_TokenInfo]) -> Text: ...
25     def compat(self, token: Tuple[int, Text], iterable: Iterable[_TokenInfo]) -> None: ...
26
27 def untokenize(iterable: Iterable[_TokenInfo]) -> Text: ...
28 def generate_tokens(
29     readline: Callable[[], Text]
30 ) -> Iterator[_TokenInfo]: ...