Solved Problem with Non-ASCII .gitignore Files (#2229)

* Solved Problem with non-alphabetical .gitignore files

When .gitignore file in the user's project directory contained non-alphabetical
characters(Japanese, Korean, Chinese, etc), Nothing works and printed this
weird message in the console('cp949' is the encoding for Korean characters
in this case). It even blocks VSCode's formatting from working. This commit
solves the problem.

Traceback (most recent call last):
  File "c:\users\username\anaconda3\envs\project-name\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\username\anaconda3\envs\project-name\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\username\anaconda3\envs\project-name\Scripts\black.exe\__main__.py", line 7, in <module>
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\__init__.py", line 1056, in patched_main
    main()
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\__init__.py", line 394, in main
    stdin_filename=stdin_filename,
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\__init__.py", line 445, in get_sources
    gitignore = get_gitignore(root)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\files.py", line 122, in get_gitignore
    lines = gf.readlines()
UnicodeDecodeError: 'cp949' codec can't decode byte 0xb0 in position 13: illegal multibyte sequence

* Made .gitignore File Reader Detect Its Encoding
* Revert "Made .gitignore File Reader Detect Its Encoding"

  This reverts commit 6c3a7ea42b5b1e441cc0026c8205d1cee68c1bba.

* Revert "Solved Problem with non-alphabetical .gitignore files"

  This reverts commit b0100b5d91c2f5db544a60f34aafab120f0aa458.

* Made .gitignore Reader Open the File with Auto Encoding Detecting

  https://docs.python.org/3.8/library/tokenize.html#tokenize.open

* Revert "Made .gitignore Reader Open the File with Auto Encoding Detecting"

  This reverts commit 50dd80422938649ccc8c7f43aac752f9f6481779.

* Made .gitignore Reader Use UTF-8
* Updated CHANGES.md for #2229
* Updated CHANGES.md for #2229
* Update CHANGES.md
* Update CHANGES.md

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>

diff --git a/CHANGES.md b/CHANGES.md
index f795c797bf55469a0f177b19f72533495df3cb3a..34fa1a2092a89827d89296fa9a2dbd9eaa9c8719 100644 (file)
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -4,6 +4,7 @@
 
 ### _Black_
 
+- Fix handling of .gitignore files containing non-ASCII characters on Windows (#2229)
 - Respect `.gitignore` files in all levels, not only `root/.gitignore` file (apply
   `.gitignore` rules like `git` does) (#2225)
 - Restored compatibility with Click 8.0 on Python 3.6 when LANG=C used (#2227)

diff --git a/src/black/files.py b/src/black/files.py
index a0c92e8c41539b282c68117b81752562d757ed03..de5161566052738a866ff10647396e702e2dbec1 100644 (file)
--- a/src/black/files.py
+++ b/src/black/files.py
@@ -118,7 +118,7 @@ def get_gitignore(root: Path) -> PathSpec:
     gitignore = root / ".gitignore"
     lines: List[str] = []
     if gitignore.is_file():
-        with gitignore.open() as gf:
+        with gitignore.open(encoding="utf-8") as gf:
             lines = gf.readlines()
     return PathSpec.from_lines("gitwildmatch", lines)