Every one of the projects in this repository is available at the canonical
URL git://git.madduck.net/madduck/pub/<projectpath> — see
each project's metadata for the exact URL.
All patches and comments are welcome. Please squash your changes to logical
commits before using git-format-patch and git-send-email to
patches@git.madduck.net.
If you'd read over the Git project's submission guidelines and adhered to them,
I'd be especially grateful.
Add option to format Jupyter Notebooks in GitHub Action (#3282)
To run the formatter on Jupyter Notebooks, Black must be installed
with an extra dependency (`black[jupyter]`). This commit adds an
optional argument to install Black with this dependency when using the
official GitHub Action [1]. To enable the formatter on Jupyter
Notebooks, just add `jupyter: true` as an argument. Feature requested
at [2].
Ofek Lev [Sun, 25 Sep 2022 21:54:33 +0000 (14:54 -0700)]
Switch build backend to Hatchling (#3233)
This implements PEP 621, obviating the need for `setup.py`, `setup.cfg`,
and `MANIFEST.in`. The build backend Hatchling (of which I am a
maintainer in the PyPA) is now used as that is the default in the
official Python packaging tutorial. Hatchling is available on all the
major distribution channels such as Debian, Fedora, and many more.
## Python support
The earliest supported Python 3 version of Hatchling is 3.7, therefore
I've also set that as the minimum here. Python 3.6 is EOL and other
build backends like flit-core and setuptools also dropped support.
Python 3.6 accounted for 3-4% of downloads in the last month.
## Plugins
Configuration is now completely static with the help of 3 plugins:
### Readme
hynek's hatch-fancy-pypi-readme allows for the dynamic construction of
the readme which was previously coded up in `setup.py`. Now it's simply:
hatch-mypyc offers many benefits over the existing approach:
- No need to manually select files for inclusion
- Avoids the need for the current CI workaround for https://github.com/mypyc/mypyc/issues/946
- Intermediate artifacts (like `build/`) from setuptools and mypyc
itself no longer clutter the project directory
- Runtime dependencies required at build time no longer need to be
manually redeclared as this is a built-in option of Hatchling
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Fix a crash when `# fmt: on` is used on a different block level than `# fmt: off` (#3281)
Previously _Black_ produces invalid code because the `# fmt: on` is used on a different block level.
While _Black_ requires `# fmt: off` and `# fmt: on` to be used at the same block level, incorrect usage shouldn't cause crashes.
The formatting behavior this PR introduces is, the code below the initial `# fmt: off` block level will be turned off for formatting, when `# fmt: on` is used on a different level or there is no `# fmt: on`. This also matches the current behavior when `# fmt: off` is used at the top-level without a matching `# fmt: on`, it turns off formatting for everything below `# fmt: off`.
Tom Fryers [Thu, 15 Sep 2022 02:31:26 +0000 (03:31 +0100)]
Improve order of paragraphs on line splitting (#3270)
These two paragraphs were tucked away at the end of the section, after
the diversion on backslashes. I nearly missed the first paragraph and
opened a nonsense issue, and I think the second belongs higher up with
it too.
Fix a crash on dicts with paren-wrapped long string keys (#3262)
Fix a crash when formatting some dicts with parenthesis-wrapped long
string keys. When LL[0] is an atom string, we need to check the atom
node's siblings instead of LL[0] itself, e.g.:
dictsetmaker
atom
STRING '"This is a really long string that can\'t be expected to fit in one line and is used as a nested dict\'s key"'
/atom
COLON ':'
atom
LSQB ' ' '['
listmaker
STRING '"value"'
COMMA ','
STRING ' ' '"value"'
/listmaker
RSQB ']'
/atom
COMMA ','
/dictsetmaker
Cooper Lees [Mon, 5 Sep 2022 22:27:39 +0000 (08:27 +1000)]
Move 3.11 tests to install aiohttp without C extensions (#3258)
* Move 311 tests to install aiohttp without C extensions
- Configure tox to install aiohttp without extensions
- i.e. use `AIOHTTP_NO_EXTENSIONS=1` for pip install
- This allows us to reenable blackd tests that use aiohttp testing helpers etc.
- Had to ignore `cgi` module deprecation warning
- Filed issue for aiohttp to fix: https://github.com/aio-libs/aiohttp/issues/6905
Test:
- `/tmp/tb/bin/tox -e 311`
* Fix formatting + linting
* Add latest aiohttp for loop fix + Try to exempt deprecation warning but failed - will ask for help
* Remove unnecessary warning ignore
Co-authored-by: Cooper Ry Lees <me@wcooperlees.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Richard Si [Mon, 5 Sep 2022 20:27:05 +0000 (16:27 -0400)]
Mitigate deprecation of aiohttp's `@middleware` decorator (#3259)
This is deprecated since aiohttp 4.0. If it doesn't exist just define a
no-op decorator that does nothing (after the other aiohttp imports
though!). By doing this, it's safe to ignore the DeprecationWarning
without needing to require the latest aiohttp once they remove
`@middleware`.
Richard Si [Wed, 31 Aug 2022 21:56:47 +0000 (17:56 -0400)]
Update stable branch after publishing to PyPI (#3223)
We've decided to a) convert stable back into a branch and b) to update
it immediately as part of the release process. We may as well automate
it. And about going back to a branch ...
Git tags are not the right tool, at all[^1]. They come with the
expectation that they will never change. Things will not work as
expected if they do change, doubly so if they change regularly. Once
you pull stable from the remote and it's copied in your local
repository, no matter how many times you run git pull you'll never see
it get updated automatically. Your only recourse is to delete the tag
via `git tag -d stable` before pulling.
This gets annoying really quickly since stable is supposed to be the
solution for folks "who want to move along as Black developers deem
the newest version reliable."[^2] See this comment for how this impacts
users using our Vim plugin[^3]. It also affects us developers[^4]. If
you have stable locally, once we cut a new release and update the stable
tag, a simple `git pull` / `git fetch` will not pull down the updated
stable tag. Unless you remember to delete stable before pulling, stable
will become stale and useless.
You can argue this is a good thing ("people should explicitly opt into
updating stable"), but IMO it does not match user expectations nor
developer expectations[^5]. Especially since not all our integrations
that use stable are bound by this security measure, for example our
GitHub Action (since it does a clean fetch of the repository every time
it's used). I believe consistency would be good here.
Finally, ever since we switched to a tag, we've been facing issues with
ReadTheDocs not picking up updates to stable unless we force a rebuild.
The initial rebuild on the stable update just pulls the commit the tag
previously pointed to. I'm not sure if switching back to a branch will
fix this, but I'd wager it will.
[^4]: In fairness, most folks working on Black probably don't use the
`stable` ref anyway, especially us maintainers who'd know what is
the latest version by heart, but it'd still be nice to make it
usable for local dev though.
[^5]: Also what benefit does a `stable` ref have over explicit version
tags like `22.6.0`? If you're going to opt into some odd pin
mechanism, might as well use explicit version tags for clarity
and consistency.
Richard Si [Wed, 31 Aug 2022 21:46:48 +0000 (17:46 -0400)]
Improve & update release process to reflect recent changes (#3242)
- Formalise release cadence guidelines
- Overhaul release steps to be easier to follow and more thorough
- Reorder changelog template to something more sensible
- Update release automation docs to reflect recent improvements (notably
the addition of in-repo mypyc wheel builds)
Co-authored-by: Felix Hildén <felix.hilden@gmail.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Add parens around implicit string concatenations where it increases readability (#3162)
Adds parentheses around implicit string concatenations when it's inside
a list, set, or tuple. Except when it's only element and there's no trailing
comma.
Looking at the order of the transformers here, we need to "wrap in
parens" before string_split runs. So my solution is to introduce a
"collaboration" between StringSplitter and StringParenWrapper where the
splitter "skips" the split until the wrapper adds the parens (and then
the line after the paren is split by StringSplitter) in another pass.
I have also considered an alternative approach, where I tried to add a
different "string paren wrapper" class, and it runs before string_split.
Then I found out it requires a different do_transform implementation
than StringParenWrapper.do_transform, since the later assumes it runs
after the delimiter_split transform. So I stopped researching that
route.
Originally function calls were also included in this change, but given
missing commas should usually result in a runtime error and the scary
amount of changes this cause on downstream code, they were removed in
later revisions.
Richard Si [Sat, 13 Aug 2022 17:46:52 +0000 (13:46 -0400)]
Delay worker count determination
os.cpu_count() can return None (sounds like a super arcane edge case
though) so the type annotation for the `workers` parameter of
`black.main` is wrong. This *could* technically cause a runtime
TypeError since it'd trip one of mypyc's runtime type checks so we
might as well fix it.
Reading the documentation (and cross-checking with the source code),
you are actually allowed to pass None as `max_workers` to
`concurrent.futures.ProcessPoolExecutor`. If it is None, the pool
initializer will simply call os.cpu_count() [^1] (defaulting to 1 if it
returns None [^2]). It'll even round down the worker count to a level
that's safe for Windows.
... so theoretically we don't even need to call os.cpu_count()
ourselves, but our Windows limit is 60 (unlike the stdlib's 61) and I'd
prefer not accidentally reintroducing a crash on machines with many,
many CPU cores.
Richard Si [Fri, 5 Aug 2022 18:04:43 +0000 (14:04 -0400)]
Load .gitignore and exclude regex at time of use
Loading .gitignore and compiling the exclude regex can take more than
15ms. We shouldn't and don't need to pay this cost if we're simply
formatting files given on the command line directly.
I would've loved to lazily import pathspec, but the patch won't be clean
until the file collection and discovery logic is refactored first.
Richard Si [Thu, 4 Aug 2022 00:18:33 +0000 (20:18 -0400)]
Lazily import parallelized format modules
`black.reformat_many` depends on a lot of slow-to-import modules. When
formatting simply a single file, the time paid to import those modules
is totally wasted. So I moved `black.reformat_many` and its helpers
to `black.concurrency` which is now *only* imported if there's more
than one file to reformat. This way, running Black over a single file
is snappier
Here are the numbers before and after this patch running `python -m
black --version`:
- interpreted: 411 ms +- 9 ms -> 342 ms +- 7 ms: 1.20x faster
- compiled: 365 ms +- 15 ms -> 304 ms +- 7 ms: 1.20x faster
Shantanu [Fri, 26 Aug 2022 21:07:25 +0000 (14:07 -0700)]
Fix misdetection of project root with `--stdin-filename` (#3216)
There are a number of places this behaviour could be patched, for
instance, it's quite tempting to patch it in `get_sources`. However
I believe we generally have the invariant that project root contains all
files we want to format, in which case it seems prudent to keep that
invariant.
This also improves the accuracy of the "sources to be formatted" log
message with --stdin-filename.
Ionite [Fri, 26 Aug 2022 19:45:31 +0000 (15:45 -0400)]
Remove hacky subprocess call in action.yml (#3226)
Updates action.yml to use the alternative $GITHUB_ACTION_PATH variable
instead of the original ${{ github.action_path }} which caused issues
with bash on the Windows runners. This removes the need for a Python
subprocess to call the main.py script.
Cooper Lees [Tue, 23 Aug 2022 03:39:48 +0000 (20:39 -0700)]
Add passing 3.11 CI by exempting blackd tests (#3234)
- Had to exempt blackd tests for now due to aiohttp
- Skip by using `sys.version_info` tuple
- aiohttp does not compile in 3.11 yet - refer to #3230
- Add a deadsnakes ubuntu workflow to run 3.11-dev to ensure we don't regress
- Have it also format ourselves
Test:
- `tox -e 311`
Co-authored-by: Cooper Ry Lees <me@wcooperlees.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Tom Fryers [Tue, 2 Aug 2022 21:22:04 +0000 (22:22 +0100)]
Remove invalid syntax in docstrings -S --preview test (#3205)
uR is not a legal string prefix, so this test breaks (AssertionError:
cannot use --safe with this file; failed to parse source file AST:
invalid syntax) if changed to one in which the file is changed. I've
changed the last test to have u alone, and added an R to the test above
instead.
Nicolò Intrieri [Tue, 2 Aug 2022 16:01:15 +0000 (18:01 +0200)]
makes install available for all users in docker image (#3202)
* makes install available for all users in docker image
moves the installation path from /root/.local to a
virtualenv. this way we still get the lightweight
multistage build without excluding non-root users.
* adds changelog entry for docker-image fix
A changelog entry has been added under the Integration
subheader
* changes dockerfile to use the venv activate script
we are now using the inbuilt venv activate script, as well
as explicitly mentioning the binary location in the entrypoint
cmd.
Co-authored-by: Nicolò <nicolo.intrieri@spinforward.it> Co-authored-by: Cooper Lees <me@cooperlees.com>
Richard Si [Sat, 30 Jul 2022 03:28:43 +0000 (23:28 -0400)]
Remove blib2to3 grammar cache logging (#3193)
As error logs are emitted often (they happen when Black's cache
directory is created after blib2to3 tries to write its cache) and cause
issues to be filed by users who think Black isn't working correctly.
These errors are expected for now and aren't a cause for concern so
let's remove them to stop worrying users (and new issues from being
opened). We can improve the blib2to3 caching mechanism to write its
cache at the end of a successful command line invocation later.
Vim plugin: prefix messages with "Black: " (#3194)
As mentioned in GH-3185, when using Black as a Vim plugin, especially
automatically on save, the plugin's messages can be confusing, as
nothing indicates that they come from Black.
The former was a regression I introduced a long time ago. To avoid
changing the stable style too much, the regression is only fixed if
--preview is enabled
Annoyingly enough, as we currently always enforce a second format pass if
changes were made, there's no good way to prove the existence of the
docstring quote normalization instability issue. For posterity, here's
one failing example:
Cooper Lees [Thu, 14 Jul 2022 22:24:34 +0000 (15:24 -0700)]
Move to explicitly creating a new loop (#3164)
* Move to explicitly creating a new loop
- >= 3.10 add a warning that `get_event_loop` will not automatically create a loop
- Move to explicit API
Test:
- `python3.11 -m venv --upgrade-deps /tmp/tb`
- `/tmp/tb/bin/pip install -e .`
- Install deps and no blackd as aiohttp + yarl can't build still with 3.11
- https://github.com/aio-libs/aiohttp/issues/6600
- `export PYTHONWARNINGS=error`
```
cooper@l33t:~/repos/black$ /tmp/tb/bin/black .
All done! ✨ 🍰 ✨
44 files left unchanged.
```
Fixes #3110
* Add to CHANGES.md
* Fix a cooper typo yet again
* Set default asyncio loop to our explicitly created one + unset on exit
* Update CHANGES.md
Fix my silly typo.
Co-authored-by: Thomas Grainger <tagrain@gmail.com> Co-authored-by: Cooper Ry Lees <me@wcooperlees.com> Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Richard Si [Thu, 14 Jul 2022 02:26:05 +0000 (22:26 -0400)]
Don't (ever) put a single-char closing docstring quote on a new line (#3166)
Doing so is invalid. Note this only fixes the preview style since the
logic putting closing docstring quotes on their own line if they violate
the line length limit is quite new.
Richard Si [Thu, 14 Jul 2022 00:02:51 +0000 (20:02 -0400)]
Copy over comments when hugging power ops (#2874)
Otherwise they'd be deleted which was a regression in 22.1.0 (oops! my
bad!). Also type comments are now tracked in the AST safety check on all
compatible platforms to error out if this happens again.
Overall the line rewriting code has been rewritten to do "the right
thing (tm)", I hope this fixes other potential bugs in the code (fwiw I
got to drop the bugfix in blib2to3.pytree.Leaf.clone since now bracket
metadata is properly copied over).
Naveen [Tue, 3 May 2022 13:08:33 +0000 (08:08 -0500)]
chore: Set permissions for GitHub actions (#3043)
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions
[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
projects / etc / vim.git / log