From: Richard Si <63936253+ichard26@users.noreply.github.com> Date: Wed, 31 Aug 2022 21:46:48 +0000 (-0400) Subject: Improve & update release process to reflect recent changes (#3242) X-Git-Url: https://git.madduck.net/etc/vim.git/commitdiff_plain/7757078ecd84d349bb24ab61e79062ba50162ef9 Improve & update release process to reflect recent changes (#3242) - Formalise release cadence guidelines - Overhaul release steps to be easier to follow and more thorough - Reorder changelog template to something more sensible - Update release automation docs to reflect recent improvements (notably the addition of in-repo mypyc wheel builds) Co-authored-by: Felix Hildén Co-authored-by: Jelle Zijlstra --- diff --git a/docs/contributing/release_process.md b/docs/contributing/release_process.md index 6a4b868..be9b08a 100644 --- a/docs/contributing/release_process.md +++ b/docs/contributing/release_process.md @@ -1,40 +1,85 @@ # Release process -_Black_ has had a lot of work automating its release process. This document sets out to -explain what everything does and how to release _Black_ using said automation. - -## Cutting a Release - -To cut a release, you must be a _Black_ maintainer with `GitHub Release` creation -access. Using this access, the release process is: - -1. Cut a new PR editing `CHANGES.md` and the docs to version the latest changes +_Black_ has had a lot of work done into standardizing and automating its release +process. This document sets out to explain how everything works and how to release +_Black_ using said automation. + +## Release cadence + +**We aim to release whatever is on `main` every 1-2 months.** This ensures merged +improvements and bugfixes are shipped to users reasonably quickly, while not massively +fracturing the user-base with too many versions. This also keeps the workload on +maintainers consistent and predictable. + +If there's not much new on `main` to justify a release, it's acceptable to skip a +month's release. Ideally January releases should not be skipped because as per our +[stability policy](labels/stability-policy), the first release in a new calendar year +may make changes to the _stable_ style. While the policy applies to the first release +(instead of only January releases), confining changes to the stable style to January +will keep things predictable (and nicer) for users. + +Unless there is a serious regression or bug that requires immediate patching, **there +should not be more than one release per month**. While version numbers are cheap, +releases require a maintainer to both commit to do the actual cutting of a release, but +also to be able to deal with the potential fallout post-release. Releasing more +frequently than monthly nets rapidly diminishing returns. + +## Cutting a release + +**You must have `write` permissions for the _Black_ repository to cut a release.** + +The 10,000 foot view of the release process is that you prepare a release PR and then +publish a [GitHub Release]. This triggers [release automation](#release-workflows) that +builds all release artifacts and publishes them to the various platforms we publish to. + +To cut a release: + +1. Determine the release's version number + - **_Black_ follows the [CalVer] versioning standard using the `YY.M.N` format** + - So unless there already has been a release during this month, `N` should be `0` + - Example: the first release in January, 2022 → `22.1.0` +1. File a PR editing `CHANGES.md` and the docs to version the latest changes + 1. Replace the `## Unreleased` header with the version number 1. Remove any empty sections for the current release - 2. Add a new empty template for the next release (template below) - 3. Example PR: [#2616](https://github.com/psf/black/pull/2616) - 4. Example title: `Update CHANGES.md for XX.X release` -2. Once the release PR is merged ensure all CI passes - 1. If not, ensure there is an Issue open for the cause of failing CI (generally we'd - want this fixed before cutting a release) -3. Open `CHANGES.md` and copy the _raw markdown_ of the latest changes to use in the - description of the GitHub Release. -4. Go and [cut a release](https://github.com/psf/black/releases) using the GitHub UI so - that all workflows noted below are triggered. - 1. The release version and tag should be the [CalVer](https://calver.org) version - _Black_ used for the current release e.g. `21.6` / `21.5b1` - 2. _Black_ uses [setuptools scm](https://pypi.org/project/setuptools-scm/) to pull - the current version for the package builds and release. -5. Once the release is cut, you're basically done. It's a good practice to go and watch - to make sure all the [GitHub Actions](https://github.com/psf/black/actions) pass, - although you should receive an email to your registered GitHub email address should - one fail. - 1. You should see all the release workflows and lint/unittests workflows running on - the new tag in the Actions UI - -If anything fails, please go read the respective action's log output and configuration -file to reverse engineer your way to a fix/soluton. - -## Changelog template + 1. (_optional_) Read through and copy-edit the changelog (eg. by moving entries, + fixing typos, or rephrasing entries) + 1. Add a new empty template for the next release above + ([template below](#changelog-template)) + 1. Update references to the latest version in + {doc}`/integrations/source_version_control` and + {doc}`/usage_and_configuration/the_basics` + - Example PR: [GH-3139] +1. Once the release PR is merged, wait until all CI passes + - If CI does not pass, **stop** and investigate the failure(s) as generally we'd want + to fix failing CI before cutting a release +1. [Draft a new GitHub Release][new-release] + 1. Click `Choose a tag` and type in the version number, then select the + `Create new tag: YY.M.N on publish` option that appears + 1. Verify that the new tag targets the `main` branch + 1. You can leave the release title blank, GitHub will default to the tag name + 1. Copy and paste the _raw changelog Markdown_ for the current release into the + description box +1. Publish the GitHub Release, triggering [release automation](#release-workflows) that + will handle the rest +1. At this point, you're basically done. It's good practice to go and [watch and verify + that all the release workflows pass][black-actions], although you will receive a + GitHub notification should something fail. + - If something fails, don't panic. Please go read the respective workflow's logs and + configuration file to reverse-engineer your way to a fix/solution. + +Congratulations! You've successfully cut a new release of _Black_. Go and stand up and +take a break, you deserve it. + +```{important} +Once the release artifacts reach PyPI, you may see new issues being filed indicating +regressions. While regressions are not great, they don't automatically mean a hotfix +release is warranted. Unless the regressions are serious and impact many users, a hotfix +release is probably unnecessary. + +In the end, use your best judgement and ask other maintainers for their thoughts. +``` + +### Changelog template Use the following template for a clean changelog after the release: @@ -45,7 +90,7 @@ Use the following template for a clean changelog after the release: -### Style +### Stable style @@ -53,93 +98,115 @@ Use the following template for a clean changelog after the release: -### _Blackd_ - - - ### Configuration -### Documentation +### Packaging - + -### Integrations +### Parser - + + +### Performance + + ### Output -### Packaging - - +### _Blackd_ -### Parser + - +### Integrations -### Performance + - +### Documentation + ``` ## Release workflows -All _Blacks_'s automation workflows use GitHub Actions. All workflows are therefore -configured using `.yml` files in the `.github/workflows` directory of the _Black_ +All of _Black_'s release automation uses [GitHub Actions]. All workflows are therefore +configured using YAML files in the `.github/workflows` directory of the _Black_ repository. +They are triggered by the publication of a [GitHub Release]. + Below are descriptions of our release workflows. -### Docker +### Publish to PyPI + +This is our main workflow. It builds an [sdist] and [wheels] to upload to PyPI where the +vast majority of users will download Black from. It's divided into three job groups: + +#### sdist + pure wheel -This workflow uses the QEMU powered `buildx` feature of docker to upload a `arm64` and -`amd64`/`x86_64` build of the official _Black_ docker image™. +This single job builds the sdist and pure Python wheel (i.e., a wheel that only contains +Python code) using [build] and then uploads them to PyPI using [twine]. These artifacts +are general-purpose and can be used on basically any platform supported by Python. -- Currently this workflow uses an API Token associated with @cooperlees account +#### mypyc wheels (…) -### pypi_upload +We use [mypyc] to compile _Black_ into a CPython C extension for significantly improved +performance. Wheels built with mypyc are platform and Python version specific. +[Supported platforms are documented in the FAQ](labels/mypyc-support). -This workflow builds a Python -[sdist](https://docs.python.org/3/distutils/sourcedist.html) and -[wheel](https://pythonwheels.com) using the latest -[setuptools](https://pypi.org/project/setuptools/) and -[wheel](https://pypi.org/project/wheel/) modules. +These matrix jobs use [cibuildwheel] which handles the complicated task of building C +extensions for many environments for us. Since building these wheels is slow, there are +multiple mypyc wheels jobs (hence the term "matrix") that build for a specific platform +(as noted in the job name in parentheses). -It will then use [twine](https://pypi.org/project/twine/) to upload both release formats -to PyPI for general downloading of the _Black_ Python package. This is where -[pip](https://pypi.org/project/pip/) looks by default. +Like the previous job group, the built wheels are uploaded to PyPI using [twine]. -- Currently this workflow uses an API token associated with @ambv's PyPI account +#### Update stable branch -### Upload self-contained binaries +So this job doesn't _really_ belong here, but updating the `stable` branch after the +other PyPI jobs pass (they must pass for this job to start) makes the most sense. This +saves us from remembering to update the branch sometime after cutting the release. -This workflow builds self-contained binaries for multiple platforms. This allows people -to download the executable for their platform and run _Black_ without a -[Python Runtime](https://wiki.python.org/moin/PythonImplementations) installed. +- _Currently this workflow uses an API token associated with @ambv's PyPI account_ -The created binaries are attached/stored on the associated -[GitHub Release](https://github.com/psf/black/releases) for download over _IPv4 only_ -(GitHub still does not have IPv6 access 😢). +### Publish executables -## Moving the `stable` tag +This workflow builds native executables for multiple platforms using [PyInstaller]. This +allows people to download the executable for their platform and run _Black_ without a +[Python runtime](https://wiki.python.org/moin/PythonImplementations) installed. -_Black_ provides a stable tag for people who want to move along as _Black_ developers -deem the newest version reliable. Here the _Black_ developers will move once the release -has been problem free for at least ~24 hours from release. Given the large _Black_ -userbase we hear about bad bugs quickly. We do strive to continually improve our CI too. +The created binaries are stored on the associated GitHub Release for download over _IPv4 +only_ (GitHub still does not have IPv6 access 😢). -### Tag moving process +### docker -#### stable +This workflow uses the QEMU powered `buildx` feature of Docker to upload an `arm64` and +`amd64`/`x86_64` build of the official _Black_ Docker image™. -From a rebased `main` checkout: +- _Currently this workflow uses an API Token associated with @cooperlees account_ + +```{note} +This also runs on each push to `main`. +``` -1. `git tag -f stable VERSION_TAG` - 1. e.g. `git tag -f stable 21.5b1` -1. `git push --tags -f` +[black-actions]: https://github.com/psf/black/actions +[build]: https://pypa-build.readthedocs.io/ +[calver]: https://calver.org +[cibuildwheel]: https://cibuildwheel.readthedocs.io/ +[gh-3139]: https://github.com/psf/black/pull/3139 +[github actions]: https://github.com/features/actions +[github release]: https://github.com/psf/black/releases +[new-release]: https://github.com/psf/black/releases/new +[mypyc]: https://mypyc.readthedocs.io/ +[mypyc-platform-support]: + /faq.html#what-is-compiled-yes-no-all-about-in-the-version-output +[pyinstaller]: https://www.pyinstaller.org/ +[sdist]: + https://packaging.python.org/en/latest/glossary/#term-Source-Distribution-or-sdist +[twine]: https://github.com/features/actions +[wheels]: https://packaging.python.org/en/latest/glossary/#term-Wheel diff --git a/docs/faq.md b/docs/faq.md index b2fe42d..aeb9634 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -114,6 +114,8 @@ errors is not a goal. It can format all code accepted by CPython (if you find an where that doesn't hold, please report a bug!), but it may also format some code that CPython doesn't accept. +(labels/mypyc-support)= + ## What is `compiled: yes/no` all about in the version output? While _Black_ is indeed a pure Python project, we use [mypyc] to compile _Black_ into a diff --git a/docs/the_black_code_style/index.md b/docs/the_black_code_style/index.md index c7f29af..e5967be 100644 --- a/docs/the_black_code_style/index.md +++ b/docs/the_black_code_style/index.md @@ -19,6 +19,8 @@ style aspects and details might change according to the stability policy present below. Ongoing style considerations are tracked on GitHub with the [design](https://github.com/psf/black/labels/T%3A%20design) issue label. +(labels/stability-policy)= + ## Stability Policy The following policy applies for the _Black_ code style, in non pre-release versions of