Remove test suite from setup.py (#2824)

[etc/vim.git] / docs / contributing / gauging_changes.md

diff --git a/docs/contributing/gauging_changes.md b/docs/contributing/gauging_changes.md
index 6b70e0bc9bc15dab0c680bd13a0257995d0445cc..59c40eb390902abc71c6cbd8774451b1158e05c9 100644 (file)
--- a/docs/contributing/gauging_changes.md
+++ b/docs/contributing/gauging_changes.md
@@ -9,53 +9,52 @@ enough to cause frustration to projects that are already "black formatted".
 
 ## black-primer
 
 
 ## black-primer
 

-`black-primer` is a tool built for CI (and humans) to have _Black_ `--check` a number of
-Git accessible projects in parallel. (configured in `primer.json`) _(A PR will be
-accepted to add Mercurial support.)_
+`black-primer` is an obsolete tool (now replaced with `diff-shades`) that was used to
+gauge the impact of changes in _Black_ on open-source code. It is no longer used
+internally and will be removed from the _Black_ repository in the future.

 
 

-### Run flow
+## diff-shades

 
 

-- Ensure we have a `black` + `git` in PATH
-- Load projects from `primer.json`
-- Run projects in parallel with `--worker` workers (defaults to CPU count / 2)
-  - Checkout projects
-  - Run black and record result
-  - Clean up repository checkout _(can optionally be disabled via `--keep`)_
-- Display results summary to screen
-- Default to cleaning up `--work-dir` (which defaults to tempfile schemantics)
-- Return
-  - 0 for successful run
-  - \< 0 for environment / internal error
-  - \> 0 for each project with an error
+diff-shades is a tool that runs _Black_ across a list of Git cloneable OSS projects
+recording the results. The main highlight feature of diff-shades is being able to
+compare two revisions of _Black_. This is incredibly useful as it allows us to see what
+exact changes will occur, say merging a certain PR.

 
 

-### Speed up runs 🏎
+For more information, please see the [diff-shades documentation][diff-shades].

 
 

-If you're running locally yourself to test black on lots of code try:
+### CI integration

 
 

-- Using `-k` / `--keep` + `-w` / `--work-dir` so you don't have to re-checkout the repo
-  each run
+diff-shades is also the tool behind the "diff-shades results comparing ..." /
+"diff-shades reports zero changes ..." comments on PRs. The project has a GitHub Actions
+workflow which runs diff-shades twice against two revisions of _Black_ according to
+these rules:

 
 

-### CLI arguments
+|                       | Baseline revision       | Target revision              |
+| --------------------- | ----------------------- | ---------------------------- |
+| On PRs                | latest commit on `main` | PR commit with `main` merged |
+| On pushes (main only) | latest PyPI version     | the pushed commit            |

 
 

-```text
-Usage: black-primer [OPTIONS]
+Once finished, a PR comment will be posted embedding a summary of the changes and links
+to further information. If there's a pre-existing diff-shades comment, it'll be updated
+instead the next time the workflow is triggered on the same PR.

 
 

-  primer - prime projects for blackening... 🏴
+The workflow uploads 3-4 artifacts upon completion: the two generated analyses (they
+have the .json file extension), `diff.html`, and `.pr-comment.json` if triggered by a
+PR. The last one is downloaded by the `diff-shades-comment` workflow and shouldn't be
+downloaded locally. `diff.html` comes in handy for push-based or manually triggered
+runs. And the analyses exist just in case you want to do further analysis using the
+collected data locally.

 
 

-Options:
-  -c, --config PATH      JSON config file path  [default: /Users/cooper/repos/
-                         black/src/black_primer/primer.json]
+Note that the workflow will only fail intentionally if while analyzing a file failed to
+format. Otherwise a failure indicates a bug in the workflow.

 
 

-  --debug                Turn on debug logging  [default: False]
-  -k, --keep             Keep workdir + repos post run  [default: False]
-  -L, --long-checkouts   Pull big projects to test  [default: False]
-  -R, --rebase           Rebase project if already checked out  [default:
-                         False]
+```{tip}
+Maintainers with write access or higher can trigger the workflow manually from the
+Actions tab using the `workflow_dispatch` event. Simply select "diff-shades"
+from the workflows list on the left, press "Run workflow", and fill in which revisions
+and command line arguments to use.

 
 

-  -w, --workdir PATH     Directory path for repo checkouts  [default: /var/fol
-                         ders/tc/hbwxh76j1hn6gqjd2n2sjn4j9k1glp/T/primer.20200
-                         517125229]
-
-  -W, --workers INTEGER  Number of parallel worker coroutines  [default: 2]
-  -h, --help             Show this message and exit.
+Once finished, check the logs or download the artifacts for local use.

 ```
 ```

+
+[diff-shades]: https://github.com/ichard26/diff-shades#readme