Improve f-string expression detection regex so ... (#2437)

we don't accidentally add backslashes to them when normalizing quotes
because that's invalid syntax!

The problem this commit fixes is that matches would eat too much
blocking important matches to occur. For example, here's one f-string
body:

    {a}{b}{c}

I know there's no risk of introducing backslashes here, but the regex
already goes sideways with this. Throwing this example at regex101
I get:

    {a}{b}{c}   # The As and Bs are the two matches, and the upper
    ---- ----   # case letters are the groups with those matches.
    aAaa bbBb

... we've missed the middle expression (so if any backslashes in a
more complex example were introduced there we wouldn't bail out
even though we should -- hence the bug). As it stands the regex
needs somesort of extra character (or the start/end of the body)
around the expressions but that isn't always the case as shown
above.

The fix implemented here is to turn the "eat a surrounding non-curly
bracket character" groups ie. `(?:[^{]|^)` and `(?:[^}]|$)` into
negative lookaheads and lookbehinds. This still guarantees the
already specified rules but without problematically eating extra
characters ^^

diff --git a/CHANGES.md b/CHANGES.md
index 3a96029bf5cd3ccfcb481ad717a5aa8462839962..22ddc423e557e3f46031bd83a5cbb6b4bf2bf1d5 100644 (file)
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -7,6 +7,8 @@
 - Add support for formatting Jupyter Notebook files (#2357)
 - Move from `appdirs` dependency to `platformdirs` (#2375)
 - Present a more user-friendly error if .gitignore is invalid (#2414)
 - Add support for formatting Jupyter Notebook files (#2357)
 - Move from `appdirs` dependency to `platformdirs` (#2375)
 - Present a more user-friendly error if .gitignore is invalid (#2414)

+- The failsafe for accidentally added backslashes in f-string expressions has been
+  hardened to handle more edge cases during quote normalization (#2437)

 
 ### Integrations
 
 
 ### Integrations
 

diff --git a/src/black/strings.py b/src/black/strings.py
index 80f588f5119c4432c4a43ce8414b014893b4be2b..d7b6c240e80215ef1c268634433074d82e4c5aeb 100644 (file)
--- a/src/black/strings.py
+++ b/src/black/strings.py
@@ -190,9 +190,9 @@ def normalize_string_quotes(s: str) -> str:
     if "f" in prefix.casefold():
         matches = re.findall(
             r"""
     if "f" in prefix.casefold():
         matches = re.findall(
             r"""

-            (?:[^{]|^)\{  # start of the string or a non-{ followed by a single {
+            (?:(?<!\{)|^)\{  # start of the string or a non-{ followed by a single {

                 ([^{].*?)  # contents of the brackets except if begins with {{
                 ([^{].*?)  # contents of the brackets except if begins with {{

-            \}(?:[^}]|$)  # A } followed by end of the string or a non-}
+            \}(?:(?!\})|$)  # A } followed by end of the string or a non-}

             """,
             new_body,
             re.VERBOSE,
             """,
             new_body,
             re.VERBOSE,

diff --git a/tests/data/string_quotes.py b/tests/data/string_quotes.py
index 5a4bc5d0b119098d3097348d8131a49f84586e6a..3384241f4adaf7ad8e1dbff5be6da70e32a40558 100644 (file)
--- a/tests/data/string_quotes.py
+++ b/tests/data/string_quotes.py
@@ -51,6 +51,11 @@ f'{y * x} \'{z}\''
 '\'{z}\' {y * " "}'
 '{y * x} \'{z}\''
 
 '\'{z}\' {y * " "}'
 '{y * x} \'{z}\''
 

+# We must bail out if changing the quotes would introduce backslashes in f-string
+# expressions. xref: https://github.com/psf/black/issues/2348
+f"\"{b}\"{' ' * (long-len(b)+1)}: \"{sts}\",\n"
+f"\"{a}\"{'hello' * b}\"{c}\""
+

 # output
 
 """"""
 # output
 
 """"""


@@ -100,3 +105,8 @@ f'\'{z}\' {y * " "}'
 f"{y * x} '{z}'"
 "'{z}' {y * \" \"}"
 "{y * x} '{z}'"
 f"{y * x} '{z}'"
 "'{z}' {y * \" \"}"
 "{y * x} '{z}'"

+
+# We must bail out if changing the quotes would introduce backslashes in f-string
+# expressions. xref: https://github.com/psf/black/issues/2348
+f"\"{b}\"{' ' * (long-len(b)+1)}: \"{sts}\",\n"
+f"\"{a}\"{'hello' * b}\"{c}\""