Match escaped square brackets in paths literally not as regexes

[ianthomas23]

This is an experiment looking at issue #838 which fails to find file with names containing square brackets. As discussed in that issue square brackets are usually interpreted as regexes and we'd rather not add an extra kwarg throughout the API to specify whether regex or literal interpretation should apply.

So one option is to escape the square brackets using backslashes. That doesn't work on the master branch, but after this one-line addition it does work. Test script:

import fsspec

fs = fsspec.filesystem("file")

# Create test files
fs.makedirs("issue838")
fs.touch("issue838/file[a].txt")
fs.touch("issue838/other.txt")

print("ls", fs.ls("issue838/"))
print("find", fs.find("issue838/"))

print("expand wildcard", fs.expand_path("issue838/file*"))
print("expand regex", fs.expand_path("issue838/[fo]*"))

# Remaining examples all fail on master branch.

# The following 3 are equivalent
print("expand literal", fs.expand_path("issue838/*\[*"))
print("expand literal", fs.expand_path("issue838/*\\[*"))
print("expand literal", fs.expand_path(r"issue838/*\[*"))

print("expand literal", fs.expand_path("issue838/*\]*"))

which using this PR gives

ls ['/tmp/issue838/file[a].txt', '/tmp/issue838/other.txt']
find ['/tmp/issue838/file[a].txt', '/tmp/issue838/other.txt']
expand wildcard ['/tmp/issue838/file[a].txt']
expand regex ['/tmp/issue838/file[a].txt', '/tmp/fsspec_temp/issue838/other.txt']
expand literal ['/tmp/issue838/file[a].txt']
expand literal ['/tmp/issue838/file[a].txt']
expand literal ['/tmp/issue838/file[a].txt']
expand literal ['/tmp/issue838/file[a].txt']

This isn't complete, it doesn't deal with anything other than square brackets and it causes some existing tests to fail.

Alternative implementations are possible. There are quite a lot of lines of string and regex processing around the change, and these could no doubt be modified to include the new behaviour but I would be wary of doing so and causing other failures elsewhere.

There is a certain irony that the one line change to force the behaviour to not be a regex is itself a regex in which the important characters have to be escaped to be interpreted correctly!

[ianthomas23]

I am closing this as I don't think it is a good solution.