Glob (programming)
Template:Short description Template:Lowercase Template:Use dmy dates
glob – the owner is dmr, short for Dennis Ritchie.glob() (Template:IPAc-en) is a libc function for globbing, which is the archetypal use of pattern matching against the names in a filesystem directory such that a name pattern is expanded into a list of names matching that pattern. Although globbing may now refer to glob()-style pattern matching of any string, not just expansion into a list of filesystem names, the original meaning of the term is still widespread.
The glob() function and the underlying gmatch() function originated at Bell Labs in the early 1970s alongside the original AT&T UNIX itself and had a formative influence on the syntax of UNIX command line utilities and therefore also on the present-day reimplementations thereof.
In their original form, glob() and gmatch() derived from code used in Bell Labs in-house utilities that developed alongside the original Unix in the early 1970s. Among those utilities were also two command line tools called glob and find; each could be used to pass a list of matching filenames to other command line tools, and they shared the backend code subsequently formalized as glob() and gmatch(). Shell-statement-level globbing by default became commonplace following the "builtin"-integration of globbing-functionality into the 7th edition of the Unix shell in 1978. The Unix shell's -f option to disable globbing — i.e. revert to literal "file" mode — appeared in the same version.
The glob pattern quantifiers now standardized by POSIX.2 (IEEE Std 1003.2) fall into two groups, and can be applied to any character sequence ("string"), not just to directory entries.
- "Metacharacters" (also called "Wildcards"):
?(not in brackets) matches any character exactly once.*(not in brackets) matches a string of zero or more characters.
- "Ranges/sets":
[...], where the first character within the brackets is not '!', matches any single character among the characters specified in the brackets. If the first character within brackets is '!', then the[!...]matches any single character that is not among the characters specified in the brackets.
- The characters in the brackets may be a list (
[abc]) or a range ([a-c]) or denote a character class (like[[:space:]]where the inner brackets are part of the classname). POSIX does not mandate multi-range ([a-c0-3]) support, which derive originally from regular expressions.
- The characters in the brackets may be a list (
As reimplementations of Bell Labs' UNIX proliferated, so did reimplementations of its Bell Labs' libc and shell, and with them glob() and globbing. Today, glob() and globbing are standardized by the POSIX.2 specification and are integral part of every Unix-like libc ecosystem and shell, including AT&T Bourne shell-compatible Korn shell (ksh), Z shell (zsh), Almquist shell (ash) and its derivatives and reimplementations such as busybox, toybox, GNU bash, Debian dash.
OriginEdit
The glob command, short for global, originates in the earliest versions of Bell Labs' Unix.<ref name="man7Unix1"/> The command interpreters of the early versions of Unix (1st through 6th Editions, 1969–1975) relied on a separate program to expand wildcard characters in unquoted arguments to a command: /etc/glob. That program performed the expansion and supplied the expanded list of file paths to the command for execution.
Glob was originally written in the B programming language. It was the first piece of mainline Unix software to be developed in a high-level programming language.<ref name="reader">Template:Cite tech report</ref> Later, this functionality was provided as a C library function, glob(), used by programs such as the shell. It is usually defined based on a function named fnmatch(), which tests for whether a string matches a given pattern - the program using this function can then iterate through a series of strings (usually filenames) to determine which ones match. Both functions are a part of POSIX: the functions defined in POSIX.1 since 2001, and the syntax defined in POSIX.2.<ref name=fnmatch3>Template:Man</ref><ref>Template:Man</ref> The idea of defining a separate match function started with wildmat (wildcard match), a simple library to match strings against Bourne Shell globs.
Traditionally, globs do not match hidden files in the form of Unix dotfiles; to match them the pattern must explicitly start with .. For example, * matches all visible files while .* matches all hidden files.
SyntaxEdit
The most common wildcards are <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight>, and <syntaxhighlight lang="text" class="" style="" inline="1">[…]</syntaxhighlight>.
| Wildcard | Description | Example | Matches | Does not match |
|---|---|---|---|---|
| <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight> | matches any number of any characters including none | <syntaxhighlight lang="text" class="" style="" inline="1">Law*</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Law</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Laws</syntaxhighlight>, or <syntaxhighlight lang="text" class="" style="" inline="1">Lawyer</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">GrokLaw</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">La</syntaxhighlight>, or <syntaxhighlight lang="text" class="" style="" inline="1">aw</syntaxhighlight> |
| <syntaxhighlight lang="text" class="" style="" inline="1">*Law*</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Law</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">GrokLaw</syntaxhighlight>, or <syntaxhighlight lang="text" class="" style="" inline="1">Lawyer</syntaxhighlight>. | <syntaxhighlight lang="text" class="" style="" inline="1">La</syntaxhighlight>, or <syntaxhighlight lang="text" class="" style="" inline="1">aw</syntaxhighlight> | ||
| <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight> | matches any single character | <syntaxhighlight lang="text" class="" style="" inline="1">?at</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Cat</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">cat</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Bat</syntaxhighlight> or <syntaxhighlight lang="text" class="" style="" inline="1">bat</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">at</syntaxhighlight> |
| <syntaxhighlight lang="text" class="" style="" inline="1">[abc]</syntaxhighlight> | matches one character given in the bracket | <syntaxhighlight lang="text" class="" style="" inline="1">[CB]at</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Cat</syntaxhighlight> or <syntaxhighlight lang="text" class="" style="" inline="1">Bat</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">cat</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">bat</syntaxhighlight> or <syntaxhighlight lang="text" class="" style="" inline="1">CBat</syntaxhighlight> |
| <syntaxhighlight lang="text" class="" style="" inline="1">[a-z]</syntaxhighlight> | matches one character from the (locale-dependent) range given in the bracket | <syntaxhighlight lang="text" class="" style="" inline="1">Letter[0-9]</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Letter0</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter1</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter2</syntaxhighlight> up to <syntaxhighlight lang="text" class="" style="" inline="1">Letter9</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Letters</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter</syntaxhighlight> or <syntaxhighlight lang="text" class="" style="" inline="1">Letter10</syntaxhighlight> |
Normally, the path separator character (<syntaxhighlight lang="text" class="" style="" inline="1">/</syntaxhighlight> on Linux/Unix, MacOS, etc. or <syntaxhighlight lang="text" class="" style="" inline="1">\</syntaxhighlight> on Windows) will never be matched. Some shells, such as Unix shell have functionality allowing users to circumvent this.<ref>https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching Template:Webarchive Bash Reference Manual</ref>
Unix-likeTemplate:AnchorEdit
On Unix-like systems <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight> is defined as above while <syntaxhighlight lang="text" class="" style="" inline="1">[…]</syntaxhighlight> has two additional meanings:<ref name="posixglob"/><ref name="linuxglob7"/>
| Wildcard | Description | Example | Matches | Does not match |
|---|---|---|---|---|
| <syntaxhighlight lang="text" class="" style="" inline="1">[!abc]</syntaxhighlight> | matches one character that is not given in the bracket | <syntaxhighlight lang="text" class="" style="" inline="1">[!C]at</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Bat</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">bat</syntaxhighlight>, or <syntaxhighlight lang="text" class="" style="" inline="1">cat</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Cat</syntaxhighlight> |
| <syntaxhighlight lang="text" class="" style="" inline="1">[!a-z]</syntaxhighlight> | matches one character that is not from the range given in the bracket | <syntaxhighlight lang="text" class="" style="" inline="1">Letter[!3-5]</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">Letter1</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter2</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter6</syntaxhighlight> up to <syntaxhighlight lang="text" class="" style="" inline="1">Letter9</syntaxhighlight> and <syntaxhighlight lang="text" class="" style="" inline="1">Letterx</syntaxhighlight> etc. | <syntaxhighlight lang="text" class="" style="" inline="1">Letter3</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter4</syntaxhighlight>, <syntaxhighlight lang="text" class="" style="" inline="1">Letter5</syntaxhighlight> or <syntaxhighlight lang="text" class="" style="" inline="1">Letterxx</syntaxhighlight> |
The ranges are also allowed to include pre-defined character classes, equivalence classes for accented characters, and collation symbols for hard-to-type characters. They are defined to match up with the brackets in POSIX regular expressions.<ref name="posixglob"/><ref name="linuxglob7"/>
Unix globbing is handled by the shell per POSIX tradition. Globbing is provided on filenames at the command line and in shell scripts.<ref name="ABSGlob"/> The POSIX-mandated case statement in shells provides pattern-matching using glob patterns.
Some shells (such as the C shell and Bash) support additional syntax known as alternation or brace expansion. Because it is not part of the glob syntax, it is not provided in case. It is only expanded on the command line before globbing.
The Bash shell also supports the following extensions:<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref>
- Extended globbing (extglob): allows other pattern matching operators to be used to match multiple occurrences of a pattern enclosed in parentheses, essentially providing the missing kleene star and alternation for describing regular languages. It can be enabled by setting the <syntaxhighlight lang="text" class="" style="" inline="1">extglob</syntaxhighlight> shell option. This option came from ksh93.<ref name="bashpat"/> The GNU fnmatch and glob has an identical extension.<ref name=fnmatch3/>
- globstar: allows
**on its own as a name component to recursively match any number of layers of non-hidden directories.<ref name="bashpat"/> Also supported by the JavaScript libraries and Python's glob.
Windows and DOSEdit
The original DOS was a clone of CP/M designed to work on Intel's 8088 and 8086 processors. Windows shells, following DOS, do not traditionally perform any glob expansion in arguments passed to external programs. Shells may use an expansion for their own builtin commands:
- Windows PowerShell has all the common syntax defined as stated above without any additions.<ref name="pwshcmdlet"/>
- COMMAND.COM and cmd.exe have most of the common syntax with some limitations: There is no <syntaxhighlight lang="text" class="" style="" inline="1">[…]</syntaxhighlight> and for COMMAND.COM the <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight> may only appear at the end of the pattern. It can not appear in the middle of a pattern, except immediately preceding the filename extension separator dot.
Windows and DOS programs receive a long command-line string instead of argv-style parameters, and it is their responsibility to perform any splitting, quoting, or glob expansion. There is technically no fixed way of describing wildcards in programs since they are free to do what they wish. Two common glob expanders include:<ref name="winWildcard"/>
- The Microsoft C Runtime (msvcrt) command-line expander, which only supports <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight> and <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight>.<ref>{{#invoke:citation/CS1|citation
|CitationClass=web }}</ref> Both ReactOS (crt/misc/getargs.c) and Wine (msvcrt/data.c) contain a compatible open-source implementation of <syntaxhighlight lang="text" class="" style="" inline="1">__getmainargs</syntaxhighlight>, the function operating under-the-hood, in their core CRT.
- The Cygwin and MSYS <syntaxhighlight lang="text" class="" style="" inline="1">dcrt0.cc</syntaxhighlight> command-line expander, which uses the unix-style <syntaxhighlight lang="text" class="" style="" inline="1">glob()</syntaxhighlight> routine under-the-hood, after splitting the arguments.
Most other parts of Windows, including the Indexing Service, use the MS-DOS style of wildcards found in CMD. A relic of the 8.3 filename age, this syntax pays special attention to dots in the pattern and the text (filename). Internally this is done using three extra wildcard characters, <syntaxhighlight lang="text" class="" style="" inline="1"><>"</syntaxhighlight>. On the Windows API end, the Template:Tt equivalent is Template:Tt, and Template:Tt corresponds to its underlying Template:Tt.<ref>Wildcards in Windows Template:Webarchive. MSDN Devblog.</ref> (Another fnmatch analogue is Template:Tt.) Both open-source msvcrt expanders use Template:Tt, so 8.3 filename quirks will also apply in them.
SQLEdit
The SQL <syntaxhighlight lang="text" class="" style="" inline="1">LIKE</syntaxhighlight> operator has an equivalent to <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight> and <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight> but not <syntaxhighlight lang="text" class="" style="" inline="1">[…]</syntaxhighlight>.
| Common wildcard | SQL wildcard | Description |
|---|---|---|
| <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">_</syntaxhighlight> | matches any single character |
| <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">%</syntaxhighlight> | matches any number of any characters including none |
Standard SQL uses a glob-like syntax for simple string matching in its LIKE operator, although the term "glob" is not generally used in the SQL community. The percent sign (<syntaxhighlight lang="text" class="" style="" inline="1">%</syntaxhighlight>) matches zero or more characters and the underscore (<syntaxhighlight lang="text" class="" style="" inline="1">_</syntaxhighlight>) matches exactly one.
Many implementations of SQL have extended the LIKE operator to allow a richer pattern-matching language, incorporating character ranges (<syntaxhighlight lang="text" class="" style="" inline="1">[…]</syntaxhighlight>), their negation, and elements of regular expressions.<ref name="transact-sql-like"/>
Compared to regular expressionsEdit
Globs do not include syntax for the Kleene star which allows multiple repetitions of the preceding part of the expression; thus they are not considered regular expressions, which can describe the full set of regular languages over any given finite alphabet.<ref>Template:Cite book</ref>
| Common wildcard | Equivalent regular expression |
|---|---|
| <syntaxhighlight lang="text" class="" style="" inline="1">?</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">.</syntaxhighlight> |
| <syntaxhighlight lang="text" class="" style="" inline="1">*</syntaxhighlight> | <syntaxhighlight lang="text" class="" style="" inline="1">.*</syntaxhighlight> |
Globs attempt to match the entire string (for example, <syntaxhighlight lang="text" class="" style="" inline="1">S*.DOC</syntaxhighlight> matches S.DOC and SA.DOC, but not POST.DOC or SURREY.DOCKS), whereas, depending on implementation details, regular expressions may match a substring.
Implementing as regular expressionsEdit
The original Mozilla proxy auto-config implementation, which provides a glob-matching function on strings, uses a replace-as-RegExp implementation as above. The bracket syntax happens to be covered by regex in such an example.
Python's fnmatch uses a more elaborate procedure to transform the pattern into a regular expression.<ref name=py>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref>
Other implementationsEdit
Beyond their uses in shells, globs patterns also find use in a variety of programming languages, mainly to process human input. A glob-style interface for returning files or an fnmatch-style interface for matching strings are found in the following programming languages:
- C and C++ do not have built-in support for glob patterns in the ISO-defined standard libraries, however on Unix-like systems C and C++ may include
<glob.h>from the C POSIX library to useglob().- C++ itself does not have direct support for glob patterns, however they may be approximated using the
<filesystem>and<regex>headers, usingstd::filesystem::directory_iterator()andstd::regex_match().
- C++ itself does not have direct support for glob patterns, however they may be approximated using the
- C# has multiple libraries available through NuGet such as
Glob<ref name="dotnet-glob"/> orDotNet.Glob.<ref name="dotnet.glob"/> - D has a
globMatchfunction in thestd.pathmodule.<ref name="dlang"/> - JavaScript has a library called
minimatchwhich is used internally by npm, andmicromatch, a purportedly more optimized, accurate and safer globbing implementation used by Babel and yarn.<ref name="minimatch"/><ref name="micromatch"/> - Go has a
Globfunction in thefilepathpackage.<ref name="golang"/> - Java has a
Filesclass in the packagejava.nio.file, containing methods that can operate on glob patterns.<ref name="java"/> - Haskell has a
Globpackage with the main moduleSystem.FilePath.Glob. The pattern syntax is based on a subset of Zsh's. It tries to optimize the given pattern and should be noticeably faster than a naïve character-by-character matcher.<ref name="hs"/> - Perl has both a
globfunction (as discussed in Larry Wall's book Programming Perl) and a Glob extension which mimics the BSD glob routine.<ref name="pl"/> Perl's angle brackets can be used to glob as well:<*.log>. - PHP has a
globfunction.<ref name="php"/> - Python has a
globmodule in the standard library which performs wildcard pattern matching on filenames,<ref name="pyglob"/> and anfnmatchmodule with functions for matching strings or filtering lists based on these same wildcard patterns.<ref name=py/> Guido van Rossum, author of the Python programming language, wrote and contributed aglobroutine to BSD Unix in 1986.<ref name="isc-glob"/> There were previous implementations ofglob, e.g., in the ex and ftp programs in previous releases of BSD. - Ruby has a
globmethod for theDirclass which performs wildcard pattern matching on filenames.<ref name="rbdir"/> Several libraries such as Rant and Rake provide aFileListclass which has a glob method or use the methodFileList.[]identically. - Rust has multiple libraries that can match glob patterns,<ref>{{#invoke:citation/CS1|citation
|CitationClass=web
}}</ref> the most popular of these being the glob crate.<ref>{{#invoke:citation/CS1|citation
|CitationClass=web
}}</ref>