Regular expressions are a commonly used method of pattern matching on strings, with regex being a catchy word for a pattern in this domain specific language. Typical problems usually solved by regular expressions include validation of user input and the ubiquitous find & replace in text processing utilities.
| Category | Functions |
|---|---|
| Matching | bmatch match matchAll matchFirst |
| Building | ctRegex escaper regex |
| Replace | replace replaceAll replaceAllInto replaceFirst replaceFirstInto |
| Split | split splitter |
| Objects | Captures Regex RegexException RegexMatch Splitter StaticRegex |
import std.regex;
import std.stdio;
void main()
{
// Print out all possible dd/mm/yy(yy) dates found in user input.
auto r = regex(r"\b[0-9][0-9]?/[0-9][0-9]?/[0-9][0-9](?:[0-9][0-9])?\b");
foreach (line; stdin.byLine)
{
// matchAll() returns a range that can be iterated
// to get all subsequent matches.
foreach (c; matchAll(line, r))
writeln(c.hit);
}
}
...
// Create a static regex at compile-time, which contains fast native code.
auto ctr = ctRegex!(`^.*/([^/]+)/?$`);
// It works just like a normal regex:
auto c2 = matchFirst("foo/bar", ctr); // First match found here, if any
assert(!c2.empty); // Be sure to check if there is a match before examining contents!
assert(c2[1] == "bar"); // Captures is a range of submatches: 0 = full match.
...
// multi-pattern regex
auto multi = regex([`\d+,\d+`,`(a-z]+):(\d+)`]);
auto m = "abc:43 12,34".matchAll(multi);
assert(m.front.whichPattern == 2);
assert(m.front[1] == "abc");
assert(m.front[2] == "43");
m.popFront();
assert(m.front.whichPattern == 1);
assert(m.front[1] == "12");
...
// The result of the `matchAll/matchFirst` is directly testable with if/assert/while.
// e.g. test if a string consists of letters:
assert(matchFirst("Letter", `^\p{L}+$`));
std.regex see a short tour of the module API and its abilities. | Pattern element | Semantics |
| Atoms | Match single characters |
| any character except [{|*+?()^$ | Matches the character itself. |
| . | In single line mode matches any character. Otherwise it matches any character except '\n' and '\r'. |
| [class] | Matches a single character that belongs to this character class. |
| [^class] | Matches a single character that does not belong to this character class. |
| \cC | Matches the control character corresponding to letter C |
| \xXX | Matches a character with hexadecimal value of XX. |
| \uXXXX | Matches a character with hexadecimal value of XXXX. |
| \U00YYYYYY | Matches a character with hexadecimal value of YYYYYY. |
| \f | Matches a formfeed character. |
| \n | Matches a linefeed character. |
| \r | Matches a carriage return character. |
| \t | Matches a tab character. |
| \v | Matches a vertical tab character. |
| \d | Matches any Unicode digit. |
| \D | Matches any character except Unicode digits. |
| \w | Matches any word character (note: this includes numbers). |
| \W | Matches any non-word character. |
| \s | Matches whitespace, same as \p{White_Space}. |
| \S | Matches any character except those recognized as \s . |
| \\ | Matches \ character. |
| \c where c is one of [|*+?() | Matches the character c itself. |
| \p{PropertyName} | Matches a character that belongs to the Unicode PropertyName set. Single letter abbreviations can be used without surrounding {,}. |
| \P{PropertyName} | Matches a character that does not belong to the Unicode PropertyName set. Single letter abbreviations can be used without surrounding {,}. |
| \p{InBasicLatin} | Matches any character that is part of the BasicLatin Unicode block. |
| \P{InBasicLatin} | Matches any character except ones in the BasicLatin Unicode block. |
| \p{Cyrillic} | Matches any character that is part of Cyrillic script. |
| \P{Cyrillic} | Matches any character except ones in Cyrillic script. |
| Quantifiers | Specify repetition of other elements |
| * | Matches previous character/subexpression 0 or more times. Greedy version - tries as many times as possible. |
| *? | Matches previous character/subexpression 0 or more times. Lazy version - stops as early as possible. |
| + | Matches previous character/subexpression 1 or more times. Greedy version - tries as many times as possible. |
| +? | Matches previous character/subexpression 1 or more times. Lazy version - stops as early as possible. |
| {n} | Matches previous character/subexpression exactly n times. |
| {n,} | Matches previous character/subexpression n times or more. Greedy version - tries as many times as possible. |
| {n,}? | Matches previous character/subexpression n times or more. Lazy version - stops as early as possible. |
| {n,m} | Matches previous character/subexpression n to m times. Greedy version - tries as many times as possible, but no more than m times. |
| {n,m}? | Matches previous character/subexpression n to m times. Lazy version - stops as early as possible, but no less then n times. |
| Other | Subexpressions & alternations |
| (regex) | Matches subexpression regex, saving matched portion of text for later retrieval. |
| (?#comment) | An inline comment that is ignored while matching. |
| (?:regex) | Matches subexpression regex, not saving matched portion of text. Useful to speed up matching. |
| A|B | Matches subexpression A, or failing that, matches B. |
| (?P<name>regex) | Matches named subexpression regex labeling it with name 'name'. When referring to a matched portion of text, names work like aliases in addition to direct numbers. |
| Assertions | Match position rather than character |
| ^ | Matches at the begining of input or line (in multiline mode). |
| $ | Matches at the end of input or line (in multiline mode). |
| \b | Matches at word boundary. |
| \B | Matches when not at word boundary. |
| (?=regex) | Zero-width lookahead assertion. Matches at a point where the subexpression regex could be matched starting from the current position. |
| (?!regex) | Zero-width negative lookahead assertion. Matches at a point where the subexpression regex could not be matched starting from the current position. |
| (?<=regex) | Zero-width lookbehind assertion. Matches at a point where the subexpression regex could be matched ending at the current position (matching goes backwards). |
| (?<!regex) | Zero-width negative lookbehind assertion. Matches at a point where the subexpression regex could not be matched ending at the current position (matching goes backwards). |
| Pattern element | Semantics |
| Any atom | Has the same meaning as outside of a character class. |
| a-z | Includes characters a, b, c, ..., z. |
| [a||b], [a--b], [a~~b], [a&&b] | Where a, b are arbitrary classes, means union, set difference, symmetric set difference, and intersection respectively. Any sequence of character class elements implicitly forms a union. |
| Flag | Semantics |
| g | Global regex, repeat over the whole input. |
| i | Case insensitive matching. |
| m | Multi-line mode, match ^, $ on start and end line separators as well as start and end of input. |
| s | Single-line mode, makes . match '\n' and '\r' as well. |
| x | Free-form syntax, ignores whitespace in pattern, useful for formatting complex regular expressions. |
format argument of replaceFirst and replaceAll. | Format specifier | Replaced by |
| $& | the whole match. |
| $` | part of input preceding the match. |
| $' | part of input following the match. |
| $$ | '$' character. |
| \c , where c is any character | the character c itself. |
| \\ | '\' character. |
| $1 .. $99 | submatch number 1 to 99 respectively. |
replace family of functions that generate a new string from the input. replaceFirstInto and replaceAllInto could come in handy as functions that avoid allocations even for replacement. std.regex by Walter Bright and Andrei Alexandrescu. Regex object holds regular expression pattern in compiled form.
Instances of this object are constructed via calls to regex. This is an intended form for caching and storage of frequently used regular expressions.
Regex!char r;
assert(r.empty);
r = regex(""); // Note: "" is a valid regex pattern.
assert(!r.empty);
Getting a range of all the named captures in the regex. import std.range; import std.algorithm; auto re = regex(`(?P<name>\w+) = (?P<var>\d+)`); auto nc = re.namedCaptures; static assert(isRandomAccessRange!(typeof(nc))); assert(!nc.empty); assert(nc.length == 2); assert(nc.equal(["name", "var"])); assert(nc[0] == "name"); assert(nc[1..$].equal(["var"]));
A StaticRegex is Regex object that contains D code specially generated at compile-time to speed up matching.
No longer used, kept as alias to Regex for backwards compatibility.
Compile regular expression pattern for the later execution.
Regex object that works on inputs having the same character width as pattern. S pattern
| A single regular expression to match. |
S[] patterns
| An array of regular expression strings. The resulting Regex object will match any expression; use whichPattern to know which. |
const(char)[] flags
| The attributes (g, i, m, s and x accepted) |
RegexException if there were any errors during compilation.// multi-pattern regex example auto multi = regex([`([a-z]+):(\d+)`, `(\d+),\d+`]); // multi regex auto m = "abc:43 12,34".matchAll(multi); writeln(m.front.whichPattern); // 1 writeln(m.front[1]); // "abc" writeln(m.front[2]); // "43" m.popFront(); writeln(m.front.whichPattern); // 2 writeln(m.front[1]); // "12"
Compile regular expression using CTFE and generate optimized native machine code for matching it.
| pattern | Regular expression |
| flags | The attributes (g, i, m, s and x accepted) |
Captures object contains submatches captured during a call to match or iteration over RegexMatch range.
First element of range is the whole match.
import std.range.primitives : popFrontN;
auto c = matchFirst("@abc#", regex(`(\w)(\w)(\w)`));
assert(c.pre == "@"); // Part of input preceding match
assert(c.post == "#"); // Immediately after match
assert(c.hit == c[0] && c.hit == "abc"); // The whole match
writeln(c[2]); // "b"
writeln(c.front); // "abc"
c.popFront();
writeln(c.front); // "a"
writeln(c.back); // "c"
c.popBack();
writeln(c.back); // "b"
popFrontN(c, 2);
assert(c.empty);
assert(!matchFirst("nothing", "something"));
// Captures that are not matched will be null.
c = matchFirst("ac", regex(`a(b)?c`));
assert(c);
assert(!c[1]);
Slice of input prior to the match.
Slice of input immediately after the match.
Slice of matched portion of input.
Range interface.
Explicit cast to bool. Useful as a shorthand for !(x.empty) in if and assert statements.
import std.regex;
assert(!matchFirst("nothing", "something"));
Number of pattern matched counting, where 1 - the first pattern. Returns 0 on no match.
import std.regex;
writeln(matchFirst("abc", "[0-9]+", "[a-z]+").whichPattern); // 2
Lookup named submatch.
import std.regex;
import std.range;
auto c = matchFirst("a = 42;", regex(`(?P<var>\w+)\s*=\s*(?P<value>\d+);`));
assert(c["var"] == "a");
assert(c["value"] == "42");
popFrontN(c, 2);
//named groups are unaffected by range primitives
assert(c["var"] =="a");
assert(c.front == "42");
Number of matches in this object.
A hook for compatibility with original std.regex.
A regex engine state, as returned by match family of functions.
Effectively it's a forward range of Captures!R, produced by lazily searching for matches in a given input.
Shorthands for front.pre, front.post, front.hit.
Functionality for processing subsequent matches of global regexes via range interface:
import std.regex;
auto m = matchAll("Hello, world!", regex(`\w+`));
assert(m.front.hit == "Hello");
m.popFront();
assert(m.front.hit == "world");
m.popFront();
assert(m.empty);
Test if this match object is empty.
Same as !(x.empty), provided for its convenience in conditional statements.
Same as .front, provided for compatibility with original std.regex.
Start matching input to regex pattern re, using Thompson NFA matching scheme.
The use of this function is discouraged - use either of matchAll or matchFirst.
Delegating the kind of operation to "g" flag is soon to be phased out along with the ability to choose the exact matching scheme. The choice of matching scheme to use depends highly on the pattern kind and can done automatically on case by case basis.
RegexMatch object holding engine state after first match.Find the first (leftmost) slice of the input that matches the pattern re. This function picks the most suitable regular expression engine depending on the pattern properties.
re parameter can be one of three types:
Initiate a search for all non-overlapping matches to the pattern re in the given input. The result is a lazy range of matches generated as they are encountered in the input going left to right.
This function picks the most suitable regular expression engine depending on the pattern properties.
re parameter can be one of three types:
RegexMatch object that represents matcher state after the first match was found or an empty one if not present.Start matching of input to regex pattern re, using traditional backtracking matching scheme.
The use of this function is discouraged - use either of matchAll or matchFirst.
Delegating the kind of operation to "g" flag is soon to be phased out along with the ability to choose the exact matching scheme. The choice of matching scheme to use depends highly on the pattern kind and can done automatically on case by case basis.
RegexMatch object holding engine state after first match.Construct a new string from input by replacing the first match with a string generated from it according to the format specifier.
To replace all matches use replaceAll.
R input
| string to search |
RegEx re
| compiled regular expression to use |
const(C)[] format
| format string to generate replacements from, see the format string. |
writeln(replaceFirst("noon", regex("n"), "[$&]")); // "[n]oon"
This is a general replacement tool that construct a new string by replacing matches of pattern re in the input. Unlike the other overload there is no format string instead captures are passed to to a user-defined functor fun that returns a new string to use as replacement.
This version replaces the first match in input, see replaceAll to replace the all of the matches.
input with all matches replaced by return values of fun. If no matches found returns the input itself.import std.conv : to;
string list = "#21 out of 46";
string newList = replaceFirst!(cap => to!string(to!int(cap.hit)+1))
(list, regex(`[0-9]+`));
writeln(newList); // "#22 out of 46"
A variation on replaceFirst that instead of allocating a new string on each call outputs the result piece-wise to the sink. In particular this enables efficient construction of a final output incrementally.
Like in replaceFirst family of functions there is an overload for the substitution guided by the format string and the one with the user defined callback.
import std.array; string m1 = "first message\n"; string m2 = "second message\n"; auto result = appender!string(); replaceFirstInto(result, m1, regex(`([a-z]+) message`), "$1"); //equivalent of the above with user-defined callback replaceFirstInto!(cap=>cap[1])(result, m2, regex(`([a-z]+) message`)); writeln(result.data); // "first\nsecond\n"
Construct a new string from input by replacing all of the fragments that match a pattern re with a string generated from the match according to the format specifier.
To replace only the first match use replaceFirst.
R input
| string to search |
RegEx re
| compiled regular expression to use |
const(C)[] format
| format string to generate replacements from, see the format string. |
input with the all of the matches (if any) replaced. If no match is found returns the input string itself.// insert comma as thousands delimiter
auto re = regex(r"(?<=\d)(?=(\d\d\d)+\b)","g");
writeln(replaceAll("12000 + 42100 = 54100", re, ",")); // "12,000 + 42,100 = 54,100"
This is a general replacement tool that construct a new string by replacing matches of pattern re in the input. Unlike the other overload there is no format string instead captures are passed to to a user-defined functor fun that returns a new string to use as replacement.
This version replaces all of the matches found in input, see replaceFirst to replace the first match only.
input with all matches replaced by return values of fun. If no matches found returns the input itself. R input
| string to search |
RegEx re
| compiled regular expression |
| fun | delegate to use |
string baz(Captures!(string) m)
{
import std.string : toUpper;
return toUpper(m.hit);
}
// Capitalize the letters 'a' and 'r':
auto s = replaceAll!(baz)("Strap a rocket engine on a chicken.",
regex("[ar]"));
writeln(s); // "StRAp A Rocket engine on A chicken."
A variation on replaceAll that instead of allocating a new string on each call outputs the result piece-wise to the sink. In particular this enables efficient construction of a final output incrementally.
As with replaceAll there are 2 overloads - one with a format string, the other one with a user defined functor.
// insert comma as thousands delimiter in fifty randomly produced big numbers
import std.array, std.conv, std.random, std.range;
static re = regex(`(?<=\d)(?=(\d\d\d)+\b)`, "g");
auto sink = appender!(char [])();
enum ulong min = 10UL ^^ 10, max = 10UL ^^ 19;
foreach (i; 0 .. 50)
{
sink.clear();
replaceAllInto(sink, text(uniform(min, max)), re, ",");
foreach (pos; iota(sink.data.length - 4, 0, -4))
writeln(sink.data[pos]); // ','
}
Old API for replacement, operation depends on flags of pattern re. With "g" flag it performs the equivalent of replaceAll otherwise it works the same as replaceFirst.
The use of this function is discouraged, please use replaceAll or replaceFirst explicitly.
Splits a string r using a regular expression pat as a separator.
| keepSeparators | flag to specify if the matches should be in the resulting range |
Range r
| the string to split |
RegEx pat
| the pattern to split on |
import std.algorithm.comparison : equal;
auto s1 = ", abc, de, fg, hi, ";
assert(equal(splitter(s1, regex(", *")),
["", "abc", "de", "fg", "hi", ""]));
import std.algorithm.comparison : equal;
import std.typecons : Yes;
auto pattern = regex(`([\.,])`);
assert("2003.04.05"
.splitter!(Yes.keepSeparators)(pattern)
.equal(["2003", ".", "04", ".", "05"]));
assert(",1,2,3"
.splitter!(Yes.keepSeparators)(pattern)
.equal([",", "1", ",", "2", ",", "3"]));
Forward range primitives.
An eager version of splitter that creates an array with splitted slices of input.
Exception object thrown in case of errors during regex compilation.
A range that lazily produces a string output escaped to be used inside of a regular expression.
import std.algorithm.comparison;
import std.regex;
string s = `This is {unfriendly} to *regex*`;
assert(s.escaper.equal(`This is \{unfriendly\} to \*regex\*`));
© 1999–2019 The D Language Foundation
Licensed under the Boost License 1.0.
https://dlang.org/phobos/std_regex.html