You are previewing Regular Expressions Cookbook, 2nd Edition.

Regular Expressions Cookbook, 2nd Edition

Cover of Regular Expressions Cookbook, 2nd Edition by Jan Goyvaerts... Published by O'Reilly Media, Inc.
  1. Regular Expressions Cookbook
  2. Preface
    1. Caught in the Snarls of Different Versions
    2. Intended Audience
    3. Technology Covered
    4. Organization of This Book
    5. Conventions Used in This Book
    6. Using Code Examples
    7. Safari® Books Online
    8. How to Contact Us
    9. Acknowledgments
  3. 1. Introduction to Regular Expressions
    1. Regular Expressions Defined
      1. Many Flavors of Regular Expressions
      2. Regex Flavors Covered by This Book
    2. Search and Replace with Regular Expressions
      1. Many Flavors of Replacement Text
    3. Tools for Working with Regular Expressions
      1. RegexBuddy
      2. RegexPal
      3. RegexMagic
      4. More Online Regex Testers
      5. More Desktop Regular Expression Testers
      6. grep
      7. Popular Text Editors
  4. 2. Basic Regular Expression Skills
    1. 2.1. Match Literal Text
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    2. 2.2. Match Nonprintable Characters
      1. Problem
      2. Solution
      3. Discussion
      4. Variations on Representations of Nonprinting Characters
      5. See Also
    3. 2.3. Match One of Many Characters
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. Flavor-Specific Features
      6. See Also
    4. 2.4. Match Any Character
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    5. 2.5. Match Something at the Start and/or the End of a Line
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    6. 2.6. Match Whole Words
      1. Problem
      2. Solution
      3. Discussion
      4. Word Characters
      5. See Also
    7. 2.7. Unicode Code Points, Categories, Blocks, and Scripts
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    8. 2.8. Match One of Several Alternatives
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    9. 2.9. Group and Capture Parts of the Match
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    10. 2.10. Match Previously Matched Text Again
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    11. 2.11. Capture and Name Parts of the Match
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. 2.12. Repeat Part of the Regex a Certain Number of Times
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    13. 2.13. Choose Minimal or Maximal Repetition
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    14. 2.14. Eliminate Needless Backtracking
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    15. 2.15. Prevent Runaway Repetition
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    16. 2.16. Test for a Match Without Adding It to the Overall Match
      1. Problem
      2. Solution
      3. Discussion
      4. Alternative to Lookbehind
      5. Solution Without Lookbehind
      6. See Also
    17. 2.17. Match One of Two Alternatives Based on a Condition
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    18. 2.18. Add Comments to a Regular Expression
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
    19. 2.19. Insert Literal Text into the Replacement Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    20. 2.20. Insert the Regex Match into the Replacement Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    21. 2.21. Insert Part of the Regex Match into the Replacement Text
      1. Problem
      2. Solution
      3. Discussion
      4. Solution Using Named Capture
      5. See Also
    22. 2.22. Insert Match Context into the Replacement Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
  5. 3. Programming with Regular Expressions
    1. Programming Languages and Regex Flavors
      1. Languages Covered in This Chapter
      2. More Programming Languages
    2. 3.1. Literal Regular Expressions in Source Code
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    3. 3.2. Import the Regular Expression Library
      1. Problem
      2. Solution
      3. Discussion
    4. 3.3. Create Regular Expression Objects
      1. Problem
      2. Solution
      3. Discussion
      4. Compiling a Regular Expression Down to CIL
      5. Discussion
      6. See Also
    5. 3.4. Set Regular Expression Options
      1. Problem
      2. Solution
      3. Discussion
      4. Additional Language-Specific Options
      5. See Also
    6. 3.5. Test If a Match Can Be Found Within a Subject String
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    7. 3.6. Test Whether a Regex Matches the Subject String Entirely
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    8. 3.7. Retrieve the Matched Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    9. 3.8. Determine the Position and Length of the Match
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    10. 3.9. Retrieve Part of the Matched Text
      1. Problem
      2. Solution
      3. Discussion
      4. Named Capture
      5. See Also
    11. 3.10. Retrieve a List of All Matches
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. 3.11. Iterate over All Matches
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    13. 3.12. Validate Matches in Procedural Code
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    14. 3.13. Find a Match Within Another Match
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    15. 3.14. Replace All Matches
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    16. 3.15. Replace Matches Reusing Parts of the Match
      1. Problem
      2. Solution
      3. Discussion
      4. Named Capture
      5. See Also
    17. 3.16. Replace Matches with Replacements Generated in Code
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    18. 3.17. Replace All Matches Within the Matches of Another Regex
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    19. 3.18. Replace All Matches Between the Matches of Another Regex
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    20. 3.19. Split a String
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    21. 3.20. Split a String, Keeping the Regex Matches
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    22. 3.21. Search Line by Line
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    23. Construct a Parser
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
  6. 4. Validation and Formatting
    1. 4.1. Validate Email Addresses
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    2. 4.2. Validate and Format North American Phone Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    3. 4.3. Validate International Phone Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    4. 4.4. Validate Traditional Date Formats
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    5. 4.5. Validate Traditional Date Formats, Excluding Invalid Dates
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    6. 4.6. Validate Traditional Time Formats
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    7. 4.7. Validate ISO 8601 Dates and Times
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    8. 4.8. Limit Input to Alphanumeric Characters
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    9. 4.9. Limit the Length of Text
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    10. 4.10. Limit the Number of Lines in Text
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    11. 4.11. Validate Affirmative Responses
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. 4.12. Validate Social Security Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    13. 4.13. Validate ISBNs
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    14. 4.14. Validate ZIP Codes
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    15. 4.15. Validate Canadian Postal Codes
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    16. 4.16. Validate U.K. Postcodes
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    17. 4.17. Find Addresses with Post Office Boxes
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    18. 4.18. Reformat Names From “FirstName LastName” to “LastName, FirstName”
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    19. 4.19. Validate Password Complexity
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    20. 4.20. Validate Credit Card Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. Extra Validation with the Luhn Algorithm
      5. See Also
    21. 4.21. European VAT Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
  7. 5. Words, Lines, and Special Characters
    1. 5.1. Find a Specific Word
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    2. 5.2. Find Any of Multiple Words
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    3. 5.3. Find Similar Words
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    4. 5.4. Find All Except a Specific Word
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    5. 5.5. Find Any Word Not Followed by a Specific Word
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    6. 5.6. Find Any Word Not Preceded by a Specific Word
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    7. 5.7. Find Words Near Each Other
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    8. 5.8. Find Repeated Words
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    9. 5.9. Remove Duplicate Lines
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    10. 5.10. Match Complete Lines That Contain a Word
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    11. 5.11. Match Complete Lines That Do Not Contain a Word
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. 5.12. Trim Leading and Trailing Whitespace
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    13. 5.13. Replace Repeated Whitespace with a Single Space
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    14. 5.14. Escape Regular Expression Metacharacters
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
  8. 6. Numbers
    1. 6.1. Integer Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    2. 6.2. Hexadecimal Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    3. 6.3. Binary Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    4. 6.4. Octal Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    5. 6.5. Decimal Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    6. 6.6. Strip Leading Zeros
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    7. 6.7. Numbers Within a Certain Range
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    8. 6.8. Hexadecimal Numbers Within a Certain Range
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    9. 6.9. Integer Numbers with Separators
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    10. 6.10. Floating-Point Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    11. 6.11. Numbers with Thousand Separators
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. 6.12. Add Thousand Separators to Numbers
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    13. 6.13. Roman Numerals
      1. Problem
      2. Solution
      3. Discussion
      4. Convert Roman Numerals to Decimal
      5. See Also
  9. 7. Source Code and Log Files
    1. Keywords
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    2. Identifiers
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    3. Numeric Constants
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    4. Operators
      1. Problem
      2. Solution
      3. Discussion
    5. Single-Line Comments
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    6. Multiline Comments
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    7. All Comments
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    8. Strings
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    9. Strings with Escapes
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    10. Regex Literals
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    11. Here Documents
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. Common Log Format
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    13. Combined Log Format
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    14. Broken Links Reported in Web Logs
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
  10. 8. URLs, Paths, and Internet Addresses
    1. 8.1. Validating URLs
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    2. 8.2. Finding URLs Within Full Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    3. 8.3. Finding Quoted URLs in Full Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    4. 8.4. Finding URLs with Parentheses in Full Text
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    5. 8.5. Turn URLs into Links
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    6. 8.6. Validating URNs
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    7. 8.7. Validating Generic URLs
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    8. 8.8. Extracting the Scheme from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    9. 8.9. Extracting the User from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    10. 8.10. Extracting the Host from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    11. 8.11. Extracting the Port from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    12. 8.12. Extracting the Path from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    13. 8.13. Extracting the Query from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    14. 8.14. Extracting the Fragment from a URL
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    15. 8.15. Validating Domain Names
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    16. 8.16. Matching IPv4 Addresses
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    17. 8.17. Matching IPv6 Addresses
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    18. 8.18. Validate Windows Paths
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    19. 8.19. Split Windows Paths into Their Parts
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    20. 8.20. Extract the Drive Letter from a Windows Path
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    21. 8.21. Extract the Server and Share from a UNC Path
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    22. 8.22. Extract the Folder from a Windows Path
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    23. 8.23. Extract the Filename from a Windows Path
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    24. 8.24. Extract the File Extension from a Windows Path
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    25. 8.25. Strip Invalid Characters from Filenames
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
  11. 9. Markup and Data Formats
    1. Processing Markup and Data Formats with Regular Expressions
      1. Basic Rules for Formats Covered in This Chapter
    2. 9.1. Find XML-Style Tags
      1. Problem
      2. Solution
      3. Discussion
      4. Skip Tricky (X)HTML and XML Sections
      5. See Also
    3. 9.2. Replace <b> Tags with <strong>
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    4. 9.3. Remove All XML-Style Tags Except <em> and <strong>
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    5. 9.4. Match XML Names
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    6. 9.5. Convert Plain Text to HTML by Adding <p> and <br> Tags
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    7. 9.6. Decode XML Entities
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    8. 9.7. Find a Specific Attribute in XML-Style Tags
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    9. 9.8. Add a cellspacing Attribute to <table> Tags That Do Not Already Include It
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    10. 9.9. Remove XML-Style Comments
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    11. 9.10. Find Words Within XML-Style Comments
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    12. 9.11. Change the Delimiter Used in CSV Files
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    13. 9.12. Extract CSV Fields from a Specific Column
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    14. 9.13. Match INI Section Headers
      1. Problem
      2. Solution
      3. Discussion
      4. Variations
      5. See Also
    15. 9.14. Match INI Section Blocks
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
    16. 9.15. Match INI Name-Value Pairs
      1. Problem
      2. Solution
      3. Discussion
      4. See Also
  12. Index
  13. About the Authors
  14. Colophon
  15. Copyright
O'Reilly logo

4.1. Validate Email Addresses

Problem

You have a form on your website or a dialog box in your application that asks the user for an email address. You want to use a regular expression to validate this email address before trying to send email to it. This reduces the number of emails returned to you as undeliverable.

Solution

Simple

This first solution does a very simple check. It only validates that the string contains an at sign (@) that is preceded and followed by one or more nonwhitespace characters.

^\S+@\S+$
Regex options: None
Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python
\A\S+@\S+\Z
Regex options: None
Regex flavors: .NET, Java, PCRE, Perl, Python, Ruby

Simple, with restrictions on characters

The domain name, the part after the @ sign, is restricted to characters allowed in domain names. Internationalized domain names are not allowed. The local part, the part before the @ sign, is restricted to characters commonly used in email local parts, which is more restrictive than what most email clients and servers will accept:

^[A-Z0-9+_.-]+@[A-Z0-9.-]+$
Regex options: Case insensitive
Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python
\A[A-Z0-9+_.-]+@[A-Z0-9.-]+\Z
Regex options: Case insensitive
Regex flavors: .NET, Java, PCRE, Perl, Python, Ruby

Simple, with all valid local part characters

This regular expression expands the previous one by allowing a larger set of rarely used characters in the local part. Not all email software can handle all these characters, but we’ve included all the characters permitted by RFC 5322, which governs the email message format. Among the permitted characters are some that present a security risk if passed directly from user input to an SQL statement, such as the single quote (') and the pipe character (|). Be sure to escape sensitive characters when inserting the email address into a string passed to another program, in order to prevent security holes such as SQL injection attacks:

^[A-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[A-Z0-9.-]+$
Regex options: Case insensitive
Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python
\A[A-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[A-Z0-9.-]+\Z
Regex options: Case insensitive
Regex flavors: .NET, Java, PCRE, Perl, Python, Ruby

No leading, trailing, or consecutive dots

Both the local part and the domain name can contain one or more dots, but no two dots can appear right next to each other. Furthermore, the first and last characters in the local part and in the domain name must not be dots:

^[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+↵
)*@[A-Z0-9-]+(?:\.[A-Z0-9-]+)*$
Regex options: Case insensitive
Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python
\A[A-Z0-9_!#$%&'*+/=?`{|}~^-]+(?:\.[A-Z0-9_!#$%&'*+/=?`{|}~^-]+↵
)*@[A-Z0-9-]+(?:\.[A-Z0-9-]+)*\Z
Regex options: Case insensitive
Regex flavors: .NET, Java, PCRE, Perl, Python, Ruby

Top-level domain has two to six letters

This regular expression adds to the previous versions by specifying that the domain name must include at least one dot, and that the part of the domain name after the last dot can only consist of letters. That is, the domain must contain at least two levels, such as secondlevel.com or thirdlevel.secondlevel.com. The top-level domain (.com in these examples) must consist of two to six letters. All country-code top-level domains (.us, .uk, etc.) have two letters. The generic top-level domains have between three (.com) and six letters (.museum):

^[\w!#$%&'*+/=?`{|}~^-]+(?:\.[\w!#$%&'*+/=?`{|}~^-]+)*@↵
(?:[A-Z0-9-]+\.)+[A-Z]{2,6}$
Regex options: Case insensitive
Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python
\A[\w!#$%&'*+/=?`{|}~^-]+(?:\.[\w!#$%&'*+/=?`{|}~^-]+)*@↵
(?:[A-Z0-9-]+\.)+[A-Z]{2,6}\Z
Regex options: Case insensitive
Regex flavors: .NET, Java, PCRE, Perl, Python, Ruby

Discussion

About email addresses

If you thought something as conceptually simple as validating an email address would have a simple one-size-fits-all regex solution, you’re quite wrong. This recipe is a prime example that before you can start writing a regular expression, you have to decide exactly what you want to match. There is no universally agreed-upon rule as to which email addresses are valid and which not. It depends on your definition of valid.

asdf@asdf.asdf is valid according to RFC 5322, which defines the syntax for email addresses. But it is not valid if your definition specifies that a valid email address is one that accepts mail. There is no top-level asdf domain.

The short answer to the validity problem is that you can’t know whether john.doe@somewhere.com is an email address that can actually receive email until you try to send email to it. And even then, you can’t be sure if the lack of response signals that the somewhere.com domain is silently discarding mail sent to nonexistent mailboxes, or if John Doe hit the Delete button on his keyboard, or if his spam filter beat him to it.

Because you ultimately have to check whether the address exists by actually sending email to it, you can decide to use a simpler or more relaxed regular expression. Allowing invalid addresses to slip through may be preferable to annoying people by blocking valid addresses. For this reason, you may want to select the “simple” regular expression. Though it obviously allows many things that aren’t email addresses, such as #$%@.-, the regex is quick and simple, and will never block a valid email address.

If you want to avoid sending too many undeliverable emails, while still not blocking any real email addresses, the regex in Top-level domain has two to six letters is a good choice.

You have to consider how complex you want your regular expression to be. If you’re validating user input, you’ll likely want a more complex regex, because the user could type in anything. But if you’re scanning database files that you know contain only valid email addresses, you can use a very simple regex that merely separates the email addresses from the other data. Even the solution in the earlier subsection may be enough in this case.

Finally, you have to consider how future-proof you want your regular expression to be. In the past, it made sense to restrict the top-level domain to only two-letter combinations for the country codes, and exhaustively list the generic top-level domains—that is, com|net|org|mil|edu. With new top-level domains being added all the time, such regular expressions now quickly go out of date.

Regular expression syntax

The regular expressions presented in this recipe show all the basic parts of the regular expression syntax in action. If you read up on these parts in Chapter 2, you can already do 90% of the jobs that are best solved with regular expressions.

All the regular expressions, except the “simple” one, require the case-insensitive matching option to be turned on. Otherwise, only uppercase characters will be allowed. Turning on this option allows you to type [A-Z] instead of [A-Za-z], saving a few keystrokes.

\S is a shorthand character class, as Recipe 2.3 explains. \S matches any character that is not a whitespace character.

@ and \. match a literal @ sign and a dot, respectively. Since the dot is a metacharacter when used outside character classes, it needs to be escaped with a backslash. The @ sign never has a special meaning with any of the regular expression flavors in this book. Recipe 2.1 gives you a list of all the metacharacters that need to be escaped.

[A-Z0-9.-] and the other sequences between square brackets are character classes. This one allows all letters between A and Z, all digits between 0 and 9, as well as a literal dot and hyphen. Though the hyphen normally creates a range in a character class, the hyphen is treated as a literal when it occurs as the first or last character in a character class. Recipe 2.3 tells you all about character classes, including combining them with shorthands, as in [A-Z0-9_!#$%&'*+/=?`{|}~^.-]. This class matches a word character, as well as any of the 19 listed punctuation characters.

+ and *, when used outside character classes, are quantifiers. The plus sign repeats the preceding regex token one or more times, whereas the asterisk repeats it zero or more times. In these regular expressions, the quantified token is usually a character class, and sometimes a group. Therefore, [A-Z0-9.-]+ matches one or more letters, digits, dots, and/or hyphens.

As an example of the use of a group, (?:[A-Z0-9-]+\.)+ matches one or more letters, digits, and/or hyphens, followed by one literal dot. The plus sign repeats this group one or more times. The group must match at least once, but can match as many times as possible. Recipe 2.12 explains the mechanics of the plus sign and other quantifiers in detail.

(?:) is a noncapturing group. The capturing group () does the same thing with a cleaner syntax, so you could replace (?: with ( in all of the regular expressions we’ve used so far without changing the overall match results. But since we’re not interested in separately capturing parts of the email address, the noncapturing group is somewhat more efficient, although it makes the regular expression somewhat harder to read. Recipe 2.9 tells you all about capturing and noncapturing groups.

In most regex flavors, the anchors ^ and $ force the regular expression to find its match at the start and end of the subject text, respectively. Placing the whole regular expression between these characters effectively requires the regular expression to match the entire subject.

This is important when validating user input. You do not want to accept drop database; -- joe@server.com haha! as a valid email address. Without the anchors, all the previous regular expressions will match because they find joe@server.com in the middle of the given text. See Recipe 2.5 for details about anchors. That recipe also explains why the “^ and $ match at line breaks” matching option must be off for these regular expressions.

In Ruby, the caret and dollar always match at line breaks. The regular expressions using the caret and dollar work correctly in Ruby, but only if the string you’re trying to validate contains no line breaks. If the string may contain line breaks, all the regexes using ^ and $ will match the email address in drop database; -- LFjoe@server.comLF haha!, where LF represents a line break.

To avoid this, use the anchors \A and \Z instead. These match at the start and end of the string only, regardless of any options, in all flavors discussed in this book, except JavaScript. JavaScript does not support \A and \Z at all. Recipe 2.5 explains these anchors.

Caution

The issue with ^ and $ versus \A and \Z applies to all regular expressions that validate input. There are a lot of these in this book. Although we will offer the occasional reminder, we will not constantly repeat this advice or show separate solutions for JavaScript and Ruby for each and every recipe. In many cases, we’ll show only one solution using the caret and dollar, and list Ruby as a compatible flavor. If you’re using Ruby, remember to use \A and \Z if you want to avoid matching one line in a multiline string.

Building a regex step-by-step

This recipe illustrates how you can build a regular expression step-by-step. This technique is particularly handy with an interactive regular expression tester, such as RegexBuddy.

First, load a bunch of valid and invalid sample data into the tool. In this case, that would be a list of valid email addresses and a list of invalid email addresses.

Then, write a simple regular expression that matches all the valid email addresses. Ignore the invalid addresses for now. ^\S+@\S+$ already defines the basic structure of an email address: a local part, an at sign, and a domain name.

With the basic structure of your text pattern defined, you can refine each part until your regular expression no longer matches any of the invalid data. If your regular expression only has to work with previously existing data, that can be a quick job. If your regex has to work with any user input, editing the regular expression until it is restrictive enough will be a much harder job than just getting it to match the valid data.

Variations

If you want to search for email addresses in larger bodies of text instead of checking whether the input as a whole is an email address, you cannot use the anchors ^ and $. Merely removing the anchors from the regular expression is not the right solution. If you do that with the final regex, which restricts the top-level domain to letters, it will match john@doe.com in john@doe.com77, for example. Instead of anchoring the regex match to the start and end of the subject, you have to specify that the start of the local part and the top-level domain cannot be part of longer words.

This is easily done with a pair of word boundaries. Replace both ^ and $ with \b. For instance, ^[A-Z0-9+_.-]+@[A-Z0-9.-]+$ becomes \b[A-Z0-9+_.-]+@[A-Z0-9.-]+\b.

See Also

RFC 5322 defines the structure and syntax of email messages, including the email addresses used in email messages. You can download RFC 5322 at http://www.ietf.org/html/rfc5322.txt.

Wikipedia maintains a comprehensive list of top-level domain names at http://en.wikipedia.org/wiki/List_of_Internet_top-level_domains.

Chapter 8 has a lot of solutions for working with URLs and Internet addresses.

Techniques used in the regular expressions in this recipe are discussed in Chapter 2. Recipe 2.1 explains which special characters need to be escaped. Recipe 2.3 explains character classes. Recipe 2.5 explains anchors. Recipe 2.6 explains word boundaries. Recipe 2.9 explains grouping. Recipe 2.12 explains repetition.

The best content for your career. Discover unlimited learning on demand for around $1/day.