diff --git a/docs/guides/quick-answers/linux/how-to-use-grep/index.md b/docs/guides/quick-answers/linux/how-to-use-grep/index.md deleted file mode 100644 index 20c058aebbe..00000000000 --- a/docs/guides/quick-answers/linux/how-to-use-grep/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -slug: how-to-use-grep -title: "Using the grep Command in Linux: Finding Text & Strings in Files" -description: "Use the grep command to filter output from administrative commands, so you get only relevant data. This guide shows you how. ✓ Click to learn today!" -authors: ["Phil Zona"] -contributors: ["Phil Zona"] -published: 2017-04-10 -modified: 2017-11-27 -keywords: ["linux", "how to", "grep", "filter"] -aliases: ['/quick-answers/linux/how-to-use-grep/','/quick-answers/how-to-use-grep/'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -tags: ["linux"] ---- - -
- -In this guide, you'll learn how to use the `grep` command. When performing administrative tasks on your Linode, many commands will give you more information than you need. Using `grep` allows you to filter that output in order to find only the data that's relevant. - -1. To search a file for a particular string, provide the string and filename as arguments: - - grep 'some text' /etc/ssh/sshd_config - -2. You may also redirect output from a command to `grep` using a [pipe](http://man7.org/linux/man-pages/man2/pipe.2.html): - - tail -f /var/log/apache/error.log | grep 'some text' - - This will monitor your Apache error logs, and display only the lines of output that contain the given string. - -3. Regex patterns are also supported by the `-E` option if you want to search for a set of strings rather than one literal: - - grep -E "[[:alpha:]]{16,20}" /etc/ssh/sshd_config - - This example will search the `/etc/ssh/sshd_config` file for strings of alphabetic characters that are 16-20 characters long, but you can use any regex pattern you like. - -These are simply a few basic ways to use `grep`. Many other options exist, and in combination with other tools, it serves as an invaluable utility for performing administrative tasks on your Linode. For more information on some of `grep`'s advanced features, check out our guide on how to [search and filter text with grep](/docs/guides/how-to-use-grep-command/). diff --git a/docs/guides/tools-reference/tools/how-to-use-grep-command/index.md b/docs/guides/tools-reference/tools/how-to-use-grep-command/index.md deleted file mode 100644 index 0a05c4f3622..00000000000 --- a/docs/guides/tools-reference/tools/how-to-use-grep-command/index.md +++ /dev/null @@ -1,228 +0,0 @@ ---- -slug: how-to-use-grep-command -title: How to Use the Grep Command to Find Information in Files -description: "An extensive guide on how to grep for text in files in Unix based systems. Understand search, match, and advanced regex to grep for text in files." -authors: ["Linode"] -contributors: ["Linode"] -published: 2010-06-30 -modified: 2023-11-14 -keywords: ["how to use grep", "grep usage", "grep tutorial"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Grep | Regular-Expressions.info](https://www.regular-expressions.info/grep.html)' - - '[Perl Regular Expressions](https://perldoc.perl.org/perlre.html)' -aliases: ['/tools-reference/how-to-grep-for-text-in-files/','/tools-reference/search-and-filter-text-with-grep/','/tools-reference/tools/how-to-grep-for-text-in-files/','/linux-tools/common-commands/grep/','/guides/how-to-grep-for-text-in-files/'] -tags: ["linux"] -image: search_and_filter_text_with_grep_smg.png ---- - -Grep is a command-line utility that can search and filter text using a common regular expression syntax. It is so ubiquitous that the verb "to grep" has emerged as a synonym for "to search". The `grep` command is a useful tool for finding all occurrences of a search term in a selection of files or the output of another command. It can also come in handy when filtering a log file or stream and when used as part of a script or chain of commands. - -This tutorial provides an overview of how to use `grep`, a brief introduction to regular expression syntax, and practical examples. - -## How to Use Grep - -This guide references recent versions of GNU grep, which are included by default in all images provided by Akamai. It is also provided as part of the common base selection of packages provided in nearly all Linux distributions. - -### The Grep Command - -A basic `grep` command uses the following syntax: - -```command -grep "string" ~/example.txt -``` - -The first argument to `grep` is a search pattern. The second (optional) argument is the name of a file to be searched. The above sequence searches for all occurrences of the word "string" in the `~/example.txt` file. - -You can use `grep` to search a single file or multiple files. If you want to search files in a directory, include the `-r` flag. It enables recursive searching through a directory tree, including subdirectories: - -```command -grep -r "string" ~/example/ -``` - -When used on a specific file, `grep` only outputs the lines that contain the matching string. In recursive mode, `grep` outputs the full path to the file, followed by a colon, and the contents of the line that matches the pattern. By default, patterns in `grep` are basic regular expressions. If you need a more expressive regular expression syntax, `grep` is capable of accepting patterns in alternate formats with the following flags: - -| Flag | Usage | -| -- | -- | -| `-E` | Use extended regular expression syntax. Equivalent to the deprecated `egrep` command. | -| `-P` | Use Perl regular expression syntax. | - -Grep provides a number of powerful options to control its output: - -| Flag | Usage | -| -- | -- | -| `-o` | Output only the matching segment of each line, rather than the full contents of each matched line. | -| `-i` | Ignore case distinctions, so that characters only differing in case still match. | -| `-n` | Print the line number of each matched line. | -| `-C 2` | Show 2 (or any number of) adjacent lines in addition to the matched line. | -| `-v` | Invert the sense of matching, to select non-matching lines. | -| `-e` | Specify a pattern. If this option is used multiple times, search for all patterns given. This option can be used to protect a pattern beginning with `-`. | - -### Piping Command Outputs to grep - -In addition to reading content from files, `grep` can read and filter text from standard input. The output of any command or stream can be piped to the `grep` command. Then `grep` then filters this output according to the match pattern specified and outputs only the matching lines. For example: - -```command -ls --help | grep "dired" -``` - -This filters the output of the `ls` command's help text and looks for appearances of "dired", and outputs them to standard out: - -```output - -D, --dired generate output designed for Emacs' dired mode -``` - -### Regular Expression Overview - -While straightforward pattern matching is sufficient for some filtering tasks, the true power of `grep` is its ability to use regular expressions for complex pattern matching. Most characters in regular expressions match with input data literally. However, there are some sequences that carry special significance: - -| Symbol | Result | -|--|--| -| `.` | Matches any character. | -| `*` | Matches zero or more instances of the preceding character. | -| `+` | Matches one or more instances of the preceding character. | -| `[]` | Matches any of the characters within the brackets. | -| `()` | Creates a sub-expression that can be combined to make more complicated expressions. | -| `\|` | The **OR** operator. For Example, (www\|ftp) matches either "www" or "ftp". | -| `^` | Matches the beginning of a line. | -| `$` | Matches the end of the line. | -| `\\` | Escapes the following character. Since `.` matches any character, to match a literal period you would need to use `\.`. | - -## Filtering Logs with Grep - -Another popular use of `grep` is to find text files generated by a system, such as logs: - -```command -grep -Eoc "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200" /srv/www/example.com/logs/access.log -``` - -Here, `grep` filters an Apache access log for lines beginning with an IP address, followed by a number of characters, a space, and `200` (representing a successful HTTP connection). The `-c` option only outputs the number of matches. To get the output of the IP address of the visitor and the path of the requested file for successful requests, omit the `-c` flag: - -```command -grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200" /srv/www/example.com/logs/access.log -``` - -The curly brackets specify the number of instances of the pattern. `{1,3}` requires that the previous character occur at least once, but no more than three times. The character class `[0-9]` matches against one or more numeric digits. You can also generate similar output that reports on unsuccessful attempts to access content by searching for `404` instead of `200`: - -```command -grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 404" /srv/www/example.com/logs/access.log -``` - -The following command generates a list of all IP addresses that have attempted to connect to your web server. Using the `-o` option, only the matching strings are sent to standard output. This output is filtered through the utility `uniq` with the *pipe* operator (`|`) to filter out duplicate entries: - -```command -grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" /srv/www/example.com/logs/access.log | uniq -``` - -The next example uses an alternative pattern for matching an IP address in a different log. The following command searches the most recent `/var/log/auth.log` file for invalid login attempts: - -```command -grep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log -``` - -You can split the above command into two layers to output a list of IP addresses with failed login attempts to your system: - -```command -grep "Invalid user" /var/log/auth.log | grep -Eo "([0-9]{1,3}\.){3}[0-9]{1,3}" | uniq -``` - -`grep` can filter the output of commands such as `tail -F` to provide real-time monitoring of specific log events: - -```command -tail ~/.procmail/procmail.log -F | grep "Subject" -``` - -In this case, `tail` follows the `~/.procmail/procmail.log` file. This output is passed to `grep`, which filters the stream and prints only lines that contain the string "Subject". - -## Filtering Commands with Grep - -`grep` can be used to filter long help files. This command filters the `tar` help text to more efficiently find the options for dealing with `bzip` files: - -```command -tar --help | grep "bzip" -``` - -`grep` is also useful for filtering the output of `ls` when listing the contents of directories with a large number of files: - -```command -ls /usr/lib | grep "xml" -``` - -## Excluding Patterns with grep - -You can also use `grep` to return non-matching lines by using the `-v` flag to perform an invert search. For example, the following command only returns lines that do *not* contain the pattern "string": - -```command -grep -v "string" ~/threads.txt -``` - -You can also exclude multiple search patterns using invert search with `grep -v` by using the `-e` flag before each pattern as follows: - -```command -grep -v -e "string" -e "yarn" ~/threads.txt -``` - -When you run the above command, it outputs all lines that do not contain "string" or "yarn". - -## Excluding grep When Using ps - -A useful application of invert searching is to exclude "grep" itself from a `grep` output, such as when using the `ps` command. For example, use the following command to search for all current processes that contain the pattern "log": - -```command -ps ax | grep log -``` - -```output -576 ? Ss 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only -583 ? Ssl 0:00 /usr/sbin/rsyslogd -n -iNONE -592 ? Ss 0:00 /lib/systemd/systemd-logind -4967 pts/0 S+ 0:00 grep --color=auto log -``` - -Notice the last line of the output contains `grep log`, which is not relevant to the purpose of the search. You can exclude this line by using a pipe operator `|` and adding `grep -v grep` after it as follows: - -```command -ps ax | grep log| grep -v grep -``` - -```output -576 ? Ss 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only -583 ? Ssl 0:00 /usr/sbin/rsyslogd -n -iNONE -592 ? Ss 0:00 /lib/systemd/systemd-logind -``` - -While using `grep -v grep` not only excludes the `grep log` line, it also skips all other lines that contain "grep", which may not be ideal. Luckily, there are other ways to remove `grep` from your output. - -You can modify the `grep log` command and add a character class match to specifically match the first character `l` from log. This works because the excluded line contains `grep [l]og`, while only matches for "log" are returned. Run the command below to do a character class match for `l` as a character: - -```command -ps ax | grep '[l]og' -``` - -```output -576 ? Ss 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only -583 ? Ssl 0:00 /usr/sbin/rsyslogd -n -iNONE -592 ? Ss 0:00 /lib/systemd/systemd-logind -``` - -Alternatively, you can use the `pgrep` command instead of `grep` to search your current processes: - -```command -pgrep -af log -``` - -```output -576 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only -583 /usr/sbin/rsyslogd -n -iNONE -592 /lib/systemd/systemd-logind -``` - -## Grep Compressed Files with zgrep - -The `zgrep` command functions identically to the `grep` command above. However, it adds the ability to run `grep` operations on files that have been compressed with gzip without requiring decompression. To search an older compressed log: - -```command -zgrep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log.2.gz -``` - -`zgrep` operations take longer than standard `grep` operations because of the additional overhead of reading the compressed files. \ No newline at end of file diff --git a/docs/guides/tools-reference/tools/how-to-use-grep-command/search_and_filter_text_with_grep_smg.png b/docs/guides/tools-reference/tools/how-to-use-grep-command/search_and_filter_text_with_grep_smg.png deleted file mode 100644 index f335fbda6b7..00000000000 Binary files a/docs/guides/tools-reference/tools/how-to-use-grep-command/search_and_filter_text_with_grep_smg.png and /dev/null differ diff --git a/docs/guides/tools-reference/tools/how-to-use-grep/index.md b/docs/guides/tools-reference/tools/how-to-use-grep/index.md new file mode 100644 index 00000000000..56321508e36 --- /dev/null +++ b/docs/guides/tools-reference/tools/how-to-use-grep/index.md @@ -0,0 +1,217 @@ +--- +slug: how-to-use-grep +title: How to use grep +description: "Learn how to use grep to search files, filter command output, and match text patterns with regular expressions on Linux systems." +authors: ["Akamai"] +contributors: ["Akamai", "Adam Overa"] +published: 2010-06-30 +modified: 2026-06-05 +keywords: ["grep", "linux grep", "grep command", "regular expressions", "text search"] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +external_resources: + - '[Grep | Regular-Expressions.info](https://www.regular-expressions.info/grep.html)' + - '[Perl Regular Expressions](https://perldoc.perl.org/perlre.html)' +aliases: ['/tools-reference/how-to-grep-for-text-in-files/','/tools-reference/search-and-filter-text-with-grep/','/tools-reference/tools/how-to-grep-for-text-in-files/','/linux-tools/common-commands/grep/','/guides/how-to-grep-for-text-in-files/','/quick-answers/linux/how-to-use-grep/','/quick-answers/how-to-use-grep/','/guides/how-to-use-grep-command/'] +tags: ["linux"] +--- + +`grep` is a command-line utility for searching and filtering text in files, command output, and log streams using pattern matching and regular expressions. + +This guide provides an overview of `grep`, a brief introduction to regular expression syntax, and practical examples. + +## Basic `grep` command syntax + +A basic `grep` command uses the following syntax: + +```command +grep "string" ~/example.txt +``` + +The first argument to `grep` is a search pattern. The second (optional) argument is the name of a file to search. The example above searches `~/example.txt` for lines containing the word "string". + +You can use `grep` to search a single file or multiple files. To search files in a directory, include the `-r` flag. It enables recursive searching through a directory tree, including subdirectories: + +```command +grep -r "string" ~/example/ +``` + +When used on a specific file, `grep` only outputs the lines that contain the matching string. In recursive mode, `grep` outputs the full path to the file, followed by a colon, and the contents of the line that matches the pattern. + +`grep` also provides a number of options to control its output: + +| Flag | Usage | +| -- | -- | +| `-o` | Output only the matching segment of each line, rather than the full contents of each matched line. | +| `-i` | Ignore case distinctions, so that characters only differing in case still match. | +| `-n` | Print the line number of each matched line. | +| `-C 2` | Show 2 (or any number of) adjacent lines in addition to the matched line. | +| `-v` | Invert the matching logic, to print non-matching lines. | +| `-e` | Specify a pattern. If this option is used multiple times, search for all patterns given. This option can be used to protect a pattern beginning with `-`. | + +### Regular expressions + +By default, `grep` uses basic regular expressions (BRE). You can also use extended regular expressions (ERE) or Perl-compatible regular expressions (PCRE) with the following flags: + +| Flag | Usage | +| -- | -- | +| `-E` | Use extended regular expression syntax. Replaces the deprecated `egrep` command. | +| `-P` | Use Perl-compatible regular expression (PCRE) syntax. Support for this option depends on the `grep` implementation available on your system. | + +The following examples use extended regular expression syntax (`grep -E`). While most characters in a regular expression match literal text, the following characters have special meaning: + +| Symbol | Result | +|--|--| +| `.` | Matches any character. | +| `*` | Matches zero or more instances of the preceding character. | +| `+` | Matches one or more instances of the preceding character. | +| `[]` | Matches any of the characters within the brackets. | +| `()` | Creates a sub-expression for grouping patterns. | +| `|` | The **OR** operator. For example, `(www|ftp)` matches either `www` or `ftp`. | +| `^` | Matches the beginning of a line. | +| `$` | Matches the end of the line. | +| `\\` | Escapes the following character. Since `.` matches any character, to match a literal period you would need to use `\.`. | + +## Filtering logs + +A common use of `grep` is searching system-generated text files such as logs: + +```command +grep -Eoc "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200" /srv/www/example.com/logs/access.log +``` + +Here, `grep` filters an Apache access log for lines beginning with an IP address, followed by a number of characters, a space, and `200` (representing a successful HTTP connection). The `-c` option only outputs the number of matches. To get the output of the IP address of the visitor and the path of the requested file for successful requests, omit the `-c` flag: + +```command +grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200" /srv/www/example.com/logs/access.log +``` + +The curly brackets specify the number of instances of the pattern. `{1,3}` requires that the previous character occur at least once, but no more than three times. The character class `[0-9]` matches a single numeric digit. Combined with `{1,3}`, it matches between one and three digits. You can also generate similar output that reports on unsuccessful attempts to access content by searching for `404` instead of `200`: + +```command +grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 404" /srv/www/example.com/logs/access.log +``` + +The following command generates a list of unique IP addresses found in the access log. Using the `-o` option, only the matching strings are sent to standard output. The results are sorted and deduplicated using `sort -u`: + +```command +grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" /srv/www/example.com/logs/access.log | sort -u +``` + +The next example searches authentication logs for invalid user login attempts. On Debian-based systems, these events are typically recorded in `/var/log/auth.log`. On RHEL-based systems, similar authentication events are usually logged to `/var/log/secure`: + +```command +grep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log +``` + +To output a unique list of IP addresses associated with failed login attempts, match only the IP address portion and sort the results: + +```command +grep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log | grep -Eo "([0-9]{1,3}\.){3}[0-9]{1,3}" | sort -u +``` + +`grep` can filter live command output to monitor specific events in real time: + +```command +journalctl -f | grep ssh +``` + +In this example, `journalctl -f` follows the system journal in real time, while `grep` filters the stream to show only lines related to SSH activity. + +## Filtering command output + +In addition to reading content from files, `grep` can read and filter text from standard input. You can pipe command output or other text streams to `grep`, which filters the incoming text according to the specified match pattern and prints only matching lines. For example: + +```command +ls --help | grep "dired" +``` + +This filters the output of the `ls` command's help text and prints lines containing "dired": + +```output + -D, --dired generate output designed for Emacs' dired mode +``` + +`grep` can be used to filter long help files. This command filters the `tar` help text to display options related to `bzip` files: + +```command +tar --help | grep "bzip" +``` + +## Excluding patterns + +You can also use `grep` to return non-matching lines by using the `-v` flag to perform an invert search. For example, the following command only returns lines that do *not* contain the pattern "string": + +```command +grep -v "string" ~/threads.txt +``` + +You can also exclude multiple search patterns using invert search with `grep -v` by using the `-e` flag before each pattern as follows: + +```command +grep -v -e "string" -e "yarn" ~/threads.txt +``` + +When you run the above command, it outputs all lines that do not contain "string" or "yarn". + +### Excluding `grep` when using `ps` + +For process searches, `pgrep` is usually a cleaner option than filtering `ps` output manually: + +```command +pgrep -af log +``` + +```output +576 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only +583 /usr/sbin/rsyslogd -n -iNONE +592 /lib/systemd/systemd-logind +``` + +Older `ps | grep` patterns are still common and useful to understand. For example, the following command searches for running processes that contain the pattern "log": + +```command +ps ax | grep log +``` + +```output +576 ? Ss 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only +583 ? Ssl 0:00 /usr/sbin/rsyslogd -n -iNONE +592 ? Ss 0:00 /lib/systemd/systemd-logind +4967 pts/0 S+ 0:00 grep --color=auto log +``` + +Notice the last line of the output contains `grep log`, which is not relevant to the purpose of the search. You can exclude this line by using a pipe operator (`|`) and adding `grep -v grep` after it as follows: + +```command +ps ax | grep log | grep -v grep +``` + +```output +576 ? Ss 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only +583 ? Ssl 0:00 /usr/sbin/rsyslogd -n -iNONE +592 ? Ss 0:00 /lib/systemd/systemd-logind +``` + +While `grep -v grep` excludes the `grep log` line, it also excludes any other lines containing the word "grep", which may not be ideal. + +Another approach uses a character class match to avoid matching the `grep` process itself: + +```command +ps ax | grep '[l]og' +``` + +```output +576 ? Ss 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only +583 ? Ssl 0:00 /usr/sbin/rsyslogd -n -iNONE +592 ? Ss 0:00 /lib/systemd/systemd-logind +``` + +## Search compressed files with `zgrep` + +On systems with `gzip` installed, the `zgrep` command provides `grep`-like searching for files compressed with `gzip`. For example, to search an older compressed log: + +```command +zgrep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log.2.gz +``` + +`zgrep` operations take longer than standard `grep` operations because of the additional overhead of reading the compressed files. \ No newline at end of file