Stream Editor (sed)

Sed is a utility program and simple scripting language used for editing text. The name is an abbreviation of "stream editor," and its purpose really is the editing of input streams — typically individual lines in a file, or a single stream from stdin (standard input).

The language is not remarkably powerful. Though sed is Turing complete, it is not at all efficient for any real programming tasks — it only has two available variables, for example. It is, however, very useful for programmatically finding and editing patterns of text in files, from the command line or from a shell script.

The most important sed command is s, which is the substitution command. The command takes two arguments, plus an options flag. For example, if you wanted to correct every instance of the British-spelling "colour" to the American "color," and write the corrected version to a new file, you might do the following from the command line:

sed s/colour/color/g <old.file >new.file

A quick explanation:

  • The s/ begins the substitution command.
  • Whatever is between the first pair of slashes is the pattern to match. In this case, it's the word "colour," but it could also be a regular expression — a pattern, rather than a specific string of characters.
  • After the pattern to match is the replacement string.
  • The g means "global," or "do this every time, not just the first time."

Both the pattern to match and the substitution can be variable. If the pattern to match is a regular expression, and you want to amend it (not just replace it) you can used the ampersand ( & ) to indicate the matched pattern. For example, suppose you wanted to turn all plain-text URLs into HTML hyperlinks. You might do this:

sed 's_http[:]//[^ ]*_<a href="&">&</a>_'
  • This uses the underscore delimiter instead of the slash, to make it easier to read with all the slashes already present in URLs.
  • The pattern to match is a regular expression that matches well-formed URLs (although in a limited way).
  • The substitution creates an anchor tag with the matched pattern (the URL) in the href attribute and text of the tag.

This sort of find-and-replace was the original reason sed was invented, and continues to be the most common usage.

Sed Online Resources and Tutorials

Books on Sed

Sed vs Awk and Other Options

The sed tool is just one of many options for searching and manipulating text from the command line or in shell scripts. The most frequently compared tool is Awk, but there is also grep. Additionally, you can use more robust scripting languages such Perl or Python.

Here are a few resources to help you determine which tool is the best for your text-searching and in-place editing needs:

To use sed effectively, you need skills in two related areas: regular expressions and shell scripting in general. Here are some books and online tutorials to help you round out your sed abilities.

In addition to these resources, you should also check out our pages on Awk and Bash.

Books

  • Introducing Regular Expressions (2012) by Michael Fitzgerald is a very helpful intro text that makes this potentially intimidating topic easy to understand; it also includes sed scripting in particular, as well as info on using regex in Perl
  • Mastering Regular Expressions (2006) by Jeffrey Friedl is your next-level regex resource, after the introductory text above;
  • Classic Shell Scripting (2005) by Robbins and Beebe is a book that covers the most important Unix command line and scripting tools, including sed; Arnold Robbins, one of the authors, is also the co-author of the classic Sed & Awk book listed above.

Online Resources

There are a lot of online resources on these topics. Here's just a few of the most helpful:

  • RegExr is an online tool to learn, build, and test regular expressions;
  • RegEx101 is another online regular expression tester, with a built-in cheat sheet and helpful parsing feedback;
  • Regex Cheat Sheet provides a comprehensive reference, for when you need to know which seemingly-random string of characters you need to type to find the seemingly-random string of characters you are looking for;
  • The Grymoire, mentioned above, is a trove of information for Unix scripting;
  • Linux Shell Scripting Tutorial is one of the best online tutorials for learning Bash.

Summary

There was a time when virtually all serious computer users knew command-line tools like sed very well. For good or bad, that time is long past. Today, even many advanced developers don't know how to use these fundamental tools.

However, if you're interested in really maximizing your efficiency as a power user — especially in a Linux, Unix, or Mac environment — having a good handle on the command line is essential. This is particularly helpful if you do any web development or administration, since most of the web runs on Linux servers, and most remote-server management is done at the command line.

Sed, which is included in nearly every Linux and Unix-like operating system, can help you find what you are looking for. When you don't have the user-friendly search tools of a desktop computer, this can become essential. It isn't that hard to learn, and you'll never be sorry you know how to use this classic tool.


Further Reading and Resources

We have more guides, tutorials, and infographics related to computers:

The Ultimate List of Webmaster Tools A-Z

Sed and Awk and Bash are just part of a large collection of tools that allow system administrators to manage operating systems. The Ultimate List of Webmaster Tools A-Z will provide you with a lot of help in doing your work.