12
\e[01;35m\e[Ks\e[m\e[K\e[01;32m\e[K:\e[m\e[K\e[01;32m\e[K360\e[m\e[K\e[01;32m\e[K:\e[m\e[K text.

When grep is using color the above is sample output captured to a file and opened in an editor. Taking that, and editing out most of the ESC[m and ESC[K seems to make no difference. What reason does grep have for going so far overboard with those codes? On the screen, colored of course, the above looks like this:

s:360: text.

Cut down to this, it looks exactly the same:

\e[1;35ms\e[32m:360:\e[m text.

The grep command I used was:

grep --color=always -siFHnI -d skip -- ' text.' somefile 
3
  • 2
    That's hard to answer without a reproducible example - i.e. please edit your question and add sample input text and the exact grep command you're using....but, generally, grep will colourise each individual match so I'd guess your regex pattern might have multiple alternations and/or your regex pattern matches multiple parts of the input line. BTW, pipe grep's output into xxd or hd to show the exact bytes in the output (you'll need to use --color=always with grep). Commented Apr 3 at 23:09
  • e.g compare the output of echo boo foo too | grep --color=always oo | hd (grep for double "o") vs echo boo foo too | grep --color=always o | hd (single "o"). With the single "o" version, each "o" is colourised individually, because each is a separate match, so there's a lot more esc codes than with the "oo" version. Commented Apr 3 at 23:10
  • Ah ... so it's covering for all possibilities. And I understand that individual matches would be individually colored, but the codes I showed were just for the coloring of the filename and line numbers. Tx. for tip about piping the output -- easier than going to an editor. Commented Apr 4 at 1:31

2 Answers 2

14

As an addition to @Stéphane Chazelas answer.

Independence from context, grep does not know how your terminal was previously formatted, so it sets colors explicitly and redundantly.

Multiple highlights for filename, line number, match, etc. each get their own color codes, so there are many switches.

Your shortened version works visually as well, but is less fault-tolerant.

grep prefers using too many codes rather than risking broken output and is intentionally overengineered so that the output remains correct under all terminal conditions, not just in the ideal case

More sources:


This is one of the best posts in my opinion that relates to "a list of all supported colors and options / ANSI escape sequence" just for your information.

1
  • 4
    That's quite clear thanks. Easy to understand that grep can't take anything for granted so it give you a belt and suspenders. Commented Apr 4 at 22:20
10

To describe those escape sequences:

  • \e[01;35m: bold + foreground-magenta
  • \e[m: reset all graphic rendition attributes to default
  • \e[K: erase up to the right edge of the screen (EL).
  • \e[01;32m: bold + foreground-green

GNU grep's highlighting is configurable with the GREP_COLORS environment variable whose default in current versions is ms=01;31:mc=01;31:sl=:cx=:fn=35:ln=32:bn=32:se=36.

Presumably, you have changed it to have fn=01;35 in there.

Each coloured section is surrounded by a start and end sequence. The start sequence is the customisable one, so for the file name (fn): \e[{fn}m\e[K and for the end sequence: \e[m to reset to default, whatever attributes the user had selected, also followed by \e[K to clear to the end of the line.

If it didn't have \e[m and you had fn=41;37 in $GREP_COLORS for instance for white foreground on a red background, the background would remain red for the line number part.

As for why the \e[K, it's explained at length in the source code:

/* Also Erase in Line (EL) to Right ("\33[K") by default.  */
/*    Why have EL to Right after SGR?
         -- The behavior of line-wrapping when at the bottom of the
            terminal screen and at the end of the current line is often
            such that a new line is introduced, entirely cleared with
            the current background color which may be different from the
            default one (see the boolean back_color_erase terminfo(5)
            capability), thus scrolling the display by one line.
            The end of this new line will stay in this background color
            even after reverting to the default background color with
            "\33[m', unless it is explicitly cleared again with "\33[K"
            (which is the behavior the user would instinctively expect
            from the whole thing).  There may be some unavoidable
            background-color flicker at the end of this new line because
            of this (when timing with the monitor's redraw is just right).
         -- The behavior of HT (tab, "\t") is usually the same as that of
            Cursor Forward Tabulation (CHT) with a default parameter
            of 1 ("\33[I"), i.e., it performs pure movement to the next
            tab stop, without any clearing of either content or screen
            attributes (including background color); try
               printf 'asdfqwerzxcv\rASDF\tZXCV\n'
            in a bash(1) shell to demonstrate this.  This is not what the
            user would instinctively expect of HT (but is ok for CHT).
            The instinctive behavior would include clearing the terminal
            cells that are skipped over by HT with blank cells in the
            current screen attributes, including background color;
            the boolean dest_tabs_magic_smso terminfo(5) capability
            indicates this saner behavior for HT, but only some rare
            terminals have it (although it also indicates a special
            glitch with standout mode in the Teleray terminal for which
            it was initially introduced).  The remedy is to add "\33K"
            after each SGR sequence, be it START (to fix the behavior
            of any HT after that before another SGR) or END (to fix the
            behavior of an HT in default background color that would
            follow a line-wrapping at the bottom of the screen in another
            background color, and to complement doing it after START).
            Piping grep's output through a pager such as less(1) avoids
            any HT problems since the pager performs tab expansion.

      Generic disadvantages of this remedy are:
         -- Some very rare terminals might support SGR but not EL (nobody
            will use "grep --color" on a terminal that does not support
            SGR in the first place).
         -- Having these extra control sequences might somewhat complicate
            the task of any program trying to parse "grep --color"
            output in order to extract structuring information from it.
      A specific disadvantage to doing it after SGR START is:
         -- Even more possible background color flicker (when timing
            with the monitor's redraw is just right), even when not at the
            bottom of the screen.
      There are no additional disadvantages specific to doing it after
      SGR END.

      It would be impractical for GNU grep to become a full-fledged
      terminal program linked against ncurses or the like, so it will
      not detect terminfo(5) capabilities.  */

However as noted by @ThomasDickey (the maintainer of xterm, vttest and ncurses (including a terminfo database of terminal behaviours)) in his xterm FAQ, while addressing the issue whereby scrolled lines are filled with the current background colour when we don't want them to, or tabulations not when we want them to, it can cause some characters to disappear from the output of grep in some circumstances in many VT100-compatible terminal emulators.

4
  • 4
    However - see FAQ Commented Apr 4 at 14:09
  • @ThomasDickey, thanks. I also came across debbugs.gnu.org/cgi/bugreport.cgi?bug=15444. So what should grep do here? Commented Apr 4 at 14:14
  • 1
    Its developers might want to reconsider that final comment about "impractical" Commented Apr 4 at 16:46
  • @TobySpeight, the @ThomasDickey formatting was intended to identify Thomas as a user here (who reacted in comment). I've reverted the edit. Commented Apr 13 at 12:40

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.