How do I remove duplicate consecutive lines using a substitute command?

Explanation

This substitute command detects pairs of identical adjacent lines and collapses them into one, using a back-reference to match the repeated content. It's a powerful complement to :sort u (which only deduplicates after sorting) because it removes duplicates in-place without reordering.

How it works

• ^ — anchors to the start of a line
• \(.\+\) — captures one or more characters (the line content) into group \1
• \n — matches the literal newline between the two lines
• \1 — requires the next line to be identical to the captured group
• $ — anchors to the end of the second line
• The replacement \1 keeps only the first of the two identical lines

Example

Before:
apple
apple
banana
cherry
cherry

After :%s/^\(.\+\)\n\1$/\1/ :
apple
banana
cherry

For three or more consecutive duplicates in a row, run the command again (@:) until the count reported reaches zero.

Tips

• To delete duplicates globally after sorting, use :sort u instead — it's simpler when order doesn't matter
• This pattern only matches lines with at least one character — blank lines are unaffected
• For visual range: :'<,'>s/^\(.\+\)\n\1$/\1/
• The related command :g/\(.\+\)\n\1/d uses :global to delete matching lines, but it leaves the second copy of the line instead of the first