About the Duplicate Line Remover
The Duplicate Line Remover keeps only the first occurrence of every line and deletes the repeats. Clean up email lists, keyword lists, log lines or any list copied from several sources.
Order is preserved, so the result reads the same as the original minus the repeats. A checkbox lets you ignore upper and lower case, so "Apple" and "apple" count as the same line.
How to use it
- Paste your list into Input, one item per line.
- Tick the option to ignore case if you want.
- Press Remove duplicates and copy the result.
Details
Lines must match exactly, including spaces, unless you ignore case; trailing spaces make lines different.
Blank lines are also treated as lines, so only the first blank line is kept.
Need alphabetical order? Paste the result into a spreadsheet and sort.
Real-world examples
Mailing lists. Remove repeated email addresses before importing.
Keyword research. Merge keyword lists from several tools and keep one of each.
Log files. Reduce repeated messages to see the different ones.
Common mistakes
A trailing space makes two lines different, so tidy whitespace first. Case differences count unless you tick ignore case. Remember the first copy is kept, so if order matters to you the result stays in your original sequence.
How it works
Lines are read in order and stored in a set. If a line (or its lowercase form, when ignoring case) is already in the set, it is skipped; otherwise it is kept and added. A set checks membership very quickly, so even long lists finish instantly.
Frequently asked questions
Does it keep the first or last copy?
The first.
Is there a size limit?
Only your device's memory.
Is the list uploaded?
No.