Finding Duplicates & Data Quality SQL Topic exerciseMediumVerified answerSQLite + PostgreSQL + MySQL + SQL Server live · 1 guided

Return Every Row in a Normalized Duplicate Group

Use a correlated EXISTS check to return every record with another record sharing its normalized email.

  • Subqueries
  • String functions
  • Filtering
  • Sorting

Exercise brief

Understand the request

Data remediation analyst The review queue needs row-level evidence for every normalized email collision, not just one aggregate per email.

The review queue needs row-level evidence for every normalized email collision, not just one aggregate per email. Use a correlated EXISTS check to return every record with another record sharing its normalized email.

Return

  • Return record_id, customer_key, email, and normalized_email.
  • Order by normalized_email, then record_id.

Constraints

  • Use EXISTS and compare against a different record_id.
  • Exclude NULL and whitespace-only emails.

Data you will use

Review the relevant tables before deciding how to join, filter, or aggregate them.

quality_records

  • record_idINTEGER
  • customer_keyTEXT
  • emailTEXT

Hints, when you need them

Open one clue at a time so you still do the reasoning.

Hint 1

EXISTS answers whether at least one matching peer is present without producing peer pairs.

Hint 2

Correlate on the normalized email and exclude the current record_id.

Hint 3

Project the canonical key and order by it plus record_id.

Verified SQL answer

Attempt the problem first, then compare structure and reasoning—not just syntax.

Reveal solution and explanation
SELECT q.record_id, q.customer_key, q.email, LOWER(TRIM(q.email)) AS normalized_email FROM quality_records q WHERE q.email IS NOT NULL AND TRIM(q.email) <> '' AND EXISTS (SELECT 1 FROM quality_records other WHERE other.record_id <> q.record_id AND other.email IS NOT NULL AND TRIM(other.email) <> '' AND LOWER(TRIM(other.email)) = LOWER(TRIM(q.email))) ORDER BY normalized_email, q.record_id;

Why this works

A correlated semi-join returns the affected source rows at original grain. Unlike a self-join, it does not create n×(n−1) pairs.

Success check

All seven colliding rows appear once each without self-matches or join multiplication.

Expected result

Use this output to verify values, aliases, ordering, and row count.

record_idcustomer_keyemailnormalized_email
1001C100 Alice@example.com alice@example.com
1002C100alice@example.comalice@example.com
1003C100ALICE@example.comalice@example.com
1004C200bob@example.combob@example.com
1005C200bob@example.combob@example.com
1009C500dave@example.comdave@example.com
1010C500dave@example.comdave@example.com

Learn the concepts behind this answer

Strengthen your understanding with these targeted learning topics:

Continue practicing

SQL Practice Online

Open the interactive workspace and practice across SQL topics.