← Data structures & algorithms
Lookup Membership, once each

Hash set

Have I seen this one before?

Every new Set() you have written to drop duplicates is answering one question, and so is the check React runs in development when two list items share a key: is this value already here? Let’s watch a birder’s life list answer it, and find out why deleting a species is harder than it looks.

TypeScriptGoOne life list, two implementations.

01 / The idea

Is it already here?

A birder keeps a life list: every species they have ever seen, each counted once. The first Northern Cardinal is a lifer. The two-hundredth is just a nice morning. So every sighting starts with the same question, have I logged this species before, and the answer has to come back straight away, whether the list holds ten species or seven hundred.

You ask that question in code all the time. [...new Set(tags)] drops repeated tags. selected.has(row.id) decides whether a checkbox is ticked. React, in development, adds each list key to a Set and warns when one is already there. Svelte collects a keyed each block’s keys in a Set and throws when there are fewer keys than items. Every time, something answers “is this here?” without looking at everything.

A hash set is that something. It stores values, never twice, and uses a hash of each value to decide where to look. The Hash map lesson kept a short list in every bucket. This one keeps every value in a single array of slots and walks forward when a slot is taken. That is open addressing, the family Go’s built-in map has belonged to since Go 1.24, and it makes one operation surprisingly subtle: deleting.

02 / Name the rule

Hash to a slot, walk forward, and never leave a hole.

The set is an array of slots, eight to start. To find a value, hash it. With eight slots, the hash’s last three bits pick where the walk starts. If that slot holds the value, it is here. If the slot is empty, it is not. If the slot holds some other value, step to the next slot and ask again, wrapping from the last slot to slot 0. This is linear probing.

Adding runs the same walk. When it reaches an empty slot, the value is not in the set, so it goes there. Two values that start at the same slot end up next to each other.

Deleting is where the rule bites. Empty a slot, and every value that walked past it on the way in is cut off from its start: the next walk reaches the hole and stops. So a deleted slot becomes a tombstone. Walks step over it, and the next add can reuse it.

The invariant: every value sits on the walk from its start slot with no empty slot in between, and values plus tombstones fill at most half the slots. That second part means every walk finds an empty slot, and keeps the walks short. When an add would break it, the table is rebuilt first: twice as big if the values need the room, or the same size with its tombstones cleared.

Why do these three codes collide?Only the low bits pick a slot

FNV-1a turns AMRO into 0x20351314, MODO into 0xdc28fc84, and SOSP into 0xff7cea54. The hashes are nothing alike, but with eight slots only the last three bits matter, and all three end in 100: slot 4. BCCH hashes to 0xfc0dfcb1, which ends in 001: slot 1.

A bigger table uses more bits. After the table grows to sixteen slots, codes that shared a start slot can land in different ones, which is how a rebuild breaks a cluster apart. NOCA (0xb335acce) and BLJA (0x586e8636) both start at slot 6 with eight slots; with sixteen, NOCA moves to slot 14. The robin, dove, and sparrow codes above all end in 0100, so they still share slot 4 at sixteen.

03 / Follow one operation

A robin that wasn’t, and a dove that stays counted once.

The list starts with three species. American Robin and Mourning Dove both hash to slot 4, so the dove walked on to slot 5. Black-capped Chickadee landed in slot 1. Then a second Mourning Dove turns up, the robin turns out to be a misidentification, and a Song Sparrow arrives.

Before you watch, predict which slot the Song Sparrow ends up in. The animation replays each recorded hash, probe, and tombstone. Try it lets you log, remove, and check species yourself, and grow the table.

Hash set

Hash, walk, and leave a tombstone.

Hash a code to choose where its walk starts.

8 slots

  1. 0 ·
  2. 1 BCCH
  3. 2 ·
  4. 3 ·
  5. 4 AMRO
  6. 5 MODO
  7. 6 ·
  8. 7 ·

3 species · 0 tombstones · 8 slots, at most 4 in use

01/ 05
The setup

Three species in eight slots.

American Robin and Mourning Dove both hash to slot 4. The dove found slot 4 taken and walked on to slot 5. Black-capped Chickadee went straight into slot 1.

Reduced motion: choose a scene to see its completed state.

Read this scene

American Robin and Mourning Dove both hash to slot 4. The dove found slot 4 taken and walked on to slot 5. Black-capped Chickadee went straight into slot 1.

American Robin and Mourning Dove both hash to slot 4. The dove found slot 4 taken and walked on to slot 5. Black-capped Chickadee went straight into slot 1.

3 species, 0 tombstones.

  • Slot 0: empty
  • Slot 1: BCCH, Black-capped Chickadee
  • Slot 2: empty
  • Slot 3: empty
  • Slot 4: AMRO, American Robin
  • Slot 5: MODO, Mourning Dove
  • Slot 6: empty
  • Slot 7: empty

Watch restarts when you return. Step through keeps your selected step. Try it starts a fresh life list each time you open it.

04 / Read the shape

The set answers “is it here?” The life list decides what counts.

Basic form is the string set alone: add, has, delete, the walk, tombstones, and rebuilds. In the wild wraps it in LifeList, which owns a birder’s rules. A code must be on the regional checklist, which is a second set. A repeat sighting changes nothing, and a misidentified sighting comes off. Still to find is the checklist minus the life list, one lookup per species.

Notice that the set never sees a species name, only a four-letter code. The Institute for Bird Populations publishes these alpha codes, so AMRO means American Robin on every list, and the set can compare codes exactly without normalizing names.

A string set with open addressing. FNV-1a picks a start slot, the walk steps past other codes and tombstones, deleting leaves a tombstone, and the table rebuilds before codes and tombstones pass half the slots. The optional trace records every probe.

TypeScriptReading
lifelist.ts
// FNV-1a over the value's UTF-8 bytes. Simple and identical in every language,
// so a value lands in the same slot in TypeScript and Go.
export function hashKey(value: string): number {
	let hash = 2166136261;
	for (const byte of new TextEncoder().encode(value)) {
		hash ^= byte;
		hash = Math.imul(hash, 16777619) >>> 0;
	}
	return hash;
}

export type Step = {
	kind: 'hash' | 'skip' | 'compare' | 'match' | 'empty' | 'insert' | 'delete' | 'rebuild' | 'move';
	value: string;
	/** The slot looked at; the old capacity for rebuild; the old slot for move. */
	slot: number;
	/** The full hash for hash; the new capacity for rebuild; the new slot for move;
	 * 1 when insert reused a tombstone, 0 when it took an empty slot; otherwise -1. */
	detail: number;
};
export type Slot = { state: 'empty' } | { state: 'tombstone' } | { state: 'live'; value: string };

const TOMBSTONE = Symbol('tombstone');
type Cell = string | typeof TOMBSTONE | undefined;
type Found = { found: number; tombstone: number; empty: number };

// Open addressing: every value lives in the slots array itself. A value starts at
// its hash's slot and walks forward until it finds itself or an empty slot.
// Deleting leaves a tombstone, so walks that pass through that slot keep going.
// Live values plus tombstones never fill more than half the slots.
export class StringSet {
	#cells: Cell[] = new Array(8).fill(undefined);
	#size = 0;
	#tombstones = 0;
	#probes = 0;
	#steps: Step[] = [];
	#capture: boolean;

	constructor(capture = false) {
		this.#capture = capture;
	}
	get size(): number {
		return this.#size;
	}
	get capacity(): number {
		return this.#cells.length;
	}
	get tombstones(): number {
		return this.#tombstones;
	}
	/** Slots looked at by every has, add, and delete so far. */
	get probes(): number {
		return this.#probes;
	}

	has(value: string): boolean {
		this.#steps = [];
		return this.#find(value).found >= 0;
	}

	add(value: string): 'added' | 'present' {
		this.#steps = [];
		let place = this.#find(value);
		if (place.found >= 0) return 'present';
		// Reusing a tombstone takes no new room. Taking an empty slot might.
		if (place.tombstone < 0 && (this.#size + this.#tombstones + 1) * 2 > this.capacity) {
			this.#rebuild();
			place = this.#find(value);
		}
		const reused = place.tombstone >= 0;
		const slot = reused ? place.tombstone : place.empty;
		this.#cells[slot] = value;
		this.#size++;
		if (reused) this.#tombstones--;
		this.#record('insert', value, slot, reused ? 1 : 0);
		return 'added';
	}

	delete(value: string): 'deleted' | 'missing' {
		this.#steps = [];
		const { found } = this.#find(value);
		if (found < 0) return 'missing';
		this.#cells[found] = TOMBSTONE;
		this.#size--;
		this.#tombstones++;
		this.#record('delete', value, found, -1);
		return 'deleted';
	}

	// Slot order: neither insertion order nor sorted.
	values(): string[] {
		return this.#cells.filter((cell): cell is string => typeof cell === 'string');
	}

	slots(): Slot[] {
		return this.#cells.map((cell) =>
			cell === undefined
				? { state: 'empty' }
				: cell === TOMBSTONE
					? { state: 'tombstone' }
					: { state: 'live', value: cell }
		);
	}

	trace(): Step[] {
		return this.#steps.map((step) => ({ ...step }));
	}

	#find(value: string): Found {
		const hash = hashKey(value);
		const mask = this.capacity - 1;
		const start = hash & mask;
		this.#record('hash', value, start, hash);
		let tombstone = -1;
		for (let i = 0; i < this.capacity; i++) {
			const slot = (start + i) & mask;
			const cell = this.#cells[slot];
			this.#probes++;
			if (cell === undefined) {
				this.#record('empty', value, slot, -1);
				return { found: -1, tombstone, empty: slot };
			}
			if (cell === TOMBSTONE) {
				this.#record('skip', value, slot, -1);
				if (tombstone < 0) tombstone = slot;
				continue;
			}
			if (cell === value) {
				this.#record('match', value, slot, -1);
				return { found: slot, tombstone, empty: -1 };
			}
			this.#record('compare', value, slot, -1);
		}
		throw new Error('No empty slot: the table is never more than half full.');
	}

	// Double if the live values alone would pass half; otherwise just clear the tombstones.
	#rebuild(): void {
		const old = this.#cells;
		const capacity = (this.#size + 1) * 2 > old.length ? old.length * 2 : old.length;
		this.#record('rebuild', '', old.length, capacity);
		this.#cells = new Array(capacity).fill(undefined);
		this.#tombstones = 0;
		const mask = capacity - 1;
		old.forEach((cell, from) => {
			if (typeof cell !== 'string') return;
			let to = hashKey(cell) & mask;
			while (this.#cells[to] !== undefined) to = (to + 1) & mask;
			this.#cells[to] = cell;
			this.#record('move', cell, from, to);
		});
	}

	#record(kind: Step['kind'], value: string, slot: number, detail: number): void {
		if (this.#capture) this.#steps.push({ kind, value, slot, detail });
	}
}
GoAlongside
lifelist.go
// HashKey is FNV-1a over the value's UTF-8 bytes. Simple and identical in every
// language, so a value lands in the same slot in Go and TypeScript.
func HashKey(value string) uint32 {
	hash := uint32(2166136261)
	for i := 0; i < len(value); i++ {
		hash ^= uint32(value[i])
		hash *= 16777619
	}
	return hash
}

type Step struct {
	Kind  string `json:"kind"`
	Value string `json:"value"`
	// The slot looked at; the old capacity for rebuild; the old slot for move.
	Slot int `json:"slot"`
	// The full hash for hash; the new capacity for rebuild; the new slot for move;
	// 1 when insert reused a tombstone, 0 when it took an empty slot; otherwise -1.
	Detail int64 `json:"detail"`
}

type cellState uint8

const (
	empty cellState = iota
	tombstone
	live
)

type cell struct {
	state cellState
	value string
}

// Slot is an inspection copy of one cell: State is "empty", "tombstone", or "live".
type Slot struct {
	State string
	Value string
}

// StringSet uses open addressing: every value lives in the cells slice itself. A
// value starts at its hash's slot and walks forward until it finds itself or an
// empty slot. Deleting leaves a tombstone, so walks through that slot keep going.
// Live values plus tombstones never fill more than half the cells.
type StringSet struct {
	cells      []cell
	size       int
	tombstones int
	probes     int
	steps      []Step
	capture    bool
}

func NewStringSet(capture bool) *StringSet {
	return &StringSet{cells: make([]cell, 8), capture: capture}
}

func (s *StringSet) Len() int        { return s.size }
func (s *StringSet) Capacity() int   { return len(s.cells) }
func (s *StringSet) Tombstones() int { return s.tombstones }

// Probes counts slots looked at by every Has, Add, and Delete so far.
func (s *StringSet) Probes() int { return s.probes }

func (s *StringSet) Has(value string) bool {
	s.steps = nil
	found, _, _ := s.find(value)
	return found >= 0
}

// Add returns "added" or "present".
func (s *StringSet) Add(value string) string {
	s.steps = nil
	found, grave, open := s.find(value)
	if found >= 0 {
		return "present"
	}
	// Reusing a tombstone takes no new room. Taking an empty slot might.
	if grave < 0 && (s.size+s.tombstones+1)*2 > len(s.cells) {
		s.rebuild()
		found, grave, open = s.find(value)
	}
	slot, reused := open, int64(0)
	if grave >= 0 {
		slot, reused = grave, 1
		s.tombstones--
	}
	s.cells[slot] = cell{live, value}
	s.size++
	s.record("insert", value, slot, reused)
	return "added"
}

// Delete returns "deleted" or "missing".
func (s *StringSet) Delete(value string) string {
	s.steps = nil
	found, _, _ := s.find(value)
	if found < 0 {
		return "missing"
	}
	s.cells[found] = cell{state: tombstone}
	s.size--
	s.tombstones++
	s.record("delete", value, found, -1)
	return "deleted"
}

// Values returns the live values in slot order: neither insertion order nor sorted.
func (s *StringSet) Values() []string {
	values := make([]string, 0, s.size)
	for _, c := range s.cells {
		if c.state == live {
			values = append(values, c.value)
		}
	}
	return values
}

func (s *StringSet) Slots() []Slot {
	slots := make([]Slot, len(s.cells))
	for i, c := range s.cells {
		slots[i] = Slot{[...]string{"empty", "tombstone", "live"}[c.state], c.value}
	}
	return slots
}

func (s *StringSet) Trace() []Step { return append([]Step{}, s.steps...) }

// find returns the value's slot or -1, the first tombstone passed or -1, and the
// empty slot that ended the walk or -1.
func (s *StringSet) find(value string) (found, grave, open int) {
	hash := HashKey(value)
	mask := len(s.cells) - 1
	start := int(hash) & mask
	s.record("hash", value, start, int64(hash))
	grave = -1
	for i := 0; i < len(s.cells); i++ {
		slot := (start + i) & mask
		s.probes++
		switch c := s.cells[slot]; {
		case c.state == empty:
			s.record("empty", value, slot, -1)
			return -1, grave, slot
		case c.state == tombstone:
			s.record("skip", value, slot, -1)
			if grave < 0 {
				grave = slot
			}
		case c.value == value:
			s.record("match", value, slot, -1)
			return slot, grave, -1
		default:
			s.record("compare", value, slot, -1)
		}
	}
	panic("no empty slot: the table is never more than half full")
}

// rebuild doubles if the live values alone would pass half; otherwise it only
// clears the tombstones.
func (s *StringSet) rebuild() {
	old := s.cells
	capacity := len(old)
	if (s.size+1)*2 > capacity {
		capacity *= 2
	}
	s.record("rebuild", "", len(old), int64(capacity))
	s.cells = make([]cell, capacity)
	s.tombstones = 0
	mask := capacity - 1
	for from, c := range old {
		if c.state != live {
			continue
		}
		to := int(HashKey(c.value)) & mask
		for s.cells[to].state != empty {
			to = (to + 1) & mask
		}
		s.cells[to] = c
		s.record("move", c.value, from, int64(to))
	}
}

func (s *StringSet) record(kind, value string, slot int, detail int64) {
	if s.capture {
		s.steps = append(s.steps, Step{kind, value, slot, detail})
	}
}
Reading the TypeScriptA symbol for tombstones

Empty slots are undefined and tombstones are a private Symbol, so no string, not even the empty string, can be mistaken for either.

hashKey encodes the value with TextEncoder, then mixes each byte in with Math.imul and >>> 0 to stay in unsigned 32-bit range. & (capacity − 1) picks the slot, which works because the capacity is always a power of two.

Reading the GoExplicit cell states

Each cell stores a state, empty, tombstone, or live, beside its string, so a freshly made slice is a table of empty cells.

HashKey loops over the string’s bytes directly, and uint32 multiplication wraps on its own. Add and Delete return strings such as "added" so both languages report the same results to the shared test cases.

What would I normally use in application code?Both languages have one, in their own way

In TypeScript, Set. Equality is SameValueZero, iteration follows insertion order, and objects are compared by identity, so a set of { id } objects usually wants to be a set of ids. union, intersection, and difference ship in Chrome 122, Firefox 127, Safari 17, and Node 22.

Go has no set type. map[string]struct{} is the idiom, with _, ok := seen[code] for membership. Since Go 1.24 the built-in map uses Swiss Tables, an open-addressing design whose control bytes mark each slot empty, deleted, or in use: the tombstone idea, built for speed.

05 / Try a decision

Why not just empty the slot?

Tombstones feel like clutter. They take up room, they make walks longer, and emptying the slot looks cleaner. Decide what goes wrong before the feedback tells you.

American Robin and Mourning Dove both start at slot 4, so the dove sits in slot 5. Suppose removing the robin emptied slot 4 instead of leaving a tombstone. Then the birder logs another Mourning Dove. What happens?

06 / Follow the cost

Half full keeps a miss to about two and a half probes.

Here is every operation at a glance, with n species on the list. The rest of this section is about the rows that say average.

Hash set life list: time and extra space per operation
OperationTimeExtra spaceWhat it assumes
Check a speciesO(1) averageO(1)Hash once, then walk until a match or an empty slot. At most half full, a miss probes about 2.5 slots on average. Codes that all hash to the same slot push a walk toward O(n).
Log a sightingO(1) amortized averageO(1) amortizedOne checklist lookup, the same walk, then one write. When codes and tombstones would pass half the slots, a rebuild moves every code first.
Remove a sightingO(1) averageO(1)The same walk, then a tombstone. Tombstones count toward the half-full limit until a rebuild clears them.
Hash a code of k bytesO(k)O(k)TypeScript encodes the code into a k-byte buffer first; Go reads the string’s bytes in place.
Still to find, for c checklist speciesO(c) averageO(c)One lookup per checklist species, in checklist order.
List every speciesO(capacity)O(n)Visits every slot, in slot order, which is neither the order they were seen nor sorted.
Hold n species—O(capacity)At least twice the codes plus tombstones. The table never shrinks, so removals leave it large.

In the animation, the second Mourning Dove took two probes: slot 4, then slot 5. Song Sparrow took three before it knew it was new. Walks stay that short because the table is never more than half full. Every walk ends at its value or at an empty slot, and at least half the slots are empty.

How short on average? Donald Knuth first analyzed linear probing in 1963. The result, as Sedgewick and Wayne state it, predicts about ½(1 + 1/(1 − α)) probes for a lookup that finds its value and ½(1 + 1/(1 − α)²) for one that misses, where α is how full the table is. A model of this lesson’s set, run on random values, reproduces it. At a quarter full, a hit averaged 1.16 probes and a miss 1.39. At half full, a hit averaged 1.50 and a miss 2.48, against a prediction of 1.5 and 2.5.

The same formula is the warning. At three-quarters full it predicts 8.5 probes for a miss, and at nine-tenths, 50.5. Linear probing slows down fast because clusters grow into each other, which is why this set rebuilds at half. Go’s Swiss Tables stay fast when fuller by checking a group of eight slots’ control bytes at once.

Tombstones are the hidden cost. They count toward the half-full limit, so a list that logs and removes species for years slowly fills with them, and walks get longer even though the list is not growing. That is why an add that would pass half rebuilds first, dropping every tombstone, at the same size when the live codes still fit.

These are probe counts, not milliseconds, and they assume the hash spreads values evenly. A hash that sends many values to the same start slot, by accident or on purpose, turns every walk into a scan of that cluster: the O(n) worst case.

07 / Give it a real job

One life list per birder, one checklist per region.

In a real app, one LifeList belongs to one birder. The sighting form calls log, fixing a misidentified sighting calls remove, and the “still to find” screen calls stillToFind. The checklist comes from the region the birder chose, so a code can be valid in one region and missing from another.

What the list leaves out is a decision too. It keeps codes, not sightings, so it cannot say when or where a species was first seen; a real app stores those records and builds the set from them. It saves nothing. It treats codes as exact strings, so an app that accepts typed names must normalize them first, the lesson of the Hash map example. And checklists change: the Institute for Bird Populations publishes code updates as species are split and renamed, so a stored code can stop meaning what it used to.

Build UIs?See the hash set already guarding your lists, and the day you have to own one.

Where it already is in your components

Render two list items with the same key, and React logs “Encountered two children with the same key” in development. Svelte goes further and throws. Neither compares every key with every other key. React’s reconciler adds each key to a Set as it goes and warns when has is already true. Svelte’s keyed each block collects the keys in a Set and compares its size with the number of items. The rule that keys must be unique is enforced by exactly the structure in this lesson.

The checkbox table is the other place you already reach for one. selected.has(row.id) is one lookup per row, where selectedIds.includes(row.id) scans every selected id for every row. The frameworks differ on how you change it. React skips a re-render when state is the same object, so toggling a row builds a new Set. Svelte does not make a plain Set inside $state reactive, so you use SvelteSet, and call add and delete directly.

When you have to own it

Picture an infinite feed with offset pagination. Page 1 is posts 0 to 19. While the reader scrolls, five new posts arrive at the top, everything shifts down five places, and page 2 starts with five posts they already have. Append them blindly and React warns about repeated keys while Svelte throws. With index keys, the reader silently sees the same posts twice.

The fix is a set of ids you have already shown. Each incoming post costs one lookup: skip it if its id is there, add it if not. Checking the rendered array instead, with posts.some((p) => p.id === post.id) for every new post, scans everything shown so far and gets slower with every page. And load one page at a time, so two responses cannot race past the check together.

A table with checkboxes. The selected ids live in a set: React replaces it on every toggle, and Svelte’s SvelteSet changes in place.

ReactAlready in your code
PeopleTable.tsx
import { useState } from 'react';

type Row = { id: string; name: string };

export function PeopleTable({ rows }: { rows: Row[] }) {
	const [selected, setSelected] = useState<ReadonlySet<string>>(() => new Set());

	function toggle(id: string) {
		setSelected((previous) => {
			// A new Set every time. Adding to the old one would leave state as the
			// same object, and React skips a re-render when state has not changed.
			const next = new Set(previous);
			if (next.has(id)) next.delete(id);
			else next.add(id);
			return next;
		});
	}

	return (
		<table>
			<tbody>
				{rows.map((row) => (
					<tr key={row.id}>
						<td>
							{/* One lookup per row. selectedIds.includes(row.id) would scan every selected id. */}
							<input
								type="checkbox"
								checked={selected.has(row.id)}
								onChange={() => toggle(row.id)}
							/>
						</td>
						<td>{row.name}</td>
					</tr>
				))}
			</tbody>
		</table>
	);
}

08 / Make the call

Ask what the question really is.

Reach for a hash set when the question is membership: have I seen it, is it allowed, is it selected. Use your language’s own: Set in TypeScript, map[string]struct{} in Go.

Look elsewhere when the question changes. A handful of values: an array and includes is simpler and fast enough. Values in order, or everything between two values: an ordered map or set. Something stored beside each value, like the date of a first sighting: a hash map. Membership in a set too large for memory, where a rare false “yes” is acceptable: a Bloom filter. Only the most recent few values: a ring buffer, or a set beside one.

09 / Take the idea with you

Explain it without saying “hash set.”

“I keep the codes in a row of slots. A code’s hash says where to start looking, and I walk forward until I find it or reach an empty slot. When I remove one, I leave a marker instead of a gap, so nothing behind it gets lost. And I never let the row get more than half full.” That describes the mechanism. The name is what you call it in a review.

Before moving on, explain three things without the name: why Song Sparrow walked to slot 6 before going into slot 4, why emptying slot 4 would have counted the dove twice, and why the table rebuilds at half full. Then find an includes inside a loop in your own code and decide whether a set would answer it better.

Connections to follow nextRelated lessons
  • Hash map hashes the same way, keeps a value beside each key, and handles collisions with a list per bucket.
  • Dynamic array doubles when it fills; the same amortized argument pays for this set’s rebuilds.
  • Ring buffer keeps only the latest few values. Beside a set, it bounds how long “seen” is remembered.