What is a Set and when should we use it?

What you need to know

Creating a set

1tags = {"upi", "refund", "upi"}      # duplicates collapse2print(len(tags))                     # 23empty = set()                        # {} would be an empty DICT4print(type({}).__name__)             # dict5print(set(["b", "a", "b"]) == {"a", "b"})   # True

Items must be hashable, so a set can hold strings, numbers and tuples, but not lists. A set has no positions, so tags[0] is an error. Printing order is not something to rely on.

Membership: O(1) vs O(n)

x in my_list compares x with each element until it finds a match — a million comparisons in the worst case for a million items. x in my_set hashes x and jumps straight to its slot. If you check membership inside a loop, this turns an O(n × m) job into O(n).

Set algebra

1keyword_hits = {"d1", "d4", "d7", "d9"}2vector_hits  = {"d2", "d4", "d9", "d11"}3print(sorted(keyword_hits & vector_hits))   # ['d4', 'd9']  -> strong matches4print(len(keyword_hits | vector_hits))      # 6             -> all candidates5print(sorted(vector_hits - keyword_hits))   # ['d11', 'd2'] -> only semantic

frozenset is the immutable version, so it can itself be a dict key or a member of another set.

A real-life example

Before embedding 50,000 scraped help-centre chunks, you remove exact duplicates so you do not pay to embed the same text twice. A set of already-seen texts does it in one pass:

1chunks = ["Reset your UPI PIN in the app.", "Refunds take 5-7 days.",2          "Reset your UPI PIN in the app.", "Contact support 24x7."]34seen = set()5unique = []6for text in chunks:7    key = " ".join(text.lower().split())     # normalise case and spaces8    if key not in seen:                       # O(1) check9        seen.add(key)10        unique.append(text)                   # keep the original order11print(len(chunks), "->", len(unique))         # 4 -> 3

With a list for seen, 50,000 chunks means up to about 1.25 billion comparisons; with a set it is 50,000 hash lookups. Notice that the output is a list, because order matters for the next step.

Follow-up questions to expect

• "Why is {} not an empty set?" — Dict literals came first, so {} means an empty dict. Use set().
• "How do you remove duplicates but keep the order?" — list(dict.fromkeys(items)), because dicts keep insertion order and ignore repeated keys.
• "What is the difference between remove and discard?" — remove(x) raises KeyError if x is missing; discard(x) does nothing.