Taxonomy of Lisp Form Shapes

What we did

Analyzed 10,455 source files across 31 open-source repositories to build a
data-driven taxonomy of what structural forms actually appear in
Lisp/Scheme/Racket/Clojure code.

Repositories analyzed

Common Lisp (12 repos, 2,926 files):

Racket (3 repos, 5,424 files):

Clojure (16 repos, 2,080 files):

Methodology

1. Mechanical extraction: Two Python parsers (one for CL/Racket/Scheme,
   one for Clojure with []/{} as data not forms) extracted all head
   symbols from s-expressions, filtering against blacklists of ~200+ standard
   forms per language and excluding calls where all arguments are atoms.
2. Agent classification: 517 haiku-level LLM agents (457 for CL/Racket,
   60 for Clojure) each received a batch of source files plus candidate head
   symbols. Each agent read the actual code and classified forms by name,
   description, and shape, filtering out false positives.
3. Deduplication: Entries were merged by form name, keeping the best
   description and tracking all source files.
4. Review: Opus-level agents reviewed each language's taxonomy, removing
   standard builtins that leaked through, merging package-qualified
   duplicates, and fixing shape templates. CL/Racket: 986 raw 737 clean.
   Clojure: 220 raw 35 clean (aggressive filtering since most Clojure forms found were standard builtins from spec, core.async, etc.).
5. Merge: The cleaned taxonomies were merged into a single file with 772
   entries tagged by language (CL, Racket, Clojure, or multi).
6. Categorization: Forms were categorized by pure structural shape
   (argument layout), ignoring domain/purpose.

Results: Structural shapes

18 distinct argument-layout patterns cover all 772 forms. The 8 structurally
interesting ones (as opposed to "it's just a function call") are:

The "Flat Alternating Pairs" and "Annotated Param-List" shapes are new
findings from the Clojure analysis that were not present in the CL/Racket
data alone.

Flat Alternating Pairs deserves special attention: where CL/Racket
pattern-matching forms wrap each clause in a sublist —
(match expr (pat1 body1) (pat2 body2)) — Meander's m/match and
m/rewrite lay them flat: (m/match expr pat1 result1 pat2 result2).
This also appears in case-enum, zmatch, and re-frame's reg-sub
with :<- markers. It's structurally distinct from both "Expr Then Clauses"
(which has sublists) and "Variadic Args" (which has independent arguments).

The remaining shapes are less structurally distinctive:

Distribution

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

Config-Group Then Body    ██████████████████████████████████████████████████ 118
Name Then Args            ████████████████████████████████████████████ 105
Name, Param-List, Body    ██████████████████████████████ 72
Binding-List Then Body    ███████████████████ 47
Flat Alternating Pairs    ███████████████ 37
Body Only                 ██████████████ 35
Expr Then Clauses         █████████████ 33
Expr Then Body            ████████████ 31
Misc / Special            ██████ 14
Annotated Param-List      ███ 8
Name Then Clauses         ██ 5
Name Then Body            █ 3
Misc Compound             █ 3
Bare Atom                 █ 2
Nullary                    1

(Variadic Args: 155 and Fixed Positional Args: 117 omitted — structurally
indistinct from function calls)

Results: Purpose categories

A separate categorization by purpose/domain (not purely structural) found 25+
categories. The top 12, now including Clojure-originated categories:

The handler-registration and pattern-match-rewrite categories are new
entries driven primarily by Clojure data:

• Handler registration (reg-event-db, reg-sub, reg-fx,
  definterceptor, defroutes, defstate) is the dominant macro pattern in
  Clojure web/UI code. The shape is typically
  (reg-X id [middleware...] handler-fn) — a name, optional interceptor
  chain, and a handler function. This pattern barely appears in CL/Racket
  because those ecosystems tend to use generic functions or method dispatch
  instead of global registries.
• Pattern-match-rewrite merges CL/Racket's match/case forms with
  Meander's m/match/m/rewrite and XTDB's zmatch. Notably, Meander's
  forms use flat alternating pairs rather than wrapped clause sublists, which
  is a structural difference from the Racket/Redex pattern-matching tradition.

Clojure-specific forms (35 entries)

The Clojure analysis added 35 domain-specific forms not found in CL/Racket.
Most standard Clojure forms (spec, core.async, test.check) were filtered as
builtins, leaving genuinely project-specific macros:

Known gaps and open questions

Things this taxonomy might be missing or underrepresenting:

Structural patterns that may be underrepresented

• Form-composition / threading: nest from Coalton takes N wrapper forms
  and a base expression, composing them: (nest f g h x) = (f (g (h x))).
  This was lumped into "Variadic Args" but is structurally distinct — the
  arguments aren't independent, they compose right-to-left. Clojure's ->,
  ->>, some->, cond-> threading macros are the most famous version of
  this pattern but were blacklisted as standard forms. The threading/composition
  shape deserves its own structural category — it appeared in both CL (via
  nest) and Clojure (via the threading macros) but was underrepresented
  because we treated the Clojure versions as builtins.
• Accumulation / collection forms: collect and do-sequence appeared but
  the "collector callback" pattern (where you pass a name that becomes a
  function you call to accumulate results) is a distinctive shape we may have
  too few examples of.
• Continuation / delimited control: call-with-continuation-prompt,
  abort-current-continuation, etc. appeared but the structural pattern
  (prompt tag + handler + body) is distinctive and may deserve its own shape.
• Place/slot mutation forms: (setf (accessor obj) value) and CL's
  define-setf-expander have a distinctive shape where the "place" is itself
  a compound form. This is hard to extract mechanically because setf was
  blacklisted.
• Reader macros / dispatch characters: #', #(, #|...|# etc. are
  structural patterns at the reader level that our s-expression parser couldn't
  see.
• Loop-clause DSLs: CL's loop macro and iterate's iter have rich
  internal clause languages (for x in list, collecting, finally, etc.)
  that are invisible to our analysis because we blacklisted loop and treated
  iter's clauses as atoms.
• Method combination / CLOS protocol forms: define-method-combination,
  :before/:after/:around qualifiers, call-next-method — the MOP has
  structural patterns around method dispatch that may be underrepresented.
• Condition system: restart-case with its named restarts and
  handler-bind with condition-type/handler pairs are distinctive structural
  patterns that were blacklisted.

Repo coverage gaps

• No Emacs Lisp repos (would add defcustom, defadvice, cl-loop,
  pcase, rx, and interactive-command patterns).
• No R7RS Scheme library code (would add syntax-rules patterns, SRFI forms).
• Alexandria and other utility libraries aren't directly included (their forms
  appear via downstream usage in McCLIM, serapeum, etc., but we may miss some).

Questions for review

1. Are there common macro patterns in production CL code that don't appear
   here because they're too project-specific to show up in library code?
2. Is the "Config-Group Then Body" vs "Binding-List Then Body" distinction
   real, or are these the same shape with cosmetic differences? (In both
   cases, the first arg is a list of pairs/specs followed by body forms.)
3. Are there important shapes in Scheme's syntax-rules / syntax-case
   pattern language that we're missing because we treated Racket's
   define-syntax-rule etc. as atomic?
4. Does the distinction between "Name Then Args" (105 forms) and "Variadic
   Args" (155 forms) hold up, or is the first-arg-is-a-name distinction
   not structural enough?
5. The "Flat Alternating Pairs" shape is distinctive to Clojure/Meander
   but cond in all Lisps has the same structure (test expr test expr ...).
   Is this actually a more general pattern that we undercounted because
   cond was blacklisted?