depgraph

Depgraph Prototype Review

This document describes the current state of index.html (~7600 lines, 290KB single file). It is structured so that each ## heading is a cluster, each - \identifier` is a **node**, and backtick cross-references within descriptions are **edges** between nodes. ###` subsections are sub-clusters: they are nodes in the parent cluster AND clusters of their children.

Overview: What The App Does (Step by Step)

  1. Fetch data – load history.csv (runtime recording) or fall back to source HTML for AST analysis. Also fetch a codemap markdown file for cluster definitions and importance scores.
  2. Parse – parseHistoryCSV or analyzeCode (acorn AST walk) produce an analysisResult with functions, globals, edges, and node types.
  3. Cluster – clusterFunctions assigns each node to a cluster via codemap sections, CSV cluster names, or union-find on shared state.
  4. Name & Score – nameClusters generates cluster labels from keywords; computeAffinities scores each node’s fractional membership across clusters.
  5. Build Graph – buildGraph creates node objects (with radius, color, affinities, minZoom) and link objects (with type, weight). In hypergraph mode, structural/value nodes are created on-the-fly from edge endpoints.
  6. Layout – computeLayout runs a multi-phase pipeline: intra-cluster force simulation, deep-node positioning along edges, cluster packing, optional warm restart.
  7. Render – renderGraph does D3 data joins to create SVG elements across 6 layer groups (hulls, meta-links, links, nodes, labels, cluster-labels). Gradient edge defs, arrowheads, hulls via d3.polygonHull.
  8. Semantic Zoom – applySemanticZoom adjusts visibility, opacity, font size, edge layer state, and meta-edge display based on zoom level currentK.
  9. Interact – Keyboard/mouse handlers dispatch to 8+ interaction modes (attractor, click-pull, x-relax, z-time-travel, t-trace, neighbor-gather, cluster-gather, space-pull).
  10. Persist – Spatial memory system saves arrangement snapshots to localStorage with decay, enabling Z-time-travel and arrangement navigation.
  11. Stream – SSE connections enable progressive graph building: base phase (bulk), tail phase (incremental with StreamPlacement), end phase (settle with GraphPhysics).

Data Pipeline

Parsing

Clustering

Graph Construction

Edge Layer System

Layer Configuration

Visibility & Opacity

Zoom Breakpoints

Layout & Physics

Initial Layout Pipeline

GraphPhysics (Streaming/Interactive)

StreamPlacement

Position Persistence

SVG Rendering

Layer Structure (bottom to top)

  1. gHulls: cluster boundary polygons + textPath boundary labels
  2. gMetaLinks: aggregated inter-cluster bezier curves with gradients
  3. gLinks: individual edge lines + arrowheads
  4. gNodes: node circles + secondary affinity rings
  5. gLabels: node text labels
  6. gClusterLabels: floating cluster name text + user cluster labels

Initialization

Semantic Zoom (line 3108)

This function conflates optical scaling with navigation-level changes:

Node Rendering

Position Updates

Cluster Visuals

Hull Computation

Meta-Edges

Cluster Labels

Cluster Importance

Interaction

Selection & Focus

Attractor / Click-Pull

X-Relax (Rewind)

Z-Time-Travel

T-Trace

Space-Gather

The d3 zoom transform is a pure optical scaling (line 3081-3094). But applySemanticZoom (line 3108) bolts navigation semantics onto it, making zoom level change WHAT you see, not just HOW you see it.

Behavior Optical (transform) Navigation (content change)
Pan/scroll YES no
Scale factor YES no
Zoom breakpoints change layer visibility no YES
Label opacity thresholds no YES (info hiding)
Deep node fade-in (k>10) no YES (new content)
Meta-edge show/hide (k<1.5) no YES
Floating label fade-out (k>2.5) no YES
TextPath label fade-in (k=3-4) no YES
Hull opacity scaling partial partial
Edge proximity boost (k>=3) partial partial

The problem: zoom is the ONLY way to navigate the hypergraph depth. There is no concept of “enter this cluster” or “go up a level” independent of the optical zoom. The zoom breakpoints (Overview at 0.3, Systems at 1.0, Structural at 4.0, Deep at 10.0) are de facto navigation levels masquerading as optical comfort settings.

What hypergraph navigation should be: A discrete operation – “expand this cluster into its members” or “collapse these nodes into their cluster” – that changes the graph topology, not the camera. Zoom could then be purely optical. The breakpoints would become navigation-level transitions triggered by zoom OR by explicit user action (click to enter, back to exit).

Where zoom is tightly coupled to functionality (all in applySemanticZoom, line 3108):

Event Handling

Keyboard Dispatch (line 6263)

15+ active keys with complex modifier combinations:

Mouse/Touch

Streaming & Live Reload

SSE Connections

Cinematic Mode

Spatial Memory

Anomalies

Bugs

Duplicate Functionality

These are places where the same concept is implemented in two or more divergent ways:

1. Cluster Centroid – two implementations

2. Edge Opacity – computed in two places

3. Node Visibility – 4+ mechanisms

4. renderHulls() + renderClusterLabels() – bundled in 6+ call sites

5. Highlight/reset – 3 implementations

6. Position saving – 3 separate triggers

7. Soft collision – similar math in 4 places

8. Cluster member filtering – repeated pattern

Parallel State That Drifts

savedPositions vs stickyNodes vs lockedNodes

selectedNode vs selectedNodes

Uncoordinated Animation Loops

13+ independent requestAnimationFrame chains, each calling renderPositions():

Problems:

  1. Multiple loops calling renderPositions() in the same frame = redundant DOM writes
  2. No priority system – if attractor and gather run simultaneously, they both modify node positions with conflicting forces
  3. isUserInteracting() (line ~401) is a flat OR guard but doesn’t prevent all conflicts
  4. Each loop has its own cleanup logic; missing cleanup in one leaks into the next interaction

Node-Cluster Duality: Analysis

Current Model

What Hypergraph Navigation Demands

At zoom level N, a cluster should appear as a single node. Zoom into it, and it expands to show its member nodes, which may themselves be clusters at a deeper level. This is the fundamental operation that is missing.

The current code has the pieces:

But these pieces are not unified into a single recursive data structure. Currently:

A cluster node should be a node that contains other nodes. When collapsed, it acts as a single node in the graph (with meta-edges as its connections). When expanded, its children become visible and its hull becomes the boundary. The expand/collapse operation should be independent of zoom – zoom changes the camera, expand/collapse changes the topology.

Hypergraph Concept Isolation

What Makes This a Hypergraph (Not Just a Graph)

  1. N-ary relations: a shared global variable connects ALL functions that read/write it. Currently modeled as pairwise edges, but the underlying relation is N-ary. A proper hyperedge would draw one “edge” connecting all participants.
  2. Typed, dynamic edge layers: the pull-layer system treats edge types as first-class with opacity and physics participation.
  3. Multi-cluster membership: affinities give each node fractional membership in multiple clusters. Primary cluster is just argmax.
  4. Zoom-dependent projection: at each zoom level, you see a different projection. Low zoom contracts clusters into nodes. High zoom reveals internals. This IS hypergraph contraction/expansion, just implemented as opacity heuristics.

What Needs Isolation

The hypergraph data model (nodes, edges, containment, projections) should be cleanly separated from:

Currently all of these are tangled in applySemanticZoom (which does rendering + navigation + edge visibility), renderGraph (which does data construction + layout + SVG creation), and the various gather/attractor loops (which do physics + interaction + rendering).

File Responsibility Key Functions
core/types.js Node, Edge, Cluster, HyperEdge type definitions (type comments/JSDoc)
core/state.js All 40+ global state variables, centralized exported state object
core/animation.js Central RAF scheduler, replaces 13 independent loops registerLoop, tick, single renderPositions per frame
data/parse.js CSV + AST + codemap parsing parseCSVLine, parseHistoryCSV, analyzeCode, parseCodemap
data/cluster.js Clustering, naming, affinity scoring clusterFunctions, nameClusters, computeAffinities
data/graph-builder.js Node/link creation from analysis buildGraph
edges/layers.js Edge layer config, pull-layer state, UI sliders EDGE_LAYERS, pullLayerState, initPullLayerUI, edgeOpacity
edges/visibility.js Unified visibility: hidden nodes + deep fade + search + trace isNodeVisible(node) – single query
layout/physics.js GraphPhysics module extracted as class force tick, spatial grid, settle detection
layout/placement.js StreamPlacement, initial layout, cluster packing computeLayout, placeNewNode
layout/positions.js Position persistence: savedPositions + sticky + locked (UNIFIED) savePosition, isSticky, isLocked
render/svg.js SVG init, layer groups, zoom behavior initSVG
render/nodes.js Node circle creation, data joins setupNodeCircle, renderIncremental
render/edges.js Edge lines, gradients, arrowheads edge rendering from renderGraph
render/hulls.js Hull computation, expansion, textPath labels renderHulls, expandHull
render/meta-edges.js Meta-link computation, bezier curves, centroids computeMetaLinks, clusterCentroid (UNIFIED)
render/labels.js Node labels, cluster labels, label placement computeClusterLabelPlacement, renderClusterLabels
render/positions.js renderPositions – transform all elements single render call per frame
navigation/semantic-zoom.js Zoom-to-navigation-level mapping, breakpoints applySemanticZoom (STRIPPED of rendering)
navigation/expand-collapse.js Future: cluster expand/collapse topology changes expandCluster, collapseCluster
interact/select.js Selection, info panel, multi-select selectNode, selectedNodes
interact/attractor.js Force press, attractor loop startAttractor, attractorLoop
interact/forces.js X-relax, click-pull, dismiss-to-T0 startXForces, clickPull, dismissNodeToT0
interact/trace.js T-trace BFS, flash, hold, mode switching startTrace, traceLoop
interact/gather.js All gather variants: neighbor, cluster, intra, space-pull startNeighborGather, startClusterGather, etc.
interact/keyboard.js Keydown/keyup dispatch delegates to interaction modules
spatial/memory.js Arrangement history, decay, comparison mode pushArrangement, navigateArrangement
spatial/time-travel.js Z-key arrangement stepping startZTimeTravel
stream/sse.js SSE connections, stream phases connectStreamSSE, connectGraphSSE
stream/cinematic.js CinematicStream zoom-tour CinematicStream
main.js Init pipeline, rebuild, polling loadAndAnalyze, initDataPipeline

Key Architectural Changes in the Split

  1. Central animation scheduler (core/animation.js): All interaction loops register their per-frame update function. Scheduler calls all active updaters, then renderPositions() exactly ONCE. Eliminates redundant DOM writes and conflicting forces.

  2. Unified visibility (edges/visibility.js): Single isNodeVisible(node) function that combines hidden-nodes, deep-fade, search, and trace state. All rendering checks this one function.

  3. Unified position persistence (layout/positions.js): Merge savedPositions, stickyNodes, and lockedNodes into one PositionState per node: {x, y, sticky: bool, locked: bool}. Single source of truth.

  4. Unified cluster centroid (render/meta-edges.js): One clusterCentroid(cid) function used by both physics and meta-edge rendering. No more divergent implementations.

  5. Navigation separated from rendering (navigation/semantic-zoom.js): Computes the navigation level from zoom, but doesn’t touch the DOM. Returns a NavigationState that the rendering modules consume.