
# Codebase Structure Analysis for Python

This guide teaches how to analyze a Python project's module structure by building a call graph, assessing modularity, and reorganizing entities (functions, classes, globals) across files for better cohesion and lower coupling. No external tools needed — you do this by reading code and reasoning about call relationships.

## Why Structure Matters

Poor module structure creates compounding problems:
- **Circular imports** that crash at runtime or force awkward workarounds
- **God modules** (utils.py, helpers.py) that everything depends on, making changes risky
- **Scattered features** where a single logical change touches 8 files
- **Cognitive overhead** — developers can't find things because placement is arbitrary

The fix is to put entities near the code that uses them most. This sounds obvious, but in practice requires systematically tracing call relationships rather than guessing.

## The Analysis Workflow

### Step 1: Entity Catalog

Read each Python file and extract:
- **Functions**: name + what they call
- **Classes**: name + methods + what each method calls
- **Globals**: constants, singletons, config objects
- **Imports**: every `from X import Y` and `import X`

Focus on **internal** calls (within the project). Ignore stdlib/third-party calls — they don't affect how your code should be organized.

### Step 2: Cross-Module Call Matrix

For every pair of modules, count how many calls flow between them. Present as a matrix:

```
| From \ To    | models | pipeline | utils | api |
|--------------|--------|----------|-------|-----|
| models       | 12     | 0        | 3     | 0   |
| pipeline     | 8      | 15       | 11    | 2   |
| utils        | 1      | 0        | 4     | 0   |
| api          | 5      | 7        | 6     | 3   |
```

This immediately reveals:
- **Dense diagonal** = good cohesion (intra-module calls are high)
- **Dense off-diagonal** = high coupling (inter-module calls are high)
- **Asymmetric rows** = one-directional dependencies (often fine)
- **Rows with tiny diagonal** = modules with low internal cohesion

### Step 3: Identify Misplaced Entities

For each entity, compute its **affinity** to each module:

```
affinity = outgoing_calls_to_module + incoming_calls_from_module
```

An entity is misplaced when another module has higher affinity than its home module. The bigger the gap, the clearer the signal.

**Example**: `normalize_text()` lives in `utils.py`. It makes 0 calls to other utils functions, but `pipeline/transform.py` calls it 5 times and `pipeline/validate.py` calls it 3 times. Affinity to utils: 0. Affinity to pipeline: 8. Strong move candidate.

### Step 4: Suggest Reorganization

Rank suggestions by impact and present to the user:
- Move entities with the strongest misplacement signal first
- Group related moves (if 3 functions in utils.py all have high affinity to pipeline, suggest moving them together)
- Flag potential issues (circular dependency creation, public API breakage)

### Step 5: Refactor

After user approval, move entities one at a time:
1. Copy entity to target file with its own imports
2. Update all import sites across the codebase
3. Add temporary re-export in old location as safety net
4. Run tests after each move
5. Commit each move separately

See `modularity-guide.md` for the detailed methodology, structural smell catalog, and safe refactoring checklist.

## Structural Smells Quick Reference

| Smell | Signal | Fix |
|-------|--------|-----|
| **God module** | One file imported by >50% of modules | Decompose by caller clusters |
| **Circular dependency** | A → B → A | Extract shared entities to C, or merge |
| **Feature envy** | Methods mostly use another module's data | Move class to where its data lives |
| **Shotgun surgery** | One change = edit 5+ files | Co-locate related entities |
| **Dead code** | Entity with zero incoming calls | Remove (after confirming with user) |
| **Middleman** | Module that just re-exports from others | Eliminate the pass-through |

## Mermaid Diagram

After building the call matrix, generate a module-level dependency diagram:

```mermaid
graph LR
    pipeline -->|11 calls| utils
    pipeline -->|8 calls| models
    api -->|7 calls| pipeline
    api -->|6 calls| utils
    api -->|5 calls| models
    models -->|3 calls| utils
```

This gives the user a visual overview before diving into specific moves.

## Reference Files

- `modularity-guide.md` — Full modularity concepts: coupling, cohesion, affinity scoring, confidence levels, structural smell detection, and the complete safe refactoring protocol
