# Analysis Methodology

## Entity Extraction: What to Catalog

When reading a Python file, extract these entities:

### Functions
- Top-level `def` statements (not nested inside classes or other functions)
- Record: name, parameters, return type (if annotated), decorators
- For the call list: every `name(...)` pattern inside the body, plus attribute calls like `obj.method()`

### Classes
- Top-level `class` statements
- Record: name, base classes, decorators, class-level attributes
- For each method: name, calls (same as functions)
- Special attention: `__init__` often reveals the most about what the class depends on

### Module-level globals
- Constants (`UPPER_CASE = ...`), module-level instances, configuration dicts
- These matter because they can create implicit coupling — if module A defines `CONFIG` and module B reads `A.CONFIG`, that's a dependency

### Imports
- `import X` and `from X import Y` — these are the declared dependency edges
- `from X import Y as Z` — record the alias so you can resolve calls to `Z` back to `X.Y`
- Conditional imports (`if TYPE_CHECKING:`) — these are type-only, not runtime dependencies

## Call Tracing: Resolving Names

The hardest part. When you see `normalize(text)` inside a function, you need to figure out which `normalize` this refers to:

1. **Check local scope**: Is `normalize` defined in the same function? (closure/nested def)
2. **Check module scope**: Is `normalize` defined at the top level of this file?
3. **Check imports**: Was `normalize` imported from another module?
4. **Attribute calls**: `self.normalize()` → look at the class definition. `obj.normalize()` → determine obj's type from context, then look up the method.

### What to ignore
- **Stdlib and third-party calls**: `os.path.join()`, `json.dumps()`, `logger.info()` — these don't affect internal modularity
- **Built-in calls**: `len()`, `range()`, `print()`, `isinstance()`
- **String/literal operations**: `"hello".upper()`, `[1,2,3].append()`
- **Dynamic calls**: `getattr(obj, method_name)()` — can't resolve statically, skip

### What to track
- **Cross-module internal calls**: `from mypackage.utils import normalize` then `normalize()` — this is the core signal
- **Intra-module calls**: `func_a()` calling `func_b()` in the same file — relevant for understanding cohesion
- **Class instantiation**: `Record(...)` when `Record` is imported from another module — this is a dependency

## Modularity Assessment

### The Core Heuristic

Good module structure means:
- **High cohesion**: Entities within a module call each other frequently (intra-module connections are dense)
- **Low coupling**: Entities in different modules rarely call each other (inter-module connections are sparse)

A **misplaced entity** is one that violates this — it lives in module A but most of its connections are to module B.

### Affinity Score

For each entity `e` and each module `m`:

```
affinity(e, m) = outgoing_calls_from_e_to_m + incoming_calls_to_e_from_m
```

The entity's **home affinity** is `affinity(e, current_module)`. If any other module has higher affinity, the entity is a move candidate.

### Confidence Levels

Not all misplacements are equally clear-cut:

- **Strong signal** (definitely suggest): affinity to another module is 3x+ the home affinity, and total connections ≥ 5
- **Medium signal** (suggest with caveat): affinity elsewhere exceeds home, but margin is small or total connections are few
- **Weak signal** (mention but don't push): roughly equal affinity — could go either way. Consider other factors (naming convention, logical grouping, circular dependency reduction)

### Structural Smells

Beyond individual misplacement, look for these module-level patterns:

1. **God module**: One module imported by >50% of other modules. Usually `utils.py`, `helpers.py`, or `common.py`. Fix: decompose into domain-specific modules based on caller clusters.

2. **Circular dependency**: A imports B, B imports A (directly or transitively). Fix: extract the shared entities into a third module, or merge A and B if they're tightly coupled.

3. **Feature envy**: A class's methods mostly operate on data from another module's classes. Fix: move the envious class closer to the data it uses.

4. **Shotgun surgery**: A single logical change requires editing many modules. This signals that related entities are scattered. Fix: co-locate related entities.

5. **Dead code**: Entities defined but never called (no incoming edges in the call graph). Flag for removal rather than movement.

## Safe Refactoring: Moving Entities

### Pre-move checklist

1. **Identify ALL references**: Use `Grep` to find every `import` and usage of the entity across the entire codebase. Include:
   - Direct imports: `from module import entity`
   - Module imports: `import module` then `module.entity`
   - Re-exports: other `__init__.py` files that re-export the entity
   - String references: `"module.entity"` in config files, serialization, etc.
   - Test files: tests that import the entity

2. **Check for re-exports in `__init__.py`**: If the module's `__init__.py` exports the entity, you need to update that too.

3. **Confirm target module exists**: If not, create it with appropriate `__init__.py` entries.

### The move sequence

For each entity:

1. **Copy** the entity definition to the target file
2. **Copy** any imports the entity itself needs (its own dependencies)
3. **Update** all files that imported the entity to point to the new location
4. **Add re-export** in the old location: `from new_module import entity` — this is a safety net
5. **Run tests** to verify nothing broke
6. **Remove** the original definition from the old file (keep the re-export for now)
7. **Commit** with a clear message

### Import update patterns

```python
# Old: from mypackage.utils import normalize_text
# New: from mypackage.pipeline.text import normalize_text

# Old: import mypackage.utils; mypackage.utils.normalize_text(...)
# New: import mypackage.pipeline.text; mypackage.pipeline.text.normalize_text(...)

# Old: from mypackage.utils import normalize_text as norm
# New: from mypackage.pipeline.text import normalize_text as norm
```

### When NOT to move

- **Public API boundaries**: If the entity is part of the package's documented public API (exported in top-level `__init__.py`), moving it is a breaking change. Discuss with the user first.
- **Circular dependency creation**: If moving entity E from A to B would create a new circular import (B already imports from A), the move makes things worse. Find a different destination.
- **Test-only entities**: Helpers used only in tests should stay in test files, not production code.
