# 7. Eliminate circular dependency between smartem_api and smartem_backend

Date: 23/08/2025

## Status

Accepted

## Context

The SmartEM Decisions project developed a circular dependency between the `smartem_api` and `smartem_backend` packages 
that violated clean architecture principles and created maintenance challenges for the scientific computing workflow 
automation system.

The circular dependency manifested in two ways:
1. **Package-level dependency**: `smartem_backend` declared dependency on `smartem_api` in its pyproject.toml 
   configuration, requiring the API package for core backend functionality
2. **Module-level imports**: `smartem_api.server` module imported extensively from `smartem_backend` components 
   including database models, message queue publishers, and utility functions

This circular relationship created several problems for the cryo-electron microscopy workflow system:
- **Build complexity**: Package installation order became critical and error-prone during development setup
- **Architectural violation**: Backend logic was split across packages in non-intuitive ways, complicating system 
  understanding
- **Maintenance burden**: Changes to core backend functionality required coordinated updates across multiple packages
- **Import confusion**: Developers struggled to understand which package provided specific functionality
- **Testing complexity**: Circular imports made unit testing and mocking more difficult

The project's multi-package structure (`smartem_backend`, `smartem_agent`, `smartem_common`, `athena_api`) was designed 
to enable modular deployment and clear separation of concerns for the high-throughput microscopy data processing 
requirements. However, the API package had evolved beyond its intended scope as a simple HTTP client interface.

Three approaches were considered for resolving the circular dependency:

1. **Dependency inversion**: Introduce abstract interfaces to break direct import cycles whilst maintaining separate 
   packages
2. **Package splitting**: Further decompose packages to eliminate circular relationships through fine-grained separation
3. **Package consolidation**: Merge `smartem_api` functionality into `smartem_backend` to eliminate the circular 
   dependency entirely

## Decision

We will **consolidate the smartem_api package into smartem_backend** to eliminate the circular dependency completely.

The consolidation will restructure the codebase as follows:
- Move `smartem_api/client.py` to `smartem_backend/api_client.py` for HTTP client functionality
- Move `smartem_api/server.py` to `smartem_backend/api_server.py` for FastAPI server implementation
- Relocate HTTP data models to `smartem_backend/model/http_*.py` alongside existing database models
- Maintain backward compatibility by creating re-export modules in the original `smartem_api/` location
- Remove `smartem_api` as a separate package from pyproject.toml configuration
- Introduce lightweight `client` dependency group for components requiring only API client functionality

The API client will remain accessible to `smartem_agent` and external consumers through the consolidated package 
structure, preserving the modularity required for distributed cryo-electron microscopy workflow processing.

## Consequences

**Positive:**
- Eliminates circular dependency completely, resolving architectural violation and build complexity
- Reduces operational overhead by managing one fewer package in the development and deployment pipeline
- Simplifies import structure with clear single-source responsibility for HTTP API functionality
- Maintains full backward compatibility for existing code using `smartem_api` imports
- Preserves API client reusability for `smartem_agent` and external scientific computing integrations
- Continues to support Swagger documentation generation from the consolidated API server
- Enables cleaner dependency management with optional `client` group for minimal installations

**Negative:**
- Increases `smartem_backend` package scope, potentially affecting cognitive load for developers
- Requires migration period where both import paths coexist, creating temporary code duplication
- May impact deployment strategies that relied on separate API package for containerised microservices
- Documentation and examples require updates to reflect new package structure

**Implementation considerations:**
- Comprehensive testing across all import paths to ensure backward compatibility during transition period
- Documentation updates for installation guides, API usage examples, and architectural diagrams
- Communication to development team about new import paths and deprecation timeline for legacy imports
- Validation of Swagger documentation generation and API client functionality post-consolidation