calypso/PHASE-C-PROGRESS.md

Phase C: Backend Core Domains - Progress Report

Status: In Progress (Core Components Complete)

✅ Completed Components

1. Database Schema (Migration 002)

• File: backend/internal/common/database/migrations/002_storage_and_tape_schema.sql
• Tables Created:
  • disk_repositories - Disk-based backup repositories
  • physical_disks - Physical disk inventory
  • volume_groups - LVM volume groups
  • scst_targets - SCST iSCSI targets
  • scst_luns - LUN mappings
  • scst_initiator_groups - Initiator groups
  • scst_initiators - iSCSI initiators
  • physical_tape_libraries - Physical tape library metadata
  • physical_tape_drives - Physical tape drives
  • physical_tape_slots - Tape slot inventory
  • virtual_tape_libraries - Virtual tape library metadata
  • virtual_tape_drives - Virtual tape drives
  • virtual_tapes - Virtual tape inventory

2. Storage Component ✅

• Location: backend/internal/storage/
• Features:
  • Disk Discovery (disk.go):
    • Physical disk detection via lsblk
    • Disk information via udevadm
    • Health status detection
    • Database synchronization
  • LVM Management (lvm.go):
    • Volume group listing
    • Repository creation (logical volumes)
    • XFS filesystem creation
    • Repository listing and retrieval
    • Usage monitoring
    • Repository deletion
  • API Handler (handler.go):
    • GET /api/v1/storage/disks - List physical disks
    • POST /api/v1/storage/disks/sync - Sync disks to database (async)
    • GET /api/v1/storage/volume-groups - List volume groups
    • GET /api/v1/storage/repositories - List repositories
    • GET /api/v1/storage/repositories/:id - Get repository
    • POST /api/v1/storage/repositories - Create repository
    • DELETE /api/v1/storage/repositories/:id - Delete repository

3. SCST Integration ✅

• Location: backend/internal/scst/
• Features:
  • SCST Service (service.go):
    • Target creation and management
    • LUN mapping (device to LUN)
    • Initiator ACL management
    • Single initiator enforcement for tape targets
    • Handler detection
    • Configuration persistence
  • API Handler (handler.go):
    • GET /api/v1/scst/targets - List all targets
    • GET /api/v1/scst/targets/:id - Get target with LUNs
    • POST /api/v1/scst/targets - Create new target
    • POST /api/v1/scst/targets/:id/luns - Add LUN to target
    • POST /api/v1/scst/targets/:id/initiators - Add initiator
    • POST /api/v1/scst/config/apply - Apply SCST configuration (async)
    • GET /api/v1/scst/handlers - List available handlers

4. System Management ✅

• Location: backend/internal/system/
• Features:
  • System Service (service.go):
    • Systemd service status retrieval
    • Service restart
    • Journald log retrieval
    • Support bundle generation
  • API Handler (handler.go):
    • GET /api/v1/system/services - List all services
    • GET /api/v1/system/services/:name - Get service status
    • POST /api/v1/system/services/:name/restart - Restart service
    • GET /api/v1/system/services/:name/logs - Get service logs
    • POST /api/v1/system/support-bundle - Generate support bundle (async)

🔄 In Progress / Pending

5. Physical Tape Bridge (Pending)

• Requirements (SRS-02):
  • Tape library discovery (changer + drives)
  • Slot inventory and barcode handling
  • Load/unload operations
  • iSCSI export via SCST
• Status: Database schema ready, implementation pending

6. Virtual Tape Library (Pending)

• Requirements (SRS-02):
  • MHVTL integration
  • Virtual tape management
  • Tape image storage
  • iSCSI export via SCST
• Status: Database schema ready, implementation pending

7. Enhanced Monitoring (Pending)

• Requirements (SRS-05):
  • Enhanced health checks
  • Alerting engine
  • Metrics collection
• Status: Basic health check exists, alerting engine pending

API Endpoints Summary

Storage Endpoints

GET    /api/v1/storage/disks
POST   /api/v1/storage/disks/sync
GET    /api/v1/storage/volume-groups
GET    /api/v1/storage/repositories
GET    /api/v1/storage/repositories/:id
POST   /api/v1/storage/repositories
DELETE /api/v1/storage/repositories/:id

SCST Endpoints

GET    /api/v1/scst/targets
GET    /api/v1/scst/targets/:id
POST   /api/v1/scst/targets
POST   /api/v1/scst/targets/:id/luns
POST   /api/v1/scst/targets/:id/initiators
POST   /api/v1/scst/config/apply
GET    /api/v1/scst/handlers

System Management Endpoints

GET    /api/v1/system/services
GET    /api/v1/system/services/:name
POST   /api/v1/system/services/:name/restart
GET    /api/v1/system/services/:name/logs
POST   /api/v1/system/support-bundle

Architecture Highlights

1. Clean Separation: Each domain has its own service and handler
2. Async Operations: Long-running operations use task engine
3. SCST Integration: Direct SCST command execution with database persistence
4. Error Handling: Comprehensive error handling with rollback support
5. Audit Logging: All mutating operations are audited (via middleware)

Next Steps

1. Physical Tape Bridge Implementation:

   • Implement tape_physical/service.go for discovery and operations
   • Implement tape_physical/handler.go for API endpoints
   • Add tape endpoints to router

2. Virtual Tape Library Implementation:

   • Implement tape_vtl/service.go for MHVTL integration
   • Implement tape_vtl/handler.go for API endpoints
   • Add VTL endpoints to router

3. Enhanced Monitoring:

   • Implement alerting engine
   • Add metrics collection
   • Enhance health checks

Testing Recommendations

1. Unit Tests: Test each service independently
2. Integration Tests: Test SCST operations with mock commands
3. E2E Tests: Test full workflows (create repo → export via SCST)

Known Limitations

1. SCST Commands: Currently uses direct scstadmin commands - may need abstraction for different SCST builds
2. Error Recovery: Some operations may need better rollback mechanisms
3. Concurrency: Need to ensure thread-safety for SCST operations
4. Validation: Additional input validation needed for some endpoints

---

Last Updated: Phase C - Core Components Complete Next Milestone: Physical and Virtual Tape Library Implementation