3aa0169af0
- Installed and configured SCST with 7 handlers - Installed and configured mhVTL with 2 Quantum libraries and 8 LTO-8 drives - Implemented all VTL API endpoints (8/9 working) - Fixed NULL device_path handling in drives endpoint - Added comprehensive error handling and validation - Implemented async tape load/unload operations - Created SCST installation guide for Ubuntu 24.04 - Created mhVTL installation and configuration guide - Added VTL testing guide and automated test scripts - All core API tests passing (89% success rate) Infrastructure status: - PostgreSQL: Configured with proper permissions - SCST: Active with kernel module loaded - mhVTL: 2 libraries (Quantum Scalar i500, Scalar i40) - mhVTL: 8 drives (all Quantum ULTRIUM-HH8 LTO-8) - Calypso API: 8/9 VTL endpoints functional Documentation added: - src/srs-technical-spec-documents/scst-installation.md - src/srs-technical-spec-documents/mhvtl-installation.md - VTL-TESTING-GUIDE.md - scripts/test-vtl.sh Co-Authored-By: Warp <agent@warp.dev>
183 lines
6.3 KiB
Markdown
183 lines
6.3 KiB
Markdown
# Phase C: Backend Core Domains - Progress Report
|
|
|
|
## Status: In Progress (Core Components Complete)
|
|
|
|
### ✅ Completed Components
|
|
|
|
#### 1. Database Schema (Migration 002)
|
|
- **File**: `backend/internal/common/database/migrations/002_storage_and_tape_schema.sql`
|
|
- **Tables Created**:
|
|
- `disk_repositories` - Disk-based backup repositories
|
|
- `physical_disks` - Physical disk inventory
|
|
- `volume_groups` - LVM volume groups
|
|
- `scst_targets` - SCST iSCSI targets
|
|
- `scst_luns` - LUN mappings
|
|
- `scst_initiator_groups` - Initiator groups
|
|
- `scst_initiators` - iSCSI initiators
|
|
- `physical_tape_libraries` - Physical tape library metadata
|
|
- `physical_tape_drives` - Physical tape drives
|
|
- `physical_tape_slots` - Tape slot inventory
|
|
- `virtual_tape_libraries` - Virtual tape library metadata
|
|
- `virtual_tape_drives` - Virtual tape drives
|
|
- `virtual_tapes` - Virtual tape inventory
|
|
|
|
#### 2. Storage Component ✅
|
|
- **Location**: `backend/internal/storage/`
|
|
- **Features**:
|
|
- **Disk Discovery** (`disk.go`):
|
|
- Physical disk detection via `lsblk`
|
|
- Disk information via `udevadm`
|
|
- Health status detection
|
|
- Database synchronization
|
|
- **LVM Management** (`lvm.go`):
|
|
- Volume group listing
|
|
- Repository creation (logical volumes)
|
|
- XFS filesystem creation
|
|
- Repository listing and retrieval
|
|
- Usage monitoring
|
|
- Repository deletion
|
|
- **API Handler** (`handler.go`):
|
|
- `GET /api/v1/storage/disks` - List physical disks
|
|
- `POST /api/v1/storage/disks/sync` - Sync disks to database (async)
|
|
- `GET /api/v1/storage/volume-groups` - List volume groups
|
|
- `GET /api/v1/storage/repositories` - List repositories
|
|
- `GET /api/v1/storage/repositories/:id` - Get repository
|
|
- `POST /api/v1/storage/repositories` - Create repository
|
|
- `DELETE /api/v1/storage/repositories/:id` - Delete repository
|
|
|
|
#### 3. SCST Integration ✅
|
|
- **Location**: `backend/internal/scst/`
|
|
- **Features**:
|
|
- **SCST Service** (`service.go`):
|
|
- Target creation and management
|
|
- LUN mapping (device to LUN)
|
|
- Initiator ACL management
|
|
- Single initiator enforcement for tape targets
|
|
- Handler detection
|
|
- Configuration persistence
|
|
- **API Handler** (`handler.go`):
|
|
- `GET /api/v1/scst/targets` - List all targets
|
|
- `GET /api/v1/scst/targets/:id` - Get target with LUNs
|
|
- `POST /api/v1/scst/targets` - Create new target
|
|
- `POST /api/v1/scst/targets/:id/luns` - Add LUN to target
|
|
- `POST /api/v1/scst/targets/:id/initiators` - Add initiator
|
|
- `POST /api/v1/scst/config/apply` - Apply SCST configuration (async)
|
|
- `GET /api/v1/scst/handlers` - List available handlers
|
|
|
|
#### 4. System Management ✅
|
|
- **Location**: `backend/internal/system/`
|
|
- **Features**:
|
|
- **System Service** (`service.go`):
|
|
- Systemd service status retrieval
|
|
- Service restart
|
|
- Journald log retrieval
|
|
- Support bundle generation
|
|
- **API Handler** (`handler.go`):
|
|
- `GET /api/v1/system/services` - List all services
|
|
- `GET /api/v1/system/services/:name` - Get service status
|
|
- `POST /api/v1/system/services/:name/restart` - Restart service
|
|
- `GET /api/v1/system/services/:name/logs` - Get service logs
|
|
- `POST /api/v1/system/support-bundle` - Generate support bundle (async)
|
|
|
|
### 🔄 In Progress / Pending
|
|
|
|
#### 5. Physical Tape Bridge (Pending)
|
|
- **Requirements** (SRS-02):
|
|
- Tape library discovery (changer + drives)
|
|
- Slot inventory and barcode handling
|
|
- Load/unload operations
|
|
- iSCSI export via SCST
|
|
- **Status**: Database schema ready, implementation pending
|
|
|
|
#### 6. Virtual Tape Library (Pending)
|
|
- **Requirements** (SRS-02):
|
|
- MHVTL integration
|
|
- Virtual tape management
|
|
- Tape image storage
|
|
- iSCSI export via SCST
|
|
- **Status**: Database schema ready, implementation pending
|
|
|
|
#### 7. Enhanced Monitoring (Pending)
|
|
- **Requirements** (SRS-05):
|
|
- Enhanced health checks
|
|
- Alerting engine
|
|
- Metrics collection
|
|
- **Status**: Basic health check exists, alerting engine pending
|
|
|
|
### API Endpoints Summary
|
|
|
|
#### Storage Endpoints
|
|
```
|
|
GET /api/v1/storage/disks
|
|
POST /api/v1/storage/disks/sync
|
|
GET /api/v1/storage/volume-groups
|
|
GET /api/v1/storage/repositories
|
|
GET /api/v1/storage/repositories/:id
|
|
POST /api/v1/storage/repositories
|
|
DELETE /api/v1/storage/repositories/:id
|
|
```
|
|
|
|
#### SCST Endpoints
|
|
```
|
|
GET /api/v1/scst/targets
|
|
GET /api/v1/scst/targets/:id
|
|
POST /api/v1/scst/targets
|
|
POST /api/v1/scst/targets/:id/luns
|
|
POST /api/v1/scst/targets/:id/initiators
|
|
POST /api/v1/scst/config/apply
|
|
GET /api/v1/scst/handlers
|
|
```
|
|
|
|
#### System Management Endpoints
|
|
```
|
|
GET /api/v1/system/services
|
|
GET /api/v1/system/services/:name
|
|
POST /api/v1/system/services/:name/restart
|
|
GET /api/v1/system/services/:name/logs
|
|
POST /api/v1/system/support-bundle
|
|
```
|
|
|
|
### Architecture Highlights
|
|
|
|
1. **Clean Separation**: Each domain has its own service and handler
|
|
2. **Async Operations**: Long-running operations use task engine
|
|
3. **SCST Integration**: Direct SCST command execution with database persistence
|
|
4. **Error Handling**: Comprehensive error handling with rollback support
|
|
5. **Audit Logging**: All mutating operations are audited (via middleware)
|
|
|
|
### Next Steps
|
|
|
|
1. **Physical Tape Bridge Implementation**:
|
|
- Implement `tape_physical/service.go` for discovery and operations
|
|
- Implement `tape_physical/handler.go` for API endpoints
|
|
- Add tape endpoints to router
|
|
|
|
2. **Virtual Tape Library Implementation**:
|
|
- Implement `tape_vtl/service.go` for MHVTL integration
|
|
- Implement `tape_vtl/handler.go` for API endpoints
|
|
- Add VTL endpoints to router
|
|
|
|
3. **Enhanced Monitoring**:
|
|
- Implement alerting engine
|
|
- Add metrics collection
|
|
- Enhance health checks
|
|
|
|
### Testing Recommendations
|
|
|
|
1. **Unit Tests**: Test each service independently
|
|
2. **Integration Tests**: Test SCST operations with mock commands
|
|
3. **E2E Tests**: Test full workflows (create repo → export via SCST)
|
|
|
|
### Known Limitations
|
|
|
|
1. **SCST Commands**: Currently uses direct `scstadmin` commands - may need abstraction for different SCST builds
|
|
2. **Error Recovery**: Some operations may need better rollback mechanisms
|
|
3. **Concurrency**: Need to ensure thread-safety for SCST operations
|
|
4. **Validation**: Additional input validation needed for some endpoints
|
|
|
|
---
|
|
|
|
**Last Updated**: Phase C - Core Components Complete
|
|
**Next Milestone**: Physical and Virtual Tape Library Implementation
|
|
|