othman.suseno/adastra-vtl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Adastra VTL Linux Distribution- Complete build system for VTL-focused Linux distro- mhvtl integration with 4 LTO drives and media changer- iSCSI target configuration for network connectivity- Automated installation and configuration scripts- Comprehensive documentation (architecture, installation, configuration)- Systemd service files and kernel tuning- Quick start wizard for easy setup

Browse Source
This commit is contained in:
Othman Suseno
2025-12-08 17:16:34 +00:00
commit fc2fb763f5
12 changed files with 1759 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

299
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,299 @@
# VTL Linux - Architecture & Design
## Overview
VTL Linux is an opinionated Linux distribution built specifically for Virtual Tape Library operations. It combines mhvtl (virtual tape library) with iSCSI target capabilities to provide enterprise-grade tape backup infrastructure over IP networks.
## Design Philosophy
### Opinionated Choices
1. **Debian-based**: Uses Debian Bookworm for stability and long-term support
2. **Minimal footprint**: Only essential packages included
3. **Pre-configured**: Ready-to-use mhvtl and iSCSI setup out of the box
4. **Performance-tuned**: Optimized kernel parameters for tape operations
5. **Network-first**: Designed for iSCSI connectivity from day one
### Target Use Cases
- Enterprise backup infrastructure
- Backup software testing and development
- Tape library simulation
- Disaster recovery testing
- Training environments
- Cost-effective alternative to physical tape libraries
## System Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ VTL Linux Host │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Kernel Space │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ mhvtl Kernel Module │ │ │
│ │ │ - SCSI Target Framework │ │ │
│ │ │ - Virtual Device Emulation │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ SCSI Generic (sg) Driver │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
│ │ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ User Space │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ mhvtl Daemons │ │ │
│ │ │ - vtltape (tape drive emulation) │ │ │
│ │ │ - vtllibrary (media changer emulation) │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ iSCSI Target (tgt) │ │ │
│ │ │ - Target management │ │ │
│ │ │ - LUN mapping │ │ │
│ │ │ - Authentication (CHAP) │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ Storage Backend │ │ │
│ │ │ /opt/mhvtl/ (tape data files) │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
│ TCP/IP (iSCSI Protocol)
│ Port 3260
│
┌─────────────────┴─────────────────┐
│ │
┌───────▼────────┐ ┌────────▼───────┐
│ Linux Client │ │ Windows Client │
│ │ │ │
│ ┌──────────┐ │ │ ┌──────────┐ │
│ │ iSCSI │ │ │ │ iSCSI │ │
│ │Initiator │ │ │ │Initiator │ │
│ └──────────┘ │ │ └──────────┘ │
│ ┌──────────┐ │ │ ┌──────────┐ │
│ │ Backup │ │ │ │ Backup │ │
│ │ Software │ │ │ │ Software │ │
│ │ (Bacula, │ │ │ │ (Veeam, │ │
│ │ Amanda) │ │ │ │ Backup │ │
│ └──────────┘ │ │ │ Exec) │ │
└────────────────┘ │ └──────────┘ │
└────────────────┘
```
## Component Details
### mhvtl (Virtual Tape Library)
**Purpose**: Emulates physical tape drives and media changers
**Components**:
- Kernel module: Provides SCSI target framework
- vtltape daemon: Emulates tape drive behavior
- vtllibrary daemon: Emulates robotic media changer
- Configuration files: Define virtual devices and media
**Default Configuration**:
- 1x STK L700 library (media changer)
- 4x IBM LTO-5/6 tape drives
- 20x LTO-5 tape cartridges
- Compression enabled (LZO algorithm)
**Storage**:
- Tape data stored as files in `/opt/mhvtl/`
- Each tape is a separate file
- Supports multiple tape formats (LTO-3 through LTO-8)
### iSCSI Target (tgt)
**Purpose**: Exports SCSI devices over IP network
**Features**:
- Multi-target support
- CHAP authentication
- Access control lists
- Performance optimization
**Configuration**:
- Exports mhvtl SCSI devices as iSCSI LUNs
- Separate targets for each tape drive
- Dedicated target for media changer
- Configurable authentication
### Network Layer
**Protocol**: iSCSI (SCSI over TCP/IP)
**Port**: 3260 (standard iSCSI port)
**Authentication**: CHAP (Challenge-Handshake Authentication Protocol)
**Benefits**:
- No physical tape hardware required
- Remote access over LAN/WAN
- Multiple simultaneous clients
- Standard protocol support
## Data Flow
### Write Operation (Backup)
1. Backup software on client initiates write to tape
2. iSCSI initiator sends SCSI commands over network
3. iSCSI target receives commands on port 3260
4. Commands forwarded to mhvtl SCSI device
5. vtltape daemon processes write commands
6. Data compressed (if enabled) and written to file in `/opt/mhvtl/`
7. Acknowledgment sent back through iSCSI to client
### Read Operation (Restore)
1. Backup software requests tape mount
2. iSCSI sends media changer commands
3. vtllibrary daemon simulates robotic arm movement
4. Virtual tape "loaded" into virtual drive
5. Read commands processed by vtltape
6. Data decompressed and sent via iSCSI to client
## Performance Considerations
### Optimizations
1. **Kernel Parameters**:
- Increased network buffers
- TCP tuning for throughput
- Reduced swappiness
- I/O scheduler optimization
2. **Compression**:
- LZO compression (fast, good ratio)
- Configurable per drive
- Typical 3:1 compression ratio
3. **Network**:
- Jumbo frames support
- TCP window scaling
- Congestion control tuning
### Bottlenecks
- Network bandwidth (1Gbps recommended minimum)
- Disk I/O for tape storage
- CPU for compression/decompression
- Memory for buffering
## Security
### Authentication
- CHAP authentication for iSCSI
- Username/password per target
- Configurable initiator ACLs
### Network Security
- Firewall rules (port 3260)
- Optional VPN/IPsec for WAN
- Network segmentation recommended
### Access Control
- User permissions on tape storage
- Systemd service isolation
- SELinux/AppArmor support (optional)
## Scalability
### Vertical Scaling
- Add more virtual drives (up to 16 per library)
- Increase tape media count
- Larger storage backend
- More CPU/RAM for compression
### Horizontal Scaling
- Multiple VTL instances
- Load balancing across servers
- Distributed storage backend
- High availability clustering (future)
## Monitoring & Management
### System Monitoring
- systemd service status
- SCSI device enumeration
- iSCSI target status
- Storage utilization
### Tools Provided
- `vtl-status`: Comprehensive system status
- `lsscsi`: SCSI device listing
- `mtx`: Media changer control
- `tgt-admin`: iSCSI target management
### Logging
- systemd journal for all services
- mhvtl debug logging (configurable)
- iSCSI connection logs
- Kernel messages for SCSI events
## Future Enhancements
### Planned Features
- Web-based management interface
- Automated tape rotation policies
- Replication to cloud storage
- High availability clustering
- Performance metrics dashboard
- Tape encryption support
- Multi-tenancy support
### Integration Opportunities
- Prometheus metrics export
- Grafana dashboards
- Ansible playbooks
- Docker containerization
- Kubernetes operators
## Comparison with Physical Tape
### Advantages
- No hardware costs
- Instant provisioning
- Easy scaling
- Remote management
- No mechanical failures
- Faster seeks
- Snapshot/backup capability
### Limitations
- Not suitable for long-term archival (use real tape)
- Dependent on disk reliability
- Network latency vs. direct attach
- No physical off-site storage
- Software emulation overhead
## Best Practices
1. **Storage**: Use dedicated disk/partition for `/opt/mhvtl/`
2. **Network**: Dedicated network interface for iSCSI traffic
3. **Backup**: Regular backup of VTL configuration and metadata
4. **Monitoring**: Set up alerts for disk space and service status
5. **Security**: Change default passwords immediately
6. **Testing**: Verify backup/restore operations regularly
7. **Documentation**: Maintain inventory of virtual tapes and contents
## References
- mhvtl project: https://github.com/markh794/mhvtl
- iSCSI specification: RFC 3720
- SCSI Architecture Model: ANSI INCITS
- Linux SCSI Target Framework documentation