300 lines
12 KiB
Markdown
300 lines
12 KiB
Markdown
# VTL Linux - Architecture & Design
|
|
|
|
## Overview
|
|
|
|
VTL Linux is an opinionated Linux distribution built specifically for Virtual Tape Library operations. It combines mhvtl (virtual tape library) with iSCSI target capabilities to provide enterprise-grade tape backup infrastructure over IP networks.
|
|
|
|
## Design Philosophy
|
|
|
|
### Opinionated Choices
|
|
|
|
1. **Debian-based**: Uses Debian Bookworm for stability and long-term support
|
|
2. **Minimal footprint**: Only essential packages included
|
|
3. **Pre-configured**: Ready-to-use mhvtl and iSCSI setup out of the box
|
|
4. **Performance-tuned**: Optimized kernel parameters for tape operations
|
|
5. **Network-first**: Designed for iSCSI connectivity from day one
|
|
|
|
### Target Use Cases
|
|
|
|
- Enterprise backup infrastructure
|
|
- Backup software testing and development
|
|
- Tape library simulation
|
|
- Disaster recovery testing
|
|
- Training environments
|
|
- Cost-effective alternative to physical tape libraries
|
|
|
|
## System Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ VTL Linux Host │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────┐ │
|
|
│ │ Kernel Space │ │
|
|
│ │ ┌────────────────────────────────────────────────┐ │ │
|
|
│ │ │ mhvtl Kernel Module │ │ │
|
|
│ │ │ - SCSI Target Framework │ │ │
|
|
│ │ │ - Virtual Device Emulation │ │ │
|
|
│ │ └────────────────────────────────────────────────┘ │ │
|
|
│ │ ┌────────────────────────────────────────────────┐ │ │
|
|
│ │ │ SCSI Generic (sg) Driver │ │ │
|
|
│ │ └────────────────────────────────────────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌──────────────────────────────────────────────────────┐ │
|
|
│ │ User Space │ │
|
|
│ │ ┌────────────────────────────────────────────────┐ │ │
|
|
│ │ │ mhvtl Daemons │ │ │
|
|
│ │ │ - vtltape (tape drive emulation) │ │ │
|
|
│ │ │ - vtllibrary (media changer emulation) │ │ │
|
|
│ │ └────────────────────────────────────────────────┘ │ │
|
|
│ │ ┌────────────────────────────────────────────────┐ │ │
|
|
│ │ │ iSCSI Target (tgt) │ │ │
|
|
│ │ │ - Target management │ │ │
|
|
│ │ │ - LUN mapping │ │ │
|
|
│ │ │ - Authentication (CHAP) │ │ │
|
|
│ │ └────────────────────────────────────────────────┘ │ │
|
|
│ │ ┌────────────────────────────────────────────────┐ │ │
|
|
│ │ │ Storage Backend │ │ │
|
|
│ │ │ /opt/mhvtl/ (tape data files) │ │ │
|
|
│ │ └────────────────────────────────────────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│
|
|
│ TCP/IP (iSCSI Protocol)
|
|
│ Port 3260
|
|
│
|
|
┌─────────────────┴─────────────────┐
|
|
│ │
|
|
┌───────▼────────┐ ┌────────▼───────┐
|
|
│ Linux Client │ │ Windows Client │
|
|
│ │ │ │
|
|
│ ┌──────────┐ │ │ ┌──────────┐ │
|
|
│ │ iSCSI │ │ │ │ iSCSI │ │
|
|
│ │Initiator │ │ │ │Initiator │ │
|
|
│ └──────────┘ │ │ └──────────┘ │
|
|
│ ┌──────────┐ │ │ ┌──────────┐ │
|
|
│ │ Backup │ │ │ │ Backup │ │
|
|
│ │ Software │ │ │ │ Software │ │
|
|
│ │ (Bacula, │ │ │ │ (Veeam, │ │
|
|
│ │ Amanda) │ │ │ │ Backup │ │
|
|
│ └──────────┘ │ │ │ Exec) │ │
|
|
└────────────────┘ │ └──────────┘ │
|
|
└────────────────┘
|
|
```
|
|
|
|
## Component Details
|
|
|
|
### mhvtl (Virtual Tape Library)
|
|
|
|
**Purpose**: Emulates physical tape drives and media changers
|
|
|
|
**Components**:
|
|
- Kernel module: Provides SCSI target framework
|
|
- vtltape daemon: Emulates tape drive behavior
|
|
- vtllibrary daemon: Emulates robotic media changer
|
|
- Configuration files: Define virtual devices and media
|
|
|
|
**Default Configuration**:
|
|
- 1x STK L700 library (media changer)
|
|
- 4x IBM LTO-5/6 tape drives
|
|
- 20x LTO-5 tape cartridges
|
|
- Compression enabled (LZO algorithm)
|
|
|
|
**Storage**:
|
|
- Tape data stored as files in `/opt/mhvtl/`
|
|
- Each tape is a separate file
|
|
- Supports multiple tape formats (LTO-3 through LTO-8)
|
|
|
|
### iSCSI Target (tgt)
|
|
|
|
**Purpose**: Exports SCSI devices over IP network
|
|
|
|
**Features**:
|
|
- Multi-target support
|
|
- CHAP authentication
|
|
- Access control lists
|
|
- Performance optimization
|
|
|
|
**Configuration**:
|
|
- Exports mhvtl SCSI devices as iSCSI LUNs
|
|
- Separate targets for each tape drive
|
|
- Dedicated target for media changer
|
|
- Configurable authentication
|
|
|
|
### Network Layer
|
|
|
|
**Protocol**: iSCSI (SCSI over TCP/IP)
|
|
**Port**: 3260 (standard iSCSI port)
|
|
**Authentication**: CHAP (Challenge-Handshake Authentication Protocol)
|
|
|
|
**Benefits**:
|
|
- No physical tape hardware required
|
|
- Remote access over LAN/WAN
|
|
- Multiple simultaneous clients
|
|
- Standard protocol support
|
|
|
|
## Data Flow
|
|
|
|
### Write Operation (Backup)
|
|
|
|
1. Backup software on client initiates write to tape
|
|
2. iSCSI initiator sends SCSI commands over network
|
|
3. iSCSI target receives commands on port 3260
|
|
4. Commands forwarded to mhvtl SCSI device
|
|
5. vtltape daemon processes write commands
|
|
6. Data compressed (if enabled) and written to file in `/opt/mhvtl/`
|
|
7. Acknowledgment sent back through iSCSI to client
|
|
|
|
### Read Operation (Restore)
|
|
|
|
1. Backup software requests tape mount
|
|
2. iSCSI sends media changer commands
|
|
3. vtllibrary daemon simulates robotic arm movement
|
|
4. Virtual tape "loaded" into virtual drive
|
|
5. Read commands processed by vtltape
|
|
6. Data decompressed and sent via iSCSI to client
|
|
|
|
## Performance Considerations
|
|
|
|
### Optimizations
|
|
|
|
1. **Kernel Parameters**:
|
|
- Increased network buffers
|
|
- TCP tuning for throughput
|
|
- Reduced swappiness
|
|
- I/O scheduler optimization
|
|
|
|
2. **Compression**:
|
|
- LZO compression (fast, good ratio)
|
|
- Configurable per drive
|
|
- Typical 3:1 compression ratio
|
|
|
|
3. **Network**:
|
|
- Jumbo frames support
|
|
- TCP window scaling
|
|
- Congestion control tuning
|
|
|
|
### Bottlenecks
|
|
|
|
- Network bandwidth (1Gbps recommended minimum)
|
|
- Disk I/O for tape storage
|
|
- CPU for compression/decompression
|
|
- Memory for buffering
|
|
|
|
## Security
|
|
|
|
### Authentication
|
|
|
|
- CHAP authentication for iSCSI
|
|
- Username/password per target
|
|
- Configurable initiator ACLs
|
|
|
|
### Network Security
|
|
|
|
- Firewall rules (port 3260)
|
|
- Optional VPN/IPsec for WAN
|
|
- Network segmentation recommended
|
|
|
|
### Access Control
|
|
|
|
- User permissions on tape storage
|
|
- Systemd service isolation
|
|
- SELinux/AppArmor support (optional)
|
|
|
|
## Scalability
|
|
|
|
### Vertical Scaling
|
|
|
|
- Add more virtual drives (up to 16 per library)
|
|
- Increase tape media count
|
|
- Larger storage backend
|
|
- More CPU/RAM for compression
|
|
|
|
### Horizontal Scaling
|
|
|
|
- Multiple VTL instances
|
|
- Load balancing across servers
|
|
- Distributed storage backend
|
|
- High availability clustering (future)
|
|
|
|
## Monitoring & Management
|
|
|
|
### System Monitoring
|
|
|
|
- systemd service status
|
|
- SCSI device enumeration
|
|
- iSCSI target status
|
|
- Storage utilization
|
|
|
|
### Tools Provided
|
|
|
|
- `vtl-status`: Comprehensive system status
|
|
- `lsscsi`: SCSI device listing
|
|
- `mtx`: Media changer control
|
|
- `tgt-admin`: iSCSI target management
|
|
|
|
### Logging
|
|
|
|
- systemd journal for all services
|
|
- mhvtl debug logging (configurable)
|
|
- iSCSI connection logs
|
|
- Kernel messages for SCSI events
|
|
|
|
## Future Enhancements
|
|
|
|
### Planned Features
|
|
|
|
- Web-based management interface
|
|
- Automated tape rotation policies
|
|
- Replication to cloud storage
|
|
- High availability clustering
|
|
- Performance metrics dashboard
|
|
- Tape encryption support
|
|
- Multi-tenancy support
|
|
|
|
### Integration Opportunities
|
|
|
|
- Prometheus metrics export
|
|
- Grafana dashboards
|
|
- Ansible playbooks
|
|
- Docker containerization
|
|
- Kubernetes operators
|
|
|
|
## Comparison with Physical Tape
|
|
|
|
### Advantages
|
|
|
|
- No hardware costs
|
|
- Instant provisioning
|
|
- Easy scaling
|
|
- Remote management
|
|
- No mechanical failures
|
|
- Faster seeks
|
|
- Snapshot/backup capability
|
|
|
|
### Limitations
|
|
|
|
- Not suitable for long-term archival (use real tape)
|
|
- Dependent on disk reliability
|
|
- Network latency vs. direct attach
|
|
- No physical off-site storage
|
|
- Software emulation overhead
|
|
|
|
## Best Practices
|
|
|
|
1. **Storage**: Use dedicated disk/partition for `/opt/mhvtl/`
|
|
2. **Network**: Dedicated network interface for iSCSI traffic
|
|
3. **Backup**: Regular backup of VTL configuration and metadata
|
|
4. **Monitoring**: Set up alerts for disk space and service status
|
|
5. **Security**: Change default passwords immediately
|
|
6. **Testing**: Verify backup/restore operations regularly
|
|
7. **Documentation**: Maintain inventory of virtual tapes and contents
|
|
|
|
## References
|
|
|
|
- mhvtl project: https://github.com/markh794/mhvtl
|
|
- iSCSI specification: RFC 3720
|
|
- SCSI Architecture Model: ANSI INCITS
|
|
- Linux SCSI Target Framework documentation
|