othman.suseno/vtl-appliance
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Adastra VTL Linux Distribution- Complete build system for VTL-focused Linux distro- mhvtl integration with 4 LTO drives and media changer- iSCSI target configuration for network connectivity- Automated installation and configuration scripts- Comprehensive documentation (architecture, installation, configuration)- Systemd service files and kernel tuning- Quick start wizard for easy setup

Browse Source
This commit is contained in:
Othman Suseno
2025-12-08 17:16:34 +00:00
commit fc2fb763f5
12 changed files with 1759 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
.gitignore vendored Normal file
View File

@@ -0,0 +1,11 @@
*.iso
*.img
iso/
*.log
*.tmp
*~
.DS_Store
*.swp
*.swo
/build/work/
/build/chroot/

109
README.md Normal file
View File

@@ -0,0 +1,109 @@
# VTL Linux Distro
An opinionated Linux distribution specifically designed for Virtual Tape Library (VTL) operations using mhvtl with iSCSI connectivity.
## Features
- **mhvtl Integration**: Pre-configured virtual tape library with multiple tape drives and media changer
- **iSCSI Target**: Network-based storage connectivity for Linux and Windows clients
- **Optimized Performance**: Tuned kernel parameters for tape operations
- **Easy Deployment**: Automated installation and configuration scripts
- **Minimal Footprint**: Stripped-down base system focused on VTL operations
## Architecture
```
┌─────────────────────────────────────┐
│ VTL Linux Distro (Server) │
│ ┌──────────────────────────────┐ │
│ │ mhvtl │ │
│ │ - Virtual Tape Drives │ │
│ │ - Media Changer │ │
│ │ - Tape Media Management │ │
│ └──────────────────────────────┘ │
│ ┌──────────────────────────────┐ │
│ │ iSCSI Target (tgt) │ │
│ │ - Network Block Devices │ │
│ │ - Multi-client Support │ │
│ └──────────────────────────────┘ │
└─────────────────────────────────────┘
│
│ iSCSI (IP Network)
│
┌──────┴──────┐
│ │
┌───▼────┐ ┌───▼────┐
│ Linux │ │Windows │
│ Client │ │ Client │
└────────┘ └────────┘
```
## Directory Structure
```
/vtl/
├── build/ # Build scripts and configurations
├── config/ # System configuration files
├── scripts/ # Installation and management scripts
├── systemd/ # Systemd service files
├── docs/ # Documentation
└── iso/ # ISO build output
```
## Quick Start
### Building the ISO
```bash
cd /vtl
sudo ./build/build-iso.sh
```
### Installation
1. Boot from the generated ISO
2. Follow the installation prompts
3. Configure network settings
4. Set up iSCSI targets and mhvtl devices
### Connecting Clients
**Linux Client:**
```bash
iscsiadm -m discovery -t st -p <VTL_IP>:3260
iscsiadm -m node --login
```
**Windows Client:**
```powershell
iscsicli QAddTargetPortal <VTL_IP>
iscsicli ListTargets
iscsicli LoginTarget <target_name>
```
## Configuration
### mhvtl Configuration
- Default library: `/etc/mhvtl/`
- Tape storage: `/opt/mhvtl/`
- Supported drives: LTO-5, LTO-6, LTO-7, LTO-8
### iSCSI Configuration
- Target configuration: `/etc/tgt/conf.d/`
- Default port: 3260
- Authentication: CHAP supported
## Requirements
- x86_64 architecture
- Minimum 2GB RAM
- 50GB+ storage for tape media
- Network interface for iSCSI
## License
MIT License
## Support
For issues and questions, please refer to the documentation in `/vtl/docs/`

176
build/build-iso.sh Executable file
View File

@@ -0,0 +1,176 @@
#!/bin/bash
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
ISO_DIR="$PROJECT_ROOT/iso"
BUILD_DIR="$PROJECT_ROOT/build"
WORK_DIR="/tmp/vtl-build"
DISTRO_NAME="VTL-Linux"
DISTRO_VERSION="1.0"
ISO_NAME="${DISTRO_NAME}-${DISTRO_VERSION}-x86_64.iso"
echo "=========================================="
echo " VTL Linux Distro Builder"
echo " Version: $DISTRO_VERSION"
echo "=========================================="
echo ""
if [ "$EUID" -ne 0 ]; then
echo "Error: This script must be run as root"
exit 1
fi
echo "[1/8] Checking dependencies..."
REQUIRED_PACKAGES="debootstrap squashfs-tools xorriso isolinux syslinux-common"
for pkg in $REQUIRED_PACKAGES; do
if ! dpkg -l | grep -q "^ii $pkg"; then
echo "Installing $pkg..."
apt-get update -qq
apt-get install -y $pkg
fi
done
echo "[2/8] Cleaning previous build..."
rm -rf "$WORK_DIR"
mkdir -p "$WORK_DIR"/{chroot,iso/{live,isolinux}}
echo "[3/8] Creating base system with debootstrap..."
debootstrap --arch=amd64 --variant=minbase bookworm "$WORK_DIR/chroot" http://deb.debian.org/debian/
echo "[4/8] Configuring chroot environment..."
mount --bind /dev "$WORK_DIR/chroot/dev"
mount --bind /dev/pts "$WORK_DIR/chroot/dev/pts"
mount --bind /proc "$WORK_DIR/chroot/proc"
mount --bind /sys "$WORK_DIR/chroot/sys"
cat > "$WORK_DIR/chroot/etc/apt/sources.list" << EOF
deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware
deb http://security.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware
EOF
echo "[5/8] Installing VTL packages in chroot..."
chroot "$WORK_DIR/chroot" /bin/bash << 'CHROOT_EOF'
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y \
linux-image-amd64 \
live-boot \
systemd-sysv \
network-manager \
openssh-server \
sudo \
vim \
curl \
wget \
net-tools \
iproute2 \
iptables \
sg3-utils \
lsscsi \
tgt \
open-iscsi \
build-essential \
git \
zlib1g-dev \
libibverbs-dev \
libconfig-dev \
libssl-dev \
uuid-dev \
mtx \
mt-st \
lsscsi
apt-get clean
rm -rf /var/lib/apt/lists/*
echo "vtl-linux" > /etc/hostname
cat > /etc/hosts << EOF
127.0.0.1 localhost
127.0.1.1 vtl-linux
EOF
useradd -m -s /bin/bash -G sudo vtladmin
echo "vtladmin:vtladmin" | chpasswd
echo "root:vtlroot" | chpasswd
systemctl enable NetworkManager
systemctl enable ssh
systemctl enable tgt
mkdir -p /opt/mhvtl
mkdir -p /etc/mhvtl
CHROOT_EOF
echo "[6/8] Copying configuration files..."
cp -r "$PROJECT_ROOT/config/"* "$WORK_DIR/chroot/tmp/" 2>/dev/null || true
cp -r "$PROJECT_ROOT/scripts/"* "$WORK_DIR/chroot/usr/local/bin/" 2>/dev/null || true
cp -r "$PROJECT_ROOT/systemd/"* "$WORK_DIR/chroot/etc/systemd/system/" 2>/dev/null || true
echo "[7/8] Creating squashfs filesystem..."
mksquashfs "$WORK_DIR/chroot" "$WORK_DIR/iso/live/filesystem.squashfs" -e boot -comp xz
cp "$WORK_DIR/chroot/boot"/vmlinuz-* "$WORK_DIR/iso/live/vmlinuz"
cp "$WORK_DIR/chroot/boot"/initrd.img-* "$WORK_DIR/iso/live/initrd"
cat > "$WORK_DIR/iso/isolinux/isolinux.cfg" << 'EOF'
UI menu.c32
PROMPT 0
TIMEOUT 100
DEFAULT vtl-linux
MENU TITLE VTL Linux Boot Menu
LABEL vtl-linux
MENU LABEL ^Start VTL Linux
KERNEL /live/vmlinuz
APPEND initrd=/live/initrd boot=live quiet splash
LABEL vtl-linux-safe
MENU LABEL Start VTL Linux (^Safe Mode)
KERNEL /live/vmlinuz
APPEND initrd=/live/initrd boot=live quiet splash nomodeset
LABEL vtl-linux-install
MENU LABEL ^Install VTL Linux to Disk
KERNEL /live/vmlinuz
APPEND initrd=/live/initrd boot=live quiet splash toram
EOF
cp /usr/lib/ISOLINUX/isolinux.bin "$WORK_DIR/iso/isolinux/"
cp /usr/lib/syslinux/modules/bios/*.c32 "$WORK_DIR/iso/isolinux/"
echo "[8/8] Creating ISO image..."
xorriso -as mkisofs \
-iso-level 3 \
-full-iso9660-filenames \
-volid "${DISTRO_NAME}" \
-eltorito-boot isolinux/isolinux.bin \
-eltorito-catalog isolinux/boot.cat \
-no-emul-boot \
-boot-load-size 4 \
-boot-info-table \
-isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin \
-output "$ISO_DIR/$ISO_NAME" \
"$WORK_DIR/iso"
echo ""
echo "=========================================="
echo " Build Complete!"
echo "=========================================="
echo "ISO Location: $ISO_DIR/$ISO_NAME"
echo "ISO Size: $(du -h "$ISO_DIR/$ISO_NAME" | cut -f1)"
echo ""
echo "Cleaning up..."
umount "$WORK_DIR/chroot/dev/pts" 2>/dev/null || true
umount "$WORK_DIR/chroot/dev" 2>/dev/null || true
umount "$WORK_DIR/chroot/proc" 2>/dev/null || true
umount "$WORK_DIR/chroot/sys" 2>/dev/null || true
rm -rf "$WORK_DIR"
echo "Done!"

17
config/sysctl-vtl.conf Normal file
View File

@@ -0,0 +1,17 @@
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.ipv4.tcp_rmem = 4096 87380 67108864
net.ipv4.tcp_wmem = 4096 65536 67108864
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_mtu_probing = 1
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_no_metrics_save = 1
vm.swappiness = 10
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5
kernel.sched_migration_cost_ns = 5000000
kernel.sched_autogroup_enabled = 0

299
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,299 @@
# VTL Linux - Architecture & Design
## Overview
VTL Linux is an opinionated Linux distribution built specifically for Virtual Tape Library operations. It combines mhvtl (virtual tape library) with iSCSI target capabilities to provide enterprise-grade tape backup infrastructure over IP networks.
## Design Philosophy
### Opinionated Choices
1. **Debian-based**: Uses Debian Bookworm for stability and long-term support
2. **Minimal footprint**: Only essential packages included
3. **Pre-configured**: Ready-to-use mhvtl and iSCSI setup out of the box
4. **Performance-tuned**: Optimized kernel parameters for tape operations
5. **Network-first**: Designed for iSCSI connectivity from day one
### Target Use Cases
- Enterprise backup infrastructure
- Backup software testing and development
- Tape library simulation
- Disaster recovery testing
- Training environments
- Cost-effective alternative to physical tape libraries
## System Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ VTL Linux Host │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Kernel Space │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ mhvtl Kernel Module │ │ │
│ │ │ - SCSI Target Framework │ │ │
│ │ │ - Virtual Device Emulation │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ SCSI Generic (sg) Driver │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
│ │ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ User Space │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ mhvtl Daemons │ │ │
│ │ │ - vtltape (tape drive emulation) │ │ │
│ │ │ - vtllibrary (media changer emulation) │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ iSCSI Target (tgt) │ │ │
│ │ │ - Target management │ │ │
│ │ │ - LUN mapping │ │ │
│ │ │ - Authentication (CHAP) │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
│ │ │ Storage Backend │ │ │
│ │ │ /opt/mhvtl/ (tape data files) │ │ │
│ │ └────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
│ TCP/IP (iSCSI Protocol)
│ Port 3260
│
┌─────────────────┴─────────────────┐
│ │
┌───────▼────────┐ ┌────────▼───────┐
│ Linux Client │ │ Windows Client │
│ │ │ │
│ ┌──────────┐ │ │ ┌──────────┐ │
│ │ iSCSI │ │ │ │ iSCSI │ │
│ │Initiator │ │ │ │Initiator │ │
│ └──────────┘ │ │ └──────────┘ │
│ ┌──────────┐ │ │ ┌──────────┐ │
│ │ Backup │ │ │ │ Backup │ │
│ │ Software │ │ │ │ Software │ │
│ │ (Bacula, │ │ │ │ (Veeam, │ │
│ │ Amanda) │ │ │ │ Backup │ │
│ └──────────┘ │ │ │ Exec) │ │
└────────────────┘ │ └──────────┘ │
└────────────────┘
```
## Component Details
### mhvtl (Virtual Tape Library)
**Purpose**: Emulates physical tape drives and media changers
**Components**:
- Kernel module: Provides SCSI target framework
- vtltape daemon: Emulates tape drive behavior
- vtllibrary daemon: Emulates robotic media changer
- Configuration files: Define virtual devices and media
**Default Configuration**:
- 1x STK L700 library (media changer)
- 4x IBM LTO-5/6 tape drives
- 20x LTO-5 tape cartridges
- Compression enabled (LZO algorithm)
**Storage**:
- Tape data stored as files in `/opt/mhvtl/`
- Each tape is a separate file
- Supports multiple tape formats (LTO-3 through LTO-8)
### iSCSI Target (tgt)
**Purpose**: Exports SCSI devices over IP network
**Features**:
- Multi-target support
- CHAP authentication
- Access control lists
- Performance optimization
**Configuration**:
- Exports mhvtl SCSI devices as iSCSI LUNs
- Separate targets for each tape drive
- Dedicated target for media changer
- Configurable authentication
### Network Layer
**Protocol**: iSCSI (SCSI over TCP/IP)
**Port**: 3260 (standard iSCSI port)
**Authentication**: CHAP (Challenge-Handshake Authentication Protocol)
**Benefits**:
- No physical tape hardware required
- Remote access over LAN/WAN
- Multiple simultaneous clients
- Standard protocol support
## Data Flow
### Write Operation (Backup)
1. Backup software on client initiates write to tape
2. iSCSI initiator sends SCSI commands over network
3. iSCSI target receives commands on port 3260
4. Commands forwarded to mhvtl SCSI device
5. vtltape daemon processes write commands
6. Data compressed (if enabled) and written to file in `/opt/mhvtl/`
7. Acknowledgment sent back through iSCSI to client
### Read Operation (Restore)
1. Backup software requests tape mount
2. iSCSI sends media changer commands
3. vtllibrary daemon simulates robotic arm movement
4. Virtual tape "loaded" into virtual drive
5. Read commands processed by vtltape
6. Data decompressed and sent via iSCSI to client
## Performance Considerations
### Optimizations
1. **Kernel Parameters**:
- Increased network buffers
- TCP tuning for throughput
- Reduced swappiness
- I/O scheduler optimization
2. **Compression**:
- LZO compression (fast, good ratio)
- Configurable per drive
- Typical 3:1 compression ratio
3. **Network**:
- Jumbo frames support
- TCP window scaling
- Congestion control tuning
### Bottlenecks
- Network bandwidth (1Gbps recommended minimum)
- Disk I/O for tape storage
- CPU for compression/decompression
- Memory for buffering
## Security
### Authentication
- CHAP authentication for iSCSI
- Username/password per target
- Configurable initiator ACLs
### Network Security
- Firewall rules (port 3260)
- Optional VPN/IPsec for WAN
- Network segmentation recommended
### Access Control
- User permissions on tape storage
- Systemd service isolation
- SELinux/AppArmor support (optional)
## Scalability
### Vertical Scaling
- Add more virtual drives (up to 16 per library)
- Increase tape media count
- Larger storage backend
- More CPU/RAM for compression
### Horizontal Scaling
- Multiple VTL instances
- Load balancing across servers
- Distributed storage backend
- High availability clustering (future)
## Monitoring & Management
### System Monitoring
- systemd service status
- SCSI device enumeration
- iSCSI target status
- Storage utilization
### Tools Provided
- `vtl-status`: Comprehensive system status
- `lsscsi`: SCSI device listing
- `mtx`: Media changer control
- `tgt-admin`: iSCSI target management
### Logging
- systemd journal for all services
- mhvtl debug logging (configurable)
- iSCSI connection logs
- Kernel messages for SCSI events
## Future Enhancements
### Planned Features
- Web-based management interface
- Automated tape rotation policies
- Replication to cloud storage
- High availability clustering
- Performance metrics dashboard
- Tape encryption support
- Multi-tenancy support
### Integration Opportunities
- Prometheus metrics export
- Grafana dashboards
- Ansible playbooks
- Docker containerization
- Kubernetes operators
## Comparison with Physical Tape
### Advantages
- No hardware costs
- Instant provisioning
- Easy scaling
- Remote management
- No mechanical failures
- Faster seeks
- Snapshot/backup capability
### Limitations
- Not suitable for long-term archival (use real tape)
- Dependent on disk reliability
- Network latency vs. direct attach
- No physical off-site storage
- Software emulation overhead
## Best Practices
1. **Storage**: Use dedicated disk/partition for `/opt/mhvtl/`
2. **Network**: Dedicated network interface for iSCSI traffic
3. **Backup**: Regular backup of VTL configuration and metadata
4. **Monitoring**: Set up alerts for disk space and service status
5. **Security**: Change default passwords immediately
6. **Testing**: Verify backup/restore operations regularly
7. **Documentation**: Maintain inventory of virtual tapes and contents
## References
- mhvtl project: https://github.com/markh794/mhvtl
- iSCSI specification: RFC 3720
- SCSI Architecture Model: ANSI INCITS
- Linux SCSI Target Framework documentation

408
docs/CONFIGURATION.md Normal file
View File

@@ -0,0 +1,408 @@
# VTL Linux - Configuration Examples
## mhvtl Device Configuration
### Basic LTO-5 Library Setup
```conf
VERSION: 5
Library: 10 CHANNEL: 00 TARGET: 00 LUN: 00
Vendor identification: STK
Product identification: L700
Unit serial number: XYZZY_A
NAA: 10:22:33:44:ab:cd:ef:00
Home directory: /opt/mhvtl
Backoff: 400
Drive: 00 CHANNEL: 00 TARGET: 01 LUN: 00
Library ID: 10 Slot: 01
Vendor identification: IBM
Product identification: ULT3580-TD5
Unit serial number: XYZZY_A1
NAA: 10:22:33:44:ab:cd:ef:01
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
```
### Multi-Drive LTO-6/7/8 Setup
```conf
VERSION: 5
Library: 20 CHANNEL: 00 TARGET: 00 LUN: 00
Vendor identification: IBM
Product identification: 03584L32
Unit serial number: XYZZY_B
NAA: 20:22:33:44:ab:cd:ef:00
Home directory: /opt/mhvtl
Backoff: 400
Drive: 10 CHANNEL: 00 TARGET: 01 LUN: 00
Library ID: 20 Slot: 01
Vendor identification: IBM
Product identification: ULT3580-TD6
Unit serial number: XYZZY_B1
NAA: 20:22:33:44:ab:cd:ef:01
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
Drive: 11 CHANNEL: 00 TARGET: 02 LUN: 00
Library ID: 20 Slot: 02
Vendor identification: IBM
Product identification: ULT3580-TD7
Unit serial number: XYZZY_B2
NAA: 20:22:33:44:ab:cd:ef:02
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
Drive: 12 CHANNEL: 00 TARGET: 03 LUN: 00
Library ID: 20 Slot: 03
Vendor identification: IBM
Product identification: ULT3580-TD8
Unit serial number: XYZZY_B3
NAA: 20:22:33:44:ab:cd:ef:03
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
```
## iSCSI Target Configuration
### Basic Target with CHAP Authentication
```conf
<target iqn.2024-01.com.vtl-linux:vtl.drive0>
backing-store /dev/sg1
initiator-address ALL
incominguser vtl-user vtl-password
write-cache on
</target>
```
### Target with IP Restrictions
```conf
<target iqn.2024-01.com.vtl-linux:vtl.drive0>
backing-store /dev/sg1
initiator-address 192.168.1.0/24
initiator-address 10.0.0.50
incominguser backup-server secure-password-here
write-cache on
</target>
```
### Multiple Targets for Different Clients
```conf
<target iqn.2024-01.com.vtl-linux:vtl.client1>
backing-store /dev/sg1
initiator-address 192.168.1.100
incominguser client1 password1
write-cache on
</target>
<target iqn.2024-01.com.vtl-linux:vtl.client2>
backing-store /dev/sg2
initiator-address 192.168.1.101
incominguser client2 password2
write-cache on
</target>
<target iqn.2024-01.com.vtl-linux:vtl.changer>
backing-store /dev/sg0
initiator-address 192.168.1.0/24
incominguser vtl-admin admin-password
device-type changer
</target>
```
### Target with Mutual CHAP
```conf
<target iqn.2024-01.com.vtl-linux:vtl.secure>
backing-store /dev/sg1
initiator-address 192.168.1.100
incominguser vtl-user vtl-password
outgoinguser initiator-user initiator-password
write-cache on
</target>
```
## Kernel Tuning
### High-Performance Network Configuration
```conf
net.core.rmem_max = 268435456
net.core.wmem_max = 268435456
net.core.rmem_default = 33554432
net.core.wmem_default = 33554432
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_mtu_probing = 1
net.core.netdev_max_backlog = 10000
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.core.default_qdisc = fq
```
### Storage-Optimized Configuration
```conf
vm.swappiness = 1
vm.dirty_ratio = 10
vm.dirty_background_ratio = 3
vm.vfs_cache_pressure = 50
kernel.sched_migration_cost_ns = 5000000
kernel.sched_autogroup_enabled = 0
```
## Backup Software Integration
### Bacula Configuration
```conf
Autochanger {
Name = VTL-Library
Device = Drive-0, Drive-1, Drive-2, Drive-3
Changer Command = "/usr/lib/bacula/scripts/mtx-changer %c %o %S %a %d"
Changer Device = /dev/sg0
}
Device {
Name = Drive-0
Media Type = LTO-5
Archive Device = /dev/nst0
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
RandomAccess = no
AutoChanger = yes
Drive Index = 0
Maximum Spool Size = 10G
Spool Directory = /var/spool/bacula
}
Device {
Name = Drive-1
Media Type = LTO-5
Archive Device = /dev/nst1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
RandomAccess = no
AutoChanger = yes
Drive Index = 1
Maximum Spool Size = 10G
Spool Directory = /var/spool/bacula
}
```
### Amanda Configuration
```conf
tapedev "chg-robot:/dev/sg0"
tpchanger "chg-robot"
changerfile "/var/lib/amanda/vtl/changer"
changerdev "/dev/sg0"
tapetype LTO-5
define tapetype LTO-5 {
comment "LTO-5 Virtual Tape"
length 1500000 mbytes
filemark 0 kbytes
speed 140000 kps
}
labelstr "^VTL-[0-9][0-9]*$"
autolabel "VTL-%%%" EMPTY VOLUME_ERROR
```
### Veritas Backup Exec (Windows)
1. Configure iSCSI initiator to connect to VTL server
2. In Backup Exec, go to Storage → Configure Storage
3. Select "Tape Drive" → "Detect and configure"
4. Backup Exec will auto-detect the tape library
5. Configure media sets and backup jobs
## Network Configuration Examples
### Static IP Configuration (NetworkManager)
```bash
nmcli con add type ethernet con-name vtl-network ifname eth0 \
ipv4.addresses 192.168.1.100/24 \
ipv4.gateway 192.168.1.1 \
ipv4.dns "8.8.8.8,8.8.4.4" \
ipv4.method manual
nmcli con up vtl-network
```
### Bonded Network Interface
```bash
nmcli con add type bond con-name bond0 ifname bond0 mode active-backup
nmcli con add type ethernet con-name bond0-slave1 ifname eth0 master bond0
nmcli con add type ethernet con-name bond0-slave2 ifname eth1 master bond0
nmcli con mod bond0 ipv4.addresses 192.168.1.100/24 \
ipv4.gateway 192.168.1.1 \
ipv4.method manual
nmcli con up bond0
```
### VLAN Configuration
```bash
nmcli con add type vlan con-name vlan100 ifname eth0.100 dev eth0 id 100
nmcli con mod vlan100 ipv4.addresses 192.168.100.100/24 \
ipv4.method manual
nmcli con up vlan100
```
## Firewall Configuration
### UFW (Ubuntu/Debian)
```bash
ufw allow from 192.168.1.0/24 to any port 3260 proto tcp
ufw allow 22/tcp
ufw enable
```
### firewalld (RHEL/CentOS)
```bash
firewall-cmd --permanent --add-port=3260/tcp
firewall-cmd --permanent --add-service=ssh
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.0/24" port port="3260" protocol="tcp" accept'
firewall-cmd --reload
```
### iptables
```bash
iptables -A INPUT -p tcp -s 192.168.1.0/24 --dport 3260 -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -j DROP
iptables-save > /etc/iptables/rules.v4
```
## Monitoring Scripts
### Tape Usage Monitor
```bash
#!/bin/bash
MHVTL_DIR="/opt/mhvtl"
THRESHOLD=80
usage=$(df -h "$MHVTL_DIR" | awk 'NR==2 {print $5}' | sed 's/%//')
if [ "$usage" -gt "$THRESHOLD" ]; then
echo "WARNING: VTL storage usage at ${usage}%"
echo "Consider adding more disk space or removing old tapes"
fi
echo "Current tape inventory:"
ls -lh "$MHVTL_DIR"/*.data 2>/dev/null | wc -l
```
### iSCSI Connection Monitor
```bash
#!/bin/bash
echo "Active iSCSI connections:"
netstat -tn | grep :3260 | grep ESTABLISHED | wc -l
echo ""
echo "Connection details:"
netstat -tn | grep :3260 | grep ESTABLISHED
```
## Systemd Service Customization
### Custom mhvtl Service with Resource Limits
```ini
[Unit]
Description=mhvtl Virtual Tape Library
After=network.target
[Service]
Type=forking
ExecStartPre=/sbin/modprobe mhvtl
ExecStart=/usr/bin/vtltape
ExecStart=/usr/bin/vtllibrary
ExecStop=/usr/bin/killall vtltape vtllibrary
Restart=on-failure
RestartSec=5s
CPUQuota=50%
MemoryLimit=2G
IOWeight=500
[Install]
WantedBy=multi-user.target
```
### Auto-restart on Failure
```ini
[Service]
Restart=always
RestartSec=10s
StartLimitInterval=200
StartLimitBurst=5
```
## Maintenance Scripts
### Tape Cleanup Script
```bash
#!/bin/bash
MHVTL_DIR="/opt/mhvtl"
DAYS_OLD=90
echo "Removing tapes older than $DAYS_OLD days..."
find "$MHVTL_DIR" -name "*.data" -mtime +$DAYS_OLD -delete
echo "Remaining tapes:"
ls -lh "$MHVTL_DIR"/*.data 2>/dev/null | wc -l
```
### Configuration Backup Script
```bash
#!/bin/bash
BACKUP_DIR="/backup/vtl-config"
DATE=$(date +%Y%m%d-%H%M%S)
mkdir -p "$BACKUP_DIR"
tar -czf "$BACKUP_DIR/vtl-config-$DATE.tar.gz" \
/etc/mhvtl/ \
/etc/tgt/conf.d/ \
/etc/sysctl.d/99-vtl.conf \
/etc/systemd/system/mhvtl.service
echo "Backup saved to: $BACKUP_DIR/vtl-config-$DATE.tar.gz"
find "$BACKUP_DIR" -name "vtl-config-*.tar.gz" -mtime +30 -delete
```

276
docs/INSTALLATION.md Normal file
View File

@@ -0,0 +1,276 @@
# VTL Linux - Installation Guide
## Prerequisites
- x86_64 compatible hardware
- Minimum 2GB RAM (4GB+ recommended)
- 50GB+ storage for tape media
- Network interface card
- USB drive or CD/DVD for installation media
## Installation Methods
### Method 1: Live Boot (Testing)
1. Write ISO to USB drive:
```bash
dd if=VTL-Linux-1.0-x86_64.iso of=/dev/sdX bs=4M status=progress
sync
```
2. Boot from USB drive
3. System will boot into live environment with VTL services
### Method 2: Full Installation
1. Boot from ISO
2. Select "Install VTL Linux to Disk" from boot menu
3. Follow installation prompts:
- Select target disk
- Configure network
- Set hostname
- Create user account
4. Reboot after installation
## Post-Installation Configuration
### 1. Network Setup
Configure static IP (recommended for iSCSI):
```bash
sudo nmcli con mod "Wired connection 1" \
ipv4.addresses 192.168.1.100/24 \
ipv4.gateway 192.168.1.1 \
ipv4.dns "8.8.8.8 8.8.4.4" \
ipv4.method manual
sudo nmcli con up "Wired connection 1"
```
### 2. Change Default Passwords
```bash
sudo passwd root
sudo passwd vtladmin
```
### 3. Configure mhvtl
Edit device configuration:
```bash
sudo vim /etc/mhvtl/device.conf
```
Restart mhvtl service:
```bash
sudo systemctl restart mhvtl
```
### 4. Configure iSCSI Targets
Edit target configuration:
```bash
sudo vim /etc/tgt/conf.d/vtl-targets.conf
```
Update credentials:
```bash
incominguser <username> <password>
```
Restart tgt service:
```bash
sudo systemctl restart tgt
```
### 5. Verify Installation
Check system status:
```bash
vtl-status
```
List SCSI devices:
```bash
lsscsi -g
```
Check library status:
```bash
mtx -f /dev/sg0 status
```
View iSCSI targets:
```bash
tgt-admin --show
```
## Client Configuration
### Linux Client
1. Install iSCSI initiator:
```bash
sudo apt-get install open-iscsi
```
2. Discover targets:
```bash
sudo iscsiadm -m discovery -t st -p <VTL_IP>:3260
```
3. Login to target:
```bash
sudo iscsiadm -m node --login
```
4. Configure CHAP authentication (if required):
```bash
sudo iscsiadm -m node -T <target_iqn> -p <VTL_IP>:3260 \
--op=update --name node.session.auth.authmethod --value=CHAP
sudo iscsiadm -m node -T <target_iqn> -p <VTL_IP>:3260 \
--op=update --name node.session.auth.username --value=vtl-user
sudo iscsiadm -m node -T <target_iqn> -p <VTL_IP>:3260 \
--op=update --name node.session.auth.password --value=vtl-password
```
5. Verify connection:
```bash
lsscsi
```
### Windows Client
1. Open iSCSI Initiator (Control Panel → Administrative Tools)
2. Go to Discovery tab, click "Discover Portal"
3. Enter VTL server IP address and port 3260
4. Go to Targets tab, select discovered target
5. Click "Connect"
6. For CHAP authentication:
- Click "Advanced"
- Enable CHAP login
- Enter username: vtl-user
- Enter password: vtl-password
7. Verify in Device Manager under "Tape drives"
## Backup Software Configuration
### Bacula
```bash
Device {
Name = VTL-Drive-0
Media Type = LTO-5
Archive Device = /dev/nst0
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
RandomAccess = no
AutoChanger = yes
}
Autochanger {
Name = VTL-Library
Device = VTL-Drive-0, VTL-Drive-1, VTL-Drive-2, VTL-Drive-3
Changer Command = "/usr/lib/bacula/scripts/mtx-changer %c %o %S %a %d"
Changer Device = /dev/sg0
}
```
### Amanda
```bash
tapedev "chg-robot:/dev/sg0"
tpchanger "chg-robot"
changerfile "/var/lib/amanda/vtl/changer"
```
### Veeam (Windows)
1. Add tape server in Veeam console
2. Rescan tape infrastructure
3. VTL devices should appear automatically
4. Configure media pools and backup jobs
## Troubleshooting
### mhvtl not starting
```bash
sudo modprobe mhvtl
sudo systemctl status mhvtl
sudo journalctl -u mhvtl -n 50
```
### iSCSI connection issues
```bash
sudo systemctl status tgt
sudo tgt-admin --show
sudo netstat -tlnp | grep 3260
```
### SCSI devices not visible
```bash
sudo modprobe sg
lsmod | grep mhvtl
dmesg | grep -i scsi
```
### Performance issues
Check system resources:
```bash
htop
iostat -x 1
iotop
```
Adjust kernel parameters in `/etc/sysctl.d/99-vtl.conf`
## Maintenance
### Adding tape media
```bash
sudo /usr/bin/mktape -l 10 -s 100 -m /opt/mhvtl -t LTO5 -d 10
sudo systemctl restart mhvtl
```
### Backup configuration
```bash
sudo tar -czf vtl-config-backup.tar.gz \
/etc/mhvtl/ \
/etc/tgt/conf.d/ \
/etc/sysctl.d/99-vtl.conf
```
### Update system
```bash
sudo apt-get update
sudo apt-get upgrade
sudo reboot
```
## Support
For issues and questions:
- Check logs: `journalctl -xe`
- Review documentation in `/vtl/docs/`
- mhvtl documentation: https://github.com/markh794/mhvtl

97
quickstart.sh Executable file
View File

@@ -0,0 +1,97 @@
#!/bin/bash
echo "=========================================="
echo " VTL Linux - Quick Start Script"
echo "=========================================="
echo ""
cat << 'EOF'
This script will help you get started with VTL Linux.
Prerequisites:
- Debian-based Linux system (for building)
- Root/sudo access
- 10GB+ free disk space
- Internet connection
What would you like to do?
1) Build VTL Linux ISO
2) Install mhvtl on current system
3) Configure iSCSI targets
4) Run post-install setup
5) Check VTL status
6) Show documentation
7) Exit
EOF
read -p "Enter your choice [1-7]: " choice
case $choice in
1)
echo ""
echo "Building VTL Linux ISO..."
echo "This will take 15-30 minutes depending on your system."
read -p "Continue? (y/n): " confirm
if [ "$confirm" = "y" ]; then
cd /vtl
sudo bash build/build-iso.sh
fi
;;
2)
echo ""
echo "Installing mhvtl..."
read -p "Continue? (y/n): " confirm
if [ "$confirm" = "y" ]; then
sudo bash /vtl/scripts/install-mhvtl.sh
fi
;;
3)
echo ""
echo "Configuring iSCSI targets..."
read -p "Continue? (y/n): " confirm
if [ "$confirm" = "y" ]; then
sudo bash /vtl/scripts/configure-iscsi.sh
fi
;;
4)
echo ""
echo "Running post-install setup..."
read -p "Continue? (y/n): " confirm
if [ "$confirm" = "y" ]; then
sudo bash /vtl/scripts/post-install.sh
fi
;;
5)
echo ""
if command -v vtl-status &> /dev/null; then
sudo vtl-status
else
echo "vtl-status command not found. Run post-install setup first."
fi
;;
6)
echo ""
echo "Available documentation:"
echo " - README: /vtl/README.md"
echo " - Installation Guide: /vtl/docs/INSTALLATION.md"
echo " - Architecture: /vtl/docs/ARCHITECTURE.md"
echo ""
read -p "Open README? (y/n): " confirm
if [ "$confirm" = "y" ]; then
less /vtl/README.md
fi
;;
7)
echo "Goodbye!"
exit 0
;;
*)
echo "Invalid choice. Please run the script again."
exit 1
;;
esac
echo ""
echo "Done!"

104
scripts/configure-iscsi.sh Executable file
View File

@@ -0,0 +1,104 @@
#!/bin/bash
set -e
echo "=========================================="
echo " iSCSI Target Configuration Script"
echo "=========================================="
echo ""
if [ "$EUID" -ne 0 ]; then
echo "Error: This script must be run as root"
exit 1
fi
TGT_CONFIG_DIR="/etc/tgt/conf.d"
ISCSI_IQN_BASE="iqn.2024-01.com.vtl-linux"
echo "[1/4] Installing iSCSI target software..."
apt-get update
apt-get install -y tgt
echo "[2/4] Configuring iSCSI targets..."
mkdir -p "$TGT_CONFIG_DIR"
cat > "$TGT_CONFIG_DIR/vtl-targets.conf" << 'EOF'
<target iqn.2024-01.com.vtl-linux:vtl.lun0>
backing-store /dev/sg1
initiator-address ALL
incominguser vtl-user vtl-password
write-cache on
</target>
<target iqn.2024-01.com.vtl-linux:vtl.lun1>
backing-store /dev/sg2
initiator-address ALL
incominguser vtl-user vtl-password
write-cache on
</target>
<target iqn.2024-01.com.vtl-linux:vtl.lun2>
backing-store /dev/sg3
initiator-address ALL
incominguser vtl-user vtl-password
write-cache on
</target>
<target iqn.2024-01.com.vtl-linux:vtl.lun3>
backing-store /dev/sg4
initiator-address ALL
incominguser vtl-user vtl-password
write-cache on
</target>
<target iqn.2024-01.com.vtl-linux:vtl.changer>
backing-store /dev/sg0
initiator-address ALL
incominguser vtl-user vtl-password
device-type changer
</target>
EOF
echo "[3/4] Configuring firewall..."
if command -v ufw &> /dev/null; then
ufw allow 3260/tcp
ufw reload
elif command -v firewall-cmd &> /dev/null; then
firewall-cmd --permanent --add-port=3260/tcp
firewall-cmd --reload
else
iptables -A INPUT -p tcp --dport 3260 -j ACCEPT
iptables-save > /etc/iptables/rules.v4
fi
echo "[4/4] Starting iSCSI target service..."
systemctl enable tgt
systemctl restart tgt
sleep 2
echo ""
echo "=========================================="
echo " iSCSI Target Configuration Complete!"
echo "=========================================="
echo ""
echo "Available targets:"
tgt-admin --show
echo ""
echo "Connection information:"
echo " - Port: 3260"
echo " - IQN Base: $ISCSI_IQN_BASE"
echo " - Username: vtl-user"
echo " - Password: vtl-password"
echo ""
echo "Client connection examples:"
echo ""
echo "Linux:"
echo " iscsiadm -m discovery -t st -p <SERVER_IP>:3260"
echo " iscsiadm -m node --login"
echo ""
echo "Windows:"
echo " iscsicli QAddTargetPortal <SERVER_IP>"
echo " iscsicli ListTargets"
echo " iscsicli LoginTarget <target_name> T * * * * * * * * * * * * * * * <username> <password>"
echo ""

133
scripts/install-mhvtl.sh Executable file
View File

@@ -0,0 +1,133 @@
#!/bin/bash
set -e
echo "=========================================="
echo " mhvtl Installation Script"
echo "=========================================="
echo ""
if [ "$EUID" -ne 0 ]; then
echo "Error: This script must be run as root"
exit 1
fi
MHVTL_VERSION="1.6-7"
MHVTL_DIR="/opt/mhvtl"
MHVTL_CONFIG="/etc/mhvtl"
echo "[1/5] Installing build dependencies..."
apt-get update
apt-get install -y \
build-essential \
git \
zlib1g-dev \
libibverbs-dev \
libconfig-dev \
libssl-dev \
uuid-dev \
linux-headers-$(uname -r) \
mt-st \
mtx \
lsscsi \
sg3-utils
echo "[2/5] Downloading mhvtl source..."
cd /tmp
if [ -d "mhvtl" ]; then
rm -rf mhvtl
fi
git clone https://github.com/markh794/mhvtl.git
cd mhvtl
echo "[3/5] Building mhvtl..."
make
echo "[4/5] Installing mhvtl..."
make install
echo "[5/5] Configuring mhvtl..."
mkdir -p "$MHVTL_DIR"
mkdir -p "$MHVTL_CONFIG"
if [ ! -f "$MHVTL_CONFIG/device.conf" ]; then
cat > "$MHVTL_CONFIG/device.conf" << 'EOF'
VERSION: 5
Library: 10 CHANNEL: 00 TARGET: 00 LUN: 00
Vendor identification: STK
Product identification: L700
Unit serial number: XYZZY_A
NAA: 10:22:33:44:ab:cd:ef:00
Home directory: /opt/mhvtl
Backoff: 400
Drive: 00 CHANNEL: 00 TARGET: 01 LUN: 00
Library ID: 10 Slot: 01
Vendor identification: IBM
Product identification: ULT3580-TD5
Unit serial number: XYZZY_A1
NAA: 10:22:33:44:ab:cd:ef:01
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
Drive: 01 CHANNEL: 00 TARGET: 02 LUN: 00
Library ID: 10 Slot: 02
Vendor identification: IBM
Product identification: ULT3580-TD5
Unit serial number: XYZZY_A2
NAA: 10:22:33:44:ab:cd:ef:02
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
Drive: 02 CHANNEL: 00 TARGET: 03 LUN: 00
Library ID: 10 Slot: 03
Vendor identification: IBM
Product identification: ULT3580-TD6
Unit serial number: XYZZY_A3
NAA: 10:22:33:44:ab:cd:ef:03
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
Drive: 03 CHANNEL: 00 TARGET: 04 LUN: 00
Library ID: 10 Slot: 04
Vendor identification: IBM
Product identification: ULT3580-TD6
Unit serial number: XYZZY_A4
NAA: 10:22:33:44:ab:cd:ef:04
Compression: factor 3 enabled 1
Compression type: lzo
Backoff: 400
EOF
fi
if [ ! -f "$MHVTL_CONFIG/library_contents.10" ]; then
/usr/bin/mktape -l 10 -s 100 -m /opt/mhvtl -t LTO5 -d 20
fi
modprobe mhvtl
systemctl daemon-reload
systemctl enable mhvtl
systemctl start mhvtl
echo ""
echo "=========================================="
echo " mhvtl Installation Complete!"
echo "=========================================="
echo ""
echo "Configuration:"
echo " - Config directory: $MHVTL_CONFIG"
echo " - Data directory: $MHVTL_DIR"
echo " - Library: STK L700 (ID: 10)"
echo " - Drives: 4x LTO-5/6 drives"
echo " - Media: 20 LTO-5 tapes"
echo ""
echo "Check status:"
echo " systemctl status mhvtl"
echo " lsscsi -g"
echo " mtx -f /dev/sg0 status"
echo ""

114
scripts/post-install.sh Executable file
View File

@@ -0,0 +1,114 @@
#!/bin/bash
set -e
echo "=========================================="
echo " VTL Linux Post-Install Setup"
echo "=========================================="
echo ""
if [ "$EUID" -ne 0 ]; then
echo "Error: This script must be run as root"
exit 1
fi
echo "[1/5] Applying system optimizations..."
if [ -f "/tmp/sysctl-vtl.conf" ]; then
cp /tmp/sysctl-vtl.conf /etc/sysctl.d/99-vtl.conf
sysctl -p /etc/sysctl.d/99-vtl.conf
fi
echo "[2/5] Installing mhvtl..."
if [ -f "/usr/local/bin/install-mhvtl.sh" ]; then
bash /usr/local/bin/install-mhvtl.sh
else
echo "Warning: mhvtl installation script not found"
fi
echo "[3/5] Configuring iSCSI targets..."
if [ -f "/usr/local/bin/configure-iscsi.sh" ]; then
bash /usr/local/bin/configure-iscsi.sh
else
echo "Warning: iSCSI configuration script not found"
fi
echo "[4/5] Setting up monitoring..."
cat > /usr/local/bin/vtl-status << 'EOF'
#!/bin/bash
echo "=========================================="
echo " VTL System Status"
echo "=========================================="
echo ""
echo "=== mhvtl Status ==="
systemctl status mhvtl --no-pager | head -n 10
echo ""
echo "=== SCSI Devices ==="
lsscsi -g
echo ""
echo "=== Library Status ==="
if [ -e /dev/sg0 ]; then
mtx -f /dev/sg0 status 2>/dev/null || echo "Library not ready"
fi
echo ""
echo "=== iSCSI Targets ==="
tgt-admin --show
echo ""
echo "=== Network Interfaces ==="
ip -br addr
echo ""
echo "=== Disk Usage ==="
df -h /opt/mhvtl 2>/dev/null || echo "/opt/mhvtl not mounted"
echo ""
EOF
chmod +x /usr/local/bin/vtl-status
echo "[5/5] Creating welcome message..."
cat > /etc/motd << 'EOF'
__ _______ _ _ _
\ \ / /_ _| | | | (_)
\ \ / / | | | | | | _ _ __ _ ___ __
\ \/ / | | | | | | | | '_ \| | | \ \/ /
\ / _| |_| |____ | |___| | | | | |_| |> <
\/ |_____|______||_____|_|_| |_|\__,_/_/\_\
Virtual Tape Library Distribution v1.0
========================================
Quick Commands:
vtl-status - Show VTL system status
systemctl status mhvtl - Check mhvtl service
lsscsi -g - List SCSI devices
tgt-admin --show - Show iSCSI targets
Default Credentials:
User: vtladmin / Password: vtladmin
Root: root / Password: vtlroot
iSCSI Authentication:
Username: vtl-user
Password: vtl-password
========================================
EOF
echo ""
echo "=========================================="
echo " Post-Install Setup Complete!"
echo "=========================================="
echo ""
echo "Next steps:"
echo " 1. Configure network settings"
echo " 2. Change default passwords"
echo " 3. Customize mhvtl configuration in /etc/mhvtl/"
echo " 4. Update iSCSI targets in /etc/tgt/conf.d/"
echo " 5. Run 'vtl-status' to verify setup"
echo ""

15
systemd/mhvtl.service Normal file
View File

@@ -0,0 +1,15 @@
[Unit]
Description=mhvtl Virtual Tape Library
After=network.target
[Service]
Type=forking
ExecStartPre=/sbin/modprobe mhvtl
ExecStart=/usr/bin/vtltape
ExecStart=/usr/bin/vtllibrary
ExecStop=/usr/bin/killall vtltape vtllibrary
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target