commit 0537709576ed1f0cca14053be61b5dfc8d7cb541 Author: Othman H. Suseno Date: Wed Dec 24 23:55:55 2025 +0700 add instruction and srs diff --git a/src/srs-technical-spec-documents/AtlasOS-Calypso-Engineering.md b/src/srs-technical-spec-documents/AtlasOS-Calypso-Engineering.md new file mode 100644 index 0000000..48bb1be --- /dev/null +++ b/src/srs-technical-spec-documents/AtlasOS-Calypso-Engineering.md @@ -0,0 +1,339 @@ +# AtlasOS – Calypso +## Engineering & Architecture Master Document +### (CURSOR.md – Single Source of Truth) + +Version: 2.0 +Status: Baseline – Product Definition +Target OS: Ubuntu Server 24.04 LTS +Category: Backup Appliance / Tape & VTL Virtualization Platform +Date: 2025 + +--- + +## 0. Project Definition + +**AtlasOS – Calypso** adalah **generic backup appliance** yang menyediakan: + +- Disk-based backup storage (iSCSI block) +- Physical tape library bridging (SAS / FC → iSCSI) +- Virtual Tape Library (VTL) menggunakan **MHVTL** +- Unified web-based management GUI +- Authentication, authorization, audit, dan monitoring terpusat + +Calypso **tidak terikat pada backup software tertentu**. + +Backup software yang didukung mencakup (namun tidak terbatas pada): +- Bacula +- Veeam +- Dell NetWorker +- Veritas NetBackup +- Commvault +- Arcserve +- Custom / proprietary SCSI-compliant backup engines + +Calypso berperan sebagai **storage & SCSI virtualization appliance**, bukan backup controller. + +--- + +## 1. Non-Negotiable Design Rules + +1. **SCST adalah satu-satunya iSCSI target framework** + - Tidak menggunakan LIO / targetcli + - Semua disk, physical tape, dan VTL diexport melalui SCST + +2. **Mapping tape device wajib konsisten** + - LUN 0 → Medium changer + - LUN 1..N → Tape drives (maksimal 8) + - Berlaku untuk: + - Physical tape library + - MHVTL virtual library + +3. **Single initiator policy untuk tape** + - Hanya satu IQN initiator boleh login ke tape target + - Pelanggaran harus terdeteksi dan ditampilkan di UI + +4. **Backup software agnostic** + - Tidak ada logic Bacula / NetWorker / Veeam di core + - Hanya SCSI semantics + optional compatibility profile + +5. **Tidak ada shell/terminal di UI (v1)** + - Semua aksi melalui API tervalidasi + +6. **Least privilege backend** + - Backend berjalan sebagai non-root + - Aksi privileged via polkit atau sudoers allowlist ketat + +7. **Audit wajib** + - Setiap perubahan konfigurasi + - Operasi tape + - Apply iSCSI + - Perubahan IAM + +8. **Operasi berat bersifat async** + - Inventory + - Load / unload + - Rescan + - Apply SCST + - Support bundle + +--- + +## 2. Core Appliance Capabilities + +### 2.1 Disk Storage +- LVM-backed repository +- Optional ZFS (advanced SKU) +- Export sebagai iSCSI block device +- Digunakan untuk: + - Disk backup target + - Backing store MHVTL + +### 2.2 Physical Tape Bridge (SAS / FC) +- Discovery changer & drive +- Inventory slot & barcode +- Load / unload tape +- Bridge ke backup software via iSCSI + +### 2.3 Virtual Tape Library (MHVTL) +- Virtual changer, drive, slot, dan tape +- Disk-backed tape image +- Barcode emulation +- Export via SCST iSCSI +- Use case: + - Backup staging + - Copy-to-tape + - Testing / development + - Air-gap simulation + +--- + +## 3. High-Level Architecture + +Backup Software (Any Vendor) +| +iSCSI +| ++--------------------------------+ +| AtlasOS – Calypso | +| | +| Disk Repository (LUN) | +| MHVTL (Virtual Tape) | +| Physical Tape Bridge | +| SCST iSCSI Core | +| | ++--------------------------------+ +| +SAS / FC +| +Physical Tape Library + +--- + +## 4. Component List (Authoritative) + +### 4.1 Base Platform +- Ubuntu Server 24.04 LTS +- systemd, journald +- udev persistent naming +- chrony +- ufw / nftables + +### 4.2 Disk Storage Layer +- LVM2 +- thin-provisioning-tools +- XFS (primary) +- ext4 (alternative) +- Optional ZFS +- smartmontools, nvme-cli +- parted, gdisk + +### 4.3 Physical Tape Subsystem +- SAS / FC HBA drivers +- multipath-tools (optional) +- lsscsi +- sg3_utils +- mt-st +- mtx + +### 4.4 Virtual Tape Library +- mhvtl +- mhvtl-utils / vtlcmd +- Disk-backed tape images + +### 4.5 iSCSI Target Stack +- scst +- iscsi-scst +- scstadmin + +### 4.6 Calypso Core Application +**Backend (Go):** +- storage +- tape_physical +- tape_vtl +- scst +- iscsi +- tasks +- system +- monitoring +- audit +- profile_engine + +**Frontend (React + Vite):** +- Dashboard +- Disk Repository +- Physical Tape +- Virtual Tape Library +- iSCSI Targets +- Clients / Initiators +- Tasks & Jobs +- Alerts & Logs +- System & IAM + +### 4.7 Authentication & IAM +- PostgreSQL +- Local auth (Argon2id) +- LDAP +- OIDC SSO +- RBAC +- Audit log + +### 4.8 Monitoring & Observability +- Built-in health checks +- Alerts engine +- Event stream (WebSocket) +- Optional Prometheus exporter +- node_exporter + +### 4.9 Web & Security Perimeter +- Caddy (recommended) / Nginx +- TLS +- Security headers +- Rate limiting + +### 4.10 Packaging & Operations +- Debian packages (.deb) +- systemd services: + - calypso-api + - scst + - mhvtl + - postgresql + - caddy/nginx +- Installer & upgrade scripts +- Support bundle generator + +--- + +## 5. Repository Structure (Monorepo) + +calypso/ +README.md +CURSOR.md + +docs/ +COMPONENT-LIST-AtlasOS-Calypso.md +SRS-00-Main-AtlasOS-Calypso.md +SRS-01-Storage-Component.md +SRS-02-VTL-and-Tape-Bridge.md +SRS-03-System-Management.md +SRS-04-Auth-and-IAM.md +SRS-05-Monitoring-and-Observability.md + +frontend/ +backend/ +deploy/ + +--- + +## 6. API Design Rules + +- Base path `/api/v1` +- JSON only +- RBAC + audit untuk endpoint mutating +- Async → `task_id` +- Task status → `/api/v1/tasks/{id}` +- WebSocket `/ws`: + - task progress + - alerts + - inventory + - iSCSI session changes + +--- + +## 7. Database Scope (PostgreSQL) + +Digunakan untuk: +- Users, roles, permissions +- Sessions & tokens +- Audit log +- Appliance configuration +- Async task state +- Alerts + +Tidak digunakan untuk: +- Backup data +- Tape data + +--- + +## 8. Coding Standards + +### Backend +- Go ≥ 1.22 +- Context everywhere +- No raw shell execution +- Strict validation +- Unit tests untuk: + - SCST config + - Tape discovery + - Task state machine + - RBAC + +### Frontend +- TypeScript strict +- API via `src/api` +- No business logic di component +- Unified error handling +- WebSocket reconnect handling + +--- + +## 9. Safety & Guardrails + +- Blok aksi berbahaya saat tape aktif +- Cegah: + - multi-initiator tape + - remove active drive + - reconfig SCST saat write +- Konfirmasi wajib untuk destructive action + +--- + +## 10. Implementation Phases + +1. Foundation & Auth +2. Disk Repository +3. Physical Tape Bridge +4. Virtual Tape Library (MHVTL) +5. Monitoring & IAM extensions +6. UI polish & hardening + +--- + +## 11. Definition of Done (DoD) + +Feature dianggap selesai jika: +- API implemented +- RBAC enforced +- Audit logged +- UI workflow complete +- Error handled +- Docs updated + +--- + +## 12. Final Authority + +Jika ada konflik antar dokumen: +**FILE INI ADALAH SUMBER KEBENARAN.** + +AtlasOS – Calypso adalah **tape & VTL appliance kelas enterprise**. \ No newline at end of file diff --git a/src/srs-technical-spec-documents/CURSOR.md b/src/srs-technical-spec-documents/CURSOR.md new file mode 100644 index 0000000..48bb1be --- /dev/null +++ b/src/srs-technical-spec-documents/CURSOR.md @@ -0,0 +1,339 @@ +# AtlasOS – Calypso +## Engineering & Architecture Master Document +### (CURSOR.md – Single Source of Truth) + +Version: 2.0 +Status: Baseline – Product Definition +Target OS: Ubuntu Server 24.04 LTS +Category: Backup Appliance / Tape & VTL Virtualization Platform +Date: 2025 + +--- + +## 0. Project Definition + +**AtlasOS – Calypso** adalah **generic backup appliance** yang menyediakan: + +- Disk-based backup storage (iSCSI block) +- Physical tape library bridging (SAS / FC → iSCSI) +- Virtual Tape Library (VTL) menggunakan **MHVTL** +- Unified web-based management GUI +- Authentication, authorization, audit, dan monitoring terpusat + +Calypso **tidak terikat pada backup software tertentu**. + +Backup software yang didukung mencakup (namun tidak terbatas pada): +- Bacula +- Veeam +- Dell NetWorker +- Veritas NetBackup +- Commvault +- Arcserve +- Custom / proprietary SCSI-compliant backup engines + +Calypso berperan sebagai **storage & SCSI virtualization appliance**, bukan backup controller. + +--- + +## 1. Non-Negotiable Design Rules + +1. **SCST adalah satu-satunya iSCSI target framework** + - Tidak menggunakan LIO / targetcli + - Semua disk, physical tape, dan VTL diexport melalui SCST + +2. **Mapping tape device wajib konsisten** + - LUN 0 → Medium changer + - LUN 1..N → Tape drives (maksimal 8) + - Berlaku untuk: + - Physical tape library + - MHVTL virtual library + +3. **Single initiator policy untuk tape** + - Hanya satu IQN initiator boleh login ke tape target + - Pelanggaran harus terdeteksi dan ditampilkan di UI + +4. **Backup software agnostic** + - Tidak ada logic Bacula / NetWorker / Veeam di core + - Hanya SCSI semantics + optional compatibility profile + +5. **Tidak ada shell/terminal di UI (v1)** + - Semua aksi melalui API tervalidasi + +6. **Least privilege backend** + - Backend berjalan sebagai non-root + - Aksi privileged via polkit atau sudoers allowlist ketat + +7. **Audit wajib** + - Setiap perubahan konfigurasi + - Operasi tape + - Apply iSCSI + - Perubahan IAM + +8. **Operasi berat bersifat async** + - Inventory + - Load / unload + - Rescan + - Apply SCST + - Support bundle + +--- + +## 2. Core Appliance Capabilities + +### 2.1 Disk Storage +- LVM-backed repository +- Optional ZFS (advanced SKU) +- Export sebagai iSCSI block device +- Digunakan untuk: + - Disk backup target + - Backing store MHVTL + +### 2.2 Physical Tape Bridge (SAS / FC) +- Discovery changer & drive +- Inventory slot & barcode +- Load / unload tape +- Bridge ke backup software via iSCSI + +### 2.3 Virtual Tape Library (MHVTL) +- Virtual changer, drive, slot, dan tape +- Disk-backed tape image +- Barcode emulation +- Export via SCST iSCSI +- Use case: + - Backup staging + - Copy-to-tape + - Testing / development + - Air-gap simulation + +--- + +## 3. High-Level Architecture + +Backup Software (Any Vendor) +| +iSCSI +| ++--------------------------------+ +| AtlasOS – Calypso | +| | +| Disk Repository (LUN) | +| MHVTL (Virtual Tape) | +| Physical Tape Bridge | +| SCST iSCSI Core | +| | ++--------------------------------+ +| +SAS / FC +| +Physical Tape Library + +--- + +## 4. Component List (Authoritative) + +### 4.1 Base Platform +- Ubuntu Server 24.04 LTS +- systemd, journald +- udev persistent naming +- chrony +- ufw / nftables + +### 4.2 Disk Storage Layer +- LVM2 +- thin-provisioning-tools +- XFS (primary) +- ext4 (alternative) +- Optional ZFS +- smartmontools, nvme-cli +- parted, gdisk + +### 4.3 Physical Tape Subsystem +- SAS / FC HBA drivers +- multipath-tools (optional) +- lsscsi +- sg3_utils +- mt-st +- mtx + +### 4.4 Virtual Tape Library +- mhvtl +- mhvtl-utils / vtlcmd +- Disk-backed tape images + +### 4.5 iSCSI Target Stack +- scst +- iscsi-scst +- scstadmin + +### 4.6 Calypso Core Application +**Backend (Go):** +- storage +- tape_physical +- tape_vtl +- scst +- iscsi +- tasks +- system +- monitoring +- audit +- profile_engine + +**Frontend (React + Vite):** +- Dashboard +- Disk Repository +- Physical Tape +- Virtual Tape Library +- iSCSI Targets +- Clients / Initiators +- Tasks & Jobs +- Alerts & Logs +- System & IAM + +### 4.7 Authentication & IAM +- PostgreSQL +- Local auth (Argon2id) +- LDAP +- OIDC SSO +- RBAC +- Audit log + +### 4.8 Monitoring & Observability +- Built-in health checks +- Alerts engine +- Event stream (WebSocket) +- Optional Prometheus exporter +- node_exporter + +### 4.9 Web & Security Perimeter +- Caddy (recommended) / Nginx +- TLS +- Security headers +- Rate limiting + +### 4.10 Packaging & Operations +- Debian packages (.deb) +- systemd services: + - calypso-api + - scst + - mhvtl + - postgresql + - caddy/nginx +- Installer & upgrade scripts +- Support bundle generator + +--- + +## 5. Repository Structure (Monorepo) + +calypso/ +README.md +CURSOR.md + +docs/ +COMPONENT-LIST-AtlasOS-Calypso.md +SRS-00-Main-AtlasOS-Calypso.md +SRS-01-Storage-Component.md +SRS-02-VTL-and-Tape-Bridge.md +SRS-03-System-Management.md +SRS-04-Auth-and-IAM.md +SRS-05-Monitoring-and-Observability.md + +frontend/ +backend/ +deploy/ + +--- + +## 6. API Design Rules + +- Base path `/api/v1` +- JSON only +- RBAC + audit untuk endpoint mutating +- Async → `task_id` +- Task status → `/api/v1/tasks/{id}` +- WebSocket `/ws`: + - task progress + - alerts + - inventory + - iSCSI session changes + +--- + +## 7. Database Scope (PostgreSQL) + +Digunakan untuk: +- Users, roles, permissions +- Sessions & tokens +- Audit log +- Appliance configuration +- Async task state +- Alerts + +Tidak digunakan untuk: +- Backup data +- Tape data + +--- + +## 8. Coding Standards + +### Backend +- Go ≥ 1.22 +- Context everywhere +- No raw shell execution +- Strict validation +- Unit tests untuk: + - SCST config + - Tape discovery + - Task state machine + - RBAC + +### Frontend +- TypeScript strict +- API via `src/api` +- No business logic di component +- Unified error handling +- WebSocket reconnect handling + +--- + +## 9. Safety & Guardrails + +- Blok aksi berbahaya saat tape aktif +- Cegah: + - multi-initiator tape + - remove active drive + - reconfig SCST saat write +- Konfirmasi wajib untuk destructive action + +--- + +## 10. Implementation Phases + +1. Foundation & Auth +2. Disk Repository +3. Physical Tape Bridge +4. Virtual Tape Library (MHVTL) +5. Monitoring & IAM extensions +6. UI polish & hardening + +--- + +## 11. Definition of Done (DoD) + +Feature dianggap selesai jika: +- API implemented +- RBAC enforced +- Audit logged +- UI workflow complete +- Error handled +- Docs updated + +--- + +## 12. Final Authority + +Jika ada konflik antar dokumen: +**FILE INI ADALAH SUMBER KEBENARAN.** + +AtlasOS – Calypso adalah **tape & VTL appliance kelas enterprise**. \ No newline at end of file diff --git a/src/srs-technical-spec-documents/Calypso-Architecture-Diagrams.md b/src/srs-technical-spec-documents/Calypso-Architecture-Diagrams.md new file mode 100644 index 0000000..c7810c1 --- /dev/null +++ b/src/srs-technical-spec-documents/Calypso-Architecture-Diagrams.md @@ -0,0 +1,164 @@ +# AtlasOS – Calypso Architecture Diagrams + +## 1) System Context (High-Level) + +```mermaid +flowchart LR + subgraph Client["Backup Software Client (Any Vendor)"] + C1["Backup Server / VM\n(Bacula / Veeam / NetWorker / NetBackup / etc)"] + end + + subgraph Net["Backup Network"] + ISCSI["iSCSI (TCP/3260)\nInitiator Sessions"] + HTTPS["HTTPS (TCP/443)\nAdmin GUI/API"] + end + + subgraph Calypso["AtlasOS – Calypso Appliance (Ubuntu 24.04)"] + RP["Reverse Proxy\n(Caddy/Nginx)"] + UI["Web GUI\n(React + Vite)"] + API["Calypso Core API\n(Go REST + WebSocket)"] + DB["PostgreSQL\n(IAM + Audit + Tasks + Alerts)"] + SCST["SCST iSCSI Target Core\n(scst + iscsi-scst + scstadmin)"] + DISK["Disk Storage Layer\n(LVM + XFS)\nBackup Repo + VTL Backing"] + MHVTL["MHVTL Engine\n(Virtual Changer/Drives/Slots/Tapes)"] + TOOLS["Tape Tools\n(lsscsi, sg3_utils, mtx, mt)"] + end + + subgraph Fabric["SAS/FC Fabric"] + HBA["SAS/FC HBA"] + end + + subgraph Phys["Physical Tape Library"] + CHG["Medium Changer"] + DRV["Tape Drives (LTO-8)\nMax 8 presented"] + SLOTS["Slots + Barcodes"] + end + + C1 --> ISCSI --> SCST + C1 --> HTTPS --> RP + RP --> UI + RP --> API + API --> DB + API --> SCST + API --> DISK + API --> MHVTL + API --> TOOLS + + SCST --> DISK + SCST --> MHVTL + + TOOLS --> HBA --> CHG + TOOLS --> HBA --> DRV + HBA --> SLOTS + +flowchart TB + subgraph ControlPlane["Control Plane"] + RP["Reverse Proxy\n(Caddy/Nginx)"] + UI["Web GUI\nReact + Vite"] + API["Calypso Core API (Go)\nREST + WS"] + IAM["IAM Module\nLocal + LDAP + OIDC"] + MON["Monitoring Module\nHealth + Alerts + Metrics"] + TASKS["Task Engine\nAsync Ops + Progress"] + AUDIT["Audit Engine\nAppend-only"] + DB["PostgreSQL\nUsers/Roles/Sessions\nTasks/Alerts/Audit\nConfig State"] + end + + subgraph DataPlane["Data Plane (Storage/SCSI)"] + SCST["SCST Target Core\niscsi-scst + scstadmin"] + DISK["Disk Repo\nLVM + XFS"] + MHVTL["MHVTL VTL\nVirtual Changer/Drives/Slots"] + PHYS["Physical Tape\nSAS/FC Changer + Drives"] + TOOLS["Tape Ops Tools\nlsscsi/sg3_utils/mtx/mt"] + end + + RP --> UI + RP --> API + + API --> IAM + API --> MON + API --> TASKS + API --> AUDIT + IAM --> DB + MON --> DB + TASKS --> DB + AUDIT --> DB + + API --> SCST + API --> DISK + API --> MHVTL + API --> TOOLS + + SCST --> DISK + SCST --> MHVTL + + TOOLS --> PHYS + API --> PHYS + +flowchart LR + subgraph Initiator["Backup Software (Initiator)"] + BSVR["Backup Server/VM\n(any vendor)"] + end + + subgraph Targets["Calypso iSCSI Targets (SCST)"] + TDISK["Target: Disk Repository\nIQN: calypso.repo\nLUN 0: Disk LUN"] + TPHYS["Target: Physical Tape Bridge\nIQN: calypso.tape.phys\nLUN 0: Changer\nLUN 1..8: Drives"] + TVTL["Target: Virtual Tape Library\nIQN: calypso.tape.vtl\nLUN 0: VTL Changer\nLUN 1..N: VTL Drives"] + end + + subgraph Backing["Backing Devices"] + DISK["LVM/XFS Repo LUN"] + PHYCHG["Physical Changer (SAS/FC)"] + PHYDRV["Physical Drives (LTO-8)"] + VTLCHG["MHVTL Changer"] + VTLDRV["MHVTL Drives"] + VTLDATA["Disk-backed Tape Images\n(under repo/pool)"] + end + + BSVR --> TDISK + BSVR --> TPHYS + BSVR --> TVTL + + TDISK --> DISK + + TPHYS --> PHYCHG + TPHYS --> PHYDRV + + TVTL --> VTLCHG + TVTL --> VTLDRV + VTLDRV --> VTLDATA + +sequenceDiagram + autonumber + participant UI as Web GUI + participant API as Calypso API (Go) + participant DB as PostgreSQL + participant SCST as SCST (iSCSI Target) + participant MHVTL as MHVTL + participant TOOLS as Tape Tools (mtx/mt/sg3) + participant WS as WebSocket Stream + + UI->>API: POST /api/v1/tape/inventory (physical or vtl) + API->>DB: create task (inventory) + API-->>UI: 202 Accepted + task_id + API->>WS: publish task_started(task_id) + + alt Physical inventory + API->>TOOLS: run inventory (mtx/sg3) + TOOLS-->>API: inventory result + else VTL inventory + API->>MHVTL: vtlcmd inventory + MHVTL-->>API: inventory result + end + + API->>DB: update task result + state=success + API->>WS: publish task_progress + task_success + UI->>WS: receive updates (realtime) + + UI->>API: POST /api/v1/iscsi/apply + API->>DB: audit + task (apply scst) + API->>SCST: scstadmin apply config (validate/rollback) + SCST-->>API: apply result + API->>WS: publish iscsi_apply_result + + + diff --git a/src/srs-technical-spec-documents/SRS-00-Main-AtlasOS-Calypso.md b/src/srs-technical-spec-documents/SRS-00-Main-AtlasOS-Calypso.md new file mode 100644 index 0000000..27f078c --- /dev/null +++ b/src/srs-technical-spec-documents/SRS-00-Main-AtlasOS-Calypso.md @@ -0,0 +1,87 @@ +# SRS-00 — AtlasOS – Calypso (Main) + +Version: 1.0 +Status: Draft – Baseline +Target OS: Ubuntu Server 24.04 LTS +Primary Use Case: Backup Appliance (Disk + Tape) + iSCSI Bridge to VM (Bacula) +Date: 2025 + +--- + +## 1. Purpose +AtlasOS – Calypso is an enterprise-grade backup appliance platform designed to unify disk-based backup repositories and physical tape libraries under a single, auditable, and storage-centric management plane. The system is optimized for reliability, determinism, and operational safety in enterprise backup environments. + +This document defines the **system-level software requirements** for AtlasOS – Calypso and acts as the parent SRS for all subsystem specifications. + +--- + +## 2. Scope + +### 2.1 In Scope +- Disk-based backup repository provisioning and management +- Physical tape library (LTO-8, SAS/FC) discovery and control +- SCST-based iSCSI target orchestration +- iSCSI bridge for tape library access by Bacula VM +- Integration with Bacula Storage Daemon +- System service management and diagnostics +- Authentication, authorization, and auditing +- Monitoring, alerting, and observability + +### 2.2 Out of Scope (Version 1) +- Bacula Director configuration and job scheduling +- Multi-tenant isolation +- High Availability (active-active) appliance clustering +- Cloud object storage tiering +- Tape encryption key lifecycle management + +--- + +## 3. System Architecture + +### 3.1 Logical Architecture + +Browser (React + Vite UI) +→ HTTPS (443) +→ Reverse Proxy (Caddy / Nginx) +→ Calypso Core API (Go) + +Calypso Core integrates with: +- SCST (Disk and Tape iSCSI targets) +- Tape utilities (lsscsi, sg3_utils, mtx, mt) +- Bacula Storage Daemon interfaces +- PostgreSQL (IAM, audit, state) +- systemd and journald + +--- + +## 4. User Roles +- Admin: Full system configuration and control +- Operator: Day-to-day tape operations and monitoring +- ReadOnly: Monitoring and reporting only + +--- + +## 5. System-Level Requirements +- Provide a web-based GUI for full appliance lifecycle management +- Expose a versioned REST API and WebSocket event stream +- Ensure deterministic SCSI behavior for disk and tape devices +- Enforce strict RBAC and full auditability +- Guarantee safe recovery after reboot or service restart + +--- + +## 6. Non-Functional Requirements +- UI response time < 300 ms for read operations +- Configuration apply time < 5 seconds +- All operations must be idempotent +- Mandatory TLS for all access +- Full compatibility with Ubuntu Server 24.04 LTS + +--- + +## 7. Acceptance Criteria +1. Bacula VM can access disk repository via iSCSI. +2. Bacula VM can access tape changer and drives via iSCSI. +3. Tape inventory, load/unload, and write operations succeed. +4. All lifecycle operations are controllable from the GUI. +5. System survives reboot without manual intervention. diff --git a/src/srs-technical-spec-documents/SRS-01-Storage-Component.md b/src/srs-technical-spec-documents/SRS-01-Storage-Component.md new file mode 100644 index 0000000..594a758 --- /dev/null +++ b/src/srs-technical-spec-documents/SRS-01-Storage-Component.md @@ -0,0 +1,61 @@ +# SRS-01 — AtlasOS – Calypso Storage Component + +Version: 1.0 +Target Capacity: 30–60 TB +Transport: iSCSI (SCST) +Date: 2025 + +--- + +## 1. Purpose +This document specifies requirements for the **Storage Repository Component**, which provides disk-based backup targets to Bacula through iSCSI. + +--- + +## 2. Scope + +### In Scope +- Physical disk discovery +- Backup repository provisioning +- iSCSI LUN export +- Capacity monitoring and alerts + +### Out of Scope +- Snapshot scheduling +- Replication +- Deduplication or compression engines + +--- + +## 3. Functional Requirements + +### 3.1 Disk Discovery +- Detect block devices and volume groups +- Display disk size, vendor, and health status + +### 3.2 Repository Provisioning +- Create repositories using LVM Logical Volumes +- Optional support for ZFS zvol +- Resize repositories with safety validation + +### 3.3 iSCSI Export +- Export repository as a dedicated iSCSI target +- Map repository as LUN 0 +- Enforce initiator ACL + +### 3.4 Capacity Monitoring +- Display used and free capacity +- Configurable warning and critical thresholds + +--- + +## 4. Non-Functional Requirements +- No data loss during resize operations +- Minimal disruption to active iSCSI sessions + +--- + +## 5. Acceptance Criteria +1. Repository is visible as an iSCSI LUN. +2. Bacula can mount and write to the disk target. +3. Threshold alerts trigger correctly. diff --git a/src/srs-technical-spec-documents/SRS-02-VTL-and-Tape-Bridge.md b/src/srs-technical-spec-documents/SRS-02-VTL-and-Tape-Bridge.md new file mode 100644 index 0000000..5c8351a --- /dev/null +++ b/src/srs-technical-spec-documents/SRS-02-VTL-and-Tape-Bridge.md @@ -0,0 +1,58 @@ +# SRS-02 — AtlasOS – Calypso VTL & Tape Bridge + +Version: 1.0 +Target: Physical Tape Library LTO-8 (SAS / FC) +Maximum Drives: 8 +Date: 2025 + +--- + +## 1. Purpose +This document defines requirements for managing and exporting **physical tape libraries** through iSCSI, allowing Bacula to access tape hardware without direct attachment. + +--- + +## 2. Scope + +### In Scope +- Tape changer and drive discovery +- Slot inventory and barcode handling +- Tape load and unload operations +- iSCSI export of changer and drives + +### Out of Scope +- Virtual tape emulation +- Tape encryption workflows +- Tape duplication policies + +--- + +## 3. Functional Requirements + +### 3.1 Discovery +- Detect tape changer and drives via SCSI inquiry +- Group devices into a single logical library + +### 3.2 Operations +- Perform slot inventory +- Load tape into drive +- Unload tape to slot +- Display drive state and errors + +### 3.3 iSCSI Mapping +- LUN 0 mapped to changer +- LUN 1–8 mapped to tape drives +- Enforce single-initiator access + +--- + +## 4. Non-Functional Requirements +- Operations must be asynchronous +- Prevent conflicting concurrent actions + +--- + +## 5. Acceptance Criteria +1. Tape library is visible in Bacula. +2. Inventory and load/unload succeed. +3. No multi-initiator conflicts occur. diff --git a/src/srs-technical-spec-documents/SRS-03-System-Management.md b/src/srs-technical-spec-documents/SRS-03-System-Management.md new file mode 100644 index 0000000..8bb3672 --- /dev/null +++ b/src/srs-technical-spec-documents/SRS-03-System-Management.md @@ -0,0 +1,31 @@ +# SRS-03 — AtlasOS – Calypso System Management + +Version: 1.0 +Target OS: Ubuntu Server 24.04 +Date: 2025 + +--- + +## 1. Purpose +Defines requirements for managing system services, diagnostics, and appliance health. + +--- + +## 2. Functional Requirements +- Display systemd service status +- Restart services with RBAC enforcement +- View journald logs +- Generate diagnostic support bundles + +--- + +## 3. Constraints +- No shell access from the UI +- All actions must be audited + +--- + +## 4. Acceptance Criteria +1. Services are manageable from the UI. +2. Logs are accessible for troubleshooting. +3. Support bundles can be generated and downloaded. diff --git a/src/srs-technical-spec-documents/SRS-04-Auth-and-IAM.md b/src/srs-technical-spec-documents/SRS-04-Auth-and-IAM.md new file mode 100644 index 0000000..06534f7 --- /dev/null +++ b/src/srs-technical-spec-documents/SRS-04-Auth-and-IAM.md @@ -0,0 +1,38 @@ +# SRS-04 — AtlasOS – Calypso Authentication & IAM + +Version: 1.0 +Identity Sources: Local (PostgreSQL), LDAP, OIDC SSO +Date: 2025 + +--- + +## 1. Purpose +Defines authentication, authorization, and audit requirements for AtlasOS – Calypso. + +--- + +## 2. Authentication Methods +- Local authentication with Argon2id password hashing +- LDAP bind authentication +- OIDC-based SSO integration + +--- + +## 3. Authorization +- Role-Based Access Control (Admin, Operator, ReadOnly) +- Permission enforcement per API endpoint + +--- + +## 4. Audit Logging +- Login success and failure +- Configuration changes +- Privileged system operations + +--- + +## 5. Acceptance Criteria +1. Local login works correctly. +2. LDAP authentication functions as expected. +3. OIDC login succeeds. +4. Audit trail is complete and immutable. diff --git a/src/srs-technical-spec-documents/SRS-05-Monitoring-and-Observability.md b/src/srs-technical-spec-documents/SRS-05-Monitoring-and-Observability.md new file mode 100644 index 0000000..4efe73f --- /dev/null +++ b/src/srs-technical-spec-documents/SRS-05-Monitoring-and-Observability.md @@ -0,0 +1,36 @@ +# SRS-05 — AtlasOS – Calypso Monitoring & Observability + +Version: 1.0 +Date: 2025 + +--- + +## 1. Purpose +Defines monitoring, alerting, and observability requirements for the Calypso appliance. + +--- + +## 2. Monitoring Scope +- Disk capacity and health +- Tape drive and changer status +- iSCSI session stability +- Service uptime + +--- + +## 3. Alerts +- Disk capacity thresholds +- Tape errors and drive offline +- Service failures + +--- + +## 4. Event Streaming +- Real-time UI updates via WebSocket + +--- + +## 5. Acceptance Criteria +1. Health states are visible in the UI. +2. Alerts trigger correctly. +3. Events stream to the UI in real time. diff --git a/src/srs-technical-spec-documents/component-list-atlasos-calypso.md b/src/srs-technical-spec-documents/component-list-atlasos-calypso.md new file mode 100644 index 0000000..734cc6f --- /dev/null +++ b/src/srs-technical-spec-documents/component-list-atlasos-calypso.md @@ -0,0 +1,262 @@ +# AtlasOS – Calypso +## Appliance Component List (Baseline v1) + +Version: 1.0 +Status: Baseline – Product Definition +Target OS: Ubuntu Server 24.04 LTS +Date: 2025 + +--- + +## 1. Overview + +AtlasOS – Calypso is a **generic backup appliance platform** that provides: + +- Disk-based backup storage +- Physical tape library bridging (SAS / FC) +- Virtual Tape Library (VTL) services +- iSCSI presentation for **any backup software** +- Centralized management via web-based GUI +- Strong security, auditability, and observability + +This document enumerates **all required components** for building, operating, and packaging the appliance. + +--- + +## 2. Base Platform Components + +### 2.1 Operating System +- Ubuntu Server 24.04 LTS +- systemd +- journald +- udev (persistent device naming) +- chrony (time synchronization) +- ufw / nftables (firewall) + +--- + +## 3. Disk Storage Layer + +Used for: +- Backup repository (disk target) +- Virtual tape backing store (MHVTL) + +Components: +- LVM2 +- thin-provisioning-tools +- XFS (primary filesystem) +- ext4 (alternative) +- (Optional) ZFS (advanced SKU) +- smartmontools +- nvme-cli +- parted / gdisk + +--- + +## 4. Physical Tape Subsystem (SAS / FC) + +Used for: +- Physical tape library discovery +- Changer and drive operations +- Slot inventory +- Load / unload operations + +Components: +- SAS or FC HBA drivers (Broadcom / LSI / QLogic / Emulex) +- multipath-tools (FC dual-path environments) +- lsscsi +- sg3_utils +- mt-st +- mtx + +--- + +## 5. Virtual Tape Library (VTL) + +Provides: +- Disk-backed virtual tape services +- Virtual medium changer +- Virtual tape drives +- Virtual slots and barcodes + +Components: +- mhvtl +- mhvtl-utils / vtlcmd (depending on build) +- VTL configuration and tape image storage + +Use Cases: +- Backup staging +- Copy-to-tape workflows +- Test / development environments +- Air-gap logic simulation + +--- + +## 6. iSCSI Target Stack (Core Appliance Function) + +All storage and tape devices are presented using **SCST**. + +Components: +- scst (kernel modules) +- iscsi-scst +- scstadmin + +Usage: +- Disk repository iSCSI targets +- Physical tape library bridge +- Virtual tape library (MHVTL) export + +Design Rules: +- SCST is the **only** target framework +- One device is owned by one framework only +- Tape targets must use: + - LUN 0 → Medium changer + - LUN 1..N → Tape drives (max 8) +- Single initiator policy enforced for tape targets + +--- + +## 7. Calypso Core Application + +### 7.1 Backend (Go) + +Responsibilities: +- Disk storage orchestration (LVM/ZFS) +- Physical tape management +- VTL (MHVTL) management +- SCST iSCSI target orchestration +- Async task execution +- State and configuration management +- Audit logging +- Backup software compatibility profiles + +Modules: +- storage +- tape-physical +- tape-vtl +- iscsi +- scst +- tasks +- system +- monitoring +- audit +- profile-engine + +--- + +### 7.2 Frontend (Web GUI) + +Technology: +- React +- Vite +- TypeScript +- TailwindCSS + shadcn/ui +- TanStack Query +- WebSocket client + +Functional Areas: +- Dashboard +- Disk Repository Management +- Physical Tape Library +- Virtual Tape Library (VTL) +- iSCSI Targets & Initiators +- Tasks & Jobs +- Alerts & Logs +- System Settings +- Authentication & Users + +--- + +## 8. Authentication & IAM + +Components: +- PostgreSQL +- Local user authentication (Argon2id) +- LDAP authentication (AD / OpenLDAP) +- SSO via OIDC (Keycloak, Azure AD, Okta) +- RBAC (Admin / Operator / ReadOnly) +- Audit logging + +Database Entities: +- users +- roles +- permissions +- user_roles +- external_identities +- sessions +- audit_log + +--- + +## 9. Monitoring & Observability + +### 9.1 Built-in Monitoring +- Disk capacity and health +- Tape library status (physical & VTL) +- SCST and iSCSI sessions +- System services health +- Alert engine +- Event stream (WebSocket) + +### 9.2 Optional External Integration +- Prometheus metrics endpoint +- node_exporter +- Log forwarding (rsyslog / Loki) + +--- + +## 10. Web Access & Security Perimeter + +Components: +- Caddy (recommended) or Nginx +- TLS certificates (ACME or internal CA) +- HTTP security headers +- Rate limiting +- Reverse proxy to API and UI + +--- + +## 11. Packaging & Appliance Operations + +Components: +- Debian packages (.deb) +- systemd service units: + - calypso-api.service + - scst.service + - mhvtl.service + - postgresql.service + - caddy.service / nginx.service +- Installer scripts +- Upgrade workflow +- Support bundle generator (logs + configs + inventory) + +--- + +## 12. Backup Software Compatibility + +AtlasOS – Calypso is **backup software agnostic**. + +Supported via iSCSI: +- Bacula +- Veeam +- Dell NetWorker +- Veritas NetBackup +- Commvault +- Arcserve +- Any SCSI-compliant backup engine + +No backup software–specific logic is embedded in the core appliance. + +--- + +## 13. Status + +This component list represents the **authoritative baseline** for: +- Architecture decisions +- SRS alignment +- Implementation planning +- Packaging and deployment + +All other documents must conform to this component list. + +--- diff --git a/src/srs-technical-spec-documents/mhvtl-integration-blueprint b/src/srs-technical-spec-documents/mhvtl-integration-blueprint new file mode 100644 index 0000000..ecac4b0 --- /dev/null +++ b/src/srs-technical-spec-documents/mhvtl-integration-blueprint @@ -0,0 +1,362 @@ +# AtlasOS – Calypso +## SCST + MHVTL Integration Blueprint +### (Single Source – Engineering Grade) + +Version: 1.0 +Status: Baseline Blueprint +Target OS: Ubuntu Server 24.04 LTS +Scope: Virtual Tape Library (MHVTL) + Physical Tape Bridging via SCST iSCSI +Date: 2025 + +--- + +## 1. Purpose + +Dokumen ini mendefinisikan **blueprint integrasi resmi** antara: + +- **MHVTL** sebagai Virtual Tape Library provider +- **SCST** sebagai satu-satunya iSCSI target framework +- **AtlasOS – Calypso** sebagai orchestration & control plane + +Blueprint ini berlaku untuk: +- Virtual Tape Library (VTL) +- Bridging physical tape library (SAS / FC) +- iSCSI presentation ke **backup software apa pun** (vendor-agnostic) + +Dokumen ini bersifat **authoritative** untuk implementasi. + +--- + +## 2. Fundamental Design Principles + +### 2.1 Ownership Model +- **MHVTL** → producer perangkat SCSI virtual +- **Physical tape library** → producer perangkat SCSI fisik +- **SCST** → exporter perangkat ke iSCSI +- **Backup software** → iSCSI initiator (consumer) + +Tidak boleh ada dua framework yang mengelola device yang sama. + +--- + +### 2.2 Mandatory Rules + +1. SCST adalah **satu-satunya iSCSI target framework** +2. Semua tape (physical & virtual) wajib diexport via SCST +3. Mapping tape device: + - LUN 0 → Medium Changer + - LUN 1..N → Tape Drives (maks 8) +4. Tape target **single-initiator only** +5. Tidak ada backup-software-specific logic di core +6. Semua operasi destructive / privileged: + - async + - auditable + - guarded + +--- + +## 3. High-Level Integration Architecture + +### 3.1 Data Plane + +Backup Software +| +iSCSI +| +SCST +| ++------------------+ +| Backend Device | ++------------------+ +| | +MHVTL Physical Tape +| +Disk Backing Store + + +### 3.2 Control Plane +- Calypso API mengontrol: + - lifecycle MHVTL + - SCST config generation & apply + - device discovery + - state & policy enforcement +- UI hanya berbicara ke API +- Tidak ada direct shell access + +--- + +## 4. Device Model + +### 4.1 Virtual Tape Library (MHVTL) +MHVTL menyediakan: +- Medium changer device +- Tape drive devices +- Slot & barcode emulation +- Tape image files (disk-backed) + +Device yang dihasilkan: +- `/dev/sgX` (generic SCSI) +- `/dev/stX` / `/dev/nstX` (tape) +- `/dev/schX` (changer) + +--- + +### 4.2 Physical Tape Library +Physical library menyediakan: +- Medium changer (SAS / FC) +- Tape drives (LTO-8) +- Slots & barcodes + +Device yang dihasilkan: +- `/dev/sgX` +- `/dev/stX` / `/dev/nstX` +- `/dev/schX` + +--- + +## 5. Stable Device Naming (Critical) + +### Problem +Nama `/dev/sg*`, `/dev/st*`, `/dev/sch*` **tidak stabil** antar reboot. + +### Blueprint Solution +1. Discover device via: + - `lsscsi -g` + - `sg_inq` +2. Ambil: + - vendor + - model + - serial + - device type +3. Generate udev rules → symlink stabil + +### Recommended Symlink Layout + +/dev/calypso/ +vtl/ +/ +changer +drive1 +drive2 +physical/ +/ +changer +drive1 +drive2 + + +SCST **hanya** boleh mereferensikan path stabil ini. + +--- + +## 6. SCST Backend Strategy + +### 6.1 Handler Selection +Karena variasi build SCST, Calypso wajib melakukan **capability detection**. + +Supported modes: +- Native tape handler (jika tersedia) +- Generic SCSI passthrough (`sg` based) + +### 6.2 Recommended Default +- **Changer** → SCSI generic passthrough +- **Drive** → tape char device (`nst`) atau sg fallback + +Handler mode dipilih **per target**, disimpan di DB. + +--- + +## 7. iSCSI Target Model + +### 7.1 Target Naming + + +iqn..atlasos.calypso:vtl. +iqn..atlasos.calypso:tape.physical. + + + +### 7.2 LUN Mapping (Mandatory) +| LUN | Device | +|----|-------| +| 0 | Medium Changer | +| 1 | Tape Drive 1 | +| 2 | Tape Drive 2 | +| … | … (max 8) | + +### 7.3 ACL Policy +- Allow only explicit initiator IQN +- Tape targets: **single initiator enforced** + +--- + +## 8. Workflow: Create & Export VTL + +### Step 1 — Prepare Backing Store +- Allocate disk path for tape images: + + +/var/lib/calypso/vtl//tapes + + +### Step 2 — Create MHVTL Library +Calypso invokes MHVTL tools to: +- create changer +- create drives (1–8) +- define slots +- enable barcode + +Result: +- Device nodes appear +- MHVTL config persisted + +--- + +### Step 3 — Device Discovery +- Run `lsscsi -g` +- Match new devices +- Persist mapping in DB +- Generate udev rules + +--- + +### Step 4 — Generate SCST Config +- Define target IQN +- Define LUN mappings +- Define ACL + +Config generated to: + +/etc/calypso/scst/generated.conf + + +--- + +### Step 5 — Apply SCST (Safe Mode) +1. Validate config +2. Apply via `scstadmin` +3. Verify target visibility +4. On failure → rollback to last-known-good + +--- + +### Step 6 — Initiator Validation +Calypso provides validation profile: +- Linux initiator test +- Windows initiator hints +- Confirm: + - changer visible + - drives visible + - basic SCSI ops OK + +--- + +## 9. Guardrails & Safety Rules + +### Hard Blocks +- No SCST apply while: + - drive busy + - tape loaded & writing +- No delete library with: + - active initiator + - loaded tape + +### Soft Alerts +- Multiple initiators → CRITICAL alert +- Backing store nearly full → WARNING / CRITICAL + +--- + +## 10. Observability Requirements + +Calypso must monitor: +- SCST target state +- iSCSI sessions +- MHVTL status: + - drives count + - slots usage + - error logs +- Disk usage (tape images) + +Events pushed via WebSocket to UI. + +--- + +## 11. Persisted State + +### PostgreSQL +- vtl_libraries +- vtl_drives +- vtl_tapes +- iscsi_targets +- lun_mappings +- tasks +- audit_log +- alerts + +### Filesystem + +/etc/calypso/ +scst/ +generated.conf +last-known-good.conf +udev/ + +/var/lib/calypso/ +vtl/ +state/ + + +--- + +## 12. Example Mapping (Conceptual) + +**Library:** `vtl01` +**Target IQN:** `iqn.2025.atlasos.calypso:vtl.vtl01` + +| LUN | Device Path | +|----|------------| +| 0 | /dev/calypso/vtl/vtl01/changer | +| 1 | /dev/calypso/vtl/vtl01/drive1 | +| 2 | /dev/calypso/vtl/vtl01/drive2 | + +--- + +## 13. Required Adapters in Calypso + +1. **MHVTL Adapter** + - create/delete library + - manage tapes + - status & logs + +2. **SCST Adapter** + - config generation + - apply / rollback + - session listing + +3. **Discovery Adapter** + - lsscsi mapping + - sg_inq identification + - udev rule generation + +4. **Policy Engine** + - single-initiator enforcement + - safe-apply checks + +--- + +## 14. Final Authority + +Dokumen ini adalah **blueprint resmi integrasi SCST + MHVTL**. + +Jika ada konflik: +- Blueprint ini +- CURSOR.md +- SRS + +maka **Blueprint ini menang untuk domain SCST + MHVTL**. + +AtlasOS – Calypso diperlakukan sebagai **enterprise tape & VTL appliance**, bukan sekadar lab tool. + +--- + diff --git a/src/srs-technical-spec-documents/mhvtl-integration-blueprint.md b/src/srs-technical-spec-documents/mhvtl-integration-blueprint.md new file mode 100644 index 0000000..ecac4b0 --- /dev/null +++ b/src/srs-technical-spec-documents/mhvtl-integration-blueprint.md @@ -0,0 +1,362 @@ +# AtlasOS – Calypso +## SCST + MHVTL Integration Blueprint +### (Single Source – Engineering Grade) + +Version: 1.0 +Status: Baseline Blueprint +Target OS: Ubuntu Server 24.04 LTS +Scope: Virtual Tape Library (MHVTL) + Physical Tape Bridging via SCST iSCSI +Date: 2025 + +--- + +## 1. Purpose + +Dokumen ini mendefinisikan **blueprint integrasi resmi** antara: + +- **MHVTL** sebagai Virtual Tape Library provider +- **SCST** sebagai satu-satunya iSCSI target framework +- **AtlasOS – Calypso** sebagai orchestration & control plane + +Blueprint ini berlaku untuk: +- Virtual Tape Library (VTL) +- Bridging physical tape library (SAS / FC) +- iSCSI presentation ke **backup software apa pun** (vendor-agnostic) + +Dokumen ini bersifat **authoritative** untuk implementasi. + +--- + +## 2. Fundamental Design Principles + +### 2.1 Ownership Model +- **MHVTL** → producer perangkat SCSI virtual +- **Physical tape library** → producer perangkat SCSI fisik +- **SCST** → exporter perangkat ke iSCSI +- **Backup software** → iSCSI initiator (consumer) + +Tidak boleh ada dua framework yang mengelola device yang sama. + +--- + +### 2.2 Mandatory Rules + +1. SCST adalah **satu-satunya iSCSI target framework** +2. Semua tape (physical & virtual) wajib diexport via SCST +3. Mapping tape device: + - LUN 0 → Medium Changer + - LUN 1..N → Tape Drives (maks 8) +4. Tape target **single-initiator only** +5. Tidak ada backup-software-specific logic di core +6. Semua operasi destructive / privileged: + - async + - auditable + - guarded + +--- + +## 3. High-Level Integration Architecture + +### 3.1 Data Plane + +Backup Software +| +iSCSI +| +SCST +| ++------------------+ +| Backend Device | ++------------------+ +| | +MHVTL Physical Tape +| +Disk Backing Store + + +### 3.2 Control Plane +- Calypso API mengontrol: + - lifecycle MHVTL + - SCST config generation & apply + - device discovery + - state & policy enforcement +- UI hanya berbicara ke API +- Tidak ada direct shell access + +--- + +## 4. Device Model + +### 4.1 Virtual Tape Library (MHVTL) +MHVTL menyediakan: +- Medium changer device +- Tape drive devices +- Slot & barcode emulation +- Tape image files (disk-backed) + +Device yang dihasilkan: +- `/dev/sgX` (generic SCSI) +- `/dev/stX` / `/dev/nstX` (tape) +- `/dev/schX` (changer) + +--- + +### 4.2 Physical Tape Library +Physical library menyediakan: +- Medium changer (SAS / FC) +- Tape drives (LTO-8) +- Slots & barcodes + +Device yang dihasilkan: +- `/dev/sgX` +- `/dev/stX` / `/dev/nstX` +- `/dev/schX` + +--- + +## 5. Stable Device Naming (Critical) + +### Problem +Nama `/dev/sg*`, `/dev/st*`, `/dev/sch*` **tidak stabil** antar reboot. + +### Blueprint Solution +1. Discover device via: + - `lsscsi -g` + - `sg_inq` +2. Ambil: + - vendor + - model + - serial + - device type +3. Generate udev rules → symlink stabil + +### Recommended Symlink Layout + +/dev/calypso/ +vtl/ +/ +changer +drive1 +drive2 +physical/ +/ +changer +drive1 +drive2 + + +SCST **hanya** boleh mereferensikan path stabil ini. + +--- + +## 6. SCST Backend Strategy + +### 6.1 Handler Selection +Karena variasi build SCST, Calypso wajib melakukan **capability detection**. + +Supported modes: +- Native tape handler (jika tersedia) +- Generic SCSI passthrough (`sg` based) + +### 6.2 Recommended Default +- **Changer** → SCSI generic passthrough +- **Drive** → tape char device (`nst`) atau sg fallback + +Handler mode dipilih **per target**, disimpan di DB. + +--- + +## 7. iSCSI Target Model + +### 7.1 Target Naming + + +iqn..atlasos.calypso:vtl. +iqn..atlasos.calypso:tape.physical. + + + +### 7.2 LUN Mapping (Mandatory) +| LUN | Device | +|----|-------| +| 0 | Medium Changer | +| 1 | Tape Drive 1 | +| 2 | Tape Drive 2 | +| … | … (max 8) | + +### 7.3 ACL Policy +- Allow only explicit initiator IQN +- Tape targets: **single initiator enforced** + +--- + +## 8. Workflow: Create & Export VTL + +### Step 1 — Prepare Backing Store +- Allocate disk path for tape images: + + +/var/lib/calypso/vtl//tapes + + +### Step 2 — Create MHVTL Library +Calypso invokes MHVTL tools to: +- create changer +- create drives (1–8) +- define slots +- enable barcode + +Result: +- Device nodes appear +- MHVTL config persisted + +--- + +### Step 3 — Device Discovery +- Run `lsscsi -g` +- Match new devices +- Persist mapping in DB +- Generate udev rules + +--- + +### Step 4 — Generate SCST Config +- Define target IQN +- Define LUN mappings +- Define ACL + +Config generated to: + +/etc/calypso/scst/generated.conf + + +--- + +### Step 5 — Apply SCST (Safe Mode) +1. Validate config +2. Apply via `scstadmin` +3. Verify target visibility +4. On failure → rollback to last-known-good + +--- + +### Step 6 — Initiator Validation +Calypso provides validation profile: +- Linux initiator test +- Windows initiator hints +- Confirm: + - changer visible + - drives visible + - basic SCSI ops OK + +--- + +## 9. Guardrails & Safety Rules + +### Hard Blocks +- No SCST apply while: + - drive busy + - tape loaded & writing +- No delete library with: + - active initiator + - loaded tape + +### Soft Alerts +- Multiple initiators → CRITICAL alert +- Backing store nearly full → WARNING / CRITICAL + +--- + +## 10. Observability Requirements + +Calypso must monitor: +- SCST target state +- iSCSI sessions +- MHVTL status: + - drives count + - slots usage + - error logs +- Disk usage (tape images) + +Events pushed via WebSocket to UI. + +--- + +## 11. Persisted State + +### PostgreSQL +- vtl_libraries +- vtl_drives +- vtl_tapes +- iscsi_targets +- lun_mappings +- tasks +- audit_log +- alerts + +### Filesystem + +/etc/calypso/ +scst/ +generated.conf +last-known-good.conf +udev/ + +/var/lib/calypso/ +vtl/ +state/ + + +--- + +## 12. Example Mapping (Conceptual) + +**Library:** `vtl01` +**Target IQN:** `iqn.2025.atlasos.calypso:vtl.vtl01` + +| LUN | Device Path | +|----|------------| +| 0 | /dev/calypso/vtl/vtl01/changer | +| 1 | /dev/calypso/vtl/vtl01/drive1 | +| 2 | /dev/calypso/vtl/vtl01/drive2 | + +--- + +## 13. Required Adapters in Calypso + +1. **MHVTL Adapter** + - create/delete library + - manage tapes + - status & logs + +2. **SCST Adapter** + - config generation + - apply / rollback + - session listing + +3. **Discovery Adapter** + - lsscsi mapping + - sg_inq identification + - udev rule generation + +4. **Policy Engine** + - single-initiator enforcement + - safe-apply checks + +--- + +## 14. Final Authority + +Dokumen ini adalah **blueprint resmi integrasi SCST + MHVTL**. + +Jika ada konflik: +- Blueprint ini +- CURSOR.md +- SRS + +maka **Blueprint ini menang untuk domain SCST + MHVTL**. + +AtlasOS – Calypso diperlakukan sebagai **enterprise tape & VTL appliance**, bukan sekadar lab tool. + +--- + diff --git a/src/srs-technical-spec-documents/scst-sample-config.md b/src/srs-technical-spec-documents/scst-sample-config.md new file mode 100644 index 0000000..1a33b82 --- /dev/null +++ b/src/srs-technical-spec-documents/scst-sample-config.md @@ -0,0 +1,297 @@ +# AtlasOS – Calypso +## SCST Sample Configuration Blueprint +### (Disk + VTL (MHVTL) + Physical Tape Bridge) + +Version: 1.0 +Status: Reference / Implementation Template +Target OS: Ubuntu Server 24.04 LTS +SCST Role: **Single iSCSI Target Framework** +Date: 2025 + +--- + +## 1. Scope & Philosophy + +Dokumen ini berisi **contoh konfigurasi SCST** yang digunakan oleh +AtlasOS – Calypso untuk: + +- Disk-based backup repository (iSCSI block) +- Virtual Tape Library (MHVTL) +- Physical tape library bridge (SAS / FC → iSCSI) + +⚠️ Catatan penting: +- **SCST tidak memiliki satu format config universal** antar distro/build. +- Pendekatan paling aman adalah **scstadmin-based configuration**. +- File ini adalah **template konseptual + praktis**, bukan hard-coded syntax + yang dipaksakan ke semua environment. + +Calypso akan **meng-generate dan meng-apply konfigurasi ini secara dinamis**. + +--- + +## 2. Global Assumptions & Placeholders + +Ganti placeholder berikut sesuai environment: + +| Placeholder | Contoh | +|------------|-------| +| `` | `iqn.1993-08.org.debian:01:backupvm` | +| `` | `iqn.2025.atlasos.calypso:repo.main` | +| `` | `iqn.2025.atlasos.calypso:vtl.vtl01` | +| `` | `iqn.2025.atlasos.calypso:tape.physical.lib01` | + +### Stable Device Paths (via udev – recommended) + +/dev/calypso/ +repo/ +lun0 +vtl/ +vtl01/ +changer +drive1 +drive2 +physical/ +lib01/ +changer +drive1 +drive2 + + +⚠️ SCST **HARUS** memakai path stabil ini, bukan `/dev/sgX` langsung. + +--- + +## 3. Pre-flight Checks + +```bash +systemctl status scst --no-pager +systemctl status iscsi-scst --no-pager + +Capability detection: + +scstadmin -list_handler +scstadmin -list_driver +scstadmin -list_target + +Calypso WAJIB mendeteksi handler yang tersedia sebelum apply config. + +4. Disk Repository Target (iSCSI Block) +4.1 Create Backend Device + +Biasanya menggunakan handler vdisk. + +scstadmin -open_dev repo_lun0 \ + -handler vdisk \ + -attributes filename=/dev/calypso/repo/lun0 + + +4.2 Create iSCSI Target & LUN Mapping +scstadmin -add_target -driver iscsi + +scstadmin -add_lun 0 \ + -target \ + -driver iscsi \ + -device repo_lun0 + +4.3 Initiator ACL +scstadmin -add_group repo_acl \ + -target \ + -driver iscsi + +scstadmin -add_init \ + -group repo_acl \ + -target \ + -driver iscsi + +(Opsional: CHAP bila diperlukan) + +5. Virtual Tape Library (MHVTL → SCST) +5.1 Handler Strategy + +Karena variasi build SCST, mode default paling aman: + +Changer → SCSI Generic (sg passthrough) + +Drives → SCSI Generic (atau native tape handler jika tersedia) + +Handler aktual ditentukan lewat: + +scstadmin -list_handler + +Gunakan placeholder . + +5.2 Create Backend Devices (VTL) +# Changer +scstadmin -open_dev vtl01_changer \ + -handler \ + -attributes filename=/dev/calypso/vtl/vtl01/changer + +# Drives +scstadmin -open_dev vtl01_drive1 \ + -handler \ + -attributes filename=/dev/calypso/vtl/vtl01/drive1 + +scstadmin -open_dev vtl01_drive2 \ + -handler \ + -attributes filename=/dev/calypso/vtl/vtl01/drive2 + + +5.3 Create iSCSI Target & LUN Mapping +scstadmin -add_target -driver iscsi + +# Mandatory LUN layout +scstadmin -add_lun 0 \ + -target \ + -driver iscsi \ + -device vtl01_changer + +scstadmin -add_lun 1 \ + -target \ + -driver iscsi \ + -device vtl01_drive1 + +scstadmin -add_lun 2 \ + -target \ + -driver iscsi \ + -device vtl01_drive2 + + +5.4 ACL (Single Initiator) +scstadmin -add_group vtl_acl \ + -target \ + -driver iscsi + +scstadmin -add_init \ + -group vtl_acl \ + -target \ + -driver iscsi + + +⚠️ Tape target WAJIB single initiator. + +6. Physical Tape Library Bridge (SAS / FC) +6.1 Backend Devices + +# Changer +scstadmin -open_dev phys_changer \ + -handler \ + -attributes filename=/dev/calypso/physical/lib01/changer + +# Drives +scstadmin -open_dev phys_drive1 \ + -handler \ + -attributes filename=/dev/calypso/physical/lib01/drive1 + +scstadmin -open_dev phys_drive2 \ + -handler \ + -attributes filename=/dev/calypso/physical/lib01/drive2 + + +6.2 Target & LUN Mapping +scstadmin -add_target -driver iscsi + +scstadmin -add_lun 0 \ + -target \ + -driver iscsi \ + -device phys_changer + +scstadmin -add_lun 1 \ + -target \ + -driver iscsi \ + -device phys_drive1 + +scstadmin -add_lun 2 \ + -target \ + -driver iscsi \ + -device phys_drive2 + + +6.3 ACL (Single Initiator) +scstadmin -add_group phys_acl \ + -target \ + -driver iscsi + +scstadmin -add_init \ + -group phys_acl \ + -target \ + -driver iscsi + +7. Persist Configuration + +Tergantung packaging SCST: + +Common Method +scstadmin -write_config /etc/scst.conf + + +Calypso Recommended Pattern +/etc/calypso/scst/ + generated.conf + last-known-good.conf + + +Flow: + +Generate → validate + +Apply + +Health check + +Promote to last-known-good + +8. Initiator Validation +iscsiadm -m discovery -t sendtargets -p +iscsiadm -m node --login +lsscsi -g + + +Untuk tape: +sg_inq /dev/sgX + + +Backup software harus melihat: + +1 medium changer +N tape drives +Slot inventory valid + +9. Operational Guardrails +Forbidden + +Apply config saat drive busy + +Delete tape target saat initiator login + +Multiple initiators ke tape target + +Mandatory + +Audit semua perubahan + +Async task execution + +Rollback otomatis jika apply gagal + +10. Authority + +Dokumen ini adalah reference resmi untuk: + +SCST config generation + +VTL export + +Physical tape bridge + +Jika ada konflik: + +Blueprint SCST + MHVTL + +CURSOR.md + +SRS + +maka dokumen ini menang untuk domain SCST configuration. + +AtlasOS – Calypso diperlakukan sebagai +enterprise-grade tape & VTL appliance, bukan sekadar target iSCSI biasa. \ No newline at end of file diff --git a/src/web/dashboard-code.html b/src/web/dashboard-code.html new file mode 100644 index 0000000..50db20c --- /dev/null +++ b/src/web/dashboard-code.html @@ -0,0 +1,453 @@ + + + + + +Backup Appliance - Monitoring & Logs + + + + + + + + + + + + + +
+ + + +
+ +
+
+
+

System Monitor

+

Real-time telemetry, ZFS health, and system event logs

+
+
+
+ + + + +System Healthy +
+ +
+
+
+ +
+
+ +
+ +
+
+

CPU Load

+memory +
+
+

12%

+ +trending_down 2% + +
+ +
+
+
+
+ +
+
+

RAM Usage

+memory_alt +
+
+

8.4 GB

+/ 32 GB +
+
+
+
+
+ +
+
+

ZFS Pool Status

+check_circle +
+
+

Online

+No Errors +
+
+
+
+
+
+
+
+ +
+
+

System Uptime

+schedule +
+
+

14d 2h 12m

+
+

Last reboot: Manual Patching

+
+
+ +
+ +
+ +
+
+
+

Network Throughput

+

Inbound vs Outbound (eth0)

+
+
+

1.2 Gbps

+

Peak: 2.1 Gbps

+
+
+
+ + + + + + + + + + + + + + + +
+10:00 +10:15 +10:30 +10:45 +11:00 +
+
+
+ +
+
+
+

ZFS ARC Hit Ratio

+

Cache efficiency

+
+
+

94%

+

Target: >90%

+
+
+
+ + + + +
+
95%
+
+
+
+ +
+
+
+

Disk Health

+Pool 1 +
+
+ +
+hard_drive +da0 + +
+
+hard_drive +da1 +
+
+hard_drive +da2 +
+
+hard_drive +da3 +
+
+hard_drive +da4 +
+ +
+ +warning +da5 +
+ +
+hard_drive +da6 +
+
+hard_drive +da7 +
+ +
+Empty +
+
+Empty +
+
+Empty +
+
+Empty +
+
+
+
+Total Capacity +64 TB +
+
+
+
+
+Used: 41.6 TB +Free: 22.4 TB +
+
+
+
+
+ +
+ +
+ + + +
+
+search + +
+
+
+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Job NameTypeProgressSpeedStatus
Daily Backup: VM-Cluster-01Replication +
+
+
+
+
+
+45% +
+

ETA: 1h 12m

+		145 MB/s + + Running + +
ZFS Scrub: Pool-01Maintenance +
+
+
+
+78% +
+		1.2 GB/s + + Running + +
+
+ +
+

Recent System Events

+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
10:45:22 +INFO +systemdStarted User Manager for UID 1000.
10:45:15 +WARN +smartdDevice: /dev/ada5, SMART Usage Attribute: 194 Temperature_Celsius changed from 38 to 41
10:44:58 +INFO +kernelix0: link state changed to UP
10:42:10 +INFO +zfszfs_arc_reclaim_thread: reclaiming 157286400 bytes ...
+
+
+
+
+
+
+
+ \ No newline at end of file diff --git a/src/web/dashboard.png b/src/web/dashboard.png new file mode 100644 index 0000000..0a2da8a Binary files /dev/null and b/src/web/dashboard.png differ diff --git a/src/web/iscsi-management-code.html b/src/web/iscsi-management-code.html new file mode 100644 index 0000000..e34e88d --- /dev/null +++ b/src/web/iscsi-management-code.html @@ -0,0 +1,401 @@ + + + + + +iSCSI Management - Backup Appliance + + + + + + + + + + +
+ +
+
+ +
+
+
+

Backup Appliance

+

Host: backup-z01

+
+
+ + +
+ +logout +Logout + +
+
+
+ +
+ +
+
+ +
+Storage +chevron_right +iSCSI Management +
+ +
+
+

iSCSI Management

+

Manage targets, portals, and initiator access control lists.

+
+
+ + +
+
+ +
+
+
+

Service Status

+check_circle +
+

Running

+

Uptime: 14d 2h

+
+
+
+

Port Binding

+dns +
+

3260

+

Listening on 0.0.0.0

+
+
+
+

Active Sessions

+swap_horiz +
+
+

12

+ +arrow_upward 2 + +
+

Total throughput: 450 MB/s

+
+
+ +
+ +
+
+ + + + +
+
+ +
+
+search + +
+
+
+Filter: + +
+
+
+ +
+ +
+ +
+
+dns +
+
+
+backup-target-01 +Online +
+
+iqn.2023-10.lan.backup:target01 + +
+
+ + +
+ +
+
+ +
+
+

Attached LUNs

+ +
+
+ +
+
+
+pie_chart +
+
+LUN 0 +zvol/tank/vm-backups/win-server +
+
+
+500 GB +
+
+ +
+
+
+pie_chart +
+
+LUN 1 +file/mnt/tank/iso-store +
+
+
+2.5 TB +
+
+
+
+ +
+
+

Access Control

+ +
+
+
+
+Auth Method +CHAP Mutual +
+
+Initiator Group +group-esxi-cluster-01 +
+
+

Allowed Initiators (3):

+

iqn.1998-01.com.vmware:esx01-4a2b...

+

iqn.1998-01.com.vmware:esx02-9c3d...

+
+
+
+
+ +
+ + + +
+
+
+
+ +
+
+
+dns +
+
+
+archive-cold-storage +Idle +
+
+iqn.2023-10.lan.backup:archive002 + +
+
+ + +
+
+ +
+
+
+dns +
+
+
+dev-sandbox-target +Offline +
+
+iqn.2023-10.lan.backup:sandbox-dev + +
+
+ + +
+
+
+ +
+

Showing 1-3 of 3 targets

+
+ + +
+
+
+
+
+
+
+ \ No newline at end of file diff --git a/src/web/iscsi-management.png b/src/web/iscsi-management.png new file mode 100644 index 0000000..b89cc21 Binary files /dev/null and b/src/web/iscsi-management.png differ diff --git a/src/web/shares-management-code.html b/src/web/shares-management-code.html new file mode 100644 index 0000000..1060c70 --- /dev/null +++ b/src/web/shares-management-code.html @@ -0,0 +1,470 @@ + + + + + +Shares Management + + + + + + + + + + + + + + +
+ + + +
+ +
+
+
+
+

Shares Management

+
+Storage +chevron_right +Shares +chevron_right +Overview +
+
+ +
+ +
+
+
+

SMB Service

+
+
+

Running

+

Port 445 Active

+
+
+
+

NFS Service

+
+
+

Running

+

Port 2049 Active

+
+
+
+

Throughput

+speed +
+

565 MB/s

+

14 Clients Connected

+
+
+
+
+ +
+ +
+ +
+
+ +search +
+
+ +
+ +
+
+folder_open +
+
+

pool/tank/home

+
+

/mnt/tank/home

+
+SMB +NFS +
+
+chevron_right +
+
+ +
+
+dns +
+
+

pool/tank/vm_storage

+
+

/mnt/tank/vm_storage

+
+SMB +NFS +
+
+
+
+ +
+
+folder_open +
+
+

pool/tank/backups

+
+

/mnt/tank/backups

+
+SMB +
+
+chevron_right +
+
+ +
+
+folder_open +
+
+

pool/tank/media

+
+

/mnt/tank/media

+
+SMB +NFS +
+
+chevron_right +
+
+
+
+

Showing 4 of 12 datasets

+
+
+ +
+ +
+
+
+
+
+dns +
+
+

vm_storage

+

pool/tank/vm_storage

+
+
+
+
+ + +
+
+ +
+ +
+
+
+folder_shared +
+
+SMB Protocol +Windows File Sharing +
+
+ +
+Use setting + +
+
+ +
+
+
+cloud_queue +
+
+NFS Protocol +Unix/Linux File Sharing +
+
+ +
+Use setting + +
+
+
+ +
+
+ + + +
+
+
+ +
+
+ +
+
+

+settings_ethernet + NFS Configuration +

+Active +
+
+
+
+ +
+ + +
+

CIDR notation supported. Use comma for multiple entries.

+
+
+ +
+ +expand_more +
+
+
+
+
+ +
+ + +
+
+
+ +
+ +expand_more +
+
+
+
+
+ +
+ + +
+ + + +
+
+ +
+
+

Top Active Clients

+View all clients +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
IP AddressUserProtocolThroughputAction
192.168.10.105esxi-host-01NFS420 MB/s + +
192.168.10.106esxi-host-02NFS105 MB/s + +
+
+
+
+
+
+
+
+
+ \ No newline at end of file diff --git a/src/web/shares-management.png b/src/web/shares-management.png new file mode 100644 index 0000000..2672592 Binary files /dev/null and b/src/web/shares-management.png differ diff --git a/src/web/snapshots-management-code.html b/src/web/snapshots-management-code.html new file mode 100644 index 0000000..99f96fa --- /dev/null +++ b/src/web/snapshots-management-code.html @@ -0,0 +1,454 @@ + + + + + +Data Protection - Snapshot Manager + + + + + + + + + + + + + + + + + +
+ + + +
+ +
+
+ +hard_drive Storage + +/ +ZFS Pools +/ +tank/data +
+
+ +
+
+

Snapshot Manager

+

Manage automated snapshots, retention policies, and replication tasks for the tank/data dataset.

+
+
+ + +
+
+ +
+
+ +
+
+

Total Snapshots

+photo_library +
+

1,245

+
+trending_up ++12 today +
+
+ +
+
+

Space Used

+pie_chart +
+

2.4 TB

+
+
+
+
+ +
+
+

Reclaimable

+recycling +
+

420 GB

+

from expired policies

+
+ +
+
+

Next Scheduled

+schedule +
+

14:00

+

Today (Auto-Daily)

+
+
+
+ +
+ + + +
+ +
+
+search + +
+
+ + +
+
+ +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +Snapshot NameCreatedUsedReferencedPolicyActions
+ + +
+camera_alt +auto-2023-10-27-1400 +
+		Oct 27, 2023 14:001.2 GB14.5 TB + + Hourly-Keep-24 + + + +
+ + +
+camera_alt +auto-2023-10-27-1300 +
+		Oct 27, 2023 13:00850 MB14.5 TB + + Hourly-Keep-24 + + + +
+ + +
+person +pre-upgrade-backup +
+		Oct 27, 2023 12:454.1 GB14.5 TB + + Manual + + +
+ + + +
+
+ + +
+camera_alt +auto-2023-10-27-1200 +
+		Oct 27, 2023 12:00920 MB14.5 TB + + Hourly-Keep-24 + + + +
+ + +
+lock +daily-2023-10-26 +
+		Oct 26, 2023 00:0012.5 GB14.4 TB + + Daily-Keep-30 + + + +
+
+ +
+

Showing 1-5 of 1,245 snapshots

+
+ + +
+
+
+ +
+
+
+sync +
+
+

Replication in progress

+

Sending 'auto-2023-10-27-1400' to backup-server (45%)

+
+
+
+
+ +
+
+
+
+ \ No newline at end of file diff --git a/src/web/snapshots-management.png b/src/web/snapshots-management.png new file mode 100644 index 0000000..6b84046 Binary files /dev/null and b/src/web/snapshots-management.png differ diff --git a/src/web/storage-management-code.html b/src/web/storage-management-code.html new file mode 100644 index 0000000..8d13ef7 --- /dev/null +++ b/src/web/storage-management-code.html @@ -0,0 +1,569 @@ + + + + + +Storage Management - Cockpit + + + + + + + + + + + + + +
+ + + +
+ +
+
+ + +
+
+

Storage Pools

+

Manage ZFS pools, datasets, and physical disks topology.

+
+
+ + +
+
+
+
+ +
+
+ +
+ +
+
+

Total Capacity

+database +
+
+

120.5 TB

+Raw +
+
+
+
+

38% Used (45 TB)

+
+ +
+
+

Health Status

+check_circle +
+
+

Optimal

+
+

All pools are online

+

Last scrub: 2 hours ago

+
+ +
+
+

Efficiency

+compress +
+
+

1.45x

+Ratio +
+
+LZ4 +DEDUP ON +
+
+ +
+
+

ARC Hit Ratio

+memory +
+
+

98.2%

+
+

Cache Usage: 58GB / 64GB

+
+
+
+
+
+
+
+
+ +
+
+

Active Pools

+
+
+search + +
+
+
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameStatusUsed / AvailTopologyCompressionActions
+arrow_right + +
+dns +
+tank0 +/mnt/tank0 +
+
+		 + + + ONLINE + + +
+
+18.5 TB +42 TB +
+
+
+
+
+		RAIDZ2 (6 disks)LZ4 + +
+
+folder +projects_vol +
+		Mounted +
+
+12 TB +
+
+		—ZSTD-3 + +
+
+folder_shared +backups_smb +
+		Shared (SMB) +
+
+6.5 TB +
+
+		—GZIP-9 + +
+arrow_right + +
+warning +
+archive_legacy +/mnt/archive +
+
+		 + + + DEGRADED + + +
+
+8.2 TB +10 TB +
+
+
+
+
+		MIRROR (2 disks)LZ4 + +
+
+ +
+Showing 2 active pools +View Archived Pools +
+
+
+ +
+
+

Physical Disks

+ +
+
+ +
+
+
+
+
+
+hard_drive +
+
+sda +Slot 1 +
+
+
+
+HGST-7200 +4 TB +
+
+
+
+
+Pool: tank0 +
+
+ +
+
+
+
+
+
+hard_drive +
+
+sdb +Slot 2 +
+
+
+
+HGST-7200 +4 TB +
+
+
+
+
+Pool: tank0 +
+
+ +
+
+
+
+
+
+hard_drive +
+
+sdc +Slot 3 +
+
+
+
+HGST-7200 +4 TB +
+
+
+
+
+Pool: tank0 +
+
+ +
+
+
+
+
+
+error +
+
+sdd +FAULTED +
+
+
+
+HGST-7200 +4 TB +
+
+ +
+
+Pool: archive_legacy +
+
+ +
+
+
+
+
+
+medical_services +
+
+sde +Slot 5 +
+
+
+
+Samsung SSD +4 TB +
+
+
+
+HOT SPARE +
+
+ +
+add +Add New Disk +
+
+
+
+
+
+ +
+ +
+

tank0 Properties

+ +
+ +
+ +
+
+info +Healthy +
+

+ This pool is operating normally. Scrub completed successfully on Oct 24, 2023. +

+
+
+ +
+ + +

Balances performance and storage efficiency.

+
+ +
+
+ + + +
+

Requires significant RAM. Use with caution.

+
+ +
+
+ + +
+

Automatically grow pool when larger disks are added.

+
+ +
+ +
+ +days +
+
+
+
+ +
+ + +
+
+ +
+
+
+ \ No newline at end of file diff --git a/src/web/storage-management.png b/src/web/storage-management.png new file mode 100644 index 0000000..a1b737f Binary files /dev/null and b/src/web/storage-management.png differ diff --git a/src/web/system-management-code.html b/src/web/system-management-code.html new file mode 100644 index 0000000..a3daaca --- /dev/null +++ b/src/web/system-management-code.html @@ -0,0 +1,459 @@ + + + + + +System Settings - Backup Appliance + + + + + + + + + + + + + + +
+ + + +
+ +
+
+ + +
+System +/ +Configuration +
+
+
+ +
+ + +
+
+ +
+
+ +
+
+

System Configuration

+

+ Manage network interfaces, time synchronization, service states, and remote management protocols. +

+
+ +
+ +
+ +
+
+
+lan +

Network Interfaces

+
+ +
+
+ +
+
+
+settings_ethernet +
+
+
+

eth0

+Management +
+

192.168.1.50 / 24

+
+
+
+ + +
+
+ +
+
+
+settings_ethernet +
+
+
+

eth1

+iSCSI +
+

10.0.50.10 / 24

+
+
+
+ + +
+
+ +
+
+
+settings_ethernet +
+
+
+

eth2

+
+

No Carrier

+
+
+
+ + +
+
+
+
+ +
+
+
+memory +

Service Control

+
+
+ +All Systems Normal +
+
+
+ +
+
+
+terminal +
+
+

SSH Service

+

Remote command line access

+
+
+
+RUNNING +
+ + +
+
+
+ +
+
+
+folder_shared +
+
+

SMB / CIFS

+

Windows file sharing

+
+
+
+RUNNING +
+ + +
+
+
+ +
+
+
+storage +
+
+

iSCSI Target

+

Block storage sharing

+
+
+
+STOPPED +
+ + +
+
+
+ +
+
+
+share +
+
+

NFS Service

+

Unix file sharing

+
+
+
+RUNNING +
+ + +
+
+
+
+
+ +
+
+
+schedule +

Date & Time

+
+UTC +
+
+
+
+ +
+ +
+expand_more +
+
+
+
+
+
+ + +
+
+
+
+
+pool.ntp.org +
+Stratum 2 • 12ms +
+
+
+
+time.google.com +
+Stratum 1 • 45ms +
+
+
+
+
+ +
+
+
+hub +

Management

+
+
+
+
+
+
+

SNMP Monitoring

+

Enable Simple Network Management Protocol

+
+
+ + +
+
+
+
+ + +
+
+ + +
+
+
+
+

Syslog Forwarding

+
+ + +
+
+
+
+
+ +
+
+
+
+
+ + \ No newline at end of file diff --git a/src/web/system-management.png b/src/web/system-management.png new file mode 100644 index 0000000..2a4f52f Binary files /dev/null and b/src/web/system-management.png differ diff --git a/src/web/system-monitoring-code.html b/src/web/system-monitoring-code.html new file mode 100644 index 0000000..50db20c --- /dev/null +++ b/src/web/system-monitoring-code.html @@ -0,0 +1,453 @@ + + + + + +Backup Appliance - Monitoring & Logs + + + + + + + + + + + + + +
+ + + +
+ +
+
+
+

System Monitor

+

Real-time telemetry, ZFS health, and system event logs

+
+
+
+ + + + +System Healthy +
+ +
+
+
+ +
+
+ +
+ +
+
+

CPU Load

+memory +
+
+

12%

+ +trending_down 2% + +
+ +
+
+
+
+ +
+
+

RAM Usage

+memory_alt +
+
+

8.4 GB

+/ 32 GB +
+
+
+
+
+ +
+
+

ZFS Pool Status

+check_circle +
+
+

Online

+No Errors +
+
+
+
+
+
+
+
+ +
+
+

System Uptime

+schedule +
+
+

14d 2h 12m

+
+

Last reboot: Manual Patching

+
+
+ +
+ +
+ +
+
+
+

Network Throughput

+

Inbound vs Outbound (eth0)

+
+
+

1.2 Gbps

+

Peak: 2.1 Gbps

+
+
+
+ + + + + + + + + + + + + + + +
+10:00 +10:15 +10:30 +10:45 +11:00 +
+
+
+ +
+
+
+

ZFS ARC Hit Ratio

+

Cache efficiency

+
+
+

94%

+

Target: >90%

+
+
+
+ + + + +
+
95%
+
+
+
+ +
+
+
+

Disk Health

+Pool 1 +
+
+ +
+hard_drive +da0 + +
+
+hard_drive +da1 +
+
+hard_drive +da2 +
+
+hard_drive +da3 +
+
+hard_drive +da4 +
+ +
+ +warning +da5 +
+ +
+hard_drive +da6 +
+
+hard_drive +da7 +
+ +
+Empty +
+
+Empty +
+
+Empty +
+
+Empty +
+
+
+
+Total Capacity +64 TB +
+
+
+
+
+Used: 41.6 TB +Free: 22.4 TB +
+
+
+
+
+ +
+ +
+ + + +
+
+search + +
+
+
+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Job NameTypeProgressSpeedStatus
Daily Backup: VM-Cluster-01Replication +
+
+
+
+
+
+45% +
+

ETA: 1h 12m

+		145 MB/s + + Running + +
ZFS Scrub: Pool-01Maintenance +
+
+
+
+78% +
+		1.2 GB/s + + Running + +
+
+ +
+

Recent System Events

+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
10:45:22 +INFO +systemdStarted User Manager for UID 1000.
10:45:15 +WARN +smartdDevice: /dev/ada5, SMART Usage Attribute: 194 Temperature_Celsius changed from 38 to 41
10:44:58 +INFO +kernelix0: link state changed to UP
10:42:10 +INFO +zfszfs_arc_reclaim_thread: reclaiming 157286400 bytes ...
+
+
+
+
+
+
+
+ \ No newline at end of file diff --git a/src/web/system-monitoring.png b/src/web/system-monitoring.png new file mode 100644 index 0000000..0a2da8a Binary files /dev/null and b/src/web/system-monitoring.png differ diff --git a/src/web/user-access-management-code.html b/src/web/user-access-management-code.html new file mode 100644 index 0000000..79a794a --- /dev/null +++ b/src/web/user-access-management-code.html @@ -0,0 +1,400 @@ + + + + + +User & Access Management - Backup Appliance + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+
+ +

User & Access Management

+

Manage local accounts, define RBAC roles, and configure directory services (LDAP/AD) integration.

+
+
+ +
+
+ +
+ +
+ + + +
+ +
+ +
+
+search + +
+ +
+ + +
+ +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
StatusUsernameFull NameRoleGroupsLast LoginActions
+
+ + Active +
+		 +
+
AD
+admin +
+		System Administrator + +verified_user + Superuser + +wheel, systemJust now + +
+
+ + Active +
+		 +
+
JD
+jdoe +
+		John Doe + +engineering + Operator + +operators, backup2 days ago + +
+
+lock + Locked +
+		 +
+
BS
+backup_svc +
+		Backup Service Account + +smart_toy + Service + +servicesNever + +
+
+ + Active +
+		 +
+
AS
+asmith +
+		Alice Smith + +visibility + Auditor + +auditors1 week ago + +
+
+ + Active +
+		 +
+
RR
+rroe +
+		Richard Roe + +storage + Storage Admin + +storage_admins5 hours ago + +
+
+ +
+Showing 1-5 of 24 users +
+ + +
+
+
+
+ +
+ +
+
+
+
+domain +
+

Directory Service

+
+Inactive +
+

No LDAP or Active Directory server is currently connected. Local authentication is being used.

+
+ + Configure Directory + arrow_forward + +
+
+ +
+
+
+
+shield +
+

Security Policy

+
+Good +
+
+
+Multi-Factor Auth +Enforced +
+
+Password Rotation +90 Days +
+
+
+ + Manage Policies + arrow_forward + +
+
+
+
+
+
+ \ No newline at end of file diff --git a/src/web/user-access-management.png b/src/web/user-access-management.png new file mode 100644 index 0000000..d376d61 Binary files /dev/null and b/src/web/user-access-management.png differ diff --git a/src/web/vtl-management-code.html b/src/web/vtl-management-code.html new file mode 100644 index 0000000..6544427 --- /dev/null +++ b/src/web/vtl-management-code.html @@ -0,0 +1,516 @@ + + + + + +Virtual Tape Library Management + + + + + + + + + + + + + + +
+ + + +
+ +
+
+ +
+Home +/ +Storage +/ +Virtual Tape Libraries +
+ +
+
+

Virtual Tape Libraries

+

Manage virtual tape devices, emulation profiles, and storage targets.

+
+
+ + +
+
+
+
+ +
+
+ +
+ +
+
+dns +
+

Total Libraries

+
+

3

+ +check_circle All Online + +
+
+ +
+
+database +
+

Total Capacity

+
+

120 TB

+
+
+ +
+
+album +
+

Tapes Online

+
+

45

+/ 200 Slots +
+
+ +
+
+swap_horiz +
+

Active Sessions

+
+

2

+Writing +
+
+
+ +
+
+
+pie_chart +

VTL Partition Usage (ZFS Pool)

+
+

14.5 TB / 50 TB Used

+
+
+
+
+
+Compression Ratio: 1.5x + Pool Healthy +
+
+ +
+ +
+
+search + +
+
+ + +
+
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +Library NameStatusEmulationTapes / SlotsiSCSI TargetActions
+ + +
+
+shelves +
+
+

VTL_Arch_01

+

ID: 001

+
+
+		 + + + Ready + + +

HP MSL2024

+

LTO-7 • 2 Drives

+		 +
+album +24 / 24 +
+
+
+
+		 +
+iqn.2023-10.com.vtl:arch01 +content_copy +
+		 +
+ + + +
+
+ + +
+
+shelves +
+
+

VTL_Prod_Daily

+

ID: 002

+
+
+		 + +refresh + Writing + + +

Quantum i3

+

LTO-8 • 4 Drives

+		 +
+album +12 / 50 +
+
+
+
+		 +
+iqn.2023-10.com.vtl:prod02 +content_copy +
+		 +
+ + + +
+
+ + +
+
+shelves +
+
+

VTL_Legacy_03

+

ID: 003

+
+
+		 + + + Offline + + +

IBM TS3100

+

LTO-5 • 1 Drive

+		 +
+album +9 / 10 +
+
+
+
+		 +
+iqn.2023-10.com.vtl:leg03 +content_copy +
+		 +
+ + + +
+
+
+ +
+

Showing 1-3 of 3 libraries

+
+ + +
+
+
+
+
+ + +
+
+
+
+cable +
+

Tape Management: VTL_Arch_01

+

Manage virtual cartridges, import/export slots, and barcodes.

+
+
+
+ + + +
+
+
+ +
+
+album +Slot 1 +
+
+

ARC001L7

+

12.0 TB / 12.0 TB

+
+ +
+ +
+
+album +Slot 2 +
+
+

ARC002L7

+

Empty / 12.0 TB

+
+ +
+ +
+add_circle +Empty Slot 3 +
+ +
+add_circle +Empty Slot 4 +
+
+
+
+
+
+ \ No newline at end of file diff --git a/src/web/vtl-management.png b/src/web/vtl-management.png new file mode 100644 index 0000000..77734d3 Binary files /dev/null and b/src/web/vtl-management.png differ