ZFS Installation and API Integration - Complete

Summary

Successfully installed ZFS on Ubuntu 24.04 and integrated ZFS pool management into the Calypso API. All CRUD operations for ZFS pools are working correctly.

Installation Details

ZFS Version: 2.2.2-0ubuntu9.4
Kernel: 6.8.0-90-generic
Services: All ZFS services active (zfs.target, zfs-zed, zfs-mount, zfs-import-cache, zfs-share, zfs-import-scan)

API Endpoints Tested

All 4 ZFS pool management endpoints are 100% functional:

✅ POST /api/v1/storage/zfs/pools - Create ZFS pool
✅ GET /api/v1/storage/zfs/pools - List all pools
✅ GET /api/v1/storage/zfs/pools/:id - Get pool details
✅ DELETE /api/v1/storage/zfs/pools/:id - Delete pool

Test Results

# Pool Creation Test
POST /api/v1/storage/zfs/pools
Body: {
  "name": "test-pool",
  "raid_level": "mirror",
  "disks": ["/dev/sdb", "/dev/sdc"]
}
Result: ✅ Pool created successfully (ID: 1ba8007f-f749-42d8-97b0-91db2cde20b4)

# List Pools Test
GET /api/v1/storage/zfs/pools
Result: ✅ Returns pool array with all details

# Get Pool Details Test
GET /api/v1/storage/zfs/pools/1ba8007f-f749-42d8-97b0-91db2cde20b4
Result: ✅ Returns complete pool information

# Delete Pool Test
DELETE /api/v1/storage/zfs/pools/1ba8007f-f749-42d8-97b0-91db2cde20b4
Result: ✅ Pool destroyed and removed from database

Technical Issues Resolved

Issue 1: Compression Property Error

Problem: zpool create command used -o compression=lz4 at pool level Error: property 'compression' is not a valid pool or vdev property Solution: Removed compression from zpool create, compression is handled at filesystem level

Issue 2: PostgreSQL Array Type Conversion

Problem: []string disks parameter couldn't be inserted into PostgreSQL TEXT[] column Error: sql: converting argument $3 type: unsupported type []string Solution:

Added github.com/lib/pq import
Wrapped disks with pq.Array(disks) in INSERT statement
Used pq.Array(&pool.Disks) in SELECT scans

Issue 3: NULL Description Field

Problem: description column allows NULL but Go struct uses non-nullable string Error: sql: Scan error on column index 2, name "description": converting NULL to string is unsupported Solution:

Used sql.NullString for scanning description
Check description.Valid before assigning to pool.Description
Applied to both ListPools() and GetPool() functions

Code Changes

File: backend/internal/storage/zfs.go

Added "github.com/lib/pq" import
Modified CreatePool(): Use pq.Array(disks) for INSERT
Modified ListPools(): Use pq.Array(&pool.Disks) and sql.NullString for description
Modified GetPool(): Same fixes as ListPools

Database Schema

Migration: 004_add_zfs_pools_table.sql

CREATE TABLE IF NOT EXISTS zfs_pools (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL UNIQUE,
    description TEXT,  -- Nullable
    raid_level VARCHAR(50) NOT NULL,
    disks TEXT[] NOT NULL,  -- PostgreSQL array
    size_bytes BIGINT NOT NULL,
    used_bytes BIGINT NOT NULL DEFAULT 0,
    compression VARCHAR(50) NOT NULL DEFAULT 'lz4',
    deduplication BOOLEAN NOT NULL DEFAULT false,
    auto_expand BOOLEAN NOT NULL DEFAULT false,
    scrub_interval INTEGER NOT NULL DEFAULT 30,
    is_active BOOLEAN NOT NULL DEFAULT true,
    health_status VARCHAR(50) NOT NULL DEFAULT 'online',
    created_at TIMESTAMP NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
    created_by UUID REFERENCES users(id)
);

Available Disks

sda: 80G (system disk)
sdb-sdf: 32G each (available for ZFS pools)

Next Steps

ZFS filesystem management (datasets)
ZFS snapshot management
ZFS replication
Pool health monitoring and alerts
Scrub scheduling

Success Metrics

API Success Rate: 100% (4/4 endpoints working)
Installation: Complete and verified
Integration: Fully functional with Calypso backend
Database: Migration applied, all queries working
CLI Verification: All operations verified with zpool commands

Date: 2025-12-25 Status: ✅ Complete and Production Ready

4.2 KiB Raw Blame History