atlas/docs/ZFS_OPERATIONS.md

ZFS Operations

Overview

AtlasOS provides comprehensive ZFS pool management including pool creation, import, export, scrubbing with progress monitoring, and health status reporting.

Pool Operations

List Pools

GET /api/v1/pools

Returns all ZFS pools.

Response:

[
  {
    "name": "tank",
    "status": "ONLINE",
    "size": 1099511627776,
    "allocated": 536870912000,
    "free": 562641027776,
    "health": "ONLINE",
    "created_at": "2024-01-15T10:30:00Z"
  }
]

Get Pool

GET /api/v1/pools/{name}

Returns details for a specific pool.

Create Pool

POST /api/v1/pools

Creates a new ZFS pool.

Request Body:

{
  "name": "tank",
  "vdevs": ["sda", "sdb"],
  "options": {
    "ashift": "12"
  }
}

Destroy Pool

DELETE /api/v1/pools/{name}

Destroys a ZFS pool. Warning: This is a destructive operation.

Pool Import/Export

List Available Pools

GET /api/v1/pools/available

Lists pools that can be imported (pools that exist but are not currently imported).

Response:

{
  "pools": ["tank", "backup"]
}

Import Pool

POST /api/v1/pools/import

Imports a ZFS pool.

Request Body:

{
  "name": "tank",
  "options": {
    "readonly": "on"
  }
}

Options:

• readonly: Set pool to read-only mode (on/off)
• Other ZFS pool properties

Response:

{
  "message": "pool imported",
  "name": "tank"
}

Export Pool

POST /api/v1/pools/{name}/export

Exports a ZFS pool (makes it unavailable but preserves data).

Request Body (optional):

{
  "force": false
}

Parameters:

• force (boolean): Force export even if pool is in use

Response:

{
  "message": "pool exported",
  "name": "tank"
}

Scrub Operations

Start Scrub

POST /api/v1/pools/{name}/scrub

Starts a scrub operation on a pool. Scrub verifies data integrity and repairs any errors found.

Response:

{
  "message": "scrub started",
  "pool": "tank"
}

Get Scrub Status

GET /api/v1/pools/{name}/scrub

Returns detailed scrub status with progress information.

Response:

{
  "status": "in_progress",
  "progress": 45.2,
  "time_elapsed": "2h15m",
  "time_remain": "30m",
  "speed": "100M/s",
  "errors": 0,
  "repaired": 0,
  "last_scrub": "2024-12-15T10:30:00Z"
}

Status Values:

• idle: No scrub in progress
• in_progress: Scrub is currently running
• completed: Scrub completed successfully
• error: Scrub encountered errors

Progress Fields:

• progress: Percentage complete (0-100)
• time_elapsed: Time since scrub started
• time_remain: Estimated time remaining
• speed: Current scrub speed
• errors: Number of errors found
• repaired: Number of errors repaired
• last_scrub: Timestamp of last completed scrub

Usage Examples

Import a Pool

curl -X POST http://localhost:8080/api/v1/pools/import \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "tank"
  }'

Start Scrub and Monitor Progress

# Start scrub
curl -X POST http://localhost:8080/api/v1/pools/tank/scrub \
  -H "Authorization: Bearer $TOKEN"

# Check progress
curl http://localhost:8080/api/v1/pools/tank/scrub \
  -H "Authorization: Bearer $TOKEN"

Export Pool

curl -X POST http://localhost:8080/api/v1/pools/tank/export \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "force": false
  }'

Scrub Best Practices

When to Scrub

• Regular Schedule: Monthly or quarterly
• After Disk Failures: After replacing failed disks
• Before Major Operations: Before pool upgrades or migrations
• After Data Corruption: If data integrity issues are suspected

Monitoring Scrub Progress

1. Start Scrub: Use POST endpoint to start
2. Monitor Progress: Poll GET endpoint every few minutes
3. Check Errors: Monitor errors and repaired fields
4. Wait for Completion: Wait until status is completed

Scrub Performance

• Impact: Scrub operations can impact pool performance
• Scheduling: Schedule during low-usage periods
• Duration: Large pools may take hours or days
• I/O: Scrub generates significant I/O load

Pool Import/Export Use Cases

Import Use Cases

1. System Reboot: Pools are automatically imported on boot
2. Manual Import: Import pools that were exported
3. Read-Only Import: Import pool in read-only mode for inspection
4. Recovery: Import pools from backup systems

Export Use Cases

1. System Shutdown: Export pools before shutdown
2. Maintenance: Export pools for maintenance operations
3. Migration: Export pools before moving to another system
4. Backup: Export pools before creating full backups

Error Handling

Pool Not Found

{
  "code": "NOT_FOUND",
  "message": "pool not found"
}

Scrub Already Running

{
  "code": "CONFLICT",
  "message": "scrub already in progress"
}

Pool in Use (Export)

{
  "code": "CONFLICT",
  "message": "pool is in use, cannot export"
}

Use force: true to force export (use with caution).

Compliance with SRS

Per SRS section 4.2 ZFS Management:

• ✅ List available disks: Implemented
• ✅ Create pools: Implemented
• ✅ Import pools: Implemented (Priority 20)
• ✅ Export pools: Implemented (Priority 20)
• ✅ Report pool health: Implemented
• ✅ Create and manage datasets: Implemented
• ✅ Create ZVOLs: Implemented
• ✅ Scrub operations: Implemented
• ✅ Progress monitoring: Implemented (Priority 19)

Future Enhancements

1. Scrub Scheduling: Automatic scheduled scrubs
2. Scrub Notifications: Alerts when scrub completes or finds errors
3. Pool Health Alerts: Automatic alerts for pool health issues
4. Import History: Track pool import/export history
5. Pool Properties: Manage pool properties via API
6. VDEV Management: Add/remove vdevs from pools
7. Pool Upgrade: Upgrade pool version
8. Resilver Operations: Monitor and manage resilver operations