386 lines
9.0 KiB
Markdown
386 lines
9.0 KiB
Markdown
# SDS-02: Database Design
|
|
|
|
## 1. Database Overview
|
|
|
|
### 1.1 Database System
|
|
- **Type**: PostgreSQL 14+
|
|
- **Encoding**: UTF-8
|
|
- **Connection Pooling**: pgxpool
|
|
- **Migrations**: Custom migration system
|
|
|
|
### 1.2 Database Schema Organization
|
|
- **Tables**: Organized by domain (users, storage, shares, etc.)
|
|
- **Indexes**: Performance indexes on foreign keys and frequently queried columns
|
|
- **Constraints**: Foreign keys, unique constraints, check constraints
|
|
|
|
## 2. Core Tables
|
|
|
|
### 2.1 Users & Authentication
|
|
```sql
|
|
users (
|
|
id UUID PRIMARY KEY,
|
|
username VARCHAR(255) UNIQUE NOT NULL,
|
|
email VARCHAR(255) UNIQUE,
|
|
password_hash VARCHAR(255) NOT NULL,
|
|
is_active BOOLEAN DEFAULT true,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP
|
|
)
|
|
|
|
roles (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
description TEXT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP
|
|
)
|
|
|
|
permissions (
|
|
id UUID PRIMARY KEY,
|
|
resource VARCHAR(255) NOT NULL,
|
|
action VARCHAR(255) NOT NULL,
|
|
description TEXT,
|
|
UNIQUE(resource, action)
|
|
)
|
|
|
|
user_roles (
|
|
user_id UUID REFERENCES users(id),
|
|
role_id UUID REFERENCES roles(id),
|
|
PRIMARY KEY (user_id, role_id)
|
|
)
|
|
|
|
role_permissions (
|
|
role_id UUID REFERENCES roles(id),
|
|
permission_id UUID REFERENCES permissions(id),
|
|
PRIMARY KEY (role_id, permission_id)
|
|
)
|
|
|
|
groups (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
description TEXT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP
|
|
)
|
|
|
|
user_groups (
|
|
user_id UUID REFERENCES users(id),
|
|
group_id UUID REFERENCES groups(id),
|
|
PRIMARY KEY (user_id, group_id)
|
|
)
|
|
```
|
|
|
|
### 2.2 Storage Tables
|
|
```sql
|
|
zfs_pools (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
description TEXT,
|
|
raid_level VARCHAR(50),
|
|
status VARCHAR(50),
|
|
total_capacity_bytes BIGINT,
|
|
used_capacity_bytes BIGINT,
|
|
health_status VARCHAR(50),
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
created_by UUID REFERENCES users(id)
|
|
)
|
|
|
|
zfs_datasets (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) NOT NULL,
|
|
pool_name VARCHAR(255) REFERENCES zfs_pools(name),
|
|
type VARCHAR(50),
|
|
mount_point VARCHAR(255),
|
|
used_bytes BIGINT,
|
|
available_bytes BIGINT,
|
|
compression VARCHAR(50),
|
|
quota BIGINT,
|
|
reservation BIGINT,
|
|
created_at TIMESTAMP,
|
|
UNIQUE(pool_name, name)
|
|
)
|
|
|
|
physical_disks (
|
|
id UUID PRIMARY KEY,
|
|
device_path VARCHAR(255) UNIQUE NOT NULL,
|
|
vendor VARCHAR(255),
|
|
model VARCHAR(255),
|
|
serial_number VARCHAR(255),
|
|
size_bytes BIGINT,
|
|
type VARCHAR(50),
|
|
status VARCHAR(50),
|
|
last_synced_at TIMESTAMP
|
|
)
|
|
|
|
storage_repositories (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
type VARCHAR(50),
|
|
path VARCHAR(255),
|
|
capacity_bytes BIGINT,
|
|
used_bytes BIGINT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP
|
|
)
|
|
```
|
|
|
|
### 2.3 Shares Tables
|
|
```sql
|
|
shares (
|
|
id UUID PRIMARY KEY,
|
|
dataset_id UUID REFERENCES zfs_datasets(id),
|
|
share_type VARCHAR(50),
|
|
nfs_enabled BOOLEAN DEFAULT false,
|
|
nfs_options TEXT,
|
|
nfs_clients TEXT[],
|
|
smb_enabled BOOLEAN DEFAULT false,
|
|
smb_share_name VARCHAR(255),
|
|
smb_path VARCHAR(255),
|
|
smb_comment TEXT,
|
|
smb_guest_ok BOOLEAN DEFAULT false,
|
|
smb_read_only BOOLEAN DEFAULT false,
|
|
smb_browseable BOOLEAN DEFAULT true,
|
|
is_active BOOLEAN DEFAULT true,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
created_by UUID REFERENCES users(id)
|
|
)
|
|
```
|
|
|
|
### 2.4 iSCSI Tables
|
|
```sql
|
|
iscsi_targets (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
alias VARCHAR(255),
|
|
enabled BOOLEAN DEFAULT true,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
created_by UUID REFERENCES users(id)
|
|
)
|
|
|
|
iscsi_luns (
|
|
id UUID PRIMARY KEY,
|
|
target_id UUID REFERENCES iscsi_targets(id),
|
|
lun_number INTEGER,
|
|
device_path VARCHAR(255),
|
|
size_bytes BIGINT,
|
|
created_at TIMESTAMP
|
|
)
|
|
|
|
iscsi_initiators (
|
|
id UUID PRIMARY KEY,
|
|
iqn VARCHAR(255) UNIQUE NOT NULL,
|
|
created_at TIMESTAMP
|
|
)
|
|
|
|
target_initiators (
|
|
target_id UUID REFERENCES iscsi_targets(id),
|
|
initiator_id UUID REFERENCES iscsi_initiators(id),
|
|
PRIMARY KEY (target_id, initiator_id)
|
|
)
|
|
```
|
|
|
|
### 2.5 Tape Tables
|
|
```sql
|
|
vtl_libraries (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
vendor VARCHAR(255),
|
|
model VARCHAR(255),
|
|
drive_count INTEGER,
|
|
slot_count INTEGER,
|
|
status VARCHAR(50),
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
created_by UUID REFERENCES users(id)
|
|
)
|
|
|
|
physical_libraries (
|
|
id UUID PRIMARY KEY,
|
|
vendor VARCHAR(255),
|
|
model VARCHAR(255),
|
|
serial_number VARCHAR(255),
|
|
drive_count INTEGER,
|
|
slot_count INTEGER,
|
|
discovered_at TIMESTAMP
|
|
)
|
|
```
|
|
|
|
### 2.6 Backup Tables
|
|
```sql
|
|
backup_jobs (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
client_id INTEGER,
|
|
fileset_id INTEGER,
|
|
schedule VARCHAR(255),
|
|
storage_pool_id INTEGER,
|
|
enabled BOOLEAN DEFAULT true,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
created_by UUID REFERENCES users(id)
|
|
)
|
|
```
|
|
|
|
### 2.7 Monitoring Tables
|
|
```sql
|
|
alerts (
|
|
id UUID PRIMARY KEY,
|
|
rule_id UUID,
|
|
severity VARCHAR(50),
|
|
source VARCHAR(255),
|
|
message TEXT,
|
|
status VARCHAR(50),
|
|
acknowledged_at TIMESTAMP,
|
|
resolved_at TIMESTAMP,
|
|
created_at TIMESTAMP
|
|
)
|
|
|
|
alert_rules (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(255) UNIQUE NOT NULL,
|
|
description TEXT,
|
|
source VARCHAR(255),
|
|
condition_type VARCHAR(255),
|
|
condition_config JSONB,
|
|
severity VARCHAR(50),
|
|
enabled BOOLEAN DEFAULT true,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP
|
|
)
|
|
```
|
|
|
|
### 2.8 Audit Tables
|
|
```sql
|
|
audit_logs (
|
|
id UUID PRIMARY KEY,
|
|
user_id UUID REFERENCES users(id),
|
|
action VARCHAR(255),
|
|
resource_type VARCHAR(255),
|
|
resource_id VARCHAR(255),
|
|
method VARCHAR(10),
|
|
path VARCHAR(255),
|
|
ip_address VARCHAR(45),
|
|
user_agent TEXT,
|
|
request_body JSONB,
|
|
response_status INTEGER,
|
|
created_at TIMESTAMP
|
|
)
|
|
```
|
|
|
|
### 2.9 Task Tables
|
|
```sql
|
|
tasks (
|
|
id UUID PRIMARY KEY,
|
|
type VARCHAR(255),
|
|
status VARCHAR(50),
|
|
progress INTEGER,
|
|
result JSONB,
|
|
error_message TEXT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
completed_at TIMESTAMP,
|
|
created_by UUID REFERENCES users(id)
|
|
)
|
|
```
|
|
|
|
## 3. Indexes
|
|
|
|
### 3.1 Performance Indexes
|
|
```sql
|
|
-- Users
|
|
CREATE INDEX idx_users_username ON users(username);
|
|
CREATE INDEX idx_users_email ON users(email);
|
|
|
|
-- Storage
|
|
CREATE INDEX idx_zfs_pools_name ON zfs_pools(name);
|
|
CREATE INDEX idx_zfs_datasets_pool_name ON zfs_datasets(pool_name);
|
|
|
|
-- Shares
|
|
CREATE INDEX idx_shares_dataset_id ON shares(dataset_id);
|
|
CREATE INDEX idx_shares_created_by ON shares(created_by);
|
|
|
|
-- iSCSI
|
|
CREATE INDEX idx_iscsi_targets_name ON iscsi_targets(name);
|
|
CREATE INDEX idx_iscsi_luns_target_id ON iscsi_luns(target_id);
|
|
|
|
-- Monitoring
|
|
CREATE INDEX idx_alerts_status ON alerts(status);
|
|
CREATE INDEX idx_alerts_created_at ON alerts(created_at);
|
|
CREATE INDEX idx_audit_logs_user_id ON audit_logs(user_id);
|
|
CREATE INDEX idx_audit_logs_created_at ON audit_logs(created_at);
|
|
```
|
|
|
|
## 4. Migrations
|
|
|
|
### 4.1 Migration System
|
|
- **Location**: `db/migrations/`
|
|
- **Naming**: `NNN_description.sql`
|
|
- **Execution**: Sequential execution on startup
|
|
- **Version Tracking**: `schema_migrations` table
|
|
|
|
### 4.2 Migration Files
|
|
- `001_initial_schema.sql`: Core tables
|
|
- `002_storage_and_tape_schema.sql`: Storage and tape tables
|
|
- `003_performance_indexes.sql`: Performance indexes
|
|
- `004_add_zfs_pools_table.sql`: ZFS pools
|
|
- `005_add_zfs_datasets_table.sql`: ZFS datasets
|
|
- `006_add_zfs_shares_and_iscsi.sql`: Shares and iSCSI
|
|
- `007_add_vendor_to_vtl_libraries.sql`: VTL updates
|
|
- `008_add_user_groups.sql`: User groups
|
|
- `009_backup_jobs_schema.sql`: Backup jobs
|
|
- `010_add_backup_permissions.sql`: Backup permissions
|
|
- `011_sync_bacula_jobs_function.sql`: Bacula sync function
|
|
|
|
## 5. Data Relationships
|
|
|
|
### 5.1 Entity Relationships
|
|
- **Users** → **Roles** (many-to-many)
|
|
- **Roles** → **Permissions** (many-to-many)
|
|
- **Users** → **Groups** (many-to-many)
|
|
- **ZFS Pools** → **ZFS Datasets** (one-to-many)
|
|
- **ZFS Datasets** → **Shares** (one-to-many)
|
|
- **iSCSI Targets** → **LUNs** (one-to-many)
|
|
- **iSCSI Targets** → **Initiators** (many-to-many)
|
|
|
|
## 6. Data Integrity
|
|
|
|
### 6.1 Constraints
|
|
- **Primary Keys**: UUID primary keys for all tables
|
|
- **Foreign Keys**: Referential integrity
|
|
- **Unique Constraints**: Unique usernames, emails, names
|
|
- **Check Constraints**: Valid status values, positive numbers
|
|
|
|
### 6.2 Cascading Rules
|
|
- **ON DELETE CASCADE**: Child records deleted with parent
|
|
- **ON DELETE RESTRICT**: Prevent deletion if referenced
|
|
- **ON UPDATE CASCADE**: Update foreign keys on parent update
|
|
|
|
## 7. Query Optimization
|
|
|
|
### 7.1 Query Patterns
|
|
- **Eager Loading**: Join related data when needed
|
|
- **Pagination**: Limit and offset for large datasets
|
|
- **Filtering**: WHERE clauses for filtering
|
|
- **Sorting**: ORDER BY for sorted results
|
|
|
|
### 7.2 Caching Strategy
|
|
- **Query Result Caching**: Cache frequently accessed queries
|
|
- **Cache Invalidation**: Invalidate on write operations
|
|
- **TTL**: Time-to-live for cached data
|
|
|
|
## 8. Backup & Recovery
|
|
|
|
### 8.1 Backup Strategy
|
|
- **Regular Backups**: Daily database backups
|
|
- **Point-in-Time Recovery**: WAL archiving
|
|
- **Backup Retention**: 30 days retention
|
|
|
|
### 8.2 Recovery Procedures
|
|
- **Full Restore**: Restore from backup
|
|
- **Point-in-Time**: Restore to specific timestamp
|
|
- **Selective Restore**: Restore specific tables
|
|
|