alpha-v1.0.0 #3
0
.scannerwork/.sonar_lock
Normal file
0
.scannerwork/.sonar_lock
Normal file
6
.scannerwork/report-task.txt
Normal file
6
.scannerwork/report-task.txt
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
projectKey=calypso
|
||||||
|
serverUrl=https://sonar.avt.data-center.id
|
||||||
|
serverVersion=25.12.0.117093
|
||||||
|
dashboardUrl=https://sonar.avt.data-center.id/dashboard?id=calypso
|
||||||
|
ceTaskId=2bb5340c-a81c-4dee-a5d2-0d498ee5e6c6
|
||||||
|
ceTaskUrl=https://sonar.avt.data-center.id/api/ce/task?id=2bb5340c-a81c-4dee-a5d2-0d498ee5e6c6
|
||||||
137
AGENTS.md
Normal file
137
AGENTS.md
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
# AGENTS
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
- Languages: Go backend with `github.com/gin-gonic/gin`, `zap`, PostgreSQL migrations and REST handlers under `backend/internal`, plus a Vite+React+TypeScript frontend under `frontend/`.
|
||||||
|
- Keep the system deterministic: Go 1.22+, PostgreSQL 14+, Node 18+, npm 10+ (use `node --version`/`npm --version` to confirm).
|
||||||
|
- Secret material lives in `/etc/calypso/config.yaml` or environment variables (never commit private keys, JWT secrets, or database passwords).
|
||||||
|
- The backend expects `CALYPSO_DB_PASSWORD` and `CALYPSO_JWT_SECRET` (minimum 32 characters) before running locally or in CI.
|
||||||
|
|
||||||
|
## Environment Setup
|
||||||
|
- Run `sudo ./scripts/install-requirements.sh` from the repo root to provision system dependencies on Ubuntu-like hosts before building the backend.
|
||||||
|
- `config.yaml.example` provides defaults; copy it to `/etc/calypso/config.yaml`, then edit the file or override with the `CALYPSO_*` environment variables before launching.
|
||||||
|
- For frontend work, `npm install` from `frontend/` once before subsequent builds or `npm run dev` sessions.
|
||||||
|
|
||||||
|
## Backend Tooling (Go)
|
||||||
|
### Build & Run
|
||||||
|
- `make build` (`go build -o bin/calypso-api ./cmd/calypso-api`) produces the local binary.
|
||||||
|
- `make run` (`go run ./cmd/calypso-api -config config.yaml.example`) is the quickest local dev server; supply `-config` pointing at your working config.
|
||||||
|
- Production cross-build: `make build-linux` sets `CGO_ENABLED=0 GOOS=linux GOARCH=amd64` and strips symbols.
|
||||||
|
- `deploy/systemd/calypso-api.service` is a template—copy it to `/etc/systemd/system/`, `daemon-reload`, `enable`, and `start` to run as service.
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
- `make test` runs `go test ./...` across all packages; it is the authoritative test runner today.
|
||||||
|
- To run a single test, narrow the package and pass `-run`: `cd backend && go test ./internal/auth -run '^TestHandler_Login$'` (use `.` to target the current package and regular expressions for test names).
|
||||||
|
- `make test-coverage` generates `coverage.out` and opens a coverage report via `go tool cover -html=coverage.out`.
|
||||||
|
|
||||||
|
### Lint & Format
|
||||||
|
- `make fmt` (`go fmt ./...`) keeps source gofmt-clean; run it after edits and before committing.
|
||||||
|
- `make lint` requires `golangci-lint`; install it with `make install-deps` (which calls `go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest`).
|
||||||
|
- `make deps` handles dependency downloads and tidies `go.mod`/`go.sum` when vendor changes are necessary.
|
||||||
|
|
||||||
|
## Backend Style Guidelines (Go)
|
||||||
|
### Imports
|
||||||
|
- Group imports with standard library packages first, blank line, then third-party packages (gofmt enforces this).
|
||||||
|
- Do not alias imports unless needed to disambiguate. Keep third-party import paths explicit (e.g., `github.com/gin-gonic/gin`).
|
||||||
|
- Prefer explicit package names (`config`, `database`, `logger`) rather than dot-importing.
|
||||||
|
|
||||||
|
### Formatting & Files
|
||||||
|
- Always run `gofmt` (or `make fmt`) before committing; gofmt enforces tabs for indentation and consistent formatting.
|
||||||
|
- Keep files small (100–300 lines) and break responsibilities into `internal/common`, `internal/iam`, etc., mirroring the current layout.
|
||||||
|
- Maintain doc comments on exported functions, types, and struct fields; these comments start with the identifier they describe.
|
||||||
|
|
||||||
|
### Types & Naming
|
||||||
|
- Use PascalCase (UpperCamel) for exported types/fields/methods and camelCase for internal helpers.
|
||||||
|
- Keep configuration structs descriptive (`ServerConfig`, `DatabaseConfig`) and align YAML tags with snake_case field names.
|
||||||
|
- When introducing new structs, mimic the existing tag conventions (`yaml:"field_name"`).
|
||||||
|
- Use short, meaningful receiver names (`func (h *Handler) ...`), ideally 1–3 characters, matching current files.
|
||||||
|
- Name boolean helpers `isX`/`shouldY` and prefer `cfg`/`db`/`ctx` as short local variables for configuration, database, and contexts.
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
- Wrap errors with `fmt.Errorf("context: %w", err)` and return early. Do not swallow errors unless there is a clear action (e.g., logging + fallback).
|
||||||
|
- Use `gin.Context` to respond with the appropriate HTTP status codes (`c.JSON(http.StatusBadRequest, gin.H{...})`).
|
||||||
|
- Log warnings vs. errors via the shared `logger.Logger` (`h.logger.Warn(...)` for expected issues, `Error` for failures, `Info` for happy paths).
|
||||||
|
- Avoid `panic` in handlers; only panic during initialization (e.g., new logger). Instead, log and return a 5xx status.
|
||||||
|
- Keep timeouts and cancellation explicit using `context.WithTimeout` when communicating with databases or external systems.
|
||||||
|
|
||||||
|
### Logging & Observability
|
||||||
|
- Use `zap`-backed `logger.NewLogger`, pass service-specific names (e.g., `router`, `auth-handler`), and attach structured pairs (`"user_id", user.ID`).
|
||||||
|
- Allow log format and level to be configured via `CALYPSO_LOG_FORMAT`/`CALYPSO_LOG_LEVEL` if present.
|
||||||
|
- Prefer adding caller info and stack traces on `Error`level logs when helpful. Do not sprinkle raw `fmt.Println` or `log.Print`.
|
||||||
|
|
||||||
|
### HTTP & Routing
|
||||||
|
- All HTTP routing lives under `internal/common/router`; register handlers through route groups and middleware (auth/authZ) as seen in `router.NewRouter`.
|
||||||
|
- Use Gin middleware for sessions, JWT validation, and auditing. Keep handler methods grouped by feature (auth, iam, tasks).
|
||||||
|
- Keep handler structs simple: inject shared dependencies (`db`, `config`, `logger`) via constructors, then refer to them as fields.
|
||||||
|
- For JSON requests/responses, mirror struct field names in `json` tags (snake_case) even if Go fields are camelCase.
|
||||||
|
|
||||||
|
### Database & Migrations
|
||||||
|
- Use prepared statements or parameterized queries (see `iam` and `sessions` tables) with `$1` style placeholders.
|
||||||
|
- Run migrations automatically from `internal/common/database/migrations/` via `database.RunMigrations` at startup.
|
||||||
|
- Hunters: use `database.DB` wrapper for connection pool handling, close connections via `defer db.Close()`.
|
||||||
|
- When adding new tables, drop SQL files into `db/migrations/` and re-run migrations via the running service or a migration helper.
|
||||||
|
|
||||||
|
### Security Notes
|
||||||
|
- JWT secrets must be strong (>=32 characters). Use environment variables, never embed secrets in code or checked-in configs.
|
||||||
|
- Database passwords, API keys, and secrets are populated via env vars (`CALYPSO_DB_PASSWORD`, `CALYPSO_JWT_SECRET`, `CALYPSO_JWT_SECRET`) or `config.yaml` on the host.
|
||||||
|
- All mutating endpoints should log audit events and guard via IAM roles (see `iam` package for role resolution).
|
||||||
|
- Bacula client registration and capability pushes require the `bacula-admin` role; enforce it via `requireRole("bacula-admin")` on `/api/v1/bacula/clients` routes.
|
||||||
|
|
||||||
|
## Frontend Tooling (React + TypeScript)
|
||||||
|
### Build & Run
|
||||||
|
- Work inside `frontend/`. Start the dev server with `npm run dev` (proxies `/api` → `http://localhost:8080`).
|
||||||
|
- Production builds use `npm run build`, which runs `tsc` before `vite build` to ensure typing correctness.
|
||||||
|
- Serve production output via `npm run preview` if you need to smoke-test a `dist/` snapshot.
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
- No automated frontend tests run currently. If you add a test harness (Jest/Vitest), ensure `npm test` or equivalent is added to `package.json`.
|
||||||
|
- For manual verification, interact with the dev server after logging into the backend API from `localhost:3000`.
|
||||||
|
|
||||||
|
### Lint & Formatting
|
||||||
|
- `npm run lint` runs ESLint across `.ts/.tsx` files with the `@typescript-eslint` recommended rules and `react-hooks` enforcement. It also blocks unused directives and sets `--max-warnings 0`.
|
||||||
|
- The linter requires you to keep hooks exhaustive and avoid unused vars; prefix intentionally unused arguments with `_` (e.g., `_event`).
|
||||||
|
- `tsconfig.json` sets `strict` mode, `noUnusedLocals`, `noUnusedParameters`, and `noFallthroughCasesInSwitch`. Respect these by keeping types precise and handling every branch.
|
||||||
|
|
||||||
|
## Frontend Style Guidelines (TypeScript + React)
|
||||||
|
### Imports & Aliases
|
||||||
|
- Use the `@/` alias defined in `tsconfig.json` for slices inside `src/` (e.g., `import Layout from '@/components/Layout'`).
|
||||||
|
- Import React/third-party packages at the top, followed by `@/` aliased modules, maintaining alphabetical order within each group.
|
||||||
|
- Prefer named exports for hooks and API utilities, default exports for page components (e.g., `export default function LoginPage()`).
|
||||||
|
|
||||||
|
### Formatting & Syntax
|
||||||
|
- Use single quotes for strings, omit semicolons (the repo follows Prettier/Vite defaults), and keep 2-space indentation inside JSX.
|
||||||
|
- Keep JSX clean: wrap complex class combinations and conditional fragments in template literals or helper functions, avoid inline object literal props unless necessary.
|
||||||
|
- Keep React components in kebab-case file names (e.g., `Login.tsx`, `Layout.tsx`) and align component names with file names.
|
||||||
|
|
||||||
|
### Types & Naming
|
||||||
|
- Use `interface` or `type` aliases for props and API shapes (e.g., `interface LoginRequest`), and add optional fields with `?`.
|
||||||
|
- Names for state setters, handlers, and store selectors follow camelCase (`setAuth`, `handleSubmit`, `loginMutation`).
|
||||||
|
- Use descriptive names for UI data (`navigation`, `userMenu`, `alerts`). Keep boolean flags prefixed with `is`, `has`, or `should`.
|
||||||
|
|
||||||
|
### State, Hooks, & Effects
|
||||||
|
- Prefer `useState`/`useEffect` for local UI state and `Zustand` stores (`useAuthStore`) for shared state.
|
||||||
|
- Keep `useEffect` dependency arrays explicit; avoid disabling exhaustive-deps rules unless documented.
|
||||||
|
- Manage async server interactions with TanStack Query (`useMutation`, `useQuery`) and centralize API clients (`src/api/client.ts`).
|
||||||
|
|
||||||
|
### Styling & Layout
|
||||||
|
- TailwindCSS classes are used for layout (`flex`, `gap`, `bg-...`). Keep className strings readable and group related classes together (spacing, color, typography).
|
||||||
|
- Use utility classes for responsive behavior (`lg:hidden`, `px-4`). For dynamic class toggling, compute the string in JS before passing it to `className`.
|
||||||
|
- Keep inline styles minimal; prefer CSS utilities and shared constants (colors, text sizing) defined in tailwind config.
|
||||||
|
|
||||||
|
### API & Networking
|
||||||
|
- Use `src/api/client.ts` as the single Axios instance (`baseURL: '/api/v1'`). Attach request/response interceptors for auth token injection/401 handling.
|
||||||
|
- Return typed promises from API helpers (e.g., `Promise<LoginResponse>`). Let callers handle success/errors via TanStack Query callbacks.
|
||||||
|
- Use `authApi.getMe`, `authApi.login`, etc., for all backend interactions; do not re-implement Axios calls directly in UI components.
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
- Surface backend errors by reading `error.response?.data?.error` (as seen in `LoginPage`) and displaying friendly messages via inline UI alerts or toast components.
|
||||||
|
- Clear authentication state on 401 responses using the Axios interceptor (`useAuthStore.getState().clearAuth()`), then redirect to `/login`.
|
||||||
|
- Display loading or pending states (`loginMutation.isPending`) instead of disabling UI abruptly; provide visual feedback for async actions.
|
||||||
|
|
||||||
|
## Repository Practices
|
||||||
|
- Keep database migration files under `db/migrations/` and refer to them from `internal/common/database/migrations/` for auto-run.
|
||||||
|
- When adding commands or packages, update the relevant `Makefile` target or `package.json` script, and document the command in this AGENTS file.
|
||||||
|
- Commit `go.mod`, `go.sum`, and `package-lock.json`/`package.json` updates together with code changes.
|
||||||
|
- Avoid committing binaries (`bin/`), coverage artifacts (`coverage.out`), or built frontend output (`frontend/dist/`). These are ignored by `.gitignore` but double-check before commits.
|
||||||
|
|
||||||
|
## Cursor & Copilot Rules
|
||||||
|
- There are no `.cursor/rules/`, `.cursorrules`, or `.github/copilot-instructions.md` files in this repository. Follow the guidelines laid out above.
|
||||||
Binary file not shown.
@@ -65,12 +65,13 @@ func main() {
|
|||||||
r := router.NewRouter(cfg, db, logger)
|
r := router.NewRouter(cfg, db, logger)
|
||||||
|
|
||||||
// Create HTTP server
|
// Create HTTP server
|
||||||
|
// Note: WriteTimeout should be 0 for WebSocket connections (they handle their own timeouts)
|
||||||
srv := &http.Server{
|
srv := &http.Server{
|
||||||
Addr: fmt.Sprintf(":%d", cfg.Server.Port),
|
Addr: fmt.Sprintf(":%d", cfg.Server.Port),
|
||||||
Handler: r,
|
Handler: r,
|
||||||
ReadTimeout: 15 * time.Second,
|
ReadTimeout: 15 * time.Second,
|
||||||
WriteTimeout: 15 * time.Second,
|
WriteTimeout: 0, // 0 means no timeout - needed for WebSocket connections
|
||||||
IdleTimeout: 60 * time.Second,
|
IdleTimeout: 120 * time.Second, // Increased for WebSocket keep-alive
|
||||||
}
|
}
|
||||||
|
|
||||||
// Setup graceful shutdown
|
// Setup graceful shutdown
|
||||||
|
|||||||
@@ -5,15 +5,19 @@ go 1.24.0
|
|||||||
toolchain go1.24.11
|
toolchain go1.24.11
|
||||||
|
|
||||||
require (
|
require (
|
||||||
|
github.com/creack/pty v1.1.24
|
||||||
github.com/gin-gonic/gin v1.10.0
|
github.com/gin-gonic/gin v1.10.0
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0
|
||||||
github.com/golang-jwt/jwt/v5 v5.2.1
|
github.com/golang-jwt/jwt/v5 v5.2.1
|
||||||
github.com/google/uuid v1.6.0
|
github.com/google/uuid v1.6.0
|
||||||
github.com/gorilla/websocket v1.5.3
|
github.com/gorilla/websocket v1.5.3
|
||||||
github.com/lib/pq v1.10.9
|
github.com/lib/pq v1.10.9
|
||||||
|
github.com/minio/madmin-go/v3 v3.0.110
|
||||||
|
github.com/minio/minio-go/v7 v7.0.97
|
||||||
github.com/stretchr/testify v1.11.1
|
github.com/stretchr/testify v1.11.1
|
||||||
go.uber.org/zap v1.27.0
|
go.uber.org/zap v1.27.0
|
||||||
golang.org/x/crypto v0.23.0
|
golang.org/x/crypto v0.37.0
|
||||||
golang.org/x/sync v0.7.0
|
golang.org/x/sync v0.15.0
|
||||||
golang.org/x/time v0.14.0
|
golang.org/x/time v0.14.0
|
||||||
gopkg.in/yaml.v3 v3.0.1
|
gopkg.in/yaml.v3 v3.0.1
|
||||||
)
|
)
|
||||||
@@ -21,29 +25,57 @@ require (
|
|||||||
require (
|
require (
|
||||||
github.com/bytedance/sonic v1.11.6 // indirect
|
github.com/bytedance/sonic v1.11.6 // indirect
|
||||||
github.com/bytedance/sonic/loader v0.1.1 // indirect
|
github.com/bytedance/sonic/loader v0.1.1 // indirect
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||||
github.com/cloudwego/base64x v0.1.4 // indirect
|
github.com/cloudwego/base64x v0.1.4 // indirect
|
||||||
github.com/cloudwego/iasm v0.2.0 // indirect
|
github.com/cloudwego/iasm v0.2.0 // indirect
|
||||||
github.com/davecgh/go-spew v1.1.1 // indirect
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
|
||||||
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||||
github.com/gin-contrib/sse v0.1.0 // indirect
|
github.com/gin-contrib/sse v0.1.0 // indirect
|
||||||
|
github.com/go-ini/ini v1.67.0 // indirect
|
||||||
|
github.com/go-ole/go-ole v1.3.0 // indirect
|
||||||
github.com/go-playground/locales v0.14.1 // indirect
|
github.com/go-playground/locales v0.14.1 // indirect
|
||||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||||
github.com/go-playground/validator/v10 v10.20.0 // indirect
|
github.com/goccy/go-json v0.10.5 // indirect
|
||||||
github.com/goccy/go-json v0.10.2 // indirect
|
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
|
||||||
|
github.com/golang/protobuf v1.5.4 // indirect
|
||||||
github.com/json-iterator/go v1.1.12 // indirect
|
github.com/json-iterator/go v1.1.12 // indirect
|
||||||
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
|
github.com/klauspost/compress v1.18.0 // indirect
|
||||||
|
github.com/klauspost/cpuid/v2 v2.2.11 // indirect
|
||||||
|
github.com/klauspost/crc32 v1.3.0 // indirect
|
||||||
github.com/leodido/go-urn v1.4.0 // indirect
|
github.com/leodido/go-urn v1.4.0 // indirect
|
||||||
|
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 // indirect
|
||||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||||
|
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
|
||||||
|
github.com/minio/crc64nvme v1.1.0 // indirect
|
||||||
|
github.com/minio/md5-simd v1.1.2 // indirect
|
||||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
||||||
github.com/modern-go/reflect2 v1.0.2 // indirect
|
github.com/modern-go/reflect2 v1.0.2 // indirect
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||||
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
|
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
|
||||||
github.com/pmezard/go-difflib v1.0.0 // indirect
|
github.com/philhofer/fwd v1.2.0 // indirect
|
||||||
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
||||||
|
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
|
||||||
|
github.com/prometheus/client_model v0.6.2 // indirect
|
||||||
|
github.com/prometheus/common v0.63.0 // indirect
|
||||||
|
github.com/prometheus/procfs v0.16.0 // indirect
|
||||||
|
github.com/prometheus/prom2json v1.4.2 // indirect
|
||||||
|
github.com/prometheus/prometheus v0.303.0 // indirect
|
||||||
|
github.com/rs/xid v1.6.0 // indirect
|
||||||
|
github.com/safchain/ethtool v0.5.10 // indirect
|
||||||
|
github.com/secure-io/sio-go v0.3.1 // indirect
|
||||||
|
github.com/shirou/gopsutil/v3 v3.24.5 // indirect
|
||||||
|
github.com/shoenig/go-m1cpu v0.1.6 // indirect
|
||||||
|
github.com/tinylib/msgp v1.3.0 // indirect
|
||||||
|
github.com/tklauser/go-sysconf v0.3.15 // indirect
|
||||||
|
github.com/tklauser/numcpus v0.10.0 // indirect
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||||
github.com/ugorji/go/codec v1.2.12 // indirect
|
github.com/ugorji/go/codec v1.2.12 // indirect
|
||||||
go.uber.org/multierr v1.10.0 // indirect
|
github.com/yusufpapurcu/wmi v1.2.4 // indirect
|
||||||
|
go.uber.org/multierr v1.11.0 // indirect
|
||||||
golang.org/x/arch v0.8.0 // indirect
|
golang.org/x/arch v0.8.0 // indirect
|
||||||
golang.org/x/net v0.25.0 // indirect
|
golang.org/x/net v0.39.0 // indirect
|
||||||
golang.org/x/sys v0.20.0 // indirect
|
golang.org/x/sys v0.34.0 // indirect
|
||||||
golang.org/x/text v0.15.0 // indirect
|
golang.org/x/text v0.26.0 // indirect
|
||||||
google.golang.org/protobuf v1.34.1 // indirect
|
google.golang.org/protobuf v1.36.6 // indirect
|
||||||
)
|
)
|
||||||
|
|||||||
137
backend/go.sum
137
backend/go.sum
@@ -2,19 +2,31 @@ github.com/bytedance/sonic v1.11.6 h1:oUp34TzMlL+OY1OUWxHqsdkgC/Zfc85zGqw9siXjrc
|
|||||||
github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4=
|
github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4=
|
||||||
github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM=
|
github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM=
|
||||||
github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
|
github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||||
github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y=
|
github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y=
|
||||||
github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
|
github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
|
||||||
github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
|
github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
|
||||||
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
|
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
|
||||||
|
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
|
||||||
|
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
|
||||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
|
||||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
|
||||||
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||||
|
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||||
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||||
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||||
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
|
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
|
||||||
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
|
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
|
||||||
github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU=
|
github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU=
|
||||||
github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
|
github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
|
||||||
|
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
|
||||||
|
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
|
||||||
|
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
|
||||||
|
github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
|
||||||
|
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
|
||||||
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||||
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||||
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||||
@@ -23,12 +35,17 @@ github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJn
|
|||||||
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||||
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
||||||
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||||
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
|
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
||||||
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||||
|
github.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=
|
||||||
|
github.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
|
||||||
github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
|
github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
|
||||||
github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
|
github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
|
||||||
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
|
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||||
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||||
|
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||||
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
|
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||||
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||||
@@ -36,25 +53,75 @@ github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aN
|
|||||||
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
|
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
|
||||||
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
||||||
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
||||||
|
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
|
||||||
|
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
|
||||||
|
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||||
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||||
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
|
github.com/klauspost/cpuid/v2 v2.2.11 h1:0OwqZRYI2rFrjS4kvkDnqJkKHdHaRnCm68/DY4OxRzU=
|
||||||
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
|
github.com/klauspost/cpuid/v2 v2.2.11/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||||
|
github.com/klauspost/crc32 v1.3.0 h1:sSmTt3gUt81RP655XGZPElI0PelVTZ6YwCRnPSupoFM=
|
||||||
|
github.com/klauspost/crc32 v1.3.0/go.mod h1:D7kQaZhnkX/Y0tstFGf8VUzv2UofNGqCjnC3zdHB0Hw=
|
||||||
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
|
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
|
||||||
|
github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
|
||||||
|
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
|
||||||
|
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||||
|
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||||
|
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 h1:PpXWgLPs+Fqr325bN2FD2ISlRRztXibcX6e8f5FR5Dc=
|
||||||
|
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
|
||||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||||
|
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
|
||||||
|
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
|
||||||
|
github.com/minio/crc64nvme v1.1.0 h1:e/tAguZ+4cw32D+IO/8GSf5UVr9y+3eJcxZI2WOO/7Q=
|
||||||
|
github.com/minio/crc64nvme v1.1.0/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
|
||||||
|
github.com/minio/madmin-go/v3 v3.0.110 h1:FIYekj7YPc430ffpXFWiUtyut3qBt/unIAcDzJn9H5M=
|
||||||
|
github.com/minio/madmin-go/v3 v3.0.110/go.mod h1:WOe2kYmYl1OIlY2DSRHVQ8j1v4OItARQ6jGyQqcCud8=
|
||||||
|
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
|
||||||
|
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
|
||||||
|
github.com/minio/minio-go/v7 v7.0.97 h1:lqhREPyfgHTB/ciX8k2r8k0D93WaFqxbJX36UZq5occ=
|
||||||
|
github.com/minio/minio-go/v7 v7.0.97/go.mod h1:re5VXuo0pwEtoNLsNuSr0RrLfT/MBtohwdaSmPPSRSk=
|
||||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
|
||||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||||
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
|
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
|
||||||
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||||
github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
|
github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
|
||||||
github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
|
github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
|
||||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
|
||||||
|
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
||||||
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
|
||||||
|
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
|
||||||
|
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
||||||
|
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
|
||||||
|
github.com/prometheus/common v0.63.0 h1:YR/EIY1o3mEFP/kZCD7iDMnLPlGyuU2Gb3HIcXnA98k=
|
||||||
|
github.com/prometheus/common v0.63.0/go.mod h1:VVFF/fBIoToEnWRVkYoXEkq3R3paCoxG9PXP74SnV18=
|
||||||
|
github.com/prometheus/procfs v0.16.0 h1:xh6oHhKwnOJKMYiYBDWmkHqQPyiY40sny36Cmx2bbsM=
|
||||||
|
github.com/prometheus/procfs v0.16.0/go.mod h1:8veyXUu3nGP7oaCxhX6yeaM5u4stL2FeMXnCqhDthZg=
|
||||||
|
github.com/prometheus/prom2json v1.4.2 h1:PxCTM+Whqi/eykO1MKsEL0p/zMpxp9ybpsmdFamw6po=
|
||||||
|
github.com/prometheus/prom2json v1.4.2/go.mod h1:zuvPm7u3epZSbXPWHny6G+o8ETgu6eAK3oPr6yFkRWE=
|
||||||
|
github.com/prometheus/prometheus v0.303.0 h1:wsNNsbd4EycMCphYnTmNY9JASBVbp7NWwJna857cGpA=
|
||||||
|
github.com/prometheus/prometheus v0.303.0/go.mod h1:8PMRi+Fk1WzopMDeb0/6hbNs9nV6zgySkU/zds5Lu3o=
|
||||||
|
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
|
||||||
|
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
|
||||||
|
github.com/safchain/ethtool v0.5.10 h1:Im294gZtuf4pSGJRAOGKaASNi3wMeFaGaWuSaomedpc=
|
||||||
|
github.com/safchain/ethtool v0.5.10/go.mod h1:w9jh2Lx7YBR4UwzLkzCmWl85UY0W2uZdd7/DckVE5+c=
|
||||||
|
github.com/secure-io/sio-go v0.3.1 h1:dNvY9awjabXTYGsTF1PiCySl9Ltofk9GA3VdWlo7rRc=
|
||||||
|
github.com/secure-io/sio-go v0.3.1/go.mod h1:+xbkjDzPjwh4Axd07pRKSNriS9SCiYksWnZqdnfpQxs=
|
||||||
|
github.com/shirou/gopsutil/v3 v3.24.5 h1:i0t8kL+kQTvpAYToeuiVk3TgDeKOFioZO3Ztz/iZ9pI=
|
||||||
|
github.com/shirou/gopsutil/v3 v3.24.5/go.mod h1:bsoOS1aStSs9ErQ1WWfxllSeS1K5D+U30r2NfcubMVk=
|
||||||
|
github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFtM=
|
||||||
|
github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ=
|
||||||
|
github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
|
||||||
|
github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k=
|
||||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||||
@@ -68,39 +135,57 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
|
|||||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||||
|
github.com/tinylib/msgp v1.3.0 h1:ULuf7GPooDaIlbyvgAxBV/FI7ynli6LZ1/nVUNu+0ww=
|
||||||
|
github.com/tinylib/msgp v1.3.0/go.mod h1:ykjzy2wzgrlvpDCRc4LA8UXy6D8bzMSuAF3WD57Gok0=
|
||||||
|
github.com/tklauser/go-sysconf v0.3.15 h1:VE89k0criAymJ/Os65CSn1IXaol+1wrsFHEB8Ol49K4=
|
||||||
|
github.com/tklauser/go-sysconf v0.3.15/go.mod h1:Dmjwr6tYFIseJw7a3dRLJfsHAMXZ3nEnL/aZY+0IuI4=
|
||||||
|
github.com/tklauser/numcpus v0.10.0 h1:18njr6LDBk1zuna922MgdjQuJFjrdppsZG60sHGfjso=
|
||||||
|
github.com/tklauser/numcpus v0.10.0/go.mod h1:BiTKazU708GQTYF4mB+cmlpT2Is1gLk7XVuEeem8LsQ=
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
|
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
||||||
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
|
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
|
||||||
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
|
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
|
||||||
|
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
|
||||||
|
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
|
||||||
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
||||||
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
||||||
go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
|
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
|
||||||
go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
|
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
|
||||||
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
|
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
|
||||||
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
|
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
|
||||||
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
||||||
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
|
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
|
||||||
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
|
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
|
||||||
golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI=
|
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||||
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
|
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||||
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
|
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
|
||||||
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
|
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
|
||||||
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
|
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||||
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
|
||||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
|
||||||
|
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||||
|
golang.org/x/sync v0.15.0 h1:KWH3jNZsfyT6xfAfKiz6MRNmd46ByHDYaZ7KSkCtdW8=
|
||||||
|
golang.org/x/sync v0.15.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
|
||||||
|
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||||
|
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
|
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
|
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
|
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
|
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
|
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||||
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
|
||||||
golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk=
|
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
|
||||||
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||||
|
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
|
||||||
|
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
|
||||||
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
|
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
|
||||||
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
|
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
|
||||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
|
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
|
||||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
|
||||||
google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg=
|
|
||||||
google.golang.org/protobuf v1.34.1/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
|
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
||||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
@@ -116,3 +116,281 @@ func (h *Handler) CreateJob(c *gin.Context) {
|
|||||||
|
|
||||||
c.JSON(http.StatusCreated, job)
|
c.JSON(http.StatusCreated, job)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ExecuteBconsoleCommand executes a bconsole command
|
||||||
|
func (h *Handler) ExecuteBconsoleCommand(c *gin.Context) {
|
||||||
|
var req struct {
|
||||||
|
Command string `json:"command" binding:"required"`
|
||||||
|
}
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "command is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
output, err := h.service.ExecuteBconsoleCommand(c.Request.Context(), req.Command)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to execute bconsole command", "error", err, "command", req.Command)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{
|
||||||
|
"error": "failed to execute command",
|
||||||
|
"output": output,
|
||||||
|
"details": err.Error(),
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"output": output,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListClients lists all backup clients with optional filters
|
||||||
|
func (h *Handler) ListClients(c *gin.Context) {
|
||||||
|
opts := ListClientsOptions{}
|
||||||
|
|
||||||
|
// Parse enabled filter
|
||||||
|
if enabledStr := c.Query("enabled"); enabledStr != "" {
|
||||||
|
enabled := enabledStr == "true"
|
||||||
|
opts.Enabled = &enabled
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse search query
|
||||||
|
opts.Search = c.Query("search")
|
||||||
|
|
||||||
|
// Parse category filter
|
||||||
|
if category := c.Query("category"); category != "" {
|
||||||
|
// Validate category
|
||||||
|
validCategories := map[string]bool{
|
||||||
|
"File": true,
|
||||||
|
"Database": true,
|
||||||
|
"Virtual": true,
|
||||||
|
}
|
||||||
|
if validCategories[category] {
|
||||||
|
opts.Category = category
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
clients, err := h.service.ListClients(c.Request.Context(), opts)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list clients", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{
|
||||||
|
"error": "failed to list clients",
|
||||||
|
"details": err.Error(),
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if clients == nil {
|
||||||
|
clients = []Client{}
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"clients": clients,
|
||||||
|
"total": len(clients),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetDashboardStats returns dashboard statistics
|
||||||
|
func (h *Handler) GetDashboardStats(c *gin.Context) {
|
||||||
|
stats, err := h.service.GetDashboardStats(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get dashboard stats", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get dashboard stats"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, stats)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListStoragePools lists all storage pools
|
||||||
|
func (h *Handler) ListStoragePools(c *gin.Context) {
|
||||||
|
pools, err := h.service.ListStoragePools(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list storage pools", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list storage pools"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if pools == nil {
|
||||||
|
pools = []StoragePool{}
|
||||||
|
}
|
||||||
|
|
||||||
|
h.logger.Info("Listed storage pools", "count", len(pools))
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"pools": pools,
|
||||||
|
"total": len(pools),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListStorageVolumes lists all storage volumes
|
||||||
|
func (h *Handler) ListStorageVolumes(c *gin.Context) {
|
||||||
|
poolName := c.Query("pool_name")
|
||||||
|
|
||||||
|
volumes, err := h.service.ListStorageVolumes(c.Request.Context(), poolName)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list storage volumes", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list storage volumes"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if volumes == nil {
|
||||||
|
volumes = []StorageVolume{}
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"volumes": volumes,
|
||||||
|
"total": len(volumes),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListStorageDaemons lists all storage daemons
|
||||||
|
func (h *Handler) ListStorageDaemons(c *gin.Context) {
|
||||||
|
daemons, err := h.service.ListStorageDaemons(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list storage daemons", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list storage daemons"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if daemons == nil {
|
||||||
|
daemons = []StorageDaemon{}
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"daemons": daemons,
|
||||||
|
"total": len(daemons),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateStoragePool creates a new storage pool
|
||||||
|
func (h *Handler) CreateStoragePool(c *gin.Context) {
|
||||||
|
var req CreatePoolRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
pool, err := h.service.CreateStoragePool(c.Request.Context(), req)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to create storage pool", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, pool)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteStoragePool deletes a storage pool
|
||||||
|
func (h *Handler) DeleteStoragePool(c *gin.Context) {
|
||||||
|
idStr := c.Param("id")
|
||||||
|
if idStr == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "pool ID is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var poolID int
|
||||||
|
if _, err := fmt.Sscanf(idStr, "%d", &poolID); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid pool ID"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
err := h.service.DeleteStoragePool(c.Request.Context(), poolID)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to delete storage pool", "error", err, "pool_id", poolID)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "pool deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateStorageVolume creates a new storage volume
|
||||||
|
func (h *Handler) CreateStorageVolume(c *gin.Context) {
|
||||||
|
var req CreateVolumeRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
volume, err := h.service.CreateStorageVolume(c.Request.Context(), req)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to create storage volume", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, volume)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateStorageVolume updates a storage volume
|
||||||
|
func (h *Handler) UpdateStorageVolume(c *gin.Context) {
|
||||||
|
idStr := c.Param("id")
|
||||||
|
if idStr == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "volume ID is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var volumeID int
|
||||||
|
if _, err := fmt.Sscanf(idStr, "%d", &volumeID); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid volume ID"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var req UpdateVolumeRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
volume, err := h.service.UpdateStorageVolume(c.Request.Context(), volumeID, req)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to update storage volume", "error", err, "volume_id", volumeID)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, volume)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteStorageVolume deletes a storage volume
|
||||||
|
func (h *Handler) DeleteStorageVolume(c *gin.Context) {
|
||||||
|
idStr := c.Param("id")
|
||||||
|
if idStr == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "volume ID is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var volumeID int
|
||||||
|
if _, err := fmt.Sscanf(idStr, "%d", &volumeID); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid volume ID"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
err := h.service.DeleteStorageVolume(c.Request.Context(), volumeID)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to delete storage volume", "error", err, "volume_id", volumeID)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "volume deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListMedia lists all media from bconsole "list media" command
|
||||||
|
func (h *Handler) ListMedia(c *gin.Context) {
|
||||||
|
media, err := h.service.ListMedia(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list media", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if media == nil {
|
||||||
|
media = []Media{}
|
||||||
|
}
|
||||||
|
|
||||||
|
h.logger.Info("Listed media", "count", len(media))
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"media": media,
|
||||||
|
"total": len(media),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
676
backend/internal/bacula/handler.go
Normal file
676
backend/internal/bacula/handler.go
Normal file
@@ -0,0 +1,676 @@
|
|||||||
|
package bacula
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/atlasos/calypso/internal/iam"
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/lib/pq"
|
||||||
|
"go.uber.org/zap"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
requestTimeout = 5 * time.Second
|
||||||
|
maxHistoryEntries = 10
|
||||||
|
)
|
||||||
|
|
||||||
|
type Handler struct {
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewHandler(db *database.DB, log *logger.Logger) *Handler {
|
||||||
|
return &Handler{db: db, logger: log.WithFields(zap.String("component", "bacula-handler"))}
|
||||||
|
}
|
||||||
|
|
||||||
|
type RegisterRequest struct {
|
||||||
|
Hostname string `json:"hostname" binding:"required"`
|
||||||
|
IPAddress string `json:"ip_address" binding:"required"`
|
||||||
|
AgentVersion string `json:"agent_version"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
BackupTypes []string `json:"backup_types" binding:"required"`
|
||||||
|
Metadata map[string]string `json:"metadata"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type UpdateCapabilitiesRequest struct {
|
||||||
|
BackupTypes []string `json:"backup_types" binding:"required"`
|
||||||
|
Notes string `json:"notes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type PingRequest struct {
|
||||||
|
Status string `json:"status"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type ClientResponse struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Hostname string `json:"hostname"`
|
||||||
|
IPAddress string `json:"ip_address"`
|
||||||
|
AgentVersion string `json:"agent_version"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
BackupTypes []string `json:"backup_types"`
|
||||||
|
PendingBackupTypes []string `json:"pending_backup_types,omitempty"`
|
||||||
|
PendingRequestedBy string `json:"pending_requested_by,omitempty"`
|
||||||
|
PendingRequestedAt *time.Time `json:"pending_requested_at,omitempty"`
|
||||||
|
PendingNotes string `json:"pending_notes,omitempty"`
|
||||||
|
Metadata map[string]interface{} `json:"metadata,omitempty"`
|
||||||
|
RegisteredBy string `json:"registered_by"`
|
||||||
|
LastSeen *time.Time `json:"last_seen,omitempty"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
UpdatedAt time.Time `json:"updated_at"`
|
||||||
|
CapabilityHistory []CapabilityHistoryEntry `json:"capability_history,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type CapabilityHistoryEntry struct {
|
||||||
|
BackupTypes []string `json:"backup_types"`
|
||||||
|
Source string `json:"source"`
|
||||||
|
RequestedBy string `json:"requested_by,omitempty"`
|
||||||
|
RequestedAt time.Time `json:"requested_at"`
|
||||||
|
Notes string `json:"notes,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type PendingUpdateResponse struct {
|
||||||
|
BackupTypes []string `json:"backup_types"`
|
||||||
|
RequestedBy string `json:"requested_by,omitempty"`
|
||||||
|
RequestedAt time.Time `json:"requested_at"`
|
||||||
|
Notes string `json:"notes,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) Register(c *gin.Context) {
|
||||||
|
var req RegisterRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request payload"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(req.BackupTypes) == 0 {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "backup_types is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
user, err := currentUser(c)
|
||||||
|
if err != nil {
|
||||||
|
c.JSON(http.StatusUnauthorized, gin.H{"error": "authentication required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
backupPayload, err := json.Marshal(req.BackupTypes)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to marshal backup types", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to encode backup types"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var metadataPayload []byte
|
||||||
|
if len(req.Metadata) > 0 {
|
||||||
|
metadataPayload, err = json.Marshal(req.Metadata)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to marshal metadata", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to encode metadata"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
status := req.Status
|
||||||
|
if status == "" {
|
||||||
|
status = "online"
|
||||||
|
}
|
||||||
|
|
||||||
|
tx, err := h.db.BeginTx(ctx, nil)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to begin database transaction", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to register client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
|
||||||
|
var row clientRow
|
||||||
|
err = tx.QueryRowContext(ctx, `
|
||||||
|
INSERT INTO bacula_clients (
|
||||||
|
hostname, ip_address, agent_version, status, backup_types, metadata,
|
||||||
|
registered_by_user_id, last_seen, updated_at
|
||||||
|
) VALUES ($1, $2, $3, $4, $5, $6, $7, NOW(), NOW())
|
||||||
|
ON CONFLICT (hostname) DO UPDATE SET
|
||||||
|
ip_address = EXCLUDED.ip_address,
|
||||||
|
agent_version = EXCLUDED.agent_version,
|
||||||
|
status = EXCLUDED.status,
|
||||||
|
backup_types = EXCLUDED.backup_types,
|
||||||
|
metadata = COALESCE(EXCLUDED.metadata, bacula_clients.metadata),
|
||||||
|
registered_by_user_id = EXCLUDED.registered_by_user_id,
|
||||||
|
last_seen = EXCLUDED.last_seen,
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING id, hostname, ip_address, agent_version, status, backup_types, metadata,
|
||||||
|
pending_backup_types, pending_requested_by, pending_requested_at, pending_notes,
|
||||||
|
registered_by_user_id, last_seen, created_at, updated_at
|
||||||
|
`, req.Hostname, req.IPAddress, req.AgentVersion, status, backupPayload, metadataPayload, user.ID).Scan(
|
||||||
|
&row.ID, &row.Hostname, &row.IPAddress, &row.AgentVersion, &row.Status, &row.BackupJSON,
|
||||||
|
&row.MetadataJSON, &row.PendingBackupJSON, &row.PendingRequestedBy, &row.PendingRequestedAt,
|
||||||
|
&row.PendingNotes, &row.RegisteredBy, &row.LastSeen, &row.CreatedAt, &row.UpdatedAt,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to ensure bacula client", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to register client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := buildClientResponse(&row)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to marshal client response", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to build response"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := insertCapabilityHistory(ctx, tx, row.ID, req.BackupTypes, "agent", user.ID, "agent registration"); err != nil {
|
||||||
|
h.logger.Error("failed to record capability history", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to record capability history"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(resp.PendingBackupTypes) > 0 && stringSlicesEqual(resp.PendingBackupTypes, resp.BackupTypes) {
|
||||||
|
if _, err := tx.ExecContext(ctx, `
|
||||||
|
UPDATE bacula_clients
|
||||||
|
SET pending_backup_types = NULL,
|
||||||
|
pending_requested_by = NULL,
|
||||||
|
pending_requested_at = NULL,
|
||||||
|
pending_notes = NULL,
|
||||||
|
updated_at = NOW()
|
||||||
|
WHERE id = $1
|
||||||
|
`, resp.ID); err != nil {
|
||||||
|
h.logger.Error("failed to clear pending capability update", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
resp.PendingBackupTypes = nil
|
||||||
|
resp.PendingRequestedBy = ""
|
||||||
|
resp.PendingRequestedAt = nil
|
||||||
|
resp.PendingNotes = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
h.logger.Error("failed to commit client registration", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to register client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) UpdateCapabilities(c *gin.Context) {
|
||||||
|
var req UpdateCapabilitiesRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request payload"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(req.BackupTypes) == 0 {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "backup_types is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
user, err := currentUser(c)
|
||||||
|
if err != nil {
|
||||||
|
c.JSON(http.StatusUnauthorized, gin.H{"error": "authentication required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
clientID := c.Param("id")
|
||||||
|
if clientID == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
tx, err := h.db.BeginTx(ctx, nil)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to begin transaction", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "unable to update capabilities"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
|
||||||
|
var exists bool
|
||||||
|
if err := tx.QueryRowContext(ctx, `SELECT EXISTS (SELECT 1 FROM bacula_clients WHERE id = $1)`, clientID).Scan(&exists); err != nil {
|
||||||
|
h.logger.Error("failed to verify client", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "unable to update client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !exists {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "client not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
backupPayload, err := json.Marshal(req.BackupTypes)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to marshal backup types", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to encode backup types"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := tx.ExecContext(ctx, `
|
||||||
|
UPDATE bacula_clients
|
||||||
|
SET pending_backup_types = $1,
|
||||||
|
pending_requested_by = $2,
|
||||||
|
pending_requested_at = NOW(),
|
||||||
|
pending_notes = $3,
|
||||||
|
updated_at = NOW()
|
||||||
|
WHERE id = $4
|
||||||
|
`, backupPayload, user.ID, req.Notes, clientID); err != nil {
|
||||||
|
h.logger.Error("failed to mark pending update", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := insertCapabilityHistory(ctx, tx, clientID, req.BackupTypes, "ui", user.ID, req.Notes); err != nil {
|
||||||
|
h.logger.Error("failed to insert capability history", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to record capability change"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
h.logger.Error("failed to commit capability update", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, PendingUpdateResponse{
|
||||||
|
BackupTypes: req.BackupTypes,
|
||||||
|
RequestedBy: user.ID,
|
||||||
|
RequestedAt: time.Now(),
|
||||||
|
Notes: req.Notes,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) GetPendingUpdate(c *gin.Context) {
|
||||||
|
clientID := c.Param("id")
|
||||||
|
if clientID == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
var pendingJSON []byte
|
||||||
|
var requestedBy sql.NullString
|
||||||
|
var requestedAt sql.NullTime
|
||||||
|
var notes sql.NullString
|
||||||
|
|
||||||
|
err := h.db.QueryRowContext(ctx, `
|
||||||
|
SELECT pending_backup_types, pending_requested_by, pending_requested_at, pending_notes
|
||||||
|
FROM bacula_clients
|
||||||
|
WHERE id = $1
|
||||||
|
`, clientID).Scan(&pendingJSON, &requestedBy, &requestedAt, ¬es)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "client not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("failed to fetch pending update", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to read pending update"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(pendingJSON) == 0 {
|
||||||
|
c.Status(http.StatusNoContent)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var backupTypes []string
|
||||||
|
if err := json.Unmarshal(pendingJSON, &backupTypes); err != nil {
|
||||||
|
h.logger.Error("failed to unmarshal pending backup types", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to read pending update"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
response := PendingUpdateResponse{
|
||||||
|
BackupTypes: backupTypes,
|
||||||
|
Notes: notes.String,
|
||||||
|
}
|
||||||
|
if requestedBy.Valid {
|
||||||
|
response.RequestedBy = requestedBy.String
|
||||||
|
}
|
||||||
|
if requestedAt.Valid {
|
||||||
|
response.RequestedAt = requestedAt.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, response)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) Ping(c *gin.Context) {
|
||||||
|
clientID := c.Param("id")
|
||||||
|
if clientID == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var req PingRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
// swallow body if absent
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
query := `
|
||||||
|
UPDATE bacula_clients
|
||||||
|
SET last_seen = NOW(),
|
||||||
|
status = COALESCE(NULLIF($2, ''), status),
|
||||||
|
updated_at = NOW()
|
||||||
|
WHERE id = $1
|
||||||
|
RETURNING id
|
||||||
|
`
|
||||||
|
|
||||||
|
var id string
|
||||||
|
err := h.db.QueryRowContext(ctx, query, clientID, req.Status).Scan(&id)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "client not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("failed to update heartbeat", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.Status(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) ListClients(c *gin.Context) {
|
||||||
|
ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
rows, err := h.db.QueryContext(ctx, `
|
||||||
|
SELECT id, hostname, ip_address, agent_version, status, backup_types, metadata,
|
||||||
|
pending_backup_types, pending_requested_by, pending_requested_at, pending_notes,
|
||||||
|
registered_by_user_id, last_seen, created_at, updated_at
|
||||||
|
FROM bacula_clients
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
`)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to query clients", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch clients"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var clients []*ClientResponse
|
||||||
|
var ids []string
|
||||||
|
for rows.Next() {
|
||||||
|
row := clientRow{}
|
||||||
|
if err := rows.Scan(&row.ID, &row.Hostname, &row.IPAddress, &row.AgentVersion, &row.Status,
|
||||||
|
&row.BackupJSON, &row.MetadataJSON, &row.PendingBackupJSON, &row.PendingRequestedBy,
|
||||||
|
&row.PendingRequestedAt, &row.PendingNotes, &row.RegisteredBy, &row.LastSeen,
|
||||||
|
&row.CreatedAt, &row.UpdatedAt); err != nil {
|
||||||
|
h.logger.Error("failed to scan client row", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch clients"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
resp, err := buildClientResponse(&row)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to build client response", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch clients"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
clients = append(clients, resp)
|
||||||
|
ids = append(ids, resp.ID)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(ids) > 0 {
|
||||||
|
if err := h.attachHistory(ctx, ids, clients); err != nil {
|
||||||
|
h.logger.Error("failed to attach history", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client history"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, clients)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) GetClient(c *gin.Context) {
|
||||||
|
clientID := c.Param("id")
|
||||||
|
if clientID == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
var row clientRow
|
||||||
|
err := h.db.QueryRowContext(ctx, `
|
||||||
|
SELECT id, hostname, ip_address, agent_version, status, backup_types, metadata,
|
||||||
|
pending_backup_types, pending_requested_by, pending_requested_at, pending_notes,
|
||||||
|
registered_by_user_id, last_seen, created_at, updated_at
|
||||||
|
FROM bacula_clients
|
||||||
|
WHERE id = $1
|
||||||
|
`, clientID).Scan(&row.ID, &row.Hostname, &row.IPAddress, &row.AgentVersion, &row.Status,
|
||||||
|
&row.BackupJSON, &row.MetadataJSON, &row.PendingBackupJSON, &row.PendingRequestedBy,
|
||||||
|
&row.PendingRequestedAt, &row.PendingNotes, &row.RegisteredBy, &row.LastSeen,
|
||||||
|
&row.CreatedAt, &row.UpdatedAt)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "client not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("failed to read client", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := buildClientResponse(&row)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("failed to build client response", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.attachHistory(ctx, []string{resp.ID}, []*ClientResponse{resp}); err != nil {
|
||||||
|
h.logger.Error("failed to attach history", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client history"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Handler) attachHistory(ctx context.Context, ids []string, clients []*ClientResponse) error {
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := h.db.QueryContext(ctx, `
|
||||||
|
SELECT client_id, backup_types, source, requested_by_user_id, requested_at, notes
|
||||||
|
FROM bacula_client_capability_history
|
||||||
|
WHERE client_id = ANY($1)
|
||||||
|
ORDER BY requested_at DESC
|
||||||
|
`, pq.Array(ids))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
clientMap := make(map[string]*ClientResponse)
|
||||||
|
for _, client := range clients {
|
||||||
|
clientMap[client.ID] = client
|
||||||
|
}
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var clientID string
|
||||||
|
var backupJSON []byte
|
||||||
|
var source string
|
||||||
|
var requestedBy sql.NullString
|
||||||
|
var requestedAt time.Time
|
||||||
|
var notes sql.NullString
|
||||||
|
|
||||||
|
if err := rows.Scan(&clientID, &backupJSON, &source, &requestedBy, &requestedAt, ¬es); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, ok := clientMap[clientID]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(resp.CapabilityHistory) >= maxHistoryEntries {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
var backupTypes []string
|
||||||
|
if err := json.Unmarshal(backupJSON, &backupTypes); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
entry := CapabilityHistoryEntry{
|
||||||
|
BackupTypes: backupTypes,
|
||||||
|
Source: source,
|
||||||
|
RequestedAt: requestedAt,
|
||||||
|
}
|
||||||
|
if requestedBy.Valid {
|
||||||
|
entry.RequestedBy = requestedBy.String
|
||||||
|
}
|
||||||
|
if notes.Valid {
|
||||||
|
entry.Notes = notes.String
|
||||||
|
}
|
||||||
|
|
||||||
|
resp.CapabilityHistory = append(resp.CapabilityHistory, entry)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type clientRow struct {
|
||||||
|
ID string
|
||||||
|
Hostname string
|
||||||
|
IPAddress string
|
||||||
|
AgentVersion string
|
||||||
|
Status string
|
||||||
|
BackupJSON []byte
|
||||||
|
MetadataJSON []byte
|
||||||
|
PendingBackupJSON []byte
|
||||||
|
PendingRequestedBy sql.NullString
|
||||||
|
PendingRequestedAt sql.NullTime
|
||||||
|
PendingNotes sql.NullString
|
||||||
|
RegisteredBy string
|
||||||
|
LastSeen sql.NullTime
|
||||||
|
CreatedAt time.Time
|
||||||
|
UpdatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildClientResponse(row *clientRow) (*ClientResponse, error) {
|
||||||
|
backupTypes, err := decodeStringSlice(row.BackupJSON)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
pendingTypes, err := decodeStringSlice(row.PendingBackupJSON)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata, err := decodeMetadata(row.MetadataJSON)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
resp := &ClientResponse{
|
||||||
|
ID: row.ID,
|
||||||
|
Hostname: row.Hostname,
|
||||||
|
IPAddress: row.IPAddress,
|
||||||
|
AgentVersion: row.AgentVersion,
|
||||||
|
Status: row.Status,
|
||||||
|
BackupTypes: backupTypes,
|
||||||
|
Metadata: metadata,
|
||||||
|
RegisteredBy: row.RegisteredBy,
|
||||||
|
CreatedAt: row.CreatedAt,
|
||||||
|
UpdatedAt: row.UpdatedAt,
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(pendingTypes) > 0 {
|
||||||
|
resp.PendingBackupTypes = pendingTypes
|
||||||
|
if row.PendingRequestedBy.Valid {
|
||||||
|
resp.PendingRequestedBy = row.PendingRequestedBy.String
|
||||||
|
}
|
||||||
|
if row.PendingRequestedAt.Valid {
|
||||||
|
resp.PendingRequestedAt = &row.PendingRequestedAt.Time
|
||||||
|
}
|
||||||
|
if row.PendingNotes.Valid {
|
||||||
|
resp.PendingNotes = row.PendingNotes.String
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if row.LastSeen.Valid {
|
||||||
|
resp.LastSeen = &row.LastSeen.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
return resp, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func currentUser(c *gin.Context) (*iam.User, error) {
|
||||||
|
user, exists := c.Get("user")
|
||||||
|
if !exists {
|
||||||
|
return nil, sql.ErrNoRows
|
||||||
|
}
|
||||||
|
authUser, ok := user.(*iam.User)
|
||||||
|
if !ok {
|
||||||
|
return nil, sql.ErrNoRows
|
||||||
|
}
|
||||||
|
return authUser, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func decodeStringSlice(data []byte) ([]string, error) {
|
||||||
|
if len(data) == 0 {
|
||||||
|
return []string{}, nil
|
||||||
|
}
|
||||||
|
var dst []string
|
||||||
|
if err := json.Unmarshal(data, &dst); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return dst, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func decodeMetadata(data []byte) (map[string]interface{}, error) {
|
||||||
|
if len(data) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
var metadata map[string]interface{}
|
||||||
|
if err := json.Unmarshal(data, &metadata); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return metadata, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func insertCapabilityHistory(ctx context.Context, tx *sql.Tx, clientID string, backupTypes []string, source, requestedBy, notes string) error {
|
||||||
|
payload, err := json.Marshal(backupTypes)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = tx.ExecContext(ctx, `
|
||||||
|
INSERT INTO bacula_client_capability_history (
|
||||||
|
client_id, backup_types, source, requested_by_user_id, notes
|
||||||
|
) VALUES ($1, $2, $3, $4, $5)
|
||||||
|
`, clientID, payload, source, requestedBy, notes)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func stringSlicesEqual(a, b []string) bool {
|
||||||
|
if len(a) != len(b) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := range a {
|
||||||
|
if a[i] != b[i] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
@@ -10,11 +10,12 @@ import (
|
|||||||
|
|
||||||
// Config represents the application configuration
|
// Config represents the application configuration
|
||||||
type Config struct {
|
type Config struct {
|
||||||
Server ServerConfig `yaml:"server"`
|
Server ServerConfig `yaml:"server"`
|
||||||
Database DatabaseConfig `yaml:"database"`
|
Database DatabaseConfig `yaml:"database"`
|
||||||
Auth AuthConfig `yaml:"auth"`
|
Auth AuthConfig `yaml:"auth"`
|
||||||
Logging LoggingConfig `yaml:"logging"`
|
Logging LoggingConfig `yaml:"logging"`
|
||||||
Security SecurityConfig `yaml:"security"`
|
Security SecurityConfig `yaml:"security"`
|
||||||
|
ObjectStorage ObjectStorageConfig `yaml:"object_storage"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ServerConfig holds HTTP server configuration
|
// ServerConfig holds HTTP server configuration
|
||||||
@@ -96,6 +97,14 @@ type SecurityHeadersConfig struct {
|
|||||||
Enabled bool `yaml:"enabled"`
|
Enabled bool `yaml:"enabled"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ObjectStorageConfig holds MinIO configuration
|
||||||
|
type ObjectStorageConfig struct {
|
||||||
|
Endpoint string `yaml:"endpoint"`
|
||||||
|
AccessKey string `yaml:"access_key"`
|
||||||
|
SecretKey string `yaml:"secret_key"`
|
||||||
|
UseSSL bool `yaml:"use_ssl"`
|
||||||
|
}
|
||||||
|
|
||||||
// Load reads configuration from file and environment variables
|
// Load reads configuration from file and environment variables
|
||||||
func Load(path string) (*Config, error) {
|
func Load(path string) (*Config, error) {
|
||||||
cfg := DefaultConfig()
|
cfg := DefaultConfig()
|
||||||
|
|||||||
@@ -0,0 +1,22 @@
|
|||||||
|
-- Migration: Object Storage Configuration
|
||||||
|
-- Description: Creates table for storing MinIO object storage configuration
|
||||||
|
-- Date: 2026-01-09
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS object_storage_config (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
dataset_path VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
mount_point VARCHAR(512) NOT NULL,
|
||||||
|
pool_name VARCHAR(255) NOT NULL,
|
||||||
|
dataset_name VARCHAR(255) NOT NULL,
|
||||||
|
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_object_storage_config_pool_name ON object_storage_config(pool_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_object_storage_config_updated_at ON object_storage_config(updated_at);
|
||||||
|
|
||||||
|
COMMENT ON TABLE object_storage_config IS 'Stores MinIO object storage configuration, linking to ZFS datasets';
|
||||||
|
COMMENT ON COLUMN object_storage_config.dataset_path IS 'Full ZFS dataset path (e.g., pool/dataset)';
|
||||||
|
COMMENT ON COLUMN object_storage_config.mount_point IS 'Mount point path for the dataset';
|
||||||
|
COMMENT ON COLUMN object_storage_config.pool_name IS 'ZFS pool name';
|
||||||
|
COMMENT ON COLUMN object_storage_config.dataset_name IS 'ZFS dataset name';
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
-- Snapshot schedules table for automated snapshot creation
|
||||||
|
CREATE TABLE IF NOT EXISTS snapshot_schedules (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
dataset VARCHAR(255) NOT NULL,
|
||||||
|
snapshot_name_template VARCHAR(255) NOT NULL, -- e.g., "auto-%Y-%m-%d-%H%M" or "daily-backup"
|
||||||
|
schedule_type VARCHAR(50) NOT NULL, -- 'hourly', 'daily', 'weekly', 'monthly', 'cron'
|
||||||
|
schedule_config JSONB NOT NULL, -- For cron: {"cron": "0 0 * * *"}, for others: {"time": "00:00", "day": 1, etc.}
|
||||||
|
recursive BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
retention_count INTEGER, -- Keep last N snapshots (null = unlimited)
|
||||||
|
retention_days INTEGER, -- Keep snapshots for N days (null = unlimited)
|
||||||
|
last_run_at TIMESTAMP,
|
||||||
|
next_run_at TIMESTAMP,
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(name)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_snapshot_schedules_enabled ON snapshot_schedules(enabled);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_snapshot_schedules_next_run ON snapshot_schedules(next_run_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_snapshot_schedules_dataset ON snapshot_schedules(dataset);
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
-- ZFS Replication Tasks Table
|
||||||
|
-- Supports both outbound (sender) and inbound (receiver) replication
|
||||||
|
CREATE TABLE IF NOT EXISTS replication_tasks (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
direction VARCHAR(20) NOT NULL, -- 'outbound' (sender) or 'inbound' (receiver)
|
||||||
|
|
||||||
|
-- For outbound replication (sender)
|
||||||
|
source_dataset VARCHAR(512), -- Source dataset on this system (outbound) or remote system (inbound)
|
||||||
|
target_host VARCHAR(255), -- Target host IP or hostname (for outbound)
|
||||||
|
target_port INTEGER DEFAULT 22, -- SSH port (default 22, for outbound)
|
||||||
|
target_user VARCHAR(255) DEFAULT 'root', -- SSH user (for outbound)
|
||||||
|
target_dataset VARCHAR(512), -- Target dataset on remote system (for outbound)
|
||||||
|
target_ssh_key_path TEXT, -- Path to SSH private key (for outbound)
|
||||||
|
|
||||||
|
-- For inbound replication (receiver)
|
||||||
|
source_host VARCHAR(255), -- Source host IP or hostname (for inbound)
|
||||||
|
source_port INTEGER DEFAULT 22, -- SSH port (for inbound)
|
||||||
|
source_user VARCHAR(255) DEFAULT 'root', -- SSH user (for inbound)
|
||||||
|
local_dataset VARCHAR(512), -- Local dataset to receive snapshots (for inbound)
|
||||||
|
|
||||||
|
-- Common settings
|
||||||
|
schedule_type VARCHAR(50), -- 'manual', 'hourly', 'daily', 'weekly', 'monthly', 'cron'
|
||||||
|
schedule_config JSONB, -- Schedule configuration (similar to snapshot schedules)
|
||||||
|
compression VARCHAR(50) DEFAULT 'lz4', -- 'off', 'lz4', 'gzip', 'zstd'
|
||||||
|
encryption BOOLEAN DEFAULT false, -- Enable encryption during transfer
|
||||||
|
recursive BOOLEAN DEFAULT false, -- Replicate recursively
|
||||||
|
incremental BOOLEAN DEFAULT true, -- Use incremental replication
|
||||||
|
auto_snapshot BOOLEAN DEFAULT true, -- Auto-create snapshot before replication
|
||||||
|
|
||||||
|
-- Status and tracking
|
||||||
|
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
status VARCHAR(50) DEFAULT 'idle', -- 'idle', 'running', 'failed', 'paused'
|
||||||
|
last_run_at TIMESTAMP,
|
||||||
|
last_run_status VARCHAR(50), -- 'success', 'failed', 'partial'
|
||||||
|
last_run_error TEXT,
|
||||||
|
next_run_at TIMESTAMP,
|
||||||
|
last_snapshot_sent VARCHAR(512), -- Last snapshot successfully sent (for outbound)
|
||||||
|
last_snapshot_received VARCHAR(512), -- Last snapshot successfully received (for inbound)
|
||||||
|
|
||||||
|
-- Statistics
|
||||||
|
total_runs INTEGER DEFAULT 0,
|
||||||
|
successful_runs INTEGER DEFAULT 0,
|
||||||
|
failed_runs INTEGER DEFAULT 0,
|
||||||
|
bytes_sent BIGINT DEFAULT 0, -- Total bytes sent (for outbound)
|
||||||
|
bytes_received BIGINT DEFAULT 0, -- Total bytes received (for inbound)
|
||||||
|
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
|
||||||
|
-- Validation: ensure required fields based on direction
|
||||||
|
CONSTRAINT chk_direction CHECK (direction IN ('outbound', 'inbound')),
|
||||||
|
CONSTRAINT chk_outbound_fields CHECK (
|
||||||
|
direction != 'outbound' OR (
|
||||||
|
source_dataset IS NOT NULL AND
|
||||||
|
target_host IS NOT NULL AND
|
||||||
|
target_dataset IS NOT NULL
|
||||||
|
)
|
||||||
|
),
|
||||||
|
CONSTRAINT chk_inbound_fields CHECK (
|
||||||
|
direction != 'inbound' OR (
|
||||||
|
source_host IS NOT NULL AND
|
||||||
|
source_dataset IS NOT NULL AND
|
||||||
|
local_dataset IS NOT NULL
|
||||||
|
)
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Create indexes
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_direction ON replication_tasks(direction);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_enabled ON replication_tasks(enabled);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_status ON replication_tasks(status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_next_run ON replication_tasks(next_run_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_source_dataset ON replication_tasks(source_dataset);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_target_host ON replication_tasks(target_host);
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Client Categories Schema
|
||||||
|
-- Version: 14.0
|
||||||
|
-- Adds category support for backup clients (File, Database, Virtual)
|
||||||
|
|
||||||
|
-- Client metadata table to store additional information about clients
|
||||||
|
-- This extends the Bacula Client table with Calypso-specific metadata
|
||||||
|
CREATE TABLE IF NOT EXISTS client_metadata (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
client_id INTEGER NOT NULL, -- Bacula Client.ClientId
|
||||||
|
client_name VARCHAR(255) NOT NULL, -- Bacula Client.Name (for reference)
|
||||||
|
category VARCHAR(50) NOT NULL DEFAULT 'File', -- 'File', 'Database', 'Virtual'
|
||||||
|
description TEXT,
|
||||||
|
tags JSONB, -- Additional tags/metadata as JSON
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
|
||||||
|
-- Ensure one metadata entry per client
|
||||||
|
CONSTRAINT unique_client_id UNIQUE (client_id),
|
||||||
|
CONSTRAINT chk_category CHECK (category IN ('File', 'Database', 'Virtual'))
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Indexes for performance
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_client_metadata_client_id ON client_metadata(client_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_client_metadata_client_name ON client_metadata(client_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_client_metadata_category ON client_metadata(category);
|
||||||
|
|
||||||
|
-- Add comment
|
||||||
|
COMMENT ON TABLE client_metadata IS 'Stores Calypso-specific metadata for backup clients, including category classification';
|
||||||
|
COMMENT ON COLUMN client_metadata.category IS 'Client category: File (file system backups), Database (database backups), Virtual (virtual machine backups)';
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Migration 015: Add Bacula clients and capability history tables
|
||||||
|
--
|
||||||
|
-- Adds tables for tracking registered Bacula agents, their backup capabilities,
|
||||||
|
-- and a history log for UI- or agent-triggered capability changes. Pending
|
||||||
|
-- updates are stored on the client row until the agent pulls them.
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS bacula_clients (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
hostname TEXT NOT NULL,
|
||||||
|
ip_address TEXT,
|
||||||
|
agent_version TEXT,
|
||||||
|
status TEXT NOT NULL DEFAULT 'online',
|
||||||
|
backup_types JSONB NOT NULL,
|
||||||
|
pending_backup_types JSONB,
|
||||||
|
pending_requested_by UUID,
|
||||||
|
pending_requested_at TIMESTAMPTZ,
|
||||||
|
pending_notes TEXT,
|
||||||
|
metadata JSONB,
|
||||||
|
registered_by_user_id UUID NOT NULL,
|
||||||
|
last_seen TIMESTAMPTZ,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
CONSTRAINT uniq_bacula_clients_hostname UNIQUE (hostname)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_clients_registered_by ON bacula_clients (registered_by_user_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_clients_status ON bacula_clients (status);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS bacula_client_capability_history (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
client_id UUID NOT NULL REFERENCES bacula_clients (id) ON DELETE CASCADE,
|
||||||
|
backup_types JSONB NOT NULL,
|
||||||
|
source TEXT NOT NULL,
|
||||||
|
requested_by_user_id UUID,
|
||||||
|
requested_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
notes TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_client_capability_history_client ON bacula_client_capability_history (client_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_client_capability_history_requested_at ON bacula_client_capability_history (requested_at);
|
||||||
|
|
||||||
|
COMMENT ON TABLE bacula_clients IS 'Tracks Bacula clients registered with Calypso, including pending capability pushes.';
|
||||||
|
COMMENT ON TABLE bacula_client_capability_history IS 'Audit history of backup capability changes per client.';
|
||||||
@@ -13,24 +13,30 @@ import (
|
|||||||
// authMiddleware validates JWT tokens and sets user context
|
// authMiddleware validates JWT tokens and sets user context
|
||||||
func authMiddleware(authHandler *auth.Handler) gin.HandlerFunc {
|
func authMiddleware(authHandler *auth.Handler) gin.HandlerFunc {
|
||||||
return func(c *gin.Context) {
|
return func(c *gin.Context) {
|
||||||
// Extract token from Authorization header
|
var token string
|
||||||
|
|
||||||
|
// Try to extract token from Authorization header first
|
||||||
authHeader := c.GetHeader("Authorization")
|
authHeader := c.GetHeader("Authorization")
|
||||||
if authHeader == "" {
|
if authHeader != "" {
|
||||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "missing authorization header"})
|
// Parse Bearer token
|
||||||
|
parts := strings.SplitN(authHeader, " ", 2)
|
||||||
|
if len(parts) == 2 && parts[0] == "Bearer" {
|
||||||
|
token = parts[1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// If no token from header, try query parameter (for WebSocket)
|
||||||
|
if token == "" {
|
||||||
|
token = c.Query("token")
|
||||||
|
}
|
||||||
|
|
||||||
|
// If still no token, return error
|
||||||
|
if token == "" {
|
||||||
|
c.JSON(http.StatusUnauthorized, gin.H{"error": "missing authorization token"})
|
||||||
c.Abort()
|
c.Abort()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Parse Bearer token
|
|
||||||
parts := strings.SplitN(authHeader, " ", 2)
|
|
||||||
if len(parts) != 2 || parts[0] != "Bearer" {
|
|
||||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid authorization header format"})
|
|
||||||
c.Abort()
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
token := parts[1]
|
|
||||||
|
|
||||||
// Validate token and get user
|
// Validate token and get user
|
||||||
user, err := authHandler.ValidateToken(token)
|
user, err := authHandler.ValidateToken(token)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -7,13 +7,17 @@ import (
|
|||||||
"github.com/atlasos/calypso/internal/audit"
|
"github.com/atlasos/calypso/internal/audit"
|
||||||
"github.com/atlasos/calypso/internal/auth"
|
"github.com/atlasos/calypso/internal/auth"
|
||||||
"github.com/atlasos/calypso/internal/backup"
|
"github.com/atlasos/calypso/internal/backup"
|
||||||
|
"github.com/atlasos/calypso/internal/bacula"
|
||||||
|
|
||||||
"github.com/atlasos/calypso/internal/common/cache"
|
"github.com/atlasos/calypso/internal/common/cache"
|
||||||
"github.com/atlasos/calypso/internal/common/config"
|
"github.com/atlasos/calypso/internal/common/config"
|
||||||
"github.com/atlasos/calypso/internal/common/database"
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
"github.com/atlasos/calypso/internal/common/logger"
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
"github.com/atlasos/calypso/internal/iam"
|
"github.com/atlasos/calypso/internal/iam"
|
||||||
"github.com/atlasos/calypso/internal/monitoring"
|
"github.com/atlasos/calypso/internal/monitoring"
|
||||||
|
"github.com/atlasos/calypso/internal/object_storage"
|
||||||
"github.com/atlasos/calypso/internal/scst"
|
"github.com/atlasos/calypso/internal/scst"
|
||||||
|
"github.com/atlasos/calypso/internal/shares"
|
||||||
"github.com/atlasos/calypso/internal/storage"
|
"github.com/atlasos/calypso/internal/storage"
|
||||||
"github.com/atlasos/calypso/internal/system"
|
"github.com/atlasos/calypso/internal/system"
|
||||||
"github.com/atlasos/calypso/internal/tape_physical"
|
"github.com/atlasos/calypso/internal/tape_physical"
|
||||||
@@ -174,6 +178,11 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
zfsPoolMonitor := storage.NewZFSPoolMonitor(db, log, 2*time.Minute)
|
zfsPoolMonitor := storage.NewZFSPoolMonitor(db, log, 2*time.Minute)
|
||||||
go zfsPoolMonitor.Start(context.Background())
|
go zfsPoolMonitor.Start(context.Background())
|
||||||
|
|
||||||
|
// Start snapshot schedule worker in background (checks schedules every 1 minute)
|
||||||
|
snapshotService := storage.NewSnapshotService(db, log)
|
||||||
|
snapshotScheduleWorker := storage.NewSnapshotScheduleWorker(db, log, snapshotService, 1*time.Minute)
|
||||||
|
go snapshotScheduleWorker.Start(context.Background())
|
||||||
|
|
||||||
storageGroup := protected.Group("/storage")
|
storageGroup := protected.Group("/storage")
|
||||||
storageGroup.Use(requirePermission("storage", "read"))
|
storageGroup.Use(requirePermission("storage", "read"))
|
||||||
{
|
{
|
||||||
@@ -194,10 +203,83 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
storageGroup.GET("/zfs/pools/:id/datasets", storageHandler.ListZFSDatasets)
|
storageGroup.GET("/zfs/pools/:id/datasets", storageHandler.ListZFSDatasets)
|
||||||
storageGroup.POST("/zfs/pools/:id/datasets", requirePermission("storage", "write"), storageHandler.CreateZFSDataset)
|
storageGroup.POST("/zfs/pools/:id/datasets", requirePermission("storage", "write"), storageHandler.CreateZFSDataset)
|
||||||
storageGroup.DELETE("/zfs/pools/:id/datasets/:dataset", requirePermission("storage", "write"), storageHandler.DeleteZFSDataset)
|
storageGroup.DELETE("/zfs/pools/:id/datasets/:dataset", requirePermission("storage", "write"), storageHandler.DeleteZFSDataset)
|
||||||
|
// ZFS Snapshots
|
||||||
|
storageGroup.GET("/zfs/snapshots", storageHandler.ListSnapshots)
|
||||||
|
storageGroup.POST("/zfs/snapshots", requirePermission("storage", "write"), storageHandler.CreateSnapshot)
|
||||||
|
storageGroup.DELETE("/zfs/snapshots/:name", requirePermission("storage", "write"), storageHandler.DeleteSnapshot)
|
||||||
|
storageGroup.POST("/zfs/snapshots/:name/rollback", requirePermission("storage", "write"), storageHandler.RollbackSnapshot)
|
||||||
|
storageGroup.POST("/zfs/snapshots/:name/clone", requirePermission("storage", "write"), storageHandler.CloneSnapshot)
|
||||||
|
// Snapshot Schedules
|
||||||
|
storageGroup.GET("/zfs/snapshot-schedules", storageHandler.ListSnapshotSchedules)
|
||||||
|
storageGroup.GET("/zfs/snapshot-schedules/:id", storageHandler.GetSnapshotSchedule)
|
||||||
|
storageGroup.POST("/zfs/snapshot-schedules", requirePermission("storage", "write"), storageHandler.CreateSnapshotSchedule)
|
||||||
|
storageGroup.PUT("/zfs/snapshot-schedules/:id", requirePermission("storage", "write"), storageHandler.UpdateSnapshotSchedule)
|
||||||
|
storageGroup.DELETE("/zfs/snapshot-schedules/:id", requirePermission("storage", "write"), storageHandler.DeleteSnapshotSchedule)
|
||||||
|
storageGroup.POST("/zfs/snapshot-schedules/:id/toggle", requirePermission("storage", "write"), storageHandler.ToggleSnapshotSchedule)
|
||||||
|
|
||||||
|
// Replication Tasks
|
||||||
|
storageGroup.GET("/zfs/replication-tasks", storageHandler.ListReplicationTasks)
|
||||||
|
storageGroup.GET("/zfs/replication-tasks/:id", storageHandler.GetReplicationTask)
|
||||||
|
storageGroup.POST("/zfs/replication-tasks", requirePermission("storage", "write"), storageHandler.CreateReplicationTask)
|
||||||
|
storageGroup.PUT("/zfs/replication-tasks/:id", requirePermission("storage", "write"), storageHandler.UpdateReplicationTask)
|
||||||
|
storageGroup.DELETE("/zfs/replication-tasks/:id", requirePermission("storage", "write"), storageHandler.DeleteReplicationTask)
|
||||||
// ZFS ARC Stats
|
// ZFS ARC Stats
|
||||||
storageGroup.GET("/zfs/arc/stats", storageHandler.GetARCStats)
|
storageGroup.GET("/zfs/arc/stats", storageHandler.GetARCStats)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Shares (CIFS/NFS)
|
||||||
|
sharesHandler := shares.NewHandler(db, log)
|
||||||
|
sharesGroup := protected.Group("/shares")
|
||||||
|
sharesGroup.Use(requirePermission("storage", "read"))
|
||||||
|
{
|
||||||
|
sharesGroup.GET("", sharesHandler.ListShares)
|
||||||
|
sharesGroup.GET("/:id", sharesHandler.GetShare)
|
||||||
|
sharesGroup.POST("", requirePermission("storage", "write"), sharesHandler.CreateShare)
|
||||||
|
sharesGroup.PUT("/:id", requirePermission("storage", "write"), sharesHandler.UpdateShare)
|
||||||
|
sharesGroup.DELETE("/:id", requirePermission("storage", "write"), sharesHandler.DeleteShare)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Object Storage (MinIO)
|
||||||
|
// Initialize MinIO service if configured
|
||||||
|
if cfg.ObjectStorage.Endpoint != "" {
|
||||||
|
objectStorageService, err := object_storage.NewService(
|
||||||
|
cfg.ObjectStorage.Endpoint,
|
||||||
|
cfg.ObjectStorage.AccessKey,
|
||||||
|
cfg.ObjectStorage.SecretKey,
|
||||||
|
log,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to initialize MinIO service", "error", err)
|
||||||
|
} else {
|
||||||
|
objectStorageHandler := object_storage.NewHandler(objectStorageService, db, log)
|
||||||
|
objectStorageGroup := protected.Group("/object-storage")
|
||||||
|
objectStorageGroup.Use(requirePermission("storage", "read"))
|
||||||
|
{
|
||||||
|
// Setup endpoints
|
||||||
|
objectStorageGroup.GET("/setup/datasets", objectStorageHandler.GetAvailableDatasets)
|
||||||
|
objectStorageGroup.GET("/setup/current", objectStorageHandler.GetCurrentSetup)
|
||||||
|
objectStorageGroup.POST("/setup", requirePermission("storage", "write"), objectStorageHandler.SetupObjectStorage)
|
||||||
|
objectStorageGroup.PUT("/setup", requirePermission("storage", "write"), objectStorageHandler.UpdateObjectStorage)
|
||||||
|
|
||||||
|
// Bucket endpoints
|
||||||
|
// IMPORTANT: More specific routes must come BEFORE less specific ones
|
||||||
|
objectStorageGroup.GET("/buckets", objectStorageHandler.ListBuckets)
|
||||||
|
objectStorageGroup.GET("/buckets/:name/objects", objectStorageHandler.ListObjects)
|
||||||
|
objectStorageGroup.GET("/buckets/:name", objectStorageHandler.GetBucket)
|
||||||
|
objectStorageGroup.POST("/buckets", requirePermission("storage", "write"), objectStorageHandler.CreateBucket)
|
||||||
|
objectStorageGroup.DELETE("/buckets/:name", requirePermission("storage", "write"), objectStorageHandler.DeleteBucket)
|
||||||
|
// User management routes
|
||||||
|
objectStorageGroup.GET("/users", objectStorageHandler.ListUsers)
|
||||||
|
objectStorageGroup.POST("/users", requirePermission("storage", "write"), objectStorageHandler.CreateUser)
|
||||||
|
objectStorageGroup.DELETE("/users/:access_key", requirePermission("storage", "write"), objectStorageHandler.DeleteUser)
|
||||||
|
// Service account (access key) management routes
|
||||||
|
objectStorageGroup.GET("/service-accounts", objectStorageHandler.ListServiceAccounts)
|
||||||
|
objectStorageGroup.POST("/service-accounts", requirePermission("storage", "write"), objectStorageHandler.CreateServiceAccount)
|
||||||
|
objectStorageGroup.DELETE("/service-accounts/:access_key", requirePermission("storage", "write"), objectStorageHandler.DeleteServiceAccount)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// SCST
|
// SCST
|
||||||
scstHandler := scst.NewHandler(db, log)
|
scstHandler := scst.NewHandler(db, log)
|
||||||
scstGroup := protected.Group("/scst")
|
scstGroup := protected.Group("/scst")
|
||||||
@@ -206,10 +288,12 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
scstGroup.GET("/targets", scstHandler.ListTargets)
|
scstGroup.GET("/targets", scstHandler.ListTargets)
|
||||||
scstGroup.GET("/targets/:id", scstHandler.GetTarget)
|
scstGroup.GET("/targets/:id", scstHandler.GetTarget)
|
||||||
scstGroup.POST("/targets", scstHandler.CreateTarget)
|
scstGroup.POST("/targets", scstHandler.CreateTarget)
|
||||||
scstGroup.POST("/targets/:id/luns", scstHandler.AddLUN)
|
scstGroup.POST("/targets/:id/luns", requirePermission("iscsi", "write"), scstHandler.AddLUN)
|
||||||
|
scstGroup.DELETE("/targets/:id/luns/:lunId", requirePermission("iscsi", "write"), scstHandler.RemoveLUN)
|
||||||
scstGroup.POST("/targets/:id/initiators", scstHandler.AddInitiator)
|
scstGroup.POST("/targets/:id/initiators", scstHandler.AddInitiator)
|
||||||
scstGroup.POST("/targets/:id/enable", scstHandler.EnableTarget)
|
scstGroup.POST("/targets/:id/enable", scstHandler.EnableTarget)
|
||||||
scstGroup.POST("/targets/:id/disable", scstHandler.DisableTarget)
|
scstGroup.POST("/targets/:id/disable", scstHandler.DisableTarget)
|
||||||
|
scstGroup.DELETE("/targets/:id", requirePermission("iscsi", "write"), scstHandler.DeleteTarget)
|
||||||
scstGroup.GET("/initiators", scstHandler.ListAllInitiators)
|
scstGroup.GET("/initiators", scstHandler.ListAllInitiators)
|
||||||
scstGroup.GET("/initiators/:id", scstHandler.GetInitiator)
|
scstGroup.GET("/initiators/:id", scstHandler.GetInitiator)
|
||||||
scstGroup.DELETE("/initiators/:id", scstHandler.RemoveInitiator)
|
scstGroup.DELETE("/initiators/:id", scstHandler.RemoveInitiator)
|
||||||
@@ -223,6 +307,16 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
scstGroup.POST("/portals", scstHandler.CreatePortal)
|
scstGroup.POST("/portals", scstHandler.CreatePortal)
|
||||||
scstGroup.PUT("/portals/:id", scstHandler.UpdatePortal)
|
scstGroup.PUT("/portals/:id", scstHandler.UpdatePortal)
|
||||||
scstGroup.DELETE("/portals/:id", scstHandler.DeletePortal)
|
scstGroup.DELETE("/portals/:id", scstHandler.DeletePortal)
|
||||||
|
// Initiator Groups routes
|
||||||
|
scstGroup.GET("/initiator-groups", scstHandler.ListAllInitiatorGroups)
|
||||||
|
scstGroup.GET("/initiator-groups/:id", scstHandler.GetInitiatorGroup)
|
||||||
|
scstGroup.POST("/initiator-groups", requirePermission("iscsi", "write"), scstHandler.CreateInitiatorGroup)
|
||||||
|
scstGroup.PUT("/initiator-groups/:id", requirePermission("iscsi", "write"), scstHandler.UpdateInitiatorGroup)
|
||||||
|
scstGroup.DELETE("/initiator-groups/:id", requirePermission("iscsi", "write"), scstHandler.DeleteInitiatorGroup)
|
||||||
|
scstGroup.POST("/initiator-groups/:id/initiators", requirePermission("iscsi", "write"), scstHandler.AddInitiatorToGroup)
|
||||||
|
// Config file management
|
||||||
|
scstGroup.GET("/config/file", requirePermission("iscsi", "read"), scstHandler.GetConfigFile)
|
||||||
|
scstGroup.PUT("/config/file", requirePermission("iscsi", "write"), scstHandler.UpdateConfigFile)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Physical Tape Libraries
|
// Physical Tape Libraries
|
||||||
@@ -260,7 +354,18 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
}
|
}
|
||||||
|
|
||||||
// System Management
|
// System Management
|
||||||
|
systemService := system.NewService(log)
|
||||||
systemHandler := system.NewHandler(log, tasks.NewEngine(db, log))
|
systemHandler := system.NewHandler(log, tasks.NewEngine(db, log))
|
||||||
|
// Set service in handler (if handler needs direct access)
|
||||||
|
// Note: Handler already has service via NewHandler, but we need to ensure it's the same instance
|
||||||
|
|
||||||
|
// Start network monitoring with RRD
|
||||||
|
if err := systemService.StartNetworkMonitoring(context.Background()); err != nil {
|
||||||
|
log.Warn("Failed to start network monitoring", "error", err)
|
||||||
|
} else {
|
||||||
|
log.Info("Network monitoring started with RRD")
|
||||||
|
}
|
||||||
|
|
||||||
systemGroup := protected.Group("/system")
|
systemGroup := protected.Group("/system")
|
||||||
systemGroup.Use(requirePermission("system", "read"))
|
systemGroup.Use(requirePermission("system", "read"))
|
||||||
{
|
{
|
||||||
@@ -268,8 +373,15 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
systemGroup.GET("/services/:name", systemHandler.GetServiceStatus)
|
systemGroup.GET("/services/:name", systemHandler.GetServiceStatus)
|
||||||
systemGroup.POST("/services/:name/restart", systemHandler.RestartService)
|
systemGroup.POST("/services/:name/restart", systemHandler.RestartService)
|
||||||
systemGroup.GET("/services/:name/logs", systemHandler.GetServiceLogs)
|
systemGroup.GET("/services/:name/logs", systemHandler.GetServiceLogs)
|
||||||
|
systemGroup.GET("/logs", systemHandler.GetSystemLogs)
|
||||||
|
systemGroup.GET("/network/throughput", systemHandler.GetNetworkThroughput)
|
||||||
systemGroup.POST("/support-bundle", systemHandler.GenerateSupportBundle)
|
systemGroup.POST("/support-bundle", systemHandler.GenerateSupportBundle)
|
||||||
systemGroup.GET("/interfaces", systemHandler.ListNetworkInterfaces)
|
systemGroup.GET("/interfaces", systemHandler.ListNetworkInterfaces)
|
||||||
|
systemGroup.GET("/management-ip", systemHandler.GetManagementIPAddress)
|
||||||
|
systemGroup.PUT("/interfaces/:name", systemHandler.UpdateNetworkInterface)
|
||||||
|
systemGroup.GET("/ntp", systemHandler.GetNTPSettings)
|
||||||
|
systemGroup.POST("/ntp", systemHandler.SaveNTPSettings)
|
||||||
|
systemGroup.POST("/execute", requirePermission("system", "write"), systemHandler.ExecuteCommand)
|
||||||
}
|
}
|
||||||
|
|
||||||
// IAM routes - GetUser can be accessed by user viewing own profile or admin
|
// IAM routes - GetUser can be accessed by user viewing own profile or admin
|
||||||
@@ -330,9 +442,33 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng
|
|||||||
backupGroup := protected.Group("/backup")
|
backupGroup := protected.Group("/backup")
|
||||||
backupGroup.Use(requirePermission("backup", "read"))
|
backupGroup.Use(requirePermission("backup", "read"))
|
||||||
{
|
{
|
||||||
|
backupGroup.GET("/dashboard/stats", backupHandler.GetDashboardStats)
|
||||||
backupGroup.GET("/jobs", backupHandler.ListJobs)
|
backupGroup.GET("/jobs", backupHandler.ListJobs)
|
||||||
backupGroup.GET("/jobs/:id", backupHandler.GetJob)
|
backupGroup.GET("/jobs/:id", backupHandler.GetJob)
|
||||||
backupGroup.POST("/jobs", requirePermission("backup", "write"), backupHandler.CreateJob)
|
backupGroup.POST("/jobs", requirePermission("backup", "write"), backupHandler.CreateJob)
|
||||||
|
backupGroup.GET("/clients", backupHandler.ListClients)
|
||||||
|
backupGroup.GET("/storage/pools", backupHandler.ListStoragePools)
|
||||||
|
backupGroup.POST("/storage/pools", requirePermission("backup", "write"), backupHandler.CreateStoragePool)
|
||||||
|
backupGroup.DELETE("/storage/pools/:id", requirePermission("backup", "write"), backupHandler.DeleteStoragePool)
|
||||||
|
backupGroup.GET("/storage/volumes", backupHandler.ListStorageVolumes)
|
||||||
|
backupGroup.POST("/storage/volumes", requirePermission("backup", "write"), backupHandler.CreateStorageVolume)
|
||||||
|
backupGroup.PUT("/storage/volumes/:id", requirePermission("backup", "write"), backupHandler.UpdateStorageVolume)
|
||||||
|
backupGroup.DELETE("/storage/volumes/:id", requirePermission("backup", "write"), backupHandler.DeleteStorageVolume)
|
||||||
|
backupGroup.GET("/media", backupHandler.ListMedia)
|
||||||
|
backupGroup.GET("/storage/daemons", backupHandler.ListStorageDaemons)
|
||||||
|
backupGroup.POST("/console/execute", requirePermission("backup", "write"), backupHandler.ExecuteBconsoleCommand)
|
||||||
|
}
|
||||||
|
|
||||||
|
baculaHandler := bacula.NewHandler(db, log)
|
||||||
|
baculaGroup := protected.Group("/bacula/clients")
|
||||||
|
baculaGroup.Use(requireRole("bacula-admin"))
|
||||||
|
{
|
||||||
|
baculaGroup.POST("/register", baculaHandler.Register)
|
||||||
|
baculaGroup.POST("/:id/capabilities", baculaHandler.UpdateCapabilities)
|
||||||
|
baculaGroup.GET("/:id/pending-update", baculaHandler.GetPendingUpdate)
|
||||||
|
baculaGroup.POST("/:id/ping", baculaHandler.Ping)
|
||||||
|
baculaGroup.GET("", baculaHandler.ListClients)
|
||||||
|
baculaGroup.GET("/:id", baculaHandler.GetClient)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Monitoring
|
// Monitoring
|
||||||
|
|||||||
@@ -88,11 +88,14 @@ func GetUserGroups(db *database.DB, userID string) ([]string, error) {
|
|||||||
for rows.Next() {
|
for rows.Next() {
|
||||||
var groupName string
|
var groupName string
|
||||||
if err := rows.Scan(&groupName); err != nil {
|
if err := rows.Scan(&groupName); err != nil {
|
||||||
return nil, err
|
return []string{}, err
|
||||||
}
|
}
|
||||||
groups = append(groups, groupName)
|
groups = append(groups, groupName)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if groups == nil {
|
||||||
|
groups = []string{}
|
||||||
|
}
|
||||||
return groups, rows.Err()
|
return groups, rows.Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -69,6 +69,17 @@ func (h *Handler) ListUsers(c *gin.Context) {
|
|||||||
permissions, _ := GetUserPermissions(h.db, u.ID)
|
permissions, _ := GetUserPermissions(h.db, u.ID)
|
||||||
groups, _ := GetUserGroups(h.db, u.ID)
|
groups, _ := GetUserGroups(h.db, u.ID)
|
||||||
|
|
||||||
|
// Ensure arrays are never nil (use empty slice instead)
|
||||||
|
if roles == nil {
|
||||||
|
roles = []string{}
|
||||||
|
}
|
||||||
|
if permissions == nil {
|
||||||
|
permissions = []string{}
|
||||||
|
}
|
||||||
|
if groups == nil {
|
||||||
|
groups = []string{}
|
||||||
|
}
|
||||||
|
|
||||||
users = append(users, map[string]interface{}{
|
users = append(users, map[string]interface{}{
|
||||||
"id": u.ID,
|
"id": u.ID,
|
||||||
"username": u.Username,
|
"username": u.Username,
|
||||||
@@ -138,6 +149,17 @@ func (h *Handler) GetUser(c *gin.Context) {
|
|||||||
permissions, _ := GetUserPermissions(h.db, userID)
|
permissions, _ := GetUserPermissions(h.db, userID)
|
||||||
groups, _ := GetUserGroups(h.db, userID)
|
groups, _ := GetUserGroups(h.db, userID)
|
||||||
|
|
||||||
|
// Ensure arrays are never nil (use empty slice instead)
|
||||||
|
if roles == nil {
|
||||||
|
roles = []string{}
|
||||||
|
}
|
||||||
|
if permissions == nil {
|
||||||
|
permissions = []string{}
|
||||||
|
}
|
||||||
|
if groups == nil {
|
||||||
|
groups = []string{}
|
||||||
|
}
|
||||||
|
|
||||||
c.JSON(http.StatusOK, gin.H{
|
c.JSON(http.StatusOK, gin.H{
|
||||||
"id": user.ID,
|
"id": user.ID,
|
||||||
"username": user.Username,
|
"username": user.Username,
|
||||||
@@ -236,6 +258,8 @@ func (h *Handler) UpdateUser(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Allow update if roles or groups are provided, even if no other fields are updated
|
// Allow update if roles or groups are provided, even if no other fields are updated
|
||||||
|
// Note: req.Roles and req.Groups can be empty arrays ([]), which is different from nil
|
||||||
|
// Empty array means "remove all roles/groups", nil means "don't change roles/groups"
|
||||||
if len(updates) == 1 && req.Roles == nil && req.Groups == nil {
|
if len(updates) == 1 && req.Roles == nil && req.Groups == nil {
|
||||||
c.JSON(http.StatusBadRequest, gin.H{"error": "no fields to update"})
|
c.JSON(http.StatusBadRequest, gin.H{"error": "no fields to update"})
|
||||||
return
|
return
|
||||||
@@ -259,13 +283,14 @@ func (h *Handler) UpdateUser(c *gin.Context) {
|
|||||||
|
|
||||||
// Update roles if provided
|
// Update roles if provided
|
||||||
if req.Roles != nil {
|
if req.Roles != nil {
|
||||||
h.logger.Info("Updating user roles", "user_id", userID, "roles", *req.Roles)
|
h.logger.Info("Updating user roles", "user_id", userID, "requested_roles", *req.Roles)
|
||||||
currentRoles, err := GetUserRoles(h.db, userID)
|
currentRoles, err := GetUserRoles(h.db, userID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.logger.Error("Failed to get current roles for user", "user_id", userID, "error", err)
|
h.logger.Error("Failed to get current roles for user", "user_id", userID, "error", err)
|
||||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to process user roles"})
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to process user roles"})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
h.logger.Info("Current user roles", "user_id", userID, "current_roles", currentRoles)
|
||||||
|
|
||||||
rolesToAdd := []string{}
|
rolesToAdd := []string{}
|
||||||
rolesToRemove := []string{}
|
rolesToRemove := []string{}
|
||||||
@@ -298,8 +323,15 @@ func (h *Handler) UpdateUser(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
h.logger.Info("Roles to add", "user_id", userID, "roles_to_add", rolesToAdd, "count", len(rolesToAdd))
|
||||||
|
h.logger.Info("Roles to remove", "user_id", userID, "roles_to_remove", rolesToRemove, "count", len(rolesToRemove))
|
||||||
|
|
||||||
// Add new roles
|
// Add new roles
|
||||||
|
if len(rolesToAdd) == 0 {
|
||||||
|
h.logger.Info("No roles to add", "user_id", userID)
|
||||||
|
}
|
||||||
for _, roleName := range rolesToAdd {
|
for _, roleName := range rolesToAdd {
|
||||||
|
h.logger.Info("Processing role to add", "user_id", userID, "role_name", roleName)
|
||||||
roleID, err := GetRoleIDByName(h.db, roleName)
|
roleID, err := GetRoleIDByName(h.db, roleName)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if err == sql.ErrNoRows {
|
if err == sql.ErrNoRows {
|
||||||
@@ -311,12 +343,13 @@ func (h *Handler) UpdateUser(c *gin.Context) {
|
|||||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to process roles"})
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to process roles"})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
h.logger.Info("Attempting to add role", "user_id", userID, "role_id", roleID, "role_name", roleName, "assigned_by", currentUser.ID)
|
||||||
if err := AddUserRole(h.db, userID, roleID, currentUser.ID); err != nil {
|
if err := AddUserRole(h.db, userID, roleID, currentUser.ID); err != nil {
|
||||||
h.logger.Error("Failed to add role to user", "user_id", userID, "role_id", roleID, "error", err)
|
h.logger.Error("Failed to add role to user", "user_id", userID, "role_id", roleID, "role_name", roleName, "error", err)
|
||||||
// Don't return early, continue with other roles
|
c.JSON(http.StatusInternalServerError, gin.H{"error": fmt.Sprintf("failed to add role '%s': %v", roleName, err)})
|
||||||
continue
|
return
|
||||||
}
|
}
|
||||||
h.logger.Info("Role added to user", "user_id", userID, "role_name", roleName)
|
h.logger.Info("Role successfully added to user", "user_id", userID, "role_id", roleID, "role_name", roleName)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Remove old roles
|
// Remove old roles
|
||||||
@@ -415,8 +448,48 @@ func (h *Handler) UpdateUser(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
h.logger.Info("User updated", "user_id", userID)
|
// Fetch updated user data to return
|
||||||
c.JSON(http.StatusOK, gin.H{"message": "user updated successfully"})
|
updatedUser, err := GetUserByID(h.db, userID)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to fetch updated user", "user_id", userID, "error", err)
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "user updated successfully"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get updated roles, permissions, and groups
|
||||||
|
updatedRoles, _ := GetUserRoles(h.db, userID)
|
||||||
|
updatedPermissions, _ := GetUserPermissions(h.db, userID)
|
||||||
|
updatedGroups, _ := GetUserGroups(h.db, userID)
|
||||||
|
|
||||||
|
// Ensure arrays are never nil
|
||||||
|
if updatedRoles == nil {
|
||||||
|
updatedRoles = []string{}
|
||||||
|
}
|
||||||
|
if updatedPermissions == nil {
|
||||||
|
updatedPermissions = []string{}
|
||||||
|
}
|
||||||
|
if updatedGroups == nil {
|
||||||
|
updatedGroups = []string{}
|
||||||
|
}
|
||||||
|
|
||||||
|
h.logger.Info("User updated", "user_id", userID, "roles", updatedRoles, "groups", updatedGroups)
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"message": "user updated successfully",
|
||||||
|
"user": gin.H{
|
||||||
|
"id": updatedUser.ID,
|
||||||
|
"username": updatedUser.Username,
|
||||||
|
"email": updatedUser.Email,
|
||||||
|
"full_name": updatedUser.FullName,
|
||||||
|
"is_active": updatedUser.IsActive,
|
||||||
|
"is_system": updatedUser.IsSystem,
|
||||||
|
"roles": updatedRoles,
|
||||||
|
"permissions": updatedPermissions,
|
||||||
|
"groups": updatedGroups,
|
||||||
|
"created_at": updatedUser.CreatedAt,
|
||||||
|
"updated_at": updatedUser.UpdatedAt,
|
||||||
|
"last_login_at": updatedUser.LastLoginAt,
|
||||||
|
},
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// DeleteUser deletes a user
|
// DeleteUser deletes a user
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package iam
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"database/sql"
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/atlasos/calypso/internal/common/database"
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
@@ -90,11 +91,14 @@ func GetUserRoles(db *database.DB, userID string) ([]string, error) {
|
|||||||
for rows.Next() {
|
for rows.Next() {
|
||||||
var role string
|
var role string
|
||||||
if err := rows.Scan(&role); err != nil {
|
if err := rows.Scan(&role); err != nil {
|
||||||
return nil, err
|
return []string{}, err
|
||||||
}
|
}
|
||||||
roles = append(roles, role)
|
roles = append(roles, role)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if roles == nil {
|
||||||
|
roles = []string{}
|
||||||
|
}
|
||||||
return roles, rows.Err()
|
return roles, rows.Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -118,11 +122,14 @@ func GetUserPermissions(db *database.DB, userID string) ([]string, error) {
|
|||||||
for rows.Next() {
|
for rows.Next() {
|
||||||
var perm string
|
var perm string
|
||||||
if err := rows.Scan(&perm); err != nil {
|
if err := rows.Scan(&perm); err != nil {
|
||||||
return nil, err
|
return []string{}, err
|
||||||
}
|
}
|
||||||
permissions = append(permissions, perm)
|
permissions = append(permissions, perm)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if permissions == nil {
|
||||||
|
permissions = []string{}
|
||||||
|
}
|
||||||
return permissions, rows.Err()
|
return permissions, rows.Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -133,8 +140,23 @@ func AddUserRole(db *database.DB, userID, roleID, assignedBy string) error {
|
|||||||
VALUES ($1, $2, $3)
|
VALUES ($1, $2, $3)
|
||||||
ON CONFLICT (user_id, role_id) DO NOTHING
|
ON CONFLICT (user_id, role_id) DO NOTHING
|
||||||
`
|
`
|
||||||
_, err := db.Exec(query, userID, roleID, assignedBy)
|
result, err := db.Exec(query, userID, roleID, assignedBy)
|
||||||
return err
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to insert user role: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if row was actually inserted (not just skipped due to conflict)
|
||||||
|
rowsAffected, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to get rows affected: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rowsAffected == 0 {
|
||||||
|
// Row already exists, this is not an error but we should know about it
|
||||||
|
return nil // ON CONFLICT DO NOTHING means this is expected
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// RemoveUserRole removes a role from a user
|
// RemoveUserRole removes a role from a user
|
||||||
|
|||||||
300
backend/internal/object_storage/handler.go
Normal file
300
backend/internal/object_storage/handler.go
Normal file
@@ -0,0 +1,300 @@
|
|||||||
|
package object_storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Handler handles HTTP requests for object storage
|
||||||
|
type Handler struct {
|
||||||
|
service *Service
|
||||||
|
setupService *SetupService
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewHandler creates a new object storage handler
|
||||||
|
func NewHandler(service *Service, db *database.DB, log *logger.Logger) *Handler {
|
||||||
|
return &Handler{
|
||||||
|
service: service,
|
||||||
|
setupService: NewSetupService(db, log),
|
||||||
|
logger: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListBuckets lists all buckets
|
||||||
|
func (h *Handler) ListBuckets(c *gin.Context) {
|
||||||
|
buckets, err := h.service.ListBuckets(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list buckets", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list buckets: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"buckets": buckets})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetBucket gets bucket information
|
||||||
|
func (h *Handler) GetBucket(c *gin.Context) {
|
||||||
|
bucketName := c.Param("name")
|
||||||
|
|
||||||
|
bucket, err := h.service.GetBucketStats(c.Request.Context(), bucketName)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get bucket", "bucket", bucketName, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get bucket: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, bucket)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateBucketRequest represents a request to create a bucket
|
||||||
|
type CreateBucketRequest struct {
|
||||||
|
Name string `json:"name" binding:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateBucket creates a new bucket
|
||||||
|
func (h *Handler) CreateBucket(c *gin.Context) {
|
||||||
|
var req CreateBucketRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create bucket request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.service.CreateBucket(c.Request.Context(), req.Name); err != nil {
|
||||||
|
h.logger.Error("Failed to create bucket", "bucket", req.Name, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create bucket: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, gin.H{"message": "bucket created successfully", "name": req.Name})
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteBucket deletes a bucket
|
||||||
|
func (h *Handler) DeleteBucket(c *gin.Context) {
|
||||||
|
bucketName := c.Param("name")
|
||||||
|
|
||||||
|
if err := h.service.DeleteBucket(c.Request.Context(), bucketName); err != nil {
|
||||||
|
h.logger.Error("Failed to delete bucket", "bucket", bucketName, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete bucket: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "bucket deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetAvailableDatasets gets all available pools and datasets for object storage setup
|
||||||
|
func (h *Handler) GetAvailableDatasets(c *gin.Context) {
|
||||||
|
datasets, err := h.setupService.GetAvailableDatasets(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get available datasets", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get available datasets: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"pools": datasets})
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetupObjectStorageRequest represents a request to setup object storage
|
||||||
|
type SetupObjectStorageRequest struct {
|
||||||
|
PoolName string `json:"pool_name" binding:"required"`
|
||||||
|
DatasetName string `json:"dataset_name" binding:"required"`
|
||||||
|
CreateNew bool `json:"create_new"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetupObjectStorage configures object storage with a ZFS dataset
|
||||||
|
func (h *Handler) SetupObjectStorage(c *gin.Context) {
|
||||||
|
var req SetupObjectStorageRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid setup request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
setupReq := SetupRequest{
|
||||||
|
PoolName: req.PoolName,
|
||||||
|
DatasetName: req.DatasetName,
|
||||||
|
CreateNew: req.CreateNew,
|
||||||
|
}
|
||||||
|
|
||||||
|
result, err := h.setupService.SetupObjectStorage(c.Request.Context(), setupReq)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to setup object storage", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to setup object storage: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, result)
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetCurrentSetup gets the current object storage configuration
|
||||||
|
func (h *Handler) GetCurrentSetup(c *gin.Context) {
|
||||||
|
setup, err := h.setupService.GetCurrentSetup(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get current setup", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get current setup: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if setup == nil {
|
||||||
|
c.JSON(http.StatusOK, gin.H{"configured": false})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"configured": true, "setup": setup})
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateObjectStorage updates the object storage configuration
|
||||||
|
func (h *Handler) UpdateObjectStorage(c *gin.Context) {
|
||||||
|
var req SetupObjectStorageRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid update request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
setupReq := SetupRequest{
|
||||||
|
PoolName: req.PoolName,
|
||||||
|
DatasetName: req.DatasetName,
|
||||||
|
CreateNew: req.CreateNew,
|
||||||
|
}
|
||||||
|
|
||||||
|
result, err := h.setupService.UpdateObjectStorage(c.Request.Context(), setupReq)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to update object storage", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update object storage: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, result)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListUsers lists all IAM users
|
||||||
|
func (h *Handler) ListUsers(c *gin.Context) {
|
||||||
|
users, err := h.service.ListUsers(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list users", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list users: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"users": users})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateUserRequest represents a request to create a user
|
||||||
|
type CreateUserRequest struct {
|
||||||
|
AccessKey string `json:"access_key" binding:"required"`
|
||||||
|
SecretKey string `json:"secret_key" binding:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateUser creates a new IAM user
|
||||||
|
func (h *Handler) CreateUser(c *gin.Context) {
|
||||||
|
var req CreateUserRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create user request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.service.CreateUser(c.Request.Context(), req.AccessKey, req.SecretKey); err != nil {
|
||||||
|
h.logger.Error("Failed to create user", "access_key", req.AccessKey, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create user: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, gin.H{"message": "user created successfully", "access_key": req.AccessKey})
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteUser deletes an IAM user
|
||||||
|
func (h *Handler) DeleteUser(c *gin.Context) {
|
||||||
|
accessKey := c.Param("access_key")
|
||||||
|
|
||||||
|
if err := h.service.DeleteUser(c.Request.Context(), accessKey); err != nil {
|
||||||
|
h.logger.Error("Failed to delete user", "access_key", accessKey, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete user: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "user deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListServiceAccounts lists all service accounts (access keys)
|
||||||
|
func (h *Handler) ListServiceAccounts(c *gin.Context) {
|
||||||
|
accounts, err := h.service.ListServiceAccounts(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list service accounts", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list service accounts: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"service_accounts": accounts})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateServiceAccountRequest represents a request to create a service account
|
||||||
|
type CreateServiceAccountRequest struct {
|
||||||
|
ParentUser string `json:"parent_user" binding:"required"`
|
||||||
|
Policy string `json:"policy,omitempty"`
|
||||||
|
Expiration *string `json:"expiration,omitempty"` // ISO 8601 format
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateServiceAccount creates a new service account (access key)
|
||||||
|
func (h *Handler) CreateServiceAccount(c *gin.Context) {
|
||||||
|
var req CreateServiceAccountRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create service account request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var expiration *time.Time
|
||||||
|
if req.Expiration != nil {
|
||||||
|
exp, err := time.Parse(time.RFC3339, *req.Expiration)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Invalid expiration format", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid expiration format, use ISO 8601 (RFC3339)"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
expiration = &exp
|
||||||
|
}
|
||||||
|
|
||||||
|
account, err := h.service.CreateServiceAccount(c.Request.Context(), req.ParentUser, req.Policy, expiration)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to create service account", "parent_user", req.ParentUser, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create service account: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, account)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteServiceAccount deletes a service account
|
||||||
|
func (h *Handler) DeleteServiceAccount(c *gin.Context) {
|
||||||
|
accessKey := c.Param("access_key")
|
||||||
|
|
||||||
|
if err := h.service.DeleteServiceAccount(c.Request.Context(), accessKey); err != nil {
|
||||||
|
h.logger.Error("Failed to delete service account", "access_key", accessKey, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete service account: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "service account deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListObjects lists objects in a bucket
|
||||||
|
func (h *Handler) ListObjects(c *gin.Context) {
|
||||||
|
bucketName := c.Param("name") // Changed from "bucket" to "name" to match route
|
||||||
|
prefix := c.DefaultQuery("prefix", "") // Optional prefix (folder path)
|
||||||
|
|
||||||
|
objects, err := h.service.ListObjects(c.Request.Context(), bucketName, prefix)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list objects", "bucket", bucketName, "prefix", prefix, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list objects: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"objects": objects})
|
||||||
|
}
|
||||||
373
backend/internal/object_storage/service.go
Normal file
373
backend/internal/object_storage/service.go
Normal file
@@ -0,0 +1,373 @@
|
|||||||
|
package object_storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
madmin "github.com/minio/madmin-go/v3"
|
||||||
|
"github.com/minio/minio-go/v7"
|
||||||
|
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Service handles MinIO object storage operations
|
||||||
|
type Service struct {
|
||||||
|
client *minio.Client
|
||||||
|
adminClient *madmin.AdminClient
|
||||||
|
logger *logger.Logger
|
||||||
|
endpoint string
|
||||||
|
accessKey string
|
||||||
|
secretKey string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewService creates a new MinIO service
|
||||||
|
func NewService(endpoint, accessKey, secretKey string, log *logger.Logger) (*Service, error) {
|
||||||
|
// Create MinIO client
|
||||||
|
minioClient, err := minio.New(endpoint, &minio.Options{
|
||||||
|
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
|
||||||
|
Secure: false, // Set to true if using HTTPS
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create MinIO client: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create MinIO Admin client
|
||||||
|
adminClient, err := madmin.New(endpoint, accessKey, secretKey, false)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create MinIO admin client: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &Service{
|
||||||
|
client: minioClient,
|
||||||
|
adminClient: adminClient,
|
||||||
|
logger: log,
|
||||||
|
endpoint: endpoint,
|
||||||
|
accessKey: accessKey,
|
||||||
|
secretKey: secretKey,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bucket represents a MinIO bucket
|
||||||
|
type Bucket struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
CreationDate time.Time `json:"creation_date"`
|
||||||
|
Size int64 `json:"size"` // Total size in bytes
|
||||||
|
Objects int64 `json:"objects"` // Number of objects
|
||||||
|
AccessPolicy string `json:"access_policy"` // private, public-read, public-read-write
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListBuckets lists all buckets in MinIO
|
||||||
|
func (s *Service) ListBuckets(ctx context.Context) ([]*Bucket, error) {
|
||||||
|
buckets, err := s.client.ListBuckets(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to list buckets: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
result := make([]*Bucket, 0, len(buckets))
|
||||||
|
for _, bucket := range buckets {
|
||||||
|
bucketInfo, err := s.getBucketInfo(ctx, bucket.Name)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to get bucket info", "bucket", bucket.Name, "error", err)
|
||||||
|
// Continue with basic info
|
||||||
|
result = append(result, &Bucket{
|
||||||
|
Name: bucket.Name,
|
||||||
|
CreationDate: bucket.CreationDate,
|
||||||
|
Size: 0,
|
||||||
|
Objects: 0,
|
||||||
|
AccessPolicy: "private",
|
||||||
|
})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
result = append(result, bucketInfo)
|
||||||
|
}
|
||||||
|
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getBucketInfo gets detailed information about a bucket
|
||||||
|
func (s *Service) getBucketInfo(ctx context.Context, bucketName string) (*Bucket, error) {
|
||||||
|
// Get bucket creation date
|
||||||
|
buckets, err := s.client.ListBuckets(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
var creationDate time.Time
|
||||||
|
for _, b := range buckets {
|
||||||
|
if b.Name == bucketName {
|
||||||
|
creationDate = b.CreationDate
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get bucket size and object count by listing objects
|
||||||
|
var size int64
|
||||||
|
var objects int64
|
||||||
|
|
||||||
|
// List objects in bucket to calculate size and count
|
||||||
|
objectCh := s.client.ListObjects(ctx, bucketName, minio.ListObjectsOptions{
|
||||||
|
Recursive: true,
|
||||||
|
})
|
||||||
|
|
||||||
|
for object := range objectCh {
|
||||||
|
if object.Err != nil {
|
||||||
|
s.logger.Warn("Error listing object", "bucket", bucketName, "error", object.Err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
objects++
|
||||||
|
size += object.Size
|
||||||
|
}
|
||||||
|
|
||||||
|
return &Bucket{
|
||||||
|
Name: bucketName,
|
||||||
|
CreationDate: creationDate,
|
||||||
|
Size: size,
|
||||||
|
Objects: objects,
|
||||||
|
AccessPolicy: s.getBucketPolicy(ctx, bucketName),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getBucketPolicy gets the access policy for a bucket
|
||||||
|
func (s *Service) getBucketPolicy(ctx context.Context, bucketName string) string {
|
||||||
|
policy, err := s.client.GetBucketPolicy(ctx, bucketName)
|
||||||
|
if err != nil {
|
||||||
|
return "private"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse policy JSON to determine access type
|
||||||
|
// For simplicity, check if policy allows public read
|
||||||
|
if policy != "" {
|
||||||
|
// Check if policy contains public read access
|
||||||
|
if strings.Contains(policy, "s3:GetObject") && strings.Contains(policy, "Principal") && strings.Contains(policy, "*") {
|
||||||
|
if strings.Contains(policy, "s3:PutObject") {
|
||||||
|
return "public-read-write"
|
||||||
|
}
|
||||||
|
return "public-read"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "private"
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateBucket creates a new bucket
|
||||||
|
func (s *Service) CreateBucket(ctx context.Context, bucketName string) error {
|
||||||
|
err := s.client.MakeBucket(ctx, bucketName, minio.MakeBucketOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to create bucket: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteBucket deletes a bucket
|
||||||
|
func (s *Service) DeleteBucket(ctx context.Context, bucketName string) error {
|
||||||
|
err := s.client.RemoveBucket(ctx, bucketName)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete bucket: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetBucketStats gets statistics for a bucket
|
||||||
|
func (s *Service) GetBucketStats(ctx context.Context, bucketName string) (*Bucket, error) {
|
||||||
|
return s.getBucketInfo(ctx, bucketName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Object represents a MinIO object (file or folder)
|
||||||
|
type Object struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Key string `json:"key"` // Full path/key
|
||||||
|
Size int64 `json:"size"` // Size in bytes (0 for folders)
|
||||||
|
LastModified time.Time `json:"last_modified"`
|
||||||
|
IsDir bool `json:"is_dir"` // True if this is a folder/prefix
|
||||||
|
ETag string `json:"etag,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListObjects lists objects in a bucket with optional prefix (folder path)
|
||||||
|
func (s *Service) ListObjects(ctx context.Context, bucketName, prefix string) ([]*Object, error) {
|
||||||
|
// Ensure prefix ends with / if it's not empty and doesn't already end with /
|
||||||
|
if prefix != "" && !strings.HasSuffix(prefix, "/") {
|
||||||
|
prefix += "/"
|
||||||
|
}
|
||||||
|
|
||||||
|
// List objects with prefix
|
||||||
|
objectCh := s.client.ListObjects(ctx, bucketName, minio.ListObjectsOptions{
|
||||||
|
Prefix: prefix,
|
||||||
|
Recursive: false, // Don't recurse, show folders as separate items
|
||||||
|
})
|
||||||
|
|
||||||
|
var objects []*Object
|
||||||
|
seenFolders := make(map[string]bool)
|
||||||
|
|
||||||
|
for object := range objectCh {
|
||||||
|
if object.Err != nil {
|
||||||
|
s.logger.Warn("Error listing object", "bucket", bucketName, "prefix", prefix, "error", object.Err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract folder name from object key
|
||||||
|
relativeKey := strings.TrimPrefix(object.Key, prefix)
|
||||||
|
|
||||||
|
// Check if this is a folder (has / in relative path)
|
||||||
|
if strings.Contains(relativeKey, "/") {
|
||||||
|
// Extract folder name (first part before /)
|
||||||
|
folderName := strings.Split(relativeKey, "/")[0]
|
||||||
|
folderKey := prefix + folderName + "/"
|
||||||
|
|
||||||
|
// Only add folder once
|
||||||
|
if !seenFolders[folderKey] {
|
||||||
|
seenFolders[folderKey] = true
|
||||||
|
objects = append(objects, &Object{
|
||||||
|
Name: folderName,
|
||||||
|
Key: folderKey,
|
||||||
|
Size: 0,
|
||||||
|
LastModified: time.Time{},
|
||||||
|
IsDir: true,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// This is a file in the current directory
|
||||||
|
objects = append(objects, &Object{
|
||||||
|
Name: relativeKey,
|
||||||
|
Key: object.Key,
|
||||||
|
Size: object.Size,
|
||||||
|
LastModified: object.LastModified,
|
||||||
|
IsDir: false,
|
||||||
|
ETag: object.ETag,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort: folders first, then files, both alphabetically
|
||||||
|
sort.Slice(objects, func(i, j int) bool {
|
||||||
|
if objects[i].IsDir != objects[j].IsDir {
|
||||||
|
return objects[i].IsDir // Folders first
|
||||||
|
}
|
||||||
|
return objects[i].Name < objects[j].Name
|
||||||
|
})
|
||||||
|
|
||||||
|
return objects, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// User represents a MinIO IAM user
|
||||||
|
type User struct {
|
||||||
|
AccessKey string `json:"access_key"`
|
||||||
|
Status string `json:"status"` // "enabled" or "disabled"
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListUsers lists all IAM users in MinIO
|
||||||
|
func (s *Service) ListUsers(ctx context.Context) ([]*User, error) {
|
||||||
|
users, err := s.adminClient.ListUsers(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to list users: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
result := make([]*User, 0, len(users))
|
||||||
|
for accessKey, userInfo := range users {
|
||||||
|
status := "enabled"
|
||||||
|
if userInfo.Status == madmin.AccountDisabled {
|
||||||
|
status = "disabled"
|
||||||
|
}
|
||||||
|
|
||||||
|
// MinIO doesn't provide creation date, use current time
|
||||||
|
result = append(result, &User{
|
||||||
|
AccessKey: accessKey,
|
||||||
|
Status: status,
|
||||||
|
CreatedAt: time.Now(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateUser creates a new IAM user in MinIO
|
||||||
|
func (s *Service) CreateUser(ctx context.Context, accessKey, secretKey string) error {
|
||||||
|
err := s.adminClient.AddUser(ctx, accessKey, secretKey)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to create user: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteUser deletes an IAM user from MinIO
|
||||||
|
func (s *Service) DeleteUser(ctx context.Context, accessKey string) error {
|
||||||
|
err := s.adminClient.RemoveUser(ctx, accessKey)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete user: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ServiceAccount represents a MinIO service account (access key)
|
||||||
|
type ServiceAccount struct {
|
||||||
|
AccessKey string `json:"access_key"`
|
||||||
|
SecretKey string `json:"secret_key,omitempty"` // Only returned on creation
|
||||||
|
ParentUser string `json:"parent_user"`
|
||||||
|
Expiration time.Time `json:"expiration,omitempty"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListServiceAccounts lists all service accounts in MinIO
|
||||||
|
func (s *Service) ListServiceAccounts(ctx context.Context) ([]*ServiceAccount, error) {
|
||||||
|
accounts, err := s.adminClient.ListServiceAccounts(ctx, "")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to list service accounts: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
result := make([]*ServiceAccount, 0, len(accounts.Accounts))
|
||||||
|
for _, account := range accounts.Accounts {
|
||||||
|
var expiration time.Time
|
||||||
|
if account.Expiration != nil {
|
||||||
|
expiration = *account.Expiration
|
||||||
|
}
|
||||||
|
|
||||||
|
result = append(result, &ServiceAccount{
|
||||||
|
AccessKey: account.AccessKey,
|
||||||
|
ParentUser: account.ParentUser,
|
||||||
|
Expiration: expiration,
|
||||||
|
CreatedAt: time.Now(), // MinIO doesn't provide creation date
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateServiceAccount creates a new service account (access key) in MinIO
|
||||||
|
func (s *Service) CreateServiceAccount(ctx context.Context, parentUser string, policy string, expiration *time.Time) (*ServiceAccount, error) {
|
||||||
|
opts := madmin.AddServiceAccountReq{
|
||||||
|
TargetUser: parentUser,
|
||||||
|
}
|
||||||
|
if policy != "" {
|
||||||
|
opts.Policy = json.RawMessage(policy)
|
||||||
|
}
|
||||||
|
if expiration != nil {
|
||||||
|
opts.Expiration = expiration
|
||||||
|
}
|
||||||
|
|
||||||
|
creds, err := s.adminClient.AddServiceAccount(ctx, opts)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create service account: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &ServiceAccount{
|
||||||
|
AccessKey: creds.AccessKey,
|
||||||
|
SecretKey: creds.SecretKey,
|
||||||
|
ParentUser: parentUser,
|
||||||
|
Expiration: creds.Expiration,
|
||||||
|
CreatedAt: time.Now(),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteServiceAccount deletes a service account from MinIO
|
||||||
|
func (s *Service) DeleteServiceAccount(ctx context.Context, accessKey string) error {
|
||||||
|
err := s.adminClient.DeleteServiceAccount(ctx, accessKey)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete service account: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
511
backend/internal/object_storage/setup.go
Normal file
511
backend/internal/object_storage/setup.go
Normal file
@@ -0,0 +1,511 @@
|
|||||||
|
package object_storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SetupService handles object storage setup operations
|
||||||
|
type SetupService struct {
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSetupService creates a new setup service
|
||||||
|
func NewSetupService(db *database.DB, log *logger.Logger) *SetupService {
|
||||||
|
return &SetupService{
|
||||||
|
db: db,
|
||||||
|
logger: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// PoolDatasetInfo represents a pool with its datasets
|
||||||
|
type PoolDatasetInfo struct {
|
||||||
|
PoolID string `json:"pool_id"`
|
||||||
|
PoolName string `json:"pool_name"`
|
||||||
|
Datasets []DatasetInfo `json:"datasets"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// DatasetInfo represents a dataset that can be used for object storage
|
||||||
|
type DatasetInfo struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
FullName string `json:"full_name"` // pool/dataset
|
||||||
|
MountPoint string `json:"mount_point"`
|
||||||
|
Type string `json:"type"`
|
||||||
|
UsedBytes int64 `json:"used_bytes"`
|
||||||
|
AvailableBytes int64 `json:"available_bytes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetAvailableDatasets returns all pools with their datasets that can be used for object storage
|
||||||
|
func (s *SetupService) GetAvailableDatasets(ctx context.Context) ([]PoolDatasetInfo, error) {
|
||||||
|
// Get all pools
|
||||||
|
poolsQuery := `
|
||||||
|
SELECT id, name
|
||||||
|
FROM zfs_pools
|
||||||
|
WHERE is_active = true
|
||||||
|
ORDER BY name
|
||||||
|
`
|
||||||
|
|
||||||
|
rows, err := s.db.QueryContext(ctx, poolsQuery)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to query pools: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var pools []PoolDatasetInfo
|
||||||
|
for rows.Next() {
|
||||||
|
var pool PoolDatasetInfo
|
||||||
|
if err := rows.Scan(&pool.PoolID, &pool.PoolName); err != nil {
|
||||||
|
s.logger.Warn("Failed to scan pool", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get datasets for this pool
|
||||||
|
datasetsQuery := `
|
||||||
|
SELECT id, name, type, mount_point, used_bytes, available_bytes
|
||||||
|
FROM zfs_datasets
|
||||||
|
WHERE pool_name = $1 AND type = 'filesystem'
|
||||||
|
ORDER BY name
|
||||||
|
`
|
||||||
|
|
||||||
|
datasetRows, err := s.db.QueryContext(ctx, datasetsQuery, pool.PoolName)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to query datasets", "pool", pool.PoolName, "error", err)
|
||||||
|
pool.Datasets = []DatasetInfo{}
|
||||||
|
pools = append(pools, pool)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
var datasets []DatasetInfo
|
||||||
|
for datasetRows.Next() {
|
||||||
|
var ds DatasetInfo
|
||||||
|
var mountPoint sql.NullString
|
||||||
|
|
||||||
|
if err := datasetRows.Scan(&ds.ID, &ds.Name, &ds.Type, &mountPoint, &ds.UsedBytes, &ds.AvailableBytes); err != nil {
|
||||||
|
s.logger.Warn("Failed to scan dataset", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
ds.FullName = fmt.Sprintf("%s/%s", pool.PoolName, ds.Name)
|
||||||
|
if mountPoint.Valid {
|
||||||
|
ds.MountPoint = mountPoint.String
|
||||||
|
} else {
|
||||||
|
ds.MountPoint = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
datasets = append(datasets, ds)
|
||||||
|
}
|
||||||
|
datasetRows.Close()
|
||||||
|
|
||||||
|
pool.Datasets = datasets
|
||||||
|
pools = append(pools, pool)
|
||||||
|
}
|
||||||
|
|
||||||
|
return pools, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetupRequest represents a request to setup object storage
|
||||||
|
type SetupRequest struct {
|
||||||
|
PoolName string `json:"pool_name" binding:"required"`
|
||||||
|
DatasetName string `json:"dataset_name" binding:"required"`
|
||||||
|
CreateNew bool `json:"create_new"` // If true, create new dataset instead of using existing
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetupResponse represents the response after setup
|
||||||
|
type SetupResponse struct {
|
||||||
|
DatasetPath string `json:"dataset_path"`
|
||||||
|
MountPoint string `json:"mount_point"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetupObjectStorage configures MinIO to use a specific ZFS dataset
|
||||||
|
func (s *SetupService) SetupObjectStorage(ctx context.Context, req SetupRequest) (*SetupResponse, error) {
|
||||||
|
var datasetPath, mountPoint string
|
||||||
|
|
||||||
|
// Normalize dataset name - if it already contains pool name, use it as-is
|
||||||
|
var fullDatasetName string
|
||||||
|
if strings.HasPrefix(req.DatasetName, req.PoolName+"/") {
|
||||||
|
// Dataset name already includes pool name (e.g., "pool/dataset")
|
||||||
|
fullDatasetName = req.DatasetName
|
||||||
|
} else {
|
||||||
|
// Dataset name is just the name (e.g., "dataset"), combine with pool
|
||||||
|
fullDatasetName = fmt.Sprintf("%s/%s", req.PoolName, req.DatasetName)
|
||||||
|
}
|
||||||
|
|
||||||
|
if req.CreateNew {
|
||||||
|
// Create new dataset for object storage
|
||||||
|
|
||||||
|
// Check if dataset already exists
|
||||||
|
checkCmd := exec.CommandContext(ctx, "sudo", "zfs", "list", "-H", "-o", "name", fullDatasetName)
|
||||||
|
if err := checkCmd.Run(); err == nil {
|
||||||
|
return nil, fmt.Errorf("dataset %s already exists", fullDatasetName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create dataset
|
||||||
|
createCmd := exec.CommandContext(ctx, "sudo", "zfs", "create", fullDatasetName)
|
||||||
|
if output, err := createCmd.CombinedOutput(); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create dataset: %s - %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get mount point
|
||||||
|
getMountCmd := exec.CommandContext(ctx, "sudo", "zfs", "get", "-H", "-o", "value", "mountpoint", fullDatasetName)
|
||||||
|
mountOutput, err := getMountCmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to get mount point: %w", err)
|
||||||
|
}
|
||||||
|
mountPoint = strings.TrimSpace(string(mountOutput))
|
||||||
|
|
||||||
|
datasetPath = fullDatasetName
|
||||||
|
s.logger.Info("Created new dataset for object storage", "dataset", fullDatasetName, "mount_point", mountPoint)
|
||||||
|
} else {
|
||||||
|
// Use existing dataset
|
||||||
|
// fullDatasetName already set above
|
||||||
|
|
||||||
|
// Verify dataset exists
|
||||||
|
checkCmd := exec.CommandContext(ctx, "sudo", "zfs", "list", "-H", "-o", "name", fullDatasetName)
|
||||||
|
if err := checkCmd.Run(); err != nil {
|
||||||
|
return nil, fmt.Errorf("dataset %s does not exist", fullDatasetName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get mount point
|
||||||
|
getMountCmd := exec.CommandContext(ctx, "sudo", "zfs", "get", "-H", "-o", "value", "mountpoint", fullDatasetName)
|
||||||
|
mountOutput, err := getMountCmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to get mount point: %w", err)
|
||||||
|
}
|
||||||
|
mountPoint = strings.TrimSpace(string(mountOutput))
|
||||||
|
|
||||||
|
datasetPath = fullDatasetName
|
||||||
|
s.logger.Info("Using existing dataset for object storage", "dataset", fullDatasetName, "mount_point", mountPoint)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure mount point directory exists
|
||||||
|
if mountPoint != "none" && mountPoint != "" {
|
||||||
|
if err := os.MkdirAll(mountPoint, 0755); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create mount point directory: %w", err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// If no mount point, use default path
|
||||||
|
mountPoint = filepath.Join("/opt/calypso/data/pool", req.PoolName, req.DatasetName)
|
||||||
|
if err := os.MkdirAll(mountPoint, 0755); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create default directory: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update MinIO configuration to use the selected dataset
|
||||||
|
if err := s.updateMinIOConfig(ctx, mountPoint); err != nil {
|
||||||
|
s.logger.Warn("Failed to update MinIO configuration", "error", err)
|
||||||
|
// Continue anyway, configuration is saved to database
|
||||||
|
}
|
||||||
|
|
||||||
|
// Save configuration to database
|
||||||
|
_, err := s.db.ExecContext(ctx, `
|
||||||
|
INSERT INTO object_storage_config (dataset_path, mount_point, pool_name, dataset_name, created_at, updated_at)
|
||||||
|
VALUES ($1, $2, $3, $4, NOW(), NOW())
|
||||||
|
ON CONFLICT (id) DO UPDATE
|
||||||
|
SET dataset_path = $1, mount_point = $2, pool_name = $3, dataset_name = $4, updated_at = NOW()
|
||||||
|
`, datasetPath, mountPoint, req.PoolName, req.DatasetName)
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
// If table doesn't exist, just log warning
|
||||||
|
s.logger.Warn("Failed to save configuration to database (table may not exist)", "error", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &SetupResponse{
|
||||||
|
DatasetPath: datasetPath,
|
||||||
|
MountPoint: mountPoint,
|
||||||
|
Message: fmt.Sprintf("Object storage configured to use dataset %s at %s. MinIO service needs to be restarted to use the new dataset.", datasetPath, mountPoint),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetCurrentSetup returns the current object storage configuration
|
||||||
|
func (s *SetupService) GetCurrentSetup(ctx context.Context) (*SetupResponse, error) {
|
||||||
|
// Check if table exists first
|
||||||
|
var tableExists bool
|
||||||
|
checkQuery := `
|
||||||
|
SELECT EXISTS (
|
||||||
|
SELECT FROM information_schema.tables
|
||||||
|
WHERE table_schema = 'public'
|
||||||
|
AND table_name = 'object_storage_config'
|
||||||
|
)
|
||||||
|
`
|
||||||
|
err := s.db.QueryRowContext(ctx, checkQuery).Scan(&tableExists)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to check if object_storage_config table exists", "error", err)
|
||||||
|
return nil, nil // Return nil if can't check
|
||||||
|
}
|
||||||
|
|
||||||
|
if !tableExists {
|
||||||
|
s.logger.Debug("object_storage_config table does not exist")
|
||||||
|
return nil, nil // No table, no configuration
|
||||||
|
}
|
||||||
|
|
||||||
|
query := `
|
||||||
|
SELECT dataset_path, mount_point, pool_name, dataset_name
|
||||||
|
FROM object_storage_config
|
||||||
|
ORDER BY updated_at DESC
|
||||||
|
LIMIT 1
|
||||||
|
`
|
||||||
|
|
||||||
|
var resp SetupResponse
|
||||||
|
var poolName, datasetName string
|
||||||
|
err = s.db.QueryRowContext(ctx, query).Scan(&resp.DatasetPath, &resp.MountPoint, &poolName, &datasetName)
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
s.logger.Debug("No configuration found in database")
|
||||||
|
return nil, nil // No configuration found
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
// Check if error is due to table not existing or permission denied
|
||||||
|
errStr := err.Error()
|
||||||
|
if strings.Contains(errStr, "does not exist") || strings.Contains(errStr, "permission denied") {
|
||||||
|
s.logger.Debug("Table does not exist or permission denied, returning nil", "error", errStr)
|
||||||
|
return nil, nil // Return nil instead of error
|
||||||
|
}
|
||||||
|
s.logger.Error("Failed to scan current setup", "error", err)
|
||||||
|
return nil, fmt.Errorf("failed to get current setup: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Debug("Found current setup", "dataset_path", resp.DatasetPath, "mount_point", resp.MountPoint, "pool", poolName, "dataset", datasetName)
|
||||||
|
// Use dataset_path directly since it already contains the full path
|
||||||
|
resp.Message = fmt.Sprintf("Using dataset %s at %s", resp.DatasetPath, resp.MountPoint)
|
||||||
|
return &resp, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateObjectStorage updates the object storage configuration to use a different dataset
|
||||||
|
// This will update the configuration but won't migrate existing data
|
||||||
|
func (s *SetupService) UpdateObjectStorage(ctx context.Context, req SetupRequest) (*SetupResponse, error) {
|
||||||
|
// First check if there's existing configuration
|
||||||
|
currentSetup, err := s.GetCurrentSetup(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to check current setup: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if currentSetup == nil {
|
||||||
|
// No existing setup, just do normal setup
|
||||||
|
return s.SetupObjectStorage(ctx, req)
|
||||||
|
}
|
||||||
|
|
||||||
|
// There's existing setup, proceed with update
|
||||||
|
var datasetPath, mountPoint string
|
||||||
|
|
||||||
|
// Normalize dataset name - if it already contains pool name, use it as-is
|
||||||
|
var fullDatasetName string
|
||||||
|
if strings.HasPrefix(req.DatasetName, req.PoolName+"/") {
|
||||||
|
// Dataset name already includes pool name (e.g., "pool/dataset")
|
||||||
|
fullDatasetName = req.DatasetName
|
||||||
|
} else {
|
||||||
|
// Dataset name is just the name (e.g., "dataset"), combine with pool
|
||||||
|
fullDatasetName = fmt.Sprintf("%s/%s", req.PoolName, req.DatasetName)
|
||||||
|
}
|
||||||
|
|
||||||
|
if req.CreateNew {
|
||||||
|
// Create new dataset for object storage
|
||||||
|
|
||||||
|
// Check if dataset already exists
|
||||||
|
checkCmd := exec.CommandContext(ctx, "sudo", "zfs", "list", "-H", "-o", "name", fullDatasetName)
|
||||||
|
if err := checkCmd.Run(); err == nil {
|
||||||
|
return nil, fmt.Errorf("dataset %s already exists", fullDatasetName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create dataset
|
||||||
|
createCmd := exec.CommandContext(ctx, "sudo", "zfs", "create", fullDatasetName)
|
||||||
|
if output, err := createCmd.CombinedOutput(); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create dataset: %s - %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get mount point
|
||||||
|
getMountCmd := exec.CommandContext(ctx, "sudo", "zfs", "get", "-H", "-o", "value", "mountpoint", fullDatasetName)
|
||||||
|
mountOutput, err := getMountCmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to get mount point: %w", err)
|
||||||
|
}
|
||||||
|
mountPoint = strings.TrimSpace(string(mountOutput))
|
||||||
|
|
||||||
|
datasetPath = fullDatasetName
|
||||||
|
s.logger.Info("Created new dataset for object storage update", "dataset", fullDatasetName, "mount_point", mountPoint)
|
||||||
|
} else {
|
||||||
|
// Use existing dataset
|
||||||
|
// fullDatasetName already set above
|
||||||
|
|
||||||
|
// Verify dataset exists
|
||||||
|
checkCmd := exec.CommandContext(ctx, "sudo", "zfs", "list", "-H", "-o", "name", fullDatasetName)
|
||||||
|
if err := checkCmd.Run(); err != nil {
|
||||||
|
return nil, fmt.Errorf("dataset %s does not exist", fullDatasetName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get mount point
|
||||||
|
getMountCmd := exec.CommandContext(ctx, "sudo", "zfs", "get", "-H", "-o", "value", "mountpoint", fullDatasetName)
|
||||||
|
mountOutput, err := getMountCmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to get mount point: %w", err)
|
||||||
|
}
|
||||||
|
mountPoint = strings.TrimSpace(string(mountOutput))
|
||||||
|
|
||||||
|
datasetPath = fullDatasetName
|
||||||
|
s.logger.Info("Using existing dataset for object storage update", "dataset", fullDatasetName, "mount_point", mountPoint)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure mount point directory exists
|
||||||
|
if mountPoint != "none" && mountPoint != "" {
|
||||||
|
if err := os.MkdirAll(mountPoint, 0755); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create mount point directory: %w", err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// If no mount point, use default path
|
||||||
|
mountPoint = filepath.Join("/opt/calypso/data/pool", req.PoolName, req.DatasetName)
|
||||||
|
if err := os.MkdirAll(mountPoint, 0755); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create default directory: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update configuration in database
|
||||||
|
_, err = s.db.ExecContext(ctx, `
|
||||||
|
UPDATE object_storage_config
|
||||||
|
SET dataset_path = $1, mount_point = $2, pool_name = $3, dataset_name = $4, updated_at = NOW()
|
||||||
|
WHERE id = (SELECT id FROM object_storage_config ORDER BY updated_at DESC LIMIT 1)
|
||||||
|
`, datasetPath, mountPoint, req.PoolName, req.DatasetName)
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
// If update fails, try insert
|
||||||
|
_, err = s.db.ExecContext(ctx, `
|
||||||
|
INSERT INTO object_storage_config (dataset_path, mount_point, pool_name, dataset_name, created_at, updated_at)
|
||||||
|
VALUES ($1, $2, $3, $4, NOW(), NOW())
|
||||||
|
ON CONFLICT (dataset_path) DO UPDATE
|
||||||
|
SET mount_point = $2, pool_name = $3, dataset_name = $4, updated_at = NOW()
|
||||||
|
`, datasetPath, mountPoint, req.PoolName, req.DatasetName)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to update configuration in database", "error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update MinIO configuration to use the selected dataset
|
||||||
|
if err := s.updateMinIOConfig(ctx, mountPoint); err != nil {
|
||||||
|
s.logger.Warn("Failed to update MinIO configuration", "error", err)
|
||||||
|
// Continue anyway, configuration is saved to database
|
||||||
|
} else {
|
||||||
|
// Restart MinIO service to apply new configuration
|
||||||
|
if err := s.restartMinIOService(ctx); err != nil {
|
||||||
|
s.logger.Warn("Failed to restart MinIO service", "error", err)
|
||||||
|
// Continue anyway, user can restart manually
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return &SetupResponse{
|
||||||
|
DatasetPath: datasetPath,
|
||||||
|
MountPoint: mountPoint,
|
||||||
|
Message: fmt.Sprintf("Object storage updated to use dataset %s at %s. Note: Existing data in previous dataset (%s) is not migrated automatically. MinIO service has been restarted.", datasetPath, mountPoint, currentSetup.DatasetPath),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// updateMinIOConfig updates MinIO configuration file to use dataset mount point directly
|
||||||
|
// Note: MinIO erasure coding requires direct directory paths, not symlinks
|
||||||
|
func (s *SetupService) updateMinIOConfig(ctx context.Context, datasetMountPoint string) error {
|
||||||
|
configFile := "/opt/calypso/conf/minio/minio.conf"
|
||||||
|
|
||||||
|
// Ensure dataset mount point directory exists and has correct ownership
|
||||||
|
if err := os.MkdirAll(datasetMountPoint, 0755); err != nil {
|
||||||
|
return fmt.Errorf("failed to create dataset mount point directory: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set ownership to minio-user so MinIO can write to it
|
||||||
|
if err := exec.CommandContext(ctx, "sudo", "chown", "-R", "minio-user:minio-user", datasetMountPoint).Run(); err != nil {
|
||||||
|
s.logger.Warn("Failed to set ownership on dataset mount point", "path", datasetMountPoint, "error", err)
|
||||||
|
// Continue anyway, might already have correct ownership
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set permissions
|
||||||
|
if err := exec.CommandContext(ctx, "sudo", "chmod", "755", datasetMountPoint).Run(); err != nil {
|
||||||
|
s.logger.Warn("Failed to set permissions on dataset mount point", "path", datasetMountPoint, "error", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Prepared dataset mount point for MinIO", "path", datasetMountPoint)
|
||||||
|
|
||||||
|
// Read current config file
|
||||||
|
configContent, err := os.ReadFile(configFile)
|
||||||
|
if err != nil {
|
||||||
|
// If file doesn't exist, create it
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
configContent = []byte(fmt.Sprintf("MINIO_ROOT_USER=admin\nMINIO_ROOT_PASSWORD=HqBX1IINqFynkWFa\nMINIO_VOLUMES=%s\n", datasetMountPoint))
|
||||||
|
} else {
|
||||||
|
return fmt.Errorf("failed to read MinIO config file: %w", err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Update MINIO_VOLUMES in config
|
||||||
|
lines := strings.Split(string(configContent), "\n")
|
||||||
|
updated := false
|
||||||
|
for i, line := range lines {
|
||||||
|
if strings.HasPrefix(strings.TrimSpace(line), "MINIO_VOLUMES=") {
|
||||||
|
lines[i] = fmt.Sprintf("MINIO_VOLUMES=%s", datasetMountPoint)
|
||||||
|
updated = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !updated {
|
||||||
|
// Add MINIO_VOLUMES if not found
|
||||||
|
lines = append(lines, fmt.Sprintf("MINIO_VOLUMES=%s", datasetMountPoint))
|
||||||
|
}
|
||||||
|
configContent = []byte(strings.Join(lines, "\n"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write updated config using sudo
|
||||||
|
// Write temp file to a location we can write to
|
||||||
|
userTempFile := fmt.Sprintf("/tmp/minio.conf.%d.tmp", os.Getpid())
|
||||||
|
if err := os.WriteFile(userTempFile, configContent, 0644); err != nil {
|
||||||
|
return fmt.Errorf("failed to write temp config file: %w", err)
|
||||||
|
}
|
||||||
|
defer os.Remove(userTempFile) // Cleanup
|
||||||
|
|
||||||
|
// Copy temp file to config location with sudo
|
||||||
|
if err := exec.CommandContext(ctx, "sudo", "cp", userTempFile, configFile).Run(); err != nil {
|
||||||
|
return fmt.Errorf("failed to update config file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set proper ownership and permissions
|
||||||
|
if err := exec.CommandContext(ctx, "sudo", "chown", "minio-user:minio-user", configFile).Run(); err != nil {
|
||||||
|
s.logger.Warn("Failed to set config file ownership", "error", err)
|
||||||
|
}
|
||||||
|
if err := exec.CommandContext(ctx, "sudo", "chmod", "644", configFile).Run(); err != nil {
|
||||||
|
s.logger.Warn("Failed to set config file permissions", "error", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Updated MinIO configuration", "config_file", configFile, "volumes", datasetMountPoint)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// restartMinIOService restarts the MinIO service to apply new configuration
|
||||||
|
func (s *SetupService) restartMinIOService(ctx context.Context) error {
|
||||||
|
// Restart MinIO service using sudo
|
||||||
|
cmd := exec.CommandContext(ctx, "sudo", "systemctl", "restart", "minio.service")
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
return fmt.Errorf("failed to restart MinIO service: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait a moment for service to start
|
||||||
|
time.Sleep(2 * time.Second)
|
||||||
|
|
||||||
|
// Verify service is running
|
||||||
|
checkCmd := exec.CommandContext(ctx, "sudo", "systemctl", "is-active", "minio.service")
|
||||||
|
output, err := checkCmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to check MinIO service status: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
status := strings.TrimSpace(string(output))
|
||||||
|
if status != "active" {
|
||||||
|
return fmt.Errorf("MinIO service is not active after restart, status: %s", status)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("MinIO service restarted successfully")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -3,11 +3,13 @@ package scst
|
|||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
|
||||||
"github.com/atlasos/calypso/internal/common/database"
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
"github.com/atlasos/calypso/internal/common/logger"
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
"github.com/atlasos/calypso/internal/tasks"
|
"github.com/atlasos/calypso/internal/tasks"
|
||||||
"github.com/gin-gonic/gin"
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/go-playground/validator/v10"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Handler handles SCST-related API requests
|
// Handler handles SCST-related API requests
|
||||||
@@ -37,6 +39,11 @@ func (h *Handler) ListTargets(c *gin.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Ensure we return an empty array instead of null
|
||||||
|
if targets == nil {
|
||||||
|
targets = []Target{}
|
||||||
|
}
|
||||||
|
|
||||||
c.JSON(http.StatusOK, gin.H{"targets": targets})
|
c.JSON(http.StatusOK, gin.H{"targets": targets})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -112,6 +119,11 @@ func (h *Handler) CreateTarget(c *gin.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Set alias to name for frontend compatibility (same as ListTargets)
|
||||||
|
target.Alias = target.Name
|
||||||
|
// LUNCount will be 0 for newly created target
|
||||||
|
target.LUNCount = 0
|
||||||
|
|
||||||
c.JSON(http.StatusCreated, target)
|
c.JSON(http.StatusCreated, target)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -119,7 +131,7 @@ func (h *Handler) CreateTarget(c *gin.Context) {
|
|||||||
type AddLUNRequest struct {
|
type AddLUNRequest struct {
|
||||||
DeviceName string `json:"device_name" binding:"required"`
|
DeviceName string `json:"device_name" binding:"required"`
|
||||||
DevicePath string `json:"device_path" binding:"required"`
|
DevicePath string `json:"device_path" binding:"required"`
|
||||||
LUNNumber int `json:"lun_number" binding:"required"`
|
LUNNumber int `json:"lun_number"` // Note: cannot use binding:"required" for int as 0 is valid
|
||||||
HandlerType string `json:"handler_type" binding:"required"`
|
HandlerType string `json:"handler_type" binding:"required"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -136,17 +148,45 @@ func (h *Handler) AddLUN(c *gin.Context) {
|
|||||||
var req AddLUNRequest
|
var req AddLUNRequest
|
||||||
if err := c.ShouldBindJSON(&req); err != nil {
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
h.logger.Error("Failed to bind AddLUN request", "error", err)
|
h.logger.Error("Failed to bind AddLUN request", "error", err)
|
||||||
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("invalid request: %v", err)})
|
// Provide more detailed error message
|
||||||
|
if validationErr, ok := err.(validator.ValidationErrors); ok {
|
||||||
|
var errorMessages []string
|
||||||
|
for _, fieldErr := range validationErr {
|
||||||
|
errorMessages = append(errorMessages, fmt.Sprintf("%s is required", fieldErr.Field()))
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("validation failed: %s", strings.Join(errorMessages, ", "))})
|
||||||
|
} else {
|
||||||
|
// Extract error message without full struct name
|
||||||
|
errMsg := err.Error()
|
||||||
|
if idx := strings.Index(errMsg, "Key: '"); idx >= 0 {
|
||||||
|
// Extract field name from error message
|
||||||
|
fieldStart := idx + 6 // Length of "Key: '"
|
||||||
|
if fieldEnd := strings.Index(errMsg[fieldStart:], "'"); fieldEnd >= 0 {
|
||||||
|
fieldName := errMsg[fieldStart : fieldStart+fieldEnd]
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("invalid or missing field: %s", fieldName)})
|
||||||
|
} else {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request format"})
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("invalid request: %v", err)})
|
||||||
|
}
|
||||||
|
}
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Validate required fields
|
// Validate required fields (additional check in case binding doesn't catch it)
|
||||||
if req.DeviceName == "" || req.DevicePath == "" || req.HandlerType == "" {
|
if req.DeviceName == "" || req.DevicePath == "" || req.HandlerType == "" {
|
||||||
h.logger.Error("Missing required fields in AddLUN request", "device_name", req.DeviceName, "device_path", req.DevicePath, "handler_type", req.HandlerType)
|
h.logger.Error("Missing required fields in AddLUN request", "device_name", req.DeviceName, "device_path", req.DevicePath, "handler_type", req.HandlerType)
|
||||||
c.JSON(http.StatusBadRequest, gin.H{"error": "device_name, device_path, and handler_type are required"})
|
c.JSON(http.StatusBadRequest, gin.H{"error": "device_name, device_path, and handler_type are required"})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Validate LUN number range
|
||||||
|
if req.LUNNumber < 0 || req.LUNNumber > 255 {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "lun_number must be between 0 and 255"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
if err := h.service.AddLUN(c.Request.Context(), target.IQN, req.DeviceName, req.DevicePath, req.LUNNumber, req.HandlerType); err != nil {
|
if err := h.service.AddLUN(c.Request.Context(), target.IQN, req.DeviceName, req.DevicePath, req.LUNNumber, req.HandlerType); err != nil {
|
||||||
h.logger.Error("Failed to add LUN", "error", err)
|
h.logger.Error("Failed to add LUN", "error", err)
|
||||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
@@ -156,6 +196,48 @@ func (h *Handler) AddLUN(c *gin.Context) {
|
|||||||
c.JSON(http.StatusOK, gin.H{"message": "LUN added successfully"})
|
c.JSON(http.StatusOK, gin.H{"message": "LUN added successfully"})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RemoveLUN removes a LUN from a target
|
||||||
|
func (h *Handler) RemoveLUN(c *gin.Context) {
|
||||||
|
targetID := c.Param("id")
|
||||||
|
lunID := c.Param("lunId")
|
||||||
|
|
||||||
|
// Get target
|
||||||
|
target, err := h.service.GetTarget(c.Request.Context(), targetID)
|
||||||
|
if err != nil {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "target not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get LUN to get the LUN number
|
||||||
|
var lunNumber int
|
||||||
|
err = h.db.QueryRowContext(c.Request.Context(),
|
||||||
|
"SELECT lun_number FROM scst_luns WHERE id = $1 AND target_id = $2",
|
||||||
|
lunID, targetID,
|
||||||
|
).Scan(&lunNumber)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "no rows") {
|
||||||
|
// LUN already deleted from database - check if it still exists in SCST
|
||||||
|
// Try to get LUN number from URL or try common LUN numbers
|
||||||
|
// For now, return success since it's already deleted (idempotent)
|
||||||
|
h.logger.Info("LUN not found in database, may already be deleted", "lun_id", lunID, "target_id", targetID)
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "LUN already removed or not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to get LUN", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get LUN"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove LUN
|
||||||
|
if err := h.service.RemoveLUN(c.Request.Context(), target.IQN, lunNumber); err != nil {
|
||||||
|
h.logger.Error("Failed to remove LUN", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "LUN removed successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
// AddInitiatorRequest represents an initiator addition request
|
// AddInitiatorRequest represents an initiator addition request
|
||||||
type AddInitiatorRequest struct {
|
type AddInitiatorRequest struct {
|
||||||
InitiatorIQN string `json:"initiator_iqn" binding:"required"`
|
InitiatorIQN string `json:"initiator_iqn" binding:"required"`
|
||||||
@@ -186,6 +268,45 @@ func (h *Handler) AddInitiator(c *gin.Context) {
|
|||||||
c.JSON(http.StatusOK, gin.H{"message": "Initiator added successfully"})
|
c.JSON(http.StatusOK, gin.H{"message": "Initiator added successfully"})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// AddInitiatorToGroupRequest represents a request to add an initiator to a group
|
||||||
|
type AddInitiatorToGroupRequest struct {
|
||||||
|
InitiatorIQN string `json:"initiator_iqn" binding:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AddInitiatorToGroup adds an initiator to a specific group
|
||||||
|
func (h *Handler) AddInitiatorToGroup(c *gin.Context) {
|
||||||
|
groupID := c.Param("id")
|
||||||
|
|
||||||
|
var req AddInitiatorToGroupRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
validationErrors := make(map[string]string)
|
||||||
|
if ve, ok := err.(validator.ValidationErrors); ok {
|
||||||
|
for _, fe := range ve {
|
||||||
|
field := strings.ToLower(fe.Field())
|
||||||
|
validationErrors[field] = fmt.Sprintf("Field '%s' is required", field)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{
|
||||||
|
"error": "invalid request",
|
||||||
|
"validation_errors": validationErrors,
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
err := h.service.AddInitiatorToGroup(c.Request.Context(), groupID, req.InitiatorIQN)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "not found") || strings.Contains(err.Error(), "already exists") || strings.Contains(err.Error(), "single initiator only") {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to add initiator to group", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to add initiator to group"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "Initiator added to group successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
// ListAllInitiators lists all initiators across all targets
|
// ListAllInitiators lists all initiators across all targets
|
||||||
func (h *Handler) ListAllInitiators(c *gin.Context) {
|
func (h *Handler) ListAllInitiators(c *gin.Context) {
|
||||||
initiators, err := h.service.ListAllInitiators(c.Request.Context())
|
initiators, err := h.service.ListAllInitiators(c.Request.Context())
|
||||||
@@ -440,6 +561,23 @@ func (h *Handler) DisableTarget(c *gin.Context) {
|
|||||||
c.JSON(http.StatusOK, gin.H{"message": "Target disabled successfully"})
|
c.JSON(http.StatusOK, gin.H{"message": "Target disabled successfully"})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// DeleteTarget deletes a target
|
||||||
|
func (h *Handler) DeleteTarget(c *gin.Context) {
|
||||||
|
targetID := c.Param("id")
|
||||||
|
|
||||||
|
if err := h.service.DeleteTarget(c.Request.Context(), targetID); err != nil {
|
||||||
|
if err.Error() == "target not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "target not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to delete target", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "Target deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
// DeletePortal deletes a portal
|
// DeletePortal deletes a portal
|
||||||
func (h *Handler) DeletePortal(c *gin.Context) {
|
func (h *Handler) DeletePortal(c *gin.Context) {
|
||||||
id := c.Param("id")
|
id := c.Param("id")
|
||||||
@@ -474,3 +612,182 @@ func (h *Handler) GetPortal(c *gin.Context) {
|
|||||||
|
|
||||||
c.JSON(http.StatusOK, portal)
|
c.JSON(http.StatusOK, portal)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CreateInitiatorGroupRequest represents a request to create an initiator group
|
||||||
|
type CreateInitiatorGroupRequest struct {
|
||||||
|
TargetID string `json:"target_id" binding:"required"`
|
||||||
|
GroupName string `json:"group_name" binding:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateInitiatorGroup creates a new initiator group
|
||||||
|
func (h *Handler) CreateInitiatorGroup(c *gin.Context) {
|
||||||
|
var req CreateInitiatorGroupRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
validationErrors := make(map[string]string)
|
||||||
|
if ve, ok := err.(validator.ValidationErrors); ok {
|
||||||
|
for _, fe := range ve {
|
||||||
|
field := strings.ToLower(fe.Field())
|
||||||
|
validationErrors[field] = fmt.Sprintf("Field '%s' is required", field)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{
|
||||||
|
"error": "invalid request",
|
||||||
|
"validation_errors": validationErrors,
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
group, err := h.service.CreateInitiatorGroup(c.Request.Context(), req.TargetID, req.GroupName)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "already exists") || strings.Contains(err.Error(), "not found") {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to create initiator group", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create initiator group"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateInitiatorGroupRequest represents a request to update an initiator group
|
||||||
|
type UpdateInitiatorGroupRequest struct {
|
||||||
|
GroupName string `json:"group_name" binding:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateInitiatorGroup updates an initiator group
|
||||||
|
func (h *Handler) UpdateInitiatorGroup(c *gin.Context) {
|
||||||
|
groupID := c.Param("id")
|
||||||
|
|
||||||
|
var req UpdateInitiatorGroupRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
validationErrors := make(map[string]string)
|
||||||
|
if ve, ok := err.(validator.ValidationErrors); ok {
|
||||||
|
for _, fe := range ve {
|
||||||
|
field := strings.ToLower(fe.Field())
|
||||||
|
validationErrors[field] = fmt.Sprintf("Field '%s' is required", field)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{
|
||||||
|
"error": "invalid request",
|
||||||
|
"validation_errors": validationErrors,
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
group, err := h.service.UpdateInitiatorGroup(c.Request.Context(), groupID, req.GroupName)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "not found") || strings.Contains(err.Error(), "already exists") {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to update initiator group", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update initiator group"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteInitiatorGroup deletes an initiator group
|
||||||
|
func (h *Handler) DeleteInitiatorGroup(c *gin.Context) {
|
||||||
|
groupID := c.Param("id")
|
||||||
|
|
||||||
|
err := h.service.DeleteInitiatorGroup(c.Request.Context(), groupID)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "not found") {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), "cannot delete") || strings.Contains(err.Error(), "contains") {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to delete initiator group", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete initiator group"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "initiator group deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetInitiatorGroup retrieves an initiator group by ID
|
||||||
|
func (h *Handler) GetInitiatorGroup(c *gin.Context) {
|
||||||
|
groupID := c.Param("id")
|
||||||
|
|
||||||
|
group, err := h.service.GetInitiatorGroup(c.Request.Context(), groupID)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "not found") {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "initiator group not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to get initiator group", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get initiator group"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListAllInitiatorGroups lists all initiator groups
|
||||||
|
func (h *Handler) ListAllInitiatorGroups(c *gin.Context) {
|
||||||
|
groups, err := h.service.ListAllInitiatorGroups(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list initiator groups", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list initiator groups"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if groups == nil {
|
||||||
|
groups = []InitiatorGroup{}
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"groups": groups})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetConfigFile reads the SCST configuration file content
|
||||||
|
func (h *Handler) GetConfigFile(c *gin.Context) {
|
||||||
|
configPath := c.DefaultQuery("path", "/etc/scst.conf")
|
||||||
|
|
||||||
|
content, err := h.service.ReadConfigFile(c.Request.Context(), configPath)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to read config file", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"content": content,
|
||||||
|
"path": configPath,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateConfigFile writes content to SCST configuration file
|
||||||
|
func (h *Handler) UpdateConfigFile(c *gin.Context) {
|
||||||
|
var req struct {
|
||||||
|
Content string `json:"content" binding:"required"`
|
||||||
|
Path string `json:"path"`
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
configPath := req.Path
|
||||||
|
if configPath == "" {
|
||||||
|
configPath = "/etc/scst.conf"
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.service.WriteConfigFile(c.Request.Context(), configPath, req.Content); err != nil {
|
||||||
|
h.logger.Error("Failed to write config file", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{
|
||||||
|
"message": "Configuration file updated successfully",
|
||||||
|
"path": configPath,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
147
backend/internal/shares/handler.go
Normal file
147
backend/internal/shares/handler.go
Normal file
@@ -0,0 +1,147 @@
|
|||||||
|
package shares
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/go-playground/validator/v10"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Handler handles Shares-related API requests
|
||||||
|
type Handler struct {
|
||||||
|
service *Service
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewHandler creates a new Shares handler
|
||||||
|
func NewHandler(db *database.DB, log *logger.Logger) *Handler {
|
||||||
|
return &Handler{
|
||||||
|
service: NewService(db, log),
|
||||||
|
logger: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListShares lists all shares
|
||||||
|
func (h *Handler) ListShares(c *gin.Context) {
|
||||||
|
shares, err := h.service.ListShares(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list shares", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list shares"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure we return an empty array instead of null
|
||||||
|
if shares == nil {
|
||||||
|
shares = []*Share{}
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"shares": shares})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetShare retrieves a share by ID
|
||||||
|
func (h *Handler) GetShare(c *gin.Context) {
|
||||||
|
shareID := c.Param("id")
|
||||||
|
|
||||||
|
share, err := h.service.GetShare(c.Request.Context(), shareID)
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "share not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "share not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to get share", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get share"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, share)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateShare creates a new share
|
||||||
|
func (h *Handler) CreateShare(c *gin.Context) {
|
||||||
|
var req CreateShareRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create share request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate request
|
||||||
|
validate := validator.New()
|
||||||
|
if err := validate.Struct(req); err != nil {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "validation failed: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get user ID from context (set by auth middleware)
|
||||||
|
userID, exists := c.Get("user_id")
|
||||||
|
if !exists {
|
||||||
|
c.JSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
share, err := h.service.CreateShare(c.Request.Context(), &req, userID.(string))
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "dataset not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "dataset not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err.Error() == "only filesystem datasets can be shared (not volumes)" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err.Error() == "at least one protocol (NFS or SMB) must be enabled" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to create share", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, share)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateShare updates an existing share
|
||||||
|
func (h *Handler) UpdateShare(c *gin.Context) {
|
||||||
|
shareID := c.Param("id")
|
||||||
|
|
||||||
|
var req UpdateShareRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid update share request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
share, err := h.service.UpdateShare(c.Request.Context(), shareID, &req)
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "share not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "share not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to update share", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, share)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteShare deletes a share
|
||||||
|
func (h *Handler) DeleteShare(c *gin.Context) {
|
||||||
|
shareID := c.Param("id")
|
||||||
|
|
||||||
|
err := h.service.DeleteShare(c.Request.Context(), shareID)
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "share not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "share not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to delete share", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "share deleted successfully"})
|
||||||
|
}
|
||||||
834
backend/internal/shares/service.go
Normal file
834
backend/internal/shares/service.go
Normal file
@@ -0,0 +1,834 @@
|
|||||||
|
package shares
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/lib/pq"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Service handles Shares (CIFS/NFS) operations
|
||||||
|
type Service struct {
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewService creates a new Shares service
|
||||||
|
func NewService(db *database.DB, log *logger.Logger) *Service {
|
||||||
|
return &Service{
|
||||||
|
db: db,
|
||||||
|
logger: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Share represents a filesystem share (NFS/SMB)
|
||||||
|
type Share struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
DatasetID string `json:"dataset_id"`
|
||||||
|
DatasetName string `json:"dataset_name"`
|
||||||
|
MountPoint string `json:"mount_point"`
|
||||||
|
ShareType string `json:"share_type"` // 'nfs', 'smb', 'both'
|
||||||
|
NFSEnabled bool `json:"nfs_enabled"`
|
||||||
|
NFSOptions string `json:"nfs_options,omitempty"`
|
||||||
|
NFSClients []string `json:"nfs_clients,omitempty"`
|
||||||
|
SMBEnabled bool `json:"smb_enabled"`
|
||||||
|
SMBShareName string `json:"smb_share_name,omitempty"`
|
||||||
|
SMBPath string `json:"smb_path,omitempty"`
|
||||||
|
SMBComment string `json:"smb_comment,omitempty"`
|
||||||
|
SMBGuestOK bool `json:"smb_guest_ok"`
|
||||||
|
SMBReadOnly bool `json:"smb_read_only"`
|
||||||
|
SMBBrowseable bool `json:"smb_browseable"`
|
||||||
|
IsActive bool `json:"is_active"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
UpdatedAt time.Time `json:"updated_at"`
|
||||||
|
CreatedBy string `json:"created_by"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListShares lists all shares
|
||||||
|
func (s *Service) ListShares(ctx context.Context) ([]*Share, error) {
|
||||||
|
query := `
|
||||||
|
SELECT
|
||||||
|
zs.id, zs.dataset_id, zd.name as dataset_name, zd.mount_point,
|
||||||
|
zs.share_type, zs.nfs_enabled, zs.nfs_options, zs.nfs_clients,
|
||||||
|
zs.smb_enabled, zs.smb_share_name, zs.smb_path, zs.smb_comment,
|
||||||
|
zs.smb_guest_ok, zs.smb_read_only, zs.smb_browseable,
|
||||||
|
zs.is_active, zs.created_at, zs.updated_at, zs.created_by
|
||||||
|
FROM zfs_shares zs
|
||||||
|
JOIN zfs_datasets zd ON zs.dataset_id = zd.id
|
||||||
|
ORDER BY zd.name
|
||||||
|
`
|
||||||
|
|
||||||
|
rows, err := s.db.QueryContext(ctx, query)
|
||||||
|
if err != nil {
|
||||||
|
if strings.Contains(err.Error(), "does not exist") {
|
||||||
|
s.logger.Warn("zfs_shares table does not exist, returning empty list")
|
||||||
|
return []*Share{}, nil
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to list shares: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var shares []*Share
|
||||||
|
for rows.Next() {
|
||||||
|
var share Share
|
||||||
|
var mountPoint sql.NullString
|
||||||
|
var nfsOptions sql.NullString
|
||||||
|
var smbShareName sql.NullString
|
||||||
|
var smbPath sql.NullString
|
||||||
|
var smbComment sql.NullString
|
||||||
|
var nfsClients []string
|
||||||
|
|
||||||
|
err := rows.Scan(
|
||||||
|
&share.ID, &share.DatasetID, &share.DatasetName, &mountPoint,
|
||||||
|
&share.ShareType, &share.NFSEnabled, &nfsOptions, pq.Array(&nfsClients),
|
||||||
|
&share.SMBEnabled, &smbShareName, &smbPath, &smbComment,
|
||||||
|
&share.SMBGuestOK, &share.SMBReadOnly, &share.SMBBrowseable,
|
||||||
|
&share.IsActive, &share.CreatedAt, &share.UpdatedAt, &share.CreatedBy,
|
||||||
|
)
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to scan share row", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
share.NFSClients = nfsClients
|
||||||
|
|
||||||
|
if mountPoint.Valid {
|
||||||
|
share.MountPoint = mountPoint.String
|
||||||
|
}
|
||||||
|
if nfsOptions.Valid {
|
||||||
|
share.NFSOptions = nfsOptions.String
|
||||||
|
}
|
||||||
|
if smbShareName.Valid {
|
||||||
|
share.SMBShareName = smbShareName.String
|
||||||
|
}
|
||||||
|
if smbPath.Valid {
|
||||||
|
share.SMBPath = smbPath.String
|
||||||
|
}
|
||||||
|
if smbComment.Valid {
|
||||||
|
share.SMBComment = smbComment.String
|
||||||
|
}
|
||||||
|
|
||||||
|
shares = append(shares, &share)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return nil, fmt.Errorf("error iterating share rows: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return shares, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetShare retrieves a share by ID
|
||||||
|
func (s *Service) GetShare(ctx context.Context, shareID string) (*Share, error) {
|
||||||
|
query := `
|
||||||
|
SELECT
|
||||||
|
zs.id, zs.dataset_id, zd.name as dataset_name, zd.mount_point,
|
||||||
|
zs.share_type, zs.nfs_enabled, zs.nfs_options, zs.nfs_clients,
|
||||||
|
zs.smb_enabled, zs.smb_share_name, zs.smb_path, zs.smb_comment,
|
||||||
|
zs.smb_guest_ok, zs.smb_read_only, zs.smb_browseable,
|
||||||
|
zs.is_active, zs.created_at, zs.updated_at, zs.created_by
|
||||||
|
FROM zfs_shares zs
|
||||||
|
JOIN zfs_datasets zd ON zs.dataset_id = zd.id
|
||||||
|
WHERE zs.id = $1
|
||||||
|
`
|
||||||
|
|
||||||
|
var share Share
|
||||||
|
var mountPoint sql.NullString
|
||||||
|
var nfsOptions sql.NullString
|
||||||
|
var smbShareName sql.NullString
|
||||||
|
var smbPath sql.NullString
|
||||||
|
var smbComment sql.NullString
|
||||||
|
var nfsClients []string
|
||||||
|
|
||||||
|
err := s.db.QueryRowContext(ctx, query, shareID).Scan(
|
||||||
|
&share.ID, &share.DatasetID, &share.DatasetName, &mountPoint,
|
||||||
|
&share.ShareType, &share.NFSEnabled, &nfsOptions, pq.Array(&nfsClients),
|
||||||
|
&share.SMBEnabled, &smbShareName, &smbPath, &smbComment,
|
||||||
|
&share.SMBGuestOK, &share.SMBReadOnly, &share.SMBBrowseable,
|
||||||
|
&share.IsActive, &share.CreatedAt, &share.UpdatedAt, &share.CreatedBy,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
return nil, fmt.Errorf("share not found")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to get share: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
share.NFSClients = nfsClients
|
||||||
|
|
||||||
|
if mountPoint.Valid {
|
||||||
|
share.MountPoint = mountPoint.String
|
||||||
|
}
|
||||||
|
if nfsOptions.Valid {
|
||||||
|
share.NFSOptions = nfsOptions.String
|
||||||
|
}
|
||||||
|
if smbShareName.Valid {
|
||||||
|
share.SMBShareName = smbShareName.String
|
||||||
|
}
|
||||||
|
if smbPath.Valid {
|
||||||
|
share.SMBPath = smbPath.String
|
||||||
|
}
|
||||||
|
if smbComment.Valid {
|
||||||
|
share.SMBComment = smbComment.String
|
||||||
|
}
|
||||||
|
|
||||||
|
return &share, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateShareRequest represents a share creation request
|
||||||
|
type CreateShareRequest struct {
|
||||||
|
DatasetID string `json:"dataset_id" binding:"required"`
|
||||||
|
NFSEnabled bool `json:"nfs_enabled"`
|
||||||
|
NFSOptions string `json:"nfs_options"`
|
||||||
|
NFSClients []string `json:"nfs_clients"`
|
||||||
|
SMBEnabled bool `json:"smb_enabled"`
|
||||||
|
SMBShareName string `json:"smb_share_name"`
|
||||||
|
SMBPath string `json:"smb_path"`
|
||||||
|
SMBComment string `json:"smb_comment"`
|
||||||
|
SMBGuestOK bool `json:"smb_guest_ok"`
|
||||||
|
SMBReadOnly bool `json:"smb_read_only"`
|
||||||
|
SMBBrowseable bool `json:"smb_browseable"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateShare creates a new share
|
||||||
|
func (s *Service) CreateShare(ctx context.Context, req *CreateShareRequest, userID string) (*Share, error) {
|
||||||
|
// Validate dataset exists and is a filesystem (not volume)
|
||||||
|
// req.DatasetID can be either UUID or dataset name
|
||||||
|
var datasetID, datasetType, datasetName, mountPoint string
|
||||||
|
var mountPointNull sql.NullString
|
||||||
|
|
||||||
|
// Try to find by ID first (UUID)
|
||||||
|
err := s.db.QueryRowContext(ctx,
|
||||||
|
"SELECT id, type, name, mount_point FROM zfs_datasets WHERE id = $1",
|
||||||
|
req.DatasetID,
|
||||||
|
).Scan(&datasetID, &datasetType, &datasetName, &mountPointNull)
|
||||||
|
|
||||||
|
// If not found by ID, try by name
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
err = s.db.QueryRowContext(ctx,
|
||||||
|
"SELECT id, type, name, mount_point FROM zfs_datasets WHERE name = $1",
|
||||||
|
req.DatasetID,
|
||||||
|
).Scan(&datasetID, &datasetType, &datasetName, &mountPointNull)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
return nil, fmt.Errorf("dataset not found")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to validate dataset: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if mountPointNull.Valid {
|
||||||
|
mountPoint = mountPointNull.String
|
||||||
|
} else {
|
||||||
|
mountPoint = "none"
|
||||||
|
}
|
||||||
|
|
||||||
|
if datasetType != "filesystem" {
|
||||||
|
return nil, fmt.Errorf("only filesystem datasets can be shared (not volumes)")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Determine share type
|
||||||
|
shareType := "none"
|
||||||
|
if req.NFSEnabled && req.SMBEnabled {
|
||||||
|
shareType = "both"
|
||||||
|
} else if req.NFSEnabled {
|
||||||
|
shareType = "nfs"
|
||||||
|
} else if req.SMBEnabled {
|
||||||
|
shareType = "smb"
|
||||||
|
} else {
|
||||||
|
return nil, fmt.Errorf("at least one protocol (NFS or SMB) must be enabled")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set default NFS options if not provided
|
||||||
|
nfsOptions := req.NFSOptions
|
||||||
|
if nfsOptions == "" {
|
||||||
|
nfsOptions = "rw,sync,no_subtree_check"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set default SMB share name if not provided
|
||||||
|
smbShareName := req.SMBShareName
|
||||||
|
if smbShareName == "" {
|
||||||
|
// Extract dataset name from full path (e.g., "pool/dataset" -> "dataset")
|
||||||
|
parts := strings.Split(datasetName, "/")
|
||||||
|
smbShareName = parts[len(parts)-1]
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set SMB path (use mount_point if available, otherwise use dataset name)
|
||||||
|
smbPath := req.SMBPath
|
||||||
|
if smbPath == "" {
|
||||||
|
if mountPoint != "" && mountPoint != "none" {
|
||||||
|
smbPath = mountPoint
|
||||||
|
} else {
|
||||||
|
smbPath = fmt.Sprintf("/mnt/%s", strings.ReplaceAll(datasetName, "/", "_"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Insert into database
|
||||||
|
query := `
|
||||||
|
INSERT INTO zfs_shares (
|
||||||
|
dataset_id, share_type, nfs_enabled, nfs_options, nfs_clients,
|
||||||
|
smb_enabled, smb_share_name, smb_path, smb_comment,
|
||||||
|
smb_guest_ok, smb_read_only, smb_browseable, is_active, created_by
|
||||||
|
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)
|
||||||
|
RETURNING id, created_at, updated_at
|
||||||
|
`
|
||||||
|
|
||||||
|
var shareID string
|
||||||
|
var createdAt, updatedAt time.Time
|
||||||
|
|
||||||
|
// Handle nfs_clients array - use empty array if nil
|
||||||
|
nfsClients := req.NFSClients
|
||||||
|
if nfsClients == nil {
|
||||||
|
nfsClients = []string{}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = s.db.QueryRowContext(ctx, query,
|
||||||
|
datasetID, shareType, req.NFSEnabled, nfsOptions, pq.Array(nfsClients),
|
||||||
|
req.SMBEnabled, smbShareName, smbPath, req.SMBComment,
|
||||||
|
req.SMBGuestOK, req.SMBReadOnly, req.SMBBrowseable, true, userID,
|
||||||
|
).Scan(&shareID, &createdAt, &updatedAt)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create share: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply NFS export if enabled
|
||||||
|
if req.NFSEnabled {
|
||||||
|
if err := s.applyNFSExport(ctx, mountPoint, nfsOptions, req.NFSClients); err != nil {
|
||||||
|
s.logger.Error("Failed to apply NFS export", "error", err, "share_id", shareID)
|
||||||
|
// Don't fail the creation, but log the error
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply SMB share if enabled
|
||||||
|
if req.SMBEnabled {
|
||||||
|
if err := s.applySMBShare(ctx, smbShareName, smbPath, req.SMBComment, req.SMBGuestOK, req.SMBReadOnly, req.SMBBrowseable); err != nil {
|
||||||
|
s.logger.Error("Failed to apply SMB share", "error", err, "share_id", shareID)
|
||||||
|
// Don't fail the creation, but log the error
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return the created share
|
||||||
|
return s.GetShare(ctx, shareID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateShareRequest represents a share update request
|
||||||
|
type UpdateShareRequest struct {
|
||||||
|
NFSEnabled *bool `json:"nfs_enabled"`
|
||||||
|
NFSOptions *string `json:"nfs_options"`
|
||||||
|
NFSClients *[]string `json:"nfs_clients"`
|
||||||
|
SMBEnabled *bool `json:"smb_enabled"`
|
||||||
|
SMBShareName *string `json:"smb_share_name"`
|
||||||
|
SMBComment *string `json:"smb_comment"`
|
||||||
|
SMBGuestOK *bool `json:"smb_guest_ok"`
|
||||||
|
SMBReadOnly *bool `json:"smb_read_only"`
|
||||||
|
SMBBrowseable *bool `json:"smb_browseable"`
|
||||||
|
IsActive *bool `json:"is_active"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateShare updates an existing share
|
||||||
|
func (s *Service) UpdateShare(ctx context.Context, shareID string, req *UpdateShareRequest) (*Share, error) {
|
||||||
|
// Get current share
|
||||||
|
share, err := s.GetShare(ctx, shareID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build update query dynamically
|
||||||
|
updates := []string{}
|
||||||
|
args := []interface{}{}
|
||||||
|
argIndex := 1
|
||||||
|
|
||||||
|
if req.NFSEnabled != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("nfs_enabled = $%d", argIndex))
|
||||||
|
args = append(args, *req.NFSEnabled)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.NFSOptions != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("nfs_options = $%d", argIndex))
|
||||||
|
args = append(args, *req.NFSOptions)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.NFSClients != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("nfs_clients = $%d", argIndex))
|
||||||
|
args = append(args, pq.Array(*req.NFSClients))
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.SMBEnabled != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("smb_enabled = $%d", argIndex))
|
||||||
|
args = append(args, *req.SMBEnabled)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.SMBShareName != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("smb_share_name = $%d", argIndex))
|
||||||
|
args = append(args, *req.SMBShareName)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.SMBComment != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("smb_comment = $%d", argIndex))
|
||||||
|
args = append(args, *req.SMBComment)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.SMBGuestOK != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("smb_guest_ok = $%d", argIndex))
|
||||||
|
args = append(args, *req.SMBGuestOK)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.SMBReadOnly != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("smb_read_only = $%d", argIndex))
|
||||||
|
args = append(args, *req.SMBReadOnly)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.SMBBrowseable != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("smb_browseable = $%d", argIndex))
|
||||||
|
args = append(args, *req.SMBBrowseable)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
if req.IsActive != nil {
|
||||||
|
updates = append(updates, fmt.Sprintf("is_active = $%d", argIndex))
|
||||||
|
args = append(args, *req.IsActive)
|
||||||
|
argIndex++
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(updates) == 0 {
|
||||||
|
return share, nil // No changes
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update share_type based on enabled protocols
|
||||||
|
nfsEnabled := share.NFSEnabled
|
||||||
|
smbEnabled := share.SMBEnabled
|
||||||
|
if req.NFSEnabled != nil {
|
||||||
|
nfsEnabled = *req.NFSEnabled
|
||||||
|
}
|
||||||
|
if req.SMBEnabled != nil {
|
||||||
|
smbEnabled = *req.SMBEnabled
|
||||||
|
}
|
||||||
|
|
||||||
|
shareType := "none"
|
||||||
|
if nfsEnabled && smbEnabled {
|
||||||
|
shareType = "both"
|
||||||
|
} else if nfsEnabled {
|
||||||
|
shareType = "nfs"
|
||||||
|
} else if smbEnabled {
|
||||||
|
shareType = "smb"
|
||||||
|
}
|
||||||
|
|
||||||
|
updates = append(updates, fmt.Sprintf("share_type = $%d", argIndex))
|
||||||
|
args = append(args, shareType)
|
||||||
|
argIndex++
|
||||||
|
|
||||||
|
updates = append(updates, fmt.Sprintf("updated_at = NOW()"))
|
||||||
|
args = append(args, shareID)
|
||||||
|
|
||||||
|
query := fmt.Sprintf(`
|
||||||
|
UPDATE zfs_shares
|
||||||
|
SET %s
|
||||||
|
WHERE id = $%d
|
||||||
|
`, strings.Join(updates, ", "), argIndex)
|
||||||
|
|
||||||
|
_, err = s.db.ExecContext(ctx, query, args...)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to update share: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-apply NFS export if NFS is enabled
|
||||||
|
if nfsEnabled {
|
||||||
|
nfsOptions := share.NFSOptions
|
||||||
|
if req.NFSOptions != nil {
|
||||||
|
nfsOptions = *req.NFSOptions
|
||||||
|
}
|
||||||
|
nfsClients := share.NFSClients
|
||||||
|
if req.NFSClients != nil {
|
||||||
|
nfsClients = *req.NFSClients
|
||||||
|
}
|
||||||
|
if err := s.applyNFSExport(ctx, share.MountPoint, nfsOptions, nfsClients); err != nil {
|
||||||
|
s.logger.Error("Failed to apply NFS export", "error", err, "share_id", shareID)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Remove NFS export if disabled
|
||||||
|
if err := s.removeNFSExport(ctx, share.MountPoint); err != nil {
|
||||||
|
s.logger.Error("Failed to remove NFS export", "error", err, "share_id", shareID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-apply SMB share if SMB is enabled
|
||||||
|
if smbEnabled {
|
||||||
|
smbShareName := share.SMBShareName
|
||||||
|
if req.SMBShareName != nil {
|
||||||
|
smbShareName = *req.SMBShareName
|
||||||
|
}
|
||||||
|
smbPath := share.SMBPath
|
||||||
|
smbComment := share.SMBComment
|
||||||
|
if req.SMBComment != nil {
|
||||||
|
smbComment = *req.SMBComment
|
||||||
|
}
|
||||||
|
smbGuestOK := share.SMBGuestOK
|
||||||
|
if req.SMBGuestOK != nil {
|
||||||
|
smbGuestOK = *req.SMBGuestOK
|
||||||
|
}
|
||||||
|
smbReadOnly := share.SMBReadOnly
|
||||||
|
if req.SMBReadOnly != nil {
|
||||||
|
smbReadOnly = *req.SMBReadOnly
|
||||||
|
}
|
||||||
|
smbBrowseable := share.SMBBrowseable
|
||||||
|
if req.SMBBrowseable != nil {
|
||||||
|
smbBrowseable = *req.SMBBrowseable
|
||||||
|
}
|
||||||
|
if err := s.applySMBShare(ctx, smbShareName, smbPath, smbComment, smbGuestOK, smbReadOnly, smbBrowseable); err != nil {
|
||||||
|
s.logger.Error("Failed to apply SMB share", "error", err, "share_id", shareID)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Remove SMB share if disabled
|
||||||
|
if err := s.removeSMBShare(ctx, share.SMBShareName); err != nil {
|
||||||
|
s.logger.Error("Failed to remove SMB share", "error", err, "share_id", shareID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return s.GetShare(ctx, shareID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteShare deletes a share
|
||||||
|
func (s *Service) DeleteShare(ctx context.Context, shareID string) error {
|
||||||
|
// Get share to get mount point and share name
|
||||||
|
share, err := s.GetShare(ctx, shareID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove NFS export
|
||||||
|
if share.NFSEnabled {
|
||||||
|
if err := s.removeNFSExport(ctx, share.MountPoint); err != nil {
|
||||||
|
s.logger.Error("Failed to remove NFS export", "error", err, "share_id", shareID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove SMB share
|
||||||
|
if share.SMBEnabled {
|
||||||
|
if err := s.removeSMBShare(ctx, share.SMBShareName); err != nil {
|
||||||
|
s.logger.Error("Failed to remove SMB share", "error", err, "share_id", shareID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Delete from database
|
||||||
|
_, err = s.db.ExecContext(ctx, "DELETE FROM zfs_shares WHERE id = $1", shareID)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete share: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// applyNFSExport adds or updates an NFS export in /etc/exports
|
||||||
|
func (s *Service) applyNFSExport(ctx context.Context, mountPoint, options string, clients []string) error {
|
||||||
|
if mountPoint == "" || mountPoint == "none" {
|
||||||
|
return fmt.Errorf("mount point is required for NFS export")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build client list (default to * if empty)
|
||||||
|
clientList := "*"
|
||||||
|
if len(clients) > 0 {
|
||||||
|
clientList = strings.Join(clients, " ")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build export line (format: /path client(options))
|
||||||
|
exportLine := fmt.Sprintf("%s %s(%s)", mountPoint, clientList, options)
|
||||||
|
|
||||||
|
// Read current exports file (use actual config file, not symlink)
|
||||||
|
exportsPath := "/opt/calypso/conf/nfs/exports"
|
||||||
|
exportsContent, err := os.ReadFile(exportsPath)
|
||||||
|
if err != nil && !os.IsNotExist(err) {
|
||||||
|
return fmt.Errorf("failed to read exports file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(string(exportsContent), "\n")
|
||||||
|
var newLines []string
|
||||||
|
found := false
|
||||||
|
|
||||||
|
// Check if this mount point already exists
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" || strings.HasPrefix(line, "#") {
|
||||||
|
newLines = append(newLines, line)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if this line is for our mount point
|
||||||
|
if strings.HasPrefix(line, mountPoint+" ") {
|
||||||
|
newLines = append(newLines, exportLine)
|
||||||
|
found = true
|
||||||
|
} else {
|
||||||
|
newLines = append(newLines, line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add if not found
|
||||||
|
if !found {
|
||||||
|
newLines = append(newLines, exportLine)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write back to file (use sudo to ensure proper permissions)
|
||||||
|
newContent := strings.Join(newLines, "\n") + "\n"
|
||||||
|
tempFile := exportsPath + ".tmp"
|
||||||
|
if err := os.WriteFile(tempFile, []byte(newContent), 0644); err != nil {
|
||||||
|
return fmt.Errorf("failed to write temporary exports file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Move temp file to actual location using sudo
|
||||||
|
cmd := exec.CommandContext(ctx, "sudo", "mv", tempFile, exportsPath)
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("failed to move exports file: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply exports (reload NFS exports)
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "exportfs", "-ra")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("failed to apply exports: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Restart NFS service to ensure changes are picked up
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "systemctl", "restart", "nfs-kernel-server")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
s.logger.Warn("Failed to restart NFS service", "error", string(output))
|
||||||
|
// Don't fail, as exportfs -ra should be sufficient
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("NFS export applied", "mount_point", mountPoint, "clients", clientList)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// removeNFSExport removes an NFS export from exports file
|
||||||
|
func (s *Service) removeNFSExport(ctx context.Context, mountPoint string) error {
|
||||||
|
if mountPoint == "" || mountPoint == "none" {
|
||||||
|
return nil // Nothing to remove
|
||||||
|
}
|
||||||
|
|
||||||
|
exportsPath := "/opt/calypso/conf/nfs/exports"
|
||||||
|
exportsContent, err := os.ReadFile(exportsPath)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil // File doesn't exist, nothing to remove
|
||||||
|
}
|
||||||
|
return fmt.Errorf("failed to read exports file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(string(exportsContent), "\n")
|
||||||
|
var newLines []string
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" || strings.HasPrefix(line, "#") {
|
||||||
|
newLines = append(newLines, line)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Skip lines for this mount point
|
||||||
|
if strings.HasPrefix(line, mountPoint+" ") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
newLines = append(newLines, line)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write back to file (use sudo to ensure proper permissions)
|
||||||
|
newContent := strings.Join(newLines, "\n")
|
||||||
|
if newContent != "" && !strings.HasSuffix(newContent, "\n") {
|
||||||
|
newContent += "\n"
|
||||||
|
}
|
||||||
|
tempFile := exportsPath + ".tmp"
|
||||||
|
if err := os.WriteFile(tempFile, []byte(newContent), 0644); err != nil {
|
||||||
|
return fmt.Errorf("failed to write temporary exports file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Move temp file to actual location using sudo
|
||||||
|
cmd := exec.CommandContext(ctx, "sudo", "mv", tempFile, exportsPath)
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("failed to move exports file: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply exports (reload NFS exports)
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "exportfs", "-ra")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("failed to apply exports: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Restart NFS service to ensure changes are picked up
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "systemctl", "restart", "nfs-kernel-server")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
s.logger.Warn("Failed to restart NFS service", "error", string(output))
|
||||||
|
// Don't fail, as exportfs -ra should be sufficient
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("NFS export removed", "mount_point", mountPoint)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// applySMBShare adds or updates an SMB share in /etc/samba/smb.conf
|
||||||
|
func (s *Service) applySMBShare(ctx context.Context, shareName, path, comment string, guestOK, readOnly, browseable bool) error {
|
||||||
|
if shareName == "" {
|
||||||
|
return fmt.Errorf("SMB share name is required")
|
||||||
|
}
|
||||||
|
if path == "" {
|
||||||
|
return fmt.Errorf("SMB path is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
smbConfPath := "/etc/samba/smb.conf"
|
||||||
|
smbContent, err := os.ReadFile(smbConfPath)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to read smb.conf: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse and update smb.conf
|
||||||
|
lines := strings.Split(string(smbContent), "\n")
|
||||||
|
var newLines []string
|
||||||
|
inShare := false
|
||||||
|
shareStart := -1
|
||||||
|
|
||||||
|
for i, line := range lines {
|
||||||
|
trimmed := strings.TrimSpace(line)
|
||||||
|
|
||||||
|
// Check if we're entering our share section
|
||||||
|
if strings.HasPrefix(trimmed, "[") && strings.HasSuffix(trimmed, "]") {
|
||||||
|
sectionName := trimmed[1 : len(trimmed)-1]
|
||||||
|
if sectionName == shareName {
|
||||||
|
inShare = true
|
||||||
|
shareStart = i
|
||||||
|
continue
|
||||||
|
} else if inShare {
|
||||||
|
// We've left our share section, insert the share config here
|
||||||
|
newLines = append(newLines, s.buildSMBShareConfig(shareName, path, comment, guestOK, readOnly, browseable))
|
||||||
|
inShare = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if inShare {
|
||||||
|
// Skip lines until we find the next section or end of file
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
newLines = append(newLines, line)
|
||||||
|
}
|
||||||
|
|
||||||
|
// If we were still in the share at the end, add it
|
||||||
|
if inShare {
|
||||||
|
newLines = append(newLines, s.buildSMBShareConfig(shareName, path, comment, guestOK, readOnly, browseable))
|
||||||
|
} else if shareStart == -1 {
|
||||||
|
// Share doesn't exist, add it at the end
|
||||||
|
newLines = append(newLines, "")
|
||||||
|
newLines = append(newLines, s.buildSMBShareConfig(shareName, path, comment, guestOK, readOnly, browseable))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write back to file
|
||||||
|
newContent := strings.Join(newLines, "\n")
|
||||||
|
if err := os.WriteFile(smbConfPath, []byte(newContent), 0644); err != nil {
|
||||||
|
return fmt.Errorf("failed to write smb.conf: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reload Samba
|
||||||
|
cmd := exec.CommandContext(ctx, "sudo", "systemctl", "reload", "smbd")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
// Try restart if reload fails
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "systemctl", "restart", "smbd")
|
||||||
|
if output2, err2 := cmd.CombinedOutput(); err2 != nil {
|
||||||
|
return fmt.Errorf("failed to reload/restart smbd: %s / %s: %w", string(output), string(output2), err2)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("SMB share applied", "share_name", shareName, "path", path)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// buildSMBShareConfig builds the SMB share configuration block
|
||||||
|
func (s *Service) buildSMBShareConfig(shareName, path, comment string, guestOK, readOnly, browseable bool) string {
|
||||||
|
var config []string
|
||||||
|
config = append(config, fmt.Sprintf("[%s]", shareName))
|
||||||
|
if comment != "" {
|
||||||
|
config = append(config, fmt.Sprintf(" comment = %s", comment))
|
||||||
|
}
|
||||||
|
config = append(config, fmt.Sprintf(" path = %s", path))
|
||||||
|
if guestOK {
|
||||||
|
config = append(config, " guest ok = yes")
|
||||||
|
} else {
|
||||||
|
config = append(config, " guest ok = no")
|
||||||
|
}
|
||||||
|
if readOnly {
|
||||||
|
config = append(config, " read only = yes")
|
||||||
|
} else {
|
||||||
|
config = append(config, " read only = no")
|
||||||
|
}
|
||||||
|
if browseable {
|
||||||
|
config = append(config, " browseable = yes")
|
||||||
|
} else {
|
||||||
|
config = append(config, " browseable = no")
|
||||||
|
}
|
||||||
|
return strings.Join(config, "\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
// removeSMBShare removes an SMB share from /etc/samba/smb.conf
|
||||||
|
func (s *Service) removeSMBShare(ctx context.Context, shareName string) error {
|
||||||
|
if shareName == "" {
|
||||||
|
return nil // Nothing to remove
|
||||||
|
}
|
||||||
|
|
||||||
|
smbConfPath := "/etc/samba/smb.conf"
|
||||||
|
smbContent, err := os.ReadFile(smbConfPath)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil // File doesn't exist, nothing to remove
|
||||||
|
}
|
||||||
|
return fmt.Errorf("failed to read smb.conf: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(string(smbContent), "\n")
|
||||||
|
var newLines []string
|
||||||
|
inShare := false
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
trimmed := strings.TrimSpace(line)
|
||||||
|
|
||||||
|
// Check if we're entering our share section
|
||||||
|
if strings.HasPrefix(trimmed, "[") && strings.HasSuffix(trimmed, "]") {
|
||||||
|
sectionName := trimmed[1 : len(trimmed)-1]
|
||||||
|
if sectionName == shareName {
|
||||||
|
inShare = true
|
||||||
|
continue
|
||||||
|
} else if inShare {
|
||||||
|
// We've left our share section
|
||||||
|
inShare = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if inShare {
|
||||||
|
// Skip lines in this share section
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
newLines = append(newLines, line)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write back to file
|
||||||
|
newContent := strings.Join(newLines, "\n")
|
||||||
|
if err := os.WriteFile(smbConfPath, []byte(newContent), 0644); err != nil {
|
||||||
|
return fmt.Errorf("failed to write smb.conf: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reload Samba
|
||||||
|
cmd := exec.CommandContext(ctx, "sudo", "systemctl", "reload", "smbd")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
// Try restart if reload fails
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "systemctl", "restart", "smbd")
|
||||||
|
if output2, err2 := cmd.CombinedOutput(); err2 != nil {
|
||||||
|
return fmt.Errorf("failed to reload/restart smbd: %s / %s: %w", string(output), string(output2), err2)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("SMB share removed", "share_name", shareName)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -195,7 +195,7 @@ func (s *DiskService) getZFSPoolForDisk(ctx context.Context, devicePath string)
|
|||||||
deviceName := strings.TrimPrefix(devicePath, "/dev/")
|
deviceName := strings.TrimPrefix(devicePath, "/dev/")
|
||||||
|
|
||||||
// Get all ZFS pools
|
// Get all ZFS pools
|
||||||
cmd := exec.CommandContext(ctx, "zpool", "list", "-H", "-o", "name")
|
cmd := exec.CommandContext(ctx, "sudo", "zpool", "list", "-H", "-o", "name")
|
||||||
output, err := cmd.Output()
|
output, err := cmd.Output()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return ""
|
return ""
|
||||||
@@ -208,7 +208,7 @@ func (s *DiskService) getZFSPoolForDisk(ctx context.Context, devicePath string)
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Check pool status for this device
|
// Check pool status for this device
|
||||||
statusCmd := exec.CommandContext(ctx, "zpool", "status", poolName)
|
statusCmd := exec.CommandContext(ctx, "sudo", "zpool", "status", poolName)
|
||||||
statusOutput, err := statusCmd.Output()
|
statusOutput, err := statusCmd.Output()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -15,14 +15,17 @@ import (
|
|||||||
|
|
||||||
// Handler handles storage-related API requests
|
// Handler handles storage-related API requests
|
||||||
type Handler struct {
|
type Handler struct {
|
||||||
diskService *DiskService
|
diskService *DiskService
|
||||||
lvmService *LVMService
|
lvmService *LVMService
|
||||||
zfsService *ZFSService
|
zfsService *ZFSService
|
||||||
arcService *ARCService
|
snapshotService *SnapshotService
|
||||||
taskEngine *tasks.Engine
|
snapshotScheduleService *SnapshotScheduleService
|
||||||
db *database.DB
|
replicationService *ReplicationService
|
||||||
logger *logger.Logger
|
arcService *ARCService
|
||||||
cache *cache.Cache // Cache for invalidation
|
taskEngine *tasks.Engine
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
cache *cache.Cache // Cache for invalidation
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetCache sets the cache instance for cache invalidation
|
// SetCache sets the cache instance for cache invalidation
|
||||||
@@ -32,14 +35,18 @@ func (h *Handler) SetCache(c *cache.Cache) {
|
|||||||
|
|
||||||
// NewHandler creates a new storage handler
|
// NewHandler creates a new storage handler
|
||||||
func NewHandler(db *database.DB, log *logger.Logger) *Handler {
|
func NewHandler(db *database.DB, log *logger.Logger) *Handler {
|
||||||
|
snapshotService := NewSnapshotService(db, log)
|
||||||
return &Handler{
|
return &Handler{
|
||||||
diskService: NewDiskService(db, log),
|
diskService: NewDiskService(db, log),
|
||||||
lvmService: NewLVMService(db, log),
|
lvmService: NewLVMService(db, log),
|
||||||
zfsService: NewZFSService(db, log),
|
zfsService: NewZFSService(db, log),
|
||||||
arcService: NewARCService(log),
|
snapshotService: snapshotService,
|
||||||
taskEngine: tasks.NewEngine(db, log),
|
snapshotScheduleService: NewSnapshotScheduleService(db, log, snapshotService),
|
||||||
db: db,
|
replicationService: NewReplicationService(db, log),
|
||||||
logger: log,
|
arcService: NewARCService(log),
|
||||||
|
taskEngine: tasks.NewEngine(db, log),
|
||||||
|
db: db,
|
||||||
|
logger: log,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -509,3 +516,417 @@ func (h *Handler) GetARCStats(c *gin.Context) {
|
|||||||
|
|
||||||
c.JSON(http.StatusOK, stats)
|
c.JSON(http.StatusOK, stats)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ListSnapshots lists all snapshots, optionally filtered by dataset
|
||||||
|
func (h *Handler) ListSnapshots(c *gin.Context) {
|
||||||
|
datasetFilter := c.DefaultQuery("dataset", "")
|
||||||
|
|
||||||
|
snapshots, err := h.snapshotService.ListSnapshots(c.Request.Context(), datasetFilter)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list snapshots", "error", err, "dataset", datasetFilter)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list snapshots: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"snapshots": snapshots})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateSnapshotRequest represents a request to create a snapshot
|
||||||
|
type CreateSnapshotRequest struct {
|
||||||
|
Dataset string `json:"dataset" binding:"required"`
|
||||||
|
Name string `json:"name" binding:"required"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateSnapshot creates a new snapshot
|
||||||
|
func (h *Handler) CreateSnapshot(c *gin.Context) {
|
||||||
|
var req CreateSnapshotRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create snapshot request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.snapshotService.CreateSnapshot(c.Request.Context(), req.Dataset, req.Name, req.Recursive); err != nil {
|
||||||
|
h.logger.Error("Failed to create snapshot", "error", err, "dataset", req.Dataset, "name", req.Name)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create snapshot: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, gin.H{"message": "snapshot created successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSnapshot deletes a snapshot
|
||||||
|
func (h *Handler) DeleteSnapshot(c *gin.Context) {
|
||||||
|
snapshotName := c.Param("name")
|
||||||
|
recursive := c.DefaultQuery("recursive", "false") == "true"
|
||||||
|
|
||||||
|
if err := h.snapshotService.DeleteSnapshot(c.Request.Context(), snapshotName, recursive); err != nil {
|
||||||
|
h.logger.Error("Failed to delete snapshot", "error", err, "snapshot", snapshotName)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete snapshot: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "snapshot deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// RollbackSnapshotRequest represents a request to rollback to a snapshot
|
||||||
|
type RollbackSnapshotRequest struct {
|
||||||
|
Force bool `json:"force"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// RollbackSnapshot rolls back a dataset to a snapshot
|
||||||
|
func (h *Handler) RollbackSnapshot(c *gin.Context) {
|
||||||
|
snapshotName := c.Param("name")
|
||||||
|
|
||||||
|
var req RollbackSnapshotRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
// Default to false if not provided
|
||||||
|
req.Force = false
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.snapshotService.RollbackSnapshot(c.Request.Context(), snapshotName, req.Force); err != nil {
|
||||||
|
h.logger.Error("Failed to rollback snapshot", "error", err, "snapshot", snapshotName)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to rollback snapshot: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "snapshot rollback completed successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CloneSnapshotRequest represents a request to clone a snapshot
|
||||||
|
type CloneSnapshotRequest struct {
|
||||||
|
CloneName string `json:"clone_name" binding:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CloneSnapshot clones a snapshot to a new dataset
|
||||||
|
func (h *Handler) CloneSnapshot(c *gin.Context) {
|
||||||
|
snapshotName := c.Param("name")
|
||||||
|
|
||||||
|
var req CloneSnapshotRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid clone snapshot request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.snapshotService.CloneSnapshot(c.Request.Context(), snapshotName, req.CloneName); err != nil {
|
||||||
|
h.logger.Error("Failed to clone snapshot", "error", err, "snapshot", snapshotName, "clone", req.CloneName)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to clone snapshot: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, gin.H{"message": "snapshot cloned successfully", "clone_name": req.CloneName})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSnapshotSchedules lists all snapshot schedules
|
||||||
|
func (h *Handler) ListSnapshotSchedules(c *gin.Context) {
|
||||||
|
schedules, err := h.snapshotScheduleService.ListSchedules(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list snapshot schedules", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list snapshot schedules: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"schedules": schedules})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetSnapshotSchedule retrieves a snapshot schedule by ID
|
||||||
|
func (h *Handler) GetSnapshotSchedule(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
|
||||||
|
schedule, err := h.snapshotScheduleService.GetSchedule(c.Request.Context(), id)
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "schedule not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "schedule not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to get snapshot schedule", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get snapshot schedule: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, schedule)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateSnapshotScheduleRequest represents a request to create a snapshot schedule
|
||||||
|
type CreateSnapshotScheduleRequest struct {
|
||||||
|
Name string `json:"name" binding:"required"`
|
||||||
|
Dataset string `json:"dataset" binding:"required"`
|
||||||
|
SnapshotNameTemplate string `json:"snapshot_name_template" binding:"required"`
|
||||||
|
ScheduleType string `json:"schedule_type" binding:"required"`
|
||||||
|
ScheduleConfig map[string]interface{} `json:"schedule_config" binding:"required"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
RetentionCount *int `json:"retention_count"`
|
||||||
|
RetentionDays *int `json:"retention_days"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateSnapshotSchedule creates a new snapshot schedule
|
||||||
|
func (h *Handler) CreateSnapshotSchedule(c *gin.Context) {
|
||||||
|
var req CreateSnapshotScheduleRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create snapshot schedule request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
userID, _ := c.Get("user_id")
|
||||||
|
userIDStr := ""
|
||||||
|
if userID != nil {
|
||||||
|
userIDStr = userID.(string)
|
||||||
|
}
|
||||||
|
|
||||||
|
schedule, err := h.snapshotScheduleService.CreateSchedule(c.Request.Context(), &CreateScheduleRequest{
|
||||||
|
Name: req.Name,
|
||||||
|
Dataset: req.Dataset,
|
||||||
|
SnapshotNameTemplate: req.SnapshotNameTemplate,
|
||||||
|
ScheduleType: req.ScheduleType,
|
||||||
|
ScheduleConfig: req.ScheduleConfig,
|
||||||
|
Recursive: req.Recursive,
|
||||||
|
RetentionCount: req.RetentionCount,
|
||||||
|
RetentionDays: req.RetentionDays,
|
||||||
|
}, userIDStr)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to create snapshot schedule", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create snapshot schedule: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, schedule)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateSnapshotSchedule updates an existing snapshot schedule
|
||||||
|
func (h *Handler) UpdateSnapshotSchedule(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
|
||||||
|
var req CreateSnapshotScheduleRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid update snapshot schedule request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
schedule, err := h.snapshotScheduleService.UpdateSchedule(c.Request.Context(), id, &CreateScheduleRequest{
|
||||||
|
Name: req.Name,
|
||||||
|
Dataset: req.Dataset,
|
||||||
|
SnapshotNameTemplate: req.SnapshotNameTemplate,
|
||||||
|
ScheduleType: req.ScheduleType,
|
||||||
|
ScheduleConfig: req.ScheduleConfig,
|
||||||
|
Recursive: req.Recursive,
|
||||||
|
RetentionCount: req.RetentionCount,
|
||||||
|
RetentionDays: req.RetentionDays,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "schedule not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "schedule not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to update snapshot schedule", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update snapshot schedule: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, schedule)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSnapshotSchedule deletes a snapshot schedule
|
||||||
|
func (h *Handler) DeleteSnapshotSchedule(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
|
||||||
|
if err := h.snapshotScheduleService.DeleteSchedule(c.Request.Context(), id); err != nil {
|
||||||
|
if err.Error() == "schedule not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "schedule not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to delete snapshot schedule", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete snapshot schedule: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "snapshot schedule deleted successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ToggleSnapshotSchedule enables or disables a snapshot schedule
|
||||||
|
func (h *Handler) ToggleSnapshotSchedule(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
|
||||||
|
var req struct {
|
||||||
|
Enabled bool `json:"enabled" binding:"required"`
|
||||||
|
}
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid toggle snapshot schedule request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.snapshotScheduleService.ToggleSchedule(c.Request.Context(), id, req.Enabled); err != nil {
|
||||||
|
if err.Error() == "schedule not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "schedule not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to toggle snapshot schedule", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to toggle snapshot schedule: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "snapshot schedule toggled successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListReplicationTasks lists all replication tasks
|
||||||
|
func (h *Handler) ListReplicationTasks(c *gin.Context) {
|
||||||
|
direction := c.Query("direction") // Optional filter: "outbound" or "inbound"
|
||||||
|
tasks, err := h.replicationService.ListReplicationTasks(c.Request.Context(), direction)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to list replication tasks", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list replication tasks: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, gin.H{"tasks": tasks})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetReplicationTask retrieves a replication task by ID
|
||||||
|
func (h *Handler) GetReplicationTask(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
task, err := h.replicationService.GetReplicationTask(c.Request.Context(), id)
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "replication task not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "replication task not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to get replication task", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get replication task: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, task)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateReplicationTaskRequest represents a request to create a replication task
|
||||||
|
type CreateReplicationTaskRequest struct {
|
||||||
|
Name string `json:"name" binding:"required"`
|
||||||
|
Direction string `json:"direction" binding:"required"`
|
||||||
|
SourceDataset *string `json:"source_dataset"`
|
||||||
|
TargetHost *string `json:"target_host"`
|
||||||
|
TargetPort *int `json:"target_port"`
|
||||||
|
TargetUser *string `json:"target_user"`
|
||||||
|
TargetDataset *string `json:"target_dataset"`
|
||||||
|
TargetSSHKeyPath *string `json:"target_ssh_key_path"`
|
||||||
|
SourceHost *string `json:"source_host"`
|
||||||
|
SourcePort *int `json:"source_port"`
|
||||||
|
SourceUser *string `json:"source_user"`
|
||||||
|
LocalDataset *string `json:"local_dataset"`
|
||||||
|
ScheduleType *string `json:"schedule_type"`
|
||||||
|
ScheduleConfig map[string]interface{} `json:"schedule_config"`
|
||||||
|
Compression string `json:"compression"`
|
||||||
|
Encryption bool `json:"encryption"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
Incremental bool `json:"incremental"`
|
||||||
|
AutoSnapshot bool `json:"auto_snapshot"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateReplicationTask creates a new replication task
|
||||||
|
func (h *Handler) CreateReplicationTask(c *gin.Context) {
|
||||||
|
var req CreateReplicationTaskRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid create replication task request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
userID, _ := c.Get("user_id")
|
||||||
|
userIDStr := ""
|
||||||
|
if userID != nil {
|
||||||
|
userIDStr = userID.(string)
|
||||||
|
}
|
||||||
|
|
||||||
|
task, err := h.replicationService.CreateReplicationTask(c.Request.Context(), &CreateReplicationRequest{
|
||||||
|
Name: req.Name,
|
||||||
|
Direction: req.Direction,
|
||||||
|
SourceDataset: req.SourceDataset,
|
||||||
|
TargetHost: req.TargetHost,
|
||||||
|
TargetPort: req.TargetPort,
|
||||||
|
TargetUser: req.TargetUser,
|
||||||
|
TargetDataset: req.TargetDataset,
|
||||||
|
TargetSSHKeyPath: req.TargetSSHKeyPath,
|
||||||
|
SourceHost: req.SourceHost,
|
||||||
|
SourcePort: req.SourcePort,
|
||||||
|
SourceUser: req.SourceUser,
|
||||||
|
LocalDataset: req.LocalDataset,
|
||||||
|
ScheduleType: req.ScheduleType,
|
||||||
|
ScheduleConfig: req.ScheduleConfig,
|
||||||
|
Compression: req.Compression,
|
||||||
|
Encryption: req.Encryption,
|
||||||
|
Recursive: req.Recursive,
|
||||||
|
Incremental: req.Incremental,
|
||||||
|
AutoSnapshot: req.AutoSnapshot,
|
||||||
|
Enabled: req.Enabled,
|
||||||
|
}, userIDStr)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to create replication task", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create replication task: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusCreated, task)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateReplicationTask updates an existing replication task
|
||||||
|
func (h *Handler) UpdateReplicationTask(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
var req CreateReplicationTaskRequest
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid update replication task request", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
task, err := h.replicationService.UpdateReplicationTask(c.Request.Context(), id, &CreateReplicationRequest{
|
||||||
|
Name: req.Name,
|
||||||
|
Direction: req.Direction,
|
||||||
|
SourceDataset: req.SourceDataset,
|
||||||
|
TargetHost: req.TargetHost,
|
||||||
|
TargetPort: req.TargetPort,
|
||||||
|
TargetUser: req.TargetUser,
|
||||||
|
TargetDataset: req.TargetDataset,
|
||||||
|
TargetSSHKeyPath: req.TargetSSHKeyPath,
|
||||||
|
SourceHost: req.SourceHost,
|
||||||
|
SourcePort: req.SourcePort,
|
||||||
|
SourceUser: req.SourceUser,
|
||||||
|
LocalDataset: req.LocalDataset,
|
||||||
|
ScheduleType: req.ScheduleType,
|
||||||
|
ScheduleConfig: req.ScheduleConfig,
|
||||||
|
Compression: req.Compression,
|
||||||
|
Encryption: req.Encryption,
|
||||||
|
Recursive: req.Recursive,
|
||||||
|
Incremental: req.Incremental,
|
||||||
|
AutoSnapshot: req.AutoSnapshot,
|
||||||
|
Enabled: req.Enabled,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
if err.Error() == "replication task not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "replication task not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to update replication task", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update replication task: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, task)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteReplicationTask deletes a replication task
|
||||||
|
func (h *Handler) DeleteReplicationTask(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
if err := h.replicationService.DeleteReplicationTask(c.Request.Context(), id); err != nil {
|
||||||
|
if err.Error() == "replication task not found" {
|
||||||
|
c.JSON(http.StatusNotFound, gin.H{"error": "replication task not found"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.logger.Error("Failed to delete replication task", "error", err, "id", id)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete replication task: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "replication task deleted successfully"})
|
||||||
|
}
|
||||||
|
|||||||
553
backend/internal/storage/replication.go
Normal file
553
backend/internal/storage/replication.go
Normal file
@@ -0,0 +1,553 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/google/uuid"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ReplicationService handles ZFS replication task management
|
||||||
|
type ReplicationService struct {
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewReplicationService creates a new replication service
|
||||||
|
func NewReplicationService(db *database.DB, log *logger.Logger) *ReplicationService {
|
||||||
|
return &ReplicationService{
|
||||||
|
db: db,
|
||||||
|
logger: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReplicationTask represents a ZFS replication task
|
||||||
|
type ReplicationTask struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Direction string `json:"direction"` // "outbound" or "inbound"
|
||||||
|
SourceDataset *string `json:"source_dataset,omitempty"`
|
||||||
|
TargetHost *string `json:"target_host,omitempty"`
|
||||||
|
TargetPort *int `json:"target_port,omitempty"`
|
||||||
|
TargetUser *string `json:"target_user,omitempty"`
|
||||||
|
TargetDataset *string `json:"target_dataset,omitempty"`
|
||||||
|
TargetSSHKeyPath *string `json:"target_ssh_key_path,omitempty"`
|
||||||
|
SourceHost *string `json:"source_host,omitempty"`
|
||||||
|
SourcePort *int `json:"source_port,omitempty"`
|
||||||
|
SourceUser *string `json:"source_user,omitempty"`
|
||||||
|
LocalDataset *string `json:"local_dataset,omitempty"`
|
||||||
|
ScheduleType *string `json:"schedule_type,omitempty"`
|
||||||
|
ScheduleConfig map[string]interface{} `json:"schedule_config,omitempty"`
|
||||||
|
Compression string `json:"compression"`
|
||||||
|
Encryption bool `json:"encryption"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
Incremental bool `json:"incremental"`
|
||||||
|
AutoSnapshot bool `json:"auto_snapshot"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
LastRunAt *time.Time `json:"last_run_at,omitempty"`
|
||||||
|
LastRunStatus *string `json:"last_run_status,omitempty"`
|
||||||
|
LastRunError *string `json:"last_run_error,omitempty"`
|
||||||
|
NextRunAt *time.Time `json:"next_run_at,omitempty"`
|
||||||
|
LastSnapshotSent *string `json:"last_snapshot_sent,omitempty"`
|
||||||
|
LastSnapshotReceived *string `json:"last_snapshot_received,omitempty"`
|
||||||
|
TotalRuns int `json:"total_runs"`
|
||||||
|
SuccessfulRuns int `json:"successful_runs"`
|
||||||
|
FailedRuns int `json:"failed_runs"`
|
||||||
|
BytesSent int64 `json:"bytes_sent"`
|
||||||
|
BytesReceived int64 `json:"bytes_received"`
|
||||||
|
CreatedBy string `json:"created_by,omitempty"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
UpdatedAt time.Time `json:"updated_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateReplicationRequest represents a request to create a replication task
|
||||||
|
type CreateReplicationRequest struct {
|
||||||
|
Name string `json:"name" binding:"required"`
|
||||||
|
Direction string `json:"direction" binding:"required"` // "outbound" or "inbound"
|
||||||
|
SourceDataset *string `json:"source_dataset"`
|
||||||
|
TargetHost *string `json:"target_host"`
|
||||||
|
TargetPort *int `json:"target_port"`
|
||||||
|
TargetUser *string `json:"target_user"`
|
||||||
|
TargetDataset *string `json:"target_dataset"`
|
||||||
|
TargetSSHKeyPath *string `json:"target_ssh_key_path"`
|
||||||
|
SourceHost *string `json:"source_host"`
|
||||||
|
SourcePort *int `json:"source_port"`
|
||||||
|
SourceUser *string `json:"source_user"`
|
||||||
|
LocalDataset *string `json:"local_dataset"`
|
||||||
|
ScheduleType *string `json:"schedule_type"`
|
||||||
|
ScheduleConfig map[string]interface{} `json:"schedule_config"`
|
||||||
|
Compression string `json:"compression"`
|
||||||
|
Encryption bool `json:"encryption"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
Incremental bool `json:"incremental"`
|
||||||
|
AutoSnapshot bool `json:"auto_snapshot"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListReplicationTasks lists all replication tasks, optionally filtered by direction
|
||||||
|
func (s *ReplicationService) ListReplicationTasks(ctx context.Context, directionFilter string) ([]*ReplicationTask, error) {
|
||||||
|
var query string
|
||||||
|
var args []interface{}
|
||||||
|
|
||||||
|
if directionFilter != "" {
|
||||||
|
query = `
|
||||||
|
SELECT id, name, direction, source_dataset, target_host, target_port, target_user, target_dataset,
|
||||||
|
target_ssh_key_path, source_host, source_port, source_user, local_dataset,
|
||||||
|
schedule_type, schedule_config, compression, encryption, recursive, incremental,
|
||||||
|
auto_snapshot, enabled, status, last_run_at, last_run_status, last_run_error,
|
||||||
|
next_run_at, last_snapshot_sent, last_snapshot_received,
|
||||||
|
total_runs, successful_runs, failed_runs, bytes_sent, bytes_received,
|
||||||
|
created_by, created_at, updated_at
|
||||||
|
FROM replication_tasks
|
||||||
|
WHERE direction = $1
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
`
|
||||||
|
args = []interface{}{directionFilter}
|
||||||
|
} else {
|
||||||
|
query = `
|
||||||
|
SELECT id, name, direction, source_dataset, target_host, target_port, target_user, target_dataset,
|
||||||
|
target_ssh_key_path, source_host, source_port, source_user, local_dataset,
|
||||||
|
schedule_type, schedule_config, compression, encryption, recursive, incremental,
|
||||||
|
auto_snapshot, enabled, status, last_run_at, last_run_status, last_run_error,
|
||||||
|
next_run_at, last_snapshot_sent, last_snapshot_received,
|
||||||
|
total_runs, successful_runs, failed_runs, bytes_sent, bytes_received,
|
||||||
|
created_by, created_at, updated_at
|
||||||
|
FROM replication_tasks
|
||||||
|
ORDER BY direction, created_at DESC
|
||||||
|
`
|
||||||
|
args = []interface{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := s.db.QueryContext(ctx, query, args...)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to query replication tasks: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var tasks []*ReplicationTask
|
||||||
|
for rows.Next() {
|
||||||
|
task, err := s.scanReplicationTask(rows)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to scan replication task", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
tasks = append(tasks, task)
|
||||||
|
}
|
||||||
|
|
||||||
|
return tasks, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetReplicationTask retrieves a replication task by ID
|
||||||
|
func (s *ReplicationService) GetReplicationTask(ctx context.Context, id string) (*ReplicationTask, error) {
|
||||||
|
query := `
|
||||||
|
SELECT id, name, direction, source_dataset, target_host, target_port, target_user, target_dataset,
|
||||||
|
target_ssh_key_path, source_host, source_port, source_user, local_dataset,
|
||||||
|
schedule_type, schedule_config, compression, encryption, recursive, incremental,
|
||||||
|
auto_snapshot, enabled, status, last_run_at, last_run_status, last_run_error,
|
||||||
|
next_run_at, last_snapshot_sent, last_snapshot_received,
|
||||||
|
total_runs, successful_runs, failed_runs, bytes_sent, bytes_received,
|
||||||
|
created_by, created_at, updated_at
|
||||||
|
FROM replication_tasks
|
||||||
|
WHERE id = $1
|
||||||
|
`
|
||||||
|
|
||||||
|
row := s.db.QueryRowContext(ctx, query, id)
|
||||||
|
task, err := s.scanReplicationTaskRow(row)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
return nil, fmt.Errorf("replication task not found")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to get replication task: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return task, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateReplicationTask creates a new replication task
|
||||||
|
func (s *ReplicationService) CreateReplicationTask(ctx context.Context, req *CreateReplicationRequest, createdBy string) (*ReplicationTask, error) {
|
||||||
|
id := uuid.New().String()
|
||||||
|
|
||||||
|
// Validate direction-specific fields
|
||||||
|
if req.Direction == "outbound" {
|
||||||
|
if req.SourceDataset == nil || req.TargetHost == nil || req.TargetDataset == nil {
|
||||||
|
return nil, fmt.Errorf("outbound replication requires source_dataset, target_host, and target_dataset")
|
||||||
|
}
|
||||||
|
} else if req.Direction == "inbound" {
|
||||||
|
if req.SourceHost == nil || req.SourceDataset == nil || req.LocalDataset == nil {
|
||||||
|
return nil, fmt.Errorf("inbound replication requires source_host, source_dataset, and local_dataset")
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
return nil, fmt.Errorf("invalid direction: must be 'outbound' or 'inbound'")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set defaults
|
||||||
|
if req.Compression == "" {
|
||||||
|
req.Compression = "lz4"
|
||||||
|
}
|
||||||
|
if req.TargetPort == nil {
|
||||||
|
defaultPort := 22
|
||||||
|
req.TargetPort = &defaultPort
|
||||||
|
}
|
||||||
|
if req.SourcePort == nil {
|
||||||
|
defaultPort := 22
|
||||||
|
req.SourcePort = &defaultPort
|
||||||
|
}
|
||||||
|
if req.TargetUser == nil {
|
||||||
|
defaultUser := "root"
|
||||||
|
req.TargetUser = &defaultUser
|
||||||
|
}
|
||||||
|
if req.SourceUser == nil {
|
||||||
|
defaultUser := "root"
|
||||||
|
req.SourceUser = &defaultUser
|
||||||
|
}
|
||||||
|
|
||||||
|
// Marshal schedule config to JSON
|
||||||
|
var scheduleConfigJSON sql.NullString
|
||||||
|
if req.ScheduleConfig != nil {
|
||||||
|
configJSON, err := json.Marshal(req.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to marshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
scheduleConfigJSON = sql.NullString{String: string(configJSON), Valid: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
query := `
|
||||||
|
INSERT INTO replication_tasks (
|
||||||
|
id, name, direction, source_dataset, target_host, target_port, target_user, target_dataset,
|
||||||
|
target_ssh_key_path, source_host, source_port, source_user, local_dataset,
|
||||||
|
schedule_type, schedule_config, compression, encryption, recursive, incremental,
|
||||||
|
auto_snapshot, enabled, status, created_by, created_at, updated_at
|
||||||
|
) VALUES (
|
||||||
|
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, NOW(), NOW()
|
||||||
|
)
|
||||||
|
RETURNING id, name, direction, source_dataset, target_host, target_port, target_user, target_dataset,
|
||||||
|
target_ssh_key_path, source_host, source_port, source_user, local_dataset,
|
||||||
|
schedule_type, schedule_config, compression, encryption, recursive, incremental,
|
||||||
|
auto_snapshot, enabled, status, last_run_at, last_run_status, last_run_error,
|
||||||
|
next_run_at, last_snapshot_sent, last_snapshot_received,
|
||||||
|
total_runs, successful_runs, failed_runs, bytes_sent, bytes_received,
|
||||||
|
created_by, created_at, updated_at
|
||||||
|
`
|
||||||
|
|
||||||
|
var scheduleTypeStr sql.NullString
|
||||||
|
if req.ScheduleType != nil {
|
||||||
|
scheduleTypeStr = sql.NullString{String: *req.ScheduleType, Valid: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
row := s.db.QueryRowContext(ctx, query,
|
||||||
|
id, req.Name, req.Direction,
|
||||||
|
req.SourceDataset, req.TargetHost, req.TargetPort, req.TargetUser, req.TargetDataset,
|
||||||
|
req.TargetSSHKeyPath, req.SourceHost, req.SourcePort, req.SourceUser, req.LocalDataset,
|
||||||
|
scheduleTypeStr, scheduleConfigJSON, req.Compression, req.Encryption, req.Recursive, req.Incremental,
|
||||||
|
req.AutoSnapshot, req.Enabled, "idle", createdBy,
|
||||||
|
)
|
||||||
|
|
||||||
|
task, err := s.scanReplicationTaskRow(row)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create replication task: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Replication task created", "id", id, "name", req.Name, "direction", req.Direction)
|
||||||
|
return task, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateReplicationTask updates an existing replication task
|
||||||
|
func (s *ReplicationService) UpdateReplicationTask(ctx context.Context, id string, req *CreateReplicationRequest) (*ReplicationTask, error) {
|
||||||
|
// Validate direction-specific fields
|
||||||
|
if req.Direction == "outbound" {
|
||||||
|
if req.SourceDataset == nil || req.TargetHost == nil || req.TargetDataset == nil {
|
||||||
|
return nil, fmt.Errorf("outbound replication requires source_dataset, target_host, and target_dataset")
|
||||||
|
}
|
||||||
|
} else if req.Direction == "inbound" {
|
||||||
|
if req.SourceHost == nil || req.SourceDataset == nil || req.LocalDataset == nil {
|
||||||
|
return nil, fmt.Errorf("inbound replication requires source_host, source_dataset, and local_dataset")
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
return nil, fmt.Errorf("invalid direction: must be 'outbound' or 'inbound'")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set defaults
|
||||||
|
if req.Compression == "" {
|
||||||
|
req.Compression = "lz4"
|
||||||
|
}
|
||||||
|
if req.TargetPort == nil {
|
||||||
|
defaultPort := 22
|
||||||
|
req.TargetPort = &defaultPort
|
||||||
|
}
|
||||||
|
if req.SourcePort == nil {
|
||||||
|
defaultPort := 22
|
||||||
|
req.SourcePort = &defaultPort
|
||||||
|
}
|
||||||
|
if req.TargetUser == nil {
|
||||||
|
defaultUser := "root"
|
||||||
|
req.TargetUser = &defaultUser
|
||||||
|
}
|
||||||
|
if req.SourceUser == nil {
|
||||||
|
defaultUser := "root"
|
||||||
|
req.SourceUser = &defaultUser
|
||||||
|
}
|
||||||
|
|
||||||
|
// Marshal schedule config to JSON
|
||||||
|
var scheduleConfigJSON sql.NullString
|
||||||
|
if req.ScheduleConfig != nil {
|
||||||
|
configJSON, err := json.Marshal(req.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to marshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
scheduleConfigJSON = sql.NullString{String: string(configJSON), Valid: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
var scheduleTypeStr sql.NullString
|
||||||
|
if req.ScheduleType != nil {
|
||||||
|
scheduleTypeStr = sql.NullString{String: *req.ScheduleType, Valid: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
query := `
|
||||||
|
UPDATE replication_tasks
|
||||||
|
SET name = $1, direction = $2, source_dataset = $3, target_host = $4, target_port = $5,
|
||||||
|
target_user = $6, target_dataset = $7, target_ssh_key_path = $8, source_host = $9,
|
||||||
|
source_port = $10, source_user = $11, local_dataset = $12, schedule_type = $13,
|
||||||
|
schedule_config = $14, compression = $15, encryption = $16, recursive = $17,
|
||||||
|
incremental = $18, auto_snapshot = $19, enabled = $20, updated_at = NOW()
|
||||||
|
WHERE id = $21
|
||||||
|
RETURNING id, name, direction, source_dataset, target_host, target_port, target_user, target_dataset,
|
||||||
|
target_ssh_key_path, source_host, source_port, source_user, local_dataset,
|
||||||
|
schedule_type, schedule_config, compression, encryption, recursive, incremental,
|
||||||
|
auto_snapshot, enabled, status, last_run_at, last_run_status, last_run_error,
|
||||||
|
next_run_at, last_snapshot_sent, last_snapshot_received,
|
||||||
|
total_runs, successful_runs, failed_runs, bytes_sent, bytes_received,
|
||||||
|
created_by, created_at, updated_at
|
||||||
|
`
|
||||||
|
|
||||||
|
row := s.db.QueryRowContext(ctx, query,
|
||||||
|
req.Name, req.Direction,
|
||||||
|
req.SourceDataset, req.TargetHost, req.TargetPort, req.TargetUser, req.TargetDataset,
|
||||||
|
req.TargetSSHKeyPath, req.SourceHost, req.SourcePort, req.SourceUser, req.LocalDataset,
|
||||||
|
scheduleTypeStr, scheduleConfigJSON, req.Compression, req.Encryption, req.Recursive, req.Incremental,
|
||||||
|
req.AutoSnapshot, req.Enabled, id,
|
||||||
|
)
|
||||||
|
|
||||||
|
task, err := s.scanReplicationTaskRow(row)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
return nil, fmt.Errorf("replication task not found")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to update replication task: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Replication task updated", "id", id)
|
||||||
|
return task, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteReplicationTask deletes a replication task
|
||||||
|
func (s *ReplicationService) DeleteReplicationTask(ctx context.Context, id string) error {
|
||||||
|
query := `DELETE FROM replication_tasks WHERE id = $1`
|
||||||
|
result, err := s.db.ExecContext(ctx, query, id)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete replication task: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rowsAffected, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to get rows affected: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rowsAffected == 0 {
|
||||||
|
return fmt.Errorf("replication task not found")
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Replication task deleted", "id", id)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanReplicationTaskRow scans a single replication task row
|
||||||
|
func (s *ReplicationService) scanReplicationTaskRow(row *sql.Row) (*ReplicationTask, error) {
|
||||||
|
var task ReplicationTask
|
||||||
|
var sourceDataset, targetHost, targetUser, targetDataset, targetSSHKeyPath sql.NullString
|
||||||
|
var sourceHost, sourceUser, localDataset sql.NullString
|
||||||
|
var targetPort, sourcePort sql.NullInt64
|
||||||
|
var scheduleType, scheduleConfigJSON sql.NullString
|
||||||
|
var lastRunAt, nextRunAt sql.NullTime
|
||||||
|
var lastRunStatus, lastRunError, lastSnapshotSent, lastSnapshotReceived sql.NullString
|
||||||
|
var createdBy sql.NullString
|
||||||
|
|
||||||
|
err := row.Scan(
|
||||||
|
&task.ID, &task.Name, &task.Direction,
|
||||||
|
&sourceDataset, &targetHost, &targetPort, &targetUser, &targetDataset,
|
||||||
|
&targetSSHKeyPath, &sourceHost, &sourcePort, &sourceUser, &localDataset,
|
||||||
|
&scheduleType, &scheduleConfigJSON, &task.Compression, &task.Encryption,
|
||||||
|
&task.Recursive, &task.Incremental, &task.AutoSnapshot, &task.Enabled, &task.Status,
|
||||||
|
&lastRunAt, &lastRunStatus, &lastRunError, &nextRunAt,
|
||||||
|
&lastSnapshotSent, &lastSnapshotReceived,
|
||||||
|
&task.TotalRuns, &task.SuccessfulRuns, &task.FailedRuns,
|
||||||
|
&task.BytesSent, &task.BytesReceived,
|
||||||
|
&createdBy, &task.CreatedAt, &task.UpdatedAt,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle nullable fields
|
||||||
|
if sourceDataset.Valid {
|
||||||
|
task.SourceDataset = &sourceDataset.String
|
||||||
|
}
|
||||||
|
if targetHost.Valid {
|
||||||
|
task.TargetHost = &targetHost.String
|
||||||
|
}
|
||||||
|
if targetPort.Valid {
|
||||||
|
port := int(targetPort.Int64)
|
||||||
|
task.TargetPort = &port
|
||||||
|
}
|
||||||
|
if targetUser.Valid {
|
||||||
|
task.TargetUser = &targetUser.String
|
||||||
|
}
|
||||||
|
if targetDataset.Valid {
|
||||||
|
task.TargetDataset = &targetDataset.String
|
||||||
|
}
|
||||||
|
if targetSSHKeyPath.Valid {
|
||||||
|
task.TargetSSHKeyPath = &targetSSHKeyPath.String
|
||||||
|
}
|
||||||
|
if sourceHost.Valid {
|
||||||
|
task.SourceHost = &sourceHost.String
|
||||||
|
}
|
||||||
|
if sourcePort.Valid {
|
||||||
|
port := int(sourcePort.Int64)
|
||||||
|
task.SourcePort = &port
|
||||||
|
}
|
||||||
|
if sourceUser.Valid {
|
||||||
|
task.SourceUser = &sourceUser.String
|
||||||
|
}
|
||||||
|
if localDataset.Valid {
|
||||||
|
task.LocalDataset = &localDataset.String
|
||||||
|
}
|
||||||
|
if scheduleType.Valid {
|
||||||
|
task.ScheduleType = &scheduleType.String
|
||||||
|
}
|
||||||
|
if scheduleConfigJSON.Valid {
|
||||||
|
if err := json.Unmarshal([]byte(scheduleConfigJSON.String), &task.ScheduleConfig); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to unmarshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if lastRunAt.Valid {
|
||||||
|
task.LastRunAt = &lastRunAt.Time
|
||||||
|
}
|
||||||
|
if lastRunStatus.Valid {
|
||||||
|
task.LastRunStatus = &lastRunStatus.String
|
||||||
|
}
|
||||||
|
if lastRunError.Valid {
|
||||||
|
task.LastRunError = &lastRunError.String
|
||||||
|
}
|
||||||
|
if nextRunAt.Valid {
|
||||||
|
task.NextRunAt = &nextRunAt.Time
|
||||||
|
}
|
||||||
|
if lastSnapshotSent.Valid {
|
||||||
|
task.LastSnapshotSent = &lastSnapshotSent.String
|
||||||
|
}
|
||||||
|
if lastSnapshotReceived.Valid {
|
||||||
|
task.LastSnapshotReceived = &lastSnapshotReceived.String
|
||||||
|
}
|
||||||
|
if createdBy.Valid {
|
||||||
|
task.CreatedBy = createdBy.String
|
||||||
|
}
|
||||||
|
|
||||||
|
return &task, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanReplicationTask scans a replication task from rows
|
||||||
|
func (s *ReplicationService) scanReplicationTask(rows *sql.Rows) (*ReplicationTask, error) {
|
||||||
|
var task ReplicationTask
|
||||||
|
var sourceDataset, targetHost, targetUser, targetDataset, targetSSHKeyPath sql.NullString
|
||||||
|
var sourceHost, sourceUser, localDataset sql.NullString
|
||||||
|
var targetPort, sourcePort sql.NullInt64
|
||||||
|
var scheduleType, scheduleConfigJSON sql.NullString
|
||||||
|
var lastRunAt, nextRunAt sql.NullTime
|
||||||
|
var lastRunStatus, lastRunError, lastSnapshotSent, lastSnapshotReceived sql.NullString
|
||||||
|
var createdBy sql.NullString
|
||||||
|
|
||||||
|
err := rows.Scan(
|
||||||
|
&task.ID, &task.Name, &task.Direction,
|
||||||
|
&sourceDataset, &targetHost, &targetPort, &targetUser, &targetDataset,
|
||||||
|
&targetSSHKeyPath, &sourceHost, &sourcePort, &sourceUser, &localDataset,
|
||||||
|
&scheduleType, &scheduleConfigJSON, &task.Compression, &task.Encryption,
|
||||||
|
&task.Recursive, &task.Incremental, &task.AutoSnapshot, &task.Enabled, &task.Status,
|
||||||
|
&lastRunAt, &lastRunStatus, &lastRunError, &nextRunAt,
|
||||||
|
&lastSnapshotSent, &lastSnapshotReceived,
|
||||||
|
&task.TotalRuns, &task.SuccessfulRuns, &task.FailedRuns,
|
||||||
|
&task.BytesSent, &task.BytesReceived,
|
||||||
|
&createdBy, &task.CreatedAt, &task.UpdatedAt,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle nullable fields (same as scanReplicationTaskRow)
|
||||||
|
if sourceDataset.Valid {
|
||||||
|
task.SourceDataset = &sourceDataset.String
|
||||||
|
}
|
||||||
|
if targetHost.Valid {
|
||||||
|
task.TargetHost = &targetHost.String
|
||||||
|
}
|
||||||
|
if targetPort.Valid {
|
||||||
|
port := int(targetPort.Int64)
|
||||||
|
task.TargetPort = &port
|
||||||
|
}
|
||||||
|
if targetUser.Valid {
|
||||||
|
task.TargetUser = &targetUser.String
|
||||||
|
}
|
||||||
|
if targetDataset.Valid {
|
||||||
|
task.TargetDataset = &targetDataset.String
|
||||||
|
}
|
||||||
|
if targetSSHKeyPath.Valid {
|
||||||
|
task.TargetSSHKeyPath = &targetSSHKeyPath.String
|
||||||
|
}
|
||||||
|
if sourceHost.Valid {
|
||||||
|
task.SourceHost = &sourceHost.String
|
||||||
|
}
|
||||||
|
if sourcePort.Valid {
|
||||||
|
port := int(sourcePort.Int64)
|
||||||
|
task.SourcePort = &port
|
||||||
|
}
|
||||||
|
if sourceUser.Valid {
|
||||||
|
task.SourceUser = &sourceUser.String
|
||||||
|
}
|
||||||
|
if localDataset.Valid {
|
||||||
|
task.LocalDataset = &localDataset.String
|
||||||
|
}
|
||||||
|
if scheduleType.Valid {
|
||||||
|
task.ScheduleType = &scheduleType.String
|
||||||
|
}
|
||||||
|
if scheduleConfigJSON.Valid {
|
||||||
|
if err := json.Unmarshal([]byte(scheduleConfigJSON.String), &task.ScheduleConfig); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to unmarshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if lastRunAt.Valid {
|
||||||
|
task.LastRunAt = &lastRunAt.Time
|
||||||
|
}
|
||||||
|
if lastRunStatus.Valid {
|
||||||
|
task.LastRunStatus = &lastRunStatus.String
|
||||||
|
}
|
||||||
|
if lastRunError.Valid {
|
||||||
|
task.LastRunError = &lastRunError.String
|
||||||
|
}
|
||||||
|
if nextRunAt.Valid {
|
||||||
|
task.NextRunAt = &nextRunAt.Time
|
||||||
|
}
|
||||||
|
if lastSnapshotSent.Valid {
|
||||||
|
task.LastSnapshotSent = &lastSnapshotSent.String
|
||||||
|
}
|
||||||
|
if lastSnapshotReceived.Valid {
|
||||||
|
task.LastSnapshotReceived = &lastSnapshotReceived.String
|
||||||
|
}
|
||||||
|
if createdBy.Valid {
|
||||||
|
task.CreatedBy = createdBy.String
|
||||||
|
}
|
||||||
|
|
||||||
|
return &task, nil
|
||||||
|
}
|
||||||
259
backend/internal/storage/snapshot.go
Normal file
259
backend/internal/storage/snapshot.go
Normal file
@@ -0,0 +1,259 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SnapshotService handles ZFS snapshot operations
|
||||||
|
type SnapshotService struct {
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSnapshotService creates a new snapshot service
|
||||||
|
func NewSnapshotService(db *database.DB, log *logger.Logger) *SnapshotService {
|
||||||
|
return &SnapshotService{
|
||||||
|
db: db,
|
||||||
|
logger: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Snapshot represents a ZFS snapshot
|
||||||
|
type Snapshot struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"` // Full snapshot name (e.g., "pool/dataset@snapshot-name")
|
||||||
|
Dataset string `json:"dataset"` // Dataset name (e.g., "pool/dataset")
|
||||||
|
SnapshotName string `json:"snapshot_name"` // Snapshot name only (e.g., "snapshot-name")
|
||||||
|
Created time.Time `json:"created"`
|
||||||
|
Referenced int64 `json:"referenced"` // Size in bytes
|
||||||
|
Used int64 `json:"used"` // Used space in bytes
|
||||||
|
IsLatest bool `json:"is_latest"` // Whether this is the latest snapshot for the dataset
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSnapshots lists all snapshots, optionally filtered by dataset
|
||||||
|
func (s *SnapshotService) ListSnapshots(ctx context.Context, datasetFilter string) ([]*Snapshot, error) {
|
||||||
|
// Build zfs list command
|
||||||
|
args := []string{"list", "-t", "snapshot", "-H", "-o", "name,creation,referenced,used"}
|
||||||
|
if datasetFilter != "" {
|
||||||
|
// List snapshots for specific dataset
|
||||||
|
args = append(args, datasetFilter)
|
||||||
|
} else {
|
||||||
|
// List all snapshots
|
||||||
|
args = append(args, "-r")
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd := zfsCommand(ctx, args...)
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to list snapshots: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(strings.TrimSpace(string(output)), "\n")
|
||||||
|
var snapshots []*Snapshot
|
||||||
|
|
||||||
|
// Track latest snapshot per dataset
|
||||||
|
latestSnapshots := make(map[string]*Snapshot)
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
fields := strings.Fields(line)
|
||||||
|
if len(fields) < 4 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
fullName := fields[0]
|
||||||
|
creationStr := fields[1]
|
||||||
|
referencedStr := fields[2]
|
||||||
|
usedStr := fields[3]
|
||||||
|
|
||||||
|
// Parse snapshot name (format: pool/dataset@snapshot-name)
|
||||||
|
parts := strings.Split(fullName, "@")
|
||||||
|
if len(parts) != 2 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
dataset := parts[0]
|
||||||
|
snapshotName := parts[1]
|
||||||
|
|
||||||
|
// Parse creation time (Unix timestamp)
|
||||||
|
var creationTime time.Time
|
||||||
|
if timestamp, err := parseZFSUnixTimestamp(creationStr); err == nil {
|
||||||
|
creationTime = timestamp
|
||||||
|
} else {
|
||||||
|
// Fallback to current time if parsing fails
|
||||||
|
creationTime = time.Now()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse sizes
|
||||||
|
referenced := parseSnapshotSize(referencedStr)
|
||||||
|
used := parseSnapshotSize(usedStr)
|
||||||
|
|
||||||
|
snapshot := &Snapshot{
|
||||||
|
ID: fullName,
|
||||||
|
Name: fullName,
|
||||||
|
Dataset: dataset,
|
||||||
|
SnapshotName: snapshotName,
|
||||||
|
Created: creationTime,
|
||||||
|
Referenced: referenced,
|
||||||
|
Used: used,
|
||||||
|
IsLatest: false,
|
||||||
|
}
|
||||||
|
|
||||||
|
snapshots = append(snapshots, snapshot)
|
||||||
|
|
||||||
|
// Track latest snapshot per dataset
|
||||||
|
if latest, exists := latestSnapshots[dataset]; !exists || snapshot.Created.After(latest.Created) {
|
||||||
|
latestSnapshots[dataset] = snapshot
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Mark latest snapshots
|
||||||
|
for _, snapshot := range snapshots {
|
||||||
|
if latest, exists := latestSnapshots[snapshot.Dataset]; exists && latest.ID == snapshot.ID {
|
||||||
|
snapshot.IsLatest = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return snapshots, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateSnapshot creates a new snapshot
|
||||||
|
func (s *SnapshotService) CreateSnapshot(ctx context.Context, dataset, snapshotName string, recursive bool) error {
|
||||||
|
// Validate dataset exists
|
||||||
|
cmd := zfsCommand(ctx, "list", "-H", "-o", "name", dataset)
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
return fmt.Errorf("dataset %s does not exist: %w", dataset, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build snapshot name
|
||||||
|
fullSnapshotName := fmt.Sprintf("%s@%s", dataset, snapshotName)
|
||||||
|
|
||||||
|
// Build command
|
||||||
|
args := []string{"snapshot"}
|
||||||
|
if recursive {
|
||||||
|
args = append(args, "-r")
|
||||||
|
}
|
||||||
|
args = append(args, fullSnapshotName)
|
||||||
|
|
||||||
|
cmd = zfsCommand(ctx, args...)
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to create snapshot: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot created", "snapshot", fullSnapshotName, "dataset", dataset, "recursive", recursive)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSnapshot deletes a snapshot
|
||||||
|
func (s *SnapshotService) DeleteSnapshot(ctx context.Context, snapshotName string, recursive bool) error {
|
||||||
|
// Build command
|
||||||
|
args := []string{"destroy"}
|
||||||
|
if recursive {
|
||||||
|
args = append(args, "-r")
|
||||||
|
}
|
||||||
|
args = append(args, snapshotName)
|
||||||
|
|
||||||
|
cmd := zfsCommand(ctx, args...)
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete snapshot: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot deleted", "snapshot", snapshotName, "recursive", recursive)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RollbackSnapshot rolls back a dataset to a snapshot
|
||||||
|
func (s *SnapshotService) RollbackSnapshot(ctx context.Context, snapshotName string, force bool) error {
|
||||||
|
// Build command
|
||||||
|
args := []string{"rollback"}
|
||||||
|
if force {
|
||||||
|
args = append(args, "-r", "-f")
|
||||||
|
} else {
|
||||||
|
args = append(args, "-r")
|
||||||
|
}
|
||||||
|
args = append(args, snapshotName)
|
||||||
|
|
||||||
|
cmd := zfsCommand(ctx, args...)
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to rollback snapshot: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot rollback completed", "snapshot", snapshotName, "force", force)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CloneSnapshot clones a snapshot to a new dataset
|
||||||
|
func (s *SnapshotService) CloneSnapshot(ctx context.Context, snapshotName, cloneName string) error {
|
||||||
|
// Build command
|
||||||
|
args := []string{"clone", snapshotName, cloneName}
|
||||||
|
|
||||||
|
cmd := zfsCommand(ctx, args...)
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to clone snapshot: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot cloned", "snapshot", snapshotName, "clone", cloneName)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseZFSUnixTimestamp parses ZFS Unix timestamp string
|
||||||
|
func parseZFSUnixTimestamp(timestampStr string) (time.Time, error) {
|
||||||
|
// ZFS returns Unix timestamp as string
|
||||||
|
timestamp, err := time.Parse("20060102150405", timestampStr)
|
||||||
|
if err != nil {
|
||||||
|
// Try parsing as Unix timestamp integer
|
||||||
|
var unixTime int64
|
||||||
|
if _, err := fmt.Sscanf(timestampStr, "%d", &unixTime); err == nil {
|
||||||
|
return time.Unix(unixTime, 0), nil
|
||||||
|
}
|
||||||
|
return time.Time{}, err
|
||||||
|
}
|
||||||
|
return timestamp, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseZFSSize parses ZFS size string (e.g., "1.2M", "4.5G", "850K")
|
||||||
|
// Note: This function is also defined in zfs.go, but we need it here for snapshot parsing
|
||||||
|
func parseSnapshotSize(sizeStr string) int64 {
|
||||||
|
if sizeStr == "-" || sizeStr == "" {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove any whitespace
|
||||||
|
sizeStr = strings.TrimSpace(sizeStr)
|
||||||
|
|
||||||
|
// Parse size with unit
|
||||||
|
var size float64
|
||||||
|
var unit string
|
||||||
|
if _, err := fmt.Sscanf(sizeStr, "%f%s", &size, &unit); err != nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert to bytes
|
||||||
|
unit = strings.ToUpper(unit)
|
||||||
|
switch unit {
|
||||||
|
case "K", "KB":
|
||||||
|
return int64(size * 1024)
|
||||||
|
case "M", "MB":
|
||||||
|
return int64(size * 1024 * 1024)
|
||||||
|
case "G", "GB":
|
||||||
|
return int64(size * 1024 * 1024 * 1024)
|
||||||
|
case "T", "TB":
|
||||||
|
return int64(size * 1024 * 1024 * 1024 * 1024)
|
||||||
|
default:
|
||||||
|
// Assume bytes if no unit
|
||||||
|
return int64(size)
|
||||||
|
}
|
||||||
|
}
|
||||||
606
backend/internal/storage/snapshot_schedule.go
Normal file
606
backend/internal/storage/snapshot_schedule.go
Normal file
@@ -0,0 +1,606 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/google/uuid"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SnapshotScheduleService handles snapshot schedule management
|
||||||
|
type SnapshotScheduleService struct {
|
||||||
|
db *database.DB
|
||||||
|
logger *logger.Logger
|
||||||
|
snapshotService *SnapshotService
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSnapshotScheduleService creates a new snapshot schedule service
|
||||||
|
func NewSnapshotScheduleService(db *database.DB, log *logger.Logger, snapshotService *SnapshotService) *SnapshotScheduleService {
|
||||||
|
return &SnapshotScheduleService{
|
||||||
|
db: db,
|
||||||
|
logger: log,
|
||||||
|
snapshotService: snapshotService,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SnapshotSchedule represents a scheduled snapshot task
|
||||||
|
type SnapshotSchedule struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Dataset string `json:"dataset"`
|
||||||
|
SnapshotNameTemplate string `json:"snapshot_name_template"`
|
||||||
|
ScheduleType string `json:"schedule_type"` // hourly, daily, weekly, monthly, cron
|
||||||
|
ScheduleConfig map[string]interface{} `json:"schedule_config"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
RetentionCount *int `json:"retention_count,omitempty"`
|
||||||
|
RetentionDays *int `json:"retention_days,omitempty"`
|
||||||
|
LastRunAt *time.Time `json:"last_run_at,omitempty"`
|
||||||
|
NextRunAt *time.Time `json:"next_run_at,omitempty"`
|
||||||
|
CreatedBy string `json:"created_by,omitempty"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
UpdatedAt time.Time `json:"updated_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateScheduleRequest represents a request to create a snapshot schedule
|
||||||
|
type CreateScheduleRequest struct {
|
||||||
|
Name string `json:"name" binding:"required"`
|
||||||
|
Dataset string `json:"dataset" binding:"required"`
|
||||||
|
SnapshotNameTemplate string `json:"snapshot_name_template" binding:"required"`
|
||||||
|
ScheduleType string `json:"schedule_type" binding:"required"`
|
||||||
|
ScheduleConfig map[string]interface{} `json:"schedule_config" binding:"required"`
|
||||||
|
Recursive bool `json:"recursive"`
|
||||||
|
RetentionCount *int `json:"retention_count"`
|
||||||
|
RetentionDays *int `json:"retention_days"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSchedules lists all snapshot schedules
|
||||||
|
func (s *SnapshotScheduleService) ListSchedules(ctx context.Context) ([]*SnapshotSchedule, error) {
|
||||||
|
query := `
|
||||||
|
SELECT id, name, dataset, snapshot_name_template, schedule_type, schedule_config,
|
||||||
|
recursive, enabled, retention_count, retention_days,
|
||||||
|
last_run_at, next_run_at, created_by, created_at, updated_at
|
||||||
|
FROM snapshot_schedules
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
`
|
||||||
|
|
||||||
|
rows, err := s.db.QueryContext(ctx, query)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to query schedules: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var schedules []*SnapshotSchedule
|
||||||
|
for rows.Next() {
|
||||||
|
schedule, err := s.scanSchedule(rows)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to scan schedule", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
schedules = append(schedules, schedule)
|
||||||
|
}
|
||||||
|
|
||||||
|
return schedules, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetSchedule retrieves a schedule by ID
|
||||||
|
func (s *SnapshotScheduleService) GetSchedule(ctx context.Context, id string) (*SnapshotSchedule, error) {
|
||||||
|
query := `
|
||||||
|
SELECT id, name, dataset, snapshot_name_template, schedule_type, schedule_config,
|
||||||
|
recursive, enabled, retention_count, retention_days,
|
||||||
|
last_run_at, next_run_at, created_by, created_at, updated_at
|
||||||
|
FROM snapshot_schedules
|
||||||
|
WHERE id = $1
|
||||||
|
`
|
||||||
|
|
||||||
|
row := s.db.QueryRowContext(ctx, query, id)
|
||||||
|
schedule, err := s.scanScheduleRow(row)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
return nil, fmt.Errorf("schedule not found")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to get schedule: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return schedule, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateSchedule creates a new snapshot schedule
|
||||||
|
func (s *SnapshotScheduleService) CreateSchedule(ctx context.Context, req *CreateScheduleRequest, createdBy string) (*SnapshotSchedule, error) {
|
||||||
|
id := uuid.New().String()
|
||||||
|
|
||||||
|
// Calculate next run time
|
||||||
|
nextRunAt, err := s.calculateNextRun(req.ScheduleType, req.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to calculate next run time: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Marshal schedule config to JSON
|
||||||
|
configJSON, err := json.Marshal(req.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to marshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
query := `
|
||||||
|
INSERT INTO snapshot_schedules (
|
||||||
|
id, name, dataset, snapshot_name_template, schedule_type, schedule_config,
|
||||||
|
recursive, enabled, retention_count, retention_days,
|
||||||
|
next_run_at, created_by, created_at, updated_at
|
||||||
|
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, NOW(), NOW())
|
||||||
|
RETURNING id, name, dataset, snapshot_name_template, schedule_type, schedule_config,
|
||||||
|
recursive, enabled, retention_count, retention_days,
|
||||||
|
last_run_at, next_run_at, created_by, created_at, updated_at
|
||||||
|
`
|
||||||
|
|
||||||
|
row := s.db.QueryRowContext(ctx, query,
|
||||||
|
id, req.Name, req.Dataset, req.SnapshotNameTemplate, req.ScheduleType,
|
||||||
|
string(configJSON), req.Recursive, true, req.RetentionCount, req.RetentionDays,
|
||||||
|
nextRunAt, createdBy,
|
||||||
|
)
|
||||||
|
|
||||||
|
schedule, err := s.scanScheduleRow(row)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to create schedule: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot schedule created", "id", id, "name", req.Name, "dataset", req.Dataset)
|
||||||
|
return schedule, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateSchedule updates an existing schedule
|
||||||
|
func (s *SnapshotScheduleService) UpdateSchedule(ctx context.Context, id string, req *CreateScheduleRequest) (*SnapshotSchedule, error) {
|
||||||
|
// Calculate next run time
|
||||||
|
nextRunAt, err := s.calculateNextRun(req.ScheduleType, req.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to calculate next run time: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Marshal schedule config to JSON
|
||||||
|
configJSON, err := json.Marshal(req.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to marshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
query := `
|
||||||
|
UPDATE snapshot_schedules
|
||||||
|
SET name = $1, dataset = $2, snapshot_name_template = $3, schedule_type = $4,
|
||||||
|
schedule_config = $5, recursive = $6, retention_count = $7, retention_days = $8,
|
||||||
|
next_run_at = $9, updated_at = NOW()
|
||||||
|
WHERE id = $10
|
||||||
|
RETURNING id, name, dataset, snapshot_name_template, schedule_type, schedule_config,
|
||||||
|
recursive, enabled, retention_count, retention_days,
|
||||||
|
last_run_at, next_run_at, created_by, created_at, updated_at
|
||||||
|
`
|
||||||
|
|
||||||
|
row := s.db.QueryRowContext(ctx, query,
|
||||||
|
req.Name, req.Dataset, req.SnapshotNameTemplate, req.ScheduleType,
|
||||||
|
string(configJSON), req.Recursive, req.RetentionCount, req.RetentionDays,
|
||||||
|
nextRunAt, id,
|
||||||
|
)
|
||||||
|
|
||||||
|
schedule, err := s.scanScheduleRow(row)
|
||||||
|
if err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
return nil, fmt.Errorf("schedule not found")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("failed to update schedule: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot schedule updated", "id", id)
|
||||||
|
return schedule, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSchedule deletes a schedule
|
||||||
|
func (s *SnapshotScheduleService) DeleteSchedule(ctx context.Context, id string) error {
|
||||||
|
query := `DELETE FROM snapshot_schedules WHERE id = $1`
|
||||||
|
result, err := s.db.ExecContext(ctx, query, id)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to delete schedule: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rowsAffected, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to get rows affected: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rowsAffected == 0 {
|
||||||
|
return fmt.Errorf("schedule not found")
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot schedule deleted", "id", id)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ToggleSchedule enables or disables a schedule
|
||||||
|
func (s *SnapshotScheduleService) ToggleSchedule(ctx context.Context, id string, enabled bool) error {
|
||||||
|
query := `UPDATE snapshot_schedules SET enabled = $1, updated_at = NOW() WHERE id = $2`
|
||||||
|
result, err := s.db.ExecContext(ctx, query, enabled, id)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to toggle schedule: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rowsAffected, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to get rows affected: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rowsAffected == 0 {
|
||||||
|
return fmt.Errorf("schedule not found")
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot schedule toggled", "id", id, "enabled", enabled)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanScheduleRow scans a single schedule row
|
||||||
|
func (s *SnapshotScheduleService) scanScheduleRow(row *sql.Row) (*SnapshotSchedule, error) {
|
||||||
|
var schedule SnapshotSchedule
|
||||||
|
var configJSON string
|
||||||
|
var lastRunAt, nextRunAt sql.NullTime
|
||||||
|
var retentionCount, retentionDays sql.NullInt64
|
||||||
|
var createdBy sql.NullString
|
||||||
|
|
||||||
|
err := row.Scan(
|
||||||
|
&schedule.ID, &schedule.Name, &schedule.Dataset, &schedule.SnapshotNameTemplate,
|
||||||
|
&schedule.ScheduleType, &configJSON, &schedule.Recursive, &schedule.Enabled,
|
||||||
|
&retentionCount, &retentionDays, &lastRunAt, &nextRunAt,
|
||||||
|
&createdBy, &schedule.CreatedAt, &schedule.UpdatedAt,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse schedule config JSON
|
||||||
|
if err := json.Unmarshal([]byte(configJSON), &schedule.ScheduleConfig); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to unmarshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle nullable fields
|
||||||
|
if lastRunAt.Valid {
|
||||||
|
schedule.LastRunAt = &lastRunAt.Time
|
||||||
|
}
|
||||||
|
if nextRunAt.Valid {
|
||||||
|
schedule.NextRunAt = &nextRunAt.Time
|
||||||
|
}
|
||||||
|
if retentionCount.Valid {
|
||||||
|
count := int(retentionCount.Int64)
|
||||||
|
schedule.RetentionCount = &count
|
||||||
|
}
|
||||||
|
if retentionDays.Valid {
|
||||||
|
days := int(retentionDays.Int64)
|
||||||
|
schedule.RetentionDays = &days
|
||||||
|
}
|
||||||
|
if createdBy.Valid {
|
||||||
|
schedule.CreatedBy = createdBy.String
|
||||||
|
}
|
||||||
|
|
||||||
|
return &schedule, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanSchedule scans a schedule from rows
|
||||||
|
func (s *SnapshotScheduleService) scanSchedule(rows *sql.Rows) (*SnapshotSchedule, error) {
|
||||||
|
var schedule SnapshotSchedule
|
||||||
|
var configJSON string
|
||||||
|
var lastRunAt, nextRunAt sql.NullTime
|
||||||
|
var retentionCount, retentionDays sql.NullInt64
|
||||||
|
var createdBy sql.NullString
|
||||||
|
|
||||||
|
err := rows.Scan(
|
||||||
|
&schedule.ID, &schedule.Name, &schedule.Dataset, &schedule.SnapshotNameTemplate,
|
||||||
|
&schedule.ScheduleType, &configJSON, &schedule.Recursive, &schedule.Enabled,
|
||||||
|
&retentionCount, &retentionDays, &lastRunAt, &nextRunAt,
|
||||||
|
&createdBy, &schedule.CreatedAt, &schedule.UpdatedAt,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse schedule config JSON
|
||||||
|
if err := json.Unmarshal([]byte(configJSON), &schedule.ScheduleConfig); err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to unmarshal schedule config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle nullable fields
|
||||||
|
if lastRunAt.Valid {
|
||||||
|
schedule.LastRunAt = &lastRunAt.Time
|
||||||
|
}
|
||||||
|
if nextRunAt.Valid {
|
||||||
|
schedule.NextRunAt = &nextRunAt.Time
|
||||||
|
}
|
||||||
|
if retentionCount.Valid {
|
||||||
|
count := int(retentionCount.Int64)
|
||||||
|
schedule.RetentionCount = &count
|
||||||
|
}
|
||||||
|
if retentionDays.Valid {
|
||||||
|
days := int(retentionDays.Int64)
|
||||||
|
schedule.RetentionDays = &days
|
||||||
|
}
|
||||||
|
if createdBy.Valid {
|
||||||
|
schedule.CreatedBy = createdBy.String
|
||||||
|
}
|
||||||
|
|
||||||
|
return &schedule, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// calculateNextRun calculates the next run time based on schedule type and config
|
||||||
|
func (s *SnapshotScheduleService) calculateNextRun(scheduleType string, config map[string]interface{}) (*time.Time, error) {
|
||||||
|
now := time.Now()
|
||||||
|
|
||||||
|
switch scheduleType {
|
||||||
|
case "hourly":
|
||||||
|
nextRun := now.Add(1 * time.Hour)
|
||||||
|
// Round to next hour
|
||||||
|
nextRun = nextRun.Truncate(time.Hour).Add(1 * time.Hour)
|
||||||
|
return &nextRun, nil
|
||||||
|
|
||||||
|
case "daily":
|
||||||
|
timeStr, ok := config["time"].(string)
|
||||||
|
if !ok {
|
||||||
|
timeStr = "00:00"
|
||||||
|
}
|
||||||
|
// Parse time (HH:MM format)
|
||||||
|
hour, min := 0, 0
|
||||||
|
fmt.Sscanf(timeStr, "%d:%d", &hour, &min)
|
||||||
|
nextRun := time.Date(now.Year(), now.Month(), now.Day(), hour, min, 0, 0, now.Location())
|
||||||
|
if nextRun.Before(now) {
|
||||||
|
nextRun = nextRun.Add(24 * time.Hour)
|
||||||
|
}
|
||||||
|
return &nextRun, nil
|
||||||
|
|
||||||
|
case "weekly":
|
||||||
|
dayOfWeek, ok := config["day"].(float64) // JSON numbers are float64
|
||||||
|
if !ok {
|
||||||
|
dayOfWeek = 0 // Sunday
|
||||||
|
}
|
||||||
|
timeStr, ok := config["time"].(string)
|
||||||
|
if !ok {
|
||||||
|
timeStr = "00:00"
|
||||||
|
}
|
||||||
|
hour, min := 0, 0
|
||||||
|
fmt.Sscanf(timeStr, "%d:%d", &hour, &min)
|
||||||
|
|
||||||
|
// Calculate next occurrence
|
||||||
|
currentDay := int(now.Weekday())
|
||||||
|
targetDay := int(dayOfWeek)
|
||||||
|
daysUntil := (targetDay - currentDay + 7) % 7
|
||||||
|
if daysUntil == 0 {
|
||||||
|
// If same day, check if time has passed
|
||||||
|
today := time.Date(now.Year(), now.Month(), now.Day(), hour, min, 0, 0, now.Location())
|
||||||
|
if today.Before(now) {
|
||||||
|
daysUntil = 7
|
||||||
|
}
|
||||||
|
}
|
||||||
|
nextRun := time.Date(now.Year(), now.Month(), now.Day(), hour, min, 0, 0, now.Location()).AddDate(0, 0, daysUntil)
|
||||||
|
return &nextRun, nil
|
||||||
|
|
||||||
|
case "monthly":
|
||||||
|
day, ok := config["day"].(float64)
|
||||||
|
if !ok {
|
||||||
|
day = 1
|
||||||
|
}
|
||||||
|
timeStr, ok := config["time"].(string)
|
||||||
|
if !ok {
|
||||||
|
timeStr = "00:00"
|
||||||
|
}
|
||||||
|
hour, min := 0, 0
|
||||||
|
fmt.Sscanf(timeStr, "%d:%d", &hour, &min)
|
||||||
|
|
||||||
|
// Calculate next occurrence
|
||||||
|
nextRun := time.Date(now.Year(), now.Month(), int(day), hour, min, 0, 0, now.Location())
|
||||||
|
if nextRun.Before(now) {
|
||||||
|
nextRun = nextRun.AddDate(0, 1, 0)
|
||||||
|
}
|
||||||
|
return &nextRun, nil
|
||||||
|
|
||||||
|
case "cron":
|
||||||
|
// For cron, we'll use a simple implementation
|
||||||
|
// In production, you'd want to use a proper cron parser library
|
||||||
|
_, ok := config["cron"].(string)
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("cron expression not found in config")
|
||||||
|
}
|
||||||
|
// Simple implementation: assume next run is in 1 hour
|
||||||
|
// TODO: Implement proper cron parsing
|
||||||
|
nextRun := now.Add(1 * time.Hour)
|
||||||
|
return &nextRun, nil
|
||||||
|
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("unknown schedule type: %s", scheduleType)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetDueSchedules retrieves all enabled schedules that are due to run
|
||||||
|
func (s *SnapshotScheduleService) GetDueSchedules(ctx context.Context) ([]*SnapshotSchedule, error) {
|
||||||
|
now := time.Now()
|
||||||
|
query := `
|
||||||
|
SELECT id, name, dataset, snapshot_name_template, schedule_type, schedule_config,
|
||||||
|
recursive, enabled, retention_count, retention_days,
|
||||||
|
last_run_at, next_run_at, created_by, created_at, updated_at
|
||||||
|
FROM snapshot_schedules
|
||||||
|
WHERE enabled = true AND next_run_at <= $1
|
||||||
|
ORDER BY next_run_at ASC
|
||||||
|
`
|
||||||
|
|
||||||
|
rows, err := s.db.QueryContext(ctx, query, now)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to query due schedules: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
var schedules []*SnapshotSchedule
|
||||||
|
for rows.Next() {
|
||||||
|
schedule, err := s.scanSchedule(rows)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to scan schedule", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
schedules = append(schedules, schedule)
|
||||||
|
}
|
||||||
|
|
||||||
|
return schedules, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExecuteSchedule executes a snapshot schedule
|
||||||
|
func (s *SnapshotScheduleService) ExecuteSchedule(ctx context.Context, schedule *SnapshotSchedule) error {
|
||||||
|
now := time.Now()
|
||||||
|
|
||||||
|
// Generate snapshot name from template
|
||||||
|
snapshotName := s.generateSnapshotName(schedule.SnapshotNameTemplate, now)
|
||||||
|
|
||||||
|
// Create snapshot
|
||||||
|
if err := s.snapshotService.CreateSnapshot(ctx, schedule.Dataset, snapshotName, schedule.Recursive); err != nil {
|
||||||
|
return fmt.Errorf("failed to create snapshot: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Snapshot created from schedule", "schedule", schedule.Name, "snapshot", snapshotName, "dataset", schedule.Dataset)
|
||||||
|
|
||||||
|
// Calculate next run time
|
||||||
|
nextRunAt, err := s.calculateNextRun(schedule.ScheduleType, schedule.ScheduleConfig)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to calculate next run time", "error", err, "schedule", schedule.Name)
|
||||||
|
// Set a default next run time (1 hour from now) if calculation fails
|
||||||
|
defaultNextRun := now.Add(1 * time.Hour)
|
||||||
|
nextRunAt = &defaultNextRun
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update schedule with last run time and next run time
|
||||||
|
query := `
|
||||||
|
UPDATE snapshot_schedules
|
||||||
|
SET last_run_at = $1, next_run_at = $2, updated_at = NOW()
|
||||||
|
WHERE id = $3
|
||||||
|
`
|
||||||
|
if _, err := s.db.ExecContext(ctx, query, now, nextRunAt, schedule.ID); err != nil {
|
||||||
|
s.logger.Error("Failed to update schedule after execution", "error", err, "schedule", schedule.Name)
|
||||||
|
// Don't return error here - snapshot was created successfully
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle retention
|
||||||
|
if err := s.handleRetention(ctx, schedule); err != nil {
|
||||||
|
s.logger.Error("Failed to handle retention", "error", err, "schedule", schedule.Name)
|
||||||
|
// Don't return error - retention is best effort
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// generateSnapshotName generates a snapshot name from a template
|
||||||
|
func (s *SnapshotScheduleService) generateSnapshotName(template string, t time.Time) string {
|
||||||
|
// Replace common time format placeholders
|
||||||
|
name := template
|
||||||
|
name = strings.ReplaceAll(name, "%Y", t.Format("2006"))
|
||||||
|
name = strings.ReplaceAll(name, "%m", t.Format("01"))
|
||||||
|
name = strings.ReplaceAll(name, "%d", t.Format("02"))
|
||||||
|
name = strings.ReplaceAll(name, "%H", t.Format("15"))
|
||||||
|
name = strings.ReplaceAll(name, "%M", t.Format("04"))
|
||||||
|
name = strings.ReplaceAll(name, "%S", t.Format("05"))
|
||||||
|
name = strings.ReplaceAll(name, "%Y-%m-%d", t.Format("2006-01-02"))
|
||||||
|
name = strings.ReplaceAll(name, "%Y-%m-%d-%H%M", t.Format("2006-01-02-1504"))
|
||||||
|
name = strings.ReplaceAll(name, "%Y-%m-%d-%H%M%S", t.Format("2006-01-02-150405"))
|
||||||
|
|
||||||
|
// If no placeholders were replaced, append timestamp
|
||||||
|
if name == template {
|
||||||
|
name = fmt.Sprintf("%s-%d", template, t.Unix())
|
||||||
|
}
|
||||||
|
|
||||||
|
return name
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleRetention handles snapshot retention based on schedule settings
|
||||||
|
func (s *SnapshotScheduleService) handleRetention(ctx context.Context, schedule *SnapshotSchedule) error {
|
||||||
|
// Get all snapshots for this dataset
|
||||||
|
snapshots, err := s.snapshotService.ListSnapshots(ctx, schedule.Dataset)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to list snapshots: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Filter snapshots that match this schedule's naming pattern
|
||||||
|
matchingSnapshots := make([]*Snapshot, 0)
|
||||||
|
for _, snapshot := range snapshots {
|
||||||
|
// Check if snapshot name matches the template pattern
|
||||||
|
if s.snapshotMatchesTemplate(snapshot.SnapshotName, schedule.SnapshotNameTemplate) {
|
||||||
|
matchingSnapshots = append(matchingSnapshots, snapshot)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
now := time.Now()
|
||||||
|
snapshotsToDelete := make([]*Snapshot, 0)
|
||||||
|
|
||||||
|
// Apply retention_count if set
|
||||||
|
if schedule.RetentionCount != nil && *schedule.RetentionCount > 0 {
|
||||||
|
if len(matchingSnapshots) > *schedule.RetentionCount {
|
||||||
|
// Sort by creation time (oldest first)
|
||||||
|
sort.Slice(matchingSnapshots, func(i, j int) bool {
|
||||||
|
return matchingSnapshots[i].Created.Before(matchingSnapshots[j].Created)
|
||||||
|
})
|
||||||
|
// Mark excess snapshots for deletion
|
||||||
|
excessCount := len(matchingSnapshots) - *schedule.RetentionCount
|
||||||
|
for i := 0; i < excessCount; i++ {
|
||||||
|
snapshotsToDelete = append(snapshotsToDelete, matchingSnapshots[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply retention_days if set
|
||||||
|
if schedule.RetentionDays != nil && *schedule.RetentionDays > 0 {
|
||||||
|
cutoffTime := now.AddDate(0, 0, -*schedule.RetentionDays)
|
||||||
|
for _, snapshot := range matchingSnapshots {
|
||||||
|
if snapshot.Created.Before(cutoffTime) {
|
||||||
|
// Check if not already marked for deletion
|
||||||
|
alreadyMarked := false
|
||||||
|
for _, marked := range snapshotsToDelete {
|
||||||
|
if marked.ID == snapshot.ID {
|
||||||
|
alreadyMarked = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !alreadyMarked {
|
||||||
|
snapshotsToDelete = append(snapshotsToDelete, snapshot)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Delete snapshots
|
||||||
|
for _, snapshot := range snapshotsToDelete {
|
||||||
|
if err := s.snapshotService.DeleteSnapshot(ctx, snapshot.Name, schedule.Recursive); err != nil {
|
||||||
|
s.logger.Error("Failed to delete snapshot for retention", "error", err, "snapshot", snapshot.Name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s.logger.Info("Deleted snapshot for retention", "snapshot", snapshot.Name, "schedule", schedule.Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// snapshotMatchesTemplate checks if a snapshot name matches a template pattern
|
||||||
|
func (s *SnapshotScheduleService) snapshotMatchesTemplate(snapshotName, template string) bool {
|
||||||
|
if template == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract the base name from template (everything before the first %)
|
||||||
|
// This handles templates like "auto-%Y-%m-%d" or "daily-backup-%Y%m%d"
|
||||||
|
baseName := template
|
||||||
|
if idx := strings.Index(template, "%"); idx != -1 {
|
||||||
|
baseName = template[:idx]
|
||||||
|
}
|
||||||
|
|
||||||
|
// If template starts with a placeholder, check if snapshot name matches the pattern
|
||||||
|
// by checking if it starts with any reasonable prefix
|
||||||
|
if baseName == "" {
|
||||||
|
// Template starts with placeholder - use a more lenient check
|
||||||
|
// Check if snapshot name matches common patterns
|
||||||
|
return true // Match all if template is just placeholders
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if snapshot name starts with the base name
|
||||||
|
// This handles cases like template "auto-%Y-%m-%d" matching snapshot "auto-2024-01-15"
|
||||||
|
return strings.HasPrefix(snapshotName, baseName)
|
||||||
|
}
|
||||||
87
backend/internal/storage/snapshot_schedule_worker.go
Normal file
87
backend/internal/storage/snapshot_schedule_worker.go
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/database"
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SnapshotScheduleWorker handles periodic execution of snapshot schedules
|
||||||
|
type SnapshotScheduleWorker struct {
|
||||||
|
scheduleService *SnapshotScheduleService
|
||||||
|
logger *logger.Logger
|
||||||
|
interval time.Duration
|
||||||
|
stopCh chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSnapshotScheduleWorker creates a new snapshot schedule worker
|
||||||
|
func NewSnapshotScheduleWorker(db *database.DB, log *logger.Logger, snapshotService *SnapshotService, interval time.Duration) *SnapshotScheduleWorker {
|
||||||
|
scheduleService := NewSnapshotScheduleService(db, log, snapshotService)
|
||||||
|
return &SnapshotScheduleWorker{
|
||||||
|
scheduleService: scheduleService,
|
||||||
|
logger: log,
|
||||||
|
interval: interval,
|
||||||
|
stopCh: make(chan struct{}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Start starts the snapshot schedule worker background service
|
||||||
|
func (w *SnapshotScheduleWorker) Start(ctx context.Context) {
|
||||||
|
w.logger.Info("Starting snapshot schedule worker", "interval", w.interval)
|
||||||
|
ticker := time.NewTicker(w.interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
// Run initial check immediately
|
||||||
|
w.processSchedules(ctx)
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
w.logger.Info("Snapshot schedule worker stopped")
|
||||||
|
return
|
||||||
|
case <-w.stopCh:
|
||||||
|
w.logger.Info("Snapshot schedule worker stopped")
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
w.processSchedules(ctx)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stop stops the snapshot schedule worker service
|
||||||
|
func (w *SnapshotScheduleWorker) Stop() {
|
||||||
|
close(w.stopCh)
|
||||||
|
}
|
||||||
|
|
||||||
|
// processSchedules processes all due snapshot schedules
|
||||||
|
func (w *SnapshotScheduleWorker) processSchedules(ctx context.Context) {
|
||||||
|
w.logger.Debug("Checking for due snapshot schedules")
|
||||||
|
|
||||||
|
// Get all schedules that are due to run
|
||||||
|
schedules, err := w.scheduleService.GetDueSchedules(ctx)
|
||||||
|
if err != nil {
|
||||||
|
w.logger.Error("Failed to get due schedules", "error", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(schedules) == 0 {
|
||||||
|
w.logger.Debug("No snapshot schedules due to run")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
w.logger.Info("Found due snapshot schedules", "count", len(schedules))
|
||||||
|
|
||||||
|
// Execute each schedule
|
||||||
|
for _, schedule := range schedules {
|
||||||
|
w.logger.Info("Executing snapshot schedule", "schedule", schedule.Name, "dataset", schedule.Dataset)
|
||||||
|
|
||||||
|
if err := w.scheduleService.ExecuteSchedule(ctx, schedule); err != nil {
|
||||||
|
w.logger.Error("Failed to execute snapshot schedule", "error", err, "schedule", schedule.Name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
w.logger.Info("Successfully executed snapshot schedule", "schedule", schedule.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -16,6 +16,16 @@ import (
|
|||||||
"github.com/lib/pq"
|
"github.com/lib/pq"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// zfsCommand executes a ZFS command with sudo
|
||||||
|
func zfsCommand(ctx context.Context, args ...string) *exec.Cmd {
|
||||||
|
return exec.CommandContext(ctx, "sudo", append([]string{"zfs"}, args...)...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// zpoolCommand executes a ZPOOL command with sudo
|
||||||
|
func zpoolCommand(ctx context.Context, args ...string) *exec.Cmd {
|
||||||
|
return exec.CommandContext(ctx, "sudo", append([]string{"zpool"}, args...)...)
|
||||||
|
}
|
||||||
|
|
||||||
// ZFSService handles ZFS pool management
|
// ZFSService handles ZFS pool management
|
||||||
type ZFSService struct {
|
type ZFSService struct {
|
||||||
db *database.DB
|
db *database.DB
|
||||||
@@ -115,6 +125,10 @@ func (s *ZFSService) CreatePool(ctx context.Context, name string, raidLevel stri
|
|||||||
var args []string
|
var args []string
|
||||||
args = append(args, "create", "-f") // -f to force creation
|
args = append(args, "create", "-f") // -f to force creation
|
||||||
|
|
||||||
|
// Set default mountpoint to /opt/calypso/data/pool/<pool-name>
|
||||||
|
mountPoint := fmt.Sprintf("/opt/calypso/data/pool/%s", name)
|
||||||
|
args = append(args, "-m", mountPoint)
|
||||||
|
|
||||||
// Note: compression is a filesystem property, not a pool property
|
// Note: compression is a filesystem property, not a pool property
|
||||||
// We'll set it after pool creation using zfs set
|
// We'll set it after pool creation using zfs set
|
||||||
|
|
||||||
@@ -155,9 +169,15 @@ func (s *ZFSService) CreatePool(ctx context.Context, name string, raidLevel stri
|
|||||||
args = append(args, disks...)
|
args = append(args, disks...)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Execute zpool create
|
// Create mountpoint directory if it doesn't exist
|
||||||
s.logger.Info("Creating ZFS pool", "name", name, "raid_level", raidLevel, "disks", disks, "args", args)
|
if err := os.MkdirAll(mountPoint, 0755); err != nil {
|
||||||
cmd := exec.CommandContext(ctx, "zpool", args...)
|
return nil, fmt.Errorf("failed to create mountpoint directory %s: %w", mountPoint, err)
|
||||||
|
}
|
||||||
|
s.logger.Info("Created mountpoint directory", "path", mountPoint)
|
||||||
|
|
||||||
|
// Execute zpool create (with sudo for permissions)
|
||||||
|
s.logger.Info("Creating ZFS pool", "name", name, "raid_level", raidLevel, "disks", disks, "mountpoint", mountPoint, "args", args)
|
||||||
|
cmd := zpoolCommand(ctx, args...)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
errorMsg := string(output)
|
errorMsg := string(output)
|
||||||
@@ -170,7 +190,7 @@ func (s *ZFSService) CreatePool(ctx context.Context, name string, raidLevel stri
|
|||||||
// Set filesystem properties (compression, etc.) after pool creation
|
// Set filesystem properties (compression, etc.) after pool creation
|
||||||
// ZFS creates a root filesystem with the same name as the pool
|
// ZFS creates a root filesystem with the same name as the pool
|
||||||
if compression != "" && compression != "off" {
|
if compression != "" && compression != "off" {
|
||||||
cmd = exec.CommandContext(ctx, "zfs", "set", fmt.Sprintf("compression=%s", compression), name)
|
cmd = zfsCommand(ctx, "set", fmt.Sprintf("compression=%s", compression), name)
|
||||||
output, err = cmd.CombinedOutput()
|
output, err = cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.logger.Warn("Failed to set compression property", "pool", name, "compression", compression, "error", string(output))
|
s.logger.Warn("Failed to set compression property", "pool", name, "compression", compression, "error", string(output))
|
||||||
@@ -185,7 +205,7 @@ func (s *ZFSService) CreatePool(ctx context.Context, name string, raidLevel stri
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
// Try to destroy the pool if we can't get info
|
// Try to destroy the pool if we can't get info
|
||||||
s.logger.Warn("Failed to get pool info, attempting to destroy pool", "name", name, "error", err)
|
s.logger.Warn("Failed to get pool info, attempting to destroy pool", "name", name, "error", err)
|
||||||
exec.CommandContext(ctx, "zpool", "destroy", "-f", name).Run()
|
zpoolCommand(ctx, "destroy", "-f", name).Run()
|
||||||
return nil, fmt.Errorf("failed to get pool info after creation: %w", err)
|
return nil, fmt.Errorf("failed to get pool info after creation: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -219,7 +239,7 @@ func (s *ZFSService) CreatePool(ctx context.Context, name string, raidLevel stri
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
// Cleanup: destroy pool if database insert fails
|
// Cleanup: destroy pool if database insert fails
|
||||||
s.logger.Error("Failed to save pool to database, destroying pool", "name", name, "error", err)
|
s.logger.Error("Failed to save pool to database, destroying pool", "name", name, "error", err)
|
||||||
exec.CommandContext(ctx, "zpool", "destroy", "-f", name).Run()
|
zpoolCommand(ctx, "destroy", "-f", name).Run()
|
||||||
return nil, fmt.Errorf("failed to save pool to database: %w", err)
|
return nil, fmt.Errorf("failed to save pool to database: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -243,7 +263,7 @@ func (s *ZFSService) CreatePool(ctx context.Context, name string, raidLevel stri
|
|||||||
// getPoolInfo retrieves information about a ZFS pool
|
// getPoolInfo retrieves information about a ZFS pool
|
||||||
func (s *ZFSService) getPoolInfo(ctx context.Context, poolName string) (*ZFSPool, error) {
|
func (s *ZFSService) getPoolInfo(ctx context.Context, poolName string) (*ZFSPool, error) {
|
||||||
// Get pool size and used space
|
// Get pool size and used space
|
||||||
cmd := exec.CommandContext(ctx, "zpool", "list", "-H", "-o", "name,size,allocated", poolName)
|
cmd := zpoolCommand(ctx, "list", "-H", "-o", "name,size,allocated", poolName)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
errorMsg := string(output)
|
errorMsg := string(output)
|
||||||
@@ -322,7 +342,7 @@ func parseZFSSize(sizeStr string) (int64, error) {
|
|||||||
|
|
||||||
// getSpareDisks retrieves spare disks from zpool status
|
// getSpareDisks retrieves spare disks from zpool status
|
||||||
func (s *ZFSService) getSpareDisks(ctx context.Context, poolName string) ([]string, error) {
|
func (s *ZFSService) getSpareDisks(ctx context.Context, poolName string) ([]string, error) {
|
||||||
cmd := exec.CommandContext(ctx, "zpool", "status", poolName)
|
cmd := zpoolCommand(ctx, "status", poolName)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("failed to get pool status: %w", err)
|
return nil, fmt.Errorf("failed to get pool status: %w", err)
|
||||||
@@ -363,7 +383,7 @@ func (s *ZFSService) getSpareDisks(ctx context.Context, poolName string) ([]stri
|
|||||||
|
|
||||||
// getCompressRatio gets the compression ratio from ZFS
|
// getCompressRatio gets the compression ratio from ZFS
|
||||||
func (s *ZFSService) getCompressRatio(ctx context.Context, poolName string) (float64, error) {
|
func (s *ZFSService) getCompressRatio(ctx context.Context, poolName string) (float64, error) {
|
||||||
cmd := exec.CommandContext(ctx, "zfs", "get", "-H", "-o", "value", "compressratio", poolName)
|
cmd := zfsCommand(ctx, "get", "-H", "-o", "value", "compressratio", poolName)
|
||||||
output, err := cmd.Output()
|
output, err := cmd.Output()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 1.0, err
|
return 1.0, err
|
||||||
@@ -406,16 +426,20 @@ func (s *ZFSService) ListPools(ctx context.Context) ([]*ZFSPool, error) {
|
|||||||
for rows.Next() {
|
for rows.Next() {
|
||||||
var pool ZFSPool
|
var pool ZFSPool
|
||||||
var description sql.NullString
|
var description sql.NullString
|
||||||
|
var createdBy sql.NullString
|
||||||
err := rows.Scan(
|
err := rows.Scan(
|
||||||
&pool.ID, &pool.Name, &description, &pool.RaidLevel, pq.Array(&pool.Disks),
|
&pool.ID, &pool.Name, &description, &pool.RaidLevel, pq.Array(&pool.Disks),
|
||||||
&pool.SizeBytes, &pool.UsedBytes, &pool.Compression, &pool.Deduplication,
|
&pool.SizeBytes, &pool.UsedBytes, &pool.Compression, &pool.Deduplication,
|
||||||
&pool.AutoExpand, &pool.ScrubInterval, &pool.IsActive, &pool.HealthStatus,
|
&pool.AutoExpand, &pool.ScrubInterval, &pool.IsActive, &pool.HealthStatus,
|
||||||
&pool.CreatedAt, &pool.UpdatedAt, &pool.CreatedBy,
|
&pool.CreatedAt, &pool.UpdatedAt, &createdBy,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.logger.Error("Failed to scan pool row", "error", err)
|
s.logger.Error("Failed to scan pool row", "error", err, "error_type", fmt.Sprintf("%T", err))
|
||||||
continue // Skip this pool instead of failing entire query
|
continue // Skip this pool instead of failing entire query
|
||||||
}
|
}
|
||||||
|
if createdBy.Valid {
|
||||||
|
pool.CreatedBy = createdBy.String
|
||||||
|
}
|
||||||
if description.Valid {
|
if description.Valid {
|
||||||
pool.Description = description.String
|
pool.Description = description.String
|
||||||
}
|
}
|
||||||
@@ -501,7 +525,7 @@ func (s *ZFSService) DeletePool(ctx context.Context, poolID string) error {
|
|||||||
// Destroy ZFS pool with -f flag to force destroy (works for both empty and non-empty pools)
|
// Destroy ZFS pool with -f flag to force destroy (works for both empty and non-empty pools)
|
||||||
// The -f flag is needed to destroy pools even if they have datasets or are in use
|
// The -f flag is needed to destroy pools even if they have datasets or are in use
|
||||||
s.logger.Info("Destroying ZFS pool", "pool", pool.Name)
|
s.logger.Info("Destroying ZFS pool", "pool", pool.Name)
|
||||||
cmd := exec.CommandContext(ctx, "zpool", "destroy", "-f", pool.Name)
|
cmd := zpoolCommand(ctx, "destroy", "-f", pool.Name)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
errorMsg := string(output)
|
errorMsg := string(output)
|
||||||
@@ -516,6 +540,15 @@ func (s *ZFSService) DeletePool(ctx context.Context, poolID string) error {
|
|||||||
s.logger.Info("ZFS pool destroyed successfully", "pool", pool.Name)
|
s.logger.Info("ZFS pool destroyed successfully", "pool", pool.Name)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Remove mount point directory (default: /opt/calypso/data/pool/<pool-name>)
|
||||||
|
mountPoint := fmt.Sprintf("/opt/calypso/data/pool/%s", pool.Name)
|
||||||
|
if err := os.RemoveAll(mountPoint); err != nil {
|
||||||
|
s.logger.Warn("Failed to remove mount point directory", "mountpoint", mountPoint, "error", err)
|
||||||
|
// Don't fail pool deletion if mount point removal fails
|
||||||
|
} else {
|
||||||
|
s.logger.Info("Removed mount point directory", "mountpoint", mountPoint)
|
||||||
|
}
|
||||||
|
|
||||||
// Mark disks as unused
|
// Mark disks as unused
|
||||||
for _, diskPath := range pool.Disks {
|
for _, diskPath := range pool.Disks {
|
||||||
_, err = s.db.ExecContext(ctx,
|
_, err = s.db.ExecContext(ctx,
|
||||||
@@ -550,7 +583,7 @@ func (s *ZFSService) AddSpareDisk(ctx context.Context, poolID string, diskPaths
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Verify pool exists in ZFS and check if disks are already spare
|
// Verify pool exists in ZFS and check if disks are already spare
|
||||||
cmd := exec.CommandContext(ctx, "zpool", "status", pool.Name)
|
cmd := zpoolCommand(ctx, "status", pool.Name)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("pool %s does not exist in ZFS: %w", pool.Name, err)
|
return fmt.Errorf("pool %s does not exist in ZFS: %w", pool.Name, err)
|
||||||
@@ -575,7 +608,7 @@ func (s *ZFSService) AddSpareDisk(ctx context.Context, poolID string, diskPaths
|
|||||||
|
|
||||||
// Execute zpool add
|
// Execute zpool add
|
||||||
s.logger.Info("Adding spare disks to ZFS pool", "pool", pool.Name, "disks", diskPaths)
|
s.logger.Info("Adding spare disks to ZFS pool", "pool", pool.Name, "disks", diskPaths)
|
||||||
cmd = exec.CommandContext(ctx, "zpool", args...)
|
cmd = zpoolCommand(ctx, args...)
|
||||||
output, err = cmd.CombinedOutput()
|
output, err = cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
errorMsg := string(output)
|
errorMsg := string(output)
|
||||||
@@ -610,6 +643,7 @@ func (s *ZFSService) AddSpareDisk(ctx context.Context, poolID string, diskPaths
|
|||||||
|
|
||||||
// ZFSDataset represents a ZFS dataset
|
// ZFSDataset represents a ZFS dataset
|
||||||
type ZFSDataset struct {
|
type ZFSDataset struct {
|
||||||
|
ID string `json:"id"`
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
Pool string `json:"pool"`
|
Pool string `json:"pool"`
|
||||||
Type string `json:"type"` // filesystem, volume, snapshot
|
Type string `json:"type"` // filesystem, volume, snapshot
|
||||||
@@ -628,7 +662,7 @@ type ZFSDataset struct {
|
|||||||
func (s *ZFSService) ListDatasets(ctx context.Context, poolName string) ([]*ZFSDataset, error) {
|
func (s *ZFSService) ListDatasets(ctx context.Context, poolName string) ([]*ZFSDataset, error) {
|
||||||
// Get datasets from database
|
// Get datasets from database
|
||||||
query := `
|
query := `
|
||||||
SELECT name, pool_name, type, mount_point,
|
SELECT id, name, pool_name, type, mount_point,
|
||||||
used_bytes, available_bytes, referenced_bytes,
|
used_bytes, available_bytes, referenced_bytes,
|
||||||
compression, deduplication, quota, reservation,
|
compression, deduplication, quota, reservation,
|
||||||
created_at
|
created_at
|
||||||
@@ -654,7 +688,7 @@ func (s *ZFSService) ListDatasets(ctx context.Context, poolName string) ([]*ZFSD
|
|||||||
var mountPoint sql.NullString
|
var mountPoint sql.NullString
|
||||||
|
|
||||||
err := rows.Scan(
|
err := rows.Scan(
|
||||||
&ds.Name, &ds.Pool, &ds.Type, &mountPoint,
|
&ds.ID, &ds.Name, &ds.Pool, &ds.Type, &mountPoint,
|
||||||
&ds.UsedBytes, &ds.AvailableBytes, &ds.ReferencedBytes,
|
&ds.UsedBytes, &ds.AvailableBytes, &ds.ReferencedBytes,
|
||||||
&ds.Compression, &ds.Deduplication, &ds.Quota, &ds.Reservation,
|
&ds.Compression, &ds.Deduplication, &ds.Quota, &ds.Reservation,
|
||||||
&ds.CreatedAt,
|
&ds.CreatedAt,
|
||||||
@@ -696,10 +730,36 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
// Construct full dataset name
|
// Construct full dataset name
|
||||||
fullName := poolName + "/" + req.Name
|
fullName := poolName + "/" + req.Name
|
||||||
|
|
||||||
// For filesystem datasets, create mount directory if mount point is provided
|
// Get pool mount point to validate dataset mount point is within pool directory
|
||||||
if req.Type == "filesystem" && req.MountPoint != "" {
|
poolMountPoint := fmt.Sprintf("/opt/calypso/data/pool/%s", poolName)
|
||||||
// Clean and validate mount point path
|
var mountPath string
|
||||||
mountPath := filepath.Clean(req.MountPoint)
|
|
||||||
|
// For filesystem datasets, validate and set mount point
|
||||||
|
if req.Type == "filesystem" {
|
||||||
|
if req.MountPoint != "" {
|
||||||
|
// User provided mount point - validate it's within pool directory
|
||||||
|
mountPath = filepath.Clean(req.MountPoint)
|
||||||
|
|
||||||
|
// Check if mount point is within pool mount point directory
|
||||||
|
poolMountAbs, err := filepath.Abs(poolMountPoint)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to resolve pool mount point: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
mountPathAbs, err := filepath.Abs(mountPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to resolve mount point: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if mount path is within pool mount point directory
|
||||||
|
relPath, err := filepath.Rel(poolMountAbs, mountPathAbs)
|
||||||
|
if err != nil || strings.HasPrefix(relPath, "..") {
|
||||||
|
return nil, fmt.Errorf("mount point must be within pool directory: %s (pool mount: %s)", mountPath, poolMountPoint)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// No mount point provided - use default: /opt/calypso/data/pool/<pool-name>/<dataset-name>/
|
||||||
|
mountPath = filepath.Join(poolMountPoint, req.Name)
|
||||||
|
}
|
||||||
|
|
||||||
// Check if directory already exists
|
// Check if directory already exists
|
||||||
if info, err := os.Stat(mountPath); err == nil {
|
if info, err := os.Stat(mountPath); err == nil {
|
||||||
@@ -748,14 +808,14 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
args = append(args, "-o", fmt.Sprintf("compression=%s", req.Compression))
|
args = append(args, "-o", fmt.Sprintf("compression=%s", req.Compression))
|
||||||
}
|
}
|
||||||
|
|
||||||
// Set mount point if provided (only for filesystems, not volumes)
|
// Set mount point for filesystems (always set, either user-provided or default)
|
||||||
if req.Type == "filesystem" && req.MountPoint != "" {
|
if req.Type == "filesystem" {
|
||||||
args = append(args, "-o", fmt.Sprintf("mountpoint=%s", req.MountPoint))
|
args = append(args, "-o", fmt.Sprintf("mountpoint=%s", mountPath))
|
||||||
}
|
}
|
||||||
|
|
||||||
// Execute zfs create
|
// Execute zfs create
|
||||||
s.logger.Info("Creating ZFS dataset", "name", fullName, "type", req.Type)
|
s.logger.Info("Creating ZFS dataset", "name", fullName, "type", req.Type)
|
||||||
cmd := exec.CommandContext(ctx, "zfs", args...)
|
cmd := zfsCommand(ctx, args...)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
errorMsg := string(output)
|
errorMsg := string(output)
|
||||||
@@ -765,7 +825,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
|
|
||||||
// Set quota if specified (for filesystems)
|
// Set quota if specified (for filesystems)
|
||||||
if req.Type == "filesystem" && req.Quota > 0 {
|
if req.Type == "filesystem" && req.Quota > 0 {
|
||||||
quotaCmd := exec.CommandContext(ctx, "zfs", "set", fmt.Sprintf("quota=%d", req.Quota), fullName)
|
quotaCmd := zfsCommand(ctx, "set", fmt.Sprintf("quota=%d", req.Quota), fullName)
|
||||||
if quotaOutput, err := quotaCmd.CombinedOutput(); err != nil {
|
if quotaOutput, err := quotaCmd.CombinedOutput(); err != nil {
|
||||||
s.logger.Warn("Failed to set quota", "dataset", fullName, "error", err, "output", string(quotaOutput))
|
s.logger.Warn("Failed to set quota", "dataset", fullName, "error", err, "output", string(quotaOutput))
|
||||||
}
|
}
|
||||||
@@ -773,7 +833,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
|
|
||||||
// Set reservation if specified
|
// Set reservation if specified
|
||||||
if req.Reservation > 0 {
|
if req.Reservation > 0 {
|
||||||
resvCmd := exec.CommandContext(ctx, "zfs", "set", fmt.Sprintf("reservation=%d", req.Reservation), fullName)
|
resvCmd := zfsCommand(ctx, "set", fmt.Sprintf("reservation=%d", req.Reservation), fullName)
|
||||||
if resvOutput, err := resvCmd.CombinedOutput(); err != nil {
|
if resvOutput, err := resvCmd.CombinedOutput(); err != nil {
|
||||||
s.logger.Warn("Failed to set reservation", "dataset", fullName, "error", err, "output", string(resvOutput))
|
s.logger.Warn("Failed to set reservation", "dataset", fullName, "error", err, "output", string(resvOutput))
|
||||||
}
|
}
|
||||||
@@ -785,30 +845,30 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
s.logger.Error("Failed to get pool ID", "pool", poolName, "error", err)
|
s.logger.Error("Failed to get pool ID", "pool", poolName, "error", err)
|
||||||
// Try to destroy the dataset if we can't save to database
|
// Try to destroy the dataset if we can't save to database
|
||||||
exec.CommandContext(ctx, "zfs", "destroy", "-r", fullName).Run()
|
zfsCommand(ctx, "destroy", "-r", fullName).Run()
|
||||||
return nil, fmt.Errorf("failed to get pool ID: %w", err)
|
return nil, fmt.Errorf("failed to get pool ID: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get dataset info from ZFS to save to database
|
// Get dataset info from ZFS to save to database
|
||||||
cmd = exec.CommandContext(ctx, "zfs", "list", "-H", "-o", "name,used,avail,refer,compress,dedup,quota,reservation,mountpoint", fullName)
|
cmd = zfsCommand(ctx, "list", "-H", "-o", "name,used,avail,refer,compress,dedup,quota,reservation,mountpoint", fullName)
|
||||||
output, err = cmd.CombinedOutput()
|
output, err = cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.logger.Error("Failed to get dataset info", "name", fullName, "error", err)
|
s.logger.Error("Failed to get dataset info", "name", fullName, "error", err)
|
||||||
// Try to destroy the dataset if we can't get info
|
// Try to destroy the dataset if we can't get info
|
||||||
exec.CommandContext(ctx, "zfs", "destroy", "-r", fullName).Run()
|
zfsCommand(ctx, "destroy", "-r", fullName).Run()
|
||||||
return nil, fmt.Errorf("failed to get dataset info: %w", err)
|
return nil, fmt.Errorf("failed to get dataset info: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Parse dataset info
|
// Parse dataset info
|
||||||
lines := strings.TrimSpace(string(output))
|
lines := strings.TrimSpace(string(output))
|
||||||
if lines == "" {
|
if lines == "" {
|
||||||
exec.CommandContext(ctx, "zfs", "destroy", "-r", fullName).Run()
|
zfsCommand(ctx, "destroy", "-r", fullName).Run()
|
||||||
return nil, fmt.Errorf("dataset not found after creation")
|
return nil, fmt.Errorf("dataset not found after creation")
|
||||||
}
|
}
|
||||||
|
|
||||||
fields := strings.Fields(lines)
|
fields := strings.Fields(lines)
|
||||||
if len(fields) < 9 {
|
if len(fields) < 9 {
|
||||||
exec.CommandContext(ctx, "zfs", "destroy", "-r", fullName).Run()
|
zfsCommand(ctx, "destroy", "-r", fullName).Run()
|
||||||
return nil, fmt.Errorf("invalid dataset info format")
|
return nil, fmt.Errorf("invalid dataset info format")
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -823,7 +883,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
|
|
||||||
// Determine dataset type
|
// Determine dataset type
|
||||||
datasetType := req.Type
|
datasetType := req.Type
|
||||||
typeCmd := exec.CommandContext(ctx, "zfs", "get", "-H", "-o", "value", "type", fullName)
|
typeCmd := zfsCommand(ctx, "get", "-H", "-o", "value", "type", fullName)
|
||||||
if typeOutput, err := typeCmd.Output(); err == nil {
|
if typeOutput, err := typeCmd.Output(); err == nil {
|
||||||
volType := strings.TrimSpace(string(typeOutput))
|
volType := strings.TrimSpace(string(typeOutput))
|
||||||
if volType == "volume" {
|
if volType == "volume" {
|
||||||
@@ -837,7 +897,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
quota := int64(-1)
|
quota := int64(-1)
|
||||||
if datasetType == "volume" {
|
if datasetType == "volume" {
|
||||||
// For volumes, get volsize
|
// For volumes, get volsize
|
||||||
volsizeCmd := exec.CommandContext(ctx, "zfs", "get", "-H", "-o", "value", "volsize", fullName)
|
volsizeCmd := zfsCommand(ctx, "get", "-H", "-o", "value", "volsize", fullName)
|
||||||
if volsizeOutput, err := volsizeCmd.Output(); err == nil {
|
if volsizeOutput, err := volsizeCmd.Output(); err == nil {
|
||||||
volsizeStr := strings.TrimSpace(string(volsizeOutput))
|
volsizeStr := strings.TrimSpace(string(volsizeOutput))
|
||||||
if volsizeStr != "-" && volsizeStr != "none" {
|
if volsizeStr != "-" && volsizeStr != "none" {
|
||||||
@@ -867,7 +927,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
|
|
||||||
// Get creation time
|
// Get creation time
|
||||||
createdAt := time.Now()
|
createdAt := time.Now()
|
||||||
creationCmd := exec.CommandContext(ctx, "zfs", "get", "-H", "-o", "value", "creation", fullName)
|
creationCmd := zfsCommand(ctx, "get", "-H", "-o", "value", "creation", fullName)
|
||||||
if creationOutput, err := creationCmd.Output(); err == nil {
|
if creationOutput, err := creationCmd.Output(); err == nil {
|
||||||
creationStr := strings.TrimSpace(string(creationOutput))
|
creationStr := strings.TrimSpace(string(creationOutput))
|
||||||
if t, err := time.Parse("Mon Jan 2 15:04:05 2006", creationStr); err == nil {
|
if t, err := time.Parse("Mon Jan 2 15:04:05 2006", creationStr); err == nil {
|
||||||
@@ -899,7 +959,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
s.logger.Error("Failed to save dataset to database", "name", fullName, "error", err)
|
s.logger.Error("Failed to save dataset to database", "name", fullName, "error", err)
|
||||||
// Try to destroy the dataset if we can't save to database
|
// Try to destroy the dataset if we can't save to database
|
||||||
exec.CommandContext(ctx, "zfs", "destroy", "-r", fullName).Run()
|
zfsCommand(ctx, "destroy", "-r", fullName).Run()
|
||||||
return nil, fmt.Errorf("failed to save dataset to database: %w", err)
|
return nil, fmt.Errorf("failed to save dataset to database: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -927,7 +987,7 @@ func (s *ZFSService) CreateDataset(ctx context.Context, poolName string, req Cre
|
|||||||
func (s *ZFSService) DeleteDataset(ctx context.Context, datasetName string) error {
|
func (s *ZFSService) DeleteDataset(ctx context.Context, datasetName string) error {
|
||||||
// Check if dataset exists and get its mount point before deletion
|
// Check if dataset exists and get its mount point before deletion
|
||||||
var mountPoint string
|
var mountPoint string
|
||||||
cmd := exec.CommandContext(ctx, "zfs", "list", "-H", "-o", "name,mountpoint", datasetName)
|
cmd := zfsCommand(ctx, "list", "-H", "-o", "name,mountpoint", datasetName)
|
||||||
output, err := cmd.CombinedOutput()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("dataset %s does not exist: %w", datasetName, err)
|
return fmt.Errorf("dataset %s does not exist: %w", datasetName, err)
|
||||||
@@ -946,7 +1006,7 @@ func (s *ZFSService) DeleteDataset(ctx context.Context, datasetName string) erro
|
|||||||
|
|
||||||
// Get dataset type to determine if we should clean up mount directory
|
// Get dataset type to determine if we should clean up mount directory
|
||||||
var datasetType string
|
var datasetType string
|
||||||
typeCmd := exec.CommandContext(ctx, "zfs", "get", "-H", "-o", "value", "type", datasetName)
|
typeCmd := zfsCommand(ctx, "get", "-H", "-o", "value", "type", datasetName)
|
||||||
typeOutput, err := typeCmd.Output()
|
typeOutput, err := typeCmd.Output()
|
||||||
if err == nil {
|
if err == nil {
|
||||||
datasetType = strings.TrimSpace(string(typeOutput))
|
datasetType = strings.TrimSpace(string(typeOutput))
|
||||||
@@ -969,7 +1029,7 @@ func (s *ZFSService) DeleteDataset(ctx context.Context, datasetName string) erro
|
|||||||
|
|
||||||
// Delete the dataset from ZFS (use -r for recursive to delete children)
|
// Delete the dataset from ZFS (use -r for recursive to delete children)
|
||||||
s.logger.Info("Deleting ZFS dataset", "name", datasetName, "mountpoint", mountPoint)
|
s.logger.Info("Deleting ZFS dataset", "name", datasetName, "mountpoint", mountPoint)
|
||||||
cmd = exec.CommandContext(ctx, "zfs", "destroy", "-r", datasetName)
|
cmd = zfsCommand(ctx, "destroy", "-r", datasetName)
|
||||||
output, err = cmd.CombinedOutput()
|
output, err = cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
errorMsg := string(output)
|
errorMsg := string(output)
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package storage
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"fmt"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"regexp"
|
"regexp"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -98,11 +99,17 @@ type PoolInfo struct {
|
|||||||
func (m *ZFSPoolMonitor) getSystemPools(ctx context.Context) (map[string]PoolInfo, error) {
|
func (m *ZFSPoolMonitor) getSystemPools(ctx context.Context) (map[string]PoolInfo, error) {
|
||||||
pools := make(map[string]PoolInfo)
|
pools := make(map[string]PoolInfo)
|
||||||
|
|
||||||
// Get pool list
|
// Get pool list (with sudo for permissions)
|
||||||
cmd := exec.CommandContext(ctx, "zpool", "list", "-H", "-o", "name,size,alloc,free,health")
|
cmd := exec.CommandContext(ctx, "sudo", "zpool", "list", "-H", "-o", "name,size,alloc,free,health")
|
||||||
output, err := cmd.Output()
|
output, err := cmd.CombinedOutput()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
// If no pools exist, zpool list returns exit code 1 but that's OK
|
||||||
|
// Check if output is empty (no pools) vs actual error
|
||||||
|
outputStr := strings.TrimSpace(string(output))
|
||||||
|
if outputStr == "" || strings.Contains(outputStr, "no pools available") {
|
||||||
|
return pools, nil // No pools, return empty map (not an error)
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("zpool list failed: %w, output: %s", err, outputStr)
|
||||||
}
|
}
|
||||||
|
|
||||||
lines := strings.Split(strings.TrimSpace(string(output)), "\n")
|
lines := strings.Split(strings.TrimSpace(string(output)), "\n")
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package system
|
|||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/atlasos/calypso/internal/common/logger"
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
"github.com/atlasos/calypso/internal/tasks"
|
"github.com/atlasos/calypso/internal/tasks"
|
||||||
@@ -131,3 +132,163 @@ func (h *Handler) ListNetworkInterfaces(c *gin.Context) {
|
|||||||
|
|
||||||
c.JSON(http.StatusOK, gin.H{"interfaces": interfaces})
|
c.JSON(http.StatusOK, gin.H{"interfaces": interfaces})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GetManagementIPAddress returns the management IP address
|
||||||
|
func (h *Handler) GetManagementIPAddress(c *gin.Context) {
|
||||||
|
ip, err := h.service.GetManagementIPAddress(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get management IP address", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get management IP address"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"ip_address": ip})
|
||||||
|
}
|
||||||
|
|
||||||
|
// SaveNTPSettings saves NTP configuration to the OS
|
||||||
|
func (h *Handler) SaveNTPSettings(c *gin.Context) {
|
||||||
|
var settings NTPSettings
|
||||||
|
if err := c.ShouldBindJSON(&settings); err != nil {
|
||||||
|
h.logger.Error("Invalid request body", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request body"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate timezone
|
||||||
|
if settings.Timezone == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "timezone is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate NTP servers
|
||||||
|
if len(settings.NTPServers) == 0 {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "at least one NTP server is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.service.SaveNTPSettings(c.Request.Context(), settings); err != nil {
|
||||||
|
h.logger.Error("Failed to save NTP settings", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"message": "NTP settings saved successfully"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetNTPSettings retrieves current NTP configuration
|
||||||
|
func (h *Handler) GetNTPSettings(c *gin.Context) {
|
||||||
|
settings, err := h.service.GetNTPSettings(c.Request.Context())
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get NTP settings", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get NTP settings"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"settings": settings})
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateNetworkInterface updates a network interface configuration
|
||||||
|
func (h *Handler) UpdateNetworkInterface(c *gin.Context) {
|
||||||
|
ifaceName := c.Param("name")
|
||||||
|
if ifaceName == "" {
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "interface name is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var req struct {
|
||||||
|
IPAddress string `json:"ip_address" binding:"required"`
|
||||||
|
Subnet string `json:"subnet" binding:"required"`
|
||||||
|
Gateway string `json:"gateway,omitempty"`
|
||||||
|
DNS1 string `json:"dns1,omitempty"`
|
||||||
|
DNS2 string `json:"dns2,omitempty"`
|
||||||
|
Role string `json:"role,omitempty"`
|
||||||
|
}
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid request body", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request body"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert to service request
|
||||||
|
serviceReq := UpdateNetworkInterfaceRequest{
|
||||||
|
IPAddress: req.IPAddress,
|
||||||
|
Subnet: req.Subnet,
|
||||||
|
Gateway: req.Gateway,
|
||||||
|
DNS1: req.DNS1,
|
||||||
|
DNS2: req.DNS2,
|
||||||
|
Role: req.Role,
|
||||||
|
}
|
||||||
|
|
||||||
|
updatedIface, err := h.service.UpdateNetworkInterface(c.Request.Context(), ifaceName, serviceReq)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to update network interface", "interface", ifaceName, "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"interface": updatedIface})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetSystemLogs retrieves recent system logs
|
||||||
|
func (h *Handler) GetSystemLogs(c *gin.Context) {
|
||||||
|
limitStr := c.DefaultQuery("limit", "30")
|
||||||
|
limit, err := strconv.Atoi(limitStr)
|
||||||
|
if err != nil || limit <= 0 || limit > 100 {
|
||||||
|
limit = 30
|
||||||
|
}
|
||||||
|
|
||||||
|
logs, err := h.service.GetSystemLogs(c.Request.Context(), limit)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get system logs", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get system logs"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"logs": logs})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetNetworkThroughput retrieves network throughput data from RRD
|
||||||
|
func (h *Handler) GetNetworkThroughput(c *gin.Context) {
|
||||||
|
// Default to last 5 minutes
|
||||||
|
durationStr := c.DefaultQuery("duration", "5m")
|
||||||
|
duration, err := time.ParseDuration(durationStr)
|
||||||
|
if err != nil {
|
||||||
|
duration = 5 * time.Minute
|
||||||
|
}
|
||||||
|
|
||||||
|
data, err := h.service.GetNetworkThroughput(c.Request.Context(), duration)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to get network throughput", "error", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to get network throughput"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"data": data})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExecuteCommand executes a shell command
|
||||||
|
func (h *Handler) ExecuteCommand(c *gin.Context) {
|
||||||
|
var req struct {
|
||||||
|
Command string `json:"command" binding:"required"`
|
||||||
|
Service string `json:"service,omitempty"` // Optional: system, scst, storage, backup, tape
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
|
h.logger.Error("Invalid request body", "error", err)
|
||||||
|
c.JSON(http.StatusBadRequest, gin.H{"error": "command is required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute command based on service context
|
||||||
|
output, err := h.service.ExecuteCommand(c.Request.Context(), req.Command, req.Service)
|
||||||
|
if err != nil {
|
||||||
|
h.logger.Error("Failed to execute command", "error", err, "command", req.Command, "service", req.Service)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{
|
||||||
|
"error": err.Error(),
|
||||||
|
"output": output, // Include output even on error
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"output": output})
|
||||||
|
}
|
||||||
|
|||||||
292
backend/internal/system/rrd.go
Normal file
292
backend/internal/system/rrd.go
Normal file
@@ -0,0 +1,292 @@
|
|||||||
|
package system
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RRDService handles RRD database operations for network monitoring
|
||||||
|
type RRDService struct {
|
||||||
|
logger *logger.Logger
|
||||||
|
rrdDir string
|
||||||
|
interfaceName string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewRRDService creates a new RRD service
|
||||||
|
func NewRRDService(log *logger.Logger, rrdDir string, interfaceName string) *RRDService {
|
||||||
|
return &RRDService{
|
||||||
|
logger: log,
|
||||||
|
rrdDir: rrdDir,
|
||||||
|
interfaceName: interfaceName,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NetworkStats represents network interface statistics
|
||||||
|
type NetworkStats struct {
|
||||||
|
Interface string `json:"interface"`
|
||||||
|
RxBytes uint64 `json:"rx_bytes"`
|
||||||
|
TxBytes uint64 `json:"tx_bytes"`
|
||||||
|
RxPackets uint64 `json:"rx_packets"`
|
||||||
|
TxPackets uint64 `json:"tx_packets"`
|
||||||
|
Timestamp time.Time `json:"timestamp"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetNetworkStats reads network statistics from /proc/net/dev
|
||||||
|
func (r *RRDService) GetNetworkStats(ctx context.Context, interfaceName string) (*NetworkStats, error) {
|
||||||
|
data, err := os.ReadFile("/proc/net/dev")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to read /proc/net/dev: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(string(data), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if !strings.HasPrefix(line, interfaceName+":") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse line: interface: rx_bytes rx_packets ... tx_bytes tx_packets ...
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
if len(parts) < 17 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract statistics
|
||||||
|
// Format: interface: rx_bytes rx_packets rx_errs rx_drop ... tx_bytes tx_packets ...
|
||||||
|
rxBytes, err := strconv.ParseUint(parts[1], 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rxPackets, err := strconv.ParseUint(parts[2], 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
txBytes, err := strconv.ParseUint(parts[9], 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
txPackets, err := strconv.ParseUint(parts[10], 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
return &NetworkStats{
|
||||||
|
Interface: interfaceName,
|
||||||
|
RxBytes: rxBytes,
|
||||||
|
TxBytes: txBytes,
|
||||||
|
RxPackets: rxPackets,
|
||||||
|
TxPackets: txPackets,
|
||||||
|
Timestamp: time.Now(),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil, fmt.Errorf("interface %s not found in /proc/net/dev", interfaceName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// InitializeRRD creates RRD database if it doesn't exist
|
||||||
|
func (r *RRDService) InitializeRRD(ctx context.Context) error {
|
||||||
|
// Ensure RRD directory exists
|
||||||
|
if err := os.MkdirAll(r.rrdDir, 0755); err != nil {
|
||||||
|
return fmt.Errorf("failed to create RRD directory: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rrdFile := filepath.Join(r.rrdDir, fmt.Sprintf("network-%s.rrd", r.interfaceName))
|
||||||
|
|
||||||
|
// Check if RRD file already exists
|
||||||
|
if _, err := os.Stat(rrdFile); err == nil {
|
||||||
|
r.logger.Info("RRD file already exists", "file", rrdFile)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create RRD database
|
||||||
|
// Use COUNTER type to track cumulative bytes, RRD will calculate rate automatically
|
||||||
|
// DS:inbound:COUNTER:20:0:U - inbound cumulative bytes, 20s heartbeat
|
||||||
|
// DS:outbound:COUNTER:20:0:U - outbound cumulative bytes, 20s heartbeat
|
||||||
|
// RRA:AVERAGE:0.5:1:600 - 1 sample per step, 600 steps (100 minutes at 10s interval)
|
||||||
|
// RRA:AVERAGE:0.5:6:700 - 6 samples per step, 700 steps (11.6 hours at 1min interval)
|
||||||
|
// RRA:AVERAGE:0.5:60:730 - 60 samples per step, 730 steps (5 days at 1hour interval)
|
||||||
|
// RRA:MAX:0.5:1:600 - Max values for same intervals
|
||||||
|
// RRA:MAX:0.5:6:700
|
||||||
|
// RRA:MAX:0.5:60:730
|
||||||
|
cmd := exec.CommandContext(ctx, "rrdtool", "create", rrdFile,
|
||||||
|
"--step", "10", // 10 second step
|
||||||
|
"DS:inbound:COUNTER:20:0:U", // Inbound cumulative bytes, 20s heartbeat
|
||||||
|
"DS:outbound:COUNTER:20:0:U", // Outbound cumulative bytes, 20s heartbeat
|
||||||
|
"RRA:AVERAGE:0.5:1:600", // 10s resolution, 100 minutes
|
||||||
|
"RRA:AVERAGE:0.5:6:700", // 1min resolution, 11.6 hours
|
||||||
|
"RRA:AVERAGE:0.5:60:730", // 1hour resolution, 5 days
|
||||||
|
"RRA:MAX:0.5:1:600", // Max values
|
||||||
|
"RRA:MAX:0.5:6:700",
|
||||||
|
"RRA:MAX:0.5:60:730",
|
||||||
|
)
|
||||||
|
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to create RRD: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
r.logger.Info("RRD database created", "file", rrdFile)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateRRD updates RRD database with new network statistics
|
||||||
|
func (r *RRDService) UpdateRRD(ctx context.Context, stats *NetworkStats) error {
|
||||||
|
rrdFile := filepath.Join(r.rrdDir, fmt.Sprintf("network-%s.rrd", stats.Interface))
|
||||||
|
|
||||||
|
// Update with cumulative byte counts (COUNTER type)
|
||||||
|
// RRD will automatically calculate the rate (bytes per second)
|
||||||
|
cmd := exec.CommandContext(ctx, "rrdtool", "update", rrdFile,
|
||||||
|
fmt.Sprintf("%d:%d:%d", stats.Timestamp.Unix(), stats.RxBytes, stats.TxBytes),
|
||||||
|
)
|
||||||
|
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to update RRD: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// FetchRRDData fetches data from RRD database for graphing
|
||||||
|
func (r *RRDService) FetchRRDData(ctx context.Context, startTime time.Time, endTime time.Time, resolution string) ([]NetworkDataPoint, error) {
|
||||||
|
rrdFile := filepath.Join(r.rrdDir, fmt.Sprintf("network-%s.rrd", r.interfaceName))
|
||||||
|
|
||||||
|
// Check if RRD file exists
|
||||||
|
if _, err := os.Stat(rrdFile); os.IsNotExist(err) {
|
||||||
|
return []NetworkDataPoint{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fetch data using rrdtool fetch
|
||||||
|
// Use AVERAGE consolidation with appropriate resolution
|
||||||
|
cmd := exec.CommandContext(ctx, "rrdtool", "fetch", rrdFile,
|
||||||
|
"AVERAGE",
|
||||||
|
"--start", fmt.Sprintf("%d", startTime.Unix()),
|
||||||
|
"--end", fmt.Sprintf("%d", endTime.Unix()),
|
||||||
|
"--resolution", resolution,
|
||||||
|
)
|
||||||
|
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to fetch RRD data: %s: %w", string(output), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse rrdtool fetch output
|
||||||
|
// Format:
|
||||||
|
// inbound outbound
|
||||||
|
// 1234567890: 1.2345678901e+06 2.3456789012e+06
|
||||||
|
points := []NetworkDataPoint{}
|
||||||
|
lines := strings.Split(string(output), "\n")
|
||||||
|
|
||||||
|
// Skip header lines
|
||||||
|
dataStart := false
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if this is the data section
|
||||||
|
if strings.Contains(line, "inbound") && strings.Contains(line, "outbound") {
|
||||||
|
dataStart = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if !dataStart {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse data line: timestamp: inbound_value outbound_value
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
if len(parts) < 3 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse timestamp
|
||||||
|
timestampStr := strings.TrimSuffix(parts[0], ":")
|
||||||
|
timestamp, err := strconv.ParseInt(timestampStr, 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse inbound (bytes per second from COUNTER, convert to Mbps)
|
||||||
|
inboundStr := parts[1]
|
||||||
|
inbound, err := strconv.ParseFloat(inboundStr, 64)
|
||||||
|
if err != nil || inbound < 0 {
|
||||||
|
// Skip NaN or negative values
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// Convert bytes per second to Mbps (bytes/s * 8 / 1000000)
|
||||||
|
inboundMbps := inbound * 8 / 1000000
|
||||||
|
|
||||||
|
// Parse outbound
|
||||||
|
outboundStr := parts[2]
|
||||||
|
outbound, err := strconv.ParseFloat(outboundStr, 64)
|
||||||
|
if err != nil || outbound < 0 {
|
||||||
|
// Skip NaN or negative values
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
outboundMbps := outbound * 8 / 1000000
|
||||||
|
|
||||||
|
// Format time as MM:SS
|
||||||
|
t := time.Unix(timestamp, 0)
|
||||||
|
timeStr := fmt.Sprintf("%02d:%02d", t.Minute(), t.Second())
|
||||||
|
|
||||||
|
points = append(points, NetworkDataPoint{
|
||||||
|
Time: timeStr,
|
||||||
|
Inbound: inboundMbps,
|
||||||
|
Outbound: outboundMbps,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
return points, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// NetworkDataPoint represents a single data point for graphing
|
||||||
|
type NetworkDataPoint struct {
|
||||||
|
Time string `json:"time"`
|
||||||
|
Inbound float64 `json:"inbound"` // Mbps
|
||||||
|
Outbound float64 `json:"outbound"` // Mbps
|
||||||
|
}
|
||||||
|
|
||||||
|
// StartCollector starts a background goroutine to periodically collect and update RRD
|
||||||
|
func (r *RRDService) StartCollector(ctx context.Context, interval time.Duration) error {
|
||||||
|
// Initialize RRD if needed
|
||||||
|
if err := r.InitializeRRD(ctx); err != nil {
|
||||||
|
return fmt.Errorf("failed to initialize RRD: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
ticker := time.NewTicker(interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
// Get current stats
|
||||||
|
stats, err := r.GetNetworkStats(ctx, r.interfaceName)
|
||||||
|
if err != nil {
|
||||||
|
r.logger.Warn("Failed to get network stats", "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update RRD with cumulative byte counts
|
||||||
|
// RRD COUNTER type will automatically calculate rate
|
||||||
|
if err := r.UpdateRRD(ctx, stats); err != nil {
|
||||||
|
r.logger.Warn("Failed to update RRD", "error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
@@ -11,18 +12,98 @@ import (
|
|||||||
"github.com/atlasos/calypso/internal/common/logger"
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// NTPSettings represents NTP configuration
|
||||||
|
type NTPSettings struct {
|
||||||
|
Timezone string `json:"timezone"`
|
||||||
|
NTPServers []string `json:"ntp_servers"`
|
||||||
|
}
|
||||||
|
|
||||||
// Service handles system management operations
|
// Service handles system management operations
|
||||||
type Service struct {
|
type Service struct {
|
||||||
logger *logger.Logger
|
logger *logger.Logger
|
||||||
|
rrdService *RRDService
|
||||||
|
}
|
||||||
|
|
||||||
|
// detectPrimaryInterface detects the primary network interface (first non-loopback with IP)
|
||||||
|
func detectPrimaryInterface(ctx context.Context) string {
|
||||||
|
// Try to get default route interface
|
||||||
|
cmd := exec.CommandContext(ctx, "ip", "route", "show", "default")
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err == nil {
|
||||||
|
lines := strings.Split(string(output), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
if strings.Contains(line, "dev ") {
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
for i, part := range parts {
|
||||||
|
if part == "dev" && i+1 < len(parts) {
|
||||||
|
iface := parts[i+1]
|
||||||
|
if iface != "lo" {
|
||||||
|
return iface
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fallback: get first non-loopback interface with IP
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "-4", "addr", "show")
|
||||||
|
output, err = cmd.Output()
|
||||||
|
if err == nil {
|
||||||
|
lines := strings.Split(string(output), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
// Look for interface name line (e.g., "2: ens18: <BROADCAST...")
|
||||||
|
if len(line) > 0 && line[0] >= '0' && line[0] <= '9' && strings.Contains(line, ":") {
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
if len(parts) >= 2 {
|
||||||
|
iface := strings.TrimSuffix(parts[1], ":")
|
||||||
|
if iface != "" && iface != "lo" {
|
||||||
|
// Check if this interface has an IP (next lines will have "inet")
|
||||||
|
// For simplicity, return first non-loopback interface
|
||||||
|
return iface
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Final fallback
|
||||||
|
return "eth0"
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewService creates a new system service
|
// NewService creates a new system service
|
||||||
func NewService(log *logger.Logger) *Service {
|
func NewService(log *logger.Logger) *Service {
|
||||||
|
// Initialize RRD service for network monitoring
|
||||||
|
rrdDir := "/var/lib/calypso/rrd"
|
||||||
|
|
||||||
|
// Auto-detect primary interface
|
||||||
|
ctx := context.Background()
|
||||||
|
interfaceName := detectPrimaryInterface(ctx)
|
||||||
|
log.Info("Detected primary network interface", "interface", interfaceName)
|
||||||
|
|
||||||
|
rrdService := NewRRDService(log, rrdDir, interfaceName)
|
||||||
|
|
||||||
return &Service{
|
return &Service{
|
||||||
logger: log,
|
logger: log,
|
||||||
|
rrdService: rrdService,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// StartNetworkMonitoring starts the RRD collector for network monitoring
|
||||||
|
func (s *Service) StartNetworkMonitoring(ctx context.Context) error {
|
||||||
|
return s.rrdService.StartCollector(ctx, 10*time.Second)
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetNetworkThroughput fetches network throughput data from RRD
|
||||||
|
func (s *Service) GetNetworkThroughput(ctx context.Context, duration time.Duration) ([]NetworkDataPoint, error) {
|
||||||
|
endTime := time.Now()
|
||||||
|
startTime := endTime.Add(-duration)
|
||||||
|
|
||||||
|
// Use 10 second resolution for recent data
|
||||||
|
return s.rrdService.FetchRRDData(ctx, startTime, endTime, "10")
|
||||||
|
}
|
||||||
|
|
||||||
// ServiceStatus represents a systemd service status
|
// ServiceStatus represents a systemd service status
|
||||||
type ServiceStatus struct {
|
type ServiceStatus struct {
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
@@ -35,31 +116,37 @@ type ServiceStatus struct {
|
|||||||
|
|
||||||
// GetServiceStatus retrieves the status of a systemd service
|
// GetServiceStatus retrieves the status of a systemd service
|
||||||
func (s *Service) GetServiceStatus(ctx context.Context, serviceName string) (*ServiceStatus, error) {
|
func (s *Service) GetServiceStatus(ctx context.Context, serviceName string) (*ServiceStatus, error) {
|
||||||
cmd := exec.CommandContext(ctx, "systemctl", "show", serviceName,
|
|
||||||
"--property=ActiveState,SubState,LoadState,Description,ActiveEnterTimestamp",
|
|
||||||
"--value", "--no-pager")
|
|
||||||
output, err := cmd.Output()
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to get service status: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
lines := strings.Split(strings.TrimSpace(string(output)), "\n")
|
|
||||||
if len(lines) < 4 {
|
|
||||||
return nil, fmt.Errorf("invalid service status output")
|
|
||||||
}
|
|
||||||
|
|
||||||
status := &ServiceStatus{
|
status := &ServiceStatus{
|
||||||
Name: serviceName,
|
Name: serviceName,
|
||||||
ActiveState: strings.TrimSpace(lines[0]),
|
|
||||||
SubState: strings.TrimSpace(lines[1]),
|
|
||||||
LoadState: strings.TrimSpace(lines[2]),
|
|
||||||
Description: strings.TrimSpace(lines[3]),
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Parse timestamp if available
|
// Get each property individually to ensure correct parsing
|
||||||
if len(lines) > 4 && lines[4] != "" {
|
properties := map[string]*string{
|
||||||
if t, err := time.Parse("Mon 2006-01-02 15:04:05 MST", strings.TrimSpace(lines[4])); err == nil {
|
"ActiveState": &status.ActiveState,
|
||||||
status.Since = t
|
"SubState": &status.SubState,
|
||||||
|
"LoadState": &status.LoadState,
|
||||||
|
"Description": &status.Description,
|
||||||
|
}
|
||||||
|
|
||||||
|
for prop, target := range properties {
|
||||||
|
cmd := exec.CommandContext(ctx, "systemctl", "show", serviceName, "--property", prop, "--value", "--no-pager")
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to get property", "service", serviceName, "property", prop, "error", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
*target = strings.TrimSpace(string(output))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get timestamp if available
|
||||||
|
cmd := exec.CommandContext(ctx, "systemctl", "show", serviceName, "--property", "ActiveEnterTimestamp", "--value", "--no-pager")
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err == nil {
|
||||||
|
timestamp := strings.TrimSpace(string(output))
|
||||||
|
if timestamp != "" {
|
||||||
|
if t, err := time.Parse("Mon 2006-01-02 15:04:05 MST", timestamp); err == nil {
|
||||||
|
status.Since = t
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -69,10 +156,15 @@ func (s *Service) GetServiceStatus(ctx context.Context, serviceName string) (*Se
|
|||||||
// ListServices lists all Calypso-related services
|
// ListServices lists all Calypso-related services
|
||||||
func (s *Service) ListServices(ctx context.Context) ([]ServiceStatus, error) {
|
func (s *Service) ListServices(ctx context.Context) ([]ServiceStatus, error) {
|
||||||
services := []string{
|
services := []string{
|
||||||
|
"ssh",
|
||||||
|
"sshd",
|
||||||
|
"smbd",
|
||||||
|
"iscsi-scst",
|
||||||
|
"nfs-server",
|
||||||
|
"nfs",
|
||||||
|
"mhvtl",
|
||||||
"calypso-api",
|
"calypso-api",
|
||||||
"scst",
|
"scst",
|
||||||
"iscsi-scst",
|
|
||||||
"mhvtl",
|
|
||||||
"postgresql",
|
"postgresql",
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -128,6 +220,108 @@ func (s *Service) GetJournalLogs(ctx context.Context, serviceName string, lines
|
|||||||
return logs, nil
|
return logs, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SystemLogEntry represents a parsed system log entry
|
||||||
|
type SystemLogEntry struct {
|
||||||
|
Time string `json:"time"`
|
||||||
|
Level string `json:"level"`
|
||||||
|
Source string `json:"source"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetSystemLogs retrieves recent system logs from journalctl
|
||||||
|
func (s *Service) GetSystemLogs(ctx context.Context, limit int) ([]SystemLogEntry, error) {
|
||||||
|
if limit <= 0 || limit > 100 {
|
||||||
|
limit = 30 // Default to 30 logs
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd := exec.CommandContext(ctx, "journalctl",
|
||||||
|
"-n", fmt.Sprintf("%d", limit),
|
||||||
|
"-o", "json",
|
||||||
|
"--no-pager",
|
||||||
|
"--since", "1 hour ago") // Only get logs from last hour
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to get system logs: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var logs []SystemLogEntry
|
||||||
|
linesOutput := strings.Split(strings.TrimSpace(string(output)), "\n")
|
||||||
|
for _, line := range linesOutput {
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var logEntry map[string]interface{}
|
||||||
|
if err := json.Unmarshal([]byte(line), &logEntry); err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse timestamp (__REALTIME_TIMESTAMP is in microseconds)
|
||||||
|
var timeStr string
|
||||||
|
if timestamp, ok := logEntry["__REALTIME_TIMESTAMP"].(float64); ok {
|
||||||
|
// Convert microseconds to nanoseconds for time.Unix (1 microsecond = 1000 nanoseconds)
|
||||||
|
t := time.Unix(0, int64(timestamp)*1000)
|
||||||
|
timeStr = t.Format("15:04:05")
|
||||||
|
} else if timestamp, ok := logEntry["_SOURCE_REALTIME_TIMESTAMP"].(float64); ok {
|
||||||
|
t := time.Unix(0, int64(timestamp)*1000)
|
||||||
|
timeStr = t.Format("15:04:05")
|
||||||
|
} else {
|
||||||
|
timeStr = time.Now().Format("15:04:05")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse log level (priority)
|
||||||
|
level := "INFO"
|
||||||
|
if priority, ok := logEntry["PRIORITY"].(float64); ok {
|
||||||
|
switch int(priority) {
|
||||||
|
case 0: // emerg
|
||||||
|
level = "EMERG"
|
||||||
|
case 1, 2, 3: // alert, crit, err
|
||||||
|
level = "ERROR"
|
||||||
|
case 4: // warning
|
||||||
|
level = "WARN"
|
||||||
|
case 5: // notice
|
||||||
|
level = "NOTICE"
|
||||||
|
case 6: // info
|
||||||
|
level = "INFO"
|
||||||
|
case 7: // debug
|
||||||
|
level = "DEBUG"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse source (systemd unit or syslog identifier)
|
||||||
|
source := "system"
|
||||||
|
if unit, ok := logEntry["_SYSTEMD_UNIT"].(string); ok && unit != "" {
|
||||||
|
// Remove .service suffix if present
|
||||||
|
source = strings.TrimSuffix(unit, ".service")
|
||||||
|
} else if ident, ok := logEntry["SYSLOG_IDENTIFIER"].(string); ok && ident != "" {
|
||||||
|
source = ident
|
||||||
|
} else if comm, ok := logEntry["_COMM"].(string); ok && comm != "" {
|
||||||
|
source = comm
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse message
|
||||||
|
message := ""
|
||||||
|
if msg, ok := logEntry["MESSAGE"].(string); ok {
|
||||||
|
message = msg
|
||||||
|
}
|
||||||
|
|
||||||
|
if message != "" {
|
||||||
|
logs = append(logs, SystemLogEntry{
|
||||||
|
Time: timeStr,
|
||||||
|
Level: level,
|
||||||
|
Source: source,
|
||||||
|
Message: message,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reverse to get newest first
|
||||||
|
for i, j := 0, len(logs)-1; i < j; i, j = i+1, j-1 {
|
||||||
|
logs[i], logs[j] = logs[j], logs[i]
|
||||||
|
}
|
||||||
|
|
||||||
|
return logs, nil
|
||||||
|
}
|
||||||
|
|
||||||
// GenerateSupportBundle generates a diagnostic support bundle
|
// GenerateSupportBundle generates a diagnostic support bundle
|
||||||
func (s *Service) GenerateSupportBundle(ctx context.Context, outputPath string) error {
|
func (s *Service) GenerateSupportBundle(ctx context.Context, outputPath string) error {
|
||||||
// Create bundle directory
|
// Create bundle directory
|
||||||
@@ -183,6 +377,9 @@ type NetworkInterface struct {
|
|||||||
Status string `json:"status"` // "Connected" or "Down"
|
Status string `json:"status"` // "Connected" or "Down"
|
||||||
Speed string `json:"speed"` // e.g., "10 Gbps", "1 Gbps"
|
Speed string `json:"speed"` // e.g., "10 Gbps", "1 Gbps"
|
||||||
Role string `json:"role"` // "Management", "ISCSI", or empty
|
Role string `json:"role"` // "Management", "ISCSI", or empty
|
||||||
|
Gateway string `json:"gateway,omitempty"`
|
||||||
|
DNS1 string `json:"dns1,omitempty"`
|
||||||
|
DNS2 string `json:"dns2,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ListNetworkInterfaces lists all network interfaces
|
// ListNetworkInterfaces lists all network interfaces
|
||||||
@@ -297,6 +494,103 @@ func (s *Service) ListNetworkInterfaces(ctx context.Context) ([]NetworkInterface
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Get default gateway for each interface
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "route", "show")
|
||||||
|
output, err = cmd.Output()
|
||||||
|
if err == nil {
|
||||||
|
lines = strings.Split(string(output), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse default route: "default via 10.10.14.1 dev ens18"
|
||||||
|
if strings.HasPrefix(line, "default via ") {
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
// Find "via" and "dev" in the parts
|
||||||
|
var gateway string
|
||||||
|
var ifaceName string
|
||||||
|
for i, part := range parts {
|
||||||
|
if part == "via" && i+1 < len(parts) {
|
||||||
|
gateway = parts[i+1]
|
||||||
|
}
|
||||||
|
if part == "dev" && i+1 < len(parts) {
|
||||||
|
ifaceName = parts[i+1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if gateway != "" && ifaceName != "" {
|
||||||
|
if iface, exists := interfaceMap[ifaceName]; exists {
|
||||||
|
iface.Gateway = gateway
|
||||||
|
s.logger.Info("Set default gateway for interface", "name", ifaceName, "gateway", gateway)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if strings.Contains(line, " via ") && strings.Contains(line, " dev ") {
|
||||||
|
// Parse network route: "10.10.14.0/24 via 10.10.14.1 dev ens18"
|
||||||
|
// Or: "192.168.1.0/24 via 192.168.1.1 dev eth0"
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
var gateway string
|
||||||
|
var ifaceName string
|
||||||
|
for i, part := range parts {
|
||||||
|
if part == "via" && i+1 < len(parts) {
|
||||||
|
gateway = parts[i+1]
|
||||||
|
}
|
||||||
|
if part == "dev" && i+1 < len(parts) {
|
||||||
|
ifaceName = parts[i+1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Only set gateway if it's not already set (prefer default route)
|
||||||
|
if gateway != "" && ifaceName != "" {
|
||||||
|
if iface, exists := interfaceMap[ifaceName]; exists {
|
||||||
|
if iface.Gateway == "" {
|
||||||
|
iface.Gateway = gateway
|
||||||
|
s.logger.Info("Set gateway from network route for interface", "name", ifaceName, "gateway", gateway)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
s.logger.Warn("Failed to get routes", "error", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get DNS servers from systemd-resolved or /etc/resolv.conf
|
||||||
|
// Try systemd-resolved first
|
||||||
|
cmd = exec.CommandContext(ctx, "systemd-resolve", "--status")
|
||||||
|
output, err = cmd.Output()
|
||||||
|
dnsServers := []string{}
|
||||||
|
if err == nil {
|
||||||
|
// Parse DNS from systemd-resolve output
|
||||||
|
lines = strings.Split(string(output), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if strings.HasPrefix(line, "DNS Servers:") {
|
||||||
|
// Format: "DNS Servers: 8.8.8.8 8.8.4.4"
|
||||||
|
parts := strings.Fields(line)
|
||||||
|
if len(parts) >= 3 {
|
||||||
|
dnsServers = parts[2:]
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Fallback to /etc/resolv.conf
|
||||||
|
data, err := os.ReadFile("/etc/resolv.conf")
|
||||||
|
if err == nil {
|
||||||
|
lines = strings.Split(string(data), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if strings.HasPrefix(line, "nameserver ") {
|
||||||
|
dns := strings.TrimPrefix(line, "nameserver ")
|
||||||
|
dns = strings.TrimSpace(dns)
|
||||||
|
if dns != "" {
|
||||||
|
dnsServers = append(dnsServers, dns)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Convert map to slice
|
// Convert map to slice
|
||||||
var interfaces []NetworkInterface
|
var interfaces []NetworkInterface
|
||||||
s.logger.Debug("Converting interface map to slice", "map_size", len(interfaceMap))
|
s.logger.Debug("Converting interface map to slice", "map_size", len(interfaceMap))
|
||||||
@@ -319,6 +613,14 @@ func (s *Service) ListNetworkInterfaces(ctx context.Context) ([]NetworkInterface
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Set DNS servers (use first two if available)
|
||||||
|
if len(dnsServers) > 0 {
|
||||||
|
iface.DNS1 = dnsServers[0]
|
||||||
|
}
|
||||||
|
if len(dnsServers) > 1 {
|
||||||
|
iface.DNS2 = dnsServers[1]
|
||||||
|
}
|
||||||
|
|
||||||
// Determine role based on interface name or IP (simple heuristic)
|
// Determine role based on interface name or IP (simple heuristic)
|
||||||
// You can enhance this with configuration file or database lookup
|
// You can enhance this with configuration file or database lookup
|
||||||
if strings.Contains(iface.Name, "eth") || strings.Contains(iface.Name, "ens") {
|
if strings.Contains(iface.Name, "eth") || strings.Contains(iface.Name, "ens") {
|
||||||
@@ -345,3 +647,401 @@ func (s *Service) ListNetworkInterfaces(ctx context.Context) ([]NetworkInterface
|
|||||||
s.logger.Info("Listed network interfaces", "count", len(interfaces))
|
s.logger.Info("Listed network interfaces", "count", len(interfaces))
|
||||||
return interfaces, nil
|
return interfaces, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GetManagementIPAddress returns the IP address of the management interface
|
||||||
|
func (s *Service) GetManagementIPAddress(ctx context.Context) (string, error) {
|
||||||
|
interfaces, err := s.ListNetworkInterfaces(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("failed to list network interfaces: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// First, try to find interface with Role "Management"
|
||||||
|
for _, iface := range interfaces {
|
||||||
|
if iface.Role == "Management" && iface.IPAddress != "" && iface.Status == "Connected" {
|
||||||
|
s.logger.Info("Found management interface", "interface", iface.Name, "ip", iface.IPAddress)
|
||||||
|
return iface.IPAddress, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fallback: use interface with default route (primary interface)
|
||||||
|
for _, iface := range interfaces {
|
||||||
|
if iface.Gateway != "" && iface.IPAddress != "" && iface.Status == "Connected" {
|
||||||
|
s.logger.Info("Using primary interface as management", "interface", iface.Name, "ip", iface.IPAddress)
|
||||||
|
return iface.IPAddress, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Final fallback: use first connected interface with IP
|
||||||
|
for _, iface := range interfaces {
|
||||||
|
if iface.IPAddress != "" && iface.Status == "Connected" && iface.Name != "lo" {
|
||||||
|
s.logger.Info("Using first connected interface as management", "interface", iface.Name, "ip", iface.IPAddress)
|
||||||
|
return iface.IPAddress, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", fmt.Errorf("no management interface found")
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateNetworkInterfaceRequest represents the request to update a network interface
|
||||||
|
type UpdateNetworkInterfaceRequest struct {
|
||||||
|
IPAddress string `json:"ip_address"`
|
||||||
|
Subnet string `json:"subnet"`
|
||||||
|
Gateway string `json:"gateway,omitempty"`
|
||||||
|
DNS1 string `json:"dns1,omitempty"`
|
||||||
|
DNS2 string `json:"dns2,omitempty"`
|
||||||
|
Role string `json:"role,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateNetworkInterface updates network interface configuration
|
||||||
|
func (s *Service) UpdateNetworkInterface(ctx context.Context, ifaceName string, req UpdateNetworkInterfaceRequest) (*NetworkInterface, error) {
|
||||||
|
// Validate interface exists
|
||||||
|
cmd := exec.CommandContext(ctx, "ip", "link", "show", ifaceName)
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
return nil, fmt.Errorf("interface %s not found: %w", ifaceName, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove existing IP address if any
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "addr", "flush", "dev", ifaceName)
|
||||||
|
cmd.Run() // Ignore error, interface might not have IP
|
||||||
|
|
||||||
|
// Set new IP address and subnet
|
||||||
|
ipWithSubnet := fmt.Sprintf("%s/%s", req.IPAddress, req.Subnet)
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "addr", "add", ipWithSubnet, "dev", ifaceName)
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to set IP address", "interface", ifaceName, "error", err, "output", string(output))
|
||||||
|
return nil, fmt.Errorf("failed to set IP address: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove existing default route if any
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "route", "del", "default")
|
||||||
|
cmd.Run() // Ignore error, might not exist
|
||||||
|
|
||||||
|
// Set gateway if provided
|
||||||
|
if req.Gateway != "" {
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "route", "add", "default", "via", req.Gateway, "dev", ifaceName)
|
||||||
|
output, err = cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to set gateway", "interface", ifaceName, "error", err, "output", string(output))
|
||||||
|
return nil, fmt.Errorf("failed to set gateway: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update DNS in systemd-resolved or /etc/resolv.conf
|
||||||
|
if req.DNS1 != "" || req.DNS2 != "" {
|
||||||
|
// Try using systemd-resolve first
|
||||||
|
cmd = exec.CommandContext(ctx, "systemd-resolve", "--status")
|
||||||
|
if cmd.Run() == nil {
|
||||||
|
// systemd-resolve is available, use it
|
||||||
|
dnsServers := []string{}
|
||||||
|
if req.DNS1 != "" {
|
||||||
|
dnsServers = append(dnsServers, req.DNS1)
|
||||||
|
}
|
||||||
|
if req.DNS2 != "" {
|
||||||
|
dnsServers = append(dnsServers, req.DNS2)
|
||||||
|
}
|
||||||
|
if len(dnsServers) > 0 {
|
||||||
|
// Use resolvectl to set DNS (newer systemd)
|
||||||
|
cmd = exec.CommandContext(ctx, "resolvectl", "dns", ifaceName, strings.Join(dnsServers, " "))
|
||||||
|
if cmd.Run() != nil {
|
||||||
|
// Fallback to systemd-resolve
|
||||||
|
cmd = exec.CommandContext(ctx, "systemd-resolve", "--interface", ifaceName, "--set-dns", strings.Join(dnsServers, " "))
|
||||||
|
output, err = cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to set DNS via systemd-resolve", "error", err, "output", string(output))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Fallback: update /etc/resolv.conf
|
||||||
|
resolvContent := "# Generated by Calypso\n"
|
||||||
|
if req.DNS1 != "" {
|
||||||
|
resolvContent += fmt.Sprintf("nameserver %s\n", req.DNS1)
|
||||||
|
}
|
||||||
|
if req.DNS2 != "" {
|
||||||
|
resolvContent += fmt.Sprintf("nameserver %s\n", req.DNS2)
|
||||||
|
}
|
||||||
|
|
||||||
|
tmpPath := "/tmp/resolv.conf." + fmt.Sprintf("%d", time.Now().Unix())
|
||||||
|
if err := os.WriteFile(tmpPath, []byte(resolvContent), 0644); err != nil {
|
||||||
|
s.logger.Warn("Failed to write temporary resolv.conf", "error", err)
|
||||||
|
} else {
|
||||||
|
cmd = exec.CommandContext(ctx, "sh", "-c", fmt.Sprintf("mv %s /etc/resolv.conf", tmpPath))
|
||||||
|
output, err = cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to update /etc/resolv.conf", "error", err, "output", string(output))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bring interface up
|
||||||
|
cmd = exec.CommandContext(ctx, "ip", "link", "set", ifaceName, "up")
|
||||||
|
output, err = cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to bring interface up", "interface", ifaceName, "error", err, "output", string(output))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return updated interface
|
||||||
|
updatedIface := &NetworkInterface{
|
||||||
|
Name: ifaceName,
|
||||||
|
IPAddress: req.IPAddress,
|
||||||
|
Subnet: req.Subnet,
|
||||||
|
Gateway: req.Gateway,
|
||||||
|
DNS1: req.DNS1,
|
||||||
|
DNS2: req.DNS2,
|
||||||
|
Role: req.Role,
|
||||||
|
Status: "Connected",
|
||||||
|
Speed: "Unknown", // Will be updated on next list
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Updated network interface", "interface", ifaceName, "ip", req.IPAddress, "subnet", req.Subnet)
|
||||||
|
return updatedIface, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SaveNTPSettings saves NTP configuration to the OS
|
||||||
|
func (s *Service) SaveNTPSettings(ctx context.Context, settings NTPSettings) error {
|
||||||
|
// Set timezone using timedatectl
|
||||||
|
if settings.Timezone != "" {
|
||||||
|
cmd := exec.CommandContext(ctx, "timedatectl", "set-timezone", settings.Timezone)
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to set timezone", "timezone", settings.Timezone, "error", err, "output", string(output))
|
||||||
|
return fmt.Errorf("failed to set timezone: %w", err)
|
||||||
|
}
|
||||||
|
s.logger.Info("Timezone set", "timezone", settings.Timezone)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Configure NTP servers in systemd-timesyncd
|
||||||
|
if len(settings.NTPServers) > 0 {
|
||||||
|
configPath := "/etc/systemd/timesyncd.conf"
|
||||||
|
|
||||||
|
// Build config content
|
||||||
|
configContent := "[Time]\n"
|
||||||
|
configContent += "NTP="
|
||||||
|
for i, server := range settings.NTPServers {
|
||||||
|
if i > 0 {
|
||||||
|
configContent += " "
|
||||||
|
}
|
||||||
|
configContent += server
|
||||||
|
}
|
||||||
|
configContent += "\n"
|
||||||
|
|
||||||
|
// Write to temporary file first, then move to final location (requires root)
|
||||||
|
tmpPath := "/tmp/timesyncd.conf." + fmt.Sprintf("%d", time.Now().Unix())
|
||||||
|
if err := os.WriteFile(tmpPath, []byte(configContent), 0644); err != nil {
|
||||||
|
s.logger.Error("Failed to write temporary NTP config", "error", err)
|
||||||
|
return fmt.Errorf("failed to write temporary NTP configuration: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Move to final location using sudo (requires root privileges)
|
||||||
|
cmd := exec.CommandContext(ctx, "sh", "-c", fmt.Sprintf("mv %s %s", tmpPath, configPath))
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to move NTP config", "error", err, "output", string(output))
|
||||||
|
os.Remove(tmpPath) // Clean up temp file
|
||||||
|
return fmt.Errorf("failed to move NTP configuration: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Restart systemd-timesyncd to apply changes
|
||||||
|
cmd = exec.CommandContext(ctx, "systemctl", "restart", "systemd-timesyncd")
|
||||||
|
output, err = cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to restart systemd-timesyncd", "error", err, "output", string(output))
|
||||||
|
return fmt.Errorf("failed to restart systemd-timesyncd: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("NTP servers configured", "servers", settings.NTPServers)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetNTPSettings retrieves current NTP configuration from the OS
|
||||||
|
func (s *Service) GetNTPSettings(ctx context.Context) (*NTPSettings, error) {
|
||||||
|
settings := &NTPSettings{
|
||||||
|
NTPServers: []string{},
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get current timezone using timedatectl
|
||||||
|
cmd := exec.CommandContext(ctx, "timedatectl", "show", "--property=Timezone", "--value")
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to get timezone", "error", err)
|
||||||
|
settings.Timezone = "Etc/UTC" // Default fallback
|
||||||
|
} else {
|
||||||
|
settings.Timezone = strings.TrimSpace(string(output))
|
||||||
|
if settings.Timezone == "" {
|
||||||
|
settings.Timezone = "Etc/UTC"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read NTP servers from systemd-timesyncd config
|
||||||
|
configPath := "/etc/systemd/timesyncd.conf"
|
||||||
|
data, err := os.ReadFile(configPath)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Warn("Failed to read NTP config", "error", err)
|
||||||
|
// Default NTP servers if config file doesn't exist
|
||||||
|
settings.NTPServers = []string{"pool.ntp.org", "time.google.com"}
|
||||||
|
} else {
|
||||||
|
// Parse NTP servers from config file
|
||||||
|
lines := strings.Split(string(data), "\n")
|
||||||
|
for _, line := range lines {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if strings.HasPrefix(line, "NTP=") {
|
||||||
|
ntpLine := strings.TrimPrefix(line, "NTP=")
|
||||||
|
if ntpLine != "" {
|
||||||
|
servers := strings.Fields(ntpLine)
|
||||||
|
settings.NTPServers = servers
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// If no NTP servers found in config, use defaults
|
||||||
|
if len(settings.NTPServers) == 0 {
|
||||||
|
settings.NTPServers = []string{"pool.ntp.org", "time.google.com"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return settings, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExecuteCommand executes a shell command and returns the output
|
||||||
|
// service parameter is optional and can be: system, scst, storage, backup, tape
|
||||||
|
func (s *Service) ExecuteCommand(ctx context.Context, command string, service string) (string, error) {
|
||||||
|
// Sanitize command - basic security check
|
||||||
|
command = strings.TrimSpace(command)
|
||||||
|
if command == "" {
|
||||||
|
return "", fmt.Errorf("command cannot be empty")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Block dangerous commands that could harm the system
|
||||||
|
dangerousCommands := []string{
|
||||||
|
"rm -rf /",
|
||||||
|
"dd if=",
|
||||||
|
":(){ :|:& };:",
|
||||||
|
"mkfs",
|
||||||
|
"fdisk",
|
||||||
|
"parted",
|
||||||
|
"format",
|
||||||
|
"> /dev/sd",
|
||||||
|
"mkfs.ext",
|
||||||
|
"mkfs.xfs",
|
||||||
|
"mkfs.btrfs",
|
||||||
|
"wipefs",
|
||||||
|
}
|
||||||
|
|
||||||
|
commandLower := strings.ToLower(command)
|
||||||
|
for _, dangerous := range dangerousCommands {
|
||||||
|
if strings.Contains(commandLower, dangerous) {
|
||||||
|
return "", fmt.Errorf("command blocked for security reasons")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Service-specific command handling
|
||||||
|
switch service {
|
||||||
|
case "scst":
|
||||||
|
// Allow SCST admin commands
|
||||||
|
if strings.HasPrefix(command, "scstadmin") {
|
||||||
|
// SCST commands are safe
|
||||||
|
break
|
||||||
|
}
|
||||||
|
case "backup":
|
||||||
|
// Allow bconsole commands
|
||||||
|
if strings.HasPrefix(command, "bconsole") {
|
||||||
|
// Backup console commands are safe
|
||||||
|
break
|
||||||
|
}
|
||||||
|
case "storage":
|
||||||
|
// Allow ZFS and storage commands
|
||||||
|
if strings.HasPrefix(command, "zfs") || strings.HasPrefix(command, "zpool") || strings.HasPrefix(command, "lsblk") {
|
||||||
|
// Storage commands are safe
|
||||||
|
break
|
||||||
|
}
|
||||||
|
case "tape":
|
||||||
|
// Allow tape library commands
|
||||||
|
if strings.HasPrefix(command, "mtx") || strings.HasPrefix(command, "lsscsi") || strings.HasPrefix(command, "sg_") {
|
||||||
|
// Tape commands are safe
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute command with timeout (30 seconds)
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
// Check if command already has sudo (reuse commandLower from above)
|
||||||
|
hasSudo := strings.HasPrefix(commandLower, "sudo ")
|
||||||
|
|
||||||
|
// Determine if command needs sudo based on service and command type
|
||||||
|
needsSudo := false
|
||||||
|
|
||||||
|
if !hasSudo {
|
||||||
|
// Commands that typically need sudo
|
||||||
|
sudoCommands := []string{
|
||||||
|
"scstadmin",
|
||||||
|
"systemctl",
|
||||||
|
"zfs",
|
||||||
|
"zpool",
|
||||||
|
"mount",
|
||||||
|
"umount",
|
||||||
|
"ip link",
|
||||||
|
"ip addr",
|
||||||
|
"iptables",
|
||||||
|
"journalctl",
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, sudoCmd := range sudoCommands {
|
||||||
|
if strings.HasPrefix(commandLower, sudoCmd) {
|
||||||
|
needsSudo = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Service-specific sudo requirements
|
||||||
|
switch service {
|
||||||
|
case "scst":
|
||||||
|
// All SCST admin commands need sudo
|
||||||
|
if strings.HasPrefix(commandLower, "scstadmin") {
|
||||||
|
needsSudo = true
|
||||||
|
}
|
||||||
|
case "storage":
|
||||||
|
// ZFS commands typically need sudo
|
||||||
|
if strings.HasPrefix(commandLower, "zfs") || strings.HasPrefix(commandLower, "zpool") {
|
||||||
|
needsSudo = true
|
||||||
|
}
|
||||||
|
case "system":
|
||||||
|
// System commands like systemctl need sudo
|
||||||
|
if strings.HasPrefix(commandLower, "systemctl") || strings.HasPrefix(commandLower, "journalctl") {
|
||||||
|
needsSudo = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build command with or without sudo
|
||||||
|
var cmd *exec.Cmd
|
||||||
|
if needsSudo && !hasSudo {
|
||||||
|
// Use sudo for privileged commands (if not already present)
|
||||||
|
cmd = exec.CommandContext(ctx, "sudo", "sh", "-c", command)
|
||||||
|
} else {
|
||||||
|
// Regular command (or already has sudo)
|
||||||
|
cmd = exec.CommandContext(ctx, "sh", "-c", command)
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd.Env = append(os.Environ(), "TERM=xterm-256color")
|
||||||
|
|
||||||
|
cmd.Env = append(os.Environ(), "TERM=xterm-256color")
|
||||||
|
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
// Return output even if there's an error (some commands return non-zero exit codes)
|
||||||
|
outputStr := string(output)
|
||||||
|
if len(outputStr) > 0 {
|
||||||
|
return outputStr, nil
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("command execution failed: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return string(output), nil
|
||||||
|
}
|
||||||
|
|||||||
328
backend/internal/system/terminal.go
Normal file
328
backend/internal/system/terminal.go
Normal file
@@ -0,0 +1,328 @@
|
|||||||
|
package system
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"os/user"
|
||||||
|
"sync"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/atlasos/calypso/internal/common/logger"
|
||||||
|
"github.com/creack/pty"
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/gorilla/websocket"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// WebSocket timeouts
|
||||||
|
writeWait = 10 * time.Second
|
||||||
|
pongWait = 60 * time.Second
|
||||||
|
pingPeriod = (pongWait * 9) / 10
|
||||||
|
)
|
||||||
|
|
||||||
|
var upgrader = websocket.Upgrader{
|
||||||
|
ReadBufferSize: 4096,
|
||||||
|
WriteBufferSize: 4096,
|
||||||
|
CheckOrigin: func(r *http.Request) bool {
|
||||||
|
// Allow all origins - in production, validate against allowed domains
|
||||||
|
return true
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// TerminalSession manages a single terminal session
|
||||||
|
type TerminalSession struct {
|
||||||
|
conn *websocket.Conn
|
||||||
|
pty *os.File
|
||||||
|
cmd *exec.Cmd
|
||||||
|
logger *logger.Logger
|
||||||
|
mu sync.RWMutex
|
||||||
|
closed bool
|
||||||
|
username string
|
||||||
|
done chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// HandleTerminalWebSocket handles WebSocket connection for terminal
|
||||||
|
func HandleTerminalWebSocket(c *gin.Context, log *logger.Logger) {
|
||||||
|
// Verify authentication
|
||||||
|
userID, exists := c.Get("user_id")
|
||||||
|
if !exists {
|
||||||
|
log.Warn("Terminal WebSocket: unauthorized access", "ip", c.ClientIP())
|
||||||
|
c.JSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
username, _ := c.Get("username")
|
||||||
|
if username == nil {
|
||||||
|
username = userID
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Info("Terminal WebSocket: connection attempt", "username", username, "ip", c.ClientIP())
|
||||||
|
|
||||||
|
// Upgrade connection
|
||||||
|
conn, err := upgrader.Upgrade(c.Writer, c.Request, nil)
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Terminal WebSocket: upgrade failed", "error", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Info("Terminal WebSocket: connection upgraded", "username", username)
|
||||||
|
|
||||||
|
// Create session
|
||||||
|
session := &TerminalSession{
|
||||||
|
conn: conn,
|
||||||
|
logger: log,
|
||||||
|
username: username.(string),
|
||||||
|
done: make(chan struct{}),
|
||||||
|
}
|
||||||
|
|
||||||
|
// Start terminal
|
||||||
|
if err := session.startPTY(); err != nil {
|
||||||
|
log.Error("Terminal WebSocket: failed to start PTY", "error", err, "username", username)
|
||||||
|
session.sendError(err.Error())
|
||||||
|
session.close()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle messages and PTY output
|
||||||
|
go session.handleRead()
|
||||||
|
go session.handleWrite()
|
||||||
|
}
|
||||||
|
|
||||||
|
// startPTY starts the PTY session
|
||||||
|
func (s *TerminalSession) startPTY() error {
|
||||||
|
// Get user info
|
||||||
|
currentUser, err := user.Lookup(s.username)
|
||||||
|
if err != nil {
|
||||||
|
// Fallback to current user
|
||||||
|
currentUser, err = user.Current()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Determine shell
|
||||||
|
shell := os.Getenv("SHELL")
|
||||||
|
if shell == "" {
|
||||||
|
shell = "/bin/bash"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create command
|
||||||
|
s.cmd = exec.Command(shell)
|
||||||
|
s.cmd.Env = append(os.Environ(),
|
||||||
|
"TERM=xterm-256color",
|
||||||
|
"HOME="+currentUser.HomeDir,
|
||||||
|
"USER="+currentUser.Username,
|
||||||
|
"USERNAME="+currentUser.Username,
|
||||||
|
)
|
||||||
|
s.cmd.Dir = currentUser.HomeDir
|
||||||
|
|
||||||
|
// Start PTY
|
||||||
|
ptyFile, err := pty.Start(s.cmd)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
s.pty = ptyFile
|
||||||
|
|
||||||
|
// Set initial size
|
||||||
|
pty.Setsize(ptyFile, &pty.Winsize{
|
||||||
|
Rows: 24,
|
||||||
|
Cols: 80,
|
||||||
|
})
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleRead handles incoming WebSocket messages
|
||||||
|
func (s *TerminalSession) handleRead() {
|
||||||
|
defer s.close()
|
||||||
|
|
||||||
|
// Set read deadline and pong handler
|
||||||
|
s.conn.SetReadDeadline(time.Now().Add(pongWait))
|
||||||
|
s.conn.SetPongHandler(func(string) error {
|
||||||
|
s.conn.SetReadDeadline(time.Now().Add(pongWait))
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-s.done:
|
||||||
|
return
|
||||||
|
default:
|
||||||
|
messageType, data, err := s.conn.ReadMessage()
|
||||||
|
if err != nil {
|
||||||
|
if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseAbnormalClosure) {
|
||||||
|
s.logger.Error("Terminal WebSocket: read error", "error", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle binary messages (raw input)
|
||||||
|
if messageType == websocket.BinaryMessage {
|
||||||
|
s.writeToPTY(data)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle text messages (JSON commands)
|
||||||
|
if messageType == websocket.TextMessage {
|
||||||
|
var msg map[string]interface{}
|
||||||
|
if err := json.Unmarshal(data, &msg); err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
switch msg["type"] {
|
||||||
|
case "input":
|
||||||
|
if data, ok := msg["data"].(string); ok {
|
||||||
|
s.writeToPTY([]byte(data))
|
||||||
|
}
|
||||||
|
|
||||||
|
case "resize":
|
||||||
|
if cols, ok1 := msg["cols"].(float64); ok1 {
|
||||||
|
if rows, ok2 := msg["rows"].(float64); ok2 {
|
||||||
|
s.resizePTY(uint16(cols), uint16(rows))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
case "ping":
|
||||||
|
s.writeWS(websocket.TextMessage, []byte(`{"type":"pong"}`))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleWrite handles PTY output to WebSocket
|
||||||
|
func (s *TerminalSession) handleWrite() {
|
||||||
|
defer s.close()
|
||||||
|
|
||||||
|
ticker := time.NewTicker(pingPeriod)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
// Read from PTY and write to WebSocket
|
||||||
|
buffer := make([]byte, 4096)
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-s.done:
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
// Send ping
|
||||||
|
if err := s.writeWS(websocket.PingMessage, nil); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
// Read from PTY
|
||||||
|
if s.pty != nil {
|
||||||
|
n, err := s.pty.Read(buffer)
|
||||||
|
if err != nil {
|
||||||
|
if err != io.EOF {
|
||||||
|
s.logger.Error("Terminal WebSocket: PTY read error", "error", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if n > 0 {
|
||||||
|
// Write binary data to WebSocket
|
||||||
|
if err := s.writeWS(websocket.BinaryMessage, buffer[:n]); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeToPTY writes data to PTY
|
||||||
|
func (s *TerminalSession) writeToPTY(data []byte) {
|
||||||
|
s.mu.RLock()
|
||||||
|
closed := s.closed
|
||||||
|
pty := s.pty
|
||||||
|
s.mu.RUnlock()
|
||||||
|
|
||||||
|
if closed || pty == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := pty.Write(data); err != nil {
|
||||||
|
s.logger.Error("Terminal WebSocket: PTY write error", "error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// resizePTY resizes the PTY
|
||||||
|
func (s *TerminalSession) resizePTY(cols, rows uint16) {
|
||||||
|
s.mu.RLock()
|
||||||
|
closed := s.closed
|
||||||
|
ptyFile := s.pty
|
||||||
|
s.mu.RUnlock()
|
||||||
|
|
||||||
|
if closed || ptyFile == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use pty.Setsize from package, not method from variable
|
||||||
|
pty.Setsize(ptyFile, &pty.Winsize{
|
||||||
|
Cols: cols,
|
||||||
|
Rows: rows,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeWS writes message to WebSocket
|
||||||
|
func (s *TerminalSession) writeWS(messageType int, data []byte) error {
|
||||||
|
s.mu.RLock()
|
||||||
|
closed := s.closed
|
||||||
|
conn := s.conn
|
||||||
|
s.mu.RUnlock()
|
||||||
|
|
||||||
|
if closed || conn == nil {
|
||||||
|
return io.ErrClosedPipe
|
||||||
|
}
|
||||||
|
|
||||||
|
conn.SetWriteDeadline(time.Now().Add(writeWait))
|
||||||
|
return conn.WriteMessage(messageType, data)
|
||||||
|
}
|
||||||
|
|
||||||
|
// sendError sends error message
|
||||||
|
func (s *TerminalSession) sendError(errMsg string) {
|
||||||
|
msg := map[string]interface{}{
|
||||||
|
"type": "error",
|
||||||
|
"error": errMsg,
|
||||||
|
}
|
||||||
|
data, _ := json.Marshal(msg)
|
||||||
|
s.writeWS(websocket.TextMessage, data)
|
||||||
|
}
|
||||||
|
|
||||||
|
// close closes the terminal session
|
||||||
|
func (s *TerminalSession) close() {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
|
||||||
|
if s.closed {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
s.closed = true
|
||||||
|
close(s.done)
|
||||||
|
|
||||||
|
// Close PTY
|
||||||
|
if s.pty != nil {
|
||||||
|
s.pty.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Kill process
|
||||||
|
if s.cmd != nil && s.cmd.Process != nil {
|
||||||
|
s.cmd.Process.Signal(syscall.SIGTERM)
|
||||||
|
time.Sleep(100 * time.Millisecond)
|
||||||
|
if s.cmd.ProcessState == nil || !s.cmd.ProcessState.Exited() {
|
||||||
|
s.cmd.Process.Kill()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close WebSocket
|
||||||
|
if s.conn != nil {
|
||||||
|
s.conn.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
s.logger.Info("Terminal WebSocket: session closed", "username", s.username)
|
||||||
|
}
|
||||||
1
bacula-config
Symbolic link
1
bacula-config
Symbolic link
@@ -0,0 +1 @@
|
|||||||
|
/opt/calypso/conf/bacula
|
||||||
BIN
dist/airgap/calypso-appliance-1.0.0-airgap.tar.gz
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap.tar.gz
vendored
Normal file
Binary file not shown.
1
dist/airgap/calypso-appliance-1.0.0-airgap.tar.gz.sha256
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap.tar.gz.sha256
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
a4040ea4ab486d4671772e4b832b4be5741bf58764fc731873b585df1d805e66 calypso-appliance-1.0.0-airgap.tar.gz
|
||||||
59
dist/airgap/calypso-appliance-1.0.0-airgap/README.md
vendored
Normal file
59
dist/airgap/calypso-appliance-1.0.0-airgap/README.md
vendored
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
# Calypso Appliance - Airgap Installation Bundle
|
||||||
|
|
||||||
|
**Version:** 1.0.0
|
||||||
|
**Target OS:** Ubuntu Server 24.04 LTS
|
||||||
|
**Installation Type:** Offline/Airgap
|
||||||
|
|
||||||
|
## Bundle Contents
|
||||||
|
|
||||||
|
- **binaries/**: Pre-built Calypso binaries
|
||||||
|
- **frontend/**: Built frontend assets
|
||||||
|
- **packages/debs/**: All required DEB packages with dependencies (59 packages + ~200-300 dependencies)
|
||||||
|
- **scripts/**: Installation scripts
|
||||||
|
- **services/**: Systemd service files
|
||||||
|
- **migrations/**: Database migration files
|
||||||
|
- **configs/**: Configuration templates
|
||||||
|
- **third_party/**: Optional third-party binaries (Go, Node.js)
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Ubuntu Server 24.04 LTS
|
||||||
|
- Root or sudo access
|
||||||
|
- Minimum 10GB free disk space
|
||||||
|
- Kernel headers installed (for kernel modules)
|
||||||
|
|
||||||
|
### Quick Install
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /path/to/bundle
|
||||||
|
sudo ./install-airgap.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
The installer will:
|
||||||
|
1. Install all packages from bundled DEB files
|
||||||
|
2. Install Calypso binaries and frontend
|
||||||
|
3. Setup database and configuration
|
||||||
|
4. Install systemd services
|
||||||
|
|
||||||
|
### Post-Installation
|
||||||
|
|
||||||
|
1. Review configuration: `/etc/calypso/calypso.yaml`
|
||||||
|
2. Install kernel modules (ZFS, SCST, mhVTL) if needed
|
||||||
|
3. Start services: `systemctl start calypso-api`
|
||||||
|
4. Enable services: `systemctl enable calypso-api`
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Check service status:
|
||||||
|
```bash
|
||||||
|
systemctl status calypso-api
|
||||||
|
curl http://localhost:8080/api/v1/health
|
||||||
|
```
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
For issues, check:
|
||||||
|
- Installation logs: `/var/log/calypso/install.log`
|
||||||
|
- Service logs: `journalctl -u calypso-api`
|
||||||
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/binaries/calypso-api
vendored
Executable file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/binaries/calypso-api
vendored
Executable file
Binary file not shown.
64
dist/airgap/calypso-appliance-1.0.0-airgap/configs/config.yaml.template
vendored
Normal file
64
dist/airgap/calypso-appliance-1.0.0-airgap/configs/config.yaml.template
vendored
Normal file
@@ -0,0 +1,64 @@
|
|||||||
|
# AtlasOS - Calypso API Configuration Template
|
||||||
|
# This file will be copied to /etc/calypso/config.yaml during installation
|
||||||
|
# Environment variables from /etc/calypso/secrets.env will be used for sensitive values
|
||||||
|
|
||||||
|
server:
|
||||||
|
port: 8080
|
||||||
|
host: "0.0.0.0"
|
||||||
|
read_timeout: 15s
|
||||||
|
write_timeout: 15s
|
||||||
|
idle_timeout: 60s
|
||||||
|
# Response caching configuration
|
||||||
|
cache:
|
||||||
|
enabled: true # Enable response caching
|
||||||
|
default_ttl: 5m # Default cache TTL (5 minutes)
|
||||||
|
max_age: 300 # Cache-Control max-age in seconds (5 minutes)
|
||||||
|
|
||||||
|
database:
|
||||||
|
host: "localhost"
|
||||||
|
port: 5432
|
||||||
|
user: "calypso"
|
||||||
|
password: "" # Set via CALYPSO_DB_PASSWORD environment variable
|
||||||
|
database: "calypso"
|
||||||
|
ssl_mode: "disable"
|
||||||
|
# Connection pool optimization
|
||||||
|
max_connections: 25
|
||||||
|
max_idle_conns: 5
|
||||||
|
conn_max_lifetime: 5m
|
||||||
|
|
||||||
|
auth:
|
||||||
|
jwt_secret: "" # Set via CALYPSO_JWT_SECRET environment variable
|
||||||
|
token_lifetime: 24h
|
||||||
|
argon2:
|
||||||
|
memory: 65536 # 64 MB
|
||||||
|
iterations: 3
|
||||||
|
parallelism: 4
|
||||||
|
salt_length: 16
|
||||||
|
key_length: 32
|
||||||
|
|
||||||
|
logging:
|
||||||
|
level: "info" # debug, info, warn, error
|
||||||
|
format: "json" # json or text
|
||||||
|
|
||||||
|
# CORS configuration
|
||||||
|
cors:
|
||||||
|
allowed_origins:
|
||||||
|
- "http://localhost:3000"
|
||||||
|
- "http://localhost:5173"
|
||||||
|
allowed_methods:
|
||||||
|
- "GET"
|
||||||
|
- "POST"
|
||||||
|
- "PUT"
|
||||||
|
- "DELETE"
|
||||||
|
- "PATCH"
|
||||||
|
allowed_headers:
|
||||||
|
- "Content-Type"
|
||||||
|
- "Authorization"
|
||||||
|
allow_credentials: true
|
||||||
|
|
||||||
|
# Rate limiting
|
||||||
|
rate_limit:
|
||||||
|
enabled: true
|
||||||
|
requests_per_minute: 100
|
||||||
|
authenticated_requests_per_minute: 200
|
||||||
|
|
||||||
66
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/chart-vendor-CnBPFalK.js
vendored
Normal file
66
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/chart-vendor-CnBPFalK.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/chart-vendor-CnBPFalK.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/chart-vendor-CnBPFalK.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index-BY0QR2-n.css
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index-BY0QR2-n.css
vendored
Normal file
File diff suppressed because one or more lines are too long
207
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index-ChZMKDzx.js
vendored
Normal file
207
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index-ChZMKDzx.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index-ChZMKDzx.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index-ChZMKDzx.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
6
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index.es-BqkUSV0N.js
vendored
Normal file
6
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index.es-BqkUSV0N.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index.es-BqkUSV0N.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/index.es-BqkUSV0N.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
3
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/purify.es-Bzr520pe.js
vendored
Normal file
3
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/purify.es-Bzr520pe.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/purify.es-Bzr520pe.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/purify.es-Bzr520pe.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/query-vendor-BeRQtQod.js
vendored
Normal file
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/query-vendor-BeRQtQod.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/query-vendor-BeRQtQod.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/query-vendor-BeRQtQod.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/react-vendor-Dvs2KPqW.js
vendored
Normal file
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/react-vendor-Dvs2KPqW.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/react-vendor-Dvs2KPqW.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/react-vendor-Dvs2KPqW.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/terminal-vendor-l0sNRNKZ.js
vendored
Normal file
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/terminal-vendor-l0sNRNKZ.js
vendored
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
|
||||||
|
//# sourceMappingURL=terminal-vendor-l0sNRNKZ.js.map
|
||||||
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/terminal-vendor-l0sNRNKZ.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/terminal-vendor-l0sNRNKZ.js.map
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
{"version":3,"file":"terminal-vendor-l0sNRNKZ.js","sources":[],"sourcesContent":[],"names":[],"mappings":""}
|
||||||
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/ui-vendor-C4xvlrdo.js
vendored
Normal file
2
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/ui-vendor-C4xvlrdo.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/ui-vendor-C4xvlrdo.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/ui-vendor-C4xvlrdo.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
7
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/utils-vendor-CJpcXyE0.js
vendored
Normal file
7
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/utils-vendor-CJpcXyE0.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/utils-vendor-CJpcXyE0.js.map
vendored
Normal file
1
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/assets/utils-vendor-CJpcXyE0.js.map
vendored
Normal file
File diff suppressed because one or more lines are too long
29
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/index.html
vendored
Normal file
29
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/index.html
vendored
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html class="dark" lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<link rel="icon" type="image/png" href="/logo.png" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<title>AtlasOS - Calypso</title>
|
||||||
|
<!-- Material Symbols -->
|
||||||
|
<link
|
||||||
|
href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:wght,FILL@100..700,0..1&display=swap"
|
||||||
|
rel="stylesheet"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
href="https://fonts.googleapis.com/css2?family=Material+Symbols+Rounded:opsz,wght,FILL,GRAD@20..48,100..700,0..1,-50..200&display=swap"
|
||||||
|
rel="stylesheet"
|
||||||
|
/>
|
||||||
|
<script type="module" crossorigin src="/assets/index-ChZMKDzx.js"></script>
|
||||||
|
<link rel="modulepreload" crossorigin href="/assets/react-vendor-Dvs2KPqW.js">
|
||||||
|
<link rel="modulepreload" crossorigin href="/assets/query-vendor-BeRQtQod.js">
|
||||||
|
<link rel="modulepreload" crossorigin href="/assets/utils-vendor-CJpcXyE0.js">
|
||||||
|
<link rel="modulepreload" crossorigin href="/assets/ui-vendor-C4xvlrdo.js">
|
||||||
|
<link rel="modulepreload" crossorigin href="/assets/chart-vendor-CnBPFalK.js">
|
||||||
|
<link rel="stylesheet" crossorigin href="/assets/index-BY0QR2-n.css">
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="root"></div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
|
||||||
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/logo.png
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/frontend/logo.png
vendored
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
215
dist/airgap/calypso-appliance-1.0.0-airgap/install-airgap.sh
vendored
Executable file
215
dist/airgap/calypso-appliance-1.0.0-airgap/install-airgap.sh
vendored
Executable file
@@ -0,0 +1,215 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
#
|
||||||
|
# Calypso Appliance - Airgap Installer
|
||||||
|
# Installs Calypso from bundled files without internet connection
|
||||||
|
#
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
BUNDLE_DIR="$SCRIPT_DIR"
|
||||||
|
VERSION="${VERSION:-1.0.0}"
|
||||||
|
|
||||||
|
# Colors
|
||||||
|
RED='\033[0;31m'
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
YELLOW='\033[1;33m'
|
||||||
|
BLUE='\033[0;34m'
|
||||||
|
NC='\033[0m'
|
||||||
|
|
||||||
|
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
|
||||||
|
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
|
||||||
|
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
|
||||||
|
log_step() { echo -e "\n${BLUE}==>${NC} $1"; }
|
||||||
|
|
||||||
|
# Check root
|
||||||
|
if [[ $EUID -ne 0 ]]; then
|
||||||
|
log_error "This script must be run as root (use sudo)"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check OS
|
||||||
|
if [ ! -f /etc/os-release ]; then
|
||||||
|
log_error "Cannot detect OS"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
. /etc/os-release
|
||||||
|
if [[ "$ID" != "ubuntu" ]] || [[ "$VERSION_ID" != "24.04" ]]; then
|
||||||
|
log_warn "This installer is designed for Ubuntu 24.04 LTS"
|
||||||
|
log_warn "Detected: $ID $VERSION_ID"
|
||||||
|
read -p "Continue anyway? (y/N) " -n 1 -r
|
||||||
|
echo
|
||||||
|
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_step "Calypso Appliance - Airgap Installation"
|
||||||
|
log_info "Bundle directory: $BUNDLE_DIR"
|
||||||
|
|
||||||
|
# 1. Install packages from bundle
|
||||||
|
log_step "Installing system packages from bundle..."
|
||||||
|
|
||||||
|
if [ -d "$BUNDLE_DIR/packages/debs" ] && [ "$(ls -A $BUNDLE_DIR/packages/debs/*.deb 2>/dev/null)" ]; then
|
||||||
|
log_info "Installing packages from bundle..."
|
||||||
|
|
||||||
|
# Create local apt repository
|
||||||
|
DEBS_DIR="$BUNDLE_DIR/packages/debs"
|
||||||
|
REPO_DIR="/tmp/calypso-local-repo"
|
||||||
|
|
||||||
|
mkdir -p "$REPO_DIR"
|
||||||
|
cp "$DEBS_DIR"/*.deb "$REPO_DIR/" 2>/dev/null || true
|
||||||
|
|
||||||
|
# Create Packages.gz if not exists
|
||||||
|
cd "$REPO_DIR"
|
||||||
|
if [ ! -f Packages.gz ]; then
|
||||||
|
dpkg-scanpackages . /dev/null 2>/dev/null | gzip -9c > Packages.gz
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Add to sources.list temporarily
|
||||||
|
echo "deb [trusted=yes] file://$REPO_DIR ./" > /tmp/calypso-local.list
|
||||||
|
mv /tmp/calypso-local.list /etc/apt/sources.list.d/calypso-local.list
|
||||||
|
|
||||||
|
# Update apt cache
|
||||||
|
apt-get update -qq
|
||||||
|
|
||||||
|
# Install packages from list
|
||||||
|
PACKAGE_LIST="$BUNDLE_DIR/packages/package-list.txt"
|
||||||
|
PACKAGES=$(grep -v '^#' "$PACKAGE_LIST" | grep -v '^$' | tr '\n' ' ')
|
||||||
|
|
||||||
|
log_info "Installing packages..."
|
||||||
|
apt-get install -y --allow-unauthenticated $PACKAGES || {
|
||||||
|
log_warn "Some packages failed to install, trying individual installation..."
|
||||||
|
while IFS= read -r package || [ -n "$package" ]; do
|
||||||
|
[[ "$package" =~ ^#.*$ ]] && continue
|
||||||
|
[[ -z "$package" ]] && continue
|
||||||
|
apt-get install -y --allow-unauthenticated "$package" 2>/dev/null || log_warn "Failed: $package"
|
||||||
|
done < "$PACKAGE_LIST"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Cleanup
|
||||||
|
rm -f /etc/apt/sources.list.d/calypso-local.list
|
||||||
|
apt-get update -qq
|
||||||
|
|
||||||
|
log_info "✓ Packages installed from bundle"
|
||||||
|
else
|
||||||
|
log_warn "Package DEB files not found in bundle"
|
||||||
|
log_warn "You need to install packages manually"
|
||||||
|
log_info "Package list: $BUNDLE_DIR/packages/package-list.txt"
|
||||||
|
read -p "Install packages from system repositories? (y/N) " -n 1 -r
|
||||||
|
echo
|
||||||
|
if [[ $REPLY =~ ^[Yy]$ ]]; then
|
||||||
|
apt-get update
|
||||||
|
xargs -a "$BUNDLE_DIR/packages/package-list.txt" -r apt-get install -y
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 2. Install Go from bundle (if available)
|
||||||
|
if [ -f "$BUNDLE_DIR/third_party/go.tar.gz" ]; then
|
||||||
|
log_step "Installing Go from bundle..."
|
||||||
|
rm -rf /usr/local/go
|
||||||
|
tar -C /usr/local -xzf "$BUNDLE_DIR/third_party/go.tar.gz"
|
||||||
|
export PATH=$PATH:/usr/local/go/bin
|
||||||
|
if ! grep -q "/usr/local/go/bin" /etc/profile; then
|
||||||
|
echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile
|
||||||
|
fi
|
||||||
|
log_info "✓ Go installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Install Node.js from bundle (if available)
|
||||||
|
if [ -f "$BUNDLE_DIR/third_party/nodejs.tar.xz" ]; then
|
||||||
|
log_step "Installing Node.js from bundle..."
|
||||||
|
rm -rf /usr/local/node
|
||||||
|
mkdir -p /usr/local/node
|
||||||
|
tar -C /usr/local/node -xJf "$BUNDLE_DIR/third_party/nodejs.tar.xz" --strip-components=1
|
||||||
|
export PATH=$PATH:/usr/local/node/bin
|
||||||
|
if ! grep -q "/usr/local/node/bin" /etc/profile; then
|
||||||
|
echo 'export PATH=$PATH:/usr/local/node/bin' >> /etc/profile
|
||||||
|
fi
|
||||||
|
log_info "✓ Node.js installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 4. Create filesystem structure
|
||||||
|
log_step "Creating filesystem structure..."
|
||||||
|
INSTALL_PREFIX="/opt/adastra/calypso"
|
||||||
|
CONFIG_DIR="/etc/calypso"
|
||||||
|
DATA_DIR="/srv/calypso"
|
||||||
|
LOG_DIR="/var/log/calypso"
|
||||||
|
LIB_DIR="/var/lib/calypso"
|
||||||
|
|
||||||
|
mkdir -p "$INSTALL_PREFIX/releases/${VERSION}"/{bin,web,migrations,scripts}
|
||||||
|
mkdir -p "$INSTALL_PREFIX/third_party"
|
||||||
|
mkdir -p "$CONFIG_DIR"/{tls,integrations,system,scst,nfs,samba,clamav}
|
||||||
|
mkdir -p "$DATA_DIR"/{db,backups,object,shares,vtl,iscsi,uploads,cache,_system,quarantine}
|
||||||
|
mkdir -p "$LOG_DIR" "$LIB_DIR" "/run/calypso"
|
||||||
|
|
||||||
|
# Create calypso user
|
||||||
|
if ! id "calypso" &>/dev/null; then
|
||||||
|
useradd -r -s /bin/false -d "$LIB_DIR" -c "Calypso Appliance" calypso
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 5. Install binaries
|
||||||
|
log_step "Installing Calypso binaries..."
|
||||||
|
cp "$BUNDLE_DIR/binaries/calypso-api" "$INSTALL_PREFIX/releases/${VERSION}/bin/"
|
||||||
|
chmod +x "$INSTALL_PREFIX/releases/${VERSION}/bin/calypso-api"
|
||||||
|
|
||||||
|
# Install frontend
|
||||||
|
cp -r "$BUNDLE_DIR/frontend"/* "$INSTALL_PREFIX/releases/${VERSION}/web/"
|
||||||
|
|
||||||
|
# Install migrations
|
||||||
|
cp -r "$BUNDLE_DIR/migrations"/* "$INSTALL_PREFIX/releases/${VERSION}/migrations/" 2>/dev/null || true
|
||||||
|
|
||||||
|
# Install scripts
|
||||||
|
cp -r "$BUNDLE_DIR/scripts"/* "$INSTALL_PREFIX/releases/${VERSION}/scripts/" 2>/dev/null || true
|
||||||
|
chmod +x "$INSTALL_PREFIX/releases/${VERSION}/scripts"/*.sh 2>/dev/null || true
|
||||||
|
|
||||||
|
# Create symlink
|
||||||
|
ln -sfn "releases/${VERSION}" "$INSTALL_PREFIX/current"
|
||||||
|
|
||||||
|
# Set ownership
|
||||||
|
chown -R calypso:calypso "$INSTALL_PREFIX" "$LIB_DIR" "$LOG_DIR" "$DATA_DIR" 2>/dev/null || chown -R root:root "$INSTALL_PREFIX" "$LIB_DIR" "$LOG_DIR" "$DATA_DIR"
|
||||||
|
|
||||||
|
log_info "✓ Binaries installed"
|
||||||
|
|
||||||
|
# 6. Install systemd services
|
||||||
|
log_step "Installing systemd services..."
|
||||||
|
if [ -d "$BUNDLE_DIR/services" ]; then
|
||||||
|
cp "$BUNDLE_DIR/services"/*.service /etc/systemd/system/ 2>/dev/null || true
|
||||||
|
systemctl daemon-reload
|
||||||
|
log_info "✓ Systemd services installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 7. Setup database
|
||||||
|
log_step "Setting up database..."
|
||||||
|
if systemctl is-active --quiet postgresql 2>/dev/null || systemctl is-enabled --quiet postgresql 2>/dev/null; then
|
||||||
|
sudo -u postgres createdb calypso 2>/dev/null || true
|
||||||
|
sudo -u postgres createuser calypso 2>/dev/null || true
|
||||||
|
sudo -u postgres psql -c "ALTER USER calypso WITH PASSWORD 'calypso_change_me';" 2>/dev/null || true
|
||||||
|
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE calypso TO calypso;" 2>/dev/null || true
|
||||||
|
log_info "✓ Database setup complete"
|
||||||
|
else
|
||||||
|
log_warn "PostgreSQL not running. Please start it and run database setup manually."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 8. Generate configuration
|
||||||
|
log_step "Generating configuration..."
|
||||||
|
if [ -f "$BUNDLE_DIR/configs/config.yaml.template" ]; then
|
||||||
|
cp "$BUNDLE_DIR/configs/config.yaml.template" "$CONFIG_DIR/calypso.yaml"
|
||||||
|
# Generate secrets
|
||||||
|
if command -v openssl &> /dev/null; then
|
||||||
|
JWT_SECRET=$(openssl rand -hex 32)
|
||||||
|
sed -i "s/CHANGE_ME_JWT_SECRET/$JWT_SECRET/g" "$CONFIG_DIR/calypso.yaml"
|
||||||
|
fi
|
||||||
|
log_info "✓ Configuration generated"
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_step "Installation Complete!"
|
||||||
|
log_info ""
|
||||||
|
log_info "Next steps:"
|
||||||
|
log_info "1. Review configuration: $CONFIG_DIR/calypso.yaml"
|
||||||
|
log_info "2. Install kernel modules (ZFS, SCST, mhVTL) if needed"
|
||||||
|
log_info "3. Start services: systemctl start calypso-api"
|
||||||
|
log_info "4. Enable services: systemctl enable calypso-api"
|
||||||
|
log_info ""
|
||||||
|
log_info "Calypso API will be available at: http://localhost:8080"
|
||||||
229
dist/airgap/calypso-appliance-1.0.0-airgap/install.sh
vendored
Executable file
229
dist/airgap/calypso-appliance-1.0.0-airgap/install.sh
vendored
Executable file
@@ -0,0 +1,229 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
#
|
||||||
|
# AtlasOS - Calypso Appliance Installer
|
||||||
|
# Complete installation script for Calypso backup appliance
|
||||||
|
# Target OS: Ubuntu Server 24.04 LTS
|
||||||
|
#
|
||||||
|
# Usage: sudo ./installer/alpha/install.sh [options]
|
||||||
|
#
|
||||||
|
# Options:
|
||||||
|
# --version VERSION Install specific version (default: auto-detect)
|
||||||
|
# --skip-deps Skip system dependencies installation
|
||||||
|
# --skip-zfs Skip ZFS installation
|
||||||
|
# --skip-scst Skip SCST installation
|
||||||
|
# --skip-mhvtl Skip MHVTL installation
|
||||||
|
# --skip-bacula Skip Bacula installation
|
||||||
|
# --config-only Only setup configuration, don't install binaries
|
||||||
|
#
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Script directory
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
INSTALLER_DIR="$SCRIPT_DIR"
|
||||||
|
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||||
|
|
||||||
|
# Colors for output
|
||||||
|
RED='\033[0;31m'
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
YELLOW='\033[1;33m'
|
||||||
|
BLUE='\033[0;34m'
|
||||||
|
NC='\033[0m' # No Color
|
||||||
|
|
||||||
|
# Logging functions
|
||||||
|
log_info() {
|
||||||
|
echo -e "${GREEN}[INFO]${NC} $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
log_warn() {
|
||||||
|
echo -e "${YELLOW}[WARN]${NC} $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
log_error() {
|
||||||
|
echo -e "${RED}[ERROR]${NC} $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
log_step() {
|
||||||
|
echo -e "\n${BLUE}==>${NC} $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Configuration
|
||||||
|
CALYPSO_VERSION="${CALYPSO_VERSION:-1.0.0-alpha}"
|
||||||
|
INSTALL_PREFIX="/opt/adastra/calypso"
|
||||||
|
CONFIG_DIR="/etc/calypso"
|
||||||
|
DATA_DIR="/srv/calypso"
|
||||||
|
LOG_DIR="/var/log/calypso"
|
||||||
|
LIB_DIR="/var/lib/calypso"
|
||||||
|
RUN_DIR="/run/calypso"
|
||||||
|
|
||||||
|
# Flags
|
||||||
|
SKIP_DEPS=false
|
||||||
|
SKIP_ZFS=false
|
||||||
|
SKIP_SCST=false
|
||||||
|
SKIP_MHVTL=false
|
||||||
|
SKIP_BACULA=false
|
||||||
|
CONFIG_ONLY=false
|
||||||
|
|
||||||
|
# Parse arguments
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case $1 in
|
||||||
|
--version)
|
||||||
|
CALYPSO_VERSION="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--skip-deps)
|
||||||
|
SKIP_DEPS=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--skip-zfs)
|
||||||
|
SKIP_ZFS=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--skip-scst)
|
||||||
|
SKIP_SCST=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--skip-mhvtl)
|
||||||
|
SKIP_MHVTL=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--skip-bacula)
|
||||||
|
SKIP_BACULA=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--config-only)
|
||||||
|
CONFIG_ONLY=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
log_error "Unknown option: $1"
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Check if running as root
|
||||||
|
if [[ $EUID -ne 0 ]]; then
|
||||||
|
log_error "This script must be run as root (use sudo)"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Detect OS
|
||||||
|
if ! grep -q "Ubuntu" /etc/os-release 2>/dev/null; then
|
||||||
|
log_warn "This installer is designed for Ubuntu. Proceeding anyway..."
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_info "=========================================="
|
||||||
|
log_info "AtlasOS - Calypso Appliance Installer"
|
||||||
|
log_info "Version: ${CALYPSO_VERSION}"
|
||||||
|
log_info "=========================================="
|
||||||
|
log_info ""
|
||||||
|
|
||||||
|
# Source helper scripts
|
||||||
|
source "$INSTALLER_DIR/scripts/helpers.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/filesystem.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/dependencies.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/components.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/application.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/database.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/services.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/configuration.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/configure-services.sh"
|
||||||
|
source "$INSTALLER_DIR/scripts/post-install.sh"
|
||||||
|
|
||||||
|
# Main installation function
|
||||||
|
main() {
|
||||||
|
log_step "Starting Calypso Appliance Installation"
|
||||||
|
|
||||||
|
# Pre-flight checks
|
||||||
|
log_step "Pre-flight Checks"
|
||||||
|
check_prerequisites
|
||||||
|
|
||||||
|
# Create filesystem structure
|
||||||
|
log_step "Creating Filesystem Structure"
|
||||||
|
create_filesystem_structure
|
||||||
|
|
||||||
|
# Install system dependencies
|
||||||
|
if [[ "$SKIP_DEPS" == "false" ]]; then
|
||||||
|
log_step "Installing System Dependencies"
|
||||||
|
install_system_dependencies
|
||||||
|
else
|
||||||
|
log_info "Skipping system dependencies installation"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Install components
|
||||||
|
if [[ "$SKIP_ZFS" == "false" ]]; then
|
||||||
|
log_step "Installing ZFS"
|
||||||
|
install_zfs || log_warn "ZFS installation failed or already installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$SKIP_SCST" == "false" ]]; then
|
||||||
|
log_step "Installing SCST"
|
||||||
|
install_scst || log_warn "SCST installation failed or already installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$SKIP_MHVTL" == "false" ]]; then
|
||||||
|
log_step "Installing MHVTL"
|
||||||
|
install_mhvtl || log_warn "MHVTL installation failed or already installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$SKIP_BACULA" == "false" ]]; then
|
||||||
|
log_step "Installing Bacula (Optional)"
|
||||||
|
install_bacula || log_warn "Bacula installation skipped or failed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# File sharing services are installed in dependencies step
|
||||||
|
# ClamAV is installed in dependencies step
|
||||||
|
|
||||||
|
# Build and install application
|
||||||
|
if [[ "$CONFIG_ONLY" == "false" ]]; then
|
||||||
|
log_step "Building and Installing Application"
|
||||||
|
build_and_install_application
|
||||||
|
else
|
||||||
|
log_info "Skipping application build (config-only mode)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Setup database
|
||||||
|
log_step "Setting Up Database"
|
||||||
|
setup_database
|
||||||
|
|
||||||
|
# Setup configuration
|
||||||
|
log_step "Setting Up Configuration"
|
||||||
|
setup_configuration
|
||||||
|
|
||||||
|
# Install systemd services
|
||||||
|
log_step "Installing Systemd Services"
|
||||||
|
install_systemd_services
|
||||||
|
|
||||||
|
# Configure file sharing and antivirus services
|
||||||
|
log_step "Configuring File Sharing and Antivirus Services"
|
||||||
|
configure_all_services
|
||||||
|
|
||||||
|
# Final verification
|
||||||
|
log_step "Verifying Installation"
|
||||||
|
verify_installation
|
||||||
|
|
||||||
|
# Post-installation setup
|
||||||
|
log_step "Post-Installation Setup"
|
||||||
|
post_install_setup
|
||||||
|
|
||||||
|
# Print summary
|
||||||
|
print_installation_summary
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run main installation
|
||||||
|
main
|
||||||
|
|
||||||
|
log_info ""
|
||||||
|
log_info "=========================================="
|
||||||
|
log_info "Installation Complete!"
|
||||||
|
log_info "=========================================="
|
||||||
|
log_info ""
|
||||||
|
log_info "Next steps:"
|
||||||
|
log_info "1. Configure /etc/calypso/config.yaml"
|
||||||
|
log_info "2. Set environment variables in /etc/calypso/secrets.env"
|
||||||
|
log_info "3. Start services: sudo systemctl start calypso-api"
|
||||||
|
log_info "4. Access web UI: http://<server-ip>:3000"
|
||||||
|
log_info "5. Login with default admin credentials (check above)"
|
||||||
|
log_info ""
|
||||||
|
|
||||||
213
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/001_initial_schema.sql
vendored
Normal file
213
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/001_initial_schema.sql
vendored
Normal file
@@ -0,0 +1,213 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Initial Database Schema
|
||||||
|
-- Version: 1.0
|
||||||
|
|
||||||
|
-- Users table
|
||||||
|
CREATE TABLE IF NOT EXISTS users (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
username VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
email VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
password_hash VARCHAR(255) NOT NULL,
|
||||||
|
full_name VARCHAR(255),
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
is_system BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
last_login_at TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Roles table
|
||||||
|
CREATE TABLE IF NOT EXISTS roles (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(100) NOT NULL UNIQUE,
|
||||||
|
description TEXT,
|
||||||
|
is_system BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Permissions table
|
||||||
|
CREATE TABLE IF NOT EXISTS permissions (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
resource VARCHAR(100) NOT NULL,
|
||||||
|
action VARCHAR(100) NOT NULL,
|
||||||
|
description TEXT,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- User roles junction table
|
||||||
|
CREATE TABLE IF NOT EXISTS user_roles (
|
||||||
|
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
role_id UUID NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
|
||||||
|
assigned_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
assigned_by UUID REFERENCES users(id),
|
||||||
|
PRIMARY KEY (user_id, role_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Role permissions junction table
|
||||||
|
CREATE TABLE IF NOT EXISTS role_permissions (
|
||||||
|
role_id UUID NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
|
||||||
|
permission_id UUID NOT NULL REFERENCES permissions(id) ON DELETE CASCADE,
|
||||||
|
granted_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (role_id, permission_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Sessions table
|
||||||
|
CREATE TABLE IF NOT EXISTS sessions (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
token_hash VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
ip_address INET,
|
||||||
|
user_agent TEXT,
|
||||||
|
expires_at TIMESTAMP NOT NULL,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
last_activity_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Audit log table
|
||||||
|
CREATE TABLE IF NOT EXISTS audit_log (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
user_id UUID REFERENCES users(id),
|
||||||
|
username VARCHAR(255),
|
||||||
|
action VARCHAR(100) NOT NULL,
|
||||||
|
resource_type VARCHAR(100) NOT NULL,
|
||||||
|
resource_id VARCHAR(255),
|
||||||
|
method VARCHAR(10),
|
||||||
|
path TEXT,
|
||||||
|
ip_address INET,
|
||||||
|
user_agent TEXT,
|
||||||
|
request_body JSONB,
|
||||||
|
response_status INTEGER,
|
||||||
|
error_message TEXT,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Tasks table (for async operations)
|
||||||
|
CREATE TABLE IF NOT EXISTS tasks (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
type VARCHAR(100) NOT NULL,
|
||||||
|
status VARCHAR(50) NOT NULL DEFAULT 'pending',
|
||||||
|
progress INTEGER NOT NULL DEFAULT 0,
|
||||||
|
message TEXT,
|
||||||
|
error_message TEXT,
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
started_at TIMESTAMP,
|
||||||
|
completed_at TIMESTAMP,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
metadata JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Alerts table
|
||||||
|
CREATE TABLE IF NOT EXISTS alerts (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
severity VARCHAR(20) NOT NULL,
|
||||||
|
source VARCHAR(100) NOT NULL,
|
||||||
|
title VARCHAR(255) NOT NULL,
|
||||||
|
message TEXT NOT NULL,
|
||||||
|
resource_type VARCHAR(100),
|
||||||
|
resource_id VARCHAR(255),
|
||||||
|
is_acknowledged BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
acknowledged_by UUID REFERENCES users(id),
|
||||||
|
acknowledged_at TIMESTAMP,
|
||||||
|
resolved_at TIMESTAMP,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
metadata JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- System configuration table
|
||||||
|
CREATE TABLE IF NOT EXISTS system_config (
|
||||||
|
key VARCHAR(255) PRIMARY KEY,
|
||||||
|
value TEXT NOT NULL,
|
||||||
|
description TEXT,
|
||||||
|
is_encrypted BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
updated_by UUID REFERENCES users(id),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Indexes for performance
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_users_active ON users(is_active);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions(user_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sessions_token_hash ON sessions(token_hash);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sessions_expires_at ON sessions(expires_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_log_user_id ON audit_log(user_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_log_created_at ON audit_log(created_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_log_resource ON audit_log(resource_type, resource_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_status ON tasks(status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_type ON tasks(type);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_created_by ON tasks(created_by);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_severity ON alerts(severity);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_acknowledged ON alerts(is_acknowledged);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_created_at ON alerts(created_at);
|
||||||
|
|
||||||
|
-- Insert default system roles
|
||||||
|
INSERT INTO roles (name, description, is_system) VALUES
|
||||||
|
('admin', 'Full system access and configuration', true),
|
||||||
|
('operator', 'Day-to-day operations and monitoring', true),
|
||||||
|
('readonly', 'Read-only access for monitoring and reporting', true)
|
||||||
|
ON CONFLICT (name) DO NOTHING;
|
||||||
|
|
||||||
|
-- Insert default permissions
|
||||||
|
INSERT INTO permissions (name, resource, action, description) VALUES
|
||||||
|
-- System permissions
|
||||||
|
('system:read', 'system', 'read', 'View system information'),
|
||||||
|
('system:write', 'system', 'write', 'Modify system configuration'),
|
||||||
|
('system:manage', 'system', 'manage', 'Full system management'),
|
||||||
|
|
||||||
|
-- Storage permissions
|
||||||
|
('storage:read', 'storage', 'read', 'View storage information'),
|
||||||
|
('storage:write', 'storage', 'write', 'Modify storage configuration'),
|
||||||
|
('storage:manage', 'storage', 'manage', 'Full storage management'),
|
||||||
|
|
||||||
|
-- Tape permissions
|
||||||
|
('tape:read', 'tape', 'read', 'View tape library information'),
|
||||||
|
('tape:write', 'tape', 'write', 'Perform tape operations'),
|
||||||
|
('tape:manage', 'tape', 'manage', 'Full tape management'),
|
||||||
|
|
||||||
|
-- iSCSI permissions
|
||||||
|
('iscsi:read', 'iscsi', 'read', 'View iSCSI configuration'),
|
||||||
|
('iscsi:write', 'iscsi', 'write', 'Modify iSCSI configuration'),
|
||||||
|
('iscsi:manage', 'iscsi', 'manage', 'Full iSCSI management'),
|
||||||
|
|
||||||
|
-- IAM permissions
|
||||||
|
('iam:read', 'iam', 'read', 'View users and roles'),
|
||||||
|
('iam:write', 'iam', 'write', 'Modify users and roles'),
|
||||||
|
('iam:manage', 'iam', 'manage', 'Full IAM management'),
|
||||||
|
|
||||||
|
-- Audit permissions
|
||||||
|
('audit:read', 'audit', 'read', 'View audit logs'),
|
||||||
|
|
||||||
|
-- Monitoring permissions
|
||||||
|
('monitoring:read', 'monitoring', 'read', 'View monitoring data'),
|
||||||
|
('monitoring:write', 'monitoring', 'write', 'Acknowledge alerts')
|
||||||
|
ON CONFLICT (name) DO NOTHING;
|
||||||
|
|
||||||
|
-- Assign permissions to roles
|
||||||
|
-- Admin gets all permissions
|
||||||
|
INSERT INTO role_permissions (role_id, permission_id)
|
||||||
|
SELECT r.id, p.id
|
||||||
|
FROM roles r, permissions p
|
||||||
|
WHERE r.name = 'admin'
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
|
-- Operator gets read and write (but not manage) for most resources
|
||||||
|
INSERT INTO role_permissions (role_id, permission_id)
|
||||||
|
SELECT r.id, p.id
|
||||||
|
FROM roles r, permissions p
|
||||||
|
WHERE r.name = 'operator'
|
||||||
|
AND p.action IN ('read', 'write')
|
||||||
|
AND p.resource IN ('storage', 'tape', 'iscsi', 'monitoring')
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
|
-- ReadOnly gets only read permissions
|
||||||
|
INSERT INTO role_permissions (role_id, permission_id)
|
||||||
|
SELECT r.id, p.id
|
||||||
|
FROM roles r, permissions p
|
||||||
|
WHERE r.name = 'readonly'
|
||||||
|
AND p.action = 'read'
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
227
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/002_storage_and_tape_schema.sql
vendored
Normal file
227
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/002_storage_and_tape_schema.sql
vendored
Normal file
@@ -0,0 +1,227 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Storage and Tape Component Schema
|
||||||
|
-- Version: 2.0
|
||||||
|
|
||||||
|
-- ZFS pools table
|
||||||
|
CREATE TABLE IF NOT EXISTS zfs_pools (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
description TEXT,
|
||||||
|
raid_level VARCHAR(50) NOT NULL, -- stripe, mirror, raidz, raidz2, raidz3
|
||||||
|
disks TEXT[] NOT NULL, -- array of device paths
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
used_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
compression VARCHAR(50) NOT NULL DEFAULT 'lz4', -- off, lz4, zstd, gzip
|
||||||
|
deduplication BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
auto_expand BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
scrub_interval INTEGER NOT NULL DEFAULT 30, -- days
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
health_status VARCHAR(50) NOT NULL DEFAULT 'online', -- online, degraded, faulted, offline
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Disk repositories table
|
||||||
|
CREATE TABLE IF NOT EXISTS disk_repositories (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
description TEXT,
|
||||||
|
volume_group VARCHAR(255) NOT NULL,
|
||||||
|
logical_volume VARCHAR(255) NOT NULL,
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
used_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
filesystem_type VARCHAR(50),
|
||||||
|
mount_point TEXT,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
warning_threshold_percent INTEGER NOT NULL DEFAULT 80,
|
||||||
|
critical_threshold_percent INTEGER NOT NULL DEFAULT 90,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Physical disks table
|
||||||
|
CREATE TABLE IF NOT EXISTS physical_disks (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
device_path VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
vendor VARCHAR(255),
|
||||||
|
model VARCHAR(255),
|
||||||
|
serial_number VARCHAR(255),
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
sector_size INTEGER,
|
||||||
|
is_ssd BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
health_status VARCHAR(50) NOT NULL DEFAULT 'unknown',
|
||||||
|
health_details JSONB,
|
||||||
|
is_used BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Volume groups table
|
||||||
|
CREATE TABLE IF NOT EXISTS volume_groups (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
free_bytes BIGINT NOT NULL,
|
||||||
|
physical_volumes TEXT[],
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- SCST iSCSI targets table
|
||||||
|
CREATE TABLE IF NOT EXISTS scst_targets (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
iqn VARCHAR(512) NOT NULL UNIQUE,
|
||||||
|
target_type VARCHAR(50) NOT NULL, -- 'disk', 'vtl', 'physical_tape'
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
description TEXT,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
single_initiator_only BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- SCST LUN mappings table
|
||||||
|
CREATE TABLE IF NOT EXISTS scst_luns (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
target_id UUID NOT NULL REFERENCES scst_targets(id) ON DELETE CASCADE,
|
||||||
|
lun_number INTEGER NOT NULL,
|
||||||
|
device_name VARCHAR(255) NOT NULL,
|
||||||
|
device_path VARCHAR(512) NOT NULL,
|
||||||
|
handler_type VARCHAR(50) NOT NULL, -- 'vdisk', 'sg', 'tape'
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(target_id, lun_number)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- SCST initiator groups table
|
||||||
|
CREATE TABLE IF NOT EXISTS scst_initiator_groups (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
target_id UUID NOT NULL REFERENCES scst_targets(id) ON DELETE CASCADE,
|
||||||
|
group_name VARCHAR(255) NOT NULL,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(target_id, group_name)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- SCST initiators table
|
||||||
|
CREATE TABLE IF NOT EXISTS scst_initiators (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
group_id UUID NOT NULL REFERENCES scst_initiator_groups(id) ON DELETE CASCADE,
|
||||||
|
iqn VARCHAR(512) NOT NULL,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(group_id, iqn)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Physical tape libraries table
|
||||||
|
CREATE TABLE IF NOT EXISTS physical_tape_libraries (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
serial_number VARCHAR(255),
|
||||||
|
vendor VARCHAR(255),
|
||||||
|
model VARCHAR(255),
|
||||||
|
changer_device_path VARCHAR(512),
|
||||||
|
changer_stable_path VARCHAR(512),
|
||||||
|
slot_count INTEGER,
|
||||||
|
drive_count INTEGER,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
discovered_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
last_inventory_at TIMESTAMP,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Physical tape drives table
|
||||||
|
CREATE TABLE IF NOT EXISTS physical_tape_drives (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
library_id UUID NOT NULL REFERENCES physical_tape_libraries(id) ON DELETE CASCADE,
|
||||||
|
drive_number INTEGER NOT NULL,
|
||||||
|
device_path VARCHAR(512),
|
||||||
|
stable_path VARCHAR(512),
|
||||||
|
vendor VARCHAR(255),
|
||||||
|
model VARCHAR(255),
|
||||||
|
serial_number VARCHAR(255),
|
||||||
|
drive_type VARCHAR(50), -- 'LTO-8', 'LTO-9', etc.
|
||||||
|
status VARCHAR(50) NOT NULL DEFAULT 'unknown', -- 'idle', 'loading', 'ready', 'error'
|
||||||
|
current_tape_barcode VARCHAR(255),
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(library_id, drive_number)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Physical tape slots table
|
||||||
|
CREATE TABLE IF NOT EXISTS physical_tape_slots (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
library_id UUID NOT NULL REFERENCES physical_tape_libraries(id) ON DELETE CASCADE,
|
||||||
|
slot_number INTEGER NOT NULL,
|
||||||
|
barcode VARCHAR(255),
|
||||||
|
tape_present BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
tape_type VARCHAR(50),
|
||||||
|
last_updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(library_id, slot_number)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Virtual tape libraries table
|
||||||
|
CREATE TABLE IF NOT EXISTS virtual_tape_libraries (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
description TEXT,
|
||||||
|
mhvtl_library_id INTEGER,
|
||||||
|
backing_store_path TEXT NOT NULL,
|
||||||
|
slot_count INTEGER NOT NULL DEFAULT 10,
|
||||||
|
drive_count INTEGER NOT NULL DEFAULT 2,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Virtual tape drives table
|
||||||
|
CREATE TABLE IF NOT EXISTS virtual_tape_drives (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
library_id UUID NOT NULL REFERENCES virtual_tape_libraries(id) ON DELETE CASCADE,
|
||||||
|
drive_number INTEGER NOT NULL,
|
||||||
|
device_path VARCHAR(512),
|
||||||
|
stable_path VARCHAR(512),
|
||||||
|
status VARCHAR(50) NOT NULL DEFAULT 'idle',
|
||||||
|
current_tape_id UUID,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(library_id, drive_number)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Virtual tapes table
|
||||||
|
CREATE TABLE IF NOT EXISTS virtual_tapes (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
library_id UUID NOT NULL REFERENCES virtual_tape_libraries(id) ON DELETE CASCADE,
|
||||||
|
barcode VARCHAR(255) NOT NULL,
|
||||||
|
slot_number INTEGER,
|
||||||
|
image_file_path TEXT NOT NULL,
|
||||||
|
size_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
used_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
tape_type VARCHAR(50) NOT NULL DEFAULT 'LTO-8',
|
||||||
|
status VARCHAR(50) NOT NULL DEFAULT 'idle', -- 'idle', 'in_drive', 'exported'
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(library_id, barcode)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Indexes for performance
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_disk_repositories_name ON disk_repositories(name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_disk_repositories_active ON disk_repositories(is_active);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_disks_device_path ON physical_disks(device_path);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_targets_iqn ON scst_targets(iqn);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_targets_type ON scst_targets(target_type);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_luns_target_id ON scst_luns(target_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_initiators_group_id ON scst_initiators(group_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_libraries_name ON physical_tape_libraries(name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_drives_library_id ON physical_tape_drives(library_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_slots_library_id ON physical_tape_slots(library_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_libraries_name ON virtual_tape_libraries(name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_drives_library_id ON virtual_tape_drives(library_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tapes_library_id ON virtual_tapes(library_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tapes_barcode ON virtual_tapes(barcode);
|
||||||
|
|
||||||
22
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/003_object_storage_config.sql
vendored
Normal file
22
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/003_object_storage_config.sql
vendored
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
-- Migration: Object Storage Configuration
|
||||||
|
-- Description: Creates table for storing MinIO object storage configuration
|
||||||
|
-- Date: 2026-01-09
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS object_storage_config (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
dataset_path VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
mount_point VARCHAR(512) NOT NULL,
|
||||||
|
pool_name VARCHAR(255) NOT NULL,
|
||||||
|
dataset_name VARCHAR(255) NOT NULL,
|
||||||
|
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_object_storage_config_pool_name ON object_storage_config(pool_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_object_storage_config_updated_at ON object_storage_config(updated_at);
|
||||||
|
|
||||||
|
COMMENT ON TABLE object_storage_config IS 'Stores MinIO object storage configuration, linking to ZFS datasets';
|
||||||
|
COMMENT ON COLUMN object_storage_config.dataset_path IS 'Full ZFS dataset path (e.g., pool/dataset)';
|
||||||
|
COMMENT ON COLUMN object_storage_config.mount_point IS 'Mount point path for the dataset';
|
||||||
|
COMMENT ON COLUMN object_storage_config.pool_name IS 'ZFS pool name';
|
||||||
|
COMMENT ON COLUMN object_storage_config.dataset_name IS 'ZFS dataset name';
|
||||||
174
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/003_performance_indexes.sql
vendored
Normal file
174
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/003_performance_indexes.sql
vendored
Normal file
@@ -0,0 +1,174 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Performance Optimization: Database Indexes
|
||||||
|
-- Version: 3.0
|
||||||
|
-- Description: Adds indexes for frequently queried columns to improve query performance
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- Authentication & Authorization Indexes
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- Users table indexes
|
||||||
|
-- Username is frequently queried during login
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
|
||||||
|
-- Email lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
|
||||||
|
-- Active user lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_users_is_active ON users(is_active) WHERE is_active = true;
|
||||||
|
|
||||||
|
-- Sessions table indexes
|
||||||
|
-- Token hash lookups are very frequent (every authenticated request)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sessions_token_hash ON sessions(token_hash);
|
||||||
|
-- User session lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions(user_id);
|
||||||
|
-- Expired session cleanup (index on expires_at for efficient cleanup queries)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sessions_expires_at ON sessions(expires_at);
|
||||||
|
|
||||||
|
-- User roles junction table
|
||||||
|
-- Lookup roles for a user (frequent during permission checks)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_user_roles_user_id ON user_roles(user_id);
|
||||||
|
-- Lookup users with a role
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_user_roles_role_id ON user_roles(role_id);
|
||||||
|
|
||||||
|
-- Role permissions junction table
|
||||||
|
-- Lookup permissions for a role
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_role_permissions_role_id ON role_permissions(role_id);
|
||||||
|
-- Lookup roles with a permission
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_role_permissions_permission_id ON role_permissions(permission_id);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- Audit & Monitoring Indexes
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- Audit log indexes
|
||||||
|
-- Time-based queries (most common audit log access pattern)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_log_created_at ON audit_log(created_at DESC);
|
||||||
|
-- User activity queries
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_log_user_id ON audit_log(user_id);
|
||||||
|
-- Resource-based queries
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_log_resource ON audit_log(resource_type, resource_id);
|
||||||
|
|
||||||
|
-- Alerts table indexes
|
||||||
|
-- Time-based ordering (default ordering in ListAlerts)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_created_at ON alerts(created_at DESC);
|
||||||
|
-- Severity filtering
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_severity ON alerts(severity);
|
||||||
|
-- Source filtering
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_source ON alerts(source);
|
||||||
|
-- Acknowledgment status
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_acknowledged ON alerts(is_acknowledged) WHERE is_acknowledged = false;
|
||||||
|
-- Resource-based queries
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_resource ON alerts(resource_type, resource_id);
|
||||||
|
-- Composite index for common filter combinations
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_alerts_severity_acknowledged ON alerts(severity, is_acknowledged, created_at DESC);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- Task Management Indexes
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- Tasks table indexes
|
||||||
|
-- Status filtering (very common in task queries)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_status ON tasks(status);
|
||||||
|
-- Created by user
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_created_by ON tasks(created_by);
|
||||||
|
-- Time-based queries
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_created_at ON tasks(created_at DESC);
|
||||||
|
-- Status + time composite (common query pattern)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_status_created_at ON tasks(status, created_at DESC);
|
||||||
|
-- Failed tasks lookup (for alerting)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tasks_failed_recent ON tasks(status, created_at DESC) WHERE status = 'failed';
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- Storage Indexes
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- Disk repositories indexes
|
||||||
|
-- Active repository lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_disk_repositories_is_active ON disk_repositories(is_active) WHERE is_active = true;
|
||||||
|
-- Name lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_disk_repositories_name ON disk_repositories(name);
|
||||||
|
-- Volume group lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_disk_repositories_vg ON disk_repositories(volume_group);
|
||||||
|
|
||||||
|
-- Physical disks indexes
|
||||||
|
-- Device path lookups (for sync operations)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_disks_device_path ON physical_disks(device_path);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- SCST Indexes
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- SCST targets indexes
|
||||||
|
-- IQN lookups (frequent during target operations)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_targets_iqn ON scst_targets(iqn);
|
||||||
|
-- Active target lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_targets_is_active ON scst_targets(is_active) WHERE is_active = true;
|
||||||
|
|
||||||
|
-- SCST LUNs indexes
|
||||||
|
-- Target + LUN lookups (very frequent)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_luns_target_lun ON scst_luns(target_id, lun_number);
|
||||||
|
-- Device path lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_luns_device_path ON scst_luns(device_path);
|
||||||
|
|
||||||
|
-- SCST initiator groups indexes
|
||||||
|
-- Target + group name lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_initiator_groups_target ON scst_initiator_groups(target_id);
|
||||||
|
-- Group name lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_initiator_groups_name ON scst_initiator_groups(group_name);
|
||||||
|
|
||||||
|
-- SCST initiators indexes
|
||||||
|
-- Group + IQN lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_initiators_group_iqn ON scst_initiators(group_id, iqn);
|
||||||
|
-- Active initiator lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_scst_initiators_is_active ON scst_initiators(is_active) WHERE is_active = true;
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- Tape Library Indexes
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- Physical tape libraries indexes
|
||||||
|
-- Serial number lookups (for discovery)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_libraries_serial ON physical_tape_libraries(serial_number);
|
||||||
|
-- Active library lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_libraries_is_active ON physical_tape_libraries(is_active) WHERE is_active = true;
|
||||||
|
|
||||||
|
-- Physical tape drives indexes
|
||||||
|
-- Library + drive number lookups (very frequent)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_drives_library_drive ON physical_tape_drives(library_id, drive_number);
|
||||||
|
-- Status filtering
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_drives_status ON physical_tape_drives(status);
|
||||||
|
|
||||||
|
-- Physical tape slots indexes
|
||||||
|
-- Library + slot number lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_slots_library_slot ON physical_tape_slots(library_id, slot_number);
|
||||||
|
-- Barcode lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_physical_tape_slots_barcode ON physical_tape_slots(barcode) WHERE barcode IS NOT NULL;
|
||||||
|
|
||||||
|
-- Virtual tape libraries indexes
|
||||||
|
-- MHVTL library ID lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_libraries_mhvtl_id ON virtual_tape_libraries(mhvtl_library_id);
|
||||||
|
-- Active library lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_libraries_is_active ON virtual_tape_libraries(is_active) WHERE is_active = true;
|
||||||
|
|
||||||
|
-- Virtual tape drives indexes
|
||||||
|
-- Library + drive number lookups (very frequent)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_drives_library_drive ON virtual_tape_drives(library_id, drive_number);
|
||||||
|
-- Status filtering
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_drives_status ON virtual_tape_drives(status);
|
||||||
|
-- Current tape lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tape_drives_current_tape ON virtual_tape_drives(current_tape_id) WHERE current_tape_id IS NOT NULL;
|
||||||
|
|
||||||
|
-- Virtual tapes indexes
|
||||||
|
-- Library + slot number lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tapes_library_slot ON virtual_tapes(library_id, slot_number);
|
||||||
|
-- Barcode lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tapes_barcode ON virtual_tapes(barcode) WHERE barcode IS NOT NULL;
|
||||||
|
-- Status filtering
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_virtual_tapes_status ON virtual_tapes(status);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- Statistics Update
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- Update table statistics for query planner
|
||||||
|
ANALYZE;
|
||||||
|
|
||||||
31
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/004_add_zfs_pools_table.sql
vendored
Normal file
31
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/004_add_zfs_pools_table.sql
vendored
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Add ZFS Pools Table
|
||||||
|
-- Version: 4.0
|
||||||
|
-- Note: This migration adds the zfs_pools table that was added to migration 002
|
||||||
|
-- but may not have been applied if migration 002 was run before the table was added
|
||||||
|
|
||||||
|
-- ZFS pools table
|
||||||
|
CREATE TABLE IF NOT EXISTS zfs_pools (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
description TEXT,
|
||||||
|
raid_level VARCHAR(50) NOT NULL, -- stripe, mirror, raidz, raidz2, raidz3
|
||||||
|
disks TEXT[] NOT NULL, -- array of device paths
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
used_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
compression VARCHAR(50) NOT NULL DEFAULT 'lz4', -- off, lz4, zstd, gzip
|
||||||
|
deduplication BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
auto_expand BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
scrub_interval INTEGER NOT NULL DEFAULT 30, -- days
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
health_status VARCHAR(50) NOT NULL DEFAULT 'online', -- online, degraded, faulted, offline
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Create index on name for faster lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_pools_name ON zfs_pools(name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_pools_created_by ON zfs_pools(created_by);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_pools_health_status ON zfs_pools(health_status);
|
||||||
|
|
||||||
35
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/005_add_zfs_datasets_table.sql
vendored
Normal file
35
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/005_add_zfs_datasets_table.sql
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Add ZFS Datasets Table
|
||||||
|
-- Version: 5.0
|
||||||
|
-- Description: Stores ZFS dataset metadata in database for faster queries and consistency
|
||||||
|
|
||||||
|
-- ZFS datasets table
|
||||||
|
CREATE TABLE IF NOT EXISTS zfs_datasets (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(512) NOT NULL UNIQUE, -- Full dataset name (e.g., pool/dataset)
|
||||||
|
pool_id UUID NOT NULL REFERENCES zfs_pools(id) ON DELETE CASCADE,
|
||||||
|
pool_name VARCHAR(255) NOT NULL, -- Denormalized for faster queries
|
||||||
|
type VARCHAR(50) NOT NULL, -- filesystem, volume, snapshot
|
||||||
|
mount_point TEXT, -- Mount point path (null for volumes)
|
||||||
|
used_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
available_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
referenced_bytes BIGINT NOT NULL DEFAULT 0,
|
||||||
|
compression VARCHAR(50) NOT NULL DEFAULT 'lz4', -- off, lz4, zstd, gzip
|
||||||
|
deduplication VARCHAR(50) NOT NULL DEFAULT 'off', -- off, on, verify
|
||||||
|
quota BIGINT DEFAULT -1, -- -1 for unlimited, bytes otherwise
|
||||||
|
reservation BIGINT NOT NULL DEFAULT 0, -- Reserved space in bytes
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Create indexes for faster lookups
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_datasets_pool_id ON zfs_datasets(pool_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_datasets_pool_name ON zfs_datasets(pool_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_datasets_name ON zfs_datasets(name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_datasets_type ON zfs_datasets(type);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_datasets_created_by ON zfs_datasets(created_by);
|
||||||
|
|
||||||
|
-- Composite index for common queries (list datasets by pool)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_datasets_pool_type ON zfs_datasets(pool_id, type);
|
||||||
|
|
||||||
50
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/006_add_zfs_shares_and_iscsi.sql
vendored
Normal file
50
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/006_add_zfs_shares_and_iscsi.sql
vendored
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Add ZFS Shares and iSCSI Export Tables
|
||||||
|
-- Version: 6.0
|
||||||
|
-- Description: Separate tables for filesystem shares (NFS/SMB) and volume iSCSI exports
|
||||||
|
|
||||||
|
-- ZFS Filesystem Shares Table (for NFS/SMB)
|
||||||
|
CREATE TABLE IF NOT EXISTS zfs_shares (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
dataset_id UUID NOT NULL REFERENCES zfs_datasets(id) ON DELETE CASCADE,
|
||||||
|
share_type VARCHAR(50) NOT NULL, -- 'nfs', 'smb', 'both'
|
||||||
|
nfs_enabled BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
nfs_options TEXT, -- e.g., "rw,sync,no_subtree_check"
|
||||||
|
nfs_clients TEXT[], -- Allowed client IPs/networks
|
||||||
|
smb_enabled BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
smb_share_name VARCHAR(255), -- SMB share name
|
||||||
|
smb_path TEXT, -- SMB share path (usually same as mount_point)
|
||||||
|
smb_comment TEXT,
|
||||||
|
smb_guest_ok BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
smb_read_only BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
smb_browseable BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
UNIQUE(dataset_id) -- One share config per dataset
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ZFS Volume iSCSI Exports Table
|
||||||
|
CREATE TABLE IF NOT EXISTS zfs_iscsi_exports (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
dataset_id UUID NOT NULL REFERENCES zfs_datasets(id) ON DELETE CASCADE,
|
||||||
|
target_id UUID REFERENCES scst_targets(id) ON DELETE SET NULL, -- Link to SCST target
|
||||||
|
lun_number INTEGER, -- LUN number in the target
|
||||||
|
device_path TEXT, -- /dev/zvol/pool/volume path
|
||||||
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
UNIQUE(dataset_id) -- One iSCSI export per volume
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Create indexes
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_shares_dataset_id ON zfs_shares(dataset_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_shares_type ON zfs_shares(share_type);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_shares_active ON zfs_shares(is_active);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_iscsi_exports_dataset_id ON zfs_iscsi_exports(dataset_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_iscsi_exports_target_id ON zfs_iscsi_exports(target_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_zfs_iscsi_exports_active ON zfs_iscsi_exports(is_active);
|
||||||
|
|
||||||
3
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/007_add_vendor_to_vtl_libraries.sql
vendored
Normal file
3
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/007_add_vendor_to_vtl_libraries.sql
vendored
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
-- Add vendor column to virtual_tape_libraries table
|
||||||
|
ALTER TABLE virtual_tape_libraries ADD COLUMN IF NOT EXISTS vendor VARCHAR(255);
|
||||||
|
|
||||||
45
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/008_add_user_groups.sql
vendored
Normal file
45
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/008_add_user_groups.sql
vendored
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
-- Add user groups feature
|
||||||
|
-- Groups table
|
||||||
|
CREATE TABLE IF NOT EXISTS groups (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
description TEXT,
|
||||||
|
is_system BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- User groups junction table
|
||||||
|
CREATE TABLE IF NOT EXISTS user_groups (
|
||||||
|
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
|
||||||
|
assigned_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
assigned_by UUID REFERENCES users(id),
|
||||||
|
PRIMARY KEY (user_id, group_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Group roles junction table (groups can have roles)
|
||||||
|
CREATE TABLE IF NOT EXISTS group_roles (
|
||||||
|
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
|
||||||
|
role_id UUID NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
|
||||||
|
granted_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (group_id, role_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Indexes
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_groups_name ON groups(name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_user_groups_user_id ON user_groups(user_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_user_groups_group_id ON user_groups(group_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_group_roles_group_id ON group_roles(group_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_group_roles_role_id ON group_roles(role_id);
|
||||||
|
|
||||||
|
-- Insert default system groups
|
||||||
|
INSERT INTO groups (name, description, is_system) VALUES
|
||||||
|
('wheel', 'System administrators group', true),
|
||||||
|
('operators', 'System operators group', true),
|
||||||
|
('backup', 'Backup operators group', true),
|
||||||
|
('auditors', 'Auditors group', true),
|
||||||
|
('storage_admins', 'Storage administrators group', true),
|
||||||
|
('services', 'Service accounts group', true)
|
||||||
|
ON CONFLICT (name) DO NOTHING;
|
||||||
|
|
||||||
34
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/009_backup_jobs_schema.sql
vendored
Normal file
34
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/009_backup_jobs_schema.sql
vendored
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Backup Jobs Schema
|
||||||
|
-- Version: 9.0
|
||||||
|
|
||||||
|
-- Backup jobs table
|
||||||
|
CREATE TABLE IF NOT EXISTS backup_jobs (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
job_id INTEGER NOT NULL UNIQUE, -- Bareos job ID
|
||||||
|
job_name VARCHAR(255) NOT NULL,
|
||||||
|
client_name VARCHAR(255) NOT NULL,
|
||||||
|
job_type VARCHAR(50) NOT NULL, -- 'Backup', 'Restore', 'Verify', 'Copy', 'Migrate'
|
||||||
|
job_level VARCHAR(50) NOT NULL, -- 'Full', 'Incremental', 'Differential', 'Since'
|
||||||
|
status VARCHAR(50) NOT NULL, -- 'Running', 'Completed', 'Failed', 'Canceled', 'Waiting'
|
||||||
|
bytes_written BIGINT NOT NULL DEFAULT 0,
|
||||||
|
files_written INTEGER NOT NULL DEFAULT 0,
|
||||||
|
duration_seconds INTEGER,
|
||||||
|
started_at TIMESTAMP,
|
||||||
|
ended_at TIMESTAMP,
|
||||||
|
error_message TEXT,
|
||||||
|
storage_name VARCHAR(255),
|
||||||
|
pool_name VARCHAR(255),
|
||||||
|
volume_name VARCHAR(255),
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Indexes for performance
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_job_id ON backup_jobs(job_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_job_name ON backup_jobs(job_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_client_name ON backup_jobs(client_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_status ON backup_jobs(status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_started_at ON backup_jobs(started_at DESC);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_job_type ON backup_jobs(job_type);
|
||||||
|
|
||||||
39
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/010_add_backup_permissions.sql
vendored
Normal file
39
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/010_add_backup_permissions.sql
vendored
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Add Backup Permissions
|
||||||
|
-- Version: 10.0
|
||||||
|
|
||||||
|
-- Insert backup permissions
|
||||||
|
INSERT INTO permissions (name, resource, action, description) VALUES
|
||||||
|
('backup:read', 'backup', 'read', 'View backup jobs and history'),
|
||||||
|
('backup:write', 'backup', 'write', 'Create and manage backup jobs'),
|
||||||
|
('backup:manage', 'backup', 'manage', 'Full backup management')
|
||||||
|
ON CONFLICT (name) DO NOTHING;
|
||||||
|
|
||||||
|
-- Assign backup permissions to roles
|
||||||
|
|
||||||
|
-- Admin gets all backup permissions (explicitly assign since admin query in 001 only runs once)
|
||||||
|
INSERT INTO role_permissions (role_id, permission_id)
|
||||||
|
SELECT r.id, p.id
|
||||||
|
FROM roles r, permissions p
|
||||||
|
WHERE r.name = 'admin'
|
||||||
|
AND p.resource = 'backup'
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
|
-- Operator gets read and write permissions for backup
|
||||||
|
INSERT INTO role_permissions (role_id, permission_id)
|
||||||
|
SELECT r.id, p.id
|
||||||
|
FROM roles r, permissions p
|
||||||
|
WHERE r.name = 'operator'
|
||||||
|
AND p.resource = 'backup'
|
||||||
|
AND p.action IN ('read', 'write')
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
|
-- ReadOnly gets only read permission for backup
|
||||||
|
INSERT INTO role_permissions (role_id, permission_id)
|
||||||
|
SELECT r.id, p.id
|
||||||
|
FROM roles r, permissions p
|
||||||
|
WHERE r.name = 'readonly'
|
||||||
|
AND p.resource = 'backup'
|
||||||
|
AND p.action = 'read'
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
209
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/011_sync_bacula_jobs_function.sql
vendored
Normal file
209
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/011_sync_bacula_jobs_function.sql
vendored
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- PostgreSQL Function to Sync Jobs from Bacula to Calypso
|
||||||
|
-- Version: 11.0
|
||||||
|
--
|
||||||
|
-- This function syncs jobs from Bacula database (Job table) to Calypso database (backup_jobs table)
|
||||||
|
-- Uses dblink extension to query Bacula database from Calypso database
|
||||||
|
--
|
||||||
|
-- Prerequisites:
|
||||||
|
-- 1. dblink extension must be installed: CREATE EXTENSION IF NOT EXISTS dblink;
|
||||||
|
-- 2. User must have access to both databases
|
||||||
|
-- 3. Connection parameters must be configured in the function
|
||||||
|
|
||||||
|
-- Create function to sync jobs from Bacula to Calypso
|
||||||
|
CREATE OR REPLACE FUNCTION sync_bacula_jobs(
|
||||||
|
bacula_db_name TEXT DEFAULT 'bacula',
|
||||||
|
bacula_host TEXT DEFAULT 'localhost',
|
||||||
|
bacula_port INTEGER DEFAULT 5432,
|
||||||
|
bacula_user TEXT DEFAULT 'calypso',
|
||||||
|
bacula_password TEXT DEFAULT ''
|
||||||
|
)
|
||||||
|
RETURNS TABLE(
|
||||||
|
jobs_synced INTEGER,
|
||||||
|
jobs_inserted INTEGER,
|
||||||
|
jobs_updated INTEGER,
|
||||||
|
errors INTEGER
|
||||||
|
) AS $$
|
||||||
|
DECLARE
|
||||||
|
conn_str TEXT;
|
||||||
|
jobs_count INTEGER := 0;
|
||||||
|
inserted_count INTEGER := 0;
|
||||||
|
updated_count INTEGER := 0;
|
||||||
|
error_count INTEGER := 0;
|
||||||
|
job_record RECORD;
|
||||||
|
BEGIN
|
||||||
|
-- Build dblink connection string
|
||||||
|
conn_str := format(
|
||||||
|
'dbname=%s host=%s port=%s user=%s password=%s',
|
||||||
|
bacula_db_name,
|
||||||
|
bacula_host,
|
||||||
|
bacula_port,
|
||||||
|
bacula_user,
|
||||||
|
bacula_password
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Query jobs from Bacula database using dblink
|
||||||
|
FOR job_record IN
|
||||||
|
SELECT * FROM dblink(
|
||||||
|
conn_str,
|
||||||
|
$QUERY$
|
||||||
|
SELECT
|
||||||
|
j.JobId,
|
||||||
|
j.Name as job_name,
|
||||||
|
COALESCE(c.Name, 'unknown') as client_name,
|
||||||
|
CASE
|
||||||
|
WHEN j.Type = 'B' THEN 'Backup'
|
||||||
|
WHEN j.Type = 'R' THEN 'Restore'
|
||||||
|
WHEN j.Type = 'V' THEN 'Verify'
|
||||||
|
WHEN j.Type = 'C' THEN 'Copy'
|
||||||
|
WHEN j.Type = 'M' THEN 'Migrate'
|
||||||
|
ELSE 'Backup'
|
||||||
|
END as job_type,
|
||||||
|
CASE
|
||||||
|
WHEN j.Level = 'F' THEN 'Full'
|
||||||
|
WHEN j.Level = 'I' THEN 'Incremental'
|
||||||
|
WHEN j.Level = 'D' THEN 'Differential'
|
||||||
|
WHEN j.Level = 'S' THEN 'Since'
|
||||||
|
ELSE 'Full'
|
||||||
|
END as job_level,
|
||||||
|
CASE
|
||||||
|
WHEN j.JobStatus = 'T' THEN 'Running'
|
||||||
|
WHEN j.JobStatus = 'C' THEN 'Completed'
|
||||||
|
WHEN j.JobStatus = 'f' OR j.JobStatus = 'F' THEN 'Failed'
|
||||||
|
WHEN j.JobStatus = 'A' THEN 'Canceled'
|
||||||
|
WHEN j.JobStatus = 'W' THEN 'Waiting'
|
||||||
|
ELSE 'Waiting'
|
||||||
|
END as status,
|
||||||
|
COALESCE(j.JobBytes, 0) as bytes_written,
|
||||||
|
COALESCE(j.JobFiles, 0) as files_written,
|
||||||
|
j.StartTime as started_at,
|
||||||
|
j.EndTime as ended_at,
|
||||||
|
CASE
|
||||||
|
WHEN j.EndTime IS NOT NULL AND j.StartTime IS NOT NULL
|
||||||
|
THEN EXTRACT(EPOCH FROM (j.EndTime - j.StartTime))::INTEGER
|
||||||
|
ELSE NULL
|
||||||
|
END as duration_seconds
|
||||||
|
FROM Job j
|
||||||
|
LEFT JOIN Client c ON j.ClientId = c.ClientId
|
||||||
|
ORDER BY j.StartTime DESC
|
||||||
|
LIMIT 1000
|
||||||
|
$QUERY$
|
||||||
|
) AS t(
|
||||||
|
job_id INTEGER,
|
||||||
|
job_name TEXT,
|
||||||
|
client_name TEXT,
|
||||||
|
job_type TEXT,
|
||||||
|
job_level TEXT,
|
||||||
|
status TEXT,
|
||||||
|
bytes_written BIGINT,
|
||||||
|
files_written INTEGER,
|
||||||
|
started_at TIMESTAMP,
|
||||||
|
ended_at TIMESTAMP,
|
||||||
|
duration_seconds INTEGER
|
||||||
|
)
|
||||||
|
LOOP
|
||||||
|
BEGIN
|
||||||
|
-- Check if job already exists (before insert/update)
|
||||||
|
IF EXISTS (SELECT 1 FROM backup_jobs WHERE job_id = job_record.job_id) THEN
|
||||||
|
updated_count := updated_count + 1;
|
||||||
|
ELSE
|
||||||
|
inserted_count := inserted_count + 1;
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Upsert job to backup_jobs table
|
||||||
|
INSERT INTO backup_jobs (
|
||||||
|
job_id, job_name, client_name, job_type, job_level, status,
|
||||||
|
bytes_written, files_written, started_at, ended_at, duration_seconds,
|
||||||
|
updated_at
|
||||||
|
) VALUES (
|
||||||
|
job_record.job_id,
|
||||||
|
job_record.job_name,
|
||||||
|
job_record.client_name,
|
||||||
|
job_record.job_type,
|
||||||
|
job_record.job_level,
|
||||||
|
job_record.status,
|
||||||
|
job_record.bytes_written,
|
||||||
|
job_record.files_written,
|
||||||
|
job_record.started_at,
|
||||||
|
job_record.ended_at,
|
||||||
|
job_record.duration_seconds,
|
||||||
|
NOW()
|
||||||
|
)
|
||||||
|
ON CONFLICT (job_id) DO UPDATE SET
|
||||||
|
job_name = EXCLUDED.job_name,
|
||||||
|
client_name = EXCLUDED.client_name,
|
||||||
|
job_type = EXCLUDED.job_type,
|
||||||
|
job_level = EXCLUDED.job_level,
|
||||||
|
status = EXCLUDED.status,
|
||||||
|
bytes_written = EXCLUDED.bytes_written,
|
||||||
|
files_written = EXCLUDED.files_written,
|
||||||
|
started_at = EXCLUDED.started_at,
|
||||||
|
ended_at = EXCLUDED.ended_at,
|
||||||
|
duration_seconds = EXCLUDED.duration_seconds,
|
||||||
|
updated_at = NOW();
|
||||||
|
|
||||||
|
jobs_count := jobs_count + 1;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
error_count := error_count + 1;
|
||||||
|
-- Log error but continue with next job
|
||||||
|
RAISE WARNING 'Error syncing job %: %', job_record.job_id, SQLERRM;
|
||||||
|
END;
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
-- Return summary
|
||||||
|
RETURN QUERY SELECT jobs_count, inserted_count, updated_count, error_count;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
-- Create a simpler version that uses current database connection settings
|
||||||
|
-- This version assumes Bacula is on same host/port with same user
|
||||||
|
CREATE OR REPLACE FUNCTION sync_bacula_jobs_simple()
|
||||||
|
RETURNS TABLE(
|
||||||
|
jobs_synced INTEGER,
|
||||||
|
jobs_inserted INTEGER,
|
||||||
|
jobs_updated INTEGER,
|
||||||
|
errors INTEGER
|
||||||
|
) AS $$
|
||||||
|
DECLARE
|
||||||
|
current_user_name TEXT;
|
||||||
|
current_host TEXT;
|
||||||
|
current_port INTEGER;
|
||||||
|
current_db TEXT;
|
||||||
|
BEGIN
|
||||||
|
-- Get current connection info
|
||||||
|
SELECT
|
||||||
|
current_user,
|
||||||
|
COALESCE(inet_server_addr()::TEXT, 'localhost'),
|
||||||
|
COALESCE(inet_server_port(), 5432),
|
||||||
|
current_database()
|
||||||
|
INTO
|
||||||
|
current_user_name,
|
||||||
|
current_host,
|
||||||
|
current_port,
|
||||||
|
current_db;
|
||||||
|
|
||||||
|
-- Call main function with current connection settings
|
||||||
|
-- Note: password needs to be passed or configured in .pgpass
|
||||||
|
RETURN QUERY
|
||||||
|
SELECT * FROM sync_bacula_jobs(
|
||||||
|
'bacula', -- Try 'bacula' first
|
||||||
|
current_host,
|
||||||
|
current_port,
|
||||||
|
current_user_name,
|
||||||
|
'' -- Empty password - will use .pgpass or peer authentication
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
-- Grant execute permission to calypso user
|
||||||
|
GRANT EXECUTE ON FUNCTION sync_bacula_jobs(TEXT, TEXT, INTEGER, TEXT, TEXT) TO calypso;
|
||||||
|
GRANT EXECUTE ON FUNCTION sync_bacula_jobs_simple() TO calypso;
|
||||||
|
|
||||||
|
-- Create index if not exists (should already exist from migration 009)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_job_id ON backup_jobs(job_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_updated_at ON backup_jobs(updated_at);
|
||||||
|
|
||||||
|
COMMENT ON FUNCTION sync_bacula_jobs IS 'Syncs jobs from Bacula database to Calypso backup_jobs table using dblink';
|
||||||
|
COMMENT ON FUNCTION sync_bacula_jobs_simple IS 'Simplified version that uses current connection settings (requires .pgpass for password)';
|
||||||
|
|
||||||
209
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/011_sync_bacula_jobs_function.sql.bak
vendored
Normal file
209
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/011_sync_bacula_jobs_function.sql.bak
vendored
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- PostgreSQL Function to Sync Jobs from Bacula to Calypso
|
||||||
|
-- Version: 11.0
|
||||||
|
--
|
||||||
|
-- This function syncs jobs from Bacula database (Job table) to Calypso database (backup_jobs table)
|
||||||
|
-- Uses dblink extension to query Bacula database from Calypso database
|
||||||
|
--
|
||||||
|
-- Prerequisites:
|
||||||
|
-- 1. dblink extension must be installed: CREATE EXTENSION IF NOT EXISTS dblink;
|
||||||
|
-- 2. User must have access to both databases
|
||||||
|
-- 3. Connection parameters must be configured in the function
|
||||||
|
|
||||||
|
-- Create function to sync jobs from Bacula to Calypso
|
||||||
|
CREATE OR REPLACE FUNCTION sync_bacula_jobs(
|
||||||
|
bacula_db_name TEXT DEFAULT 'bacula',
|
||||||
|
bacula_host TEXT DEFAULT 'localhost',
|
||||||
|
bacula_port INTEGER DEFAULT 5432,
|
||||||
|
bacula_user TEXT DEFAULT 'calypso',
|
||||||
|
bacula_password TEXT DEFAULT ''
|
||||||
|
)
|
||||||
|
RETURNS TABLE(
|
||||||
|
jobs_synced INTEGER,
|
||||||
|
jobs_inserted INTEGER,
|
||||||
|
jobs_updated INTEGER,
|
||||||
|
errors INTEGER
|
||||||
|
) AS $$
|
||||||
|
DECLARE
|
||||||
|
conn_str TEXT;
|
||||||
|
jobs_count INTEGER := 0;
|
||||||
|
inserted_count INTEGER := 0;
|
||||||
|
updated_count INTEGER := 0;
|
||||||
|
error_count INTEGER := 0;
|
||||||
|
job_record RECORD;
|
||||||
|
BEGIN
|
||||||
|
-- Build dblink connection string
|
||||||
|
conn_str := format(
|
||||||
|
'dbname=%s host=%s port=%s user=%s password=%s',
|
||||||
|
bacula_db_name,
|
||||||
|
bacula_host,
|
||||||
|
bacula_port,
|
||||||
|
bacula_user,
|
||||||
|
bacula_password
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Query jobs from Bacula database using dblink
|
||||||
|
FOR job_record IN
|
||||||
|
SELECT * FROM dblink(
|
||||||
|
conn_str,
|
||||||
|
$$
|
||||||
|
SELECT
|
||||||
|
j.JobId,
|
||||||
|
j.Name as job_name,
|
||||||
|
COALESCE(c.Name, 'unknown') as client_name,
|
||||||
|
CASE
|
||||||
|
WHEN j.Type = 'B' THEN 'Backup'
|
||||||
|
WHEN j.Type = 'R' THEN 'Restore'
|
||||||
|
WHEN j.Type = 'V' THEN 'Verify'
|
||||||
|
WHEN j.Type = 'C' THEN 'Copy'
|
||||||
|
WHEN j.Type = 'M' THEN 'Migrate'
|
||||||
|
ELSE 'Backup'
|
||||||
|
END as job_type,
|
||||||
|
CASE
|
||||||
|
WHEN j.Level = 'F' THEN 'Full'
|
||||||
|
WHEN j.Level = 'I' THEN 'Incremental'
|
||||||
|
WHEN j.Level = 'D' THEN 'Differential'
|
||||||
|
WHEN j.Level = 'S' THEN 'Since'
|
||||||
|
ELSE 'Full'
|
||||||
|
END as job_level,
|
||||||
|
CASE
|
||||||
|
WHEN j.JobStatus = 'T' THEN 'Running'
|
||||||
|
WHEN j.JobStatus = 'C' THEN 'Completed'
|
||||||
|
WHEN j.JobStatus = 'f' OR j.JobStatus = 'F' THEN 'Failed'
|
||||||
|
WHEN j.JobStatus = 'A' THEN 'Canceled'
|
||||||
|
WHEN j.JobStatus = 'W' THEN 'Waiting'
|
||||||
|
ELSE 'Waiting'
|
||||||
|
END as status,
|
||||||
|
COALESCE(j.JobBytes, 0) as bytes_written,
|
||||||
|
COALESCE(j.JobFiles, 0) as files_written,
|
||||||
|
j.StartTime as started_at,
|
||||||
|
j.EndTime as ended_at,
|
||||||
|
CASE
|
||||||
|
WHEN j.EndTime IS NOT NULL AND j.StartTime IS NOT NULL
|
||||||
|
THEN EXTRACT(EPOCH FROM (j.EndTime - j.StartTime))::INTEGER
|
||||||
|
ELSE NULL
|
||||||
|
END as duration_seconds
|
||||||
|
FROM Job j
|
||||||
|
LEFT JOIN Client c ON j.ClientId = c.ClientId
|
||||||
|
ORDER BY j.StartTime DESC
|
||||||
|
LIMIT 1000
|
||||||
|
$$
|
||||||
|
) AS t(
|
||||||
|
job_id INTEGER,
|
||||||
|
job_name TEXT,
|
||||||
|
client_name TEXT,
|
||||||
|
job_type TEXT,
|
||||||
|
job_level TEXT,
|
||||||
|
status TEXT,
|
||||||
|
bytes_written BIGINT,
|
||||||
|
files_written INTEGER,
|
||||||
|
started_at TIMESTAMP,
|
||||||
|
ended_at TIMESTAMP,
|
||||||
|
duration_seconds INTEGER
|
||||||
|
)
|
||||||
|
LOOP
|
||||||
|
BEGIN
|
||||||
|
-- Check if job already exists (before insert/update)
|
||||||
|
IF EXISTS (SELECT 1 FROM backup_jobs WHERE job_id = job_record.job_id) THEN
|
||||||
|
updated_count := updated_count + 1;
|
||||||
|
ELSE
|
||||||
|
inserted_count := inserted_count + 1;
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Upsert job to backup_jobs table
|
||||||
|
INSERT INTO backup_jobs (
|
||||||
|
job_id, job_name, client_name, job_type, job_level, status,
|
||||||
|
bytes_written, files_written, started_at, ended_at, duration_seconds,
|
||||||
|
updated_at
|
||||||
|
) VALUES (
|
||||||
|
job_record.job_id,
|
||||||
|
job_record.job_name,
|
||||||
|
job_record.client_name,
|
||||||
|
job_record.job_type,
|
||||||
|
job_record.job_level,
|
||||||
|
job_record.status,
|
||||||
|
job_record.bytes_written,
|
||||||
|
job_record.files_written,
|
||||||
|
job_record.started_at,
|
||||||
|
job_record.ended_at,
|
||||||
|
job_record.duration_seconds,
|
||||||
|
NOW()
|
||||||
|
)
|
||||||
|
ON CONFLICT (job_id) DO UPDATE SET
|
||||||
|
job_name = EXCLUDED.job_name,
|
||||||
|
client_name = EXCLUDED.client_name,
|
||||||
|
job_type = EXCLUDED.job_type,
|
||||||
|
job_level = EXCLUDED.job_level,
|
||||||
|
status = EXCLUDED.status,
|
||||||
|
bytes_written = EXCLUDED.bytes_written,
|
||||||
|
files_written = EXCLUDED.files_written,
|
||||||
|
started_at = EXCLUDED.started_at,
|
||||||
|
ended_at = EXCLUDED.ended_at,
|
||||||
|
duration_seconds = EXCLUDED.duration_seconds,
|
||||||
|
updated_at = NOW();
|
||||||
|
|
||||||
|
jobs_count := jobs_count + 1;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
error_count := error_count + 1;
|
||||||
|
-- Log error but continue with next job
|
||||||
|
RAISE WARNING 'Error syncing job %: %', job_record.job_id, SQLERRM;
|
||||||
|
END;
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
-- Return summary
|
||||||
|
RETURN QUERY SELECT jobs_count, inserted_count, updated_count, error_count;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
-- Create a simpler version that uses current database connection settings
|
||||||
|
-- This version assumes Bacula is on same host/port with same user
|
||||||
|
CREATE OR REPLACE FUNCTION sync_bacula_jobs_simple()
|
||||||
|
RETURNS TABLE(
|
||||||
|
jobs_synced INTEGER,
|
||||||
|
jobs_inserted INTEGER,
|
||||||
|
jobs_updated INTEGER,
|
||||||
|
errors INTEGER
|
||||||
|
) AS $$
|
||||||
|
DECLARE
|
||||||
|
current_user_name TEXT;
|
||||||
|
current_host TEXT;
|
||||||
|
current_port INTEGER;
|
||||||
|
current_db TEXT;
|
||||||
|
BEGIN
|
||||||
|
-- Get current connection info
|
||||||
|
SELECT
|
||||||
|
current_user,
|
||||||
|
COALESCE(inet_server_addr()::TEXT, 'localhost'),
|
||||||
|
COALESCE(inet_server_port(), 5432),
|
||||||
|
current_database()
|
||||||
|
INTO
|
||||||
|
current_user_name,
|
||||||
|
current_host,
|
||||||
|
current_port,
|
||||||
|
current_db;
|
||||||
|
|
||||||
|
-- Call main function with current connection settings
|
||||||
|
-- Note: password needs to be passed or configured in .pgpass
|
||||||
|
RETURN QUERY
|
||||||
|
SELECT * FROM sync_bacula_jobs(
|
||||||
|
'bacula', -- Try 'bacula' first
|
||||||
|
current_host,
|
||||||
|
current_port,
|
||||||
|
current_user_name,
|
||||||
|
'' -- Empty password - will use .pgpass or peer authentication
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
-- Grant execute permission to calypso user
|
||||||
|
GRANT EXECUTE ON FUNCTION sync_bacula_jobs(TEXT, TEXT, INTEGER, TEXT, TEXT) TO calypso;
|
||||||
|
GRANT EXECUTE ON FUNCTION sync_bacula_jobs_simple() TO calypso;
|
||||||
|
|
||||||
|
-- Create index if not exists (should already exist from migration 009)
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_job_id ON backup_jobs(job_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_backup_jobs_updated_at ON backup_jobs(updated_at);
|
||||||
|
|
||||||
|
COMMENT ON FUNCTION sync_bacula_jobs IS 'Syncs jobs from Bacula database to Calypso backup_jobs table using dblink';
|
||||||
|
COMMENT ON FUNCTION sync_bacula_jobs_simple IS 'Simplified version that uses current connection settings (requires .pgpass for password)';
|
||||||
|
|
||||||
23
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/012_add_snapshot_schedules_table.sql
vendored
Normal file
23
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/012_add_snapshot_schedules_table.sql
vendored
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
-- Snapshot schedules table for automated snapshot creation
|
||||||
|
CREATE TABLE IF NOT EXISTS snapshot_schedules (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
dataset VARCHAR(255) NOT NULL,
|
||||||
|
snapshot_name_template VARCHAR(255) NOT NULL, -- e.g., "auto-%Y-%m-%d-%H%M" or "daily-backup"
|
||||||
|
schedule_type VARCHAR(50) NOT NULL, -- 'hourly', 'daily', 'weekly', 'monthly', 'cron'
|
||||||
|
schedule_config JSONB NOT NULL, -- For cron: {"cron": "0 0 * * *"}, for others: {"time": "00:00", "day": 1, etc.}
|
||||||
|
recursive BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
retention_count INTEGER, -- Keep last N snapshots (null = unlimited)
|
||||||
|
retention_days INTEGER, -- Keep snapshots for N days (null = unlimited)
|
||||||
|
last_run_at TIMESTAMP,
|
||||||
|
next_run_at TIMESTAMP,
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
UNIQUE(name)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_snapshot_schedules_enabled ON snapshot_schedules(enabled);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_snapshot_schedules_next_run ON snapshot_schedules(next_run_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_snapshot_schedules_dataset ON snapshot_schedules(dataset);
|
||||||
76
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/013_add_replication_tasks_table.sql
vendored
Normal file
76
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/013_add_replication_tasks_table.sql
vendored
Normal file
@@ -0,0 +1,76 @@
|
|||||||
|
-- ZFS Replication Tasks Table
|
||||||
|
-- Supports both outbound (sender) and inbound (receiver) replication
|
||||||
|
CREATE TABLE IF NOT EXISTS replication_tasks (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
direction VARCHAR(20) NOT NULL, -- 'outbound' (sender) or 'inbound' (receiver)
|
||||||
|
|
||||||
|
-- For outbound replication (sender)
|
||||||
|
source_dataset VARCHAR(512), -- Source dataset on this system (outbound) or remote system (inbound)
|
||||||
|
target_host VARCHAR(255), -- Target host IP or hostname (for outbound)
|
||||||
|
target_port INTEGER DEFAULT 22, -- SSH port (default 22, for outbound)
|
||||||
|
target_user VARCHAR(255) DEFAULT 'root', -- SSH user (for outbound)
|
||||||
|
target_dataset VARCHAR(512), -- Target dataset on remote system (for outbound)
|
||||||
|
target_ssh_key_path TEXT, -- Path to SSH private key (for outbound)
|
||||||
|
|
||||||
|
-- For inbound replication (receiver)
|
||||||
|
source_host VARCHAR(255), -- Source host IP or hostname (for inbound)
|
||||||
|
source_port INTEGER DEFAULT 22, -- SSH port (for inbound)
|
||||||
|
source_user VARCHAR(255) DEFAULT 'root', -- SSH user (for inbound)
|
||||||
|
local_dataset VARCHAR(512), -- Local dataset to receive snapshots (for inbound)
|
||||||
|
|
||||||
|
-- Common settings
|
||||||
|
schedule_type VARCHAR(50), -- 'manual', 'hourly', 'daily', 'weekly', 'monthly', 'cron'
|
||||||
|
schedule_config JSONB, -- Schedule configuration (similar to snapshot schedules)
|
||||||
|
compression VARCHAR(50) DEFAULT 'lz4', -- 'off', 'lz4', 'gzip', 'zstd'
|
||||||
|
encryption BOOLEAN DEFAULT false, -- Enable encryption during transfer
|
||||||
|
recursive BOOLEAN DEFAULT false, -- Replicate recursively
|
||||||
|
incremental BOOLEAN DEFAULT true, -- Use incremental replication
|
||||||
|
auto_snapshot BOOLEAN DEFAULT true, -- Auto-create snapshot before replication
|
||||||
|
|
||||||
|
-- Status and tracking
|
||||||
|
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
status VARCHAR(50) DEFAULT 'idle', -- 'idle', 'running', 'failed', 'paused'
|
||||||
|
last_run_at TIMESTAMP,
|
||||||
|
last_run_status VARCHAR(50), -- 'success', 'failed', 'partial'
|
||||||
|
last_run_error TEXT,
|
||||||
|
next_run_at TIMESTAMP,
|
||||||
|
last_snapshot_sent VARCHAR(512), -- Last snapshot successfully sent (for outbound)
|
||||||
|
last_snapshot_received VARCHAR(512), -- Last snapshot successfully received (for inbound)
|
||||||
|
|
||||||
|
-- Statistics
|
||||||
|
total_runs INTEGER DEFAULT 0,
|
||||||
|
successful_runs INTEGER DEFAULT 0,
|
||||||
|
failed_runs INTEGER DEFAULT 0,
|
||||||
|
bytes_sent BIGINT DEFAULT 0, -- Total bytes sent (for outbound)
|
||||||
|
bytes_received BIGINT DEFAULT 0, -- Total bytes received (for inbound)
|
||||||
|
|
||||||
|
created_by UUID REFERENCES users(id),
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
|
||||||
|
-- Validation: ensure required fields based on direction
|
||||||
|
CONSTRAINT chk_direction CHECK (direction IN ('outbound', 'inbound')),
|
||||||
|
CONSTRAINT chk_outbound_fields CHECK (
|
||||||
|
direction != 'outbound' OR (
|
||||||
|
source_dataset IS NOT NULL AND
|
||||||
|
target_host IS NOT NULL AND
|
||||||
|
target_dataset IS NOT NULL
|
||||||
|
)
|
||||||
|
),
|
||||||
|
CONSTRAINT chk_inbound_fields CHECK (
|
||||||
|
direction != 'inbound' OR (
|
||||||
|
source_host IS NOT NULL AND
|
||||||
|
source_dataset IS NOT NULL AND
|
||||||
|
local_dataset IS NOT NULL
|
||||||
|
)
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Create indexes
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_direction ON replication_tasks(direction);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_enabled ON replication_tasks(enabled);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_status ON replication_tasks(status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_next_run ON replication_tasks(next_run_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_source_dataset ON replication_tasks(source_dataset);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_replication_tasks_target_host ON replication_tasks(target_host);
|
||||||
30
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/014_add_client_categories_table.sql
vendored
Normal file
30
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/014_add_client_categories_table.sql
vendored
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Client Categories Schema
|
||||||
|
-- Version: 14.0
|
||||||
|
-- Adds category support for backup clients (File, Database, Virtual)
|
||||||
|
|
||||||
|
-- Client metadata table to store additional information about clients
|
||||||
|
-- This extends the Bacula Client table with Calypso-specific metadata
|
||||||
|
CREATE TABLE IF NOT EXISTS client_metadata (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
client_id INTEGER NOT NULL, -- Bacula Client.ClientId
|
||||||
|
client_name VARCHAR(255) NOT NULL, -- Bacula Client.Name (for reference)
|
||||||
|
category VARCHAR(50) NOT NULL DEFAULT 'File', -- 'File', 'Database', 'Virtual'
|
||||||
|
description TEXT,
|
||||||
|
tags JSONB, -- Additional tags/metadata as JSON
|
||||||
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||||
|
|
||||||
|
-- Ensure one metadata entry per client
|
||||||
|
CONSTRAINT unique_client_id UNIQUE (client_id),
|
||||||
|
CONSTRAINT chk_category CHECK (category IN ('File', 'Database', 'Virtual'))
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Indexes for performance
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_client_metadata_client_id ON client_metadata(client_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_client_metadata_client_name ON client_metadata(client_name);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_client_metadata_category ON client_metadata(category);
|
||||||
|
|
||||||
|
-- Add comment
|
||||||
|
COMMENT ON TABLE client_metadata IS 'Stores Calypso-specific metadata for backup clients, including category classification';
|
||||||
|
COMMENT ON COLUMN client_metadata.category IS 'Client category: File (file system backups), Database (database backups), Virtual (virtual machine backups)';
|
||||||
44
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/015_add_bacula_clients_table.sql
vendored
Normal file
44
dist/airgap/calypso-appliance-1.0.0-airgap/migrations/015_add_bacula_clients_table.sql
vendored
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
-- AtlasOS - Calypso
|
||||||
|
-- Migration 015: Add Bacula clients and capability history tables
|
||||||
|
--
|
||||||
|
-- Adds tables for tracking registered Bacula agents, their backup capabilities,
|
||||||
|
-- and a history log for UI- or agent-triggered capability changes. Pending
|
||||||
|
-- updates are stored on the client row until the agent pulls them.
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS bacula_clients (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
hostname TEXT NOT NULL,
|
||||||
|
ip_address TEXT,
|
||||||
|
agent_version TEXT,
|
||||||
|
status TEXT NOT NULL DEFAULT 'online',
|
||||||
|
backup_types JSONB NOT NULL,
|
||||||
|
pending_backup_types JSONB,
|
||||||
|
pending_requested_by UUID,
|
||||||
|
pending_requested_at TIMESTAMPTZ,
|
||||||
|
pending_notes TEXT,
|
||||||
|
metadata JSONB,
|
||||||
|
registered_by_user_id UUID NOT NULL,
|
||||||
|
last_seen TIMESTAMPTZ,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
CONSTRAINT uniq_bacula_clients_hostname UNIQUE (hostname)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_clients_registered_by ON bacula_clients (registered_by_user_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_clients_status ON bacula_clients (status);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS bacula_client_capability_history (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
client_id UUID NOT NULL REFERENCES bacula_clients (id) ON DELETE CASCADE,
|
||||||
|
backup_types JSONB NOT NULL,
|
||||||
|
source TEXT NOT NULL,
|
||||||
|
requested_by_user_id UUID,
|
||||||
|
requested_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
notes TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_client_capability_history_client ON bacula_client_capability_history (client_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_bacula_client_capability_history_requested_at ON bacula_client_capability_history (requested_at);
|
||||||
|
|
||||||
|
COMMENT ON TABLE bacula_clients IS 'Tracks Bacula clients registered with Calypso, including pending capability pushes.';
|
||||||
|
COMMENT ON TABLE bacula_client_capability_history IS 'Audit history of backup capability changes per client.';
|
||||||
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/Packages.gz
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/Packages.gz
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/adduser_3.137ubuntu1_all.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/adduser_3.137ubuntu1_all.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/apt-utils_2.8.3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/apt-utils_2.8.3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/apt_2.8.3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/apt_2.8.3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/attr_1%3a2.5.2-1build1.1_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/attr_1%3a2.5.2-1build1.1_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-bscan_13.0.4-1build3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-bscan_13.0.4-1build3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-client_13.0.4-1build3_all.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-client_13.0.4-1build3_all.deb
vendored
Normal file
Binary file not shown.
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-common_13.0.4-1build3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-common_13.0.4-1build3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-console_13.0.4-1build3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-console_13.0.4-1build3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-fd_13.0.4-1build3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-fd_13.0.4-1build3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-sd_13.0.4-1build3_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/bacula-sd_13.0.4-1build3_amd64.deb
vendored
Normal file
Binary file not shown.
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/binutils-common_2.42-4ubuntu2.8_amd64.deb
vendored
Normal file
BIN
dist/airgap/calypso-appliance-1.0.0-airgap/packages/debs/binutils-common_2.42-4ubuntu2.8_amd64.deb
vendored
Normal file
Binary file not shown.
Binary file not shown.
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user