diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..31ff5c6 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,137 @@ +# AGENTS + +## Overview +- Languages: Go backend with `github.com/gin-gonic/gin`, `zap`, PostgreSQL migrations and REST handlers under `backend/internal`, plus a Vite+React+TypeScript frontend under `frontend/`. +- Keep the system deterministic: Go 1.22+, PostgreSQL 14+, Node 18+, npm 10+ (use `node --version`/`npm --version` to confirm). +- Secret material lives in `/etc/calypso/config.yaml` or environment variables (never commit private keys, JWT secrets, or database passwords). +- The backend expects `CALYPSO_DB_PASSWORD` and `CALYPSO_JWT_SECRET` (minimum 32 characters) before running locally or in CI. + +## Environment Setup +- Run `sudo ./scripts/install-requirements.sh` from the repo root to provision system dependencies on Ubuntu-like hosts before building the backend. +- `config.yaml.example` provides defaults; copy it to `/etc/calypso/config.yaml`, then edit the file or override with the `CALYPSO_*` environment variables before launching. +- For frontend work, `npm install` from `frontend/` once before subsequent builds or `npm run dev` sessions. + +## Backend Tooling (Go) +### Build & Run +- `make build` (`go build -o bin/calypso-api ./cmd/calypso-api`) produces the local binary. +- `make run` (`go run ./cmd/calypso-api -config config.yaml.example`) is the quickest local dev server; supply `-config` pointing at your working config. +- Production cross-build: `make build-linux` sets `CGO_ENABLED=0 GOOS=linux GOARCH=amd64` and strips symbols. +- `deploy/systemd/calypso-api.service` is a template—copy it to `/etc/systemd/system/`, `daemon-reload`, `enable`, and `start` to run as service. + +### Testing +- `make test` runs `go test ./...` across all packages; it is the authoritative test runner today. +- To run a single test, narrow the package and pass `-run`: `cd backend && go test ./internal/auth -run '^TestHandler_Login$'` (use `.` to target the current package and regular expressions for test names). +- `make test-coverage` generates `coverage.out` and opens a coverage report via `go tool cover -html=coverage.out`. + +### Lint & Format +- `make fmt` (`go fmt ./...`) keeps source gofmt-clean; run it after edits and before committing. +- `make lint` requires `golangci-lint`; install it with `make install-deps` (which calls `go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest`). +- `make deps` handles dependency downloads and tidies `go.mod`/`go.sum` when vendor changes are necessary. + +## Backend Style Guidelines (Go) +### Imports +- Group imports with standard library packages first, blank line, then third-party packages (gofmt enforces this). +- Do not alias imports unless needed to disambiguate. Keep third-party import paths explicit (e.g., `github.com/gin-gonic/gin`). +- Prefer explicit package names (`config`, `database`, `logger`) rather than dot-importing. + +### Formatting & Files +- Always run `gofmt` (or `make fmt`) before committing; gofmt enforces tabs for indentation and consistent formatting. +- Keep files small (100–300 lines) and break responsibilities into `internal/common`, `internal/iam`, etc., mirroring the current layout. +- Maintain doc comments on exported functions, types, and struct fields; these comments start with the identifier they describe. + +### Types & Naming +- Use PascalCase (UpperCamel) for exported types/fields/methods and camelCase for internal helpers. +- Keep configuration structs descriptive (`ServerConfig`, `DatabaseConfig`) and align YAML tags with snake_case field names. +- When introducing new structs, mimic the existing tag conventions (`yaml:"field_name"`). +- Use short, meaningful receiver names (`func (h *Handler) ...`), ideally 1–3 characters, matching current files. +- Name boolean helpers `isX`/`shouldY` and prefer `cfg`/`db`/`ctx` as short local variables for configuration, database, and contexts. + +### Error Handling +- Wrap errors with `fmt.Errorf("context: %w", err)` and return early. Do not swallow errors unless there is a clear action (e.g., logging + fallback). +- Use `gin.Context` to respond with the appropriate HTTP status codes (`c.JSON(http.StatusBadRequest, gin.H{...})`). +- Log warnings vs. errors via the shared `logger.Logger` (`h.logger.Warn(...)` for expected issues, `Error` for failures, `Info` for happy paths). +- Avoid `panic` in handlers; only panic during initialization (e.g., new logger). Instead, log and return a 5xx status. +- Keep timeouts and cancellation explicit using `context.WithTimeout` when communicating with databases or external systems. + +### Logging & Observability +- Use `zap`-backed `logger.NewLogger`, pass service-specific names (e.g., `router`, `auth-handler`), and attach structured pairs (`"user_id", user.ID`). +- Allow log format and level to be configured via `CALYPSO_LOG_FORMAT`/`CALYPSO_LOG_LEVEL` if present. +- Prefer adding caller info and stack traces on `Error`level logs when helpful. Do not sprinkle raw `fmt.Println` or `log.Print`. + +### HTTP & Routing +- All HTTP routing lives under `internal/common/router`; register handlers through route groups and middleware (auth/authZ) as seen in `router.NewRouter`. +- Use Gin middleware for sessions, JWT validation, and auditing. Keep handler methods grouped by feature (auth, iam, tasks). +- Keep handler structs simple: inject shared dependencies (`db`, `config`, `logger`) via constructors, then refer to them as fields. +- For JSON requests/responses, mirror struct field names in `json` tags (snake_case) even if Go fields are camelCase. + +### Database & Migrations +- Use prepared statements or parameterized queries (see `iam` and `sessions` tables) with `$1` style placeholders. +- Run migrations automatically from `internal/common/database/migrations/` via `database.RunMigrations` at startup. +- Hunters: use `database.DB` wrapper for connection pool handling, close connections via `defer db.Close()`. +- When adding new tables, drop SQL files into `db/migrations/` and re-run migrations via the running service or a migration helper. + +### Security Notes +- JWT secrets must be strong (>=32 characters). Use environment variables, never embed secrets in code or checked-in configs. +- Database passwords, API keys, and secrets are populated via env vars (`CALYPSO_DB_PASSWORD`, `CALYPSO_JWT_SECRET`, `CALYPSO_JWT_SECRET`) or `config.yaml` on the host. +- All mutating endpoints should log audit events and guard via IAM roles (see `iam` package for role resolution). +- Bacula client registration and capability pushes require the `bacula-admin` role; enforce it via `requireRole("bacula-admin")` on `/api/v1/bacula/clients` routes. + +## Frontend Tooling (React + TypeScript) +### Build & Run +- Work inside `frontend/`. Start the dev server with `npm run dev` (proxies `/api` → `http://localhost:8080`). +- Production builds use `npm run build`, which runs `tsc` before `vite build` to ensure typing correctness. +- Serve production output via `npm run preview` if you need to smoke-test a `dist/` snapshot. + +### Testing +- No automated frontend tests run currently. If you add a test harness (Jest/Vitest), ensure `npm test` or equivalent is added to `package.json`. +- For manual verification, interact with the dev server after logging into the backend API from `localhost:3000`. + +### Lint & Formatting +- `npm run lint` runs ESLint across `.ts/.tsx` files with the `@typescript-eslint` recommended rules and `react-hooks` enforcement. It also blocks unused directives and sets `--max-warnings 0`. +- The linter requires you to keep hooks exhaustive and avoid unused vars; prefix intentionally unused arguments with `_` (e.g., `_event`). +- `tsconfig.json` sets `strict` mode, `noUnusedLocals`, `noUnusedParameters`, and `noFallthroughCasesInSwitch`. Respect these by keeping types precise and handling every branch. + +## Frontend Style Guidelines (TypeScript + React) +### Imports & Aliases +- Use the `@/` alias defined in `tsconfig.json` for slices inside `src/` (e.g., `import Layout from '@/components/Layout'`). +- Import React/third-party packages at the top, followed by `@/` aliased modules, maintaining alphabetical order within each group. +- Prefer named exports for hooks and API utilities, default exports for page components (e.g., `export default function LoginPage()`). + +### Formatting & Syntax +- Use single quotes for strings, omit semicolons (the repo follows Prettier/Vite defaults), and keep 2-space indentation inside JSX. +- Keep JSX clean: wrap complex class combinations and conditional fragments in template literals or helper functions, avoid inline object literal props unless necessary. +- Keep React components in kebab-case file names (e.g., `Login.tsx`, `Layout.tsx`) and align component names with file names. + +### Types & Naming +- Use `interface` or `type` aliases for props and API shapes (e.g., `interface LoginRequest`), and add optional fields with `?`. +- Names for state setters, handlers, and store selectors follow camelCase (`setAuth`, `handleSubmit`, `loginMutation`). +- Use descriptive names for UI data (`navigation`, `userMenu`, `alerts`). Keep boolean flags prefixed with `is`, `has`, or `should`. + +### State, Hooks, & Effects +- Prefer `useState`/`useEffect` for local UI state and `Zustand` stores (`useAuthStore`) for shared state. +- Keep `useEffect` dependency arrays explicit; avoid disabling exhaustive-deps rules unless documented. +- Manage async server interactions with TanStack Query (`useMutation`, `useQuery`) and centralize API clients (`src/api/client.ts`). + +### Styling & Layout +- TailwindCSS classes are used for layout (`flex`, `gap`, `bg-...`). Keep className strings readable and group related classes together (spacing, color, typography). +- Use utility classes for responsive behavior (`lg:hidden`, `px-4`). For dynamic class toggling, compute the string in JS before passing it to `className`. +- Keep inline styles minimal; prefer CSS utilities and shared constants (colors, text sizing) defined in tailwind config. + +### API & Networking +- Use `src/api/client.ts` as the single Axios instance (`baseURL: '/api/v1'`). Attach request/response interceptors for auth token injection/401 handling. +- Return typed promises from API helpers (e.g., `Promise`). Let callers handle success/errors via TanStack Query callbacks. +- Use `authApi.getMe`, `authApi.login`, etc., for all backend interactions; do not re-implement Axios calls directly in UI components. + +### Error Handling +- Surface backend errors by reading `error.response?.data?.error` (as seen in `LoginPage`) and displaying friendly messages via inline UI alerts or toast components. +- Clear authentication state on 401 responses using the Axios interceptor (`useAuthStore.getState().clearAuth()`), then redirect to `/login`. +- Display loading or pending states (`loginMutation.isPending`) instead of disabling UI abruptly; provide visual feedback for async actions. + +## Repository Practices +- Keep database migration files under `db/migrations/` and refer to them from `internal/common/database/migrations/` for auto-run. +- When adding commands or packages, update the relevant `Makefile` target or `package.json` script, and document the command in this AGENTS file. +- Commit `go.mod`, `go.sum`, and `package-lock.json`/`package.json` updates together with code changes. +- Avoid committing binaries (`bin/`), coverage artifacts (`coverage.out`), or built frontend output (`frontend/dist/`). These are ignored by `.gitignore` but double-check before commits. + +## Cursor & Copilot Rules +- There are no `.cursor/rules/`, `.cursorrules`, or `.github/copilot-instructions.md` files in this repository. Follow the guidelines laid out above. diff --git a/backend/internal/bacula/handler.go b/backend/internal/bacula/handler.go new file mode 100644 index 0000000..e36089a --- /dev/null +++ b/backend/internal/bacula/handler.go @@ -0,0 +1,676 @@ +package bacula + +import ( + "context" + "database/sql" + "encoding/json" + "net/http" + "time" + + "github.com/atlasos/calypso/internal/common/database" + "github.com/atlasos/calypso/internal/common/logger" + "github.com/atlasos/calypso/internal/iam" + "github.com/gin-gonic/gin" + "github.com/lib/pq" + "go.uber.org/zap" +) + +const ( + requestTimeout = 5 * time.Second + maxHistoryEntries = 10 +) + +type Handler struct { + db *database.DB + logger *logger.Logger +} + +func NewHandler(db *database.DB, log *logger.Logger) *Handler { + return &Handler{db: db, logger: log.WithFields(zap.String("component", "bacula-handler"))} +} + +type RegisterRequest struct { + Hostname string `json:"hostname" binding:"required"` + IPAddress string `json:"ip_address" binding:"required"` + AgentVersion string `json:"agent_version"` + Status string `json:"status"` + BackupTypes []string `json:"backup_types" binding:"required"` + Metadata map[string]string `json:"metadata"` +} + +type UpdateCapabilitiesRequest struct { + BackupTypes []string `json:"backup_types" binding:"required"` + Notes string `json:"notes"` +} + +type PingRequest struct { + Status string `json:"status"` +} + +type ClientResponse struct { + ID string `json:"id"` + Hostname string `json:"hostname"` + IPAddress string `json:"ip_address"` + AgentVersion string `json:"agent_version"` + Status string `json:"status"` + BackupTypes []string `json:"backup_types"` + PendingBackupTypes []string `json:"pending_backup_types,omitempty"` + PendingRequestedBy string `json:"pending_requested_by,omitempty"` + PendingRequestedAt *time.Time `json:"pending_requested_at,omitempty"` + PendingNotes string `json:"pending_notes,omitempty"` + Metadata map[string]interface{} `json:"metadata,omitempty"` + RegisteredBy string `json:"registered_by"` + LastSeen *time.Time `json:"last_seen,omitempty"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` + CapabilityHistory []CapabilityHistoryEntry `json:"capability_history,omitempty"` +} + +type CapabilityHistoryEntry struct { + BackupTypes []string `json:"backup_types"` + Source string `json:"source"` + RequestedBy string `json:"requested_by,omitempty"` + RequestedAt time.Time `json:"requested_at"` + Notes string `json:"notes,omitempty"` +} + +type PendingUpdateResponse struct { + BackupTypes []string `json:"backup_types"` + RequestedBy string `json:"requested_by,omitempty"` + RequestedAt time.Time `json:"requested_at"` + Notes string `json:"notes,omitempty"` +} + +func (h *Handler) Register(c *gin.Context) { + var req RegisterRequest + if err := c.ShouldBindJSON(&req); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request payload"}) + return + } + + if len(req.BackupTypes) == 0 { + c.JSON(http.StatusBadRequest, gin.H{"error": "backup_types is required"}) + return + } + + user, err := currentUser(c) + if err != nil { + c.JSON(http.StatusUnauthorized, gin.H{"error": "authentication required"}) + return + } + + ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout) + defer cancel() + + backupPayload, err := json.Marshal(req.BackupTypes) + if err != nil { + h.logger.Error("failed to marshal backup types", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to encode backup types"}) + return + } + + var metadataPayload []byte + if len(req.Metadata) > 0 { + metadataPayload, err = json.Marshal(req.Metadata) + if err != nil { + h.logger.Error("failed to marshal metadata", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to encode metadata"}) + return + } + } + + status := req.Status + if status == "" { + status = "online" + } + + tx, err := h.db.BeginTx(ctx, nil) + if err != nil { + h.logger.Error("failed to begin database transaction", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to register client"}) + return + } + defer tx.Rollback() + + var row clientRow + err = tx.QueryRowContext(ctx, ` + INSERT INTO bacula_clients ( + hostname, ip_address, agent_version, status, backup_types, metadata, + registered_by_user_id, last_seen, updated_at + ) VALUES ($1, $2, $3, $4, $5, $6, $7, NOW(), NOW()) + ON CONFLICT (hostname) DO UPDATE SET + ip_address = EXCLUDED.ip_address, + agent_version = EXCLUDED.agent_version, + status = EXCLUDED.status, + backup_types = EXCLUDED.backup_types, + metadata = COALESCE(EXCLUDED.metadata, bacula_clients.metadata), + registered_by_user_id = EXCLUDED.registered_by_user_id, + last_seen = EXCLUDED.last_seen, + updated_at = NOW() + RETURNING id, hostname, ip_address, agent_version, status, backup_types, metadata, + pending_backup_types, pending_requested_by, pending_requested_at, pending_notes, + registered_by_user_id, last_seen, created_at, updated_at + `, req.Hostname, req.IPAddress, req.AgentVersion, status, backupPayload, metadataPayload, user.ID).Scan( + &row.ID, &row.Hostname, &row.IPAddress, &row.AgentVersion, &row.Status, &row.BackupJSON, + &row.MetadataJSON, &row.PendingBackupJSON, &row.PendingRequestedBy, &row.PendingRequestedAt, + &row.PendingNotes, &row.RegisteredBy, &row.LastSeen, &row.CreatedAt, &row.UpdatedAt, + ) + if err != nil { + h.logger.Error("failed to ensure bacula client", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to register client"}) + return + } + + resp, err := buildClientResponse(&row) + if err != nil { + h.logger.Error("failed to marshal client response", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to build response"}) + return + } + + if err := insertCapabilityHistory(ctx, tx, row.ID, req.BackupTypes, "agent", user.ID, "agent registration"); err != nil { + h.logger.Error("failed to record capability history", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to record capability history"}) + return + } + + if len(resp.PendingBackupTypes) > 0 && stringSlicesEqual(resp.PendingBackupTypes, resp.BackupTypes) { + if _, err := tx.ExecContext(ctx, ` + UPDATE bacula_clients + SET pending_backup_types = NULL, + pending_requested_by = NULL, + pending_requested_at = NULL, + pending_notes = NULL, + updated_at = NOW() + WHERE id = $1 + `, resp.ID); err != nil { + h.logger.Error("failed to clear pending capability update", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"}) + return + } + resp.PendingBackupTypes = nil + resp.PendingRequestedBy = "" + resp.PendingRequestedAt = nil + resp.PendingNotes = "" + } + + if err := tx.Commit(); err != nil { + h.logger.Error("failed to commit client registration", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to register client"}) + return + } + + c.JSON(http.StatusOK, resp) +} + +func (h *Handler) UpdateCapabilities(c *gin.Context) { + var req UpdateCapabilitiesRequest + if err := c.ShouldBindJSON(&req); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request payload"}) + return + } + + if len(req.BackupTypes) == 0 { + c.JSON(http.StatusBadRequest, gin.H{"error": "backup_types is required"}) + return + } + + user, err := currentUser(c) + if err != nil { + c.JSON(http.StatusUnauthorized, gin.H{"error": "authentication required"}) + return + } + + clientID := c.Param("id") + if clientID == "" { + c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"}) + return + } + + ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout) + defer cancel() + + tx, err := h.db.BeginTx(ctx, nil) + if err != nil { + h.logger.Error("failed to begin transaction", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "unable to update capabilities"}) + return + } + defer tx.Rollback() + + var exists bool + if err := tx.QueryRowContext(ctx, `SELECT EXISTS (SELECT 1 FROM bacula_clients WHERE id = $1)`, clientID).Scan(&exists); err != nil { + h.logger.Error("failed to verify client", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "unable to update client"}) + return + } + if !exists { + c.JSON(http.StatusNotFound, gin.H{"error": "client not found"}) + return + } + + backupPayload, err := json.Marshal(req.BackupTypes) + if err != nil { + h.logger.Error("failed to marshal backup types", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to encode backup types"}) + return + } + + if _, err := tx.ExecContext(ctx, ` + UPDATE bacula_clients + SET pending_backup_types = $1, + pending_requested_by = $2, + pending_requested_at = NOW(), + pending_notes = $3, + updated_at = NOW() + WHERE id = $4 + `, backupPayload, user.ID, req.Notes, clientID); err != nil { + h.logger.Error("failed to mark pending update", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"}) + return + } + + if err := insertCapabilityHistory(ctx, tx, clientID, req.BackupTypes, "ui", user.ID, req.Notes); err != nil { + h.logger.Error("failed to insert capability history", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to record capability change"}) + return + } + + if err := tx.Commit(); err != nil { + h.logger.Error("failed to commit capability update", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"}) + return + } + + c.JSON(http.StatusOK, PendingUpdateResponse{ + BackupTypes: req.BackupTypes, + RequestedBy: user.ID, + RequestedAt: time.Now(), + Notes: req.Notes, + }) +} + +func (h *Handler) GetPendingUpdate(c *gin.Context) { + clientID := c.Param("id") + if clientID == "" { + c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"}) + return + } + + ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout) + defer cancel() + + var pendingJSON []byte + var requestedBy sql.NullString + var requestedAt sql.NullTime + var notes sql.NullString + + err := h.db.QueryRowContext(ctx, ` + SELECT pending_backup_types, pending_requested_by, pending_requested_at, pending_notes + FROM bacula_clients + WHERE id = $1 + `, clientID).Scan(&pendingJSON, &requestedBy, &requestedAt, ¬es) + if err != nil { + if err == sql.ErrNoRows { + c.JSON(http.StatusNotFound, gin.H{"error": "client not found"}) + return + } + h.logger.Error("failed to fetch pending update", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to read pending update"}) + return + } + + if len(pendingJSON) == 0 { + c.Status(http.StatusNoContent) + return + } + + var backupTypes []string + if err := json.Unmarshal(pendingJSON, &backupTypes); err != nil { + h.logger.Error("failed to unmarshal pending backup types", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to read pending update"}) + return + } + + response := PendingUpdateResponse{ + BackupTypes: backupTypes, + Notes: notes.String, + } + if requestedBy.Valid { + response.RequestedBy = requestedBy.String + } + if requestedAt.Valid { + response.RequestedAt = requestedAt.Time + } + + c.JSON(http.StatusOK, response) +} + +func (h *Handler) Ping(c *gin.Context) { + clientID := c.Param("id") + if clientID == "" { + c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"}) + return + } + + var req PingRequest + if err := c.ShouldBindJSON(&req); err != nil { + // swallow body if absent + } + + ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout) + defer cancel() + + query := ` + UPDATE bacula_clients + SET last_seen = NOW(), + status = COALESCE(NULLIF($2, ''), status), + updated_at = NOW() + WHERE id = $1 + RETURNING id + ` + + var id string + err := h.db.QueryRowContext(ctx, query, clientID, req.Status).Scan(&id) + if err != nil { + if err == sql.ErrNoRows { + c.JSON(http.StatusNotFound, gin.H{"error": "client not found"}) + return + } + h.logger.Error("failed to update heartbeat", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update client"}) + return + } + + c.Status(http.StatusNoContent) +} + +func (h *Handler) ListClients(c *gin.Context) { + ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout) + defer cancel() + + rows, err := h.db.QueryContext(ctx, ` + SELECT id, hostname, ip_address, agent_version, status, backup_types, metadata, + pending_backup_types, pending_requested_by, pending_requested_at, pending_notes, + registered_by_user_id, last_seen, created_at, updated_at + FROM bacula_clients + ORDER BY created_at DESC + `) + if err != nil { + h.logger.Error("failed to query clients", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch clients"}) + return + } + defer rows.Close() + + var clients []*ClientResponse + var ids []string + for rows.Next() { + row := clientRow{} + if err := rows.Scan(&row.ID, &row.Hostname, &row.IPAddress, &row.AgentVersion, &row.Status, + &row.BackupJSON, &row.MetadataJSON, &row.PendingBackupJSON, &row.PendingRequestedBy, + &row.PendingRequestedAt, &row.PendingNotes, &row.RegisteredBy, &row.LastSeen, + &row.CreatedAt, &row.UpdatedAt); err != nil { + h.logger.Error("failed to scan client row", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch clients"}) + return + } + resp, err := buildClientResponse(&row) + if err != nil { + h.logger.Error("failed to build client response", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch clients"}) + return + } + clients = append(clients, resp) + ids = append(ids, resp.ID) + } + + if len(ids) > 0 { + if err := h.attachHistory(ctx, ids, clients); err != nil { + h.logger.Error("failed to attach history", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client history"}) + return + } + } + + c.JSON(http.StatusOK, clients) +} + +func (h *Handler) GetClient(c *gin.Context) { + clientID := c.Param("id") + if clientID == "" { + c.JSON(http.StatusBadRequest, gin.H{"error": "client id is required"}) + return + } + + ctx, cancel := context.WithTimeout(c.Request.Context(), requestTimeout) + defer cancel() + + var row clientRow + err := h.db.QueryRowContext(ctx, ` + SELECT id, hostname, ip_address, agent_version, status, backup_types, metadata, + pending_backup_types, pending_requested_by, pending_requested_at, pending_notes, + registered_by_user_id, last_seen, created_at, updated_at + FROM bacula_clients + WHERE id = $1 + `, clientID).Scan(&row.ID, &row.Hostname, &row.IPAddress, &row.AgentVersion, &row.Status, + &row.BackupJSON, &row.MetadataJSON, &row.PendingBackupJSON, &row.PendingRequestedBy, + &row.PendingRequestedAt, &row.PendingNotes, &row.RegisteredBy, &row.LastSeen, + &row.CreatedAt, &row.UpdatedAt) + if err != nil { + if err == sql.ErrNoRows { + c.JSON(http.StatusNotFound, gin.H{"error": "client not found"}) + return + } + h.logger.Error("failed to read client", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client"}) + return + } + + resp, err := buildClientResponse(&row) + if err != nil { + h.logger.Error("failed to build client response", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client"}) + return + } + + if err := h.attachHistory(ctx, []string{resp.ID}, []*ClientResponse{resp}); err != nil { + h.logger.Error("failed to attach history", "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to fetch client history"}) + return + } + + c.JSON(http.StatusOK, resp) +} + +func (h *Handler) attachHistory(ctx context.Context, ids []string, clients []*ClientResponse) error { + if len(ids) == 0 { + return nil + } + + rows, err := h.db.QueryContext(ctx, ` + SELECT client_id, backup_types, source, requested_by_user_id, requested_at, notes + FROM bacula_client_capability_history + WHERE client_id = ANY($1) + ORDER BY requested_at DESC + `, pq.Array(ids)) + if err != nil { + return err + } + defer rows.Close() + + clientMap := make(map[string]*ClientResponse) + for _, client := range clients { + clientMap[client.ID] = client + } + + for rows.Next() { + var clientID string + var backupJSON []byte + var source string + var requestedBy sql.NullString + var requestedAt time.Time + var notes sql.NullString + + if err := rows.Scan(&clientID, &backupJSON, &source, &requestedBy, &requestedAt, ¬es); err != nil { + return err + } + + resp, ok := clientMap[clientID] + if !ok { + continue + } + + if len(resp.CapabilityHistory) >= maxHistoryEntries { + continue + } + + var backupTypes []string + if err := json.Unmarshal(backupJSON, &backupTypes); err != nil { + return err + } + + entry := CapabilityHistoryEntry{ + BackupTypes: backupTypes, + Source: source, + RequestedAt: requestedAt, + } + if requestedBy.Valid { + entry.RequestedBy = requestedBy.String + } + if notes.Valid { + entry.Notes = notes.String + } + + resp.CapabilityHistory = append(resp.CapabilityHistory, entry) + } + + return nil +} + +type clientRow struct { + ID string + Hostname string + IPAddress string + AgentVersion string + Status string + BackupJSON []byte + MetadataJSON []byte + PendingBackupJSON []byte + PendingRequestedBy sql.NullString + PendingRequestedAt sql.NullTime + PendingNotes sql.NullString + RegisteredBy string + LastSeen sql.NullTime + CreatedAt time.Time + UpdatedAt time.Time +} + +func buildClientResponse(row *clientRow) (*ClientResponse, error) { + backupTypes, err := decodeStringSlice(row.BackupJSON) + if err != nil { + return nil, err + } + + pendingTypes, err := decodeStringSlice(row.PendingBackupJSON) + if err != nil { + return nil, err + } + + metadata, err := decodeMetadata(row.MetadataJSON) + if err != nil { + return nil, err + } + + resp := &ClientResponse{ + ID: row.ID, + Hostname: row.Hostname, + IPAddress: row.IPAddress, + AgentVersion: row.AgentVersion, + Status: row.Status, + BackupTypes: backupTypes, + Metadata: metadata, + RegisteredBy: row.RegisteredBy, + CreatedAt: row.CreatedAt, + UpdatedAt: row.UpdatedAt, + } + + if len(pendingTypes) > 0 { + resp.PendingBackupTypes = pendingTypes + if row.PendingRequestedBy.Valid { + resp.PendingRequestedBy = row.PendingRequestedBy.String + } + if row.PendingRequestedAt.Valid { + resp.PendingRequestedAt = &row.PendingRequestedAt.Time + } + if row.PendingNotes.Valid { + resp.PendingNotes = row.PendingNotes.String + } + } + + if row.LastSeen.Valid { + resp.LastSeen = &row.LastSeen.Time + } + + return resp, nil +} + +func currentUser(c *gin.Context) (*iam.User, error) { + user, exists := c.Get("user") + if !exists { + return nil, sql.ErrNoRows + } + authUser, ok := user.(*iam.User) + if !ok { + return nil, sql.ErrNoRows + } + return authUser, nil +} + +func decodeStringSlice(data []byte) ([]string, error) { + if len(data) == 0 { + return []string{}, nil + } + var dst []string + if err := json.Unmarshal(data, &dst); err != nil { + return nil, err + } + return dst, nil +} + +func decodeMetadata(data []byte) (map[string]interface{}, error) { + if len(data) == 0 { + return nil, nil + } + var metadata map[string]interface{} + if err := json.Unmarshal(data, &metadata); err != nil { + return nil, err + } + return metadata, nil +} + +func insertCapabilityHistory(ctx context.Context, tx *sql.Tx, clientID string, backupTypes []string, source, requestedBy, notes string) error { + payload, err := json.Marshal(backupTypes) + if err != nil { + return err + } + + _, err = tx.ExecContext(ctx, ` + INSERT INTO bacula_client_capability_history ( + client_id, backup_types, source, requested_by_user_id, notes + ) VALUES ($1, $2, $3, $4, $5) + `, clientID, payload, source, requestedBy, notes) + return err +} + +func stringSlicesEqual(a, b []string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i] != b[i] { + return false + } + } + return true +} diff --git a/backend/internal/common/database/migrations/015_add_bacula_clients_table.sql b/backend/internal/common/database/migrations/015_add_bacula_clients_table.sql new file mode 100644 index 0000000..add67e0 --- /dev/null +++ b/backend/internal/common/database/migrations/015_add_bacula_clients_table.sql @@ -0,0 +1,44 @@ +-- AtlasOS - Calypso +-- Migration 015: Add Bacula clients and capability history tables +-- +-- Adds tables for tracking registered Bacula agents, their backup capabilities, +-- and a history log for UI- or agent-triggered capability changes. Pending +-- updates are stored on the client row until the agent pulls them. + +CREATE TABLE IF NOT EXISTS bacula_clients ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + hostname TEXT NOT NULL, + ip_address TEXT, + agent_version TEXT, + status TEXT NOT NULL DEFAULT 'online', + backup_types JSONB NOT NULL, + pending_backup_types JSONB, + pending_requested_by UUID, + pending_requested_at TIMESTAMPTZ, + pending_notes TEXT, + metadata JSONB, + registered_by_user_id UUID NOT NULL, + last_seen TIMESTAMPTZ, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + CONSTRAINT uniq_bacula_clients_hostname UNIQUE (hostname) +); + +CREATE INDEX IF NOT EXISTS idx_bacula_clients_registered_by ON bacula_clients (registered_by_user_id); +CREATE INDEX IF NOT EXISTS idx_bacula_clients_status ON bacula_clients (status); + +CREATE TABLE IF NOT EXISTS bacula_client_capability_history ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + client_id UUID NOT NULL REFERENCES bacula_clients (id) ON DELETE CASCADE, + backup_types JSONB NOT NULL, + source TEXT NOT NULL, + requested_by_user_id UUID, + requested_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + notes TEXT +); + +CREATE INDEX IF NOT EXISTS idx_bacula_client_capability_history_client ON bacula_client_capability_history (client_id); +CREATE INDEX IF NOT EXISTS idx_bacula_client_capability_history_requested_at ON bacula_client_capability_history (requested_at); + +COMMENT ON TABLE bacula_clients IS 'Tracks Bacula clients registered with Calypso, including pending capability pushes.'; +COMMENT ON TABLE bacula_client_capability_history IS 'Audit history of backup capability changes per client.'; diff --git a/backend/internal/common/router/router.go b/backend/internal/common/router/router.go index 0b8e769..0335cf3 100644 --- a/backend/internal/common/router/router.go +++ b/backend/internal/common/router/router.go @@ -7,6 +7,8 @@ import ( "github.com/atlasos/calypso/internal/audit" "github.com/atlasos/calypso/internal/auth" "github.com/atlasos/calypso/internal/backup" + "github.com/atlasos/calypso/internal/bacula" + "github.com/atlasos/calypso/internal/common/cache" "github.com/atlasos/calypso/internal/common/config" "github.com/atlasos/calypso/internal/common/database" @@ -457,6 +459,18 @@ func NewRouter(cfg *config.Config, db *database.DB, log *logger.Logger) *gin.Eng backupGroup.POST("/console/execute", requirePermission("backup", "write"), backupHandler.ExecuteBconsoleCommand) } + baculaHandler := bacula.NewHandler(db, log) + baculaGroup := protected.Group("/bacula/clients") + baculaGroup.Use(requireRole("bacula-admin")) + { + baculaGroup.POST("/register", baculaHandler.Register) + baculaGroup.POST("/:id/capabilities", baculaHandler.UpdateCapabilities) + baculaGroup.GET("/:id/pending-update", baculaHandler.GetPendingUpdate) + baculaGroup.POST("/:id/ping", baculaHandler.Ping) + baculaGroup.GET("", baculaHandler.ListClients) + baculaGroup.GET("/:id", baculaHandler.GetClient) + } + // Monitoring monitoringHandler := monitoring.NewHandler(db, log, alertService, metricsService, eventHub) monitoringGroup := protected.Group("/monitoring") diff --git a/docs/alpha/components/bacula/Bacula-Installation-Guide.md b/docs/alpha/components/bacula/Bacula-Installation-Guide.md index fa7dd6e..f3b1177 100644 --- a/docs/alpha/components/bacula/Bacula-Installation-Guide.md +++ b/docs/alpha/components/bacula/Bacula-Installation-Guide.md @@ -151,3 +151,42 @@ sudo systemctl status bacula-director bacula-sd bacula-fd ## 8. SELinux/AppArmor If you are using SELinux or AppArmor, you may need to adjust the security policies to allow Bacula to access the new configuration directory and storage directory. The specific steps will depend on your security policy. + +## 9. Calypso Bacula Agent Registration Flow + +The Calypso Director exposes a REST API that allows a lightweight Bacula agent to register itself, report its capability matrix, and receive UI-driven capability pushes. + +### 9.1. Authenticate with Calypso +1. Log in against `POST /api/v1/auth/login` with a Calypso user that has the `bacula-admin` role. The same JWT token is used for all subsequent agent calls. +2. Store the token (and refresh it whenever a 401 is returned) so the agent can re-authenticate automatically. + +### 9.2. Register the client +- Call `POST /api/v1/bacula/clients/register` with the following payload: + ```json + { + "hostname": "backup-client-01", + "ip_address": "10.0.0.15", + "agent_version": "1.0.0", + "backup_types": ["files", "database"], + "metadata": { + "platform": "ubuntu", + "location": "datacenter-1" + } + } + ``` +- Store the returned `id` (client identifier) so the agent can re-register subsequently. +- Include the `status` string (`online`, `maintenance`, etc.) as part of the payload so the Director knows the client state. + +### 9.3. Responding to UI-driven capability pushes +- The Calypso console exposes a `Bacula Clients` page (see the new navigation entry in the web UI) where operators can edit the backup types. +- When an operator submits a change, the UI calls `POST /api/v1/bacula/clients/{id}/capabilities`. This request is gated by the `bacula-admin` role and records an audit entry via the `bacula_client_capability_history` table. +- The agent should poll `GET /api/v1/bacula/clients/{id}/pending-update`. The endpoint never expires a pending list and returns the full desired `backup_types`, the operator notes, and who requested the change. Once the agent re-registers and matches the desired list, the pending update is cleared automatically. +- Each registration (agent push or UI trigger) appends a history row so the console can surface the last few changes for auditing. + +### 9.4. Heartbeats and status updates +- Use `POST /api/v1/bacula/clients/{id}/ping` to refresh the agent's `last_seen` timestamp and propagate `status` changes without altering the capability matrix. +- If the agent needs to push new metadata (e.g., a new `agent_version` or additional `backup_types`), simply call `POST /api/v1/bacula/clients/register` again with the updated values. The same endpoint handles both initial registration and re-registration, and it clears pending capability pushes once the requested list matches the agent's `backup_types`. + +### 9.5. Agent configuration location +- Store agent settings under `/opt/calypso/conf/bacula/agent.yaml` and keep the JWT token, Calypso API URL, and desired backup types there so the service can re-run after restarts. The installation script should add a systemd service that runs the agent and ensures it restarts on failure. + diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 74e553c..69a1799 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -12,6 +12,8 @@ import ISCSITargetsPage from '@/pages/ISCSITargets' import ISCSITargetDetailPage from '@/pages/ISCSITargetDetail' import SystemPage from '@/pages/System' import BackupManagementPage from '@/pages/BackupManagement' +import BaculaClientsPage from '@/pages/BaculaClients' + import TerminalConsolePage from '@/pages/TerminalConsole' import SharesPage from '@/pages/Shares' import IAMPage from '@/pages/IAM' @@ -65,7 +67,9 @@ function App() { } /> } /> } /> + } /> } /> + } /> } /> } /> diff --git a/frontend/src/api/bacula.ts b/frontend/src/api/bacula.ts new file mode 100644 index 0000000..120f9a8 --- /dev/null +++ b/frontend/src/api/bacula.ts @@ -0,0 +1,83 @@ +import apiClient from './client' + +export interface BaculaCapabilityHistory { + backup_types: string[] + source: string + requested_by?: string + requested_at: string + notes?: string +} + +export interface BaculaClient { + id: string + hostname: string + ip_address: string + agent_version: string + status: string + backup_types: string[] + pending_backup_types?: string[] + pending_requested_by?: string + pending_requested_at?: string + pending_notes?: string + metadata?: Record + registered_by: string + last_seen?: string + created_at: string + updated_at: string + capability_history?: BaculaCapabilityHistory[] +} + +export interface RegisterBaculaClientRequest { + hostname: string + ip_address: string + agent_version?: string + backup_types: string[] + status?: string + metadata?: Record +} + +export interface UpdateCapabilitiesRequest { + backup_types: string[] + notes?: string +} + +export interface PendingBaculaUpdate { + backup_types: string[] + requested_by?: string + requested_at: string + notes?: string +} + +export const baculaApi = { + listClients: async (): Promise => { + const response = await apiClient.get('/bacula/clients') + return response.data + }, + + getClient: async (id: string): Promise => { + const response = await apiClient.get(`/bacula/clients/${id}`) + return response.data + }, + + registerClient: async (payload: RegisterBaculaClientRequest): Promise => { + const response = await apiClient.post('/bacula/clients/register', payload) + return response.data + }, + + updateCapabilities: async (id: string, payload: UpdateCapabilitiesRequest): Promise => { + const response = await apiClient.post(`/bacula/clients/${id}/capabilities`, payload) + return response.data + }, + + getPendingUpdate: async (id: string): Promise => { + const response = await apiClient.get(`/bacula/clients/${id}/pending-update`) + if (response.status === 204) { + return null + } + return response.data + }, + + ping: async (id: string, status?: string): Promise => { + await apiClient.post(`/bacula/clients/${id}/ping`, { status }) + }, +} diff --git a/frontend/src/components/Layout.tsx b/frontend/src/components/Layout.tsx index 3c03f4d..10f7afe 100644 --- a/frontend/src/components/Layout.tsx +++ b/frontend/src/components/Layout.tsx @@ -16,8 +16,10 @@ import { Activity, Box, Camera, - Shield + Shield, + ShieldCheck } from 'lucide-react' + import { useState, useEffect } from 'react' export default function Layout() { @@ -55,7 +57,9 @@ export default function Layout() { { name: 'Tape Libraries', href: '/tape', icon: Database }, { name: 'iSCSI Management', href: '/iscsi', icon: Network }, { name: 'Backup Management', href: '/backup', icon: Archive }, + { name: 'Bacula Clients', href: '/bacula/clients', icon: ShieldCheck }, { name: 'Terminal Console', href: '/terminal', icon: Terminal }, + { name: 'Share Shield', href: '/share-shield', icon: Shield }, { name: 'Monitoring & Logs', href: '/monitoring', icon: Activity }, { name: 'Alerts', href: '/alerts', icon: Bell }, diff --git a/frontend/src/pages/BaculaClients.tsx b/frontend/src/pages/BaculaClients.tsx new file mode 100644 index 0000000..332d18c --- /dev/null +++ b/frontend/src/pages/BaculaClients.tsx @@ -0,0 +1,212 @@ +import { FormEvent, useEffect, useState } from 'react' +import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' +import { baculaApi } from '@/api/bacula' + +const backupTypeOptions = [ + { label: 'Files', value: 'files' }, + { label: 'Database', value: 'database' }, + { label: 'Application', value: 'application' }, + { label: 'Exchange', value: 'exchange' }, + { label: 'Mail', value: 'mail' }, +] + +function formatTimestamp(value?: string) { + if (!value) { + return 'Never' + } + const date = new Date(value) + return date.toLocaleString() +} + +export default function BaculaClientsPage() { + const queryClient = useQueryClient() + const { data: clients, isLoading, error } = useQuery(['bacula-clients'], baculaApi.listClients) + const [editingClientId, setEditingClientId] = useState(null) + const [selectedTypes, setSelectedTypes] = useState([]) + const [notes, setNotes] = useState('') + const [statusMessage, setStatusMessage] = useState('') + + const mutation = useMutation( + ({ id, backupTypes, notes }: { id: string; backupTypes: string[]; notes: string }) => + baculaApi.updateCapabilities(id, { backup_types: backupTypes, notes }), + { + onSuccess: () => { + queryClient.invalidateQueries(['bacula-clients']) + setStatusMessage('Capability update requested. The agent will pull changes shortly.') + }, + onError: () => { + setStatusMessage('Failed to request capability update. Please try again.') + }, + } + ) + + const editingClient = clients?.find((client) => client.id === editingClientId) ?? null + + useEffect(() => { + if (!editingClient) { + setSelectedTypes([]) + setNotes('') + return + } + + setSelectedTypes( + editingClient.pending_backup_types?.length + ? editingClient.pending_backup_types + : editingClient.backup_types + ) + setNotes(editingClient.pending_notes ?? '') + }, [editingClient]) + + const toggleType = (type: string) => { + setSelectedTypes((prev) => + prev.includes(type) ? prev.filter((value) => value !== type) : [...prev, type] + ) + } + + const handleSubmit = (event: FormEvent) => { + event.preventDefault() + if (!editingClient) { + return + } + + mutation.mutate({ id: editingClient.id, backupTypes: selectedTypes, notes }) + } + + if (isLoading) { + return

Loading Bacula clients...

+ } + + if (error) { + return

Failed to load Bacula clients.

+ } + + return ( +
+
+

Bacula Client Management

+

Register agents, monitor their capabilities, and push updates.

+
+ +
+
+
+

Edit capabilities

+

+ {editingClient ? editingClient.hostname : 'Select a client from the list'} +

+
+ {statusMessage &&

{statusMessage}

} +
+ +
+
+ Backup types +
+ {backupTypeOptions.map((option) => ( + + ))} +
+
+ +
+ +