logging and diagnostic features added

docs/API_SECURITY.md

@@ -0,0 +1,278 @@
+# API Security & Rate Limiting
+
+## Overview
+
+AtlasOS implements comprehensive API security measures including rate limiting, security headers, CORS protection, and request validation to protect the API from abuse and attacks.
+
+## Rate Limiting
+
+### Token Bucket Algorithm
+
+The rate limiter uses a token bucket algorithm:
+- **Default Rate**: 100 requests per minute per client
+- **Window**: 60 seconds
+- **Token Refill**: Tokens are refilled based on elapsed time
+- **Per-Client**: Rate limiting is applied per IP address or user ID
+
+### Rate Limit Headers
+
+All responses include rate limit headers:
+
+```
+X-RateLimit-Limit: 100
+X-RateLimit-Window: 60
+```
+
+### Rate Limit Exceeded
+
+When rate limit is exceeded, the API returns:
+
+```json
+{
+  "code": "SERVICE_UNAVAILABLE",
+  "message": "rate limit exceeded",
+  "details": "too many requests, please try again later"
+}
+```
+
+**HTTP Status**: `429 Too Many Requests`
+
+### Client Identification
+
+Rate limiting uses different keys based on authentication:
+
+- **Authenticated Users**: `user:{user_id}` - More granular per-user limiting
+- **Unauthenticated**: `ip:{ip_address}` - IP-based limiting
+
+### Public Endpoints
+
+Public endpoints (login, health checks) are excluded from rate limiting to ensure availability.
+
+## Security Headers
+
+All responses include security headers:
+
+### X-Content-Type-Options
+- **Value**: `nosniff`
+- **Purpose**: Prevents MIME type sniffing
+
+### X-Frame-Options
+- **Value**: `DENY`
+- **Purpose**: Prevents clickjacking attacks
+
+### X-XSS-Protection
+- **Value**: `1; mode=block`
+- **Purpose**: Enables XSS filtering in browsers
+
+### Referrer-Policy
+- **Value**: `strict-origin-when-cross-origin`
+- **Purpose**: Controls referrer information
+
+### Permissions-Policy
+- **Value**: `geolocation=(), microphone=(), camera=()`
+- **Purpose**: Disables unnecessary browser features
+
+### Strict-Transport-Security (HSTS)
+- **Value**: `max-age=31536000; includeSubDomains`
+- **Purpose**: Forces HTTPS connections (only on HTTPS)
+- **Note**: Only added when request is over TLS
+
+### Content-Security-Policy (CSP)
+- **Value**: `default-src 'self'; script-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net; style-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net; img-src 'self' data:; font-src 'self' https://cdn.jsdelivr.net; connect-src 'self';`
+- **Purpose**: Restricts resource loading to prevent XSS
+
+## CORS (Cross-Origin Resource Sharing)
+
+### Allowed Origins
+
+By default, the following origins are allowed:
+
+- `http://localhost:8080`
+- `http://localhost:3000`
+- `http://127.0.0.1:8080`
+- Same-origin requests (no Origin header)
+
+### CORS Headers
+
+When a request comes from an allowed origin:
+
+```
+Access-Control-Allow-Origin: http://localhost:8080
+Access-Control-Allow-Methods: GET, POST, PUT, DELETE, PATCH, OPTIONS
+Access-Control-Allow-Headers: Content-Type, Authorization, X-Requested-With
+Access-Control-Allow-Credentials: true
+Access-Control-Max-Age: 3600
+```
+
+### Preflight Requests
+
+OPTIONS requests are handled automatically:
+
+- **Status**: `204 No Content`
+- **Headers**: All CORS headers included
+- **Purpose**: Browser preflight checks
+
+## Request Size Limits
+
+### Maximum Request Body Size
+
+- **Limit**: 10 MB (10,485,760 bytes)
+- **Enforcement**: Automatic via `http.MaxBytesReader`
+- **Error**: Returns `413 Request Entity Too Large` if exceeded
+
+### Content-Type Validation
+
+POST, PUT, and PATCH requests must include a valid `Content-Type` header:
+
+**Allowed Types:**
+- `application/json`
+- `application/x-www-form-urlencoded`
+- `multipart/form-data`
+
+**Error Response:**
+```json
+{
+  "code": "BAD_REQUEST",
+  "message": "Content-Type must be application/json"
+}
+```
+
+## Middleware Chain Order
+
+Security middleware is applied in the following order (outer to inner):
+
+1. **CORS** - Handles preflight requests
+2. **Security Headers** - Adds security headers
+3. **Request Size Limit** - Enforces 10MB limit
+4. **Content-Type Validation** - Validates request content type
+5. **Rate Limiting** - Enforces rate limits
+6. **Error Recovery** - Catches panics
+7. **Request ID** - Generates request IDs
+8. **Logging** - Logs requests
+9. **Audit** - Records audit logs
+10. **Authentication** - Validates JWT tokens
+11. **Routes** - Handles requests
+
+## Public Endpoints
+
+The following endpoints are excluded from certain security checks:
+
+- `/api/v1/auth/login` - Rate limiting, Content-Type validation
+- `/api/v1/auth/logout` - Rate limiting, Content-Type validation
+- `/healthz` - Rate limiting, Content-Type validation
+- `/metrics` - Rate limiting, Content-Type validation
+- `/api/docs` - Rate limiting, Content-Type validation
+- `/api/openapi.yaml` - Rate limiting, Content-Type validation
+
+## Best Practices
+
+### For API Consumers
+
+1. **Respect Rate Limits**: Implement exponential backoff when rate limited
+2. **Use Authentication**: Authenticated users get better rate limits
+3. **Include Content-Type**: Always include `Content-Type: application/json`
+4. **Handle Errors**: Check for `429` status and retry after delay
+5. **Request Size**: Keep request bodies under 10MB
+
+### For Administrators
+
+1. **Monitor Rate Limits**: Check logs for rate limit violations
+2. **Adjust Limits**: Modify rate limit values in code if needed
+3. **CORS Configuration**: Update allowed origins for production
+4. **HTTPS**: Always use HTTPS in production for HSTS
+5. **Security Headers**: Review CSP policy for your use case
+
+## Configuration
+
+### Rate Limiting
+
+Rate limits are currently hardcoded but can be configured:
+
+```go
+// In rate_limit.go
+rateLimiter := NewRateLimiter(100, time.Minute) // 100 req/min
+```
+
+### CORS Origins
+
+Update allowed origins in `security_middleware.go`:
+
+```go
+allowedOrigins := []string{
+    "https://yourdomain.com",
+    "https://app.yourdomain.com",
+}
+```
+
+### Request Size Limit
+
+Modify in `app.go`:
+
+```go
+a.requestSizeMiddleware(10*1024*1024) // 10MB
+```
+
+## Error Responses
+
+### Rate Limit Exceeded
+
+```json
+{
+  "code": "SERVICE_UNAVAILABLE",
+  "message": "rate limit exceeded",
+  "details": "too many requests, please try again later"
+}
+```
+
+**Status**: `429 Too Many Requests`
+
+### Request Too Large
+
+```json
+{
+  "code": "BAD_REQUEST",
+  "message": "request body too large"
+}
+```
+
+**Status**: `413 Request Entity Too Large`
+
+### Invalid Content-Type
+
+```json
+{
+  "code": "BAD_REQUEST",
+  "message": "Content-Type must be application/json"
+}
+```
+
+**Status**: `400 Bad Request`
+
+## Monitoring
+
+### Rate Limit Metrics
+
+Monitor rate limit violations:
+
+- Check audit logs for rate limit events
+- Monitor `429` status codes in access logs
+- Track rate limit headers in responses
+
+### Security Events
+
+Monitor for security-related events:
+
+- Invalid Content-Type headers
+- Request size violations
+- CORS violations (check server logs)
+- Authentication failures
+
+## Future Enhancements
+
+1. **Configurable Rate Limits**: Environment variable configuration
+2. **Per-Endpoint Limits**: Different limits for different endpoints
+3. **IP Whitelisting**: Bypass rate limits for trusted IPs
+4. **Rate Limit Metrics**: Prometheus metrics for rate limiting
+5. **Distributed Rate Limiting**: Redis-based for multi-instance deployments
+6. **Advanced CORS**: Configurable CORS via environment variables
+7. **Request Timeout**: Configurable request timeout limits