# Server Advanced Features Declarative hooks, agent factory, idle shutdown management, and the RAG pipeline (indexing, chunking, retrieval). ## Declarative Hook System The `com.tnsai.server.hooks` package provides a YAML-driven hook system that attaches lifecycle callbacks to agents without writing Java code. ### Architecture Overview ```text hooks.yaml --> DeclarativeHookRegistry --> ToolCallListener on Agent | | | PRE_TOOL_USE / POST_TOOL_USE | DeclarativeHookExecutor | COMMAND (shell) / LOG (message) ``` ### HookEvent Lifecycle events that hooks can listen for: | Event | Description | |-------|-------------| | `PRE_TOOL_USE` | Before a tool executes | | `POST_TOOL_USE` | After a tool executes | | `ON_START` | When the agent starts | | `ON_STOP` | When the agent stops | | `ON_ERROR` | When an error occurs | Parsing supports both camelCase (`preToolUse`) and UPPER_SNAKE (`PRE_TOOL_USE`) from YAML. ### HookActionType | Type | Description | |------|-------------| | `COMMAND` | Execute a shell command | | `LOG` | Log a message | ### HookAction Record defining what to execute when a hook fires. ```java // Factory methods HookAction cmd = HookAction.command("echo 'tool finished'"); HookAction log = HookAction.log("Tool ${tool} is about to execute"); ``` Fields: `type` (HookActionType), `command` (for COMMAND), `message` (for LOG). Supports `${variable}` placeholders that are substituted at execution time. ### HookMatcher Matches tool names against patterns. Case-insensitive using `Locale.ROOT`. ```java // Exact match HookMatcher exact = new HookMatcher("shell"); exact.matches("shell"); // true exact.matches("Shell"); // true (case-insensitive) // Glob wildcard HookMatcher glob = new HookMatcher("file_*"); glob.matches("file_read"); // true glob.matches("file_write"); // true glob.matches("shell"); // false // Match all tools HookMatcher all = HookMatcher.ALL; // pattern "*" ``` ### HookConfig A single hook definition, typically parsed from YAML. ```java // Record: HookConfig(event, matcher, action) HookConfig config = new HookConfig( HookEvent.PRE_TOOL_USE, new HookMatcher("file_*"), HookAction.log("Tool ${tool} is about to execute") ); ``` ### HookExecutor Interface ```java @FunctionalInterface public interface HookExecutor { void execute(HookAction action, Map context); } ``` ### DeclarativeHookExecutor Default executor that runs shell commands (with 30-second timeout) and logs messages. All execution is best-effort: failures are logged but never propagate. Variable substitution replaces `${key}` placeholders with values from the context map. ### DeclarativeHookRegistry Parses YAML hook definitions and registers them as runtime listeners on agents. **YAML schema:** ```yaml hooks: - event: preToolUse matcher: "file_*" action: type: log message: "Tool ${tool} is about to execute" - event: postToolUse matcher: "shell" action: type: command command: "echo 'shell tool finished'" - event: onStart action: type: log message: "Agent started" ``` **Loading and applying hooks:** ```java // Parse from YAML InputStream yaml = new FileInputStream("hooks.yaml"); DeclarativeHookRegistry registry = DeclarativeHookRegistry.fromYaml(yaml); // Parse with custom executor DeclarativeHookRegistry registry = DeclarativeHookRegistry.fromYaml(yaml, customExecutor); // Programmatic creation DeclarativeHookRegistry registry = DeclarativeHookRegistry.of(hookConfigs); // Apply to an agent (registers ToolCallListener + fires ON_START) registry.applyTo(agent); ``` **Manual event firing:** ```java // Fire a generic event registry.fireEvent(HookEvent.ON_START, Map.of("agent", agentId)); // Fire a tool-specific event (checks HookMatcher against tool name) registry.fireToolEvent(HookEvent.PRE_TOOL_USE, "shell", Map.of("event", "preToolUse")); ``` **Querying hooks:** ```java List all = registry.getHooks(); List preHooks = registry.getHooksForEvent(HookEvent.PRE_TOOL_USE); ``` When applied to an agent via `applyTo()`, the registry creates a `ToolCallListener` that: - Fires `PRE_TOOL_USE` hooks in `onToolCallStart` - Fires `POST_TOOL_USE` hooks in `onToolCallComplete` - Fires `ON_ERROR` hooks when `success` is false ## Agent Factory `AgentFactory` (`com.tnsai.server.agent`) creates agents for chat sessions with built-in tools and configurable LLM providers. ### Built-in Tools Every agent created by `AgentFactory` includes: | Tool | Description | |------|-------------| | `ShellTool` | Shell command execution with risk classification | | `FileReadTool` | Read file contents (auto-approved) | | `FileWriteTool` | Write file contents (requires approval) | ### Creating Agents ```java // Setup Supplier llmSupplier = () -> new AnthropicClient("claude-sonnet-4-20250514"); AgentFactory factory = new AgentFactory(llmSupplier); // Or with a custom working directory AgentFactory factory = new AgentFactory(llmSupplier, Path.of("/project")); // Simple creation (role name only) Agent agent = factory.createAgent("agent-001", "assistant"); // With additional tools (e.g., MCP proxy tools) Agent agent = factory.createAgent("agent-001", "developer", additionalTools); // With provider and model override Agent agent = factory.createAgent("agent-001", "developer", "ollama", "glm-5:cloud", additionalTools); // Full configuration with custom goal and domain Agent agent = factory.createAgent("agent-001", "developer", "ollama", "glm-5:cloud", "Write clean, tested code for the TnsAI framework", "tnsai-core", additionalTools); // Multi-role agent from RoleDef list Agent agent = factory.createAgent("agent-001", "assistant", "ollama", "glm-5:cloud", roleDefs, additionalTools); ``` ### Built-in Role Goals | Role | Goal | |------|------| | `assistant` | Help users by answering questions. Use tools to read files, execute commands, and write code. | | `developer` | Write clean, well-tested code. Use shell and file tools to explore and implement. | | `reviewer` | Analyze code for bugs, style, and improvements. Read files and run tests. | | `architect` | Design systems and make technical decisions. Explore codebase structure. | | `tester` | Write tests and find edge cases. Use shell tools to run tests. | | (other) | Perform the role of `{name}` effectively. Use available tools as needed. | ### Provider Resolution Provider overrides on Maven Central `0.14.1` fail loudly. The 0.13.0 release was the last line that warned and fell back. #### Maven Central 0.14.1 `TnsAI@609a661d` is in the published jars. The server no longer silently substitutes a different provider: - A null or blank `provider` uses the configured default `LLMClient` supplier. - `"ollama"` is the only supported explicit override. - Any other explicit value throws `IllegalArgumentException`. To run a cloud client such as Anthropic, configure that client as the server's default supplier and omit `provider`. Supplying `provider: "anthropic"` does not select an `AnthropicClient`. #### Maven Central 0.13.0 (historical) - A null or blank `provider` used the configured default supplier. - `"ollama"` created an `OllamaClient` with the requested model, or `"glm-5:cloud"` when the model was blank. - Any other nonblank value logged a warning and fell back to the default supplier. ## Idle Shutdown Manager `IdleShutdownManager` (`com.tnsai.server.health`) auto-shuts down the server after a configurable idle period when no active WebSocket connections exist (ADR-2: hybrid auto-daemon). ```java IdleShutdownManager manager = new IdleShutdownManager( connectionManager, // WsConnectionManager instance Duration.ofMinutes(30), // idle timeout () -> server.stop() // shutdown action ); manager.start(); // If no connections exist at start, schedules shutdown // Called by WebSocket handlers manager.onConnectionOpened(); // cancels pending shutdown manager.onConnectionClosed(); // schedules shutdown if no connections remain manager.stop(); // cleanup ``` Behavior: - On connection open: cancels any pending shutdown - On connection close: if no active connections remain, schedules shutdown after timeout - On timeout: re-checks for connections before shutting down (double-check safety) - Uses a daemon thread (`tns-idle-shutdown`) for the scheduler ## RAG Pipeline The RAG (Retrieval-Augmented Generation) system in `com.tnsai.server.rag` provides file indexing, code-aware chunking, and hybrid retrieval. ### RagService Orchestrates the full RAG pipeline for a single session. Thread-safe: indexing is serialized via a lock; reads (search) are concurrent. ```java RagService rag = new RagService(); // Index a directory rag.indexDirectory(Path.of("/project/src"), progress -> System.out.printf("Indexing: %d/%d - %s%n", progress.indexedFiles(), progress.totalFiles(), progress.currentFile())); // Search the knowledge base List results = rag.search("authentication logic", 5); // Build a context-augmented prompt String augmentedPrompt = rag.buildContextPrompt("How does auth work?", 3); // Returns: // [Relevant code context] // --- file: src/main/java/Auth.java (lines 10-30) --- // // // [User question] // How does auth work? ``` **Document management:** ```java // Add a document manually String docId = rag.addDocument("Some text content", Map.of("source", "manual", "tag", "notes")); // Remove a document boolean removed = rag.removeDocument(docId); // List all managed documents List docs = rag.listDocuments(); // DocumentInfo(id, preview, contentLength, metadata) // Get a specific document Optional doc = rag.getDocument(docId); // Get index status IndexStatus status = rag.getStatus(); // IndexStatus(fileCount, chunkCount, lastIndexedAt, indexedPath) // Clear everything rag.clear(); ``` ### FileIndexer Walks a directory tree and indexes supported source files into a `KnowledgeBase`. **Features:** - Respects `.gitignore` and `.tnsignore` patterns - Skips binary files and known build/dependency directories - SHA-256 content hashing for incremental re-indexing (unchanged files are skipped) - Progress reporting via callback **Supported file extensions:** `java`, `ts`, `tsx`, `js`, `jsx`, `py`, `md`, `json`, `yml`, `yaml`, `xml`, `html`, `css`, `sh`, `sql`, `go`, `rs`, `rb`, `kt`, `scala`, `c`, `cpp`, `h` **Skipped directories:** `.git`, `node_modules`, `build`, `dist`, `target`, `.idea`, `.vscode`, `.gradle`, `__pycache__`, `vendor`, `.next`, `out`, `coverage`, `.svn`, `.hg` **Maximum file size:** 512 KB ```java FileIndexer indexer = new FileIndexer(); FileIndexer.IndexResult result = indexer.index( Path.of("/project"), knowledgeBase, bm25Stream, progress -> System.out.println(progress.currentFile()) ); // IndexResult(fileCount, chunkCount) // Force full re-index on next run indexer.clearHashes(); ``` ### CodeChunker Splits source code files into semantically meaningful chunks for RAG indexing. Each chunk is a `Document` with metadata: `file`, `startLine`, `endLine`, `language`. **Chunking strategies by language:** | Language | Strategy | |----------|----------| | Java, Kotlin, Scala | Class and method boundary detection via regex | | TypeScript, JavaScript | Function, class, and arrow-function boundary detection | | Markdown | Heading boundary detection | | Everything else | Fixed-size line groups (max 100 lines) | Files under 100 lines are kept as a single chunk. ```java // Chunk a file List chunks = CodeChunker.chunk( Path.of("src/Auth.java"), // file path (metadata) fileContent, // file content string "java" // language key ); // Detect language from filename String lang = CodeChunker.detectLanguage("Auth.java"); // "java" String lang = CodeChunker.detectLanguage("app.tsx"); // "tsx" String lang = CodeChunker.detectLanguage("README.md"); // "md" String lang = CodeChunker.detectLanguage("unknown.xyz"); // "text" ``` ### Hybrid Retrieval `RagService` uses a `HybridRetriever` that combines two retrieval streams: - **BM25Stream** (weight 0.6) -- keyword-based BM25 scoring - **KnowledgeBaseStream** (weight 0.4) -- vector similarity from the `KnowledgeBase` Results are merged and ranked by weighted score. ## Cross-References - [WebSocket Protocol](websocket.md) -- WebSocket v1 protocol for client communication - [Tool Approval](tool-approval.md) -- tool approval flow via WebSocket - [RAG Pipeline](../capabilities/rag/pipeline.md) -- RAG pipeline overview - [Observability](../observability.md) -- evaluation hooks and tracing