Agentic Project Plan — Framework

📢 This is a proposed plan and vision. The content reflects current thinking and is subject to change as the project evolves, requirements are clarified, and decisions are made with the team.

1. Vision & Goals

One repository, two kinds of code: the platform (built once, reused everywhere) and the projects (composed per use case).

This framework defines how to build, structure, and operate an agentic platform that hosts multiple use cases within a single monorepo. The platform's purpose is to eliminate duplication across use cases by providing reusable, audit-grade building blocks that any project pod can compose without reimplementing.

North-star metric: The third use case onboards in under 2 sprints, reusing the platform with zero copy-paste.

Goals

• Deliver agentic automation for domain-specific workflows that is explainable, auditable, and deterministic where rules apply
• Establish a reusable platform so each new use case costs less than the last
• Enforce governance (PII, data residency, role-based access) uniformly across all projects
• Give operators full observability into cost, latency, and agent decisions

2. Scope & Use Cases

In Scope

Populate this table with the actual use cases for the engagement.

Out of Scope

• General-purpose chatbot or Q&A outside the defined domain
• Direct end-user UI (the platform exposes APIs; front-end is owned by consuming teams)
• Use cases outside the agreed domain boundary

3. Client Requirements

What Clients Want

What Clients Say vs. What Actually Blocks Projects

4. Architecture Overview

Monorepo Zones

Dependency Rules

apps/*  →  projects/*  →  packages/*

Platform Response to Client Requirements

5. Code Structure

Top-Level Layout

agent-platform/
  packages/                  # Platform zone — reusable, owned by Central Team
    agent_factory/           # Agent framework wrappers: load_declarative_agent, create_code_agent
    enterprise_tools/        # @tool-decorated async functions for external system integrations
    shared_models/           # Canonical Pydantic models and snapshot schema
    shared_utils/            # Logging, retry, config helpers
  projects/                  # Project zone — one folder per use case
    <project>/
      workflow/
        <project>_workflow.py      # WorkflowBuilder wiring (Graph API)
        messages.py                # Pydantic input/result/output types
        workflow_config.yml        # workflow_api: graph
        executors/
          <name>_executor.py       # Executor subclasses with @handler
      agents/
        <project>_agents.py        # Agent construction via agent_factory
      tools/
        <project>_tool.py          # @tool-decorated async functions (project-specific)
      skills/
        SKILL.md                   # MAF Skills stub
        references/
      configs/
        agent_profiles/            # kind: Prompt YAML files
        prompts/                   # System prompt .md files
        rules/                     # Deterministic rule definitions
      services/
        <name>_decision_service.py # Business rule enforcement
      tests/
        test_workflow.py
      evals/
  apps/                      # Worker entrypoints — one per project
    <project>-worker/
      main.py
      Dockerfile
  evals/                     # Cross-project eval runner
  infra/                     # IaC — shared + per-project
    shared/
    <project>/
  docs/
    standards/               # Platform coding standards (this file lives here)
    runbooks/                # Per-project operational runbooks
  scripts/                   # CI helpers, code generators
  ci_matrix.yml              # Change-aware CI: only test affected projects
  CODEOWNERS                 # Enforces ownership boundaries

Agent & Workflow Design Standards

• Graph API is production default — WorkflowBuilder, Executor subclass + @handler, IWorkflowContext.yield_output()
• Functional API (@workflow/@step) is prototyping only — never in production
• Every workflow must have a Default() fallback edge on every switch node
• Every workflow must have a HITLExecutor node for decisions above a confidence threshold
• Agents are created via agent_factory — never instantiated directly from MAF classes
• Tools are plain async functions decorated with @tool — no BaseTool class

Declarative Agent YAML Schema

kind: Prompt
name: "agent_name"
description: "..."
instructions: "..."
model:
  id: "gpt-4o"
  provider: "AzureOpenAI"
  apiType: "azure"
  options: { temperature: 0, max_tokens: 2048 }
# outputSchema: optional JSON Schema for structured output

6. Integration Points

Replace placeholder names with actual systems for the engagement.

Rule: Project pods never instantiate integration clients directly. All external calls go through packages/enterprise_tools/.

7. Testing & Eval Strategy

Layers

Eval Standards

• Every project must have a minimum of 10 golden cases before going to production
• Eval score threshold: ≥ 90% pass rate required to merge
• Golden cases cover: happy path, edge cases, HITL trigger conditions, rule enforcement
• Evals run against the same LLM endpoint as production (no mocked LLM in evals)

Testing Methods

• Executor unit tests — instantiate executor, call handler directly, assert on yield_output captures
• Workflow integration tests — build full graph, inject mock tool responses, assert final output model
• Decision service tests — pure Python, no LLM, 100% deterministic, must have 100% coverage
• Eval runner — scripts/run_evals.py --project <name> scores against golden cases

8. Observability & Cost

Tracing

• Every workflow node logs: node_id, input_hash, output_summary, latency_ms, token_usage
• Traces stored in persistent store; queryable per session_id and workflow_run_id
• Structured logs emitted to the agreed observability platform (e.g. Azure Monitor, Datadog, ELK)

Cost Tracking

• LLM client middleware captures prompt tokens, completion tokens, model ID per call
• Cost rolled up per: individual call → executor → workflow run → project → daily aggregate
• Chargeback tags: project, use_case, environment

Dashboards (stub — to be built in Sprint 2)

• Cost per workflow run (by project)
• P50/P95 latency per executor node
• HITL trigger rate (% of runs requiring human intervention)
• Eval pass rate trend over time

Alerting (stub)

• Cost spike: > 2× 7-day average triggers on-call alert
• Eval regression: < threshold on any project blocks deployment pipeline

9. Governance & Compliance

CODEOWNERS Enforcement

• packages/ → Central Platform Team; any PR requires Platform Architect approval
• projects/<p>/ → Pod Tech Lead; cross-pod changes require both pod leads
• CODEOWNERS is the physical enforcement of team boundaries — it is not optional

Data & Security

• All model calls use an LLM client bound to an approved in-region endpoint
• No data leaves the approved tenant/cloud boundary
• PII detection runs as a guardrail before any data is sent to the LLM
• Role-based access: agent inherits the permissions of the calling user's identity

Audit Evidence

• Every workflow output is signed with workflow_run_id, timestamp, user_id, model_version
• Outputs exportable as JSON for compliance review
• Audit log retained for the agreed retention period (minimum 90 days)

Change Governance

• Any new shared abstraction in packages/ requires Platform Architect sign-off
• Project-specific logic must not creep into packages/ — the Architect says no
• Model/prompt version changes require a passing eval run before deployment

10. Team Formation & Ownership

Hub-and-Spoke Structure

                    ┌─────────────────────┐
                    │  Platform Architect  │
                    └──────────┬──────────┘
                               │
              ┌────────────────▼────────────────┐
              │     Central Platform Team        │
              │  (packages/, scripts/, infra/)   │
              └──┬──────────┬──────────┬─────────┘
                 │          │          │
          ┌──────▼──┐  ┌────▼────┐  ┌─▼───────┐
          │  Pod A  │  │  Pod B  │  │  Pod C  │
          │Use Case1│  │Use Case2│  │Use CaseN│
          └─────────┘  └─────────┘  └─────────┘

Central Platform Team (owns packages/)

Typical size: 3–5 people. Protect this team's capacity — if it's a bottleneck, every pod slows down.

Project Pods (own projects/<p>/ + 4 sibling folders)

Typical size: 2–4 people per pod (roles double up on small pods).

Staffing Summary

Rules of Engagement

• Pods import packages/* — they do not edit it. Need a change? Raise a PR to the Central Team
• No pod imports another pod
• The Platform Architect approves any new shared abstraction
• Project-specific logic must never creep into packages/

11. Delivery Milestones

The second project is the real test. Clean reuse proves the platform. Friction proves the abstraction is wrong.

Weekly Cadence

12. Development Lifecycle

Eval iterations run throughout Development — step 4 is the exit gate, not the first time evals run.

Key Differences from Standard SDLC

13. Risks & Mitigations

Related: Architecture Decision Records · Agent Standards · Eval Strategy · CI/CD Pipeline