MAS_Metax V1 — Offline Multi-Agent Deep Research Framework

Version 1.1.0 | Runtime 沐曦 GPU + vLLM | Mode Offline containerized inference

Overview

MAS_Metax V1 is an offline multi-agent research system designed for MetaX GPU containers. Six specialized LLM agents collaborate to execute deep research tasks — from topic decomposition through multi-round retrieval, evidence analysis, report writing, and quality review — producing a complete Markdown report with full audit trails and visualizations.

Key Capabilities

Capability	Description
Per-Agent Model Selection	Each of the 6 agents can use a different local model at runtime
Multi-Round Retrieval	Iterative query generation → recall → relevance scoring → query refinement
Knowledge Refinement	High-quality retrieval results are persisted to the local corpus for reuse
Quality Control Loop	Reviewer scores the report, ReviewFeedback drives revisions, loop closes on passed=true
GPU Memory Management	Automatic model unloading before loading a different model (V1.1)
Full Audit Trail	Every LLM call is logged with agent, purpose, temperature, tokens, and latency
Run Visualizations	Auto-generated SVG/HTML dashboards showing workflow, timeline, and agent usage
Offline Operation	No internet required; optional web search when available

---

Quick Start

# Smoke test with mock backend (no GPU required)
python run.py --demo --print-report

# List available models
python run.py --list-models

# Real run with per-agent model selection
python run.py \
  --topic "沐曦容器内基于 vLLM 的离线多智能体推理框架" \
  --backend vllm \
  --agent-model Orchestrator=Qwen3-4B \
  --agent-model Researcher=Qwen3-4B \
  --agent-model Analyst=Qwen3.5-9B \
  --agent-model Writer=Qwen3.5-27B-W8A8 \
  --agent-model Reviewer=Qwen3-30B-A3B-Thinking-2507 \
  --agent-model ReviewFeedback=Qwen3-30B-A3B-Thinking-2507 \
  --source examples/metax_local_notes.md

# Using a vLLM OpenAI-compatible server instead of direct load
python run.py \
  --topic "国产 GPU 离线 LLM 推理框架评估" \
  --backend openai \
  --base-url http://127.0.0.1:8000/v1 \
  --model-name Qwen3-8B \
  --source examples/metax_local_notes.md

---

Architecture

User Topic / Query
       │
       ▼
┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│  Orchestrator   │────▶│   Researcher     │────▶│    Analyst      │
│  Task Planning  │     │  Multi-Round     │     │  Evidence       │
│  & Decomposition│     │  Retrieval +     │     │  Analysis       │
│                 │     │  Knowledge Refine│     │                 │
└─────────────────┘     └────────┬─────────┘     └────────┬────────┘
                                 │                        │
                    ┌────────────┴────────┐               │
                    ▼                     ▼               │
            ┌──────────────┐    ┌─────────────────┐       │
            │ Local Corpus │    │  Web Search     │       │
            │ (lexical)    │    │  (DDGS/Bing)    │       │
            └──────────────┘    └─────────────────┘       │
                                 │                        │
                                 ▼                        │
                        ┌─────────────────┐               │
                        │ Relevance Score │               │
                        │ (LLM + lexical  │               │
                        │  fallback)      │               │
                        └────────┬────────┘               │
                                 │                        │
                    ┌────────────┴────────────┐           │
                    ▼                         ▼           │
            ┌──────────────┐         ┌──────────────┐     │
            │ Refined KB   │         │ Evidence Pool│◀────┘
            │ (persisted)  │────────▶│ source_hits  │
            └──────────────┘         └──────┬───────┘
                                            │
                                            ▼
                                   ┌─────────────────┐
                                   │     Writer      │
                                   │  Markdown Report│
                                   └────────┬────────┘
                                            │
                                            ▼
                                   ┌─────────────────┐     ┌──────────────────┐
                                   │    Reviewer     │────▶│ ReviewFeedback   │
                                   │  Quality Audit  │     │ Revision Plan +  │
                                   │  JSON output    │     │ Closure Check    │
                                   └────────┬────────┘     └────────┬─────────┘
                                            │                       │
                                            │    ┌──────────────────┘
                                            ▼    ▼ (backtrack)
                                   ┌─────────────────┐
                                   │  Final Report   │
                                   │  + Visuals      │
                                   └─────────────────┘

Key architectural properties:

• Each agent independently selects its model. Model assignments are persisted in model_assignments.json and visualized in the dashboard.
• The Researcher queries both a local lexical corpus and an optional web search engine. Web results record url, provider, and retrieved_at.
• Retrieval runs multiple rounds: initial query → recall → LLM relevance scoring → query refinement → re-recall.
• After each retrieval cycle, refined knowledge is written to data/knowledge/refined/ and the corpus index is refreshed.
• The Analyst distinguishes between locally sourced evidence, web-sourced information, model inference, and identified gaps.
• The Reviewer checks for <think> leakage, unsourced performance numbers, fabricated comparison tables, and missing URL citations.
• ReviewFeedback maps review findings to a backtrack plan: evidence gaps trigger Researcher, analysis weaknesses trigger Analyst, structure/writing issues trigger Writer.
• Every LLM call is logged with agent identity, purpose, model, temperature, max_tokens, character counts, and elapsed time.
• All logs are persisted to log/ with an auto-generated error summary on failure.

---

Agents

Agent	Role	Temperature	Max Tokens	Key Output
Orchestrator	Task decomposition & assignment	0.2	1200	JSON task list
Researcher	Multi-round retrieval & relevance scoring	0.2–0.3	600–2200	Structured research notes
Analyst	Evidence synthesis & risk identification	0.35	2200	Analysis Markdown
Writer	Full Markdown report generation	0.45	3200	draft_report
Reviewer	Quality audit against red-line criteria	0.2	1400	JSON review
ReviewFeedback	Revision brief & closure validation	0.2	1400–1600	Feedback JSON

Reviewer Red-Line Criteria

• <think> or internal reasoning content present → fail
• Unsourced throughput/latency/memory/QPS figures → fail
• Comparison tables using fabricated values without explicit caveat → fail
• Web hits exist but report contains zero URLs or freshness notes → fail
• Score below 8 (configurable) → forced passed=false

---

Pipeline (5 Stages)

1. Planning — Orchestrator decomposes the topic into 3–5 subtasks.
2. Execution — Subtasks run sequentially. If Writer runs before analysis is done, Analyst is invoked first.
3. Fallback Drafting — Ensures analysis_results and draft_report exist before review.
4. Review Loop — Reviewer scores, ReviewFeedback plans revisions, relevant agents re-execute. Repeats up to max_review_rounds times.
5. Artifact Persistence — Report, state, logs, visualizations, and knowledge deposits are saved to outputs/.

---

Backend Modes

Backend	CLI Flag	Description
Direct vLLM	--backend vllm	Loads the model in-process via vllm.LLM(). Supports automatic model swapping.
OpenAI Compatible	--backend openai	Connects to a vLLM/OpenAI-compatible HTTP server. No local model loading.
Mock	--backend mock	Deterministic responses for smoke testing. No GPU required.

Model Swapping (V1.1)

When backend=vllm and agents use different model paths, _ensure_agent_model_ready() automatically unloads the previously loaded model before loading the next one, preventing GPU out-of-memory errors.

---

Model Catalog

Models are resolved from /mnt/moark-models/. The framework ships with catalog entries for 9 models:

Model	Capability	Recommended Agents
Qwen3-8B	Text / Reasoning	All (default)
Qwen3-4B	Text / Fast	Orchestrator, Researcher, ReviewFeedback
Qwen3-0.6B	Text / Ultra-fast	Researcher
Qwen3.5-9B	Text / Reasoning	Analyst, Writer, Reviewer
Qwen3.5-27B-W8A8	Text / High-quality	Analyst, Writer, Reviewer
Qwen3-30B-A3B-Thinking-2507	Thinking / High-quality	Analyst, Reviewer, ReviewFeedback
Qwen3-VL-4B-Instruct	Vision-Language	Researcher, Reviewer
Qwen3-VL-8B-Instruct	Vision-Language	Researcher, Reviewer
ERNIE-4.5-VL-28B-A3B-Thinking	VL / Thinking	Analyst, Reviewer

Per-agent models can be set via CLI (--agent-model Researcher=Qwen3-4B), environment variables (METAX_RESEARCHER_MODEL), or a JSON map (METAX_AGENT_MODEL_MAP).

---

Multi-Round Retrieval & Knowledge Refinement

The Researcher agent performs iterative retrieval (default: 3 rounds, 4 queries/round, 8 hits/round):

1. Query Generation — LLM produces 4–6 targeted queries from the task description.
2. Recall — Queries hit local_search (lexical, TF-based) and optionally web_search (DDGS / Bing fallback).
3. Relevance Scoring — LLM scores results 0–1; lexical fallback uses TF coverage × 2 + density.
4. Query Refinement — LLM generates next-round queries focused on information gaps.
5. Synthesis — Top retained hits are summarized into structured notes for the Analyst.
6. Knowledge Deposit — Queries, summaries, and top sources are persisted to data/knowledge/refined/*.md and the corpus index is refreshed.

# Customize retrieval parameters
python run.py \
  --retrieval-rounds 5 \
  --retrieval-queries-per-round 6 \
  --retrieval-max-hits-per-round 10 \
  --retrieval-relevance-threshold 0.3 \
  --knowledge-refine \
  --topic "..."

---

Quality Control Loop

Reviewer (score, passed, required_fixes)
    │
    ▼
ReviewFeedback (should_revise, target_agents, backtrack_plan)
    │
    ├── Researcher  ← evidence gaps, missing sources, freshness issues
    ├── Analyst     ← logic conflicts, confidence issues, inference gaps
    └── Writer      ← structure, citations, expression, completeness
    │
    ▼
validate_closure (closed, final_decision, remaining_risks)
    │
    ├── closed=true & should_revise=false → Final Report
    └── otherwise → next review round

Default: max 3 rounds, score threshold 8. All actions are recorded in quality_actions.json.

---

Output Artifacts

Every run produces the following files under outputs/<timestamp>_<topic>/:

File	Description
report.md	Final Markdown report
state.json	Full run state (tasks, sources, reviews, events)
config.json	Resolved runtime configuration
model_assignments.json	Agent-to-model mapping
llm_calls.json	All LLM and tool call logs
agent_llm_usage.json	Aggregated LLM usage per agent
sources.json	Loaded corpus files and retrieval hits
review_feedback.json	Per-round feedback and revision briefs
validation.json	Closure validation results
quality_actions.json	Backtrack actions triggered by reviews
visual/workflow.svg	Agent collaboration flow
visual/timeline.svg	LLM call timeline
visual/llm_usage.svg	Per-agent LLM usage chart
visual/feedback_loop.svg	Review & revision loop
visual/model_assignment.svg	Agent-to-model assignment
visual/retrieval_strategy.svg	Multi-round retrieval visualization
visual/dashboard.html	Interactive run dashboard
data/knowledge/refined/*.md	Persisted refined knowledge

---

Configuration Reference

Copy .env.example to .env to set defaults:

cp .env.example .env

Variable	Default	Description
METAX_BACKEND	vllm	Backend: vllm, openai, or mock
METAX_MODEL_PATH	/mnt/moark-models/Qwen3-8B	Path to model for direct vLLM
METAX_MODEL_NAME	Qwen3-8B	Model name for OpenAI-compatible API
METAX_GPU_MEMORY_UTILIZATION	0.88	Fraction of GPU memory for vLLM
METAX_MAX_MODEL_LEN	8192	Max context length
METAX_TEMPERATURE	0.6	Sampling temperature
METAX_MAX_TOKENS	2048	Max generated tokens per call
METAX_RETRIEVAL_ROUNDS	3	Max retrieval rounds
METAX_RETRIEVAL_QUERIES_PER_ROUND	4	Max queries per round
METAX_RETRIEVAL_MAX_HITS_PER_ROUND	8	Max hits retained per round
METAX_RETRIEVAL_RELEVANCE_THRESHOLD	0.25	Minimum relevance score to keep
METAX_MAX_REVIEW_ROUNDS	3	Max review-revision iterations
METAX_REVIEW_SCORE_THRESHOLD	8	Minimum score for passed=true
METAX_ENABLE_WEB_SEARCH	false	Enable optional web search
METAX_KNOWLEDGE_DIR	data/knowledge	Local corpus root
METAX_KNOWLEDGE_REFINE_DIR	data/knowledge/refined	Refined knowledge output
METAX_OUTPUT_DIR	outputs	Run artifacts directory
METAX_ENABLE_VISUALS	true	Generate SVG/HTML visualizations
METAX_LOG_LEVEL	INFO	Logging verbosity

Per-agent overrides: METAX_ORCHESTRATOR_MODEL, METAX_RESEARCHER_MODEL, METAX_ANALYST_MODEL, METAX_WRITER_MODEL, METAX_REVIEWER_MODEL, METAX_REVIEW_FEEDBACK_MODEL.

---

Project Structure

V1/
├── run.py                          # CLI entry point
├── requirements.txt                # Dependencies
├── .env.example                    # Environment variable template
├── generate_pptx.py                # PPTX report generator
├── metax_mas/
│   ├── __init__.py
│   ├── cli.py                      # Argument parsing & log setup
│   ├── api.py                      # FastAPI service
│   ├── pipeline.py                 # End-to-end orchestration
│   ├── agents/
│   │   ├── base.py                 # Agent base class
│   │   ├── orchestrator.py         # Task planning
│   │   ├── researcher.py           # Multi-round retrieval
│   │   ├── analyst.py              # Evidence analysis
│   │   ├── writer.py               # Report generation & revision
│   │   ├── reviewer.py             # Quality audit
│   │   └── review_feedback.py      # Feedback & closure validation
│   └── core/
│       ├── config.py               # RuntimeConfig
│       ├── llm.py                  # Backend adapters (vLLM/OpenAI/Mock)
│       ├── model_catalog.py        # Model registry
│       ├── state.py                # ResearchState & TaskStatus
│       ├── tools.py                # Local corpus & web search
│       ├── artifacts.py            # Output persistence
│       └── visualization.py        # SVG/HTML dashboard
├── data/
│   └── knowledge/                  # Local corpus & refined knowledge
├── examples/
│   └── metax_local_notes.md        # Sample source document
├── log/                            # Run logs (auto-generated)
└── outputs/                        # Run artifacts (auto-generated)

---

Deployment Guide

Recommended Workflow

1. Smoke test with --demo to verify the code path.
2. Single-model test with --backend vllm --model-path /mnt/moark-models/Qwen3-8B.
3. Multi-model setup using --agent-model AGENT=MODEL flags.
4. Increase quality by raising --review-score-threshold 9 and --max-review-rounds 4.

GPU Memory Notes

• Running multiple large models in one process requires V1.1's automatic model swapping.
• If memory is still tight, use a single model for all agents, or run models as separate vLLM serve instances and connect via --backend openai.
• Lower --gpu-memory-utilization (e.g., 0.75) if other processes share the GPU.

First-Run Latency

The first invocation with backend=vllm incurs one-time costs from model loading, torch.compile, and CUDA graph capture. Use --enforce-eager to skip graph capture for faster startup (at the cost of lower throughput on repeated calls).

Web Search

Enable with --web-search. Requires network access and the ddgs / duckduckgo_search package. If unavailable, the tool returns empty results and the report notes the limitation.

API Mode

uvicorn metax_mas.api:app --host 0.0.0.0 --port 8000

---

V1.1 Resilience Improvements

Issue	Fix
Multi-model GPU OOM	_ensure_agent_model_ready() auto-unloads previous models
Single task failure kills pipeline	Non-fatal: logs error, marks FAILED, continues
Stage 3/4 exceptions crash run	All fallback stages wrapped in try/except
No persistent logs	_setup_logging() writes both stderr and log/
Unstructured error output	Auto-generated error_summary_<ts>.txt with full config
Missing tokenizer config	tokenizer_mode passed to LLM()

---

License

Internal research use. MetaX / MAS_Metax.