Troubleshooting: common VeriOps pipeline issues¶

Your VeriOps documentation pipeline is failing with contract validation errors, quality gate warnings, or build failures. This guide fixes 95% of pipeline issues in under 5 minutes.

Quick diagnosis (30 seconds)¶

Check the pipeline exit code and stage summary first:

python3 -c "
import json
s = json.load(open('reports/pipeline_stage_summary.json'))
for stage in s.get('stages', []):
    status = 'EXISTS' if stage.get('exists') else 'MISSING'
    print(f'  {stage[\"name\"]}: {status}')
m = json.load(open('reports/review_manifest.json'))
print(f'Exit code: {m[\"weekly_rc\"]}')
print(f'Available: {m[\"available_artifacts\"]}, Missing: {m[\"missing_artifacts\"]}')"

Diagnosis table¶

Symptom	Frequency	Fix time	Solution
gRPC contract fails	40%	2 min	Contract validation fails for gRPC
Quality score below 80	25%	5 min	Quality score drops below 80
MkDocs build fails	15%	1 min	MkDocs build fails with theme error
WebSocket tester error	10%	1 min	WebSocket tester shows connection error
Reports directory empty	5%	2 min	Pipeline reports directory is empty
RAG retrieval low precision	5%	10 min	Retrieval precision below threshold

Contract validation fails for gRPC¶

You see: The multi_protocol_contract_report.json shows grpc in failed_protocols.

{
  "failed_protocols": ["grpc"],
  "protocols": ["rest", "graphql", "grpc", "asyncapi", "websocket"]
}

Root cause: The protoc compiler is not installed, or the proto files reference missing imports.

Fix gRPC contract in 2 minutes¶

Install the protobuf compiler:
```
apt-get install -y protobuf-compiler
protoc --version
```
Expected output: libprotoc 3.21.x or later.

Verify proto file paths match client_runtime.yml:

grpc:
  proto_paths:
    - reports/contracts/grpc

Re-run the pipeline:

python3 scripts/run_autopipeline.py \
  --docsops-root . \
  --reports-dir reports \
  --runtime-config reports/client_runtime.yml \
  --mode veridoc

Verify gRPC passes:

python3 -c "
import json
r = json.load(open('reports/multi_protocol_contract_report.json'))
print('Failed:', r.get('failed_protocols', []))"

Expected output: Failed: []

Quality score drops below 80¶

You see: The kpi-wall.json shows quality_score below 80.

Root cause: Stale documents, missing frontmatter, or unresolved documentation gaps lower the quality score below the 80 threshold.

Fix quality score in 5 minutes¶

Check the gap report for high-priority items:

python3 -c "
import json
r = json.load(open('reports/doc_gaps_report.json'))
high = [g for g in r.get('gaps', []) if g.get('priority') == 'high']
print(f'{len(high)} high-priority gaps:')
for g in high:
    print(f'  [{g[\"priority\"]}] {g[\"title\"]}')"

Address each high-priority gap by creating or updating the relevant document. Common gaps include:

Gap category	Typical fix
`authentication`	Add authentication guide with token management
`webhook`	Document webhook setup and payload schemas
`database_schema`	Add data model reference with field descriptions
`error_handling`	Create error code reference with resolution steps

Re-run the pipeline and verify the score:

python3 -c "
import json
print(json.load(open('reports/kpi-wall.json'))['quality_score'])"

MkDocs build fails with theme error¶

You see: mkdocs build exits with Theme 'material' not found or Module 'mkdocs_macros' not found.

Root cause: The required Python packages are not installed in the current environment.

Fix MkDocs theme in 1 minute¶

pip install mkdocs-material mkdocs-macros-plugin pymdown-extensions

Verify the build succeeds:

cd demo-showcase/veridoc && mkdocs build --strict

Expected output: INFO - Documentation built in X.XX seconds

WebSocket tester shows connection error¶

You see: The interactive WebSocket tester on the WebSocket event playground displays "Connection error."

Root cause: The browser blocks insecure WebSocket connections (ws://) from HTTPS pages, or the endpoint is unreachable.

Fix WebSocket connection in 1 minute¶

Verify the endpoint uses wss:// (not ws://). The correct endpoint is wss://api.veriops.example/realtime.

Confirm the endpoint is accessible from your network. Try from the command line:

curl -s -o /dev/null -w "%{http_code}" https://api.veriops.example/realtime

Check browser developer tools (Console tab) for specific error messages.
If you use a corporate proxy, configure WebSocket passthrough or use the AsyncAPI event docs with direct AMQP instead.

Pipeline reports directory is empty¶

You see: The reports/ directory contains no JSON files after running the autopipeline.

Root cause: The runtime config path is incorrect, or the --reports-dir argument points to a different location.

Fix empty reports in 2 minutes¶

Verify the runtime config exists:
```
ls -la reports/client_runtime.yml
```

Run the pipeline with explicit paths:

python3 scripts/run_autopipeline.py \
  --docsops-root . \
  --reports-dir reports \
  --runtime-config reports/client_runtime.yml \
  --mode veridoc

Verify reports are generated:
```
ls reports/*.json | head -10
```
Expected output: at least 5 JSON report files.

Retrieval precision below threshold¶

You see: The retrieval_evals_report.json shows precision below 0.5 or recall below 0.5.

Root cause: The eval dataset queries may not match the current module IDs after re-extraction. Re-run module extraction and update the eval dataset. Enable advanced retrieval features (hybrid search, HyDE, cross-encoder reranking) for production-grade precision and recall.

Fix retrieval precision in 10 minutes¶

Check the retrieval evaluation report:

python3 -c "
import json
r = json.load(open('reports/retrieval_evals_report.json'))
print(f'Status: {r[\"status\"]}')
print(f'Precision: {r[\"precision\"]}')
print(f'Recall: {r[\"recall\"]}')
print(f'Hallucination rate: {r[\"hallucination_rate\"]}')"

Run a multi-mode comparison to identify the best search strategy:

python3 scripts/run_retrieval_evals.py \
  --mode all \
  --dataset config/retrieval_eval_dataset.yml \
  --report reports/retrieval_comparison.json

This compares token, semantic, hybrid, and hybrid+rerank modes side by side.

Improve knowledge module coverage:
- Add more detailed content to documentation pages (each page should have at least 100 words)
- Include code examples with inline comments (the knowledge extractor indexes code blocks)
- Add tables with specific values (the knowledge extractor indexes structured data)

Rebuild the retrieval index with chunking:

python3 scripts/validate_knowledge_modules.py
python3 scripts/generate_knowledge_retrieval_index.py
python3 scripts/generate_embeddings.py --chunk \
  --index docs/assets/knowledge-retrieval-index.json \
  --output-dir docs/assets/

Verify advanced retrieval features are enabled in config/ask-ai.yml:

hybrid_search:
  enabled: true
hyde:
  enabled: true
reranking:
  enabled: true
embedding_cache:
  enabled: true

Re-run retrieval evaluations:

python3 scripts/run_autopipeline.py \
  --docsops-root . \
  --reports-dir reports \
  --runtime-config reports/client_runtime.yml \
  --mode veridoc

Prevention checklist¶

Prevent 90% of pipeline issues with these practices:

[ ] Install all dependencies: protoc, mkdocs-material, pymdown-extensions
[ ] Run the pipeline before every release: Do not wait for the weekly schedule
[ ] Address high-priority gaps immediately: They compound and lower the quality score
[ ] Keep contract files current: Update OpenAPI, GraphQL schema, and proto files with every API change
[ ] Monitor the KPI dashboard: Check kpi-wall.json quality score after every run

Performance baseline¶

After resolving issues, your pipeline should show:

Metric	Good	Warning	Critical
Quality score	80+	70-79	Below 70
Failed protocols	0	1	2 or more
High-priority gaps	0-2	3-5	6 or more
Pipeline exit code	0	1 (non-critical)	2 (critical failure)
Retrieval precision	0.7+	0.5-0.69	Below 0.5

Next steps¶

How-to: keep docs aligned with every release for the operational workflow
Quality evidence and gate results for current gate status
Concept: pipeline-first documentation lifecycle to understand the pipeline architecture