0 · Pipeline at a glance
Orchestration entry point: cv_toolkit.py (all sub-commands). The batch driver
run_batch_v2.sh runs steps 1–12 for the whole company list using the broader
framework_v2/ rubrics, writing to outputs_v2/.
Prerequisites & one-time setup
conda activate cv_env
pip install -r requirements.txt
export TOKENIZERS_PARALLELISM=false # silence HuggingFace fork warning
# API key — stored in a git-ignored .env at repo root (loaded by cv_config.load_dotenv())
echo 'ANTHROPIC_API_KEY=sk-ant-...' > .env
Build the scoring rubrics once (converts the framework definitions into scoring bands +
key indicators, cached in .cv_cache/):
python cv_toolkit.py rubrics # build outputs/rubrics.json from framework/*.txt (cv_framework.py)
python cv_toolkit.py describe # add plain-English brief_descriptions (Haiku)
# v2 rubrics (broader definitions) are built automatically by run_batch_v2.sh → outputs_v2/rubrics.json
| Program | Role in setup | Creates |
|---|---|---|
cv_config.py | Single source of truth: dimensions, colours, paths, model pin, windowing / ensemble / caching toggles; loads .env. | — |
cv_framework.py | Builds rubrics from framework/ & framework_v2/ definition files via Claude. | outputs/rubrics.json, outputs_v2/rubrics.json |
The 12 steps
Document identification & fetch
Resolve the ticker to an SEC entity, locate its latest 10-K, and download the complete filing plus structured financials. Everything is cached so repeat runs are instant.
Preprocess into narrative sections
Strip XBRL/HTML boilerplate and isolate the analytical text — Items 1, 1A, 2, 7, and all of
Item 8 including the financial-statement footnotes. If the input is already a clean narrative,
it is detected (_looks_like_clean_narrative) and used as-is to avoid lossy re-processing
that would drop Item 8 notes.
Build the RAG index
Split the narrative into ~1,000-char passages and build a hybrid retrieval index:
BM25 + all-MiniLM-L6-v2 sentence embeddings. The index supports
index.query() = BM25 (≈60 candidates) re-ranked by cosine, blended 60% BM25 / 40% cosine.
Entity discovery (the RAG consumer)
Identify the company's named funds, vehicles, and portfolio companies, in three stages:
- Stage 0 — curated context (not RAG): load known funds / portfolio companies and
per-dimension
dimension_hintsfromcompany_context.json. - Stage 1 — RAG retrieval: for each investment-themed seed query, call
index.query(seed_query, top_k=5); keep the top ~20 highest-scoring passages — i.e. RAG surfaces the investment-relevant slice of the filing. - Stage 1b — pattern matching: regexes scan those passages for fund names, "formerly / through our X" patterns, portfolio companies, and vehicle language.
- Stage 2 — Claude refinement (only if <2 funds found): send the top passages to Claude for additional entities.
Windowed extraction (all 15 dimensions)
Tile the entire narrative into overlapping 60K-char windows (6K overlap) and scan every window for all 15 dimensions via Claude. Nothing is truncated; evidence (verbatim sentences, key terms, financial figures) is merged and de-duplicated across windows. A cleanup pass re-extracts any practice still showing zero evidence.
Curated-web augmentation
OPTIONALPull curated CVC / Accelerator / Impact web pages and fold them into the evidence. Two parts:
- Scrape (
scrapesub-command → wrapscv_url_scraper.py): reads the URL workbook, fetches each page (requests/httpx + BeautifulSoup), and a Claude pass extracts structured CVC/accelerator/impact facts. - Merge (
analyze --with-scraped): the scraped full text is run through the same windowed all-15-dimension extractor (so it informs every dimension incl. all 7 objectives) and merged into the 10-K evidence before scoring. The result is taggedscraped_evidence_used: true.
Financial extraction + XBRL merge
One Claude pass extracts financial metrics from the financial section; then authoritative XBRL values (revenue, R&D, CVC portfolio value, buybacks, etc.) override the text extraction for all standard GAAP metrics.
Ensemble scoring (median + uncertainty)
All dimension evidence is sent together for scoring. With ENSEMBLE_SCORING_N > 1 (default 5),
scoring runs N independent passes on the same evidence (temperature=0); the
reported value per dimension is the median, with min/max/std as the
uncertainty. Ranks are recomputed from medians, the P×O matrix cells are medianed, and the
headline total is the median of per-pass totals (range stored; sum_of_medians
kept separately). Each pass uses a distinct cache slot, so first run draws fresh samples and reruns
reproduce them.
Investment & funds extraction
A dedicated synthesis call combines the merged evidence, the discovered fund descriptions, the scraper's
structured output, and the financial figures into a structured investment_and_funds field:
number of investments, fund name(s)/size(s), and investment preferences
(check size, stage, geography, technology) — with supporting quotes.
Persist the analysis
All of the above is assembled and written as the single structured record for the company.
Per-company visuals + ensemble report
Render the per-company charts. Two dashboard variants are produced when ensemble data is present.
Portfolio aggregation
Flatten every company's analysis.json into one table, then build the cross-company analytics
and the self-contained interactive dashboard.
The 15-dimension framework
| 4 Practices | 4 Activities | 7 Objectives |
|---|---|---|
| Corporate Venture Capital · Accelerators and Incubators · Venture Building · Venture Clienting | Mentorship · Business Services · Workspace · Events | Risk Management · Market Expansion · Access to New Technology · Develop Business Ecosystems · Social and Environmental Impact · Developing Entrepreneurial Culture and Capabilities · Financial Return |
Each dimension scores 0–100 (emphasis). Rubrics are conceptual guides, not vocabulary
checklists — the model scores the underlying activity regardless of wording. framework/ holds
detailed v1 definitions; framework_v2/ holds broader v1-paragraph definitions used by
run_batch_v2.sh.
How scores are computed — LLM vs. coded algorithms
A · Each dimension's emphasis score (0–100)
An LLM judgment, aggregated by code:
- Assemble (code) —
_scoring_prompt()packs, per dimension, the extracted evidence (verbatim sentences, key terms, financial figures), the rubric (scoring bands + brief description), and company context into one prompt. - Score (LLM) — Claude (
temperature=0) returns per dimension anemphasis_score0–100 (+ rank, description, top_quote, rubric_band, rationale). The number is the model's holistic judgment, anchored by explicit rubric bands: 20–39 incidental · 40–59 generic/enterprise-scale · 60–79 explicit/named/quantified · 80–100 comprehensive/strategic with metrics. No arithmetic maps evidence → score. - Ensemble (code) — the scoring call runs N times (default 5);
_aggregate_ensemble()reports the per-dimension median (statistics.median) with min/max/std as uncertainty. - Rank (code) — recomputed by sorting median scores within each group (practices 1–4, activities 1–4, objectives 1–7) — not taken from the LLM.
- Total (code) — headline = median of the per-pass totals;
sum_of_mediansalso stored.
B · Practice/Activity → Objective heatmap (P×O matrix)
A hybrid: LLM-judged cells where evidence is strong, deterministic theoretical alignment everywhere else.
- LLM cells — the scoring call also returns
practice_x_objective_matrix(8 practice/activity rows × 7 objectives, 0–100) = "how strongly each row serves each objective." The model typically fills only rows with strong evidence; ensemble medians these cells. - Code fallback fill (
cv_analyzer.py) — any row the LLM left empty/zero is filled from the rubric's fixedcross_objective_relevancemap (high/medium/low/none→70/40/20/0), scaled by the dimension's emphasis score:
So a 0-scored dimension still shows its theoretical alignment at 25% intensity; an 80-scored one at ~85%. Rows the LLM scored keep the LLM values; the rest get this computation.scale = 0.25 + 0.75 × (emphasis_score / 100) cell = relevance_number × scale
The cross_objective_relevance map is authored by the LLM once when
rubrics are built (cv_framework.py), then frozen in rubrics.json and applied
deterministically at scoring time.
C · Division of labour
| LLM (Claude, temperature=0) | Coded algorithms (Python, deterministic) |
|---|---|
| Evidence extraction — which sentences support each dimension (windowed passes) | Document fetch / SGML-HTML parsing / section extraction (regex) |
| Dimension scores (0–100) — the core judgment | Windowing — tiling text into overlapping blocks (_window_text) |
| P×O matrix cells it chooses to fill | BM25 + cosine retrieval — entity discovery only (cv_rag) |
Descriptions, top quotes, rationales; investment_and_funds synthesis; entity refinement |
Evidence merge/dedup across windows; ensemble aggregation (median/std) |
Rubric authoring — bands, indicators, cross_objective_relevance (one-time) |
Ranks (sort), totals (median/sum), P×O fallback fill |
| XBRL merge (authoritative financials), caching (md5), all charts (matplotlib) |
Bottom line: scoring and P×O alignment-where-evidence-exists are LLM judgments; aggregation (median across passes), ranking, totals, and the theoretical P×O fill for unscored rows are deterministic code.
Final outputs & files created
Caches (built once, reused)
| Path | What |
|---|---|
sec_cache/sec_filings.sqlite | Downloaded EDGAR filings (SGML), keyed by CIK + accession |
.cv_cache/<md5>.txt | On-disk Claude response cache (exact reproducibility); rubric builds + every API call |
outputs/<TICKER>_narrative.txt · _xbrl_financials.json | Preprocessed narrative + structured financials (shared by v1 & v2) |
outputs_v2/rubrics.json | Scoring rubrics (bands, indicators, brief descriptions) |
Per company — outputs_v2/<TICKER>/
| File | Contents |
|---|---|
analysis.json | The master record: scores, ranks, P×O matrix, evidence, ensemble, score_uncertainty, investment_and_funds, investment_entities, financials, scraped_evidence_used |
dashboard.png / dashboard_uncertainty.png | Per-company dashboard — plain, and with min–max ensemble error bars |
practice_x_objective_heatmap.png | Practice/activity → objective alignment matrix |
ensemble_scores.json · .png · _table.csv/.md | Ensemble samples, error-bar figure, and table |
<TICKER>_report.docx | Narrative Word report |
extraction_checkpoint.json | Cached merged evidence (resume aid) |
word_clouds/ | Per-dimension word clouds (only if --word-clouds) |
Curated web (optional) — scraped/<TICKER>/
| File | Contents |
|---|---|
result.json | Sources checked, doc links, structured extraction, cv_analyzer_input block |
<TICKER>_{CVC,Accelerator,Impact}.txt | Cleaned full text per source category (the merge input) |
Portfolio — outputs_v2/
| File | Contents |
|---|---|
master_dataframe.csv | One row per company — all dimension scores, ranks, descriptions, financials |
portfolio_dashboard.html | Self-contained (zero-CDN) interactive cross-company dashboard |
portfolio_report.docx | Portfolio Word report (heatmaps, benchmarks, "what works") |
portfolio_charts/ | Portfolio heatmap, score distributions, aggregate P×O matrix |
Command quick reference
Single company (full chain)
# 1) fetch + preprocess (Steps 1–2)
python cv_toolkit.py preprocess --ticker GOOGL --output outputs
# 2) [optional] scrape curated web (Step 6a)
python cv_toolkit.py scrape --ticker GOOGL
# 3) analyze: Steps 3–11 (windowed extraction, ensemble scoring, investment_and_funds, visuals)
python cv_toolkit.py --output outputs_v2 --rubrics outputs_v2/rubrics.json --ensemble 5 \
analyze --ticker GOOGL --company "Alphabet Inc." \
--file outputs/GOOGL_narrative.txt --with-scraped --force
# 4) ensemble table + figure (Step 11)
python ensemble_report.py outputs_v2/GOOGL
# 5) portfolio aggregation (Step 12)
python cv_toolkit.py --output outputs_v2 --rubrics outputs_v2/rubrics.json csv
python cv_toolkit.py --output outputs_v2 --rubrics outputs_v2/rubrics.json portfolio-report
Whole portfolio (batch — runs Steps 1–12 for every company)
bash run_batch_v2.sh --force # all companies, v2 framework → outputs_v2/
bash run_batch_v2.sh --ticker GOOGL --force # single company through the v2 batch
cv_config.py / CLI):
ENSEMBLE_SCORING_N (--ensemble N, 1 = off) ·
GENERATE_WORD_CLOUDS (--word-clouds, off by default) ·
--with-scraped (off by default) ·
EXTRACT_WINDOW_CHARS / EXTRACT_OVERLAP_CHARS ·
ENABLE_PROMPT_CACHE · pinned CLAUDE_MODEL = claude-sonnet-4-5-20250929.temperature=0 is greedy but not bit-deterministic at the API
level. Exact reproducibility comes from the .cv_cache response cache; the ensemble (Step 8)
quantifies the residual scoring uncertainty as error bars.Appendix — Future extensions (not implemented)
Core idea — separate perception from judgment
Today the LLM both reads the evidence and emits the 0–100 number (the judgment is opaque). The hybrid splits these:
- LLM = perception → extracts a few well-defined, bounded sub-signals per dimension.
- Code = judgment → a transparent formula maps those signals → 0–100 (auditable, reproducible, tunable).
Step 1 — LLM returns structured features, not a score
| Feature | Type | Meaning |
|---|---|---|
directness | direct / indirect / none | activity explicitly described vs. inferred |
specificity | named+quantified / named / generic / vague | named funds/programmes, $ figures present? |
materiality | strategic / moderate / incidental | centrality/scale to the business |
evidence_count | int (capped) | # distinct supporting sentences |
financials_present | bool / count | $ figures attached |
Categorical buckets are far more stable across runs than a free 0–100 number → less ensemble noise.
Step 2 — deterministic scoring function (weights in cv_config)
score = BASE[directness]
+ W_spec * LEVEL[specificity]
+ W_evidence * min(evidence_count, CAP)/CAP
+ W_fin * financials_present
score = clamp(score, 0, 100)
Every score then decomposes traceably, e.g.
CVC = 72 = direct(40) + named+quantified(20) + evidence 8/cap(8) + financials(4).
Step 3 — blend knob (keeps current behaviour available)
final = α · formula_score + (1 − α) · llm_holistic_score
α=1 → fully transparent/reproducible; α=0 → today's behaviour; in between → hybrid.
Plus an optional LLM "override flag" for genuine edge cases the formula misses.
Step 4 (optional) — calibrate the formula as an interpretable model
Replace hand-set weights with a small ordinal/linear model fit on a labelled sample (human scores, or current LLM scores as a starting target). LLM features → interpretable coefficients → score, making scoring a defensible, reportable model.
P×O matrix analog
The fallback fill already does this (relevance × emphasis-scale). The hybrid would make the
whole matrix formula-based for consistency:
cell = f(row emphasis, cross_objective_relevance, objective emphasis, evidence co-occurrence) —
removing the "LLM cells vs. coded cells" split.
Trade-offs
| Gains | Costs / risks |
|---|---|
| Full traceability (score → named contributions) | Must design + validate the feature set and weights |
| Reproducibility (deterministic formula; stable categorical features) | Features still come from the LLM (perception not eliminated) |
| Tunability / calibration against ground truth | A rigid formula can miss nuance — mitigated by the α blend |
| Comparability across companies (one scoring rule) | Extra LLM schema + a small validation harness |
Footprint when implemented: new extraction schema, a
score_from_features() in cv_analyzer, weights in cv_config, keep the
current LLM-score path behind the α blend, and a harness comparing formula vs. LLM scores.