FinEvolveKG Constrains Prompt Evolution for Auditable Financial Knowledge Graphs

Summary:FinEvolveKG addresses knowledge-graph construction from shared SEC 10-K filings for financial risk analysis, portfolio management, and investment research. Its main design choice is to separate an approved general prompt block from independently versioned persona-policy blocks, while mechanically restricting GEPA's reflective optimizer to the designated block. On ticker-disjoint 10-K splits, the complete two-stage workflow improves over a frozen prompt by 19.4–30.9% and over the tested unconstr

---

Core Idea: Treat Prompts as Evolvable Engineering Assets

FinEvolveKG does not allow GEPA to optimize a monolithic prompt containing every rule. Instead, it divides the prompt into two areas with different responsibilities:

• A general block that carries entity grounding, typing constraints, temporal relations, and baseline KG-quality rules;
• A persona policy block that encodes role-specific focus areas, facts, motifs, and relational preferences.

The optimizer may edit only the persona policy block. The approved general block remains byte-stable throughout persona adaptation and is versioned separately from the persona policy. In effect, the system treats prompt evolution like a scoped code change: an optimizer mistake cannot silently rewrite the institution's ontology, temporal semantics, or entity-alignment rules.

The dual objective jointly measures generic temporal-KG quality and persona-relevant facts, motifs, and multi-hop connectivity. The goal is not to create isolated data silos for each persona. It is to reuse a common grounding and temporal foundation while applying controlled policy changes for different analytical roles.

Mechanism: From Unconstrained Prompt Search to Bounded Editing

Let the full prompt be:

[ P = P_g \oplus P_p ]

where (P_g) is the general block and (P_p) is the persona policy. FinEvolveKG permits the optimizer to propose (\Delta P_p), while enforcing:

[ P_g' = P_g, \qquad P' = P_g \oplus (P_p + \Delta P_p) ]

The primary benefit is governance rather than architectural novelty. General rules cannot drift as personas are optimized, and each organization can independently approve and roll back persona policies. Every change is localized and versioned, allowing reviewers to inspect a defined diff rather than infer what changed inside a monolithic natural-language prompt. Multiple roles can also share one grounding and temporal specification, reducing contradictory graph policies.

A mechanical boundary does not, however, guarantee semantic correctness. GEPA may still introduce selective disclosure, over-extraction, or incorrect relations for a particular persona. A production pipeline therefore still needs schema checks, entity normalization, temporal-consistency validation, and human review. The SME audit supports using generated triplets as reference ground truth, but it does not establish that the automated pipeline can operate without governance.

Evaluation: Leakage Control, but Still a Limited Scope

The experiments compare five mutation scopes on ticker-disjoint 10-K splits across three personas. The persona-conditioned variants v3, v4, and v5 each use three random seeds. v4 is the complete two-stage block-constrained workflow, while v5 permits edits to the full prompt.

Across three-seed means, v4 improves over the frozen prompt v1 by 19.4–30.9% and over the tested unconstrained full-prompt workflow v5 by 2.2–6.4%. It leads v5 on both objective components, wins all nine paired-seed comparisons, and has the lowest seed variance for every persona. Without further optimization, v4 also exceeds v5 across all evaluated OOD shifts.

Ticker-disjoint splits reduce the risk of company-level entity and event leakage, making this evaluation more credible than a random document split. Several limitations nevertheless remain:

1. Three personas and a limited set of tickers do not cover investment banking, asset management, short selling, or compliance auditing;
2. The two objective components may share information and model outputs, while the abstract does not specify normalization, absolute scores, or significance tests;
3. The 2.2–6.4% margin over v5 requires confidence intervals and repeated experiments to establish practical stability;
4. The paper explicitly avoids attributing the gains causally to the boundary. Improvements may also arise from a different optimization budget, a smaller search space, or task-structure differences.

The evidence therefore supports constrained prompt evolution as a more auditable engineering pattern, but not as a universally superior method across LLMs, optimizers, and financial documents.

Human Audit: Better Reviewability Is Not Verification

An SME audit supports treating generated triplets as ground truth. In blinded, contrast-enriched judgments, two SMEs favored v4 over v5 in 70.1% of cases.

Human review can catch issues missed by automatic metrics, such as whether temporal relations make financial sense, whether entities and events are correctly linked, and whether a persona policy genuinely supports its intended research role rather than merely increasing formal multi-hop connectivity. However, 70.1% is not unanimity, and the abstract does not disclose sample size, confidence intervals, disagreement resolution, or the full protocol. Two experts' judgments are useful complementary evidence, not a substitute for broader annotation and replication.

Deployment: Put Prompt Versions into Knowledge Governance

The direct implication for financial institutions is that persona adaptation should not require copying an entire prompt stack. It should preserve a stable base and isolate a mutable policy. A practical deployment can use the following structure:

• Store the general block under code review and require approval from risk, legal, and data teams;
• Record prompt diffs, GEPA traces, objective scores, and rollback points for every persona policy;
• Validate merged outputs for entities, temporal relations, types, and source-text support;
• Compare in-distribution and OOD performance before each release;
• Route high-risk triplets to human review while retaining evidence links to the original 10-K filings.

A higher persona-policy score must not be interpreted as proof that the extracted facts are more complete or more correct. An optimizer may simply produce graph structures that better match role preferences. Robust deployment still requires an independent labeled set, temporal splits, ablation studies, and replication across different LLMs.

Engineering Trade-offs and Conclusion

FinEvolveKG's contribution is not a new model architecture. It redefines the mutation boundary of prompt optimization: trading some unconstrained search freedom for rule stability, change traceability, and shared infrastructure across personas. This trade-off is attractive in finance, compliance, continual learning, and agentic workflows where auditability matters.

The limitations are equally clear: absolute gains, external generalization, expert-audit protocol, and the cost of the extra optimization stage remain underreported. Whether the constrained method is intrinsically more robust or merely closes an optimization-budget mismatch with v5 requires equal-compute and equal-call-count ablations. Overall, this is a practical and governance-oriented prompt-engineering contribution rather than a demonstrated universal advantage of autonomous prompt evolution.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-11-09)

— END —

Tags: #FinEvolveKG Follow #GEPA Follow #Prompt Optimization Follow #Financial Knowledge Graphs Follow #SEC 10-K Follow