Delegation in EM allocation research

For each stage of EM allocation research, which level of agent autonomy maintains acceptable research quality while minimising analyst effort and cost?

Estimate the effect of assigning decision authority at a stage, conditional on its permitted choices and input difficulty. Then validate selected combinations across the complete workflow. No assumption that autonomy is beneficial or that one configuration is universally best.

At a fixed month-end cutoff, recommend country and sector tilts around one broad EM equity benchmark over three months. USD reporting, fully invested, long-only; no leverage, derivatives, stock selection or live trading. Use six countries selected by a rule fixed at study start, plus Other EM, and fixed sector groups. Retain underlying constituent detail.

Supply a frozen archive with point-in-time benchmark weights, prices/total returns, earnings estimates and dated documents. Permit only information published by the cutoff, using the vintage available then. Freeze universe, data access, implementation conventions and hard portfolio limits across conditions.

Let i=(c,s), b_i benchmark weights, w_i^- pre-trade drifted weights and w_i proposed weights. Country and sector totals are sums of the same cell matrix.

Illustrative features: valuation V_i = current earnings yield minus its preceding 60-month mean; revisions E_i = weighted constituent consensus-EPS percentage revisions over three months for the same fiscal-year target; momentum M_i = \frac{TR_i(t-1)}{TR_i(t-12)}-1 in USD. Fix EPS denominator, coverage and aggregation definitions during calibration.

Standardise eligible features using date-t cross-sectional statistics: z_{ji}=\frac{x_{ji}-\mu_j}{\sigma_j}. Apply the selected permitted outlier/missing-data policy. Score S_i=\sum_j\alpha_j z_{ji}, with \alpha_j\geq0 and \sum_j\alpha_j=1.

Target q_i=b_i[1+\eta(S_i-\sum_m b_mS_m)]. Code solves w^*=\arg\min_w\{\sum_i(w_i-q_i)^2+\gamma C(w)\} subject to hard limits. Scores are not expected returns. Code performs all calculations and enforces constraints at every autonomy level.

Baseline \alpha=(\tfrac13,\tfrac13,\tfrac13). Alternative parameter choices come only from a finite menu calibrated beforehand. This baseline does not train a deep learning model.

w_i\geq0; \sum_i w_i=1; b_i=0\Rightarrow w_i=0. Illustrative country and sector deviations ≤3 percentage points; cell deviations ≤1.5 points. Active allocation A=\tfrac12\sum_i|w_i-b_i|\leq0.10. One-way turnover \tau=\tfrac12\sum_i|w_i-w_i^-|\leq0.10.

Trading cost C=k\sum_i|w_i-w_i^-|=2k\tau. Illustrative k=10 bps per dollar bought or sold; sensitivity k\in\{5,10,25\} bps. At \tau=10\%, C=2 bps of portfolio value.

R_{\mathrm{net}}=(1-C)(1+R_{\mathrm{gross}})-1 for one rebalance; compound net subperiod factors for multiple rebalances. Exclude market impact, taxes, FX conversion and within-cell trades. Report gross/net results. Fixed-allocation cost sensitivity and re-optimised cost scenarios are separate analyses.

No autonomy level may change these hard limits, information cutoff or trading permissions. Test feasibility and define common fallback rules before runs.

L0: Human-directed. The analyst selects actions and assumptions; the LLM executes requested tasks. Record all active steering time. This is a live human-directed comparator, distinct from a fixed scripted workflow.

L1: Approval required. The agent investigates and recommends a choice; a human must approve or revise it before downstream use.

L2: Conditional delegation. The agent applies choices within the permitted menu, but predefined observable triggers require escalation before proceeding.

L3: Bounded autonomy. The agent selects and applies permitted choices without discretionary stage approval. Hard validators remain active; final portfolio sign-off remains human.

Keep decision menus, LLM, tools and available information identical across levels. At L2 freeze escalation triggers before evaluation; do not rely solely on the LLM’s self-reported confidence. Record pre-sign-off and final outputs to measure the burden shifted to final review.

S1. Evidence selection: choose sources, retrieval order and follow-up investigations within the dated archive. Escalation candidates: conflicting sources, missing provenance or inadequate coverage. Audit relevance, provenance and missed material evidence.

S2. Input validation: retain, replace from an authorised source, or mark an observation missing. Require evidence for changes; preserve original records. Escalation candidates: unit/currency ambiguity, fiscal-period mismatch or material discrepancies. Audit corrections, false changes and missed errors.

S3. Feature construction: choose from prespecified coverage, outlier and missing-data policies and approved lookback variants. For example, compare 5th/95th-percentile capping with an uncapped policy; require a coverage floor under every option. Audit leakage, consistent application and sensitivity. Exact choices and thresholds are illustrative and frozen before evaluation.

S4. Allocation assumptions: choose a score-weight vector and tilt/cost-penalty settings from a small calibrated menu. Include equal weights and a few justified alternatives, never unrestricted parameter search. Escalation candidates: unstable allocations across approved settings or a large change from the baseline. Assess justification and robustness; there is no uniquely correct parameter vector.

S5. Interpretation and recommendation: choose supporting evidence, counterarguments, scenarios and whether to recommend the calculated allocation or abstain. No manual weight editing. Escalation candidates: unsupported material claims or conclusions inconsistent with computed results. Audit accuracy, coverage, uncertainty and reviewer effort.

At every stage, apply L0–L3 to the same decision rights. The level changes who decides and when approval occurs, not the available menu. Fix materiality and trigger thresholds in calibration.

For each decision record: case/stage, assigned level, owner, input snapshot, available options, selected action/assumption, evidence, trigger status, human intervention, elapsed/active time and downstream effects.

Save the initial proposal, post-stage decision, pre-final-review portfolio and signed-off output. Log accepted, rejected, revised and unresolved decisions. Use the same final-review rubric and cap across workflows. An unresolved material issue leads to abstention or the predefined fallback, not automatic approval.

Operational analysts follow assigned authority. Independent evaluators assess outcomes without seeing the assigned level where feasible. They do not supply answers during runs. Avoid reviewers seeing paired variants of the same case; otherwise counterbalance order and report carry-over. Include final review and escalation time in total human effort.

Create identical incoming snapshots for each stage from audited upstream outputs. Run all four autonomy levels against the same decision packet, freezing downstream settings when measuring portfolio impact. This separates local decision quality from upstream errors.

Use a small case set covering routine, ambiguous and stressed inputs. Define these categories through observable characteristics before testing. Natural cases are primary; controlled injected-error cases can supplement them and must be reported separately.

Assign analysts to conditions/cases in a balanced randomised schedule, controlling for experience where possible. Keep machine ceilings equal and measure human time rather than artificially making it equal. Compare each level with L0 on paired case-stage outcomes. The intervention is the complete oversight policy, including its escalation rules.

Stage results do not reveal all interactions. Use development cases to select a small set of policies a=(a_1,\ldots,a_5), where a_s\in\{0,1,2,3\}. Candidate policies: all-human-directed, approval at every stage, conditional delegation, and a mixed policy selected from the stage map. Include bounded-autonomy stages only where evidence justifies testing them.

Freeze candidate policies before new held-out cases. Run the complete pipeline, preserving upstream errors and their downstream consequences. Compare quality, total human time, cost and final-review burden. Do not infer that individually acceptable stages necessarily form an acceptable combined workflow.

Keep a fixed scripted LLM workflow as an optional resource baseline, labelled separately from L0 live human direction. Avoid testing all 4^5 combinations.

Quality: independently verified input corrections, false changes, missed material errors, citation support/coverage, assumption consistency, robustness and portfolio feasibility. Use audited dated data for checkable facts and a blinded rubric for judgement-dependent assumptions. Report evaluator disagreement.

Human effort H=T_{\mathrm{steering}}+T_{\mathrm{approval}}+T_{\mathrm{escalation}}+T_{\mathrm{review}}+T_{\mathrm{correction}}, measured in active minutes. Report waiting time and wall-clock latency separately. Record model/tool cost K, token use, failures and abstentions.

For stage s and autonomy level l, estimate Q_{s\ell}, material-error rate E_{s\ell}, human minutes H_{s\ell}, cost K_{s\ell} and escalation rate G_{s\ell}. Save pre-review and post-review values. Error reduction after review is an observed review change, not automatically a causal estimate of human involvement.

Secondary: deviation from the original portfolio, turnover, net benchmark-relative returns and risk. An allocation different from an analyst’s preference is not itself an error.

Produce a stage × autonomy-level map with final quality, material-error rate, human minutes, cost and uncertainty in each cell. Show separate maps for case difficulty only when sample size permits.

Paired time saving versus human direction: \Delta H_{s\ell}=\operatorname{mean}_d(H_{s0d}-H_{s\ell d}). Paired quality difference: \Delta Q_{s\ell}=\operatorname{mean}_d(Q_{s\ell d}-Q_{s0d}). Positive values mean time saved and quality improved, respectively. Aggregate repeated runs within case before estimating uncertainty.

Define acceptable delegation using prespecified criteria: \operatorname{LCB}(\Delta Q_{s\ell})\geq-\delta_Q and \operatorname{UCB}(E_{s\ell})\leq\varepsilon, plus hard-constraint compliance. \delta_Q is the tolerated quality loss and \varepsilon the material-error ceiling, agreed with practitioners before evaluation. Use intervals appropriate to the metric and case/analyst clustering.

Among acceptable options, report the Pareto frontier: configurations for which no alternative is at least as good on quality, human time and cost and strictly better on one. Avoid collapsing all outcomes into an arbitrary score.

For a chosen cost budget B, an illustrative policy is \ell_s^*=\arg\min_\ell H_{s\ell} subject to acceptability and K_{s\ell}\leq B. Validate the combined policy on held-out cases. If no level qualifies or uncertainty is too wide, label the boundary unresolved. Do not assume quality changes monotonically with autonomy.

Illustrative feasibility pilot: 6\text{ cases}\times5\text{ stages}\times4\text{ levels}=120\text{ trials}. Use it to estimate variance, review workload and escalation frequency, not to certify delegation boundaries. Add repeated runs selectively to measure model variability.

Historical validation example: 3\text{ policies}\times6\text{ dates}\times2\text{ repeats}=36\text{ runs}. Select one policy using prespecified quality, effort and cost criteria, then freeze it for the prospective phase. Sample sizes depend on reviewer capacity and the precision needed for \delta_Q and \varepsilon.

Illustrative stage ceiling: 5 minutes and 10 tool calls; workflow ceiling: 20 minutes and 40 calls, plus a calibrated monetary cap. Set separate review caps during calibration. Freeze common budgets within each experiment; pilot failures count in results.

Separate calibration, policy-selection and held-out evaluation cases. Preregister primary measures, contrasts and thresholds. Report uncertainty and multiple-comparison limitations. Distinguish case-level variation from stochastic repetition; human repetitions also reflect reviewer variability.

Question: does the frozen selected policy maintain the prespecified research-quality requirements while reducing total analyst effort on unseen live cases, relative to a fixed oversight baseline? Validate the complete workflow, including upstream error propagation and final-review burden.

Before the first case, freeze stage autonomy levels, escalation triggers, decision menus, feature/model parameters, hard constraints, resource/review budgets, quality thresholds, schedule and analysis plan. Use the same LLM version where available. Log unavoidable model/data changes and report affected cases separately. Protocol changes start a new labelled phase rather than revising earlier results.

Run the selected policy and a prespecified approval-at-every-stage baseline on identical timestamped live information snapshots. Use the same numerical engine, permitted choices and machine ceilings. Balance analyst assignment and review order; avoid the same analyst seeing both paired versions where feasible. Count all steering, escalation, approval, correction and final-review minutes.

Use paper portfolios only. Start both from the same benchmark holdings, then track each policy’s own holdings, drift, rebalance trades and costs under the fixed implementation rules. Timestamp recommendations before outcomes are known; record next eligible implementation prices. Preserve original proposals, interventions and signed-off allocations. No real capital is required.

Assess quality promptly against cutoff-compliant evidence using independent audits and blinded review where feasible. Compare paired quality, material errors, total human minutes, cost, latency and failures. Apply the same acceptability criteria as the delegation map; if uncertainty is too wide, report prospective feasibility evidence rather than claiming a confirmed boundary. Track whether early time savings merely shift work into final review.

Illustrative schedule: three monthly recommendation rounds, each with a three-month horizon, require roughly five months from the first decision to the final outcome. For overlapping horizons, maintain separate paper-portfolio cohorts and charge each cohort’s required trades. Treat overlapping returns as dependent. This is a small operational pilot, not a powered investment-performance test.

Research-quality and effort outcomes are available immediately after review; portfolio outcomes mature later. If the MSc deadline precedes the final horizon, report completed research outcomes and only matured return results, with a prespecified follow-up. Do not shorten horizons or stop early because returns look favourable. Record deviations, abstentions and policy failures.

Deliver: decision-rights specification, stage/autonomy map with uncertainty, quality–effort–cost frontiers, tested mixed policies, escalation rules and a prospective validation report. Call the final configuration the selected policy, not a universally optimal rule. Recommendations are conditional on task, difficulty, analysts, permitted choices and observed period.

Sequence: agree mandate and quality tolerances; build archive/engine/decision menus; calibrate; run stage trials; validate complete policies on held-out historical cases; select and freeze one policy; run prospective paired cases; report research outcomes and subsequently matured portfolio outcomes.

Historical cutoffs cannot remove future knowledge embedded in LLM training. Prospective timestamped decisions reduce this concern for subsequent events but do not eliminate data contamination or operational errors. Report coverage, model changes, reviewer effects and sample-size limits. Assumptions must use permitted evidence, never future returns. Deep learning or unrestricted feature discovery requires a separate experiment.