Part IV, Edge and Risk Chapter 15

From Discretionary to Systematic

Every discretionary trade is a system. The question is whether the system is written down.

15.0Why this chapter exists

Discretionary trading and systematic trading are often presented as opposites. They are not. A discretionary trader makes a series of decisions according to internal rules; a systematic trader makes the same decisions according to written rules. The difference is whether the rules are accessible to inspection and validation.

Most discretionary traders would benefit from making more of their decisions explicit. Most systematic researchers would benefit from understanding the discretionary judgments embedded in their data preprocessing. The two camps are closer than they typically present.

The translation pipeline comes first. Rule extraction, parameterisation, walk-forward validation: a procedure for turning the things you do automatically when trading into things a backtest can evaluate. After that, the case for translation as an exercise in self-knowledge, even for traders who have no intention of going fully systematic. Then the honest limits, which deserve a chapter to themselves. Parameters decay. Regimes are not stationary. The cost of re-validation is real and accumulates. Finally, the hybrid architecture, which is what most working desks actually run: a systematic regime classifier and level scorer feeding a discretionary execution layer within the framework's playbook.

This chapter is the bridge between the discretionary frameworks of Parts I to III and the rigorous validation methods of Chapter 16.

15.1The translation pipeline

To convert a discretionary edge into a testable system:

Step 1, Articulate the rules

Write down, in plain English, the rules a discretionary trader follows for entry, sizing, and exit. Include the implicit rules, the ones the trader never says out loud:

"I take this setup only when the regime is X."
"I require at least Y confluence."
"I never take this in the first 15 minutes."
"I size up only when conviction is high."
"I exit if the level is broken on the wrong side."

The discipline of writing the rules exposes how many of them there are. Most discretionary traders find they have 15 to 30 implicit rules they were not consciously articulating.

Step 2, Parameterize

Turn each qualitative rule into a quantitative one. "Strong trend" becomes "KER > 0.6 or ADX > 25." "Confluent level" becomes "level score > X" (per Chapter 5). "First 15 minutes" becomes "bars in [09:30, 09:45) ET."

The parameterization makes each rule mechanical. The thresholds (0.6, 25, X) become parameters of the strategy.

Step 3, Backtest with realistic assumptions

Run the parameterized strategy on historical data with realistic assumptions:

Slippage: 1 to 2 ticks per round trip on liquid intraday strategies.
Commission: $2 to $4 per contract round trip.
Fill probability: limit orders fill only at favorable prices; market orders fill at the next tick beyond the bid/offer.
Data quality: no lookahead; entry only after the bar that triggered confirms.

The backtest produces a baseline performance estimate.

Step 4, Walk-forward validate

Train parameters on the first 70% of the data, test on the last 30%. Or use rolling walk-forward: train on a window, test on the next window, slide forward. The out-of-sample performance is the honest estimate.

Chapter 16 treats walk-forward and other validation methods rigorously.

Step 5, Stress test

Vary each parameter by ±20%. The strategy should degrade gracefully. If a 10% parameter change collapses performance, the strategy is overfit.

Step 6, Compare to discretionary

The systematic version's performance is a calibration of the discretionary trader's edge. If the systematic version is profitable but lower-Sharpe than the discretionary, the difference is the discretionary trader's tactical skill (skipping bad setups, sizing to conviction). If the systematic version is unprofitable but the discretionary is, the difference is either tactical skill or self-deception about discretionary performance.

The honest answer requires the trader's discretionary journal as comparison data. Few discretionary traders maintain this rigor; most who do are surprised to find their actual edge is smaller than they believed.

15.2Why translate?

A discretionary trader who has no intention of becoming fully systematic still benefits from the translation exercise:

Benefit 1: It exposes implicit assumptions

Every implicit rule that the translation forces explicit is a rule the trader can now examine. Some will be sound; some will be old habits that no longer apply; some will be biases. The audit is valuable independently of whether the system is deployed.

Benefit 2: It reveals overconfidence

The translation typically reveals that the trader's "high-conviction" setups have lower conditional probability than the trader believed. The data is humbling and useful.

Benefit 3: It enables stress testing

A trader who knows their setup performs at 60% in Range-Calm and 40% in Range-Vol can size accordingly. A trader who lumps both into "this setup works" cannot.

Benefit 4: It supports regime-conditional sizing

Translating to systematic exposes the regime-conditional performance, which is the basis for regime-conditional sizing. A discretionary trader who sizes uniformly across regimes is leaving money on the table.

Benefit 5: It provides a default

A systematic version of the strategy can be the default execution; the discretionary trader overrides only when they have specific reason. The discipline reduces the variance of execution and surfaces the cases where discretion adds value.

15.3Common patterns in discretionary-to-systematic translation

Patterns that recur in the practitioner literature:

Pattern 1: The "I only take this when..." cascade

A discretionary trader's rules often include 3 to 5 conditional gates. Translating each into a parameter and requiring all to be true at once produces a setup that fires far less frequently than the trader expected. The frequency drop is itself informative; it suggests the trader takes setups that don't actually meet all the implicit conditions.

Pattern 2: The "execution" gap

The trader's rules describe entries and exits, but the live execution differs. The systematic version captures the rules; the discretionary version is what actually gets done. The gap is the trader's psychology and execution skill, both as positive and negative contributors.

Pattern 3: The "regime shift" awareness

Discretionary traders often have implicit regime awareness ("this doesn't feel like a trend day") but the awareness is post-hoc. Translating to systematic and using a formal regime classifier (Chapter 2) is what makes the awareness real-time.

Pattern 4: The "high-frequency drift"

Discretionary traders often take more trades than their setup justifies. The systematic version trades less; this can feel uncomfortable but is correct. Lower trade frequency on cleaner setups is the institutional norm.

Pattern 5: The "sizing drift"

Discretionary sizing varies with mood, conviction, and recent P&L. The systematic version uses a fixed formula. The systematic version's Sharpe is often higher despite the discretionary version's higher win rate, because the discretionary sizing erratically rewards lucky moments.

15.4The systematic researcher's reverse problem

Researchers who start systematic face the reverse problem: their data preprocessing and "obvious" choices contain implicit discretionary judgments.

Examples

"I'll use 5-min bars" is a choice. Why not 1-min? 15-min? The choice affects everything downstream.
"I'll exclude data from before 2020" is a choice. Why? What regime did that pre-2020 data represent?
"I'll define a swing as N=3 ATR-conditioned" is a choice. The N and the multiplier are discretionary.
"I'll use the trailing 30-day window for percentile" is a choice. 14? 60? Same.

Each choice is reasonable; collectively they define the strategy as much as the explicit rules do. A systematic researcher who is unaware of how many discretionary judgments are baked into the preprocessing is overconfident in the strategy's purely-mechanical character.

The cure

Acknowledge the discretionary judgments. Sensitivity-test them. A strategy whose Sharpe is robust across reasonable variations of the preprocessing choices is genuinely systematic; one that requires the specific choices is partly discretionary regardless of what the algorithm looks like.

15.5The hybrid approach

The most institutionally common architecture: systematic for the regime classification and signal generation; discretionary for tactical execution within the regime's playbook.

Example architecture

Systematic regime classifier: the composite from Chapter 2 produces a regime label every bar. The classifier is mechanical, validated, and runs on every session.
Systematic level set: the quality score from Chapter 5 ranks horizontal levels. The top 5 to 10 levels for the day are pre-computed.
Systematic open-type classifier: the open type is locked by 09:45 ET per Chapter 13. The framework is selected by 10:30.
Discretionary tactical execution: within the framework, the trader applies discretion for entry timing, scaling, and exit. The discretion is bounded by the framework's structure.
Systematic risk management: position size from the formula in Chapter 17; per-session loss cap is hard.

The architecture preserves the discretionary trader's tactical skill while mechanizing the regime and risk decisions. It is the configuration most professional desks actually run.

Why this works

The hard parts of trading (regime classification, level identification, sizing discipline) are systematic; the parts that benefit from human judgment (reading the tape in real time, adjusting to breaking news, sensing exhaustion) are discretionary. Each part of the architecture plays to its strength.

The discipline

The hybrid requires the discipline to not override the systematic regime classification when intuition disagrees. Most failures of hybrid systems come from discretionary overrides of the systematic regime, where the trader convinces themselves the regime is different from what the classifier says. The classifier is right more often than not; ride it.

15.6Edge decay and the case for ongoing validation

Even validated systems decay. Markets adapt; participants discover edges; competitors build similar systems; the edge erodes.

Examples of edge decay

The Monday effect (Mondays returned less than other weekdays) was robust in equity research from the 1970s through 1990s. By 2000s, the effect was weak. By 2020s, gone.
The classical trend-following Sharpe of futures CTAs was 0.6 to 0.8 in the 1990s on standard parameters. By the 2010s it was closer to 0.3 to 0.4 with much higher capital deployed.
The post-earnings drift in equities was a durable factor for decades; by the 2010s, much of it had been arbitraged out.

The pattern is common: edges that were durable for 10 to 30 years decay over similar timeframes. New edges replace them, but the assumption that any specific edge will work indefinitely is wrong.

Implications for systematic strategies

Re-validate periodically (every 6 to 12 months, or after any major market regime shift).
Track strategy performance against expected; if performance degrades materially, investigate whether the edge is decaying or whether the regime simply doesn't favor the strategy this year.
Maintain multiple strategies with different edge sources; correlated decay across strategies is the danger sign.
Allow strategies to be retired. A strategy that has stopped working should be turned off, not "given another month to recover."

15.7The honest limits of systematic trading

For the trader considering full systematization, the limits:

Limit 1: Parameter selection is curve-fitting risk

Every parameter chosen is a degree of freedom that may be optimized in-sample. The more parameters, the higher the curve-fit risk. Strategies with 3 to 5 parameters have manageable curve-fit risk; strategies with 15 are almost certainly overfit.

Limit 2: Walk-forward results decay

Walk-forward backtests are the gold standard, but even they overestimate live performance because:

The data was selected (you know it exists).
The strategy was selected from a larger pool of trial strategies.
Implementation differs slightly from the backtest (slippage, fills).
Markets adapt during the backtest period; future performance is conditional on regime persistence.

The realistic guideline: live performance is typically 50 to 80% of walk-forward Sharpe in the first 6 to 12 months. Plan for the lower bound.

Limit 3: Black-swan exposure is hard to size

Strategies that look good on backtests often have hidden tail risk. The 2008 crisis, the 2020 COVID crash, and other extreme events frequently produced losses 2 to 5x larger than the backtest's worst observed loss. Sizing for the backtest's worst loss is sizing for known risk; sizing for unknown risk requires reducing leverage further.

Limit 4: Implementation drift

A strategy that runs reliably for 6 months may fail not because the edge decayed but because of platform changes, broker changes, data feed changes, or regulatory changes. The implementation has its own life cycle.

Limit 5: Capacity

A strategy that works on $100K may not work on $10M because the slippage at scale is materially different. Capacity is a real constraint and one that backtests rarely capture well.

15.8The recommended path

For a discretionary trader becoming more systematic:

Start with rule articulation. Write down every rule. Audit. (Weeks 1 to 2.)
Build a regime classifier. Use the composite from Chapter 2 as a starting point. Validate on your own contract. (Weeks 3 to 4.)
Build a level scorer. Use the framework from Chapter 5. Calibrate base weights to your contract. (Weeks 5 to 6.)
Translate one setup to systematic. Pick the highest-confidence setup. Parameterize it. Backtest. Walk-forward. (Weeks 7 to 12.)
Run the systematic version alongside discretionary. Track both. Compare. (Months 3 to 6.)
Iterate. Refine the systematic version based on live experience. Add additional setups as they validate. (Months 6+.)

The path is not 6 months to fully systematic; it's 6 months to one validated systematic component. Each subsequent component takes similar time. Building a full systematic stack typically takes 1 to 3 years of part-time work.

The reward is durable: a validated systematic regime classifier is a capability that compounds, and the discipline of building it changes how the trader sees markets even on days they trade discretionarily.

15.9Failure modes specific to the transition

Premature deployment. A backtest looks good; the trader deploys live without walk-forward. Performance disappoints. Always walk-forward before live.
Insufficient sample size. A strategy validated on 50 trades is statistically underpowered. Most strategies need 200+ trades for the metrics to be reliable.
Backtest contamination. Building the strategy and validating it on the same data. The fix: hold out a strict out-of-sample window the strategy never sees during development.
Survivorship in strategy selection. Building 100 strategies, picking the 5 that backtested well, deploying those. The 5 are by selection-bias the lucky ones; expect them to underperform. Bonferroni correction or Deflated Sharpe Ratio (Chapter 16) handles this.
Over-confidence in walk-forward. Walk-forward is the floor of validation, not the ceiling. Even perfect walk-forward overestimates live performance for the reasons above.
Discretionary override fatigue. The trader cannot resist overriding the system. Over time, the overrides accumulate, and the live strategy is neither the system nor pure discretion. Discipline matters.

15.10The integrated stack treatment

Systematic infrastructure is Layer 9 of the institutional stack: structural levels, profile, AVWAP, liquidity, OF, regime composite, open type, cross-market, and now systematic infrastructure as the discipline that ensures consistent execution of all the above.

The hybrid architecture is the practical default: systematic regime classifier and level scorer feeding discretionary execution within validated frameworks.

15.11Diagram concepts referenced in this chapter

D15.1: Translation pipeline flowchart. Six stages: articulate, parameterize, backtest, walk-forward, stress-test, compare. Each stage with characteristic inputs and outputs.
D15.2: Hybrid architecture diagram. A box-and-arrow diagram showing systematic components (regime, level scorer, open type, sizing) feeding a discretionary execution box that produces orders.
D15.3: Edge decay timeline. A long-horizon chart showing the decay of well-known historical edges (Monday effect, basic trend following Sharpe) over decades.
D15.4: Walk-forward vs live performance. Hypothetical equity curves: walk-forward is steady upward; live is similar shape but lower slope.
D15.5: Parameter sensitivity heatmap. A 2D grid of Sharpe ratio over two parameters; a robust strategy shows a plateau, an overfit strategy shows a peak.

15.13Exercises

Exercise 15.1: Rule articulation. Write down every rule you follow when taking your most-common setup. Aim for 15+ rules; if you stop at 5, you are missing implicit ones. Have a fellow trader read the list and identify gaps.

Exercise 15.2: Translation of one setup. Pick a single setup and translate it from discretionary to systematic. Parameterize each rule. Backtest on the most recent 100 instances. Note the in-sample win rate and average R.

Exercise 15.3: Walk-forward validation. Take the systematic version from 15.2. Split data into train (first 70%) and test (last 30%). Re-fit parameters on train. Test on test. Compare in-sample and out-of-sample performance.

Exercise 15.4: Parameter sensitivity. Vary each parameter of your systematic strategy by ±20%. Plot Sharpe vs each parameter. The plot should show a plateau, not a peak. If a peak, the strategy is overfit.

Exercise 15.5: Hybrid execution audit. For one trading week, run both your systematic and discretionary versions in parallel. Compare trade-by-trade. Where did discretion add value? Where did it subtract?

Next chapter: statistical validation, walk-forward, parameter robustness, regime stratification, multiple-testing correction, the rigorous foundations beneath the methods of this chapter.