Case History:  Using Design of Experiments Technology to Improve the HR Trading System

 

Ronald Schoenberg, Ph.D.

Trading Desk Strategies, LLC

 

Here at Trading Desk Strategies we’ve developed a trading system based on options on the S&P 500.  The basic unit is a spread we call the HR spread.  The spread is opened with a specified credit, usually $40,000, and is neutral with respect to short and long options.  Profits are made when the short options are bought back for less than we sold them.  The sale of short options generates excess long options which are held for protection against large movements in the market.  Both call and put HR spreads are opened creating, at least initially, a risk profile of a short strangle with a wide profit plateau in the middle where all options expire out of the money and we keep the remaining credit.   The actual progress of the portfolio over time is somewhat more complicated however because as the market moves up and down short positions are bought back and more spreads opened creating a complex arrangement of short and long call and put options. 

A computer program was written using C# with the GAUSS Run-Time Module [1] for statistical and mathematical calculations to manage the trading of the HR spreads.   An option chain, a table of calls and puts for all currently available strikes and expires, is downloaded in real time each minute of the trading day and processed for eligible spreads.  If any is found that satisfies a rather complex set of requirements, a signal is issued.   For paper-trading the signals are maintained in a database with a paper account.  For actual trading, the signal would be sent to a trader.

Over ten thousand candidate spreads are scrutinized in every option chain.  Market conditions are also measured and analyzed.  Whether or not a signal is issued depends on a large set of parameter settings.   Initially these settings were determined by what we call “brute force back-testing”.  Many simulated runs over many expiries were conducted with a variety of settings looking for the best overall outcome. 

What emerged from this early testing of the model was that it was going to be difficult to find consistent settings with profitable outcomes in all types of market conditions.  The overall outcome tended to be small with settings that worked well in one type of condition but poorly in another condition. 

We then decided to apply Design of Experiments (DOE) technology (http://en.wikipedia.org/wiki/Optimal_design) to our trading model.  This would have the following beneficial effects:  first, we seek to find optimal parameter settings for each market condition, allowing us to see whether any of them can be found to vary by some market measure such as realized volatility.  Second, we might find some set of parameters that produce good outcomes over all market conditions. 

 

Design of Experiments

To start we select “factors”, here the parameters we want to investigate.  Our earlier back-testing showed the outcomes to be generally insensitive to many of them.  We settled on four parameters, two involving spread selection,

And two involving exit management

Next we set a region of interest by establishing minimums and maximums for each of our factors.  They are [.5, 3] for the selection factors, and [.1, 1] for the exit management factors.

Now a design matrix is generated. The design matrix is a set of trials with parameter settings chosen to satisfy a statistical criterion.   We choose the I-Optimal criterion.  This criterion minimizes the prediction variance over the region of interest and has been shown generally to produce better results than other criteria.  We must also choose the type of polynomial we’ll be using for the response surface analysis.  There’s a trade-off between the order of the polynomial and the number of trials.  For the initial study we’ll choose a cubic model.  We expect the response surface to be somewhat complex and the cubic model provides us with more opportunity to explore that surface.  It does entail 35 trials but the greater elasticity of the response surface will be worth the effort.

Using the computer program Gosset (http://www.research.att.com/~njas/gosset/) based on methods developed by Hardin and Sloane [1], the following design matrix was generated based on the specifications described above:

1

               1.000

               1.000

               2.751

               0.500

2

               0.260

               0.345

               3.000

               1.041

3

               0.100

               1.000

               1.265

               3.000

4

               0.211

               0.620

               1.163

               0.546

5

               0.320

               1.000

               2.010

               1.069

6

               0.716

               0.348

               3.000

               2.635

7

               0.671

               0.662

               2.520

               0.500

8

               0.100

               0.591

               0.500

               3.000

9

               0.383

               0.907

               0.692

               2.278

10

               1.000

               0.357

               2.406

               1.306

11

               0.100

               0.100

               3.000

               3.000

12

               0.100

               0.226

               1.099

               1.844

13

               0.575

               0.100

               0.633

               2.974

14

               1.000

               1.000

               0.500

               3.000

15

               1.000

               0.327

               0.500

               0.500

16

               0.269

               0.332

               2.000

               3.000

17

               0.672

               1.000

               0.564

               0.500

18

               1.000

               0.837

               0.996

               1.157

19

               0.100

               0.104

               2.332

               0.500

20

               0.417

               0.100

               2.417

               2.066

21

               0.707

               0.776

               1.493

               3.000

22

               0.100

               1.000

               0.500

               0.971

23

               1.000

               0.100

               0.500

               1.576

24

               0.660

               0.439

               0.500

               1.350

25

               0.100

               1.000

               3.000

               0.500

26

               0.934

               0.100

               3.000

               0.500

27

               0.258

               0.100

               0.500

               0.500

28

               0.985

               0.100

               2.418

               3.000

29

               0.728

               0.153

               1.383

               0.742

30

               0.893

               1.000

               2.049

               2.359

31

               0.369

               1.000

               3.000

               3.000

32

               0.100

               0.734

               2.563

               2.204

33

               1.000

               0.808

               3.000

               3.000

34

               0.762

               0.871

               3.000

               1.422

35

               1.000

               0.414

               1.025

               2.549

 

Each row is a trial with selected set of parameters values.  The HR trading system begins trading spreads about 65 days before expiry, and all profit/losses are realized at expiry.   Such a run for a selected expiry constitutes an experiment.  We will want to conduct experiments across market conditions.   Ultimately we want to find settings that will profit across market conditions.   This may be achieved by finding either a set of parameters that succeeds for all market conditions, or some way of tying the parameters to market measures such as realized volatility.

 Security Issues

The design matrix generated by the Gossett program is in a ₋1,₊1  scale where ₋1 is the minimum value and ₊1  the maximum value of the parameter.  In practice the Design of Experiments could be conducted by a third party, such as Trading Desk Strategies, on behalf of a client without knowing anything about the parameter settings or the model.  The factors could be given neutral names such as A, B, etc.  The client would be sent the design matrix on the ₋1,₊1 scale.  The client would transform it to the scale of their parameters, conduct the trials, and then return only the measured outcomes to Trading Desk Strategies.  The response surface analysis would be done in the ₋1,₊1 scale and sweet spots and analysis returned to the client in the ₋1,₊1 scale who would then transform to the original scales of the parameters.   In other words, the Design of Experiments technology can be applied without a client having to reveal anything at all about their trading system.   The original scales and the names of the parameters are revealed here in this article for verisimilitude, but even as much as has been revealed here doesn’t really cause us any worry that we’ve shown too much of our trading system.

Trial Runs

For our purposes there are three types of market conditions we’ll investigate, (A) the bear conditions around the October 2008 meltdown, the (B) bull condition from early March through the end of May 2009, and (C) the calmer conditions of June and July 2009. 

The November expiry is the most vulnerable to the October meltdown.  The run starts right after the August expiry on August 18th when the S&P 500 was 1278, and ends on November 20th when it was 752.44.  The fall in the market was sufficiently unremitting over this time that opportunities for buying back short puts were too infrequent.  The same, but opposite problem, occurred for the May expiry where the unprecedented 30% rise in the market over two months also failed to provide enough opportunities for buying back short calls. 

All the remaining expiries under study were like June and July where the only issue is the size of the profit.  The task of the analysis, then, is to find either that sweet spot that loses small amounts for November and May while allowing for large profits the remaining expiries, or produce the clues we need to be able to vary the parameters according to some market measure.

Each trial run assumed a $3,000,000 account.  The following table shows results for six months

Jul-09

Jun-09

May-09

Apr-09

Dec-08

Nov-08

Oct-08

Sep-08

1

24200

11450

11650

37075

10825

-2031175

15700

26600

2

65850

38950

46375

31525

13800

-2455100

33175

33075

3

398050

411250

-137525

108700

33950

-3203200

-2134170

98875

4

148075

114825

-360775

78325

77450

-3108550

22575

57825

5

324650

237950

212025

111200

75825

-3106625

-2480640

76775

6

108125

110800

-203425

34825

-467800

-3205800

13200

42125

7

45700

21250

40850

50775

50575

-2325050

21050

20950

8

254100

240750

-155275

68825

420775

-2567250

21225

52950

9

405950

404750

-68725

106125

294700

-2587650

-2311340

76550

10

93050

87950

126825

13650

80650

-3105600

9100

33275

11

57175

75025

-88775

13425

-520550

4925

6400

17650

12

118725

98700

-135700

22625

-400825

-3203200

9100

24825

13

56250

100425

-83850

17575

74400

4925

6400

16150

14

432350

504100

-131125

-2336425

48875

-2426700

9100

87525

15

134700

103950

-260350

24575

-415475

-2634725

9100

42125

16

159275

132075

-132500

31675

162275

-2335400

13200

44875

17

377225

257300

-251300

-2285625

-337150

-3205800

18500

95550

18

399150

299600

-214200

-2154950

-280500

-3074425

9100

63000

19

9725

10700

5200

4650

18125

7200

4825

9000

20

50125

49000

-269975

13425

-312800

4925

6400

17650

21

363875

404025

-113650

48350

368550

-2523125

18500

82325

22

409775

393750

-121450

82550

-375450

-2780475

-2105195

98875

23

58375

44900

-217650

7725

77225

4925

6400

16750

24

156500

208250

-60000

84450

-268175

-2534100

16500

63375

25

54050

26200

-92400

86525

148200

-3072150

31950

26600

26

6725

2750

4050

6950

10875

4500

4825

4175

27

60875

48650

-181450

13425

-295775

4925

6400

17650

28

68400

35525

-226000

7475

42175

4925

6400

17650

29

56025

44450

173575

13050

57200

-2333850

6400

15250

30

333500

307025

-207325

-1587500

-329850

-2333125

13200

72400

31

434500

404125

-182800

82550

-386500

-2581900

-2013230

117175

32

283775

253825

-213900

61650

-654375

-2740350

-2113550

83300

33

329750

241125

-236125

-1026300

-643950

-3153200

9100

63000

34

218725

99300

120225

-2241425

192400

-2427775

15825

63175

35

162075

159200

-247425

55550

428925

-2588525

9100

60175

 

As can be seen, the expiry months widely vary in their profit/loss profiles.  The percent of trials to generate a profit by expiry is

Jul-09

Jun-09

May-09

Apr-09

Dec-08

Nov-08

Oct-08

Sep-08

100%

100%

26%

82%

60%

22%

82%

100%

 

78% of the HR model’s parameter settings lose money in the Nov-08 expiry and 74% in the May-09 expiry.  Each of these situations are the opposite type of market condition, the former including the Meltdown, and the latter the fastest rise in the market for that period of time.  It will be a challenge to find parameter settings that reconcile such dramatically variable conditions, not to mention finding even one for Nov-08.

The Analysis

The trials results are fit to a cubic model.  The response surface is searched for the parameter settings that generate the maximum profit, called the “sweet spot”.   For this search we use the Sqpsolvemt function in the GAUSS Run-Time Library which solves the nonlinear programming problem with general constraints on parameters.  The cubic model is nonquadratic and may have multiple maximums.  For this reason the parameter space is divided into 2^4 or 16 quadrants and a maximum is sought within each of these quadrants.  Finally, any maximum found on an internal boundary is rejected because it is merely pointing to a maximum in the adjoining quadrant. 

The sweet spots for the eight months are

Profit/Loss

P1

P2

P3

P4

Jul-09

463829

0.417

1.000

0.500

1.437

Jun-09

547188

0.698

1.000

0.500

3.000

May-09

236745

0.880

0.285

2.191

0.789

174879

0.550

0.272

2.053

0.886

Apr-09

1181911

0.348

0.367

0.500

2.192

831618

0.376

0.339

3.000

3.000

Dec-08

827047

0.385

0.825

0.614

3.000

754326

0.460

1.000

3.000

0.500

Nov-08

-1837566

0.346

0.880

2.395

3.000

Oct-08

1016845

0.842

0.100

3.000

1.671

Sep-08

135123

0.100

1.000

3.000

3.000

We see in this table wide variation in sweet spots by expiry.  This indicates a sensitivity of the parameters to market conditions.  It does appear unlikely that a single set of parameters will work for all expiries.  Our next step will be to introduce some kind of process control in which the parameters will be adjusted in real time in accordance with measures of market conditions such as realized volatility, market volume, moving average trend information, etc.   The Kalman Filter is a well-known method for accomplishing this kind of task (http://en.wikipedia.org/wiki/Kalman_filter). 

If we were successful in implementing a type of process control for the parameters, we would have a result similar to a run where the individual sweet spots above prevailed for each expiry.  To show what that might be like, we executed a run on the complete set of expiries from Feb-08 through Jul-09.  For expiries for which we don’t have a sweet spot, we used the Jun-09 sweet spot.  The following table displays these results:

 

Jul-09

426325

Jun-09

475925

May-09

46375

Apr-09

46675

Mar-09

250475

Feb-09

268975

Jan-09

553250

Dec-08

578125

Nov-08

4925

Oct-08

6400

Sep-08

108775

Aug-08

63775

Jul-08

126000

Jun-08

117225

May-08

148600

Apr-08

31900

Mar-08

175025

Feb-08

71975

Total

3500725

 

From one to three portfolios are open at any one time for an average total of $7.5 million at risk.  The above result is a 31.1% annualized (uncompounded) rate of return.  We recommend for this type of investment that not more than 50% of the account be at risk in which case the return would be 15.6%.

 

 

 

References

[1] Aptech Systems, Inc., http://www.aptech.com. 

[2] R. H. Hardin and N. J. A. Sloane, "A New Approach to the Construction of Optimal Designs", Journal of Statistical Planning and Inference, vol. 37, 1993, pp. 339-369