Topic: How to code Natural Cubic Spline?

Posting in case anyone can code an indicator for a Natural Cubic Spline, just needs to get the spline values and does not need any logic. I'm trying to develop it from this wiki article:
http://en.wikipedia.org/wiki/Spline_%28mathematics%29

but they give a description, not the code. I'm trying to work through it, but if anyone has it already it would help a lot if you could post it. It will probably take me quite a bit of time, and I'm not sure how to verify the answers.

Hoping to explore something presented in this thread:
http://www.elitetrader.com/vb/showthrea … genumber=1

If anyone can help -- thank you very much in advance !!

Re: How to code Natural Cubic Spline?

hey Krog! can you tell me how does it work if it doesn't involve logic? you need logic to read signals or whatever out of it, right? and I can't understand how does it predict the prices, what I've read is it draws smooth curves through all data points but how does the prediction part step in?

Re: How to code Natural Cubic Spline?

Kind of funny -- I don't know how it is supposed to work!! For now, I'm hoping to do research, so it's best to reduce the experience to raw data as much as possible at the beginning. If someone has a spline indicator with logic added in, that's cool; if not, I was trying to save a step in case someone was going to code it from scratch. Mostly, I was interested in getting the values and seeing it drawn on the price chart, then later develop and test models as to how the values should be interpreted.

I will try to code one myself. To verify the values I get, I will search for a utility or tool when I get to that point. It's fairly complicated, I'm trying to make sense from some other coding examples I've found in Excel and Java and C++ etc.

As for the prediction, I don't know. But this is part of something a poster in the thread above presented as clues to lead the reader along. Cliff-hanging ending, he announced he was leaving the thread before the answer is reached. However, I suppose it's a good style -- it certainly has hooked my interest to work to solve his puzzle. If the poster directly stated it, in all honesty I probably would have drifted away after a couple of pages of posts.

It has something to do with the deviation from the "natural cubic spline". I don't want to paraphrase too much because I may not have a good grasp of the thread.

Re: How to code Natural Cubic Spline?

looked into it, tried to find mql indicators but failed, but some suggested to use HMA instead, I know it's not the same but it produces similar results, reduces lag and at the same time it smoothens data very well. so coding the HMA using the source of MA, RSI/MAcross into FSB should be easy (tried myself, mql seems easy but this C# FSB is higher rocket science for my brain, my result was a disaster).

Re: How to code Natural Cubic Spline?

thanks for looking into it. I I was able to create one in C# for FSB from here:
http://www.cse.unsw.edu.au/~lambert/splines/NatCubic.java
It's Java but C# is very close so was able to adapt it.
After looking at it, it seemed like a useless indicator. It fit the prices perfectly, but that is the goal of a spline - make a curve that fits the data. As you noted above -- it follows the data perfectly to the last known point, but there is no prediction.
I'm not sure what the point of the ET thread was. Following it, the results from the cubic spline were a random distribution. The main benefit I've got is that it was interesting to learn about splines and interpolation, and probability distributions and random walks -- but nothing useful to help with my next trade. Oh well, that's how the search goes sometimes.

Re: How to code Natural Cubic Spline?

Krog, do you mind sharing your indicator?  maybe we can sort things out.

7 (edited by krog 2010-06-08 19:14:38)

Re: How to code Natural Cubic Spline?

Hi Footon, thanks for taking an interest, I really appreciate the discussion and feedback.
Posting what I have now -- Cubic Spline 3. "3" does not mean anything, that's the number of the version of coding I've got to that works or makes sense.
For more info on cubics, see http://en.wikipedia.org/wiki/Cubic_spline. Basically, you have "knots", which are fixed points, like 5 or so, then draw a smoothly curved line that passes through the knots.
In Cubic Spline 3, you can select the number of knots and the "spacer". Spacer is the number of bars between the knots.
It draws the spline curve on the prices as a green line. Because of the way it's calculated, you only see the curve line for the part. Before that, it is on the close for the bar. If you select a large number for spacer, it will be more visible.
There is also a blue line for the "Projected Y value". That is, go to the last knot, use its cubic values (they describe the slope of the curve), and extend it forward one knot -- as if the spline curve were projecting out into the future. It's shifted back to the current bar.
Ex: spacer = 10, current close = 1.1234, Projected Y = 1.290: this means in 10 more bars, if the spline curve were drawn forward, the next knot would be at 1.290, above the current close.
At first, the Projected Y looked very bumpy, so I added a parameter to smooth it out. It takes a moving average.
It's an Open/Close Filter, so have to add after "Bar Closing". I've kept it simple for now, if there's more promise I may go back and add Open/Close slot.
Logic is simliar to Moving Average -- it rises, falls, the price opens above or below the Projected Y, or closes above or below the Projected Y. And -- "bar is higher/lower" - the whole bar has to be above or below the Projected Y.

In using it, it looks like a momentum indicator. It seems to go up and down with the waves, overshooting the peaks and undershooting the troughs. And it seems to pass through the prices at the middle of the wave. This is what led me to think it's not very useful -- it seems to exaggerate the movements, with no advance signal of price reversal. Or it's like it is more volatile and adds noise instead of reducing noise as an indicator should. Also, it could be just curve fitting which looks great in back testing but never works in real trading.

Feel free to try it out, perhaps there is something else to do with it that I have not thought of. Let me know, I'll see if it I can put it in.

Re: How to code Natural Cubic Spline?

thanks for your upload, great work!

Tried to be as good as you are, got some progress with HMA, but the errors are not ending, so the whole night was obviously not successful...

Ok, about the spline, what you think about using it as a moving average, maybe as a crossover indicator? A little optimization might prove it as a weapon of mass destruction, if you know what I mean lol   
Tried optimizing it but FSB crashed everytime, before the crash the results were promising, it said something that "index was outside the bounds of array"

9 (edited by krog 2010-06-24 18:31:35)

Re: How to code Natural Cubic Spline?

I have one with an MA crossover, and compared it to a regular 2 MA crossover. It got a better result with ending balance, but then testing over another time period, it was worse. The key would be to find consistency across data sets, to build confidence one has not simply hit upon a random backtest of good luck. I think that was one of the points of the ET thread, and he was leading to recognizing edge events in terms of a random probability distribution. Using the cubic spline here as recommended, it still looked like a random distribution to me, meaning I couldn't figure out his puzzle. Crap !!
This means Cubic Spline 3 is just like any other indicator. If you can get similar results with something else, may be better to go with that because CS 3 is pretty CPU intensive.
Could you post your strategy before optimization that crashes? I'll take a look and see if I can fix the bug.
The CS 3 above had a bug in the "Draw only" option. I've fixed it in the zip below. The MA crossover is an oscillator, so it's in the bottom chart. Add CS 3 and Moving Average Visual and set them to "Draw Only". Then the oscillator will give signals, the other two will draw on the chart so you can see what they are doing.

Re: How to code Natural Cubic Spline?

How about what you have for HMA? I'd be happy to take a look at it, see if I could help with some of the errors.

Re: How to code Natural Cubic Spline?

Alright, firstly the HMA - I uploaded HMA.cs, which I made, it's based on the FSB's Moving Average Indicator (this has "only" one type of error 'maMethod' is a 'variable' but is used like a 'method'). Secondly, @HMA, which should be a proper C# HMA from Ninjatrader forum. And the third text file is a list of HMA's formulas.

And now I'm turning to the CubicSpline, I'll post my more thorough results, findings, and failures afterwards.

Post's attachments

HMA.cs 17.22 kb, 33 downloads since 2010-06-08 

You don't have the permssions to download the attachments of this post.

Re: How to code Natural Cubic Spline?

couldn't attach all of them at the same time... so here is next:

Post's attachments

@HMA.cs 6.12 kb, 36 downloads since 2010-06-08 

You don't have the permssions to download the attachments of this post.

Re: How to code Natural Cubic Spline?

and the last:

Post's attachments

Formulas.txt 374 b, 16 downloads since 2010-06-08 

You don't have the permssions to download the attachments of this post.

14 (edited by footon 2010-06-09 00:51:06)

Re: How to code Natural Cubic Spline?

first update: CS3 crashes at the very end of optimizing:
http://s4.postimage.org/hvr6r.jpg


and it really is a CPU hog, didn't expect it to be so intense. The strategy:
Market: EURJPY 15 Minutes
Spread in pips: 4
Swap Long in pips: 1
Swap Short in pips: -1
Commission per lot at opening and closing in pips: 0
Slippage in pips: 0

Maximum open lots: 1
Entry lots: 1
Adding lots: 1
Reducing lots: 1

Intrabar scanning: Not accomplished
Interpolation method: Pessimistic scenario
Ambiguous bars: 0
Tested bars: 2331
Balance: 311 pips (10327.37 JPY)
Minimum account: -8 pips (9991.58 JPY)
Maximum drawdown: 96 pips (101.05 JPY)
Time in position: 14 %

A same direction signal - Does nothing
An opposite direction signal - Does nothing
Permanent Stop Loss - 20
Permanent Take Profit - 20

[Opening Point of the Position]
Bar Opening
     Enter the market at the beginning of the bar
     Base price  -  Open

[Opening Logic Condition]
Entry Time
     Enter the market between the specified hours
     From hour (incl.)  -  9
     From min (incl.)  -  0
     Until hour (excl.)  -  12
     Until min( excl.)  -  0

[Opening Logic Condition]
Cubic Spline 3
     The bar closes above the Projected Y
     Base Price  -  Close
     Smoothing method  -  Smoothed
     Knots  -  5
     Spacing between knots  -  10
     Smoothing Period   -  5
     Use previous bar value  -  Yes

[Closing Point of the Position]
Bar Closing
     Exit the market at the end of the bar
     Base price  -  Close

[Closing Logic Condition]
Bollinger Bands
     The bar closes below the Lower Band
     Smoothing method  -  Simple
     Base price  -  Close
     MA period  -  20
     Multiplier  -  2.00
     Use previous bar value  -  No


Added the complete error message as well.

Other thoughts: only 2 logic conditions seem to be profitable ("The bar closes above the Projected Y" and "The bar is higher than the Projected Y"). In the crossover CS3 there has to be a cross between the spline and a MA, right? Why not between 2 splines?

edit:
it crashes if more than 1 parameter is being optimized.

Post's attachments

error message.txt 5.23 kb, 8 downloads since 2010-06-08 

You don't have the permssions to download the attachments of this post.

Re: How to code Natural Cubic Spline?

Try this for the HMA. Good news -- it seems to work, looks very fast compared to the prices. Bad news -- I don't know how to get it to work for my data when used as Open, only works for me after Bar Closing then add HMA. In fact, Moving Average is the same for me. I think it might be because my data has fractional pips. I'm not too put out by it -- I always use closing values, after the Bar Closing slot.
Best to verify it against a different app or some standard set of data and expected values, to be sure the FSB values from my coding match what's expected.

Post's attachments

HMA.cs 17.94 kb, 81 downloads since 2010-06-09 

You don't have the permssions to download the attachments of this post.

Re: How to code Natural Cubic Spline?

krog wrote:

Try this for the HMA.

Brilliant stuff, thank you!

krog wrote:

I don't know how to get it to work for my data when used as Open, only works for me after Bar Closing then add HMA.

My brief test showed what I expected, and no problems with open/close prices or bars, it should be the data fault, I'll run a comparison tomorrow to see if lines are lining up right.

Re: How to code Natural Cubic Spline?

Awesome -- glad it worked out for you.

Crash -- sorry, I don't know. From the error message, that looks like something inside of FSB. It might be from CS3, unless you're able to get a similar crash with another indicator. I was not able to reproduce it, I got to the end. Something about "Reset" -- but it would take an FSB developer to track it down. I have FSB 2.11.

It is pretty CPU intensive -- it does like 50 or 100 steps or more to get one number!! The other thing to consider is that if you were to use it in real trading, if you use a different platform, you'd have to port it over (like from FSB C# to NinjaTrader C#). That might be a bit tricky.

Re: How to code Natural Cubic Spline?

Crossover -- yes, it's when the spline curve and MA cross, or the MA and spline curve cross. I chose MA arbitrarily, didn't think of using 2 splines. It might not work because 1 spline curve takes 3 numeric stepper parameters, so 2 curves would take 6. But that's all the numeric steppers FSB allows -- there would be no way to enter a Level parameter.

Re: How to code Natural Cubic Spline?

Hey Krog! Things went my way finally - CS3 stopped crashing, I didn't give it a lot of thought, but it seems that parameters should be kept low (not over 20), and higher TF is more than welcomed. The latter brings me to a conclusion that TF must be at least H1, otherwise there's a jumpin' jack flash in action (I added an indicator to filter CS3's high activity - enter once). My first evaluation showed the same that you stated - it is a poor price predictor, but there is definitely potential. My thoughts are a bit raw, I'll think and test a little, and then bring out my opinion for discussion.

krog wrote:

The other thing to consider is that if you were to use it in real trading, if you use a different platform, you'd have to port it over (like from FSB C# to NinjaTrader C#). That might be a bit tricky.

I swear I don't have anything similar in my mind lol  Read the brilliant code you wrote for HMA, I was astonished - like a first visit to the city, if you know what I mean. It is a Terra Incognita for me, so I'll stop using my programmer skills for a while at least big_smile

Re: How to code Natural Cubic Spline?

Glad to hear it stopped crashing; but if there's more info pointing to the indicator please post, I'll try to fix it, it's not good etiquette for me to post something that crashes the app  smile  Good luck with it and programming -- it's just lots of practice ...

Re: How to code Natural Cubic Spline?

check your inbox, I fired a PM

22 (edited by krog 2010-06-24 18:39:57)

Re: How to code Natural Cubic Spline?

workaround

Re: How to code Natural Cubic Spline?

they both look alike, this can be a good thing!

http://s4.postimage.org/80FJ9.jpg

but no, reality bites and everything goes to hell FST style!

http://s4.postimage.org/8169J.jpg

Re: How to code Natural Cubic Spline?

I'll take a look, for now I'm taking all of the attachments off. If it doesn't work properly, not good to leave it up.

Re: How to code Natural Cubic Spline?

krog wrote:

I'll take a look, for now I'm taking all of the attachments off. If it doesn't work properly, not good to leave it up.

it might not be the indicator's fault, you're probably familiar with this thread, but I'll post the link anyway
http://forexsb.com/forum/topic/1496/liv … an-in-fsb/