Thread: Automated trading algo generation

Greetings People,

I made a thread on automated system development a week but it was immediately moved to the industrial department although I had absolutely no intention to sell anything -- I guess since we said the progr men and women sell that can do this (you can look at my threads if you want to follow that you as well). No problemo, I completely understand that but I need to have a trading discussion thread concerning the theory of this .

Soooooo, to get things on the Perfect track this is what I need with this thread:Talk about the theory of automated trading platform creation (things like data mining prejudice, look ahead bias, etc) Discuss about features Which May be useful or not Helpful within an implementation (without mentioning anything commercial) Discuss about publicly accessible (non-commercial) applications for this purpose As you fine people might already know I'm heavily involved with machine learning (https://forexintuitive.com/discussio...3-trading.html) and this is the way I've created all of the systems I currently utilize reside. There are many different places for trading platform creation. I hope we could go through them on this thread and provide some worthwhile discussion to the merits and issues that each might have.

I'm a busy guy so this thread will progress in bursts (as the one on ML has up til now) in whatever moment my kids and trading business allow for,

Cheers,

Jo

What am I talking about when I say automated system generation?

In very simple terms I'm talking about applications where you can simply say the simulation results you want and the software delivers an algo that makes that happen.

Say I wish a system that makes money from 2005 to 2015 using a Sharpe ratio of 0.8, so I tell the program offer me a trading system that has made money in this time and also has a Sharpe more than 0.8 the software works its magic and spits a part of code that when simulated shows whatever performance metrics I've asked for...

You'll find a gazillion problems that come out when you program software to perform so magic trick... Here is what I want the thread to be around!

When an app does algo generation ask yourselves these questions. How does the app...Decide what logic to originally use for the systems? Implement money management? Implement trade administration? carry out the simulations? Include trading prices? Act throughout the process? Assess the data mining bias of the research procedure? Apart from these you might also want to consider an implementation. . .is fast enough for whatever you need is efficient enough finding systems makes efficient use of computational tools generates code to your trading platform provides the Identical simulation results as your trading platform

Interesting.... A field I don't have any true comprehension in so forgive the next query based on insufficient comprehension.

With your example a system that makes money from 2005 to 2015 using a Sharpe ratio of 0.8 - How is that any different than simply curve fitting and tweaking indior settings on any specific egy to meet that criteria? I suppose since its mimicked the algo is going to do that automatically but am I right in this thinking?

AlgoTraderJo quoted from another thread:

Originally Posted by ;

IMHO there is a really very very very big problem when you utilize genetic programming for the construction of algo trading systems.

Originally Posted by ;

In [genetic programming] applications you have hereditary searching which introduces a bus load of prejudice into the procedure that's terribly difficult to account for.

Are we talking about prejudice in the statistical sense where high prejudice means high performance/accuracy and low prejudice means low performance/accuracy? I ask since it is reasonable that to test for high prejudice you would compare against randomly created egies and see if your results are very much superior to the random system results. But doesn't this also imply a system which is more complex will also perform better when comparing against this random system standard?

But when complexity increases, the chance that the system will have high variance also increases. In my experience large variance appears to be a bigger problem than high prejudice for traders. The reasoning is that the action of optimizing system performance based on some performance metric is in and of itself a means of increasing performance and decreasing prejudice. Done to the intense, this may lead to greater variance which will show up as poor results (poor generalization) on unseen data. In other words with high variance, the system performance is so good on the training information it is overfit and performance doesn't generalize to unseen data.

As http://scott.fortmann-roe.com/docs/BiasVariance.html (section 4.4 picture) as model complexity increases bias always declines. At some point variance begins to increase. High variance can be as big of a problem for systematic traders as large prejudice. Questions:

1) Does the data mining prejudice evaluation speech variance either indirectly or directly?
2) Is there any evaluation that's specifically designed to target the variance of system effects? Why is only the data mining prejudice evaluation mentioned?

Thank you for your gifts.

Hi Guys

Interesting thread.

How I see it and remember I'm a total novice in this area and also the guie from these skilled in machine learning with these assumptions will be greatly valued is as follows.

I find that there are 3 broad streets that could be obtained to EA growth that attempts to attain the objectives of machine learning, specifically to be able to navigate uncharted waters (from sample information ):

1. An unsupervised machine learning pattern recognition algorithm that evaluates a specified current pattern against a background of it's phenomenon (stored in memory), and if materially important against a specified confidence level, predicts the most effective trade method to harvest opportunity with that pattern. This kind of algorithm essentially so is forward looking and ready to navigate from sample information and hone in on the perfect way to exploit that new terrain. An approach similar to this will necessarily be'lagging' to new emerging states because it requires a sufficient historical sample to allow for pattern recognition.... But responsive to'learn' new market conditions within a rather brief time period. Possible curve matching however would be widespread as the quicker the reaction required to adapt to new states, the less history in memory which therefore reduces the sample size for testing and makes the process very subject to curve matching (data mining prejudice ).

2. An instructed algorithm where key performance metrics are given to the algorithm and then by interrogation of a sample dataset and extensive processing and iteration starting from a key set of factors, creates an coded outcome that accomplishes those performance metrics over the data sample. Obviously the ability for data mining bias arises in the size of data sample chosen (if sufficiently small to not be representative of the people ), nevertheless there might be workarounds with this approach by ensuring that the sample information is sufficiently large to be representative of the majority of potential from sample market conditions that may arise rather than just a random or uncontrollable characteristic. Note: This is describing the choice that algoTradeJo want to go over in detail in this thread.

3. A comprehensive history of information is made accessible to an algorithm in addition to a range of egies that may be successful employed for those historic market conditions whereby the algorithm only searches using pattern recognition for formative patterns that are statistically estimated as needing to potential to mimic past history. . .and thereby an present trading solution is used to exchange that nascent pattern. There's not much difference here to this achieved by an professional optional trader. The problem with this approach is that the inherent assumption that market conditions are used as a guide for your future. Obviously with a restricted sample dataset, in contrast to the other possibilities, this too could suffer with data mining bias.

Certainly the potential for all three potential approaches could be defeat in the utilisation of a satisfactorily large historic sample that holds inside the great majority of potential market conditions that may be thrown at it in the future. If so, then it's a problem of memory and processing grunt as opposed to the concept of this approach itself.

I'd be interested to hear other thoughts.

Originally Posted by ;

AlgoTraderJo quoted in the other thread: quote quote Are we speaking about bias in the statistical sense where high bias means high performance/accuracy and very low bias means low performance/accuracy? I ask because it makes sense that to check for high bias you'd compare against randomly created egies and see whether your results are very much superior to the arbitrary system results. But does not this also imply a system which is more complicated will also perform better when comparing this arbitrary system standard? But when sophistication...

Nice FXEZ. I want to chew on this a bit and get my thoughts between what is termed'bias' and what is termed'variance'.

PS Thanks for the connection. I'll throw in http://scott.fortmann-roe.com/docs/BiasVariance.htmldefinitions as they are useful for this discussion.

Conceptual DefinitionError because of Bias: The error because of bias is taken as the difference between the expected (or average) prediction of the model and the correct value which we are attempting to predict. Of course you have just one version so speaking about anticipated or normal prediction values may seem a little strange. But, imagine you can repeat the whole model construction procedure more than once: each time you gather new information and run a new analysis creating a new version. Due to randomness in the data sets, the models will have a range of forecasts. Bias steps how far away in general these models' predictions are from the appropriate value. Error because of Variance: The error due to variance is taken because a model prediction for a given data point's variability. Again, imagine the model construction procedure can be repeated times by you. The variance is the forecasts for any particular point vary between different realizations of the model. PS FXEZ based on the illuion below in the article, are not we searching both bias and low variance? You may want to help me out here.

Originally Posted by ;

When an app does algo generation ask yourselves these questions.

How can the app

... determine what logic to originally use for the systems?

Their must be at least one initial requirement premise upon which it then iterates using key variables to do so utilizing correlation for a procedure to cross compare iterations against performance metrics.

As an instance, this is merely a punt guess...I am assuming you begin with a period interval range established for your historical data sequence (eg. 1-2 decades of data utilizing a 10 pub time interval period to start) with state 3-4 key variables being say entry, timeframe, profit target and stop (or simply trailing stop). Set one to some non-random condition and the others to random and iterate for each time interval segment across the dataset at a walk ahead manner on each new bar (which then applies for another time interval range). Describe correlation between input metrics and outcome of each iteration. Pick closest correlation result for non-random condition and then interrogate another variable in precisely the same manner. After correlations established between variables and performance metrics then rate them and run tests on grouped non-random variables to ascertain result against performance metrics. Progressively add to non-random variable inputs until each variable is non-random.

Then pick another time interval range (eg. 20 bars) and go again....and so on and so forth until you hone in on the desired performance metrics.

Scale up or down so with position dimensions to attain nearest consequence????????

Originally Posted by ;

quotePS based on the simple illuion below in the article, aren't we seeking equally low bias and low variance? You might want to help me out here. picture

Hi, yes that's the way I see it. Ideally you want models that are highly accurate (low bias) and with little dispersion (low variance) so they reach the target directly in the center.

P.S. I like that site because it's really good graphics that produce a fairly intricate subject, intuitively apparent.

Originally Posted by ;

quote Hi, yes that's the way I view it. Ideally you want models that are highly accurate (low bias) and with little dispersion (low variance) so they hit the target right in the middle.

Thank you mate. That clears it up. :-)