Optimization Tactics

**winias** · 05-11-2009 06:30

So many EA's, so little time.

Like to open a new thread here to discuss fast / better ways to optimize EA's.
Optimizing EA's takes time, and oftentimes I find myself scrapping results on one outcome, only to realize I should have started with other parameters instead.

If you have any tips / tricks / ideas to share based on your experience I would love to hear about them.

Optimization can be a grueling process but if you have a sound strategy to go about it a lot of time can get shaven off if you zone in on the right things.

My biggest issue is the sheer time it takes to go through a single ea. Running the genetic algorithm helps, but only to a certain extent. Letting metatrader go for 3 days while it optimizes an ea seems rediculous, so genius', please post away and share some ideas.

Floor is open to anything that strikes you.....I'm intermediate in terms of metatrader. I'm skilled when it comes to trading tactics, not as much programming. (This thread could be considered a silent cry for help on my behalf, if you've ever been at the same skill-level as me

).

I find a lot of times many EA's that have excellent long term potential get pushed aside for newer EA's just because they got 'lucky' on initial backtesting results. I'd like to start this discussion in an effort to teach old dogs new tricks and keep them in the portfolio where they belong.

Thanks!!

**Neo-Rio** · 05-15-2009 04:45

I'm relatively new to forex trading (only a couple of months), but I learn quick. So far I've learnt that you can create a multiple EA trading strategy, but you must pick your EAs wisely, and then OPTIMIZE THE HELL OUT OF THEM before putting them to work on a demo account, and then onto a live account.

Personally I take the brute-force optimization position. With enough computing power and enough time, you can optimize the hell out of a particular EA. As computer technology gets better and faster, this becomes more viable.

Now to do such heavy optimization, you are going to need a FAST computer(s) because a rigourous optimization is going to take a LOT of time (such as a whole week).

Unfortunately Metatrader only uses one logical processor for testing, so you will AT LEAST need a modern dual-core processor with the highest clock speed you can afford.
Quad core and upwards processors will not help you much for an individual optimization, as it's the clock speed that counts - but the more cores you have, the more simultaneous Metatraders you can have open and simultaneous optimizations you can carry out! Just remember to leave one core free so you can use your computer normally in the meantime! If you've got the cash (you're a rich trader), something like the latest dual quadcore CPU Apple Mac Pro workstation running Windows would be ideal for this kind of work. For example, that beast has 8 cores and 16 logical processors, so you could ramp it up to simultaneously optimize 15 EA setups.

It's not enough to backtest over the space of a year either. At least 5 years of historical data is good enough. 10 years if possible. To illustrate this point, I optimized one EA over a period of one year, and it managed to turn $1,000 into $18,000. Then I changed the time frame to a completely different year, and used the optimized settings. What happened? The test account blew up! Seems to make more sense to backtest over as many years data as possible so that the EA can handle just about anything the market can throw at it.
Some EAs don't optimize well over 10 years (either they blow up accounts, or do not profit much), and some really good ones will actually survive and THRIVE off 10 year backtests!

Certain EAs are brilliant for optimization. Others aren't. Your first task is to find an EA that optimizes well.

Then take a look at the default settings. Click all the boxes that pertain to the EAs performace. Some things can be left alone (like the Magic number, no need to optimize that!). If there are GMT settings, then find one that fits. Autodetecting GMT filters will cause the EA to refuse a backtest and ultimately an optimization.
Also, if there are trading sets in the EA (such as time frame sets), optimize only one timeframe set at a time. This is so that you don't overdo the optimization settings beyond the computer's capabilities.

Then you have to decide on a pair and timeframe. Something that I like doing is taking a successful EA, and then running it on the same timeframe but with a different currency pair and optimizing it for the different pair. In trading, it does get problematic when you have many EAs competing over the same currency pair, you risk having them modify each other's trades.

Then I take the default value of a particular setting, change the Start to 0 (typically, unless there's a possibility of using a negative value), change the step to something reasonable, and then the end value to something double the default value.

I like to begin testing with $1000 in the test account. This allows the bot to optimize itself to avoid huge drawdown. If it can profit from a startup of $1000, guess what it can do for $10,000 ? On the other hand, if it can margin call $1000 in a short space of time, would we trust the EA?
Also, since we are EA trading, most people will start off with a small account anyway. We're on a quest for the holy-grail after all.

After that it's really a matter of waiting for the optimization to finish. Optimization usually starts out slowly. You have runs which don't generate much profit (such as $100 over 10 years!), but eventually the system smartens up and within a number of days, you've trained the EA to turn $1000 into a substantial sum of money.

If anyone has some better optimization strategies please let me know.

**ContraTrader** · 06-19-2009 16:03

Try not to over-optimize. This is the worst thing that can happen to any trading system.

**fairylord** · 09-27-2009 12:17

I found this tutorial post very informative and professional...I found it in the post by MetaTrader7, thanks to him so much for sharing this tutorial..

Friends, learn it for your pocket

Indeed, fairylord still digesting it

__________________________________________________ _

Most folks are EA freeloaders, trained by click bank to think EA running is an easy biz. They try original file, run it, see problems, have no idea what to do with it and give up. I would like to say few words on what it really takes. EA testing is a tedious job that takes time, patience, lots of prep work, record keeping, data keeping. It's a computer science project.
******
You need a system for testing and a to do list:

One data acc per broker you intend to do business with, to collect your historical data running 24/7 , ideally you need a VPS for that, if you don't, then just have them and run periodically. Go Tools>Options>Charts and set bars in history 360000 and bars in chart 360000. Restart MT4. Open M1 (one minute time frame) charts for all pairs and on the first run scroll them all the way to the back to pull as much broker data as possible. Save profile: DataCollectionProfile, copy to all similar MT4s.This is your data/optimization setup and should eventually collect one year of intraday data. Don't run any EAs on it, use it only for back testing and optimization.
Then you need another optimization demo account, broker doesn't matter.
set bars in history 360000 and bars in chart 360000 again, restart. Delete demo account. Then you go to history center, delete all data and download data from Metaquotes. That will give you several years of data. You don't need all that, unless you want to do heavy back tests. The problem it really takes lots of ram and will freeze your laptop. So you keep about 1 year, 2 max. so after you download MQ data, delete excess and keep one year per currency.

So this is your generic optimization setup and you don't have to run this one 24/7, only when optimizing.

***********
Then you need EA running setups that will run EAs with few rams. If you trade manually use same economical setup. Go Tools>Options>Charts and set bars in history 6000 and bars in chart 6000. Restart MT4. Open fresh charts with all pairs with nothing on them. Go Files>Profile>Save As, name the profile. You can have several profiles: AllPairs, AllGoodSpread pairs, ...MySetup. Then copy these to all similar MT4 instances. Profiles are saved in profiles folder. Run Each EA or same EA with different settings on separate MT4 and compare side by side.

***********************************

Save detailed statements after each cycle run or test as: EaNameSettingsDateBroker.
If you know excel, you can crunch it further when needed.

********************************
Now you will be running dozens of MT4s and facing with the laborious task installing EAs - pasting lots of files to lots of folders. Thanks God there is a freeware for that. See attached file. So the way you do it, go to windows explorer C:>Programs and run search let's say Experts, that will give you list of all Experts folders. You simply highlight drag and drop them into the ware I attached and files you want to copy and press the button. So now you can install everything fast to all MT4s.

****************************
Also after each MT4 install go to Tools>Options>Expert Advisers, Make 1,4,6,8 from the top boxes checked, the rest unchecked.

*****************************
To set EA , drag and drop to the chart, check chart time frame. In EA Properties either change settings or if there is .set, press load, select and OK. Then you can save it as Template (faster that way) and open on all other charts. If you run same EA on different currencies you don't have to have or have to change magic number, so simply open and run same template. Different magic number is only needed when 2 or same EAs are running on same currency

***********
When back testing make sure you load the .set file in Strategy Tester. Even you loaded it or tweaked setting on the chart, Strategy tester has its separate EA Properties setup. So you have to set it up there. I saw some posts complaining they are getting a way different results, perhaps they just didn't loaded .set in Strategy tester. That should be basically it. And now you can operate with test data and improve the results.

**************
Many are stacked with back testing. Popular (Click Bank) myth : if I get a 90% back test then it holds 90% in real trading that I can bank. It isn't, it's only 90% modeling quality, not even modeling accuracy.
This is just statistics of modeling process, not real trading and not even demo.
That is more like 50-70% of demo test depending on the complexity of the system. Simpler systems back tests more accurately with standard settings. To prove the point, run EA for a week, then back test on exact same data with same settings, you will see real difference and real %.There is also another myth that forward test = live trading. It isn't. Actually even a forward test isn't 100% and only 80%-90% of live trading. So one has to cover quite a distance, from setup, to back test, to forward, then tweaking and back/forward testing again and again. Remaining inaccuracy % can sway either direction, some systems are better on demos then back tests and some systems are better live and with all sorts of combination in between.

**************
All EAs need tweaking for particular broker, different market and same market changes. One need access to code for that. Commercial EAs are all locked with intentionally only few options left. that makes them impossible to tweak. The only option is to crack them or not to use at all, because when they go bad, sellers aren't always adept to fix them. 99% of comercial EAs are bad tweaks anyway or not lasting. Most commercial EAs are simply the tweaks of free EAs that are being sold under a marketing scheme "Licenses to print money". This is a myth, there is no such thing as a license to print money (unless u are a central bank). In reality EA is a tool and only as good as the trader skill . So the best way is to find best tool for the job and develop your own tweak.

****************************
Most EAs are optimized on particular data and results may differ a lot. Original .set is provided as a starting point to put EA in a profit zone, from where it can be further optimized. So it isn't final or one fits all. It is rather a starting point. If EAs are optimized on Meta Quotes data, then they will work best with 4 digit brokers who use filtered data. So Each EA has to be optimized for your broker and one has to know how to do that or doesn't deserve $$$. Then each EA has to be continuously optimized for changes in the market. Frequency differs from a week to a month, 6month, year. So realistically one has to have a skill of installing, running, optimizing EAs in order to have confidence and running them successfully.

**********************************
Just to see EA potential at least 2 weeks of forward test is needed.
After you found a great tweak that forward tests well and steady at least for a month, you can live test it with lowest lot size with real money. Never put 2 different accounts on the same MT4 installation and switch between them, keep the latest delete all other accounts. Run always one account per installation. MT4 doesn't handle that always well and sometimes confuse accounts start managing live account while you run test on demo. Also never point several MT4 to one account or you will confuse broker server and it will eventually start freezing one or another.

***********************
From Chatting with ITs in the field I learned that MT4 has some internal flaw at the kernel level. (MQ is overcoming it in MT5) So one has to be an experienced programmer to work around. Trading Results can also sway both ways
It makes it more like an art rather then science. So optimization and entire MT4 is like that, it can work great in the hands of those who know it and can go real bad for those not experienced enough. MT4 is Russian creation and there is more documentation and tech chat in Russian. Russian speaking coders and tweakers have a definite advantage.

****************.
It looks I just created a popular guide for pro back testing. At least for the setup. Bottom line if one wants to enjoy the income of a professional trader, one has to perform duties of a trader.
Click bank myth that you can buy a license to print money is still a myth. What technology did, it made the job even more equally accessible with more tools. It became easier, and also easier to burn. So learning technology is one thing, trading markets for $$ is another and connecting both is another task.

original post: http://www.tradingsystemforex.com/16663-post117.html
.

**bobish** · 12-21-2009 04:29

Here's the deal. I backtest for 2 WEEKS, and no more. I believe the ONLY thing we know for sure is the PRICE NOW, i.e. price and time.

Backtesting over one year or even several months is IMO a waste of time. Conditions in forex change by the second and price action last year is ancient history and irrelevant. If an EA shows a nice graph over a year this is NO indication it won't crash the following week.

Here's what I do.
I backtest for two weeks and my EAs show a PF of 2 to 63 and above. THEN I extend the span to THREE weeks ahead, so that one week is NOT within the optimized period. If the graph goes up I extend the period to FOUR weeks. Now 2 weeks are outside the opt. period. If the graph is still UP I add another week and so on. This method gives me the Strenght Value of the EA which I use as a judgement factor.

Of course you must test with historical data, so I start with data that are 4 weeks old, optimize for the first 2 weeks and test for the remaining 2 weeks. If the graph holds up I re-Optimize for the 2 weeks just before I want to deploy the EA, then attach it. So far I have gotten good results on forward demo. Last week my EA made a net of $55 over 4 days trading with 500 cap. Time for 2 week opt. is about 1H 30m on my slow PC and about 45m on the VPS

Bobish

**XRanger** · 12-21-2009 10:04

From my experience coding a lot of EAs and systems: If you need to optimize- you are already in a bad system. The best systems automatically adapt themselves to the markets, whether by using ATR instead on constant pips, or by identifying chart patterns (some of my best EAs use chart patterns and have less than 4 settings!). Basically, the less you optimize, the less chance you are to overoptimize and the more likely your system is actually profitable.

**bobish** · 12-21-2009 22:08

Originally Posted by XRanger

From my experience coding a lot of EAs and systems: If you need to optimize- you are already in a bad system. The best systems automatically adapt themselves to the markets, whether by using ATR instead on constant pips, or by identifying chart patterns (some of my best EAs use chart patterns and have less than 4 settings!). Basically, the less you optimize, the less chance you are to overoptimize and the more likely your system is actually profitable.

Yes, I agree wholeheartedly. That's why I criticise the "Steinitz" garbage floating around. By last count it had more ADJUSTABLE Params than the lenght of most EAs. It would take several LIFETIMES to optimise it. My EAs have typically 5 opt params. I never include Risk or magic number as an opt, param.

I do not know enough about AI and the other esoteric algos in vogue, but my latest is based on the ubiquitous BB and adapts to breakouts where longs become shorts and shorts, longs.

I have to opt because there are 2 params that need to be adjusted each week. So my EAs are based on a weekly cycle done over weekends when Broker shuts down. Those beautiful graphs with NO kinks and NO stops are a trap for suckers. Trade without stops and you are sure to LOSE!

I use a short MA of (3) on the Median to close out bad entries, so that my protective stop is very seldom hit. I think one has to use a system which limits time in a trade, and to keep out of bad times to trade. But I always opt 2 params besides SL TP and Trail.

Too many params and over opt, are a recipe for disaster. An opt should not take more that 1 hour even on a slow PC. It would be nice to use some of the extrapolation techniques out there in Math kingdom, like Fourier series and Newton's forward or backward differences etc, but that's for the mathematicians among us.

Bobish

**cico707** · 01-06-2010 08:26

Hi,
I am new to this forum.
I purchased this and and I can not use it.
You can you help me?

**funyoo** · 01-06-2010 14:49

Originally Posted by cico707

Hi,
I am new to this forum.
I purchased this and and I can not use it.
You can you help me?

Hi cico707,

Attachments removed. You can not post a commercial code.

**shpock** · 08-10-2010 06:06

very good

Thread: Optimization Tactics

Thread Tools

Display

Optimization Tactics

ProSetup for EA testing & running (by MetaTrader7)

Backtesting tips

Optimization EA

Similar Threads

How do i use optimization!

General Settings For 'Optimization'

Posting Permissions