Open Source Trading Software Or Not?

I have built a few trading systems from scratch during my years with investment banks. So, I have learnt from the many mistakes made. I am recently reviewing some open source system, and would like to share some thoughts.

In general, I am against building software in house. Funds and banks are not software firms. They should focus on making money instead of IT development. Producing software is best left to Microsoft, Google, Apple and IT vendors. These good folks spend their life time doing nothing other than building software. There is no chance that a (small) financial organization can build better systems than they do.

Now, there is an exception to this rule of thumb. When it comes to building trading systems, I suggest that we should do this in house. Alternatively, you can hire a vendor and retain them on contract basis to co-develop the trading system, if you do not have proper talents in house. (It is notoriously difficult to hire very talented programmers in finance because the top quality guys go to GOOG, MSFT, AAPL or startups.)

The worst is to download an open source trading system and simply hope that it will work. Let’s examine why. Firstly, for a piece of very complicated software such as trading system, it is unlikely that the downloaded code works out of the box. There are often too many configurations to do. Even with proper documentations (which is a big assumption), there are still too many plumbing to set up before you can make the first trade, e.g., connectivity to exchanges, data sources, news and announcements, writing and testing strategies in their framework, book-keeping, middle office, clearance. Chances are that you will end up paying the vendor’s service to speed up the set up process.

Secondly, you may think that you have the option to edit their (open) source code if you like. The reality is that you probably cannot do so. There are two problems. The first problem is that, without the authors walking thru the code with you, it is very difficult to understand the source code, the architecture, the assumptions, the hacks and quick-fixes, and even the comments. For the open source trading system that I am playing with, I don’t think I can reasonably edit it without spending a month or two studying it without assistance.

The second and the biggest problem is that even though you can edit the code to your taste, how do you merge your changes with the vendor’s future releases? The vendor knows absolutely nothing about your changes. Whenever they make an update, you will have to manually merge all the changes, assuming that this is even possible. You will then find yourself either 1) rewriting the same code over and over again or 2) maintaining a separate branch from the vendor’s altogether. Neither is satisfactory. Chances are that you will end up paying the vendor to write the code for you.

Thirdly, assuming that you cannot reasonably edit their code even if it is open source, you and your trading strategies must then work under their framework and work with all their assumptions and limitations. There is no one standard way to build a trading system. There are too many possibilities: order routing, order placement, order cancelation, order filling, error handling, alerting.

Take order cancel-replace as an example. There are at least two ways to do it. Number one: you cancel an order, wait for a confirmation, and then place a new order. The problem is that this method is very slow so you may miss the trading opportunity. You may even miss the confirmation that you never send out the replacement order. Number two: you place the new order before sending in the cancelation message. The problem in this case is that you may over-fill your order when both the old and new orders are executed. There is no “correct” way to do it. It all depends on the traders and the trading strategies being executed. Another detail is whether you still want to cancel the order if it is already partially filled. The open source trading code I am reading now does not address this altogether. Chances are that you will end up paying the vendor to customize the code for your applications.

Fourth, if even the trading system is open source, you cannot reasonably understand the code enough to modify it. You at best can only be a user of the system. In this case, what do you do in the future when you want to add additional components to enhance your trading process? For instance, you may want independent and hence external modules such as order aggregators, flow controls, safety guards, etc. You will have to build these components around their framework that you don’t understand. Chances are that you will end up paying the vendor to build these add-on modules for you.

Fifth, some trading systems, for the sake of generality, take a script as a trading strategy. The script may be in proprietary languages such as TradeStation EasyLanguage or in popular language like Ruby. Either way, the problem with scripting is that it is very slow, relatively speaking. The competition in the algorithmic trading space is very fierce. Many firms, e.g., GETCO, spend a lot of resources to tweak their systems or colocation just to get a few milliseconds edge over competitors. Executing scripts line-by-line is very slow. Not being able to tweak, hence optimize, your system is making it even slower. You are already losing the trading game to others in speed. Chances are that you will end up paying the vendor to directly access the underlying API.

Sixth, some solid trading systems come with many exchanges/ECNs gateway access. Many don’t. For those smaller ones, such as the open source ones, they might at best have implemented the FIX protocol. While FIX is a popular protocol in finance, many exchanges and ECNs have their own protocols. Some ECNs may even have different APIs for liquidity takers and liquidity providers. I cannot image the vendor would make open source many these gateway API’s. (Some major businesses live on providing these API for fees not for free after all.) Chances are that you will end up paying the vendor to develop certain gateway access to the markets that you want to trade.

In other words, whether it is open source or not, it is not very relevant. You will end up paying big money either for the software with support or for the services. In either case, you will need a copy of the source code so that you can modify as the last resort. Therefore, from the IT perspective, it does not make sense to buy a piece of trading software from a vendor.

From also the business perspective, you should build the trading system in-house. Or, at least, you need an in-house IT team to work with the 3rd party vendor to co-develop the trading system. The main reason is that a trading system is an embodiment of many business secrets, such as trading strategies, capital allocation, execution optimization, stop loss handling, etc. For instance, when designing a trading system, one question is how to optimize execution to reduce the transaction costs, such as bid-ask. You may want to split the order. You may have some tick prediction mechanism. Etc. You do not want to outsource these business logic or proprietary knowledge to a 3rd party vendor. They are best kept in-house.

Going back to my general principle that a fund is not a software house so it should do as minimal IT as possible, so, where is the line drawn? My suggestion is that we do not build a trading system all from scratch. You want to buy as many “standard” components as possible and only the “standard” components. For instance, it makes no sense to reinvent another FIX engine because some vendors have done it so well. The same go true for historical data database, communication bus, analytic tools, back testing system, market gateways and APIs, GUI controls, etc. What you should build in-house are only the parts that incorporate your business knowledge, e.g., execution optimization, as discussed above.

In conclusion, a trading system is an embodiment of the business secrets of a fund or a bank. You should not out-source this work to a 3rd party vendor. You should develop this in-house. Over time, you grow your internal team of trading system experts. More importantly, your trading system, albeit starting small, will eventually grow to a powerful weapon that gives you an edge over competitors. After all, algorithmic trading is a game of technology competition.