The Role of Technology in Quantitative Trading Research

I posted my presentation titled “The Role of Technology in Quantitative Trading Research” presented in

You can find the powerpoint here.

Abstract:

There needs a technology to streamline the quantitative trading research process. Typically, quants/traders, from idea generation to strategy deployment, may take weeks if not months. This means not only loss of trading opportunity, but also a lengthy, tedious, erroneous process marred with ad-hoc decisions and primitive tools. From the organization’s perspective, comparing the paper performances of different traders is like comparing apples to oranges. The success of the firm relies on hiring the right geniuses. Our solution is a technological process that standardizes and automates most of the mechanical steps in quantitative trading research. Creating a new trading strategy should be as easy and fun as playing Legos by assembling together simpler ideas. Consequently, traders can focus their attention on what they are supposed to be best at – imagining new trading ideas/strategies.

Excerpts:

  • In reality, the research process for a quantitative trading strategy, from conceptual design to actual execution, is very time consuming, e.g., months. The backtesting step, in the broadest sense, takes the longest time. There are too many details that we can include in the backtesting code. To just name a few, data cleaning and preparation, mathematics algorithms, mock market simulation, execution and slippage assumptions, parameter calibration, sensitivity analysis, and worst of all, debugging. In practice, most people will ignore many details and make unfortunate “approximation”. This is one major reason why real and paper p&l’s are different.
  • Before AlgoQuant, there is no publicly available quantitative trading research platform that alleviates quants/traders from coding up those “infrastructural” components. Most of the existing tools are either lacking extensive built-in math libraries, or lacking modern programming language support, or lacking plug-and-play “trading toolboxes”.
  • Technology can change the game by enhancing productivity. Imagine there is a system that automates and runs in a parallel grid of 100s of CPUs for you all those tedious and mundane tasks, data cleaning, mock market, calibration, and mathematics. You can save 80% of coding time and can focus your attention to trading ideas and analysis. Jim, using Matlab, may find a successful trading strategy in 3 months. You, equipped with the proper technology, may find 3 strategies in a month! The success of a hedge fund shall no longer rely on hiring genius.
  • After we code up a strategy and choose a parameter set, there is a whole suite of analysis that we can go through and many measures that we can compute to evaluate the strategy. For instance, we can see how the strategy performs for historical data, simulated data generated from Monte Carlo simulation (parametric) or bootstrapping (non-parametric), as well as scenario data (hand crafted). We can construct the p&l distribution (it is unfortunate that historical p&l seems to be the popular performance measure; we traders do not really care about what we could make in the past but care only about our bonuses in the future; so what we really want to see is the future p&l distribution for uncertainty not historical p&l); we can do sensitivity analysis of parameters; we can compute the many performance statistics. All these are very CPU-intensive tasks. Using AlgoQuant, you simply feed your strategy into the system. AlgoQuant runs all these tasks on a parallel grid and generates a nice report for you.
  • The academic community publishes very good papers on quantitative trading strategies. Unfortunately they are by-and-large unexplored. First, they are very difficult to understand because they are written for peer reviewers not laymen. Second, they are very difficult to reproduce because most authors do not publish source code. Third, they are very difficult to apply in real trading because the source code is not meant for public use, even if available.

Java vs C++ performance

It is very unfortunate that some people are still not aware of the fact that Java performance is comparable to that of C++. This blog piece collects the evidence to support this claim.

The wrong perception about Java slowness is by-and-large because Java 1 in 1995 was indeed slower than C++. Java has improved a lot since then, e.g., hotspot. It is now version 6 and soon will be version 7. Java is now a competitive technology comparing to C/C++. In fact, in order to realistically optimize for C/C++, you need to find the “right” programmer to code it. This programmer needs to be aware of all the performance issues of C/C++, profiling, code optimization such as loop unfolding, and may even need to write code snippets in assembly. An average Joe coding in C/C++ is probably not any faster than coding in Java.

(I am in general against code optimization techniques because they make the code unreadable to humans, hence unmaintainable, such as a lot of the FORTRAN/C/C++ code found in Netlib and Statlib.)

More importantly, most modern software runs on multiple cores. Code optimization techniques are dwarfed by parallel computing technologies. It is significantly easier and more efficient (and more enjoyable) to write concurrent programming code in Java than in C++. Therefore, to code high performance software, I personally prefer to code for multi-core, multi-CPU, and cloud in java rather than doing code optimization in C/C++.

(I AM NOT SURE WHY FORTRAN SURVIVES IN 2011. HOW ARE YOU SUPPOSED TO READ THOUSDANDS LINES OF CODE ALL IN UPPER/LOWER CASES WITH A BUNCH OF C’S AND GOTO’S EVERYWHERE?)

Briefly, my conclusion is that, among the general purpose programming languages (hence excluding Scala and etc.), we should use Java instead of C/C++, FORTRAN, Assembly and etc. whenever possible because Java is the easiest programming language to learn and work with without a sacrifice in performance.

(For me, Java is an easier language than C# because the Java IDE technology is far much better than the C# counterparts.)

The evidence I collect are listed below. Please feel free to expand the list.

  1. Java has recently won some major benchmark competitions.
    1. http://developer.yahoo.com/blogs/hadoop/posts/2008/07/apache_hadoop_wins_terabyte_sort_benchmark/
    2. http://developer.yahoo.com/blogs/hadoop/posts/2009/05/hadoop_sorts_a_petabyte_in_162/
    3. http://news.cnet.com/8301-13846_3-10242392-62.html
  2. Recent independent studies seem to show that Java performance for high performance computing (HPC) is similar to FORTRAN on computation intensive benchmarks.
  3. http://blog.cfelde.com/2010/06/c-vs-java-performance/
  4. http://www.amazon.com/Fixed-Income-Analytics-Developer-Circa/lm/R3FV39FJRU3FE9

 

Open Source Trading Software Or Not?

I have built a few trading systems from scratch during my years with investment banks. So, I have learnt from the many mistakes made. I am recently reviewing some open source system, and would like to share some thoughts.

In general, I am against building software in house. Funds and banks are not software firms. They should focus on making money instead of IT development. Producing software is best left to Microsoft, Google, Apple and IT vendors. These good folks spend their life time doing nothing other than building software. There is no chance that a (small) financial organization can build better systems than they do.

Now, there is an exception to this rule of thumb. When it comes to building trading systems, I suggest that we should do this in house. Alternatively, you can hire a vendor and retain them on contract basis to co-develop the trading system, if you do not have proper talents in house. (It is notoriously difficult to hire very talented programmers in finance because the top quality guys go to GOOG, MSFT, AAPL or startups.)

The worst is to download an open source trading system and simply hope that it will work. Let’s examine why. Firstly, for a piece of very complicated software such as trading system, it is unlikely that the downloaded code works out of the box. There are often too many configurations to do. Even with proper documentations (which is a big assumption), there are still too many plumbing to set up before you can make the first trade, e.g., connectivity to exchanges, data sources, news and announcements, writing and testing strategies in their framework, book-keeping, middle office, clearance. Chances are that you will end up paying the vendor’s service to speed up the set up process.

Secondly, you may think that you have the option to edit their (open) source code if you like. The reality is that you probably cannot do so. There are two problems. The first problem is that, without the authors walking thru the code with you, it is very difficult to understand the source code, the architecture, the assumptions, the hacks and quick-fixes, and even the comments. For the open source trading system that I am playing with, I don’t think I can reasonably edit it without spending a month or two studying it without assistance.

The second and the biggest problem is that even though you can edit the code to your taste, how do you merge your changes with the vendor’s future releases? The vendor knows absolutely nothing about your changes. Whenever they make an update, you will have to manually merge all the changes, assuming that this is even possible. You will then find yourself either 1) rewriting the same code over and over again or 2) maintaining a separate branch from the vendor’s altogether. Neither is satisfactory. Chances are that you will end up paying the vendor to write the code for you.

Thirdly, assuming that you cannot reasonably edit their code even if it is open source, you and your trading strategies must then work under their framework and work with all their assumptions and limitations. There is no one standard way to build a trading system. There are too many possibilities: order routing, order placement, order cancelation, order filling, error handling, alerting.

Take order cancel-replace as an example. There are at least two ways to do it. Number one: you cancel an order, wait for a confirmation, and then place a new order. The problem is that this method is very slow so you may miss the trading opportunity. You may even miss the confirmation that you never send out the replacement order. Number two: you place the new order before sending in the cancelation message. The problem in this case is that you may over-fill your order when both the old and new orders are executed. There is no “correct” way to do it. It all depends on the traders and the trading strategies being executed. Another detail is whether you still want to cancel the order if it is already partially filled. The open source trading code I am reading now does not address this altogether. Chances are that you will end up paying the vendor to customize the code for your applications.

Fourth, if even the trading system is open source, you cannot reasonably understand the code enough to modify it. You at best can only be a user of the system. In this case, what do you do in the future when you want to add additional components to enhance your trading process? For instance, you may want independent and hence external modules such as order aggregators, flow controls, safety guards, etc. You will have to build these components around their framework that you don’t understand. Chances are that you will end up paying the vendor to build these add-on modules for you.

Fifth, some trading systems, for the sake of generality, take a script as a trading strategy. The script may be in proprietary languages such as TradeStation EasyLanguage or in popular language like Ruby. Either way, the problem with scripting is that it is very slow, relatively speaking. The competition in the algorithmic trading space is very fierce. Many firms, e.g., GETCO, spend a lot of resources to tweak their systems or colocation just to get a few milliseconds edge over competitors. Executing scripts line-by-line is very slow. Not being able to tweak, hence optimize, your system is making it even slower. You are already losing the trading game to others in speed. Chances are that you will end up paying the vendor to directly access the underlying API.

Sixth, some solid trading systems come with many exchanges/ECNs gateway access. Many don’t. For those smaller ones, such as the open source ones, they might at best have implemented the FIX protocol. While FIX is a popular protocol in finance, many exchanges and ECNs have their own protocols. Some ECNs may even have different APIs for liquidity takers and liquidity providers. I cannot image the vendor would make open source many these gateway API’s. (Some major businesses live on providing these API for fees not for free after all.) Chances are that you will end up paying the vendor to develop certain gateway access to the markets that you want to trade.

In other words, whether it is open source or not, it is not very relevant. You will end up paying big money either for the software with support or for the services. In either case, you will need a copy of the source code so that you can modify as the last resort. Therefore, from the IT perspective, it does not make sense to buy a piece of trading software from a vendor.

From also the business perspective, you should build the trading system in-house. Or, at least, you need an in-house IT team to work with the 3rd party vendor to co-develop the trading system. The main reason is that a trading system is an embodiment of many business secrets, such as trading strategies, capital allocation, execution optimization, stop loss handling, etc. For instance, when designing a trading system, one question is how to optimize execution to reduce the transaction costs, such as bid-ask. You may want to split the order. You may have some tick prediction mechanism. Etc. You do not want to outsource these business logic or proprietary knowledge to a 3rd party vendor. They are best kept in-house.

Going back to my general principle that a fund is not a software house so it should do as minimal IT as possible, so, where is the line drawn? My suggestion is that we do not build a trading system all from scratch. You want to buy as many “standard” components as possible and only the “standard” components. For instance, it makes no sense to reinvent another FIX engine because some vendors have done it so well. The same go true for historical data database, communication bus, analytic tools, back testing system, market gateways and APIs, GUI controls, etc. What you should build in-house are only the parts that incorporate your business knowledge, e.g., execution optimization, as discussed above.

In conclusion, a trading system is an embodiment of the business secrets of a fund or a bank. You should not out-source this work to a 3rd party vendor. You should develop this in-house. Over time, you grow your internal team of trading system experts. More importantly, your trading system, albeit starting small, will eventually grow to a powerful weapon that gives you an edge over competitors. After all, algorithmic trading is a game of technology competition.

The Demand of Good Financial Engineers

It is very difficult to find people who can produce effective code for mathematical models. In general, mathematicians do not write good code and computer programmers do not understand the (advanced) mathematics. Yet, computer language is the only language we use to turn an idea into a product that people can use. I foresee that the truly valuable talents are those who have creative ideas and are able to implement them. They are the most-sought after in many engineering disciplines, e.g., finance.

Speaking from personal experience, I have been working exclusively in the financial industry, Algorithmic Trading in particular. This field is where mathematics and computer science meet. As an algo-trader, my job is to develop mathematical models and write computer software for automatic execution. In addition, I lead a team of mathematicians to design trading models and a team of programmers to build these systems. From these years of job experience, I find it exceedingly difficult to hire the perfect candidates to work in this algorithmic trading industry.

I have hired some very good statisticians and mathematicians from the top schools. They produce good research and design very sophisticated mathematical models. Unfortunately, they are not able to program to the professional standard their models that are ready to be used by other people. Often the programmers need to translate into C++/C#/Java their prototypes in Matlab/R. On the other hands, it is unrealistic to expect the programmers to develop these mathematical models simply because they do not have the training. In summary, my dilemma is: mathematicians cannot differentiate between inheritance and interface; programmers do not know about hidden Markov Chain.

It occurs very surprising to me that given the prevalence of computer programming in the industries, such as finance and bio-chemistry, universities have not been placing enough emphasize and proper training on their students. In my opinion, in a scientific corporation programming skill is as essential as speaking English. It is a way to communicate ideas (models) in a form that other people can actually use, i.e., a product. Our school curriculums have a number of drawbacks.

First, programming courses are not mandatory for many students, even science students. For instance, statistics students can graduate without ever taking a course in computer programming. They might very well be proficient in Matlab, R, and other specialized software. However, these are not real programming languages. These have little use in an industrial production environment. The students are still not trained in terms of object-oriented concepts, debugging skills, software engineering principles, team collaboration, quick adaptation of new tools and technologies.

Secondly, the professors, instructors, or lecturers teaching the programming courses usually have little industrial programming experience. Speaking from my personal experience, I thought I was a good programmer when I graduated with a PhD in computer science and after spending many years programming for my thesis and homework. It turned out that I was very naïve and ignorant. I was proved to know nothing about industrial programming on my first job. Looking back, the professors teaching programming in universities are probably in the stage where I once was. Most have never delivered a real product (not hands-on anyway).

Therefore, as we are now in the era of a technology driven world, the truly valuable talents are those who can have creative ideas and are able to implement them. I foresee that, gradually, the schools begin to recognize that having good programming skill is as essential as having good communication skill on the job. I am looking forward to changes in the academic curriculums. More emphases are placed on computer programming training across all majors. At the least, all engineering students must be proficient in one modern programming language. Equally important, these programming courses should be taught by experienced professionals rather than academic people who are trained to write journal papers.

In conclusion, I would like to see in universities a new course to teach numerical programming. This is a course to educate science students (not just computer science students) how to code mathematical models. That includes a modern programming language, software engineering methodology, debugging, algorithm design and analysis, effective implementation, and design pattern.

What is missing in our computer science curriculum?

From my years of experience of recruiting computer science students, it seems that universities fail to train students who can build good software. When I hire freshly graduated students, I often need to rewrite their code (sometimes from scratch) before putting the code in production. Most graduates know what inheritance and interface are, but few of them can properly choose and defend which to use. Most graduates have taken classes in “Operating Systems” and thus can talk a lot about threads and scheduling, but few actually have read books like the one from Doug Lea. They cannot be entrusted to write multi-threaded code. Most graduates memorize the visited nodes when writing a function to detect cycle in a linked list, but few of them are aware of Floyd’s cycle-finding algorithm. The funnies thing is that a lot of them think that they are writing object-oriented code because they have replaced ‘struct’ in C with ‘class’ in C++. They are yet to read a book on design pattern.

In other words, these graduates cannot use the programming languages effectively (ineffective). They cannot communicate their ideas to colleagues in their code (incomprehensible). They do not evaluate the space and time complexities of their algorithms (inefficient). They produce spaghetti code that is so difficult to modify (unmaintainable) so we have this motto in the industry: if it ain’t broke, don’t fix it. It is next to impossible to add new requirements without rewriting it from scratch (inextensible). Often when I do code review, the code looks like scribble to me. I have no way to tell whether it is right or wrong (incorrect).

I would like to point out that computer programming is a real professional that requires more than merely reading “C++/Java/C# for Dummies”. We need the programmers to know more than just the programming constructs and some irrelevant theories they never use on the job. Given a project, a competent programmer should be able to understand the problem, design efficient algorithms in terms of space and time complexities, evaluate different implementations, type in legible code, and provide systematic evidences that the code work. It is very unfortunate that universities fail to deliver students that qualify.

Why don’t universities produce good programmers? The major problem lies in the curriculum. Speaking from personal experience, I managed to finished all B.S. computer science requirements, obtained a M.S. and a Ph.D. in this field without ever taking a course in software engineering. In fact, I have never seen a university course with the purpose to teach students how to write good programs, not at the top-tier “research” universities anyway. Somehow the schools assume everyone can read the book such as “C++/Java/C# for Dummies” on their own. Consequently, most fresh graduates write dummy code.

Yes, we do programming projects for courses like Data structure, Operating System, Artificial Intelligence and etc. But these assignments aim to enhance the students’ knowledge in the particular topics. The code is never graded based on how professional it is written. Take Operating System as an example, the assignments aim to enhance the students’ knowledge on the concepts of thread and the OS scheduling algorithms but they are never designed to train students on writing good multi-threaded programs. The textbooks used in class teach about mutex and semaphore but they do not teach how to use them effectively. How many students, after taking an OS class, can design a thread-safe stack? I bet that most of them know what threads are and what a stack is, but few can design an efficient and effective thread-safe stack class.

The second problem is faculty. As a university, most faculty members hold doctorate degrees. The undergraduate courses are usually taught by newly graduated doctorate students. These doctorate students are trained to publish journal papers. Many never deliver any industrial applications in their life time (not hands-on anyway). Not only do they not know how to write good code, but they cannot appreciate how important it is to produce good code. For instance, in a Data Structure course, a professor teaches about stack, but he is unlikely to implement a thread-safe stack class for professional use. Without ever working in the industries, it is very difficult for a professor to understand the details and issues associated with using a thread-safe stack. It would even be more difficult for him to appreciate how important it is to provide systematic evidences to show that his class works and is ready for a third-party to use.

One may argue that “Computer Science” is about science so we study Artificial Intelligence, Operating System, etc. Science is about research and innovation. It is not about programming. My dilemma is that if we cannot go to computer science schools to seek qualified programmers, where should we turn to? Also, what exactly is the use of these university trained computer science graduates if they cannot write good code? After all, not everyone becomes a scientist. Most of them are engineers that code the science ideas into real products that benefit our societies.