This Blog has posted some of the content in the website/community named Re.Work in the past. It is a London-based thriving and exciting community of people engaged and keen on combining technology, science and entrepreneurship to shape the future landscape of business.
Today I come back to this website to share the content of an interesting interview kindly given by David Samuel, co-founder of RKR Epsilon & Predictive Machines, a physicist and entrepreneur fully committed with machine learning/deep learning business applications.
He will be a presenter at the next Deep Learning in Finance Summit in Singapore. Re.Work talked with him about the topics in Deep Learning in Finance that are currently in development, namely the deep reinforcement learning techniques that are being polished that will certainly be features in future trading desks, becoming de facto standards of algorithmic trading:
At the Deep Learning in Finance Summit in Singapore, David will be sharing expertise on methods using Q-function based reinforcement learning and DQNs trained on simulation models for markets, with data provided by generative models that mimic both the randomness and salient features of actual markets. I asked him a few questions ahead of the summit to learn more.
Please tell us more about your work, and a teaser for your session at the upcoming summit.
I’m David Samuel, co-founder of Prediction Machines which is developing IP and conducting research into the fields of commercial transactions and markets, where the prices of products (commodities, assets, financial instruments, including digital/virtual forms) are uncertain at future time horizons and where there is a need to determine optimal trading strategies. I have worked in the financial markets for over 20 years in roles including quantitative research, trading, risk management and business management. I have also taught classes for the Master’s Degree in Quantitative Finance at Oxford University and presented research at a number of industry events.
At the Deep Learning in Finance Summit I shall be presenting some of our latest research into the use of Q-Function Reinforcement Learning (QRL) algorithms for trading financial instruments, where the implementation is via the use of Deep Q-Networks (DQNs).The starting point of this research is the observation that some experienced traders are able to generate fairly consistent profits from trading the financial markets (accepting that there will be some variability in performance relating to ever-changing market opportunities). Indeed one can specialise this observation to cases where the information available to the trader is limited to the order books, historical price evolution, and trade event history of a small set of products. The actions of the trader are not random, they are informed decisions based on what is happening in the market. An interesting question is then is whether these decisions / actions can be learned?
Ok. Now people familiar with machine learning will rightly ask the question: but isn’t the models used by the quantitative trading desks all over the markets mostly plain old vanilla supervised learning types? Yes they are, but the new implementations of an unsupervised technique such as reinforcement learning is quickly gaining traction, and specially within the more experienced traders with most robust computer science backgrounds. These are algorithms that require the best and most capable expertise:
Traditional approaches to quantitative trading typically involve the use of well-known techniques in machine learning to identify structure (features) within the market data which can then be exploited via an appropriate trading strategy. Whilst this machine learning approach can offer up some profitable features they are often not the ones spotted by the experienced traders.
Reinforcement Learning provides a potential framework for learning how to trade but traditional methods, when presented with a relatively small amount of noisy market data, are plagued by various complexities that make the approach difficult to tackle. However, Q-Learning (a form of Reinforcement Learning) applied to specially designed trading simulation games do provide some promising results. Generating synthetic data that largely mimics the random behaviour of markets but also contains the salient features that will result in exploits presenting themselves is an important part of the process of getting Q-RL to successfully learn how to trade.
Dr. David Samuel gives a nice quick overview of the recent development of transition from traditional machine learning algorithms to the more sophisticated deep reinforcement learning state-of-the-art in the quantitative trading field, whilst also reporting on his own experience as a witness and active involved agent:
How did you get into deep learning?
My background is in theoretical physics and I initially joined the financial markets industry as a quantitative researcher / derivatives trader. I became interested in predictive models of short-term price moves in financial instruments using machine learning methods that were growing in popularity in the industry in the early 2000’s, and had some collaborations with researchers at Oxford University and Cambridge University machine learning groups.A few years later I was fortunate to be able to participate in the Machine Learning Summer School held at Cambridge University (2009) in which one of the lecturers was Professor Geoff Hinton, who presented work on Deep Boltzmann Machines. This was my first introduction to deep learning methods. At that time most of the financial industry was still focused on the use of the more widely known techniques in machine learning, as these methods were more accessible and easier to implement with financial markets data.However, over the past three years a combination of the progress made in deep learning and the resources that have been made available for general use (e.g., TensorFlow) have made deep learning methods more accessible and have therefore become a part of our research efforts at Prediction Machines.
Notwithstanding, the measured tone of critical appraisal as to the true, real impact deep learning shall have with the wider implementations in all kinds of business sectors and industries is welcomed counterpoint:
Which industries or areas do you feel deep learning will have the most beneficial impact?
This is a good question, and a difficult question. Beneficial for whom? A number of very smart people believe that deep learning and its extension into AI will have a significant and disruptive effect on all aspects of global socio-economic. It is too difficult to predict specific domains of beneficial impact because everything is coupled in a complex manner and no doubt where there are benefits in some areas there will also be negative consequences. Obviously, one tends to think about advancements and applications in medicine and health care to be the most important benefits for a global society. However, it is a very complex ecosystem to consider and we are likely to be surprised with some new products or services that don’t yet exist. It is encouraging to see some smart leaders in this space participating in events such as Asilomar 2017 to help steer the path forward.
A glimpse of the near future by someone in-the-know of what it is all about:
What advancements in deep learning would you hope to see in the next 3 years?
It is encouraging to see that the pioneers and leaders in deep learning have adopted an open approach to publishing their work and making specialised resources generally available. The pace of development in deep learning is currently phenomenal and so extrapolating out three years is challenging. I hope that whatever advancements take place continue to be open. I suspect that some of the main technology advancements will be in specialised hardware designed specifically for implementing deep nets.
Last but not least, a point of view about how all this might fit in with current FinTech developments, where Dr.Samuel doesn’t talk much about for instance blockchain developments taking place with its integrations with Artificial Intelligence (AI) frameworks. But we sense that the ecosystems broadly suffer some thinning out of profits due to excessively crowded marketplaces; first-mover advantage with such environments isn’t always strategically sensible:
Will fintech startups continue to influence industry financial services?
In areas such as payments, secure transactions, compliance, investment advice, and micro-finance it would appear that there has been a very good start. There is now an expected crowded field of start-ups ready for some thinning out. Of course, first-mover advantage doesn’t always hold and there is scope for new entrants with better operating models or technology solutions.For trading of financial markets it would seem that there is still scope to fully take into account the enormous amount of data that presents itself. Bringing all this information together in a timely manner and in a way that it can all be combined into optimal decision-making processes remains work-in-progresss.