Given the precision of speech-to-text AI models these days, I decided to use OpenAI’s whisper model together with pyannote-audio in order to build myself an automated podcast transcription tool. As a test example I used the tool to automatically generate a German transcript of a podcast interview that I did some years ago. It actually worked pretty well even for German language. Afterwards I translated the transcript into English using DeepL (and didn’t further modify the output).


You can find further info on the episode’s page of the Money, Markets and Machines Podcast of Scalable Capital.

We talked about:

  • varying degrees of automation and AI
  • problems of overfitting if investment rules are generated by the model
  • the combination of human and machine
  • pecularities of financial data
  • alternative data sources for asset management

You can find the German transcript here:


Transcript

0:00:09 - 0:00:30: Tobias


Good afternoon, welcome again to the Scalable Capital Podcast. I am Tobias Aigner and today I would like to take you into the world of artificial intelligence and machine learning. Specifically, we want to talk about how much automation investment actually tolerates and whether, for example, Scalable's investment algorithm learns its investment rules itself. Our expert on this topic is Christian Groll, Scalable's Head of Quantitative Investment Strategy, and he's sitting here with me today. Hello Christian, good to have you here.


0:00:30 - 0:00:31: Christian


Hi.


0:00:31 - 0:00:50: Tobias


Christian, maybe you could first tell us a little bit about yourself. How did you come to Scalable and what exactly do you do here?


0:00:50 - 0:01:56: Christian


Yes, very much so. I studied business mathematics in Munich and then did my doctorate in statistics at the chair of Professor Stefan Mittnik, who is one of the founders of Scalable Capital. When I saw through Stefan that Scalable Capital offered me the opportunity to put my academic knowledge into practice, I joined Scalable in 2015. With Stefan, we have always been very academically oriented and research and quantitative modeling are basically just as much a part of everyday life here as they were at university. The motivation and in principle our main task is to understand and model the financial market as well as possible. And I don't think we are far away from the university in terms of motivation. But beyond that, I also see it as part of our task to be as transparent as possible to the outside world. We also have a quant blog, where we are gradually describing components of the algorithm and working through financial market topics that are relevant for modeling.


0:01:58 - 0:02:04: Tobias


Let's get into the topic. How would you describe the describe Scalable Capital's investment model?


0:02:05 - 0:02:38: Christian


Yes, I would say the main pillars of the model are a global and across multiple asset classes, diversified investment universe with low product costs, so we use ETFs, automated and rule-based and dynamic risk management, individual portfolio monitoring and adjustment, and importantly also, yes I would say, a strong belief that any investment decisions should be based on as solid empirical data as possible.


0:02:38 - 0:02:46: Tobias


And when you say automated, does that mean that the computer really makes the decisions itself? Can you say that?


0:02:46 - 0:05:40: Christian


Yes, I think I'll have to elaborate a bit more on that first. Not all automation is the same. There are extreme differences in what can be hidden behind the term. In the media, there's always talk of artificial intelligence and machine learning, although no one ever really defines what they mean. In particular, I think it is very important to understand how high the degree of human influence is in an automated allocation decision. So I'll try to sketch it out using the example that I recently read in a really great didactic article by AQR, which is a US hedge fund. The idea is to implement an algorithm that decides for a given text input, i.e. a character string consisting of letters, digits and special characters, whether it is a valid e-mail address or not. So the first variant is, I simply look up all the defined rules for e-mail addresses myself and then hand them over to the computer already ready. For example, I can define that a valid e-mail address must always have an "@" character, must not have any spaces and must not have more than 253 characters. I can now simply pass this set of rules to the computer as a sequence of if-then checks. So basically it tests every requirement I defined and checks if it is fulfilled or not. And if even one requirement is not fulfilled, the string would then be classified as a non-valid e-mail address. So now the computer has no freedom of decision in principle, so it just stubbornly executes the rules given by me. But nevertheless, at the end of the day, I have of course built an algorithm that works fully automated for me. Given inputs, the rules I specify are applied and the corresponding output is produced. So and on the other end of the spectrum, there was now a completely different approach. Namely, I no longer give the computer any rules, but in the end simply give myself a comprehensive collection of input-output combinations. So in our example, let's say we give the computer five million input strings for each of which we have already classified whether it is a valid e-mail address or not. So based on the example data set, the computer then looks for the rules itself with which it will classify future e-mail addresses. So instead of the subject-specific expert knowledge or the assumptions that which we have to translate into a set of rules, we only need this we only need this collection of already evaluated sample data as a prerequisite. evaluated sample data. But "only" in quotation marks, which is is not a matter of course that such data is already available. are already available. Someone has to classify the five million strings to determine whether they are valid e-mail addresses or not. valid e-mail address or not.


0:05:41 - 0:05:47: Tobias


So both variants outlined. The exciting question now is, of course, which one is better or which one should be preferred?


0:05:48 - 0:07:57: Christian


Yes, before we think about that, let's talk briefly about the potential weaknesses of the two approaches. Sure, in the traditional approach, we are of course highly dependent on whether the rules given by the expert are actually correct. That is, of course, relatively clear. With the second approach, you can talk about artificial intelligence to a certain extent, and the problems with the rules that are found are always a bit more difficult to uncover. Let me give you a few examples of what could go wrong. First, let's assume that the computer is suddenly confronted with an input that has not yet occurred in the previous example data set. So in our example now a test address with a space comes along for the first time. If such an input has not occurred in the data so far, the computer must ultimately decide without any point of reference what the best rule is here. So it has to extrapolate and apply a rule beyond the existing set of observed inputs. Second, and of course this is a problem with statistics in general, existing relationships can always change. Historical data may no longer be meaningful for the present. In the past, for example, e-mail addresses always had to end in domains like .com or .de etc.. Nowadays the restrictions are less. For example, my work email ends with @scalable.capital. If it was not yet mapped in the example data set, then the computer is of course working with outdated rules. And yes, thirdly, the rules found by the computer are often extremely intransparent and difficult for humans to understand. So what exactly prompted the computer to classify an e-mail as an invalid is usually unclear. As a result, it is quite possible that an e-mail was classified correctly, but only by chance by applying an actually wrong rule. This may be extremely difficult to notice at first, because the classification on the sample data set was correct. But when applying it to new data, you will soon realize that the wrong rules were learned in the background.


0:07:58 - 0:08:07: Tobias


The bottom line is that which approach is better is decided by whether you trust the computer's rules or the expert's rules more, right?


0:08:07 - 0:09:55: Christian


Yes, you could formulate it like that in general or now translated specifically only to the rules of the computer. The trustworthiness ultimately results from how comprehensive and representative the sample data set is and the extent to which it can be ruled out that the computer does not inadvertently pick up patterns in the data that do not actually exist. The statistical term for this is the so-called overfitting. One of my favorite anecdotes to illustrate the problem I read the other day in one of your texts. For one application, the computer was supposed to automatically recognize tanks in digital images. The basic idea was to somehow use artificial intelligence to create an algorithm that would recognize whether the given image contained a tank or not. So, the computer was fed with example data, so that it can learn a set of rules and on the example data set itself it also worked outstandingly. But when the algorithm was used in reality, it failed miserably. And the reason for that was that the computer had not actually learned to recognize a tank at all, but in the example data set, all the tank images had simply been taken in the sunshine. And so what the computer had actually recognized was just the sunshine in the images. So it basically then identified every sunshine image as a tank image because it had built a completely wrong set of rules, but it happened to work on the sample data anyway. But out of sample, that is, outside the known example data set, it just didn't work anymore. And the avoidance of overfitting, that is, this avoidance of incorrectly picked up patterns in the data, that is certainly one of the greatest challenges in the application of artificial intelligence.


0:09:55 - 0:10:02: Tobias


Let's take a look at the algorithm of scalable. To what extent does artificial artificial intelligence really play a role?


0:10:02 - 0:11:22: Christian


Yes, so far we have only dealt with rather extreme examples of algorithms in order to hopefully make it more or less understandable how different approaches to automation can be and what one can roughly imagine by artificial intelligence. In reality, however, there is of course no reason whatsoever to subject oneself to such extreme black-and-white thinking. So why not just try to combine the best of both worlds? Perhaps I could refer again to the example of e-mail addresses. Even if I, as an expert, am not sure which special characters are actually allowed, I can tell the algorithm to pay special attention to them and then create a suitable set of rules. Or I can simply specify that an @ symbol is mandatory and that there is some maximum length of allowed characters and so on. With this help, the self-learning part of the algorithm can then use the existing sample data much more efficiently, which in the end simply reduces the risk of potential misinterpretation. Transferred to the world of finance, this would mean, for example, that I would tell the algorithm how best to calculate risk. But you could still let it learn how best to handle this information.


0:11:23 - 0:11:53: Tobias


Scalable's dynamic risk management is designed to determine the risk in the portfolio and then derive decisions for action from that. For example, whether to reduce the equity quota or increase commodities or bonds, whatever. In this way, the customer's risk specifications are always adhered to. That is the idea behind it. Is it really the case that I have to imagine this risk management system as receiving part of the rules from you and working out the other part by itself? Can you put it that way?


0:11:53 - 0:15:14: Christian


Yes, here again I would say that there are very many different gradations as to how much leeway I could leave to the computer itself to react to a calculated risk level. And I would say that, for many reasons, we have tightened the thumbscrews rather tightly. The algorithm can certainly not simply create its own rules. But a certain amount of freedom should be allowed to ensure that the investment model is in line with the historical financial market data. A small example perhaps of what I mean, so let's just assume that we are convinced that an increase in financial market risks is generally a rather undesirable phenomenon, to which we want to respond by rebalancing in a certain way. So for simplicity, let's say the desired response would be to reduce the proportion of risky securities in the portfolio and increase the proportion of low-risk securities. That's specified in what way I want to respond, but just not yet how much. So, in other words, for a given increase in risk, I could still either shift only rather slightly or sell all high-risk securities right away. Which reaction is optimal depends on how sustainable the increase in risk is. And in order to find the right measure here, it makes sense to let the computer learn with the help of historical data which measure of reaction had proven to be useful in the past. However, my feeling is that I would not normally speak of machine learning here. In principle, a statistical model is simply estimated with data in a very classical way. The term learning doesn't really mean anything else, but I think it was introduced to give it a special extravagant touch. In any case, I think there are a lot of possibilities in portfolio management to let the computer refine an economically motivated, predefined set of rules with the help of historical data in order to hopefully arrive at an optimal investment decision. This has always been the idea of economics or financial economics, simply the symbiosis of economic theory with empirical modeling. And another example where you would give the computer some leeway is something like conflicting goals that you have to solve. So in general, the investor prefers, let's say, high returns, low risk and low transaction costs. A conflict of objectives exists when an improvement in one of the three objectives always automatically leads to a deterioration in at least one of the other objectives. In other words, if I want to have a portfolio that is optimally positioned in terms of return and risk every day, then I would have to make transactions on an ongoing basis, which in turn would cause costs to skyrocket. So even in such cases, the computer can help to determine the sweet spot, so to speak, the optimal point in the conflict of objectives. Overall, however, I think that in the world of finance in particular, you have to be extremely careful about where you leave the computer and how much leeway you leave it to determine its own rules.


0:15:14 - 0:15:22: Tobias


You just said in the financial world, what do you mean, how is the financial world different from other areas?


0:15:22 - 0:18:58: Christian


I would say, well, the signal-to-noise ratio, that is, the ratio of relevant patterns in the data to simple, random patterns that are ultimately meaningless for further modeling, is extremely low for financial data. Let's somehow take soccer score prediction as a comparison. Assuming now that we consider a knockout match of any two teams, there are only two possible outcomes. Either team A will advance or team B. If I know absolutely nothing about both teams, then my best guess is, well, that the probability for both teams to advance is 50-50. So in principle we can't do anything but guess. But if I know that Team A is the quite successful German national team and Team B is an absolute underdog, then anyone with a little bit of soccer knowledge could predict that probably the German national team will advance. Of course, we're not always right, but let's say we're right eight times out of ten. That's because the structural difference in quality between the two teams is so clear that it overrides the chance that also exists in the game. So in games with relatively unbalanced opponents, I think it's quite easy to make a prediction that works out much better than 50-50. But if you look at the stock market for comparison, even systematic strategies that have been successful for decades have a probability of a positive return of at least over 50 percent on a randomly selected day. The predictive power is therefore almost as precise as a coin toss, and a knowledge advantage only becomes noticeable over a really long period of time. Yes, then the question is, why is that? I think the key point about the financial market is that it is an adaptive system in which the development of prices is influenced by the predictions and views of individual market participants. So if I have a knowledge edge in the financial market, that always means that there is an opportunity for a profitable investment. And if you now assume that market participants generally maximize profit, think and act, then of course they would always have an incentive to try to turn this knowledge advantage into money. So, and now market prices are ultimately also only the result of supply and demand. So if the demand for a security increases due to a certain knowledge advantage, the market price will automatically change as well. And it will continue to do so until there is nothing left to be gained from the original knowledge advantage. At least this is the hypothesis of the efficient financial market, for which the American economist Eugene Fama was awarded the Nobel Prize. Through profit maximization and the interplay of supply and demand, predictions ultimately have a direct influence on market prices themselves, i.e. on the quantity that one actually wanted to predict. This is a fundamental difference to other scientific fields and especially a problem with machine learning. I recently heard a very funny anecdote about this from AQR, who compared the prediction on the financial market with the recognition of cats on photos. The statement was something like, yes, cats don't start turning into dogs as soon as the algorithm gets too good at recognizing cats. That's why machine learning and artificial intelligence play a bigger role in image recognition than in the adaptive financial market.


0:18:58 - 0:19:07: Tobias


Okay, but the bottom line then is that in the financial world, you should just rely much more on predetermined rules and not give the computer too much leeway.


0:19:07 - 0:20:46: Christian


Yes, I do think that empirical studies currently still tend to indicate that less leeway for the computer is definitely advisable. In other words, the risk of otherwise possibly picking up on wrong patterns in the data is simply enormously high and would otherwise add an additional and unnecessarily dangerous component to the investment decision. The requirements for a good set of rules should be that it should prove robust with respect to several dimensions. What do I mean by that? First, yes, it should be consistent. So over a long period of time, and ideally across different investment universes, there should have been demonstrable positive results. It should be robust. So if I now make any slight changes to the parameters, then the whole thing should behave stably. So, for example, if I now only marginally change the model for determining risks, of course it should not come out with fundamentally different investment decisions. And third, it should be plausible. In other words, it should be broadly consistent with existing economic theory. And with some algorithms, as they exist in artificial intelligence, you usually end up with a so-called black box. In principle, this is a set of rules that makes decisions, but without providing much insight into how these decisions are made. And accordingly, the rules chosen by the computer cannot be translated directly into rules that can be understood and interpreted by humans. Of course, this automatically excludes a bit of comparison with existing economic theory.


0:20:47 - 0:21:14: Tobias


Now we have talked a lot about novel algorithms. What I'm also interested in now, what is known, are, for example, new types of data sources. For example, satellite images of parking lots in front of supermarkets that are used to make decisions or compile information. Social media comments, smartphone geolocation data to figure out consumer behavior. Internet search engine data, all that stuff. What do you think about that?


0:21:14 - 0:21:46: Christian


Yes, the question fits perfectly here. I can answer about the same thing again right away. Here, too, my first hint would always be, yes, fixed rules should be consistent, i.e. they should be verifiable over a longer period of time. So, now let's assume I use novel data sources to derive any investment decisions. It doesn't matter whether the rules come from experts or from the computer. If I have the necessary data sources only over a very limited period of time back into the past, then the rules and the resulting benefit can of course only be empirically tested to a very limited extent.


0:21:47 - 0:21:49: Tobias


Why is this so dangerous or what does it lead to?


0:21:50 - 0:23:52: Christian


Well, the full risk profile of an investment strategy can only be guessed at once it has been observed over at least one full economic cycle. Even after such a full economic cycle or even several, it is also possible that one still has no observation of certain extreme events, because they may occur only once in 100 years or so. Ideally, therefore, one has as long as possible observation periods and, if possible, also observations from several different framework conditions of the global economy and financial market. Only then can it be said with sufficient conviction that a set of rules will actually generate positive added value in the long term. The classic metaphor for this is the time series of a turkey before Thanksgiving. In the months before Thanksgiving, he always gets enough to eat. However, anyone who extrapolates on the basis of experience that the turkey will still be in good health and well-fed after Thanksgiving could hardly be more wrong. One needs data from all environmental states, i.e. both the time of fattening and the time of slaughter, to be clear about the risk profile. Is a very short history now an exclusion criterion with these innovative data sources? No, of course not. And we also consider the trend toward more and more available and different data sources to be promising. But I think you have to be careful that certain innovative data sources, such as social media posts, smartphones, geo-locations or something like that, have only been available for a few years. And you just always have to keep that in mind. During that time, we simply haven't had any major upheavals in the stock markets and we've also had, let's say, an extremely extraordinary interest rate environment since then. So the supposed benefits of innovative data sources cannot be transferred to other financial market environments in the same way.


0:23:52 - 0:24:03: Tobias


Finally, after all the algorithms, let's come back to the topic of people, namely the portfolio manager when investing. What makes a good portfolio manager for you?


0:24:04 - 0:25:24: Christian


Humility and the honesty to analyze past decisions relentlessly. In my opinion, every successful investment strategy requires two things. Firstly, a good assessment of the market situation and secondly, you have to draw the right conclusions from it. And overconfidence is a massive risk for investment success. Let's take the example of someone who can correctly predict a coin toss with a 55 percent probability. That is an outstanding ability, ultimately, no human being, no person in the world can do. But if the person now thinks that he would be right 100 percent of the time instead of 55 percent of the time, that is, if he would overestimate himself, so to speak, then he would be inclined to bet almost all his money on a single coin toss. He thinks that there is a 100 percent probability that he will get it right. In reality, however, he would be bankrupt with a 45 percent probability. And yes, that's one of the reasons why I'm a big fan of quantitative, rule-based investment strategies. Numbers don't lie, and when you check the existing set of rules with historical data, you get the probability of misjudgements in black and white, so to speak. In my opinion, this is the best means against overestimating oneself.


0:25:24 - 0:25:37: Tobias


Very interesting, so I humbly say, thank you very much for the interview, Christian.


0:25:37 - 0:25:39: Christian


Thank you too.


0:25:39 - 0:25:47: Tobias


This was our podcast episode on the topic of automation in investment. If you would like more information on this, you can find it on our website or send us an email at podcast@scalable.capital. Thank you very much for listening.


0:25:52 - 0:26:18: Risk Disclaimer


Scalable Capital Vermögensverwaltung GmbH, does not provide investment, legal or tax advice. Should this podcast contain information about the capital market, financial instruments and or other topics relevant to the investment, this information serves solely to explain the services provided. The capital investment is associated with risks. Please refer to the information on our website.