autonomous-agents

Autonomous Agents and Multi-Agent Systems – Part 3

Introduction

This is the third and final installment of a 3-part blog post on Autonomous Agents and Multi-Agent Systems (MAS). If you haven’t done so already, I highly recommend a quick skim over Part 1 and Part 2 for background on Autonomous Agents and Multi-Agent Systems! To summarize, in Parts 1 & 2, we learned that Agents are programs or entities capable of autonomous action and meet their design objectives through exploitation of learned knowledge and exploration of their environment. We also saw that Multi-Agent Systems are systems in which multiple Agents operate, cooperate and/or behave adversarialy to achieve individual or collective goals.

To further emphasize the nature of Agent-based systems, it would be a good shout to compare the difference between systems with Agents and systems without Agents. Of course no text on this topic would be complete without wrapping up with a potential future use case, and I believe regime change detection in financial markets could really benefit from the adoption of MAS. Read on!

Directional Change

Possible Application of MAS: Directional Change - Fig. Source: Wikipedia

Agent vs Non-Agent-Based Systems

Agent-Based Systems

To hypothesize around how an Agent-based system might differs one that does not use Agents, it is worth considering what traits Agents possess and how they might be manifested and observed in a system. As previously mentioned, Agents sometimes take random actions, or ‘explore’ their environment. When a system has entities taking random actions there is going to be a degree of irrationality attributed to certain system events. In Multi-Agent Systems where interaction between Agents exists, Bloembergen et. al attribute “highly dynamic and non-deterministic”[1, p. 659] to the environments hosting such Agents.

Financial Market Trading

One area above all that has been drenched in activity from Agents is financial market trading[2]. Studies show that as a result of Agent presence in this arena, financial markets could be more challenging towards the end of the trading day as opposed to the beginning[3]. Kluger and McBride claim that this is a result of Agents showing up towards the start of the trading day and become “informed agents” by market close[3, p. 27].

The previous example uses a hostile environment in which numerous Agents, presumably some untrained, compete against each other for their own gain. However, no two environments are the same. To highlight how Agent-based systems behave differently to non-Agent-based systems, the notion of system efficiency in an engineering or manufacturing environment could also work. Consider two systems, A and B. System A does not use Agents and system B uses Agents trained using Reinforcement Learning. As system B has been optimized with a MAS, the behavior of system B would be such that KPIs and other readings of efficiency would be much more favorable than those of system A.

Non-Agent-Based Systems

What about software? Programs such as payroll software and standard spell-checkers also operate in environments in which the program senses and takes action. However, such programs do not exhibit “temporal continuity” – a property of Autonomous Agents[4, p. 4]. In the spell-checker example, the non-Agent-based system would simply review the text after word or text completion, whereas an Agent would sense the state of the text as the user types and actuate in real-time[4].

It’s immediately obvious that without Autonomous Agents, systems will not exhibit this so called ‘stochastic’ behavior, i.e. actions that from the outside, appear to have no rationale behind them. However, without autonomous learning, the same systems fall short of self-improvement over time.

Future Impact of Agents

Financial Market Regime Change Detection

One very important phenomenon in financial markets is the occurrence of regime change[5]. A regime is a representation of the collective sentiment of a financial instrument. When the overall sentiment changes such that the market enters into a completely different trend, it is said that a ‘Regime Change’ (RC) has occurred[6, p. 40]. Much work has been done on RC detection as well as its uses and implications[5], [7]–[9]. A tried and tested approach for financial market analysis is using time-series methods[10].

Directional Change

Recently however, a supplemental methodology known as Directional Change (DC) is emerging[11], [12]. Albeit they can both be used for regime change detection, the DC technique is fundamentally different to time-series methodology in that the temporal aspect of analysis is measured in intrinsic time[13]. This means that time-intervals are not the main unit of abstraction, but rather the time between events. As well as this fundamental difference, Directional Change introduces a new set of indicators and a parameter known as the DC threshold[14]. A full review of DC indicators is outside the scope of this report but the DC threshold requires explanation to motivate the utility of Agents in this use case.

Directional Change Threshold

The DC threshold is the percentage increase or decrease in a market price since the last extreme point, such that upon hitting the threshold a DC event is triggered[14, p. 1]. As the DC threshold is lowered, so too is the bar for triggering DC events. Conceptually speaking, arbitrarily small DC thresholds could be used to detect regime change at low timeframes (e.g. 1/2/5/10 minute intervals). Likewise, arbitrarily large DC thresholds could be used to detect regime change at the highest timeframes (e.g. 1/4/12 hour intervals up to daily and weekly).

Financial Market Early Warning System

High-Level Approach

As Chen and Tsang suggested, the DC technique could be applied to serve as a financial market early warning signal[6, p. 100]. Agents could play a key role in such an endeavor. Having covered the necessary background knowledge, it is possible to hypothesize around the utility of Agents in creating such a signal. At a high-level, we would adopt a multi-Agent approach. For each level range of DC threshold specificity we would assign an Agent to autonomously determine the optimal DC threshold to use within its respective DC level range. This could be done with effective application of reinforcement learning, with rewards and penalties accrued as appropriate. Indeed, there has already been achieved[12]. Once the learning has converged at each time scale of the financial market, the system is ready to begin issuing signals.

Signal Functionality

How do the signals work? For simplification, assume that we are aiming for regime change detection at a daily time interval. Therefore, the agents would be configured to be able to communicate with one another, either directly or through a bus. The workflow begins with the Agents operating at the lowest DC thresholds. Once a regime change has been detected as say the lowest level, say through Agent A it communicates with the Agent operating at the level up, say Agent B. If and only if Agent B receives a signal from Agent A, Agent B waits a period of time to see if a regime change takes place. If a regime change occurs, Agent B signals Agent C, and so propagation continues in the upward direction. If no break in the chain occurs, and the last Agent receives a signal from it is predecessor, its job is to fire off an early warning signal, stating that there is potential for a regime change in its respective timeframe, at a 1-day interval for example.

Closing Thoughts

Agents & MAS

In summary, Agents share some properties of traditional computer systems and algorithms but the similarities begin and end there. Whereas algorithms are constructed with the ‘how’ built into the procedure, Agents are autonomous in achieving their objectives. With autonomy comes decision making and Agents use various techniques to make decisions and explore their environments. Multi-Agent systems are systems consisting of an environment with two or more Agents operating autonomously and interactively to fulfill their objectives, either collaboratively or in an adversarial fashion. There are also other types of interaction between Agents in Multi-Agent Systems, such as negotiation and cooperating as part of a team. Research has shown that like Humans, Agents in MAS can benefit from utilizing trust mechanisms in so far as increasing their chances of obtaining rewards.

Agent vs Non-Agent-Based Systems

In terms of a comparison of systems with and without agents, systems with Agents acting therein have been shown to be very dynamic and difficult in terms of determining event outcome. In another case, namely the stock market, the presence of Autonomous Agents were said to increase the difficulty of profitability towards the end of the trading day. It was also shown that programs and Agents are not the same thing. We can conclude that Agents have exclusive properties such as being temporally-continuous, autonomous and adaptive that differentiates them from programs.

Potential Use Case

One potential area which could benefit from Autonomous Agents and Multi-Agent Systems is risk management in the financial sector. Albeit MAS already exist in this area, there is a considerable gap in the application of same in combination with the Directional Change technique for financial market regime change detection. Whilst traditional time-series methods have worked to some degree to predict regime change, the combination of directional change and time-series methods is more effective. One area of further research is to use Directional Change to build a financial market early warning system. Indeed, we have seen how a MAS with Agents dedicated to regime change detection at various timeframes could serve as an early warning system for regime change at a macro-economic level.

Future Outlook

Looking to the future, it is possible that Agents will have human-like intelligence and be able to maintain autonomy whilst determining and achieving altered objectives as circumstances change. For now, Agents aim to fulfill objectives handed down by the designer and this alone has already led to advances in applications across various sectors.

If you enjoyed this post, please leave a comment below. Likewise, and for more content and news, why not follow me on Twitter or subscribe to my YouTube channel. For direct contact, feel free to use the contact form.

References

  1. D. Bloembergen, K. Tuyls, D. Hennes, and M. Kaisers, “Evolutionary dynamics of multi-agent learning: A survey,” Journal of Artificial Intelligence Research, vol. 53, pp. 659–697, 2015.
  2. M. P. Wellman and U. Rajan, “Ethical issues for autonomous trading agents,” Minds and Machines, vol. 27, no. 4, pp. 609–624, 2017.
  3. B. D. Kluger and M. E. McBride, “Intraday trading patterns in an intelligent autonomous agent-based stock market,” Journal of Economic Behavior & Organization, vol. 79, no. 3, pp. 226–245, 2011.
  4. S. Franklin and A. Graesser, “Is it an agent, or just a program?: A taxonomy for autonomous agents,” in International workshop on agent theories, architectures, and languages, Springer, 1996, pp. 21–35.
  5. A. Ang and G. Bekaert, “International asset allocation with regime shifts,” The review of financial studies, vol. 15, no. 4, pp. 1137–1187, 2002.
  6. J. Chen and E. P. Tsang, Detecting Regime Change in Computational Finance: Data Science, Machine Learning and Algorithmic Trading. Chapman and Hall/CRC, 2020.
  7. E. Tsang and J. Chen, “Regime change detection using directional change indicators in the foreign exchange market to chart brexit,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 3, pp. 185–193, 2018.
  8. A. Ang and A. Timmermann, “Regime changes and financial markets,” Annu. Rev. Financ. Econ., vol. 4, no. 1, pp. 313–337, 2012.
  9. S. Baek, S. K. Mohanty, and M. Glambosky, “Covid-19 and stock market volatility: An industry level analysis,” Finance research letters, vol. 37, p. 101 748, 2020.
  10. R. S. Tsay, Analysis of financial time series. John wiley & sons, 2005.
  11. M. Aloud, E. Tsang, R. Olsen, and A. Dupuis, “A directional-change event approach for studying financial time series,” Economics, vol. 6, no. 1, 2012.
  12. N. Alkhamees and M. Aloud, “Dcrl: Approach for pattern recognition in price time series using directional change and reinforcement learning,”
  13. [13] U. A. Muller, M. M. Dacorogna, R. D. Dave, O. V. Pictet, R. B. Olsen, and J. R. Ward, “Fractals and intrinsic time: A challenge to econometricians,” Unpublished manuscript, Olsen & Associates, Zurich, p. 130, 1993.
  14. [14] E. Tsang, “Directional changes, definitions,” Working Paper WP050-10 Centre for Computational Finance and Economic Agents (CCFEA), University of Essex Revised 1, Tech. Rep., 2010.

Share this post

Share on twitter
Share on linkedin
Share on email

Also by Jonathan:

5 1 vote
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x