This is the second of a 3-part blog post on Autonomous Agents and Multi-Agent Systems (MAS). Herein we will be covering Multi-Agent Systems. If you would like to learn more about Autonomous Agents, check out Part 1.
Introduction
Previously, we saw that Agents are programs or entities capable of autonomous action. They meet their design objectives through exploitation of learned knowledge and exploration of their environment. Hence, Multi-Agent Systems are systems in which at least two such Agents operate. However, this definition is vastly incomplete. Defining Multi-Agent Systems simply by the quantity of Agents would be ignoring volumes of research and evidence. Indeed, it would discount the actual nature of Multi-Agent Systems. Firstly, we must must consider the interaction between and impact of having multiple Agents in a system. It is this interaction between Agents, be it cooperative, adversarial or otherwise, that gets one closer to defining what a Multi-Agent System actually is. Then we can begin to cover use cases and potential applications of MAS.
Multi-Agent Systems
Multi-Agent Systems are weaving their way into the fabric of modern-day computer systems in various domains[1]. Conceptually, the reasoning behind MAS is that there is strength in numbers. Fundamentally, MAS are systems in which “several agents attempt, through their interaction, to jointly solve tasks or to maximize utility”[2, p. 387]. Indeed, cooperation is one of the core traits of Agents as they interact with each other in an environment; more specifically, the same environment[1]. But cooperation it is not the only type of interaction. Agents can also negotiate[3] with each other, behave as adversaries[4] and even learn how to operate as part of a team[5], for instance.
A Note on Trust
In Part 1, we discussed how Agents are similar to humans in certain ways. This comparison also extends to Multi-Agent Systems and humans. Like human to human interaction, Agents in Multi-Agent Systems can benefit from being able to trust each other. In fact, Huynh et al. showed that Agents which utilized FIRE (a trust and reputation system), saw an increase in their reward or utility[6]. This is quite astonishing. In addition, research has led to the discovery that Agents which use trust models can strategize about which other Agents can be trusted. They do so by computing the expected rewards of interacting with those Agents and working that into their decision making[7].
Examples of Multi-Agent Systems
Real-Time Manufacturing Control
Notwithstanding good faith efforts to describe Multi-Agent Systems, no description stand up to thoroughness without providing some examples of applications of same. For example, one area which is undergoing active development in MAS is real-time manufacturing control[8]. To avoid cases where breakdowns or errors at the minute end of the manufacturing process causes widespread disruption, MAS have been deployed to mitigate this risk. Each material, piece of equipment or other entity category of manufacturing gets assigned an Agent. The Agents collaborate with each other to determine the best course of action in resolving the situation[8, p. 28].
Microgrid Control
In the energy sector, microgrids are localized power grids at a local/micro scale, that consist of Distributed Energy Resources (DERs) and are capable of supplying power to their parent grid, and/or or serving the local region with power[9]. Considering that DERs can be anything from a battery to a wind generator, one can already envisage the complexity of managing power generation and load in a microgrid structure. Research shows that Multi-Agent Systems could be applied to tackle this issue[10]. Each DER is assigned its own Agent which can operate in primary or secondary control. In primary control, the Agent deals with the internal operation and potential output of the DER. Whereas in secondary control, the Agent collaborates with Agents of other DERs in order to optimize transmission and load on the microgrid.
Wrap-Up
That’s it, you now know what Autonomous Agents and Multi-Agent Systems are! Keep an eye out for Parts 3, where I will cover how systems might behave with and without Agents. Also we will get to see a very exciting use case where Agents could be very effective. Stay tuned!
If you enjoyed this post, please leave a comment below. Likewise, and for more content and news, why not follow me on Twitter or subscribe to my YouTube channel. For direct contact, feel free to use the contact form.
References
- L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 2, pp. 156–172, 2008.
- L. Panait and S. Luke, “Cooperative multi-agent learning: The state of the art,” Autonomous agents and multi-agent systems, vol. 11, no. 3, pp. 387–434, 2005.
- S. Kraus, “Negotiation and cooperation in multi-agent environments,” Artificial intelligence, vol. 94, no. 1-2, pp. 79–97, 1997.
- L. Pinto, J. Davidson, R. Sukthankar, and A. Gupta, “Robust adversarial reinforcement learning,” in International Conference on Machine Learning, PMLR, 2017, pp. 2817–2826.
- H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, and E. Osawa, “Robocup: The robot world cup initiative,” in Proceedings of the first international conference on Autonomous agents, 1997, pp. 340–347.
- T. D. Huynh, N. R. Jennings, and N. R. Shadbolt, “An integrated trust and reputation model for open multi-agent systems,” Autonomous Agents and Multi-Agent Systems, vol. 13, no. 2, pp. 119–154, 2006.
- S. D. Ramchurn, D. Huynh, and N. R. Jennings, “Trust in multi-agent systems,” The knowledge engineering review, vol. 19, no. 1, pp. 1–25, 2004.
- V. Marik and D. McFarlane, “Industrial adoption of agent-based technologies,” IEEE intelligent systems, vol. 20, no. 1, pp. 27–35, 2005.
- Wikipedia contributors, Microgrid — Wikipedia, the free encyclopedia, https://en.wikipedia.org/w/index.php?title=Microgrid&oldid=1064092132, [Online; accessed 20-March-2022], 2022.
- W.-D. Zheng and J.-D. Cai, “A multi-agent system for distributed energy resources control in microgrid,” in 2010 5th International Conference on Critical Infrastructure (CRIS), IEEE, 2010, pp. 1–5.
