机器学习与数据科学博士生系列论坛（第七十二期）—— Federated Reinforcement Learning with Heterogeneity among Agents
报告人：金昊 (北京大学)
时间：2024-05-23 16:00-17:00
地点：腾讯会议 446-9399-1513
摘要：
Federated reinforcement learning (FedRL) is a relatively novel field of multi-agent reinforcement learning, which allows agents located in different environments to cooperatively learn a shared policy without sharing locally collected data.
Challenges of FedRL mainly lie in heterogeneity among agents, which indicates that there exists difference among detailed settings of participated agents.
Based on the classical formulation of a Markov Decision Process (MDP), this paper discusses heterogeneity among agents from three different sources: heterogeneity among transition dynamics, heterogeneity among constraint signals, and heterogeneity among state spaces.
These sources of heterogeneity respectively correspond to different realistic scenarios of applying FedRL algorithms.
Extending the formulation of an MDP with heterogeneity among agents, this paper manages to provide theoretical justification for the proposed FedRL algorithms.
In this talk, we have designed proper communication protocols for the derivation of FedRL algorithms.
Specifically, agents utilizing FedRL algorithms independently update their policies with locally collected experience, and periodically communicate their knowledge via exchanging their policy-related parameters.
Such communication protocols are simple but attractive, which enable us to derive various FedRL algorithms from classical single-agent RL algorithms.
论坛简介：该线上论坛是由张志华教授机器学习实验室组织，每两周主办一次（除了公共假期）。论坛每次邀请一位博士生就某个前沿课题做较为系统深入的介绍，主题包括但不限于机器学习、高维统计学、运筹优化和理论计算机科学。
