Maestriot

Multi-Agent Trust Decision Process for the Internet of Things

Methodology

The research works of the project are structured in 5 scientific workpackages (WP) alongside project management WP.

WP1 focuses on the sensors data model used in the two fields of application. This data model will integrate properties related to their reliability, accuracy and context of use. WP2 will propose a method for security context diagnosis during execution and communication taking into account hardware and software aspects. WP3 will build on the productions of these 2 WPs to use them in a trust management system for the evaluation and update of the trust granted to other agents by analyzing their behaviors and comparing data on the same object but from different sources. All this information will be used by WP4 to propose a decision algorithm adapted to the presence of observations with different levels of trust. The decision processes of WP4 will also be involved in WP1 for active perception and WP3 for a feedback towards trust processes.
All these theoretical realizations will be deployed in demonstrators by WP5. Scenarios defined early in WP5 will give information on sensors and actuators studied in WP1.

Workpackage 1 : Potential threats, efficiency and suitability of sensors

The first goal of this WP is to define use cases from two existing applications that are currently at the cutting edge of IoT technologies and thus could be sensitive to data and communication security concerns. These use cases will serve to build realistic scenarios from real data issued by Industry 4.0 and Connected Cooperative Automated Mobility (CCAM) testbeds (WP5). The second objective will be to identify and model potential threats (e.g. attackers strategies or the source of each feared event) from state-of-the-art research and from the partners’ experience with these applications.

Workpackage 2 : Individual vigilance state model for IoT agent

This task aims at defining a generic multi-level observation model, along with its communication mechanisms that are both compatible with the hardware and software components which compose an IoT autonomous agent. In addition, it will help to identify the “causal chain” that led to the “bad” decision if an attack was not detected.

Workpackage 3 : Decentralized multi-target trust management

Trust management algorithms and protocols will be developed in this WP. A decentralized approach will be adopted to keep sensitive data locally in agents and to reduce communication costs.
WP3 is divided in 3 tasks to address trust according to 3 dimensions:

  1. intra-agent trust: an agent perceives raw data from sensors, both embedded on its execution platform and off-board. This task will develop a trust evaluation method to define their reliability. Two aspects will be considered: the risk of having corrupted data according to the security context (as defined in WP 2) of the sensor and its communication channel; the quality of the sensors according to the execution context (as defined in WP 1).
  2. inter-agent trust: an agent will maintain a trust model of other agents based on their past behaviors and communication. Trust values will first be built by requesting the same information to different sources (including itself) and compare their similarity. Communication on unsecured channels can be also performed to check the presence of attackers. Both the situations of authenticated and unauthenticated agents will be considered (LITIS and LCIS have previously work together on trust management in the absence of authentication for wireless sensors network).
  3. distributed reputation system: a reputation system is a way to share trust information in a multi-agent system to fasten trust learning in the whole system. The proposed reputation system will take into account traditional attacks such as collusion, badmouthing or fake promotions. Additionally, the reputation system have to consider the security context when transmitting trust information and the risk of corruption. Secured zones with trusted agents will be defined to facilitate the transfer of trust values to new agents in order to reduce the cold start problem.

Workpackage 4 : Sequential decision making under uncertainty

This WP aims at the formal and algorithmic study of decentralized decision making for a network of heterogeneous sensors and actuators possibly failing and/or malicious. The focus of this study will be to integrate into the classical game-theoretic models and objectives, e.g., a trust model from WP 3 on the reliability of sensors and information exchanged. We investigate a model that can formalize and help solving a multi-agent sequential decision making under partial observability and trust. This WP also aims at providing a certain security level in a dynamic system populated by defensive or attacking agents without prior knowledge of their type and the complete game model. That naturally leads to the study of a corresponding reinforcement learning problem for security under trust uncertainty and partially observability. To tackle these issues, we will target the following three tasks:

  1. The primary goal of this WP is to define and conduct a complexity study for a formal model of sequential decision-making under partial observability and trust in systems involving self-interest agents, building upon general-sum, zero-sum, or Stackelberg partially observable stochastic games. We will assume a trust model over sensors and actuators is available in this task. While defining the formal model is not a challenging task on its own, determining the class of complexity it belongs will be a important step towards understanding how hard is our problem. We may consider some tractable assumption on which previous work leverage upon, e.g., transition and observation independence amongst agents. 
  2. The second task of the WP involves the study of structural properties underlying these sequential decision-making problems—i.e., different form of uniform continuity properties (e.g., Lipschitz, convexity and concavity properties) regarding the optimal solution of the problems. We build upon previous prior results on various partially observable stochastic games, including decentralized partially observable Markov decision processes or zero-sum partially observrable stochastic games. Little is known regarding structural results of Stackelberg stochastic games under partially observability and trust.
  3. The last task of the WP involves the design of efficient planning and reinforcement learning algorithms for calculating secure strategies under partial observability and trust. Multi-agent deep reinforcement learning techniques will target settings where trust and dynamics models are unknown. We shall build upon previous algorithmic schemes we used in related domains, such as decentralized partially observable Markov decision process .

Workpackage 5 : Demonstrator with full and hybrid simulations

The demonstrator will consist in a Web platform ( “Plateforme Territoire”) [15] with interfaces to both a simulation platform and the technological platforms ITM’factory and or DIWII for the application domain Industry 4.0 and the platform PVAC for CCAM. That platform will expose a single Web API (based on W3C Web of Things standards) to collect space-time data from either environment, such that any approach implemented during the project can be exported and reproduced on other environments. There are three issues. The first one concerns integration of the trust architecture in existing simulation and physical platforms. The second concerns the coupling between these platforms for hybrid simulations.
The third concerns the design process of the demonstrator with two application domains. The following tasks have to be achieved:

  1. design of scenarios illustrating the threats to these systems and how the trust framework should ensure their good behavior. In the Industry 4.0 demonstrator, the scenarios will cover the problematic of simultaneous cooperation and competition of agents in a factory. Performance metrics will include the quantitative and qualitative quality of overall production (or the degradation thereof), In the CCAM testbed, investigated scenarios will include the use of sensor technology among conventional (cameras) and non-conventional (polarimetric, infrared, lidars) that is compatible with real autonomous driving conditions (ideal or adverse conditions). The quality of classification on the input data will be used as the main performance metric to show how the system reacts to potential threats. 
  2. study of the existing/in development technological platforms and simulation platform of the consortium with the highlighting of the sensor/actuator and space/time data management. the result is the specification of a web API for pushing/collecting data to/from the web platform (server side) to the simulation platform and the technological platforms (client side).
  3. implementation of the API specification in each physical simulator and simulators of the system.
  4. integration of the trust framework in each component of the demonstrator.
  5. validation of the implementation thanks to the scenario execution.