Generating protocols
Note
This is currently an experimental feature. To try it follow this guide.
How to run
First make sure you are inside your AEA's folder (see here on how to create a new agent).
Then run
aea generate protocol <path-to-protocol-specification>
where <path-to-protocol-specification>
is the path to a protocol specification file.
If there are no errors, this command will generate the protocol and place it in your AEA project. The name of the protocol's directory will match the protocol name given in the specification. The author will match the registered author in the CLI. The generator currently produces the following files (assuming the name of the protocol in the specification is sample
):
message.py
: defines messages valid under thesample
protocolserialisation.py
: defines how messages are serialised/deserialised__init__.py
: makes the directory a packageprotocol.yaml
: contains package information about thesample
protocolsample.proto
protocol buffer schema filesample_pb2.py
: the generated protocol buffer implementationcustom_types.py
: stub implementations for custom types (created only if the specification contains custom types)
Protocol Specification
A protocol can be described in a YAML file. As such, it needs to follow the YAML format. The following is an example protocol specification:
---
name: two_party_negotiation
author: fetchai
version: 0.1.0
license: Apache-2.0
aea_version: '>=0.9.0, <0.10.0'
description: 'A protocol for negotiation over a fixed set of resources involving two parties.'
speech_acts:
cfp:
query: ct:DataModel
propose:
offer: ct:DataModel
price: pt:float
accept: {}
decline: {}
match_accept: {}
...
---
ct:DataModel: |
bytes data_model = 1;
...
---
reply:
cfp: [propose, decline]
propose: [accept, decline]
accept: [decline, match_accept]
decline: []
match_accept: []
roles: {buyer, seller}
end_states: [successful, failed]
...
Each protocol specification YAML file must have a minimum of one, and a maximum of three YAML documents (each YAML document is enclosed within --- and ...).
Basic Protocol Detail and Messages Syntax
The first YAML document is mandatory in any protocol specification. It contains some basic information about the protocol and describes the syntax of communicative messages allowed under this protocol.
The allowed fields and what they represent are:
name
: The name of the protocol (written in snake_case)author
: The creator of the protocolversion
: The current version of the protocollicense
: Licensing informationaea_version
: The version(s) of the framework that support this protocol. The format is described here.description
: A short description of the protocol
All of the above fields are mandatory and each is a key/value pair, where both key and value are YAML strings.
In addition, the first YAML document in a protocol specification must describe the syntax of valid messages according to this protocol. Therefore, there is another mandatory field: speech-acts
, which defines the set of performatives valid under this protocol, and a set of contents for each performative.
A performative defines the type of a message (e.g. propose, accept) and has a set of contents (or parameters) of varying types.
The format of the speech-act
is as follows: speech-act
is a dictionary, where each key is a unique performative (YAML string), and the value is a content dictionary. If a performative does not have any content, then its content dictionary is empty, e.g. accept
, decline
and match_accept
in the above specification.
A content dictionary in turn is composed of key/value pairs, where each key is the name of a content (YAML string) and the value is its type (YAML string). For example, the cfp
(short for 'call for proposal') performative has one content whose name is query
and whose type is ct:DataModel
.
Types
The specific types which could be assigned to contents in a protocol specification are described in the table below.
Types are either user defined (i.e. custom types) or primitive:
- Custom types are prepended with
ct:
and their format is described using regular expression in the table below. - Primitive types are prepended with
pt:
. There are different categories of primitive types, e.g.<PT>
such as integers and booleans,<PCT>
such as sets and lists, and so on. Primitive types are compositional:- For example, consider
pt:set[...]
under<PCT>
, i.e. an unordered collection of elements without duplicates. Apt:set[...]
describes the type of its elements (called "sub-type") in square brackets. The sub-type of apt:set[...]
must be a<PT>
(e.g.pt:int
,pt:bool
). - In describing the format of types,
/
between two sub-types should be treated as "or". For example, the sub-type of apt:optional[...]
is either a<PT>
,<CT>
,<PCT>
,<PMT>
or an<MT>
.
- For example, consider
A multi type denotes an "or" separated set of sub-types, e.g. pt:union[pt:str, pt:int]
as the type of a content
means content
should either be a pt:int
or a pt:float
.
An optional type for a content denotes that the content's existence is optional, but if it is present, its type must match pt:optional[...]
's sub-type.
Type | Code | Format | Example | In Python |
---|---|---|---|---|
Custom types1 | <CT> |
ct:RegExp(^[A-Z][a-zA-Z0-9]*$) |
ct:DataModel |
Custom Class |
Primitive types | <PT> |
pt:bytes |
pt:bytes |
bytes |
pt:int |
pt:int |
int |
||
pt:float |
pt:float |
float |
||
pt:bool |
pt:bool |
bool |
||
pt:str |
pt:str |
str |
||
Primitive collection types | <PCT> |
pt:set[<PT>] |
pt:set[pt:str] |
FrozenSet[str] |
pt:list[<PT>] |
pt:list[pt:int] |
Tuple[int, ...] * |
||
Primitive mapping types2 | <PMT> |
pt:dict[<PT>, <PT>] |
pt:dict[pt:str, pt:bool] |
Dict[str, bool] |
Multi types | <MT> |
pt:union[<PT>/<CT>/<PCT>/<PMT>, ..., <PT>/<CT>/<PCT>/<PMT>] |
pt:union[ct:DataModel, pt:set[pt:str]] |
Union[DataModel, FrozenSet[str]] |
Optional types | <O> |
pt:optional[<MT>/<PMT>/<PCT>/<PT>/<CT>] |
pt:optional[pt:bool] |
Optional[bool] |
* This is how variable length tuples containing elements of the same type are declared in Python; see here.
Protocol Buffer Schema
Currently, there is no official method provided by the AEA framework for describing custom types in a programming language independent format. This means that if a protocol specification includes custom types, the required implementations must be provided manually.
Therefore, if any of the contents declared in speech-acts
is of a custom type, the specification must then have a second YAML document, containing the protocol buffer schema code for every custom type.
You can see an example of the second YAML document in the above protocol specification.
Dialogues
You can optionally describe some of the structural details of dialogues conforming to your protocol in a third YAML document in the protocol specification.
The allowed fields and what they represent are:
reply
: The reply structure of speech-actsroles
: The roles of agents participating in dialoguesend_states
: The final states a dialogue could end up in.
All of the above fields are mandatory.
reply
specifies for every performative, what its valid replies are. If a performative per_1
is a valid reply to another per_2
, this means a message with performative per_1
can target a message whose performative is per_2
.
reply
is a dictionary, where the keys are the performatives (YAML string) defined in speech-acts
. For each performative key, its value is a list of performatives which are defined to be a valid reply.
For example, valid replies to cfp
are propose
and decline
.
roles
lists the roles are agents which participate in dialogues conforming with your protocol.
roles
takes a set, which may contain two or one roles, each role being a YAML string.
If there are two roles, each agent has a distinguished role in the dialogue (e.g. buyer and seller in the above specification). If there is one role, then the two agents in a dialogue take the same role.
end_states
lists the final states a dialogue based on your protocol may terminate in.
end_states
is a list of YAML strings.
Notes
- Currently, there is no way to describe custom types in a programming language independent format. This means that if a protocol specification includes custom types, the required implementations must be provided manually.
- Before generating the protocol, the protocol buffer schema code for every custom type must be provided in the protocol specification.
- Once the generator is called, it produces a
custom_types
module containing stub implementations for every custom type in the specification. The user must then modify this module and add implementations for every custom type in the specification. This includes implementations of how an object of a custom type can be encoded and decoded using protocol buffer. - Note, currently the way custom types are dealt with in the generator is admittedly inconvenient. The reason is, the generator does not know the structure of custom types and how they may be serialised/deserialised. Although this approach works, it is only a temporary solution until further work on a programming language-independent type description language is finished (similar to how the generator is designed to be a programming language-independent protocol description language).
- Currently, the first element in
pt:dict
cannot be a<CT>
,pt:float
orpt:bytes
. This is because of a constraint in protocol buffer version 3 which is the framework's underlying serialisation mechanism. In a future version, we may address this limitation, in which case we will relax this constraint. - In protocol buffer version 3, which is the version used by the generator, there is no way to check whether an optional field (i.e. contents of type
pt:optional[...]
) has been set or not (see discussion here). In proto3, all optional fields are assigned a default value (e.g.0
for integers types,false
for boolean types, etc). Therefore, given an optional field whose value is the default value, there is no way to know from the optional field itself, whether it is not set, or in fact is set but its value happens to be the default value. Because of this, in the generated protocol schema file (the.proto
file), for every optional content there is a second field that declares whether this field is set or not. We will maintain this temporary solution until a cleaner alternative is found. - Be aware that currently, using the generated protocols in python, there might be some rounding errors when serialising and then deserialising values of
pt:float
contents.
Demo instructions
First, create a new AEA project:
aea create my_aea
cd my_aea
Second, run the generator on the sample specification:
aea generate protocol ../examples/protocol_specification_ex/sample.yaml
This will generate the protocol and place it in your AEA project.
Third, try generating other protocols by first defining a specification, then running the generator.