# How collective learning works¶

A Colearn experiment begins when a group of entities, referred to as *learners*, decide on a model architecture and
begin learning. Together they will train a single global model. The goal is to train a model that performs better
than any of the learners can produce by training on their private data set.

### How Training Works¶

Training occurs in rounds; during each round the learners attempt to improve the performance of the global shared
model.
To do so each round an **update** of the global model (for example new set of weights in a neural network) is proposed.
The learners then **validate** the update and decide if the new model is better than the current global model.

If enough learners *approve* the update then the global model is updated. After an update is approved or rejected a
new round begins.

The detailed steps of a round updating a global model *M* are as follows:

- One of the learners is selected and proposes a new updated model
*M'* - The rest of the learners
**validate***M'* - If
*M'*has better performance than*M*against their private data set then the learner votes to approve - If not, the learner votes to reject
- The total votes are tallied
- If more than some threshold (typically 50%) of learners approve then
*M'*becomes the new global model. If not,*M*continues to be the global model - A new round begins.

By using a decentralized ledger (a blockchain) this learning process can be run in a completely decentralized, secure and auditable way. Further security can be provided by using differential privacy to avoid exposing your private data set when generating an update.

## Learning algorithms that work for collective learning¶

Collective learning is not just for neural networks; any learning algorithm that can be trained on subsets of the data and which can use the results of previous training rounds as the basis for subsequent rounds can be used. Neural networks fit both these constraints: training can be done on mini-batches of data and each training step uses the weights of the previous training step as its starting point. More generally, any model that is trained using mini-batch stochastic gradient descent is fine. Other algorithms can be made to work with collective learning as well. For example, a random forest can be trained iteratively by having each learner add new trees (see example in mli_random_forest_iris.py). For more discussion, see here.

## The driver¶

The driver implements the voting protocol, so it handles selecting a learner to train, sending the update out for voting, calculating the vote and accepting or declining the update. Here we have a very minimal driver that doesn't use networking or a blockchain. Eventually the driver will be a smart contract. This is the code that implements one round of voting:

```
def run_one_round(round_index: int, learners: Sequence[MachineLearningInterface],
vote_threshold=0.5):
proposer = round_index % len(learners)
new_weights = learners[proposer].mli_propose_weights()
prop_weights_list = [ln.mli_test_weights(new_weights) for ln in learners]
approves = sum(1 if v.vote else 0 for v in prop_weights_list)
vote = False
if approves >= len(learners) * vote_threshold:
vote = True
for j, learner in enumerate(learners):
learner.mli_accept_weights(prop_weights_list[j])
return prop_weights_list, vote
```

The driver has a list of learners, and each round it selects one learner to be the proposer. The proposer does some training and proposes an updated set of weights. The driver then sends the proposed weights to each of the learners, and they each vote on whether this is an improvement. If the number of approving votes is greater than the vote threshold the proposed weights are accepted, and if not they're rejected.

## The Machine Learning Interface¶

```
# ------------------------------------------------------------------------------
#
# Copyright 2021 Fetch.AI Limited
#
# Licensed under the Creative Commons Attribution-NonCommercial International
# License, Version 4.0 (the "License"); you may not use this file except in
# compliance with the License. You may obtain a copy of the License at
#
# http://creativecommons.org/licenses/by-nc/4.0/legalcode
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# ------------------------------------------------------------------------------
import abc
from enum import Enum
from typing import Any, Optional
import onnx
import onnxmltools
import sklearn
import tensorflow as tf
import torch
from pydantic import BaseModel
from tensorflow import keras
model_classes_keras = (tf.keras.Model, keras.Model, tf.estimator.Estimator)
model_classes_scipy = (torch.nn.Module)
model_classes_sklearn = (sklearn.base.ClassifierMixin)
def convert_model_to_onnx(model: Any):
"""
Helper function to convert a ML model to onnx format
"""
if isinstance(model, model_classes_keras):
return onnxmltools.convert_keras(model)
if isinstance(model, model_classes_sklearn):
return onnxmltools.convert_sklearn(model)
if 'xgboost' in model.__repr__():
return onnxmltools.convert_sklearn(model)
if isinstance(model, model_classes_scipy):
raise Exception("Pytorch models not yet supported to onnx")
else:
raise Exception("Attempt to convert unsupported model to onnx: {model}")
class DiffPrivBudget(BaseModel):
target_epsilon: float
target_delta: float
consumed_epsilon: float
consumed_delta: float
class ErrorCodes(Enum):
DP_BUDGET_EXCEEDED = 1
class TrainingSummary(BaseModel):
dp_budget: Optional[DiffPrivBudget]
error_code: Optional[ErrorCodes]
class Weights(BaseModel):
weights: Any
training_summary: Optional[TrainingSummary]
class DiffPrivConfig(BaseModel):
target_epsilon: float
target_delta: float
max_grad_norm: float
noise_multiplier: float
class ProposedWeights(BaseModel):
weights: Weights
vote_score: float
test_score: float
vote: Optional[bool]
class ModelFormat(Enum):
PICKLE_WEIGHTS_ONLY = 1
ONNX = 2
class ColearnModel(BaseModel):
model_format: ModelFormat
model_file: Optional[str]
model: Optional[Any]
def deser_model(model: Any) -> onnx.ModelProto:
"""
Helper function to recover a onnx model from its deserialized form
"""
return onnx.load_model_from_string(model)
class MachineLearningInterface(abc.ABC):
@abc.abstractmethod
def mli_propose_weights(self) -> Weights:
"""
Trains the model. Returns new weights. Does not change the current weights of the model.
"""
pass
@abc.abstractmethod
def mli_test_weights(self, weights: Weights) -> ProposedWeights:
"""
Tests the proposed weights and fills in the rest of the fields
"""
@abc.abstractmethod
def mli_accept_weights(self, weights: Weights):
"""
Updates the model with the proposed set of weights
:param weights: The new weights
"""
pass
@abc.abstractmethod
def mli_get_current_weights(self) -> Weights:
"""
Returns the current weights of the model
"""
pass
@abc.abstractmethod
def mli_get_current_model(self) -> ColearnModel:
"""
Returns the current model
"""
pass
```

There are four methods that need to be implemented:

`propose_weights`

causes the model to do some training and then return a new set of weights that are proposed to the other learners. This method shouldn't change the current weights of the model - that only happens when`accept_weights`

is called.`test_weights`

- the models takes some new weights and returns a vote on whether the new weights are an improvement. As with propose_weights, this shouldn't change the current weights of the model - that only happens when`accept_weights`

is called.`accept_weights`

- the models accepts some weights that have been voted on and approved by the set of learners. The old weights of the model are discarded and replaced by the new weights.`current_weights`

should return the current weights of the model.

For more details about directly implementing the machine learning interface see the tutorial here