As a cursory performance evaluation to inform the discussion below we also implemented MLPs with the same type of layer scaling for the MNIST task \cite{lecun1998gradient}. The architectures we used were:
- Narrowing Network: (\(784\rightarrow800\rightarrow600\rightarrow400\rightarrow200\rightarrow10\)) Accuracy: 85% on MNIST
- Constant Width Network: (\(784\rightarrow500\rightarrow500\rightarrow500\rightarrow500\rightarrow10\)) Accuracy: 58% on MNIST
- Widening Network: (\(784\rightarrow200\rightarrow400\rightarrow600\rightarrow800\rightarrow10\)) Accuracy: 61% on MNIST
This leads us to consider the narrowing network as the better collaborating network of the three.
Naturally Occurring Networks
The two naturally occurring networks were used were an airport connection network, where directed edges show connections between airports \citep{nr}, and the weights of the edges represent the amount of traffic between the nodes. The second network is an ecology network \citep{nr}, where nodes are species in an environment, edges represent dependence of one species on another, and the weight is the amount of dependence of one species on another. We see the airport network as a more collaborative network as flights must share common resources (gates, runways, airspace) and work together to get people where they are going, as opposed to the ecology network which describes the interactions between animals (i.e. which animals each which others)