How Comes this Unity?

In the second chapter of ‘Out of Control‘, Kevin Kelly describes the surprising unity that emerges in a large flock of mallard ducks:

At dawn, on a weedy Michigan lake, ten thousand mallards fidget. In the soft pink glow of morning, the ducks jabber, shake out their wings, and dunk for breakfast. Ducks are spread everywhere. Suddenly, cued by some imperceptible signal, a thousand birds rise as one thing. They lift themselves into the air in a great thunder. As they take off they pull up a thousand more birds from the surface of the lake with them, as if they were all but part of a reclining giant now rising. The monstrous beast hovers in the air, swerves to the east sun, and then, in a blink, reverses direction, turning itself inside out. A second later, the entire swarm veers west and away, as if steered by a single mind. In the 17th century, an anonymous poet wrote: “…and the thousands of fishes oved as a huge beast, piercing the water. They appeared united, inexorably bound to a common fate. How comes this unity?”

Just as each duck has some set of criteria by which it decides when to swim and when to eat and when to fly off with the crowd, each individual in the flŭd network has some simple rules for when and where to store bits of its data, and for when and from whom to accept bits of data. The trick is to tease out an emergent behavior where each node in this massive flock of computers cooperates to provide extremely robust protection against data loss. The local rules employed to produce this behavior are not so different from those used by the mallard; each individual in the flock must be free to act for itself, each decision must have self-interest driving its outcome and must be made only with local stimulus from the part of the world that it perceives directly, and there is no central leader, chain-of-command, or hierarchical control. The behavior that emerges from the flŭd backup network also shares some common characteristics with the flock. Both survive even when a large number of individuals leave or die, and both continue to function as new individuals join. The system is extremely robust to radically changing conditions and rogue individuals.

In the initial flŭd network, all the nodes are identical — clones programmed to make the same decisions given the same input, over and over again. But flŭd has been designed to encourage the richness of diversity. The software is open. Alternative clients with new behaviors can be created and distributed. No one can anticipate where such diversity will take the network, but the internal mechanisms of flŭd plan for such events in a robust way. All interactions are based on trust, and trust is a value that each individual node determines for itself. Nodes increase their trust for others that behave in ways that they view as beneficial, and decrease their trust for nodes that they deem as misbehaving. Nodes prefer to exchange data with other nodes that they trust. This internal trust mechanism regulates the entire network, brokering how resources can be consumed and dispersed, ensuring that you’ll always be able to recover the data that you store to it.

We believe that no model more robust to failure can be found than those that nature has so generously provided. flŭd’s goal to provide the world’s easiest to manage and most resilient backup system is accomplished by leveraging these natural models.

Posted in emergence, resilience | 1 Comment »