Complex systems do not follow linear, predictable chains of cause and effect. Instead, system trajectories can diverge wildly into entirely different regimes.

Most of us are familiar with the phrase 'tipping point'. We tend to associate it with moments of no return: when overfishing crosses a threshold that causes fish stocks to collapse or when social unrest reaches a breaking point resulting in riot or revolution. The concept is often associated with an extreme shift, brought about by what seems to be a slight variance in what had been incremental change. A system that seemed stable is pushed until it reaches a breaking point, at which point a small additional push results in a dramatic shift in outcomes.

Example:

Water molecules respond to two critical points: zero degrees, when they shifts from fluid to solid state; and one hundred degrees, when they shift from fluid to vapor state. We see that the kinds of behavior that water molecules will obey is context dependent:  they maintain fluid behaviors within, and only within, the context of a certain temperature range. If we examine why the behavior of the water changes, we realize that fluid behavior within the zero to 100 range is the behavior that involves the least possible energy expenditure on the part of the water molecules given their environmental context. Once this context shifts - becoming too cold or too hot - a particular behavioral mode is no longer that which best conserves energy. Water molecules have the capacity to enact three different kinds of behavioral modes - frozen, fluid, or vapor - and the way these modes come to be enacted is subject to whichever mode involved the least energy expenditure within a given context.

Minimizing Processes:

Another way to think about this, using a complex systems perspective, is that the global behavioral dynamics are moving from one Attractor States to another. When the context changes, the water molecules are forced into a different "basin of attraction" (another word for an attractor state), and this triggers a switch in their mode.

In all complex systems this switch from one basin of attraction to another is simply the result of a system moving from a regime of behavior that, up until a certain point, involved a minimized energy expenditure. Beyond that point (the tipping point) another kind of behavioral regime encounters less resistance, conserving energy expenditures given a shifting context.

A tipping point, or critical point is one where a system moves from one regime of 'fit' behavior into another. We can imagine the point above as a water molecule poised at zero degrees - with the capacity to manifest either in a fluid or frozen energy state.

Of course, what we mean by 'conserving energy' is highly context-dependent. For example, even though the individual members of a political uprising are very different actors from individual water molecules in a fluid medium, the dynamics at play are in fact very similar. Up until a certain critical mass is obtained, resisting a government or a policy involves encountering a great deal of resistance. The effort might feel futile - 'a waste of energy'. But when a movement begins to gain momentum, there can be a sense that the force of the movement is stronger than the institutions that it opposes. Being 'carried along' with the movement (joining an uprising), is in fact the course of action that is most in alignment with the forces being unleashed.

Further, once a critical mass is reached, a movement will tend to accelerate its pace due to positive feedback. This can have both positive and negative societal consequences: some mass movement such as lynching mobs or bank-runs show us the downside of tipping points that move beyond a threshold and then spiral out of control.

That said, understanding that critical points may exist in the system (beyond which new kinds of behavior become feasible), can help us move outside of 'ruts' or 'taken for granted' scenarios. In the North American context, smoking was an acceptable social practice in public space. Over time, societal norms pushed public smoking beyond a threshold of acceptability, at which point smoking went from being a normative behavior to something that, while tolerated, is ostracized in the public realm.

What other kinds of activities might we wish to encourage and discourage? If we realize that a behavioral norm is close to a critical point, then perhaps with minimal effort we can provide that additional 'push' that moves it over the edge.

Shifting Environmental Context:

Of course these examples are somewhat metaphoric in nature, but the point being made is that there can be changes in physical dynamics and changes in cultural dynamics that cause different kinds of behaviors to become more (or less) viable within the constraints of the surrounding context.

Returning to physical systems, slime mould is a very unique organism that has the capacity to operate either as a collective unit, or as a collection of individual cells, depending on the inputs provided by the environmental context. As long as food sources are readily available, the mould operates as single cells. That said, when food becomes scarce, a critical point is reached when cells agglomerate to form a collective body with differentiated functions. This new body has capacities for movement and food detection not available at the individual cell level, as well as other kinds of reproductive capacities.

Accordingly, we cannot think about the behavior of a complex system without considering the context within which it is embedded. The system may have the different kinds of capacities depending on how the environment interacts with and 'triggers' the system. It is therefore important to be very aware of the environmental coupling of a system. What might appear to be stable behavior might in fact be behavior that is relying on certain environmental features being present - change these features and entirely new kinds of behaviors might manifest.

This is to say that tipping points might be extended both from intrinsic forces and extrinsic forces (also termed endogenous vs exogenus aspects). This is to say that a shift might be due to dynamics at play within the system, that push it beyond a critical threshold, or they may be due to dynamics external to the system, that alter the system context or inputs in such a way that a system's particular behavior can no longer be maintained and is pushed into a new regime. When the forces are external, we can think of this as shifts to the Fitness Landscape, where a particular mode of operation is no longer viable due to differences in the environmental context.

Back to {{key-concepts}}

Back to {{complexity}}

Strike a match and drop it in the forest. How big will the resulting fire be? The forest is dry but not overly so... vegetation is relatively thick. Will the fire burn a few trees and then flame out, or will it jump from branch to branch, burning thousands of acres to the ground?

Weirdly uncorrelated cause and effect:

We might think that the scale of an event is relative to the scale of a cause, and in some instances this is indeed the case. But in the context of complex systems, we find an interesting phenomena. These systems appear to 'tune' themselves to a point whereby system inputs of identical intensities (two matches lit on two different days, otherwise same conditions), result in outputs that diverge wildly (a small fire; a massive fire event). The frequency distribution of intense system outputs (relative to equivalent system inputs) follows power-law regularities.

According to Per Bak, a variety of systems naturally 'tune' themselves to operate at a threshold where such dynamics occurs. He defined this 'tuning' as Self-Organized Criticality.  A feature of critical states is that, once reached, system components become highly correlated or linked to other system components. That said, the links are exactly balanced: the system elements are linked just tightly enough so that an input at any point can cascade through the entire system, but just loosely enough, so that there are no redundant links needed to make this occur.

Example:

One might think about this like an array of domino-like entities that, instead of being rectangular, are vertical cylinders: able to topple in any direction. The dominos, instead of being arranged in rows, are arranged in a field, with gaps between some cylinders. Accordingly, when a cylinder falls it might strike a gap in the field, with no additional cylinders toppling. Alternately, it might strike an adjacent neighbor, in which case this neighbor will also fall in a particular direction, potentially striking another or potentially dying out. The analogy is made stronger if we imagine that an arrangement whereby, regardless of the direction from which a cylinder is struck, it will wobble and then can fall in any direction.  When a system is self-critical, it has reached a state where we can randomly choose any domino to topple and the impact on the overall field will vary according to a power-law distribution. That is to say, that some disturbance will affect only a small number of surrounding dominos, while others will propogate throughout the entire system, causing all cylinders to fall. The occurrence of these large scale versus small-scale cascades follow Power Laws distributions.

Sand Piles and Avalanches

We can imagine that it would be very difficult to, from the top down, create a specific arrangement where such dynamics occur. What is surprising, and what Bak and his colleagues showed, is that natural systems will independently 'tune' themselves to such arrangements. Bak famously provides us with the 'sand pile' model as an example of self-organized criticality:

Imagine that we begin to drop a steady stream of grains of sand onto a surface. The sand begins to pile up, forming a cone shape. As more sand is added, the height of the sand cone grows, and there begins to be a series of competing forces: the force of gravity that tends to drag grains of sand downwards, the friction between grains of sand that tends to hold them in place, and the input of new sand grains that tends to put pressure on both of these forces.

What Bak demonstrates is that, as grains are added sand will dislodge itself from the pile, cascading downwards. What is amazing is that it is impossible to predict whether dropping an individual sand grain will result in a tiny dislodgment of sand cascades, or a massive sand avalanche. That said, it is possible to predict the ratio of cascade events over time - which follows a power-law distribution.

What this suggests is that the sand grains cease to operate independently to forces, and instead their response to forces is highly correlated with that of the other sand grains. We no longer have a collection of grains, acting independently, but a system of grains whereby system-wide behaviors are displayed. Accordingly, an input that affects one element in the system might die out then and there, or, because of the correlation amongst all elements, create a chain reaction.

Information Transfer

It remains unclear exactly how such system-wide correlations emerge, but we do know something about the nature of these correlations - they are tuned to the point where information is able to propagate through the system with maximum efficiency. In other words, a message or input at one node in the system (a grain of sand, burning tree, or toppling cylinder) has the capacity to reach all other nodes, but this with the least redundancy possible. In other words, there are gaps in the system which means that a majority of inputs ultimately die out, but not so many gaps that it is impossible for an input to reach all elements of the system.

Coming back to our original example, when we strike a match in a forest, if the forest has achieved a 'self-critical' state, then we cannot know whether the resulting fire will spread only to a few trees, a large cluster of trees, or cascade through the entire forest. The only thing that we can know is that the largest scale events will happen with diminishing frequency in comparison to the small scale events.

One possible way of understanding why self-organized criticality occurs is to position it as a process that emerges in systems that are affected both by a pressure to have elements couple with one another (sand-grains becoming interlocked by friction or 'sticky') and some mechanism that acts upon the system to loosen such couplings (the force of gravity pulling grains apart). The feedback between these two pressures 'tunes' the system to a critical state.

Complex systems that exhibit power-laws would seem to exhibit such interactions between two competing and unbalanced forces.

A system is considered to be self-organizing when the behavior of elements in the system can, together, arrive at a globally more optimal functional regimes compared to if each system element behaved independently. This occurs without the benefit of any controller or director of action. Instead, the system contains elements acting in parallel that will gradually manifest organized, correlated behaviors: Emergence.

Emergent  behaviors become organized into a regular form or pattern. Furthermore, this pattern has properties that do not exist at the level of the independent elements - that is, there is a degree of unexpectedness or novelty in what manifests at the group level as opposed to what occurs at the individual level.

An example of an emergent phenomena generated by self-organization is flock behavior, where the flock manifests an overall identity distinct from that of any individual bird.

Characterizing 'the self' in 'Self'-organization

Let us begin by disambiguating self-organizing emergence from other kinds of processes that might also lead to global, collective outcomes.

Example - Back to School:

Imagine you are a school teacher, telling your students to form a line leading to their classroom. After a bit of chaos and jostling you will see a linear pattern form that is composed of individual students. At this point, 'the line' has a collective identity that transcends that of any given individual: it is a collective manifestation with an intrinsic identity (don't cut in the line!).  The line is created by individual components, expresses new global properties, but it's appearance is not the result of self-organization, it is the result of a top-down control mechanism.

Clearly 'selves' organize in this example, but not in ways that are 'self-organizing'.

Now imagine instead that you are a school teacher wanting the same group of students to play a game of tug-a-war in the school gym. Beginning with a blended room of classmates, you ask the students to pick teams. The room quickly partitions into two collectives:  one composed entirely of girls and the other entirely of boys. As a teacher, you might not appreciate this self-organization, and attempt to exert top-down control in an effort to balance team gender. What is interesting about this case is that it does not require any one boy calling out 'all the boys on this side', or any one girl doing the same: the phenomena of 'boys versus girls' self-organizes.

In the example above, we can well imagine the reasons why school teams might tend to partition into 'girls vs boys' even without explicit coordination (of course these dynamics don't always appear, but I am sure the reader can imagine lots of situations where they do).

Here, there are slight preferences (we can think of these as differentials), that generate a tendency for the elements of the system to adjust their behaviors one way vs another. In the case of the school children, the tendencies of girls to cluster with girls manifests due to tacit practices: friends cluster near friends, and as clusters appear students switch sides to be nearer those most 'like' them. Even if an individual child within this group has no strong preference - is equally friends with girls and boys - the pressures of patterns formed by the collective will tend to tip the balance. One girl alone in a team of boys will register that their behavior is non-conforming and feel pressured to switch sides, even if this is not explicitly stated.

Here there are 'selves' with individual preferences, but global behaviors are tipped into uniformity by virtue of slight system differences that tend to coordinate action.

Conscious vs unconscious self-organization:

While the gym example should be pretty intuitive, what is interesting is that there are many physical systems that produce this same kind of pattern formation but that do not require social cues or other forms of intentional volition. Instead, self-organization occurs naturally in a host of processes. Whether we are talking about schools of fish, ripples of wind-blown sand, or water molecules freezing into snowflakes, self-organization leading to emergent global features is a ubiquitous phenomena.

While the features of self-organization manifest differently depending on the nature of the system, there are common dynamics at play regardless of system. Agents in the system participate in a shared context wherein there exists some form of differential. The agents in the system adjust their behaviors in accordance with slight biases in their shared context and these adjustments, though initially minor, are then amplified through reinforcing feedback that cascades through the system. Finally an emergent phenomena can be recognized.

Sync!

Let us consider the sound of cicadas chirping:

cicadas chirping in sync

The cicadas chirp in a regular rhythm. There is no conductor to orchestrate the beat of the rhythm, no head cicada leading the chorus, no one in charge. The process by which the rhythm of sound (an emergent phenomena) manifests is governed purely by the mechanism of self-organization. Let us break down the system:

1. Agents: Chirping Cicadas
2. Shared Context: the acoustic environment shared by all cicadas
3. Differential: the timing of the chirps
4. Agent Bias: adjust chirp to minimize timing differences with nearby chirps
5. Feedback: As more agents begin to chirp in more regular rhythms, this reinforces a rhythmic tendency, further syncing chirping rhythms.
6. Emergent Phenomena: Regular chirping rhythm.

Even if all agents in the system start off with completely different  (random) behaviors, the system dynamics will lead to the coordination of chirping behaviors.

For another example of the power of self-organization, consider this proposition: You are tasked with getting one thousand people to walk across a bridge, with their movements coordinated so that their steps are aligned in perfect rhythm. You must achieve this feat on the first try (with a group of strangers of all ages who have never met one another).

It is difficult to imagine this top-down directive ending in anything other than an uncoordinated mess. But place people on the Millennium bridge in London for its grand opening and this is precisely what we get:

as the video progresses, watch the movement of people fall into sync

There are a variety of mechanisms that permit such self-organization to occur. In the millennium bridge video, the bridge provides the shared context or environment for  the walkers (who are the agents in the system). As this shared context sways slightly (differential) it throws each agent just a little bit off balance (feedback).  Each individual then slightly adjusts their stance and weight to counteract this sway (agent bias), which serves only to reinforces the collective sway direction. Over time, as the bridge sways ever more violently, people are forced to move in a coordinated collective motion (emergence) in order to traverse the bridge.

What is important to note in this example is that we do not require the agents to agree with one another in order for self-organization to occur. In our earlier example - that of school children forming teams - we can imagine that a variety of factors are at work that have to do with active volition on the part of the children. But in the example above, movement preferences have nothing to do with observed walking behavior or individual preferences. Instead, the agents have become entangled with their context (which is partially formed of other agents), in ways that constrain their movement options.

Enslaved Behavior

Accordingly, in self-organizing systems agents that might initially possess a high number of possible states that are able to enact (see also Degrees of Freedom) the possible range of freedom becoming increasingly limited, until such time as only a narrow band of behavior is possible.

Further, while the shared context of the agents might initially be the source of difference in the system (with difference gradually being amplified over time), in reality the context for each agent is a combination of two aspects: both the broader shared context (the bridge) and the emerging behaviors of all the other agents within that context. This is to say that once a global behavior emerges, subsequent self-organization of the agents is constrained by the emerged context agents are also a part of.

Back to {{key-concepts}}

Back to {{complexity}}

Complex systems tend towards scale-free, nested hierarchies. By 'Scale-free', we mean to say that we can zoom in on the system at any level of magnification, and observe the same kind of structural relations.

A good example here is the fractal features of a leaf:

We can think of the capillary network as the minimum structure required to reach the maximum surface area.

Nature's Optimizing Algorithm

Here, the scale-free structure of the capillary network allows the most efficient transport of nutrients to all parts of the leaf surface within the overall shortest capillary path length. This 'shortest overall path length'  is one of the reasons that we might often see scale-free features in nature: this may well be the natural outcome of nature 'solving' the problem of how to best economize flow networks.

minimum global path length to reach all nodes

The two images serve to illustrate the idea of shortest overall path length. If we wish to get resources from a central node to 16 nodes distributed along a surrounding boundary, we can either trace a direct path to each point from the center, or we can partition the path into splitting segments that gradually work their way towards the boundary. While each individual pathway from the center to an individual node is longer in the right hand image, the total aggregate of all pathways to reach all nodes from the center is shorter. Thus the image on the right (which shows scale-free characteristics), is the more efficient delivery network.

Example - Street Networks:

We should therefore expect to see such forms of scale-free dynamics in other non-natural systems that carry and distribute flows: thus, if we think of size distribution of road networks in a city, we would expect a small number of key expressways carrying large traffic flows, followed by a moderate number of mid-scaled arteries carrying mid-scale flows, then a large number of neighborhood streets carrying moderate flows, and finally a very high number of extremely small alleys and roads that each carry very small flows to their respective destinations.

mud fractals and street networks

Fractals, scale-free networks, self-similar entities and power-law distributions are concepts that can be difficult to disambiguate. Not all scale-free networks look like fractals, but all fractals and scale-free networks follow power-laws. Finally, there are many power-law distributions that neither 'look' like fractals, nor follow scale-free network characteristics: if we take a frozen potato and smash it on the ground, then classify the size of each piece, we would find that the distribution of smashed potato pieces follows a power law (but is not nearly as pretty as a fractal!). Finally, self-similar entities (like the romanesco broccoli shown below) are fractal-like (you can zoom in and see similar structure at different scales), but are not as mathematically precise as a fractal.

credit: Wikimedia commons  (Jon Sullivan)

Back to {{key-concepts}}

Back to {{complexity}}

One of the intriguing characteristics of complex systems is that highly sophisticated emergent phenomena can be generated by seemingly simple agents. These agents follow very simple rules - with dramatic results.

Simple Rules - Complex Outcomes

Slime mold forming the Tokyo subway map

Take it in Context

We can conceptualize  bottom-up agents as simple entities with limited action possibilities. The decision of which action possibility to deploy is regulated by basic rules that pertain to the context in which the agents find themselves. Another way to think of 'rules' is therefore to relate them to the idea of a simple set of input/output criteria.

An agent exists within a particular context that contains a series of factors considered as relevant inputs: one input might pertain to the agent's previous state (moving left or right); one might pertain to some differential in the agent's context (more or less light; and one might relate to the state of surrounding agents (greater or fewer). An agent processes these inputs and, according to a particular rule set, generates an output: 'stay the course', 'shift left', 'back up'.

input/output rule factoring three variables

In complex adaptive systems, an aspect of this 'context' must include the output behaviors generated by surrounding agents. Further, while for natural systems the agent's context might include all kinds of factors that serve as relevant inputs, in artificial complex systems novel emergent behavior can manifest even if the only thing informing the context is surrounding agent behaviors.

Example:

Early complexity models focused  precisely on the generative capacity of simple rules within a context composed purely of other agents. For example, John Conway's 'Game of Life' is a prime example of how a very basic rule set can generate a host of complex phenomena. Starting from agents arranged on a cellular grid, with fixed rules of being either 'on' or 'off' depending on the status of the agents in neighboring cells, we see the generation of a host of rich forms. The game unfolds using only four rules, that govern whether an agent is 'on' (alive) or 'off' (dead).  For every iteration:

1. 'Off' cells turn 'On' IF they have three 'alive' neighbors;
2. 'On' cells stay 'On' IF they have two or three 'alive' neighbors;
3. 'On' cells turn 'Off' IF they have one or fewer 'alive' neighbors;
4. 'On' cells turn 'Off' IF they have four or more 'alive' neighbors.

The resulting behavior has an 'alive' quality: agents flash on and off over multiple iterations, seem to converge, move along the grid, swallow other forms, duplicate, and reproduce.

Conway's Game of Life

Principle: One agent's output is another agent's input!

As we can see from the Game of Life, starting with very basic agents, who rely only on other agents outputs as their input, a basic rule set can nonetheless generate extremely rich outputs.

While the Game of Life is an artificial complex system (modeled in a computer), we can, in all real-world examples of complexity, observe that the agents of the system are both responders to inputs from their environmental context, as well as shapers of that same environmental context. This means that the behaviors of all agents necessarily become entangled - entering into feedback loops with one another.

Adjusting rules to targets

It is intriguing to observe that, simply by virtue of simple rule protocols that are pre-set by the programmer and played out over multiple iterations, complex emergent behavior can be produced. Here we observe the 'fact' of emergence from simple rules. But we can also imagine natural complex systems where agent rules also shift over time. While this could happen arbitrarily, it makes sense from an evolutionary perspective when some agent rules are more 'fit' then others. This results in a kind of selection pressure, determining which rule protocols are preserved and maintained. Here, the discovery of simple rule sets that yield better enacted results exemplifies the 'function' of emergence.

When we couple the notions of 'rules' with context, we are therefore stating that we are not interested in just any rule set that can generate emergent outcomes, but with specific rule sets that generate emergent outcomes that are in some way  'better' with respect to a given context. Successful rule systems imply a fit between the rules the agents are employing, and how well these rules assists agents (as a collective) in achieving a particular goal within a given setting.

As a general principle we can think of successful rules as ones that minimize agent effort (energy output) to resolve a given task. That said, in complex systems we need to go a step further to analyze the collective energy output. Thus in a system the best rules will be the ones that result in minimal energy output for the system as a whole to resolve a given task. This may require 'sacrifice' on the part of an individual agent, but this sacrifice (from a game theory perspective), is still worth it from the overall system level.

As agents in a complex system enact particular rule sets, rules might be revised based on how quickly or effectively they succeed at reaching a particular target.

When targets are achieved - 'food found!' - this information becomes a relevant system input.  Agents that receive this input may have a rule that advises them to persist in the behavior that led to the input, whereas agents that fail to achieve this input may have a rule that demands they revise their rule set!

Agents are therefore not only be conditioned by a set of pre-established inputs and outputs but also be able to revise their rules. This requires them to gain feedback about the success of their rules and test modications. A way of thinking about this is captured in the idea of an agent held {{schemata}} about their behavior relative to their context that can be updated over time so as to better align. Further, if multiple agents test different rule regimes simultaneously, then there may be other 'rules' that help agents learn from one another. If a particular rule leads agents to food, on average, in ten steps, and another rule, on average, leads agents to food in 6 steps, then agents adopting the second rule should have the capacity to disseminate their rule set to other agents, eventually suppressing the first, weaker rule. This processor dissemination requires some form of communication or steering, which is often done via the use of Stigmergy.

Enacted 'rules' are therefore provisional tests of how well an output protocol achieves  a given goal. The test results then become another form of input:

bad test results also become an agent input,  telling the agent to: "generate a rule mutation as part of your next enacted output".

Novel Rule formation:

Rules might be altered in different ways. At the level of the individual -

• an agent might choose to revise how it values or factors inputs in a new way;
• an agent might choose to revise the nature of its outputs in a new way.

In the first instance, the impact or value assigned to particular inputs (needed to trigger an output) might change based on how successfully previous input weighting strategies were in relationship to reaching a target goal.  In order for this to occur, the agent must have the capacity to assign new 'weights' (the value or significance) for an input, in different ways.

In the second instance, the agent requires enough inherent flexibility or 'Degrees of Freedom' to generate more than one kind of output. For example, if an agent can only be in one of two states, it has very little ability to realign outputs. But if an agent has the capacity to deploy itself in multiple ways, then there is more flexibility in the character rules it can formulate. This ties back to the idea of {{adaptive-capacity}}.

Rules might also be revised through processes occurring at the group level. Here, even if agents are unable to alter their performance at the individual level, there may still be mechanisms operating at the level of the group which result in better rules propagating. In this case, we would have a population of agents, each with specific rule sets that vary amongst them. Even if each individual agent has no ability to revise their particular rules, at the level of the population -

• poor rules result in agent death - there is no internal recalibration - but agents with bad rules simply cease to exist;
• 'good' rules can be reproduced - there is no internal recalibration - but agents with good rules persist and reproduce.

We can imagine that the two means of rule revision - those working at the individual level and those at the population level - might work in tandem. While all of this should not seem new (it is analogous to General Darwinism), since complex systems are not always biological ones, it can be helpful to consider how the processes of system adaptation (evolution) can instead be thought of as a process of rule revision.

Through agent to agent interaction, over multiple iterations, weaker protocols are filtered out, and stronger protocols are maintained and grow. That said, the ways in which rules are revised is not entirely predictable - there are many ways in which rules might be revised, and more than one kind of revision may prove successful (as the saying goes - there is more than one way to skin a cat). Accordingly, the trajectory of these systems is always contingent and  subject to historical conditions.

Fixed Rules with thresholds of enactment

Not all complex adaptive behaviors require that rules be revised. We began with artificial systems - cellular automata - where the agent rules are fixed but we still see complex behaviors. There are also example of natural complex systems where rules are fixed, but still drive complex behaviors. These rules, rather than being the result of a computer programmer arbitrarily determining an input/output protocol, are the result of fundamental laws  (or rules) of physics or chemistry.

One particularly beautiful example of non-programmed natural rules resulting in complex behaviors is the Belousov-Zhabotinsky (BZ) chemical oscillator.  Here, fixed chemical interaction rules lead to complex form generation:

BZ chemical oscillator

In this particular reaction, as in other chemical oscillators, there are two interacting chemicals, or two 'agent populations' which react in ways that are auto-catalytic. The output generated by the production of one of the chemicals, becomes the input needed for the generation of the other chemical. Each chemical is associated with a particular color, which appears only when that chemical present in sufficient concentrations. The concentrations of these chemicals augments and diminishes at different reaction speeds, leading to shifting concentrations of the coupled pair.  As concentrations rise and fall, we see emergent and oscillating color arrays.

Back to {{key-concepts}}

Back to {{complexity}}

Power laws are particular mathematical distributions that appear in contexts where a very small number of system events or entities exist that, while rare, are highly impactful, alongside of a very large number of system events or entities exist that, while plentiful, have very little impact.

Power laws arise in both natural and social system, in contexts as diverse as earthquake intensities, city population sizes, and word frequency use.

'Normal' vs 'Power Law' Distributions

Complex systems are often characterized by power law distributions. A power law is a kind of mathematical distribution that we see in many different kinds of systems. It has different properties from the well known 'bell curve' 'normal' or 'Gaussian' distribution.

 Let's look at the two here:

Power-law (left) vs Bell-curve (right)

Most people likely remember the bell curve from high school. The fat middle (highlighted) is the 'norm' and the two sides or edges represent the extremes. Accordingly, a bell curve can illustrate things like people's heights - with 'typical' heights being distributed around a large cluster at the middle, and extreme heights (both very tall and very short people), being represented by much smaller numbers at the extremes. There are many, many, phenomena that can be graphed using a bell curve. It is suitable for depicting systems that hover around a normative 'middle' and for systems where there are no driving correlations amongst members of the set. That is to say: the height of one person in classroom is not constrained or affected by heights of other people.

Power-law distributions are likely as common as bell-curve distributions, but for some reason people are not as familiar with them. They occur in systems where there is no normative middle where most phenomena occur. Furthermore, entities within a power-law set enjoy some kind of calibration feedback relation amongst them - meaning that the size of one entity in the system is in someway correlated with, (or has an impact) on the size and frequency of other entities. These  systems are characterized by a small percent of phenomena or entities in the system, accounting for a great deal of influence or system impact.

This small percent is shown on the far left hand side of the diagram (highlighted), where the 'y' axis (vertical) indicates intensity or impact (of some phenomena), and the 'x' axis indicates the frequency of events, actors, or components associated with the impact. The left hand side of the diagram is sometimes called the 'fat head', and as we move along to the right hand side of the diagram, we see what is called 'the long tail'. Like the bell curve, which we can use to chart phenomena such as housing prices, heights, test scores, or household water consumption, the power law distribution can illustrate many different kinds of things. 

Occasionally, we can illustrate the same phenomena using bell curves and power law distributions, while simultaneously highlighting different aspects of the same phenomena.

Example:

Let's say we chart income levels on a bell curve. The majority of people earn a moderate income, and a smaller number of people earn both very high and the very low incomes at the extreme sides. Showing this data, we get a chart that looks like the one below:

Wealth in the USA plotted as a bell curve (source: pseudoerasmus)

But, we can think of income distribution another way - the impact or intensity of incomes. Consider this fact of wealth distribution: in the US, if we look at the right side of the bell curve above (the wealthiest people who make up a small fraction or 1% of the population) these few people control around 45% of entire US wealth. Clearly, the bell curve does not capture the importance of this small fraction of extreme wealth holders.

Imagine that instead of plotting the number of people in different income brackets we were to instead plot the intensities of incomes themselves. In this case we would generate a plot showing:

• 1% (a few people)  controlling  45% (a large chunk) of total wealth;
• 19% (a moderate number of people) controlling  35% ( a moderate chunk) of total wealth;
• 80% (the bulk of the population) controlling 20% (a small fraction) to total wealth.

These ratios plot as a power law, with approximately 20% of the people controlling 80% of the wealth resource.

80/20 Rule

These numbers, while not precisely aligning with US statistics, are not that far off, and they align with what is referred to as the '80/20' rule: where 20 percent of a system's components are responsible for 80 percent of the system's key functions or impacts. This phenomena was first noted by {{Pareto}}, and is also referred to as a a Pareto Distribution. We can find Pareto distributions in many different kinds of phenomena where the distributions might be applied to aspects such as - quantities, frequencies, or intensities. Thus:

• 20% of our wardrobe is worn 80% of the time;
• 20% of all English words are used 80% of the time;
• 20% of all roads attract 80% of all traffic;
• 20% of all grocery items account for 80% of all grocery sales;

Finally, if we smash a frozen potato against a wall and sort out the resulting broken chunks:

• 20% of the potato chunks will account for 80% of the total smashed potato.

Such ratios are so common that if you are unsure of a statistic then - provided it follows the 80/20 rule - you are likely safe to make it up! (the frozen potato being a case in point :))

Source: themediaconsortium.org

Rank Order

Another way to help understand how power law distributions work is to consider systems in terms of what is called their 'rank order'.  We can illustrate this with language. Consider a few words from English:

• 'The' is the most commonly used word in the English language -
  • We rank it 'first' and it accounts for 7% of all word use (rank 1) .

• "Of" is the second most commonly used word -
  • We rank it 'second' and it accounts for 3.5% of all word use (1/2 of the rank 1 word)

If we were to continues, say looking at the 7th most frequently used word, we would expect to see it use 1/7th as frequently as the most commonly used word. And in fact -

• 'For' is the seventh most commonly word,
  • We rank it seventh  and it accounts for 1% of all word use  (1/7 of the rank 1).

This power-law phenomena is known as 'Zipf's Law' for George Kingsley Zipf, the man who first identified it. Zipf's law indicates that if, for example,  you have 100 items in a group, the 99th item will occur 1/99th as frequently as the first item.  For any element in the group, you simply need to know its rank in the order - 1st, 3rd, 25th - to understand its frequency (relative to the top ranked item in the group).

The constant in Zipf's law is '1/n' , where the 'nth' ranked word in a list is used 1/nth as often as the most popular word.

Were all power-laws to follow a Zipf's law then:

• the 20th largest city would be 1/20th the size of the largest;
• the 10th most popular child's name would be used 1/10 of the time compared to the most popular;
• the 3rd largest earthquake in California in 100 years would be 1/3 of the size of the largest;
• the 50th most popular product would sell 1/50th as often as the most popular .

This is a very easy and neat set, and it is represents perhaps the most straightforward power law.  That said, there can be other power law ratios between elements which, - while remaining constant, are not always such a 'clean' constant.  These follow the same principle but are just more difficult to express (and calculate). For example"

'1.07/n'  would be a power-law where the 'nth' ranked word in a list is used 1/1.07 times as often as the most popular word.

Pretty in Pink

Clearly '1.07/f' is a less satisfactory ratio then 1/n. In fact, the 1/n ratio is so pleasing that it has a few different names. 1/n is mathematically equivalent to 1/f ratio where, but instead of highlighting the rank in the list, 1/f highlights the frequency of an element in a list (the format is different but the meaning is the same).

'1/f' is also described as 'pink noise' - which is a statistical pattern distinct from 'brown' or 'white' noise. Each class of 'noise' pertains to different kinds of randomness in a system. In other words, while many systems exhibit random behaviors, some random behaviors differ from others. We can think of 'pink', 'white', and 'brownian', noise as being different 'flavors' of randomness. Without getting into too much detail here, 1/f noise seems to occur frequently in natural systems, and can be associated with beauty. In non-mathematical terms, pink noise involves a frequency ratio of component distributions such that there is just enough correlation between elements to provide a sense of unity, and just enough unexpectedness to provide variety. The human mind seems to enjoy this balance between the two, which is why pink noise can be found in music or artworks that we find beautiful. White noise is too random (no correlation) and brownian noise is too correlated (no unexpected interested).

Dynamics generating Power-laws

Power laws distributions have been identified in many complex system behaviors, such as:

• earthquake size and frequency
• neuron activity
• stock prices
• web site popularity
• academic citation network structure
• city sizes
• word use frequency
• ....and much more!

Much time and energy has gone into identifying where these distributions occur and also trying to understand why they occur.

Growing Riches

A strong contender for the presence of power-law dynamics is that they may be present in  systems that involve both growth and Preferential Attachment. Understood colloquially as 'the rich get richer', preferential attachment is the idea is that popular things tend to attract more attention, thereby becoming more popular. Similarly, wealth begets wealth.  The idea of growth and preferential attachment is therefore associated with positive feedback. It can be used to explain the presence of power-law distributions in the size and number of cities (bigger cities attract more industry thereby attracting more people...) the distributions of citations in academic publishing (highly cited authors are read more thereby attracting more citations), and the accumulation of wealth (rich people can make more investments, thereby attracting more wealth).

Push forward and Push back

Further, power-laws might be understood as a phenomena that occur in systems that involve both positive and negative feedback interactions as co-evolving drivers of the entities within the system. Such systems would involve feedback dynamics that are out of balance: some feedback dynamics ({{positive-feedback}}) are amplifying certain system features, while others system dynamics ({{negative-feedback}}) are simultaneously  'dampening' or constraining these same system features. Simultaneously there is a correlation between these push and pull dynamics - so the greater the push forward the more it generates a pull back, and vice versa. The imbalance between this push and pull interplay between interacting forces creates feedback loops that lead to power-law features.

An example of this would be that of reproducing species in an eco-system with limited carrying capacity. Plentiful food would tend to amplify reproduction and survival rates (positive feedback), but as population expands this begins to put pressure on the food resources, leading to a push back (lower survival rates), and consequently a drop in population levels. The two driving factors in the system -  growing population and dwindling food - are causally intertwined with one another and are not necessarily in balance. If the system achieves a perfect balance then the system will find an equilibrium - the reproduction rate will settle to a point where it matches the carrying capacity. But if there are forces that drive the system out of balance, or if there is a lag time between how the two 'push' and 'pull' (amplifying and constraining) dynamics interact, then the system cannot reach equilibrium and instead keeps oscillating between states (see {{Bifurcations}} ). 

Example: What's in a Name?

It has been shown that the frequency of baby name occurrences follows a power-law distribution. In this example, what is the push/pull interplay that might lead to the emergence of this regularity?

While each set of parents chooses their child's name independently, they do so within a system where their choices are somewhat driven or constrained by the choices being made by parents around them. Suppose there is a name that, for some reason, has become prevalent in popular consciousness - perhaps a character name in a popular book or tv series.  It is not necessary to know the precise reasons why this particular name becomes popular, but we can imagine that certain names seem to resonate in popular consciousness or 'the zeitgeist'. Let us take the name 'Jennifer'. An obscure name in the 1930s,  it became the most popular girl's name in the 1970s. During that time, if you were one of the approximately 16 million girls born in the US, there was  a 2.8% chance you would be named Jennifer!  And yet,  the name had plummeted back to 1940s levels by the time we get to 2012.

the rise and fall of Jennifer

But how can the rise and fall of 'Jennifer' be described using push and pull forces? We can imagine a popular name being like a contagion, where a given name catches on in popular consciousness. During its initial spread, the name is highlighted even further in popular consciousness, potentially expanding its appeal.  At the same time, the very fact that the name is popular causes a tendency for resistance - if Jennifer is on a short list of possible baby names, but a sibling or close friend names their child 'Jennifer', this has an impact on your naming choice. In fact, the more popular the name becomes, the more pullback we can expect. As more and more people tap into the popularity of a name, it becomes more and more commonplace, leading to a sense of overuse, leading to a search for new novelty. The interactions of push and pull cause the name to both rise and fall. In a system of names, Jennifer is a name that had an expansion rate caused by rising popularity feedback, but then a decay rate caused by overuse and loss of freshness.

The Long Tail

An additional feature of power law distribution that should not be overlooked is what is sometimes called the "power of the long tail". While power law systems have a few strongly performing elements in the upper 20%, there are still many important actors in the remaining 80% of the distribution. One recent feature of information technologies is that it is easier to "find" the specificity of this 80%. If we think about bookstores from only a decade ago, they needed to carry only the "best-sellers": if your reading interests fell outside of the norm than it would be difficult to find books that would serve as the right "fit" or "niche" for your reading interests. Today, with information flows having become so inexpensive, online bookstores are not limited by the number of titles they can carry, so people can find the niche books they actually want to read rather than having to compromise around the average. In some ways this echoes eco-systems, where there can be a few top players, but where there also exist many viable micro-niches that can be populated. There are many domains where accessing this "long tail" will lead to more choice and precision in complex systems. 

Proviso

While power-laws are often pointed to as 'the fingerprint of complexity', It should. be noted that their recent ubiquity is not without controversy.  While many studies highlight the presence of these mathematical regularities in a host of diverse systems, other argue that the statistics upon which these findings are based are often skewed, and that power-laws may not be as common in systems as is frequently stated. It is a problem of researchers looking to affirm the existence of these patterns that may cause them to ignore results where they do not occur, and attribute their presence in systems that may or may not actually hold these properties. 

Back to {{key-concepts}}

Back to {{complexity}}

Complex systems can follow many potential trajectories: the actualization of any given trajectory can be dependent on small variables, or "changes to initial conditions" that are actually pretty trivial. Accordingly, if we truly wish to understand system dynamics, we need to pay attention to all system pathways (or the system's phase space) rather than the pathway that happened to unfold.

Inherent vs Contingent causality

Why is one academic cited more than another, one song more popular than another, or one city more populated than another? We tend to imagine that the reason must have to do with inherent differences between academics, songs or cities. While this may be the case, the dynamics of complex systems may lead one to doubt such seemingly common-sense assumptions.

We describe complex systems as being non-linear - this means that small changes in the system can have cascading large effects - think the butterfly effect - but what it also implies is that history, in a very real way matters. If we were to play out the identical system with very slight changes, the specific history of each system would play a tangible role in what we perceive to be significant or insignificant.

Think about a cat video going viral. Why this video? Why this particular cat? If on a given day 100 new cat videos are uploaded, what is to say that the one going viral is inherently cuter than the other 99 out there? Perhaps this particular cat video really is more special. But a complexity perspective might counter with the idea of path-dependency: that amongst many potentially viral cat videos, a particular one played  this potentiality out - but this is an accident of a specific historical trajectory, rather than a statement about the cuteness of this particular cat.

Butterfly Effects:

The reason for this returns to the idea of the Path Dependency nature of the system, the fact that it is Sensitive to Initial Conditions. Suppose we have six cat videos that are of inherently equal entertainment value. All are posted at the same time. We now roll a six sided die to determine which of these gets an initial 'like'.  This initial roll of the die now causes subsequent rolls to be slighted weighted - whatever received an initial 'like' has a fractionally larger chance of being highlighted in subsequent video feeds. Let us assume that subsequent rolls reinforce, in a non-linear manner, the first 'like'. Over time, like begets like, the rich get richer, and we see one video going viral.

If we were to play out the identical scenario in a parallel universe, with the first random toss of the dice falling differently, then an entirely different trajectory would unfold. Such is the notion of 'path-dependency'. Of course, it is normal to assume that given the choice of two pathways into an unknown future,  the path we take matters, and will change outcomes. But in complex systems this constitutes an inherent part of the dynamics, and a 'choice' is not something that one actively elects to make,  as much as something that arises due to random system fluctuations.

Another way to think about this is with regards to the concept of Phase Space. Any complex system has a broad state of potential trajectories (its phase space), and the actualization of any given trajectory is subject to historical conditions. Thus, if we want to understand the dynamics of the system, we should not only attune to the path that actually unfolded - rather we should consider the trajectories of all possible pathways. This is because the actual unfolding of any given pathway within a system is not inherently more important then all of the other pathways that could equally have unfolded.

One of the reasons that computer modeling is popular in understanding complex systems has to do with this notion of phase space and path dependency. A computer model allows us to 'explore the phase space' of a complex system: seeing if system trajectories are inherently stable and repeat themselves consistently, or if they are inherently unstable and might manifest in quite different ways.

Sometimes we can imagine that a system does unfold differently in phase space, but that this unfolding tends towards particular behaviors. We call these system tendencies Attractor States. One of the features of complex systems is that they often have multiple attractors, and it is only by allowing the system to unfold that we are able to determine which attractor the system ultimately converges towards. It would be a mistake, however, to grant a particular attractor as being more important than another based only upon one given instance of a system unfolding.

Another feature of path dependency is that, once a particular path is enacted, it can be very difficult to move the system away from that pathway, even if better alternatives exist.

A great example of path dependency is the battle between VHS and BETA as competing video formats. According to most analysts, BETA was the superior format, but due to factors involving path dependency, VHS was able to take over the market and squeeze out its superior competitor.

Another example is that of the QWERTY key board. While initially a solution to the problem of keys jamming when pressed too quickly on a manual keyboard, the solution actually slows down the process of typing. However, even though we have long since moved to electronic and digital keyboards where jamming is not a factor, we are 'stuck' in the attractor space that is the QWERTY system. This is partially due to the historical trajectory of the system, but also all of the reinforcing feedback that works to maintain QWERTY: once people have learnt to type on one system, it is difficult to instigate change. One way of saying this is to refer to the system being 'locked-in' or referring to "Enslaved States".

An Urban example may also be instructive: In Holland people bike as a normal mode of transport, in North America they drive. We can make arguments that there are inherent differences in North American and Dutch cultures that create these differences, but a complexity argument might propose, instead, that such differences are due to path-dependency. Perhaps any preferences that the Dutch have for biking are only random. That being said, over time, infrastructure has been created in the Netherlands that incentivizes biking (routes everywhere), and disincentives driving (many streets closed to traffic, lack of parking, inconvenient, slow commutes). In North America, we have created infrastructure that incentivizes driving: big streets, huge parking areas close to where we work, and lack of other transport alternatives. We then arrive at a situation where the Dutch bike and the North-American drives. But place a North American in Holland and they will soon find themselves happily biking, and place a Dutchman in the USA and they will soon find themselves purchasing a vehicle to drive along with everyone else. Neither driving nor biking is inherently 'better' in so far as the commuter is concerned (although there may be more environmental and health benefits associated with one versus the other), but the pathways each country have taken wind up mattering, and reinforcing behaviors through feedback systems.

If we are able to better understand how to break out of ill-suited path-dependency, we may be able to solve a variety of problems that seem to be 'inherent' or 'natural' choices or preferences.

Back to {{key-concepts}}

Back to {{complexity}}

A system is considered to be open and dissipative when energy or inputs can be absorbed into the system, and 'waste' discharged. Here, system inputs like heat, energy, food, etc., can traverse the open boundaries of the system and ‘drive’ it towards order: seemingly in violation of the second law of thermodynamics.

Complex OutLaws!

For those who are haven't dusted off their high school science textbooks recently, it is worth a quick refresher on the 2nd law of thermo-dynamics. Initially formulated by Sadi Carnot in 1824 (he was looking at the flow of heat in steam engines), the law has been expressed in various technically precise ways. For our purposes, the importance characteristic of these definitions is the idea of loss of order. Any ordered system will eventually move towards disorder. There is no way of getting around it. Things get messy over time - that's the Second Law. Everything ultimately decays. You, me, the world, the universe.

We can contemplate the metaphysical implications of this (the 2nd law is a bit of a downer) over a cup of coffee, while watching this video. We see illustrated the sad,  inevitable decrease in the cream's order as it meets with coffee (it's pretty relaxing actually):

Cream dis-ordering as it enters coffee

What the 2nd Law states is that something is ultimately lost in every interaction, and because of that, more and more disorder is ultimately created. We can ask heat to do work in driving a steam engine, but some of the heat will always be lost in translation, so that even if we are able to produce localized work or order, more disorder has ultimately been created in the universe as a whole. We call this inevitable increase in disorder 'entropy'.

But wait - you say - there is order all around us! While this may appear true, it is because what appear to be violations of the 2nd Law are achieved within the boundaries of a particular system. While a particular system can gain order, it is only because its disorder is simultaneously being dissipated into the surrounding context. Local order (within the system) is thus maintained at the expense of global disorder (within the environment). Were the system to be fully closed from its context, it would be unable to maintain this local order.

Thus, the ability to increase order in violation of the 2nd Law is called Negentropy - and one of ways in which Negentropy can be generated is by creating a system that is 'open and dissipative': meaning that an energy source can flow in to drive order, and waste can flow out to dissipate disorder.

Example:

A famous example of this dynamic is in  Benard/Rayleigh convection Rolls (a phenomena studied by {{ilya-prigogine-isabelle-stengers}} as an example of self-organizing behavior). In this example, we have fluid in a small Petri dish, heated by a source placed under the dish. The behavior of the fluid is the system that we wish to observe, but this system is not closed: it is open to the input of heat that traverses the boundary of the Petri dish. Further, while heat can 'get into' the system, it can also be lost to the air above as the fluid cools. Note that the overall system clearly has a defined 'inside'  (the fluid in the Petri dish), and a defined 'outside' (the surrounding environment and the the heat acting upon the Petri dish), but there is not full closure between the inside and outside. This is what is meant when we say that complex systems are Open / Dissipative . We understand them as bounded, (with relations primarily internal to that boundary), but nonetheless interacting in some way with their surroundings.  Were the boundary fully closed no increase in order could not occur.

Let us turn now to the flows driving the system. As heat is increased, the energy of this heat is transferred to the fluid, and the temperature differential between the top and the bottom of the liquid causes heated molecules to be driven upwards. At the same time, the force of gravity causes the cooler, molecules in the fluid to be driven downwards. Finally, the drag forces acting between rising and falling molecules cause their behaviors to become coordinated, resulting in 'roll' patterns associated with Benard convection.

Rayleigh/Benard Convection (fluid of oil/ silver paint)

The roll patterns that we observe are a pattern: a global structure that emerges from the interactions of many agitated molecules without being 'coordinated' by them. What helps drive this coordination is the dynamics of the interacting forces that the molecules are subjected to (driving heat flows and counteracting gravity pressures), as well as how the independent molecular responses to these pressures feedback to reinforce one another (through the drag forces exerted between molecules).  That said, the fluid molecules do nothing on their own absent the input of heat. Instead, heat is the flow that drives the system behavior. Further, as the intensity of this flow is amplified (more heat added), the behavior of the fluid shifts from that of regular roll patterns to more turbulent patterns.

Setting boundaries

{{ilya-prigogine-isabelle-stengers}} were the first to highlight the importance of open dissipative structures in generating complexity dynamics. Earlier works in General Systems Theory ({{ludwig-v-bertalanffy}}) attuned to the complex dynamics at work within an internal structure, but did not make a distinction between open and closed structures. Closed structures in contrast to open structures do not process new inputs, and therefore are unable to generate novelty.

At the same time, systems need some sort of boundary or structure so as to hold together components of enough collective identity that they can work in tandem to process flows. It is therefore important to determine what is the appropriate boundary of any complex system under study, and what kinds of flows are relevant in terms of crossing those boundaries. 

Often, complexity involves multiple overlapping systems, each with their own internal dynamics and external flows, but systems can become entangled as one systems exports become another's inputs. In order to simplify these dynamics, it is perhaps helpful to try to identify which groups of agents in a system belong to a particular class of that shares a common driving flow and then examine the dynamics with respect to only those flows and behaviors. Systems can then be layered onto systems to build a more complete understanding of the dynamics at play. 

Back to {{key-concepts}}

Back to {{complexity}}

Network theory is a huge topic in and of itself, and can be looked at on its own, or in relation to complex systems. There are various formal, mathematical ways of studying networks, as well as looser, more fluid ways of understanding how networks can serve as a structuring mechanism.

Why Networks?

We can think of networks in fairly simple terms: imagine, for example, a network of aircraft traveling between hubs and terminals, or a network of people working together in an office. Network analysis operates under the premise that,  by looking at the structure of the network alone, we can deduce something about how the network will function, and potential information about potential nodes within the network. For example, the image below could illustrate many different kinds of networks: perhaps it is an amazon delivery network, or a social network, or an academic citation network. What is interesting is that, even without knowing anything about the kind of network it is, we can still say some things about how it is structured. The network below has some pretty big hubs - around 6 of them that are well connected to other nodes, but not strongly connected to one another. What would be the dynamics of this network if it were a social network, or a network of a company?

What might we learn from the network?

By looking at the diagram we might learn about how information or control is exerted, about which entities are isolated, and about how protracted communication channels might be. A work network in which I need to talk to my superior, who in turn talks to his boss, who in turn is one of three bosses who only talk to each other, creates very different dynamics than a network where I have connections to everyone, or where there is only one chain of command rather than three.

Network theory attempts to understand how different network structures might lead to different kinds of system performances. The field uses domain specific language - speaking of nodes, edges, degree centrality, etc. - with much of this detail falling outside of the scope of this website.

What is important is that complex systems are made up of individual entities and, accordingly, the ways in which these entities relate to one another matter in terms of how the whole is structured. Networks in complex adaptive systems are composed of individual agents, and the relationships between these agents tend to evolve in ways that lead to power law distributions between highly and weakly connected agents. This is due to the dynamics of Preferential Attachment whereby 'the rich get richer'.

At its most extreme, network theory advances the idea that relationships between objects can have primacy over the objects themselves. Here, the causal chain is flipped from considering objects or entities as being the primary causal figure that structures relationships, to instead exploring how relationships might in fact be the primary driver that act to structure objects or entities.

Generalizing Network Knowledge

In the social sciences, Systems Theory (developed by Ludwig V. Bertalanffy), was the first to endeavor to examine how networks could play a key structuring role in how a range of entities function. Systems theory positioned itself as a meta-framework that could be applied in disparate domains - including  physics, biology, and the social sciences - and it attracted a wide following. Rather than focusing upon the atomistic properties of the things that make up the system, systems theory instead attuned to the relationships that joined entities, and how these relationships were structured.

Gregory Bateson, illustrates this point nicely when he considers the notion of a  hand: he asks, what kind of entity are we looking at when considering a hand? The answer depends on one's perspective: We can say we are looking at five digits, and this is perhaps the most common answer (or four fingers and a thumb). If we look at the components of the hand in this manner, we remain focused on the nature of the parts - we might look at the properties of each finger and how these are structured. However, we can answer the question another way: instead of seeing five digits we can say that we see four relationships. Bateson's point was that the way in which the genome of an organism understands or structures the entity 'hand' is more closely aligned with the notion of relationships rather than that of digits or objects. Accordingly, if we are to better understand natural entities we should begin to examine these from the perspective of relations rather than objects.

“You have probably been taught that you have five fingers. That is, on the whole, incorrect. It is the way language subdivides things into things. Probably the biological truth is that in the growth of this thing – in your embryology, which you scarcely remember – what was important was not five, but four relations between pairs of fingers.” - Gregory Bateson

In a similar vein, {{Alan-Turing}} (father of the computer!) tried to analyze the range of fur patterns that are seen on animals (spots, patches or lines), as being different manifestations of a common driving mechanism  - where shifting the timing and intensities of the relationships of the driving mechanism would result in shifts in which pattern manifests. Rather than thinking of these distinctive markings as things 'in and of themselves' Turing wanted to understand how they might simply be different manifestation of more fundamental driving relationships.

Turing based his ideas on a reaction/diffusion model showing how shifting intensities of chemical relationships could create different distinct patterns.

Networks in Complexity Theory

Network theory is important in complexity thinking because of how the structure of the network can affect the way in which emergence occurs: certain dynamics manifest in more tightly or loosely bound networks, and information, a key driver of complex processes, moves differently depending on the nature of a given network.

Small Worlds, Growth & Preferential Attachment, Boolean Networks

Key work of network theorists include that of:

• {{steven-strogatz}} who developed small world networks where information can move quickly across the network; 
• {{Albert-laszlo-barabasi}} who showed how networks that observe {{power-laws}} can be generated, following the rules involving both 'growth' and '{{preferential-attachment}}'.
• {{Stuart-Kauffman}} who developed the theory of 'boolean' networks, where any series of linked nodes will ultimately move into regular regimes or cycles of behavior over multiple iterations in time.

Philosophical Interpretations

Alongside of these more technical ways of understanding networks, the appreciation of the more fundamental role of networks in structuring reality has also gained prominence. Networks would imply that functionality is something that is distributed, non-centralized, and shifting. In the social sciences, Actor Network Theory considers how agents power can be formed through network interactions. For philosopher Gilles Deleuze, the world is composed of what he terms a {{rhizomes}}, a concept that parallels that of a network in the sense of it being non-centralized, shifting, and entangled.

Historic Roots

The origins of network theory stretch back to earlier 'graph theory' a branch of mathematics developed by Leonard Euler, and made famous by his using graph theory to solve the "Konigsberg bridge problem". For a quick intro watch the video here:

This kind of graph analysis was considered as a relatively minor sub-field of mathematics, and only resurged when Barabasi reinvigorated the field (and transformed it into Network theory), with his network analysis work. Barabasi's work gained prominence as he was able to show how network theory could be applied to understanding the structural and functional properties of things like the world-wide-web. Today, network analysis is used in a huge array of disciplines in order to try to understand how the structure of relationships affects the functioning of a given entity - both at the level of the entire structure, and at the level of individual nodes (people, roads, websites, etc.), within the network.

Limitations?

It is perhaps worth noting that, along with computational modeling, network analysis is one of the central ways in which complexity dynamics are explored in many fields. While this kind of analysis can potentially be very helpful, the ubiquity of this strategy may have overshadowed some of its potential shortcomings. Network analysis can be very effective at demonstrating how {{driving-flows}} can move through a system, and how {{information-theory}} that steers the system can be relayed, but the precise configuration of networks often has surprisingly little to do with "classic" complex systems that we observe in the natural world.

If we are interested in the dynamics that form ripples in sand dunes, roll patterns in Benard cells, murmurations of starlings or even the emergence of ordered entities in Conway's Game of Life, then network structures do not appear to be playing any particular role. It is not as though graphing relationships between individual grains of sand on a dune will help us unravel the dynamics that form the emergent ripples. While network analysis often tries to pinpoint distinct actors in a system, very often agents in a complex system do not behave in distinctive ways. It is therefore somewhat surprising that Network Analysis has garnered so much strength as a key tool in complex systems research. Again, this is not to say that networks do not matter - certainly there are certain features of complex systems like the internet have key nodes like "wikipedia" that once entrenched help steer the system dynamics. It is just that there are many other features of complexity dynamics that may be overlooked if our primary focus is only on network relationships in a system.

Back to {{key-concepts}}

Back to {{complexity}}

The concept of interactive, incremental shifts in a system might seem innocent - but with enough agents and enough increments we are able to tap into something incredibly powerful. Evolutionary change proceeds in incremental steps - and with enough of these steps, accompanied by feedback at each step, we can achieve fit outcomes. Any strategies for increasing the frequency of these iterations will further drive the effectiveness of this iterative search.

One of the keys to enabling complex adaptation to manifest in a given system is the ability for these systems to unfold, with system complexity or fitness being enhanced with each ensuing step.

That said, the kinds of outcomes we see being derived from iterative unfolding differ somewhat in kind: some iterative processes lead to 'fitness' with respect to a given context, whereas other kinds of iterative processes generate pattern, but not necessarily fitness as the term would more generally be understood. Whether or not patterns might fulfill some other kind of fitness criteria is a more nebulous question, which we will get into later.

Differences and Repetitions 

Prior to that, let us first clarify what we mean by an iteration. We can think of an iteration in two distinct ways: the first involving sequential iterations, and the second involving parallel iterations. Thus we can imagine a system that unfolds over the course of 100 generations, or we can imagine a system that has 100 components. Each generation can undergo a mutation, testing a different strategy of performance, or, in the case of the simultaneous system, each component of the system can have a slightly different performance strategy. Thus, while we tend to think of iterations as sequential 'versions' of a class of elements, in essence we can also have multiple 'versions' that operate in parallel rather than sequentially. If we recall the example of ants searching for food, we have many ants performing search in parallel - many iterations of ant behavior proceeding simultaneously.

That said the notion of sequence is important, because it implies the possibility of feedback: that each version of action can be assessed, and undergo a modification at every time step based on feedback: has a particular strategy moved closer to or further from a goal?

Example

Lets start with 100 ants and give them 10 seconds to scurry around a table where we have placed one tasty peanut butter sandwich. Let's say only one ant finds this big cache of food on the first go  - Victory! This particular food locating strategy played out successfully for this particular ant in what we can call 'Version 1.0': What now needs to propagate through the colony as a whole is how to repeat this success - and this is where feedback enters into the picture. As part of Version 1.0 the victorious ant has gleefully deposited a bunch of pheromone traces enroute to the cache.  "Version 2.0"  has a couple of ants pick up on that trail, find the food, and pump up the pheromone signal, and so forth: through an iterative sequence of time steps - seeking, finding, and signaling - more and more ants are drawn to the yummy sandwich.

It is worth noting that even in round one, all ants had equal capacity to find food - the fact that one versus another ant was successful was effectively random. Thus what needed to propagate through the system was not some unique new superpower that this particular ant had (like some extra peanut butter receptors), but instead what needed to be replicated was the way in which the ants random search strategies were directed.

We can think of other examples of sequential iterations where the dynamics differ slightly in terms of what is being iterated. For example, we can state that the pathway from the first PC to today's smartphone was one that also proceeded by iteration, but the feedback driving each iteration entailed incremental system enhancements - a step by step learning from the previous model, (feedback), and then adjustments and improvements in the next round. 

There is thus a subtle difference between iterations that involve feedback that is generative in terms of modifying or enhancing the inherent nature of the agents in the system and feedback that is more about propagating a behavior that is already available to all the agents in the system (but only randomly enacted by some and not other agents).  

Many of the dynamics observed in complex systems have more to do with the iterative propagation of a particular behavioral regime. One form of propagation dynamics involves relaying a particular strategy that helps deliver a given resource or energy source to the group as a whole (patterns emerging that help direct slime mould or ants to viable food sources). Another propagation strategy involves driving a system towards regimes that minimize frictions or energy expenditures the system is encountering: water molecules coalescing into movement patterns that reduce internal drag differentials (generated by processing heat in Benard Rolls), metronomes synching so as to minimize movement frictions produced by their differentials). 

With each iteration of these systems, the overall performance gets just a little bit better: energy sources that fuel a group become easier to find and access, and energy expenditures demanded of a group (due to the forces imposed by an external driver) are modified so as to process these drives in a more frictionless, smooth manner. In both cases we can think of the system as trying to enter into regimes that  minimize global effort

Iterations for Fitness:

If a system can exist in many different kinds of states, with some states being more 'fit' than others, it is helpful if that system has an opportunity to explore different state possibilities. The faster it can explore the possibilities, the more likely it is to chance upon a state that is more productive or useful than another. This is why it is useful if a complex system has either a lot of agents, a lot of generations of agents, or both.

If we imagine a complex system as being capable of existing in many different kinds of states, than we can think of iterations as ways in which this {{phase-space}} of system possibilities is explored. It is therefore useful to think about whether a given group of agents in a system is being offered enough iterative capacity to explore this phase space quickly enough to learn anything of use. An ant colony of only 10 ants might do a very poor job of finding food - exhausting itself to death before it succeeds in finding nourishment. In principle, nothing is wrong with the ants (agents), the driving flows (food), or the signaling (pheromones). There is simply not enough system iterative capacity to learn.

Iterations for Pattern

Fractals:

The examples described above pertain to how iterations combined with feedback can steer a system towards effective behavior. But there is another way in which iterations are explored, in terms of their capacity to produce emergent pattern through simple step by step rules.

Here we would describe the nature of {{fractals-1}} generations, and how only a few rule steps, repeated over and over, can generate complex form that might be described as "emergent". Fractals like the Koch Curve or Serpinski Triangle are generated by simple geometric steps, (which we can call iterations) and more complex fractals like the Mandelbrot set can be created by creating a simple formulae that proceeds in recursive steps.

While these processes are iterative, and the patterns produced are emergent in that their spectacular aesthetic and harmonious qualities are not self-evident from their generative rules, these kinds of phenomena can not be seen to be 'learning' or becoming more 'fit'  in the same way as described above.  

Automata

Similar in terms of pattern generation, Conway's Game of Life (see Rules) is a prime example of complexity generated by such simple rules, that, repeated over multiple time step iterations yields highly complex behaviors.

Returning to the question of fitness and learning, this famous example of emergent complexity is entitle 'the game of life', but is it really life? While the emergent outcomes of the automata are rich in variety, can we say that the system adapts or learns, or becomes more fit? One feature of the output is that some of the 'creatures' generated in the game are able to enter into iterative loops, meaning that once these forms emerge they continuously reproduce versions of themselves. If proliferation within the grid of the game is thus considered to be a form of higher evolution (or Fitness), then perhaps this could be seen as a form of learning. That said, the Game of Life does not seem to 'learn' in ways we would associate with the word. 

Game of Life from Wikimedia Commons

Explorations

Returning to notions of fitness in the more traditional sense,  it is helpful to think of iterations as the way in which a complex system explores the scope of possibility within a {{fitness-landscape}}. As described in more detail elsewhere, a fitness landscape represents the differential structure of possibilities within a space of all possible behaviors {{phase-space}}, where more successful strategies within that space of possibility are conceptualized as peaks. Agent iterations can then be seen as processes of stepping around the fitness landscape, testing to see which steps take us up to higher peaks. In terms of these exploratory journeys, some agents may choose to incrementally modify whatever strategy they initially stumble upon (making small modifications and testing to see whether or not those modifications moved them higher or lower), and other strategies involve a more random 'jumping': abandoning a given set of strategies to test an altogether different set of alternatives. These jumps can be productive if they land agents on what are fundamentally higher peaks. These dynamics are unpacked in more detail on the pages referenced, but what is important to note is that the size of a step (or iteration), can vary between small local steps, and big global jumps.

In order to be responsive to a world consisting of different kinds of inputs, complex systems tune themselves to states holding just enough variety to be interesting (keeping responsive) and just enough homogeneity to remain organized (keeping stable). To understand how this works, we need to understand flows of information in complex systems, and what "information" means.

Complex Systems are ones that would appear to violate the second law of thermo-dynamics: that is to say, order manifests out of disorder. Another way to state this it that actions within the boundary of the systems are one where order ({{negentropy}}), increases over time. This appears counter to the second law of thermodynamics which states that, left to its own devices, a system's disorder (entropy) will increase. Thus, we expect that, over time, buildings break down, and a stream of cream poured into a cup of coffee will dissipate. We don’t expect a building to rise from the dust, nor a creamy cup of coffee to partition itself into distinct layers of and cream and coffee.

Yet similar forms of unexpected order arise in complex systems. The reason this can occurs is that complex systems are not fully bounded - they are {{open-dissipative}} structures that are subject to some form of energy entering from the outside, and within these "loose" boundaries, we see glimpses of temporary order.  Disorder is, however, still being ejected outside of these same boundaries - stuff comes in, stuff goes out -  in some other form. It is only within the boundaries that we see temporary pockets of order. In order to get a better grasp on how these pockets of temporary order appear, we need to understand the relationship between entropy  (disorder, or randomness) and information.

PART I: Understanding Information

Shannonian Information

An important way of thinking about this increase in order relates to concepts based in information theory.  Information theory, as developed by Claude Shannon, evaluates systems based upon the amount of information or 'bits' required to describe them.

Shannon might ask, what is the amount of information required to know where a specific molecule of cream is located in a cup of coffee? Further, in what kinds of situations would we require more or less information to specify a location?

Example:

In a mixed, creamy cup of coffee, any location is equally probable for any molecule of cream. We therefore have maximum uncertainty about location:  the situation has high entropy, high uncertainty, and requires high information content to specify a location.  By contrast, if the cream and coffee were to be separated (say in two equal layers with the cream at the top) we would now have a more limited range of locations where a particular bit of cream might be placed. Our degree of uncertainty about the cream's location has been reduced by half, since we now know that any bit of cream has to be located somewhere in the upper half of the cup - all locations at the bottom of the cup can be safely ignored.

Information vs Knowledge

Counterintuitively, the more Shannonian information required to describe a system, the less structured or "orderly" it appears to us.  Thus, as a system become more differentiated and orderly - or as emergence features arise - its level of Shannon information diminishes.

This, in a way, is unfortunate: our colloquial understanding of 'having a lot of information', pertains to us knowing more about something. Thus, seeing a cup of coffee divided in cream and coffee layers, we perceive something with more structure, more logic, and we might assume this it should follow that this conveys more information to us (at least in our normal ways of thinking about information - in this case that coffee and cream are different things!). A second, stirred cup appears more homogenous – it has less structure or organization. And yet, it requires more Shannon information to describe it.

A difficulty thus lies in how we tend to intuitively consider the words ‘disorder’ and ‘information’. We associate disorder with lack of structure (or low amounts of information) and order with more knowledge and, therefore, more information).

While intuitively correct, unfortunately this is not how things works from the perspective of information and communication signals - which is what Shannon was concerned with when formulating his ideas.  Shannon  (who worked for Bell Laboratories) was trying to understand the required bits of information needed to relay the state of a system (or a signal).

Example:

Imagine I have an extremely messy dresser and I am looking for my favorite shirt. I open my dresser drawers and see a jumble of miscellaneous clothes: socks, shirts, shorts, underwear. I rifle through each drawer examining each item to see if it, indeed, is the shirt I am seeking. To find the shirt I want (which could be anywhere in the dresser), I require maximum information, since the dresser is in a state of maximum disorder.

Thankfully I spend the weekend sorting through my clothes. I divide the dresser by category, with separate socks, shirts, shorts, and underwear drawers. Now, if I wish to find my shirt,  my uncertainty about its location has been reduced by one quarter (assuming four drawers in the dresser). To discover the shirt in the dresser's more ordered state requires less information:  I can limit myself to looking in one drawer only.

Let us take the above example a little further:

Imagine that I love this particular shirt so much that I buy 100 copies of it, so many that they now fill my entire dresser. The following morning, upon waking, I don't even bother to turn on the lights. I reach into a drawer (any drawer will do), and pull out my favorite shirt!

My former, messy dresser had maximum disorder (high entropy), and required a maximum amount of Shannon Information ('bits' of information to find a particular shirt).  By contrast, the dresser of identical shirts, has maximum order (negentropy), and requires a minimal amount of Shannon Information (bits) to find the desired shirt.

Interesting information:  States that matter!

It should be noted that the two extreme states illustrated above are both pretty uninteresting. A fully random dresser (maximum information) is pretty meaningless, but so is a dresser filled will identical shirts (minimum information). While each are described by contrasting states of Shannonian information, neither maximum nor minimum information systems appear very interesting.

One might also imagine that neither the random nor the homogeneous systems are all that functional. A dresser filled with identical shirts does not do a very good job of meeting my diverse requirements for dressing (clothing for different occasions or different body parts), but my random dresser, while meeting these needs, can't function well because it takes me forever to sort through.

Similarly, systems with too much order cannot respond to a world filled with different kinds of situations. Furthermore, they are more vulnerable to system disruption. If you have a forest filled with identical tree species, one destructive insect infestation might have the capacity to wipe out the entire system. If I own 100 identical shirts and it goes out of style, I suddenly have nothing to wear.

Meanwhile, if everything is distributed at random then functional differences can't arise: a mature forest eco-system has collections of species that work together, processing environmental inputs in ways that syphon resources effectively - certain species are needed moreso than others. In my dresser, I need to find the right balance between shirts, socks, and shorts: some things are worn more than others and I will run into shortages of some, and excesses of others, if I am not careful.

PART II:  Information Sorting in Complex Systems

Between Order and Disorder

What is interesting in Complexity, is that it appears that, in order to be responsive to a world that consists of different kinds of inputs, complex systems tune themselves to information states involving just enough variety (lots of different kinds of clothes/lots of different tree species) and just enough homogeneity (clusters of appropriately scaled groups of clothing or species). While within their boundaries these systems violate the second-law of thermodynamics (gaining order), they do not gain so much order as to become homogenous. The phrase 'poised at the edge of order and chaos' seems to capture this dynamic.

Tuning a complex system -  decreasing uncertainty

Imagine we have a system looking to optimize a particular behavior - say an ant colony seeking food. We place an assortment of various-sized bread crumbs on a kitchen table, and leave our kitchen window open overnight. Ants march in through the window, along the floor, and up the leg of the table.

Which way should they go?

From the ants perspective, there is maximum uncertainty about the situation: or maximum Shannonian information. The ants spread out in all directions, seeking food at random. Suddenly, one ant finds food, and joyfully secretes some pheromones as it carries it away. The terrain of the table is no longer totally random: there is a signal - food here! Nearby ants pick up the pheromone signal and, rather than moving at random, they slightly adjust their trajectories. The ant's level of uncertainty about the situation has been reduced or, put another way, the pheromone trail represents a compression of informational uncertainty - going from 'maximum information required' (search every space), to 'reduced information required' (search only spaces near the pheromone trace).

If all ants had to independently search every square inch of tabletop to find food, each would require maximum information about all table states. If, instead, they can be steered by signals (see {{stigmergy}}) deployed by other ants, they can then limit their search to only some table states. By virtue of the collective, the table has become more ‘organized’ in that it requires less information to navigate towards food. There is a reduction of uncertainty, or reduction of 'information bits', required by each ant to find the location of 'food bits'. Accordingly, these are more easily discovered. It is worth noting that in this particular system, the "food bits" are effectively the {{driving-flows}} that energize the system and thereby help fuel the localized order. The second law is preserved, since the ants will ultimately dissipate this order (through heat generated in their movements, through ant deffication as they process food, and ultimately through death and decay). 

Reduce information | Reduce effort

Suppose we are playing 20 questions.  I am thinking of the concept ‘gold’, and you are required to go through all lists of persons, places, and things in order to eventually identify ‘gold’ as the correct entity. Out of a million possible entities that I might be thinking of, how long would it take to find the right one in a sequential manner? Clearly, this would involve a huge length of time. The system has maximum uncertainty (1 million bits), and each sequential random guess reduces that uncertainty by only 1 bit (999,999 bits to go after the first guess!). While I might 'strike gold' at any point, the odds are low!

From an information perspective, we can greatly reduce the time it takes to guess the correct answer if we structure our questions so as to minimize our uncertainty at every step. Thus if I have 1,000,000 possible answers in the game ‘twenty questions’, I am looking for questions that will reduce these possibilities to the greatest extent at each step. If, with every question, I can reduce the possible answers in half (binary search) then, within 20 question, I can generally arrive at the solution. In fact the game, when played by a computer, can solve for any given entity within an average of six guesses! With each guess, the degree of uncertainty regarding the correct answer (or the amount of Shannonian information required), is reduced.

Sorting a system so there is less to sort

From a Complex Systems standpoint, this information sorting by agents within a system will allow it to channel resources more effectively – that is, focus on work (or questions) that move towards success while engaging in less wasted effort.

To illustrate:  imagine that I wish to move to a new city to find a job. I can choose one of ten cities, but other than their names, I know nothing about them, including their populations. I relocate at random and find myself in a city of 50 people, with no job postings. My next random choice might bring me to a bigger center, but, without any information, I need to keep re-locating until I land in a place where I can find work.

If, instead, the only piece of information that I have is the city populations, I can make a judgement: If I start off my job hunt in larger centers then there is a better chance that jobs matching my skills will be on offer. I use the population sizes as a way to filter out certain cities from my search - perhaps with a 'rule' stating that I won't consider relocating to cities with less than 1 million inhabitants. This rule might cross out six cities from my search list, and this 'crossing out' is equivalent to reducing information bits required to find a job: I can decide that my efforts are better spent focusing on a job search in only four cities instead of ten (this may also be the reason why, in studying cities as complex systems, we often observe the phenomena of growth and preferential attachment, which manifests as {{power-laws}} in population distributions).

By now it should have become clear that this is equivalent to my looking for a given cream molecule in only half the coffee cup, or ants looking for food only on some parts of the table, or my search in 20 questions being limited only to items in the 'mineral' category.

All these processes involve a kind of information sorting that gives rise to order, which in turn makes things go smoother: from random cities to differentiated cities; from random words to differentiated categories of words.

What complex systems are able to do is take a context that, while initially undifferentiated, can be sorted by the agents in the system such that the agents in the system can navigate through it more efficiently. This always involves a local violation of the second law of thermodynamics, since the amount of Shannonian information (the entropy or disorder of the system), is always being reduced. That said, this can only occur if there is some inherent difference in the system, or 'something to sort' in the first place. If a context is truly homogeneous (going back to our dresser of identical shirts), then no amount of system rearranging can make it easier to navigate. Note that an undifferentiated system is different from a homogenous system. A random string of letters is undifferentiated; a string composed solely of the letter 'A' is homogeneous.

Accordingly, complex systems need to operate in a context where some kind of differential  (in the form of {{driving-flows}} are present. The system then has something to work with, in terms of sorting through the kinds of differences that might be relevant.

One thing to be very aware of in the above example, is how difficult it is to disambiguate information from orderliness. As our knowledge of probable system states becomes more orderly, Shannonian information is reduced.  This is a frustrating aspect of the term ‘information’, and can lead to a lot of confusion.

This Christmas Story illustrates how binary search can quickly identify an entity

Back to {{key-concepts}}

Back to {{complexity}}

What do we mean when we speak of Fitness? For ants, fitness might be discovering a source of food that is abundant and easy to reach. For a city, fitness might be moving the maximum number of people in the minimum amount of time. But fitness criteria can also vary - what might be fit for one agent isn't necessarily fit for all.

Getting Fit!

The idea of fitness in any complex system is not necessarily a fixed point. There can be many different kinds of fitness, and we need to examine each specific system to determine what factors are at play. For example, what makes a hotel room 'fit'? Is it location, or price, or cleanliness, or amenities, or all of the above? For different people, these various factors or parameters have different 'weights'. For a backpacker traveling through Europe, maybe the price is the only thing worth worrying about, whereas for a wealthy business person it may not factor in at all.

Despite these variations, there are certain principles that remain somewhat consistent, and this pertains to the idea of minimizing processes. We can imagine that certain behaviors in a system require more or less energy to perform. Agents in a system are always trying to minimize an energy expenditure, but what might entail a high energy expenditure for one agent might be a low energy expenditure for another (depending on what forms of energy they each have available to them. If an ant wants to find food, it prefers to find a source that takes less time to get to than one that is further away. Further, a bigger source of food is better than a smaller source of food, as more ants in the colony can benefit. Complex systems generally gravitate towards regimes that therefore in some way minimize energy expenditure to achieve a particular goal. However, this energy rationing depends both on the nature of the goal, and the resources available to reach it.

Example:

Returning to the example of finding a hotel room, consider the popular website 'Airbnb' as a complex adaptive system. Here, two sets of bottom-up agents (room providers and room seekers) coordinate their actions in order for useful room occupancy patterns to emerge. Some of these patterns might be unexpected. For example, a particular district in Paris might emerge as a very popular neighborhood for travelers to stay in, even though it is not in the center of the city. Perhaps it is just at a 'sweet-spot' in terms of price, amenities, and access to transport to the center. This is an example of an emergent phenomena that might not be predictable but nonetheless emerges over the course of time. In that case, rooms in that district might be more 'fit' than in another, because the factors listed (its relevant parameter settings in that particular zone) are highly appealing to a broad swath of room-seekers.

So in what way is the above example 'energy minimizing'? We can think of the room seekers as having different packages of energy rations they are willing to expend over the course of their holiday. One package might hold money, one might hold time, and one might hold patience for dealing with irritations (noisy neighbors that keep them from sleeping, or willingness to tolerate a dirty bathroom...). Each agent in the system is trying to manage these packets of energy in the most effective way possible to minimize discomfort and maximize holiday pleasure. So if a room is close to the center of the city, it might preserve time energy, but this needs to be balanced with preserving money energy.

We can begin to see that fitness is not going to come in a 'one size fits all' form. Some agents will have more energy resources available to spend on time, and others will have more energy resources to allocate in the form of money. Further, an agent in the system might be willing to spend much more money if it results in much more time being saved, or vice versa. We can imagine that an agent might reach a decision point where two equally viable trajectories are placed in front of them. The choice of time or money might be likened to a flipping of a coin, but the resulting 'fit' regime might appear as very different.

In order to better understand these dynamics, two features of CAS, that of a Fitness Landscape and ideas surrounding Bifurcations, clarify how CAS can unfold in multiple fit trajectories, but despite these differences the underlying principles of energy minimizing holds true.

Avoiding Work and the Prisoner's Dilemma

In the above example the agents (room seekers), employ cognitive decision-making processes to determine what a 'fit' regime is. But physical systems will all naturally gravitate to these energy minimizing regimes.

Example: 

When molecules in a soap bubble solution are subject to being blown through a soap wand, nobody tells them to form a bubble, and the molecules themselves don't consider this outcome. Instead, the bubble is the soap mixture's solution to the problem of finding a form that minimizes surface area and therefore frictions. The soap bubble can therefore be considered as an energy minimizing emergent phenomena  (for a detailed explanination, follow this link to an article on the subject: note the phrase, 'a bubble's surface will minimize until the force of the air pressures within is equal to the 'pull' of the soap film'). We can also think of a sphere as being the natural Attractor States of a soap solution: seeking to absorb maximum air with minimum surface - or doing the most with the least.

We can derive from these examples that one way we can examine complex systems is to equate 'fitness' with avoiding unnecessary work or effort. While this is important for individual cases of agents (specific birds in a flock, or specific fish in a school), what is also interesting in systems exhibiting {{self-organization}} (bird flocks and fish schools), is that this principle is extended to the include the group level. Thus the system, as a whole, finds a regime that expends the minimum effort to achieve a goal on the part of the group rather than on the part of the individual. This might involve individual sacrifices in order to enable overall group behavior to succeed.

These kinds of dynamics involving individual sacrifices (or trade-offs) where group performance ultimately matters are the subject of game theory. The Prisoner's Dilemma, for example, is a classic case where the most 'fit' long term strategy is for both players to sacrifice some potential individual gain, in favor of longer term collective gain. Fit strategies differ depending on whether or not the game is played once or multiple times, so in natural systems that have ongoing interactions of agents than there are different fitness incentives than in non-repeating scenarios.

Short Explanation of the Prisoner's Dilemma:

Back to {{key-concepts}}

Back to {{complexity}}

This coupling between input affecting output - thereby affecting input - creates unique dynamics and interdependencies between the two.

There are two kinds of feedback that are important in our study of complex systems: {{positive-feedback}}  and {{negative-feedback}}. Despite the value-laden connotations of these designations, there is no inherent value judgement regarding 'positive' (good) versus 'negative' (bad) feedback. Instead, the terms can more accurately be described as referring to reinforcing deviation (positive) versus suppressing or dampening deviation (negative). Reinforcing feedback thus amplifies slight tendencies in a system's behavior, whereas dampening feedback works to restrain any changes to system behavior.

Negative Feedback

We can think of a thermostat and temperature regulation as a classic example of dampening (negative) feedback at work. The thermostat has a desired state that it wishes to maintain, and it is constantly monitoring an input about whether or not it is achieving that target. If the temperature exceeds the target, then the thermostat activates a cooling mechanism; if the temperature falls short of the target then the thermostat activates a heating mechanism. The thermostat is therefore situated within an environment, (acted upon by outside forces) but is simultaneously helping create this environment (by being one of the environmental activating forces). It is able to respond to the input of the environment by activating and output that suppressed any deviation from the goal state (the goldilocks temperature).

Because of how Negative Feedback helps maintain a particular status quo, it is an important dimension of life, or {{homeostasis}}. Our body's ability to maintain a somewhat steady state is something we often take for granted, but it is worth pausing to reflect upon the amount of constant adjustments that are required in order to keep things like our temperature or glucose levels steady in light of extreme environmental fluctuations. 

Maintaining our own body temperatures within a narrow, healthy range requires three aspects: an input (temperature) a sensor (or brain) and a viable output (shiver to raise temperature if cold; sweat to lower temperature if hot).  While this is somewhat similar to the thermostat example, there are some slight differences: even though the act of sweating or shivering does in fact have a minute impact on the environment we are located within, these outputs do not have a significant enough impact on the environment to alter the input.

Cybernetics 

While homeostasis refers specifically to biological systems able to maintain themselves, in fact for any system where the goal is to avoid deviance - to maintain a steady state or goal for some given target such as temperature - these same three elements - inputs, sensors and outputs - need to be present.

{{Cybernetics}} is a field dedicated to understanding a whole host of systems from different disciplines in light of these characteristics, to better understand the means of self regulation in entities that seek to maintain a particular target behavior. The field emerged in the 1940s, and it, (along with general systems theory, which shares many similarities with complexity research but deals with closed rather than open-systems), is in many ways is a pre-cursor to complex adaptive systems thinking.

In many cybernetic systems, the dynamics become quite interesting, in that an output can flow back into the system as an input, in ways that we can think of as more directly 'self-regulating'.  A fly-ball governor is one such self-regulating mechanism (described on the {{cybernetics}} page), where the self-regulating dynamics of the mechanisms cause it to slow down when it exceeds a particular speed. Anther such self-regulating or self-governing dynamics can be observed in eco-systems, where if a population of animals increases beyond the environment's carrying capacity, that environment ceases to sustain those high numbers resulting in a die-off of excess animals.  Similarly, if population numbers drop significantly, then those remaining will have a high availability of food, and any offspring will thrive, leading to population growth. These two competing forces  -population growth and carrying capacity - work in tandem to  dampen the fluctuation of population numbers, preventing them from getting too high or too low. 

Another other classic example is the idea of an oarsman on a boat, trying to reach an island, and constantly adjusting the movement of the oar to compensate for the deviations caused by the environmental factors (water currents, wind, etc. ).

Cybernetic systems differ from complex adaptive systems in that the CAS features such as  {{emergence}} are typically associated as being the result of amplifying or positive feedback  vs Cybernetic systems that work more to maintain a stable state

Positive  Feedback

If negative feedback relied on an input, a sensor, and an output, then positive feedback operates in a equivalent way: the difference being that the output does not counteract the input, but instead builds upon, or reinforces it in some way.

We can observe that in many systems driven by simple rules, such as {{Fractals-1}} that over iterative sequences of graphic generation, become differentiated by more detail, more variation, and more pattern becoming apparent.

But Fractals are a specific class of entity that is limited to the domain of mathematics. Again, these kinds of positive feedback systems can exist in a wide range of non-mathematical domains, with the same principles at work.

Viral Orders:

In discussion the {{non-linear}} of Complex Systems, we used the example of a cat video going viral, in order to illustrate how a small, early amplification of a system preference can cause a massive shift in system outcome. Using the analogy of the 'rich get richer', cat videos that initially might get a few more clicks are recommended more frequently, leading to more views, leading to more recommendations and so forth. This illustrates the power of  positive feedback to amplify a particular aspect of a system such that it grows in importance in a non-linear way.

Another example of this comes from Network Theory dynamics, that examines how networks characterized by {{power-laws}} can be generated when the network is constantly growing, and when new nodes of the network can be added anywhere at random, but will affix preferentially to nodes in the network that are already highly linked. This, phenomena of "growth and preferential attachment" is again an example of positive feedback, and such dynamics are thought to explain things like scaling patterns seen in different cities within a given region.

Network examples are of interest because while behaviors are initially random, because of the nature of positive feedback, these random intensities come to be reinforced over time: leading to an increase in structure and pattern. The situation is almost the opposite of that of fractals: in fractal growth a very simple formula ultimately leads to greater and greater visual complexity: in complex systems steered by positive feedback, an initially random distribution of entities (human habitations, websites, cat videos, cricket chirping), gradually become more ordered and organized, with a few dominant entities emerging and thereafter constraining the performance, behaviors, or success of other entities in the system. We can say that the system, after a time, moves into {{enslaved-states}} with only a few behavioral regimes succeeding following feedback. 

In these instances, an initially random situation has small variations amplified, to the extent that  an initially random factor or actor becomes dominant, now steering the system. I do wish to point out that this would seem to muddy our earlier contrast of 'amplifying' vs 'restraining': once a particular behavior is amplified, it in turn winds up constraining the system, as deviance from that behavior is now more difficult. To illustrate: Wikipedia has become the default encyclopedic website due to positive feedback: now that it exists, it is in fact stabilized and resists being disrupted. Its amplified strength as a website is part of what now gives it stability, dampening further disruptions. This is a characteristic of {{Emergence}}, in that emergent systems like schools of fish or flocks of birds are driven into being through positive feedback, but then exert a kind of top down resistance to future change.

Dynamics of systems subject to both Positive & Negative Feedback:

Some very interesting complex systems are governed by a combination of both positive and negative feedback. 

For instance, in the example of governing animal population fluctuations described above, we can imagine that rather than settling into one steady-state population, a particular species might oscillate between two regimes - booms and busts in population as the carrying capacity environment undergoes stress and then recovery. When we examine the system more closely, we realize that there are actually both kinds of feedback at play: reproduction rate is an example of amplifying feedback - if every two rabbits that reproduces make four rabbits and those four rabbits go on to make 8 rabbits (and so forth), then we have the kind of accelerating growth associated with positive feedback. What then happens is that this drive towards amplification, is suppressed by resistance (the carrying capacity), which works to counter balance the growth. So if we start with 8 garden rows of carrots and at every generation of new rabbits the rate of carrot row consumption proceeds faster and faster, pretty quickly all the carrots are done (and by extension, all the unfed hungry rabbit are done too). In a way, the  terminology can be muddy in that our definitions of positive and negative can rely on what is considered to be "amplifying" feedback. If we shift the lens, we could think of carrots as the agent in the system (rather than rabbits), and we could state that, due to the positive feedback in their environment (rabbit reproduction), the rate at which carrots are being consumed is increasing (even as the number of carrots is diminishing). Accordingly, what we mean by 'positive' and 'negative' are often context dependent, and can shift depending on how we describe the features of 'amplification' or 'suppression'.

What is nonetheless very interesting is that we can have systems that involve competing forces of feedback - one that drives the system forward, the other that resists or suppresses this drive (as in the case of rabbit reproduction and dwindling carrot supplies). Depending on the extent to which these co-evolving system features are out of sync in terms of these respective rates (rate of rabbit reproduction vs rate of carrot growth), the system can begin to oscillate in irregular ways. These kinds of irregular oscillations can be observed in the logistic map (also called bifurcation diagram), which illustrate how systems can cycle between many different behavioral states - with extremes arising, being dampened and then arising again (to greater and lesser degrees). Many interesting complex systems are therefore neither being entirely steered towards stability (like cybernetic systems), nor steered towards unified amplification (like crickets chirping in sync), but instead ride cascading waves between different states.

The characteristics of how feedback is moving through the system, and whether or not the system is subject to one or more interdependent feedback loops is therefore at the heart of some of the most complex dynamics we observe in complex system, and why systems composed of seemingly simple agents can nonetheless produce very complex dynamics (the complexity is in the nature of the feedback, rather than the inherent characteristics of the system.

As a final thought on this, we can observe the double pendulum experiment, where we see the irregular motion of a pendulum, the motion of which is subject to interwoven feedback from competing sources - while a simple system, the patterns traced exhibit complex dynamics:

source, wikipedia

Back to {{key-concepts}}

Back to {{complexity}}

The Second Law of Thermodynamics is typically at play in most systems - shattered glasses don't reconstitute themselves and pencils don't stay balanced on their tips. But Complex Systems exhibit some pretty strange behaviors that violate these norms...

Equilibrium

In order to appreciate what we mean by 'far from equilibrium' we first need to start by understanding what is meant by 'equilibrium'. We can understand equilibrium using two examples: that of a pendulum, and that of a glass containing ice cubes and water.

If we set a pendulum in motion, it will oscillate back and forth, slowing down gradually, and coming 'to rest' in a position where it hangs vertically downwards. We would not expect the pendulum to rest sideways, nor to stand vertically from its fulcrum point.

We understand that the pendulum has expended its energy and now finds itself in the position where there is no energy - or competing forces -  left to be expended. The forces exerted upon it are that of gravity, and this causes the weight to hang low. The pendulum has arrived at the point where all acting forces have been canceled out : equilibrium.

Similarly, if we place ice cubes in a glass of water, we initially have a system (ice and water) where the water molecules within the system have very different states (solid and liquid). Over time, the water will cool slightly, while the ice will warm slightly (beginning to melt), and gradually we will arrive at a point in time when all the differences in the system will have cancelled out. Ignoring the temperature of the external environment, we can consider that all water molecules in the glass will come to be of the same temperature.

Again, we have a system where competing differences in the system are gradually smoothed out, until such time as the system arrives at a state where no change can occur: equilibrium.

In a complex system, we see very different dynamics: part of the strangeness of emergence arises from the idea that we might see ice spontaneously manifesting out of a glass of water! This is what we mean by 'far from equilibrium': systems that are constantly being driven away from the most neutral state (which would follow the second law of thermodynamics), towards states that are more complex or improbable. In order to understand how this can occur, we need to look at the flows that drive the system, and how these offer an ongoing input source that pushes the system away from equilibrium.

Example:

Lets take a look at one of our favorite examples, an ant colony seeking food. Lets start 100 ants off on a kitchen table (we left them there earlier when we were looking at {{driving-flows}}. The ants begin to wander around the table, moving at random, looking for food. If there are crumbs on the table, then some ants will find them, and direct the colony towards food sources through the intermediary signal of pheromones. As we see trails form (a clear line forming out of randomness like an ice cube fusing itself out of a glass of water!), we observe the system moving far from equilibrium. But imagine instead that there is no food. The ants just keep moving at random. No emergence, nothing of statistical interest happening. When we remove the driving external flow (food) that is outside of the ant system itself then the ants become like our molecules of water in a glass. Moving around in neutral, random configurations.  Eventually, without food, the ants will die - arriving at an even more extreme form of equilibrium (and then decay)!

Origins

The phrase "far from equilibrium" was originally coined by Ilya Prigogine, and was used to characterize such phenomena as Benard Rolls (see also {{ilya-prigogine-isabelle-stengers}}). Prigogine and Stengers were interested in how system that were driven by external inputs could gain order (as exhibited by the rolls), and how the increase in these external inputs could in turn drive order in increasingly interesting ways. 

Another way to say this is that systems in equilibrium lack energy inputs needing processing  whereas system from from equilibrium are characterized by having some kind of energy driver or differential at play.  

Muddying the Waters

While the above should now be somewhat clear, it is also true that complex systems, while indeed operating "far from equilibrium" can exhibit behaviors that imply a different kind of equilibrium: one that is not part of the domain of physics or chemistry but rather that of Game Theory (and economics). 

There are various multi-actor systems examined by Game Theorists and Economists, where actors (or agents) use competing strategies to see which will yield (or 'win') some form of allocation. Such games might be played once, to show optimum game choices, or multiple times, to see what occurs when past strategies plays a role in current strategies. Depending on how multiple agents deploy their strategies games might produce win/win outcomes (where multiple agents gain allocations),  win/lose outcomes (where my win results in your loss or vice versa), or lose/lose scenarios (where in efforts to outcompete one another, all agents wind up leaving empty handed). Game Theory can examine the kinds of strategies most viable for an individual agent in the system, but they can also analyze what strategies are most viable not solely for an individual agent, but for the collection gain of all agents in the system.

Such 'collective benefit' systems are described as being "Pareto efficient": and occur in instances when strategies result in dynamics whereby no agent can improve its own effectiveness without diminishing the effectiveness of the overall group. Another way to frame this is in terms of what would constitute a Pareto improvement: when system behavior can be enhanced in such a way that at least one agent is better off, and the system performance as a whole has not been made worse off by this change.

Example:

Imagine we are placing 100 trash cans in a park. We don't know where they should go, so we distribute them at random, but we add a few special features:

1. Each trash can has a sensor that can track how quickly it is filled

2. Each is also able to receive and relay a signal to its nearest neighbors - indicating its             rate of trash accumulation

3. Each is set on a rolling platform, that allowing it to navigate to a new location in the park.

Accordingly: the agent in the system is the trash can; the fitness criteria is gathering trash; the adaptive capacity is the ability to relocate; and the differential driving flows are the variable intensities of trash generation.

We can imagine this system to be driven now by simple rules: each trash can monitors, broadcasts, and receives information about its own rate of trash acquisition, as well as that of its nearest neighbors. At various time steps it makes a decision: remain in place or move - with movement direction weighted depending in accordance with more successful neighbors. Each movement entails a Pareto improvement.

It should be relatively intuitive to note that, over time, trash cans will move until such point as all cans are collecting identical amounts. At that point, the system has arrived at a Pareto Optima, where movement cannot occur without a reduction in overall system fitness (it should be noted that this state may only be a local optimum:  for more information). The system has calibrated itself to perform in the most effective way possible, restricted only by the scope of state spaces it was able to explore.*

* One proviso regarding this example is that the system may be  trapped in a local optima (see {{fitness-landscape}}). As a result, the system above will function more effectively if individual agents occasionally engage in random search regardless of neighboring states. This allows potential untapped domains of trash production to be discovered and the recruited for.

The reason it is worth pointing out this particular dynamic, is that game theory often discusses such optimizing strategies as finding "Equilibria".  Accordingly, we have the famous "Nash Equilibrium" as a kind of game theory state (see the Prisoner's Dilemma Game), as well as other game theory protocols that use the term "Equilibrium" to refer to end states strategies. While we normally speak of "Pareto efficient" or "Pareto Optimum" rather than "Pareto Equilibrium", there is a notional slipperiness at work here, meaning that it is easy to think of complex systems as arriving at a kind of steady state where the system has found a kind of poised balance (as in the trash cans above). This kind of calibration and balancing act within their environment might be described as existing in a state of ecological equilibrium (rather than being far from it).

The muddiness comes from how the term is technically applied in physics or chemistry versus how it is used in economics and game theory. 

Back to {{key-concepts}}

Back to {{complexity}}

Understanding the degrees of freedom available within a complex system is important because it helps us understand the overall scope of potential ways in which a system can unfold. We can imagine that a given complex system is subject to a variety of inputs (many of which are unknown), but then we must ask, what is the system's range of possible outputs?

The notion of degrees of freedom comes to us from physics and statistics, where it describes the number of possible states a system can occupy. For example, a swinging pendulum is constrained to a fixed number of 'states' (positions in space) that the pendulum can occupy. We can imagine that it is possible to map out all the potential locations of the pendulum's swing, and therefore the limits of all it behaviors. The degrees of freedom thus tells us something about what a system is capable of doing: it's potential. The system cannot act outside of the boundaries of this action potential.

For example, if we wanted to examine the maximum capacity of motion for a three-dimensional object in space, this can be provided using just six degrees of freedom, which together define changes both in orientation: (rotation) that occurs via the 'roll', 'yaw' and 'pitch' motions; and for changes related to displacement in space: through the 'up/down', 'back/forward' and 'left/right' parameters. We can see that all potentialities of movement are covered within this framework.

If we were to eliminate any of these parameters - for example the 'up/down' potential, then we have fewer degrees of freedom, certain types of movement would no longer be possible. Phase Space - thereby captures the sum total of all potential behaviors - sometimes referred to as a system's 'possibility space'.

(image courtesy of Wikimedia commons)

In addition to there being a range of phase space potentials, there may also be particular behaviors in phase space that are more likely to occur. Accordingly, if we were to map all of the potential states of a pendulum's behavior from any given starting position in phase space, we  would have what is known as a 'phase portrait' of that pendulum. This is to say that there are particular trajectories that the pendulum will follow within phase space. Different systems might have phase portraits that highlight certain special regions of phase space as being {{attractor-statesbasins}} which a system will tend to gravitate towards.

Human Systems

So far we have been speaking about physical degrees of freedom, but we might also imagine degrees of freedom in relation to behavioral possibilities.

Example:

Imagine we are wanting to stay at an airbnb. We could think of each airbnb option as being an agent in a complex system, competing to win us over by broadcasting its 'fitness' for our stay. Each Airbnb would be able to adjust a number of parameters that one might consider as important in choosing accommodation. These parameters could include cost, cleanliness, distance to center, size, and quietness. Different people might value (or weigh) these parameters differently, and choose their airbnb accordingly. At the same time, we can imagine that each airbnb has a capacity to adjust its 'state' to different degrees. Location is clearly a limited parameter: a given airbnb has no capacity to simply change its location. But it does have the capacity to adjust its price point. Size is also difficult to alter. But cleanliness might have more flexibility. Thus certain categories have more range in terms of their degrees of freedom than others. If airbnbs are considered as agents in a complex system, each competing to find patrons who wish to stay at their location, then they each have to operate within their particular bounds of freedom in terms of how they adjust to align themselves better with user needs. Thus if they can't compete on the basis of location then they can attempt to compete on the basis of cost.

More Than Three Dimensions - No Problem!

The airbnb case should also serve to illustrate that, in many scenarios, the degrees of freedom available to an agent in a complex system cannot be easily plotted in three dimensional space (that is a 'space' bounded by an x, y, and z axis). That said, just because phase space cannot always be easily visualized in three-dimensional space, it doesn't mean we have to bend our minds to imagine more than 3 degrees of freedom. Just because we can't easily draw a graph of all these potential parameters, we can certainly imagine sorting different priorities simply as parameter bars with weights. We then have a multi-parameter space that the agent is calibrated within.

Multiple Degrees of Freedom thought of as sliding parameter bars: 

Requisite Variety

Analyzing  agents in complex system according to their Degrees of freedom can thus be thought about as examining their range of possible parameter settings. This can be an extremely helpful way of thinking about the {{adaptive-capacity}} of the system: or what it can and cannot do in response to environmental changes or fluctuations. Another way to describe this is the idea of a system's {{large-number-elements}}: a phrase coined by Ross Ashby to highlight the amount of variability a system can enact. According to Ashby, a system needs to have a variety of responses commensurate with the variety of inputs. This responsive capacity can be defined more precisely by means of defining the agent's degrees of freedoms. 

Back to {{key-concepts}}

Back to {{complexity}}

The word Cybernetics comes from the Greek 'Kybernetes', meaning 'steersman' or 'oarsman'. It is the etymological root of the English 'Governor'. Cybernetics is related to an interest in dynamics that lead to internal rather than external governing.

Cybernetic thought is an important early precursor to Complex Systems thinking.

Imagine a ship, sailing towards a target (say an island). There are various forces (wind and currents) that act upon the ship to push it away from its trajectory. In order to maintain a trajectory towards the island, the steersman need not be aware of the speed or direction of the wind, or the velocity of the waves. Instead, he (or she), just needs to keep their eye on the target, and keep adjusting the rudder of the ship to correct for any deviations from the route.

In a sense, we have here a complete system that works to correct for any disturbances. The system is comprised of the target, any and all forces pushing the ship away from the target, the steersman registering the amount of deviation, and subsequently counterbalancing this through means of interaction with the rudder.

While it is true that the steersman is the agent that 'activates' the rudder,  it is also true that the amount of deviation the target presents also 'activates' the steersman.  Finally, the forces acting upon the ship are what activates the deviation. We thus have a complete cybernetic system, where the forces at work form a continuous loop, and where the loop, in turn, is able to self-regulate.

A cybernetic system works to dampen  any disturbances or amplifying feedback that would move the trajectory away from a given optimum range. Thermostats work on cybernetic principles, where temperature fluctuations are dampened.

Like CAS, Cybernetics is concerned with how a system interacts with its environment. However, Cybernetics focus on systems subject to negative feedback: ones self-regulating to maintain regimes of stable equilibrium where disruptions (or Perturbations) are dampened.

Macy Conferences

Control

Stafford Beer, an early proponents of Cybernetics, discusses the Watt Flyball Regulator

Complex Adaptive Systems do not obey predictable, linear trajectories. They are "Sensitive to Initial Conditions", such that small changes in these conditions can lead the system to unfold in unexpected ways. That said, in some systems, particular 'potential unfoldings' are more likely to occur than others. We can think of these as 'attractor states' to which a system will tend to gravitate.

What's so Attractive?

Often a system has the capacity to unfold in many different ways - it has a large 'possibility space'. That being said, there can be regions of this possibility space that exert more force or 'attraction' for the system.

In some kinds of systems these zones of attraction exist because of pre-determined energy minimizing characteristics of these regimes. For example, if we blow a soap bubble it 'wants' to become a sphere: this is the state where there is the most volume for the least surface area, and therefore also the best configuration for soapy molecules given that it locates them in their lowest energy state: one that best balances competing forces, being the expansion forces of the air pushing the system outwards, and the resistance forces of the soapy solution not wanting to waste any surface area. The spherical shape of the bubble is thus a kind of pre-given, and when we blow a bubble this it is this shape - rather than a cube or a conical form, that we can safely anticipate the form will take. 

Similarly, if we toss a marble in a vortex sphere at a science museum we know it will spin around the surface, but then ultimately make its way down to the bottom: this is the state of minimum resistance to the forces of gravity acting upon it.

It is this 'minimizing behavior' that is characteristic of attractor states - of all possible states within a given system {{phase-space}} (the space of all possibilities) some regions may require less energy expenditure to move towards others. We will see that there can also be systems that have more than one such minimizing regime.

Lock In!

While the two physical systems described above have natural attractors, there are also social system dynamics that can cause similar attractor dynamics to arise.

In these scenarios, attractor states are not necessarily pre-determined by natural forces, but can instead emerge over time, as the system evolves, and in light of {{feedback-loops}} forces.  That said, once present they can reinforce themselves by constraining subsequent actions of agents forming the system.

Example:

We can think of Silicon Valley as being an emergent attractor for tech firms, that has, over time, reinforced its position. What is interesting about this example, is that even though it comes from the social sciences rather than physical sciences, in some way the same minimizing principle applies - it is just a different form of minimization that does not have to do with the laws of nature, but instead the social laws of human interaction.

To put this another way, once Silicon Valley established itself as the main tech hub, any new entrants to the tech field could, in principle, have chosen to locate themselves elsewhere - there were multiple locational possibilities win {{phase-space}}. However, if they were to choose these other locations, they would be far more likely to encounter additional "resistance" or frictions that would inhibit success. This is because these non-silicon valley sites would lack factors such as of supporting infrastructure, abundant knowledge spillovers, experienced and readily available workers, etc. In a sense, the smoothest, least resistant course of business action for a technology firm is thus to locate where these kinds of external inputs are most easily accessed: a 'state of least resistance' - which in this case equates to Silicon Valley.

The emergence of such clusters of expertise is not limited to Silicon Valley. We often see that groupings of similar business co-locate in space (referred to as agglomerations), rather than distributing themselves evenly across a region. In a particular city we will see groupings ranging from jewelry stores, or cell phone service providers, or bridal salons, tending to coalesce in co-located groupings.

The precise locations of these groupings is not something that is established in advance in the way that the spherical shape of the soap bubble is. Instead, in these instances it is the processes of {{feedback-loops}} that, over time, reinforce minor locational advantages, such that the kind of spill-over advantages discussed in the Silicon Valley example lead businesses that co-locate to have a better chance of success compared to their far-flung competitors. 

Once these kinds of concentrations of expertise have coalesced in a particular region, it then attracts new entrants to the field, in the same way that the spherical form attracts the soap molecules. Any systems that enter into this kind of regime, where new behavior is directed according to what has occurred before in ways that are constraining and directing, can be considered to have entered into "Enslaved States" (a term popularized by Hermann Haken. The concept of 'Enslavement' captures the notion that certain attractor states can emerge from agent interaction, and once present,  will constrain the future action of these agents and all that come after them. The same idea is referred to as 'Lock-in' in the field of Evolutionary Economic Geography.

Shake it Up

We can see that in the example of the soap bubble, and the example of Silicon Valley, we have two very different kinds of system that are nonetheless both still trying to limit unnecessary energy expenditures. For the soap, the concern is minimizing surface tensions or stress, for the business owner, minimizing the tensions and stresses involved with finding good employees, or  access to good internet, etc. In this way the dynamics, while at first completely different, nonetheless run parallel. What is different is that in the human system the 'laws' at play are not stable over time. What might be best practice at one instant is not necessarily best practice at a later time. This is the risk of Lock-in:  that systems begin to perpetuate themselves beyond the point that they were helpful (the QWERTY key board, designed to slow down typists to ensure that the mechanical typing hammers would not jam, is a great example of this kind of lock-in).

In these kinds of lock-in systems not governed by physical laws, it is occasionally worth 'shaking the system up' in order to see if it can be dislodged from a weak regime and encouraged to explore alternative behaviors. This is described as introducing a system {{perturbation}}, a disturbance intended to jostle a system and then see what it settles back into.

Example: 

For much of human history, the most effective way for individuals to access goods was for them to converge towards a central market-place. This was the area for trade, and by being centralized and co-located, efforts to find goods could be minimized on the part of the consumer, and efforts to find customers could be minimized on the part of the seller.  This was the most "fit" way of achieving the goal of the acquiring and dispersing goods.

In recent decades, this model has begun to shift on its head. With the advent of information technologies, combined with innovations in transport logistics, it has become increasingly viable for companies to deliver goods directly to the homes of consumers. Rather than coming to a central market-place, goods are able to move directly from manufacturer to consumer. Frictions about what is needed where have been reduced, and costs and energy associated with physical markets vs virtual markets have been similarly reduced. 

We can think of each of these regimes of behavior and as each two separate attractor basins within a variegated possibility space of goods acquisition and dispersal strategies.  With changes in technology, one basin of attraction has, over time become more viable (therefore deeper), and the other seems to have shrunk back in relevance and depth. We seem to have arrived at a tipping point today, where the minimizing forces favoring e-commerce vs physical commerce have shifted. That said, the legacy system tends to persist (old habits die hard).

Enter a global pandemic: this is a great example of a system perturbation, which shakes up standard patterns of behavior. Indeed, Covid caused many people who had never shopped online to try this behavior, and realize that it does, indeed, minimize effort in new ways. This kind of system disturbance has moved many people out of their taken for granted regime of behavior, and caused them to move into new regimes. 

We can see from this example how a system perturbation can act as a kind of productive 'shock' that, if large enough, is able to move a system out of a prior attractor state and potentially into a new regime.

Multiple Attractors

In discussing the example above, we slipped in the idea that a system may have more than one 'well' or basin of attraction. It is worth exploring this a bit more, since we can imagine different kinds of possibility spaces - some that only have one deep well to which everything will ultimately  tumble (a single attractor like that which the pendulum moves towards), others can have multiple attractors, some deeper, some shallower, with a system able to explore multiple regimes of behaviors within the space.

Further, complex systems can sometimes oscillate between attractor states, both of which are equally viable. This can be described as a system having Multiple Equilibria. The example of Benard Rolls is a case in point - liquid is heated from below, and forces churn the water molecules so as to cause them to minimize resistance by moving into a "roll" pattern. That said, the direction of the roll -cascading left or cascading right- or two equally viable minimizing behaviors, both of which the liquid can move into. The system therefore has multiple equilibria

In addition, we can have systems that oscillate between attractors, rather than settling into a specific regime. An example would be a predator/prey system, where the population numbers of each species each rise and crash in recurring patterns over multiple generations. In this case, two attractors are coupled whereby as one intensifies (the prey reproduces a lot), in generate a response in another part of the system that is counterbalancing  (the predator finds a ready food source and is able to reproduce a lot). This creates a back and forth oscillation between high prey and high predator numbers, with each regime counterbalancing the other.

The same dynamics can be seen in what are known as chemical oscillators, where we have a phenomena of multiple attractors described as follows:

• a reaction intensifies certain chemical behaviors;
• beyond a certain threshold  these behaviors catalyze a new, counter behavior;
• this counter behavior intensifies...;
• beyond a certain threshold this counter behavior catalyzes the first behavior;
• etc. 

The result of these reactions can be quite surprising, as seen below!

Check out the Multiple Attractors in the Briggs Rauscher chemical oscillator.

Back to {{key-concepts}}

Back to {{complexity}}