Power laws arise in both natural and social system, in contexts as diverse earthquake behaviors, city population sizes, and word frequency use.
'Normal' vs 'Power Law' Distributions
Complex systems are often characterized by power law distributions. A power law is a kind of mathematical distribution that we see in many different kinds of systems. It has different properties from a well known distribution - a 'bell curve' 'normal' or 'Gaussian' distribution.
Let's look at the two here:
Power-law (left) vs Bell-curve (right)
Most people likely remember the bell curve from high school. The fat middle (highlighted) is the 'norm' and the two sides or edges represent the extremes. Accordingly, a bell curve can illustrate things like people's heights - with 'typical' heights being distributed around a large middle, and extreme heights (both very tall and very short people), being represented by much smaller numbers at the extremes. There are many, many, phenomena that can be graphed using a bell curve. It is suitable for depicting systems that hover around a normative 'middle', and for systems where there are no driving correlations amongst members of the set. In other words, the height of one person in classroom is not constrained or affected by heights of other people.
Power-law distributions are likely as common as bell-curve distributions, but for some reason people are not as familiar with them. They occur in systems where there is no normative middle where most phenomena occur. Furthermore, entities within a power-law set enjoy some kind of relations amongst them - meaning that the size of one entity in the system is in someway correlated with, (or has an impact) on the size and frequency of other entities. These systems are characterized by a small percent of phenomena or entities in the system, accounting for a great deal of influence or system impact.
This small percent is shown on the far left hand side of the diagram (highlighted), where the 'y' axis (vertical) indicates intensity or impact (of some phenomena), and the 'x' axis indicates the frequency of events, actors, or components associated with the impact. The left hand side of the diagram is sometimes called the 'fat head', and as we move along to the right hand side of the diagram, we see what is called 'the long tail'. Like the bell curve, which we can use to chart phenomena such as housing prices, heights, test scores, or household water consumption, the power law distribution can illustrate many different kinds of things.
Occasionally, we can illustrate the same phenomena using bell curves and power law distributions, while simultaneously highlighting different aspects of the same phenomena.
Let's say we chart income levels on a bell curve. The majority of people earn a moderate income, and a smaller number of people earn both very high and the very low incomes at the extreme sides. Showing this data, we get a chart that looks like the one below:
Wealth in the USA plotted as a bell curve (source: pseudoerasmus)
But, we can think of income distribution another way - the impact or intensity of incomes. Consider this fact of wealth distribution: in the US, if we look at the right side of the bell curve above (the wealthiest people who make up a small fraction or 1% of the population) these few people control around 45% of entire US wealth. Clearly, the bell curve does not capture the importance of this small fraction of extreme wealth holders.
Imagine that instead of plotting the number of people in different income brackets we were to instead plot the intensities of incomes themselves. In this case we would generate a plot showing:
1% (a few people) controlling 45% (a large chunk) of total wealth;
19% (a moderate number of people) controlling 35% ( a moderate chunk) of total wealth;
80% (the bulk of the population) controlling 20% (a small fraction) to total wealth.
These ratios plot as a power law, with approximately 20% of the people controlling 80% of the wealth resource.
These numbers, while not precisely aligning with US statistics, are not that far off, and they align with what is referred to as the '80/20' rule: where 20 percent of a system's components are responsible for 80 percent of the system's key functions or impacts. This might refer to many different kinds of things - quantities, frequencies, or intensities. Thus:
20% of our wardrobe is worn 80% of the time;
20% of all English words are used 80% of the time;
20% of all roads attract 80% of all traffic;
20% of all grocery items account for 80% of all grocery sales;
Finally, if we smash a frozen potato against a wall and sort out the resulting broken chunks:
20% of the potato chunks will account for 80% of the total smashed potato.
Such ratios are so common that if you are unsure of a statistic then - provided it follows the 80/20 rule - you are likely safe to make it up! (the frozen potato being a case in point :))
Another way to help understand how power law distributions work is to consider systems in terms of what is called their 'rank order'. We can illustrate this with language. Consider a few words from English:
'The' is the most commonly used word in the English language -
We rank it 'first' and it accounts for 7% of all word use (rank 1) .
"Of" is the second most commonly used word -
We rank it 'second' and it accounts for 3.5% of all word use (1/2 of the rank 1 word)
If we were to continues, say looking at the 7th most frequently used word, we would expect to see it use 1/7th as frequently as the most commonly used word. And in fact -
'For' is the seventh most commonly word,
We rank it seventh and it accounts for 1% of all word use (1/7 of the rank 1).
This is perhaps the most straightforward ratio driving a power-law, known as 'Zipf's Law' for George Kingsley Zipf, the man who first idenitified it. Zipf's law indicates that if, for example, you have 100 items in a group, the 99th item will occur 1/99th as frequently as the first item. For any element in the group, you simply need to know its rank in the order - 1st, 3rd, 25th - to understand its frequency (relative to the top ranked item in the group).
The constant in Zipf's law is '1/n' , where the 'nth' ranked word in a list is used 1/nth as often as the most popular word.
Were all power-laws to follow a zip's law then:
the 20th largest city would be 1/20th the size of the largest;
the 10th most popular child's name would be used 1/10 of the time compared to the most popular;
the 3rd largest earthquake in California in 100 years would be 1/3 of the size of the largest;
the 50th most popular product would sell 1/50th as often as the most popular .
This is a very easy and neat set, and it is represents perhaps the most straightforward power law. That said, there can be other power law ratios between elements which, - while remaining constant, are not always such a 'clean' constant. These follow the same principle but arejust are just more difficult to calculate. For example"
'1.07/n' would be a power-law where the 'nth' ranked word in a list is used 1/1.07 times as often as the most popular word.
Pretty in Pink
Clearly '1.07/f' is a less satisfactory ratio then 1/n. In fact, the 1/n ratio is so pleasing that it has a few different names. 1/n is mathematically equivalent to 1/f ratio where, but instead of highlighting the rank in the list, 1/f highlights the frequency of an element in a list (the format is different but the meaning is the same).
'1/f' is also described as 'pink noise' - which is a statistical pattern distinct from 'brown' or 'white' noise. Each class of 'noise' pertains to different kinds of randomness in a system. In other words, while many systems exhibit random behaviors, some random behaviors differ from others. We can think of 'pink' 'white' and 'brownian' being different 'flavors' of randomness. Without getting into too much detail here, 1/f noise seems to occur frequently in natural systems, and are also associated with beauty. Without going into the mathematics of pink noise, it can be described as a frequency ratio of component distributions such that there is just enough correlation between elements to provide a sense of unity, and just enough unexpectedness to provide variety.
Dynamics generating Power-laws
Power laws distributions have been identified in many complex system behaviors, such as:
earthquake size and frequency
web site popularity
academic citation network structure
word use frequency
....and much more!
Much time and energy has gone into identifying where these distributions occur and also trying to understand why they occur.
A strong contender for the presence of power-law dynamics is that they may be the result of systems that involve both growth and Preferential Attachment. Understood colloquially as 'the rich get richer', preferential attachment is the idea is that popular things tend to attract more attention, thereby becoming more popular. Similarly, wealth begets wealth. The idea of growth and preferential attachment is therefore associated with positive feedback. It can be used to explain the presence of power-law distributions in the size and number of cities (bigger cities attract more industry thereby attracting more people...) the distributions of citations in academic publishing (highly cited authors are read more thereby attracting more citations), and the accumulation of wealth (rich people can make more investments, thereby attracting more wealth).
Further, power-laws might be understood as a phenomena that occur in systems that involve both positive and negative feedback as interacting and co-evolving drivers operating amongst the entities within the system. Such systems would involve feedback dynamics that are out of balance: some feedback dynamics (positive) are amplifying certain system features, while others system dynamics (negative) are simultaneously 'dampening' or constraining these same system features.Their is a correlation between these push and pull dynamics - so the greater the push forward the more it generates a pull back, and vice versa. The imbalance between this push and pull interplay between interacting forces creates feedback loops that lead to power-law features.
An example of this would be that of reproducing species in an eco-system with limited carrying capacity. Plentiful food would tend to amplify reproduction and survival rates (positive feedback), but as population expands this begins to put pressure on the food resources, leading to a push back (lower survival rates), and consequently a drop in population levels. The two driving factors in the system - growing population and dwindling food - are causally entwined with one another and are not necessarily in balance. If the system achieves a perfect balance then the system will find an equilibrium - the reproduction rate will settle to a point where it matches the carrying capacity. But if there are forces that drive the system out of balance, or if there is a lag time between how the two 'push' and 'pull' (amplifying and constraining) dynamics interact, then the system cannot reach equilibrium and instead keeps oscillating between states (see Bifurcations ).
It has been shown that the frequency of baby name occurrences follows a power-law distribution. In this example, what is the push/pull interplay that might lead to the emergence of this regularity? While each set of parents chooses their child's name independently, they do so within a system where their choices are somewhat driven or constrained by the choices being made by parents around them. Suppose there is a name that, for some reason, has become prevalent in popular consciousness - perhaps a character name in a popular book or tv series. It is not necessary to know the precise reasons why this particular name becomes popular, but we can imagine that certain names seem to resonate in popular consciousness or 'the zeitgeist'. Let us take the name 'Jennifer'. An obscure name in the 1930s, it became the most popular girl's name in the 1970s. During that time, if you were one of the approximately 16 million girls born in the US, there was a 2.8% chance you would be named Jennifer ! And yet, the name had plummeted back to 1940s levels by the time we get to 2012.
the rise and fall of Jennifer
But how can the rise and fall of 'Jennifer' be described using push and pull forces? We can imagine a popular name being like a contagion, where a given name catches on in popular consciousness. While spreading, the name it brought even further into popular consciousness, potentially expanding its appeal. At the same time, the very fact that the name is popular causes a tendency for resistance - if Jennifer is on a short list of possible baby names, but a sibling or close friend names their child 'Jennifer', this has an impact on your naming choice. In fact, the more popular the name becomes, the more pullback we can expect. As more and more people tap into the popularity of a name, it becomes more and more commonplace, leading to a sense of overuse, leading to a search for new novelty. The interactions of push and pull cause the name to both rise and fall. In a system of names, Jennifer is a name that had an expansion rate caused by rising popularity feedback, but then a decay rate caused by overuse and loss of freshness.
Again, these dynamics find their expression in the ratios associated with power laws. The distribution is always characterized by a small number of system components wielding a high degree of system energy or impact. It should, however, be noted that power laws are not without controversy: while power laws are often described as 'the fingerprint of complexity', some argue that the statistics upon which they are based are often skewed, and that power-laws may not be as common in systems as is frequently stated.
Because of their contributions to our understanding of power law distributions (Zipf for word frequency and Pareto for city size), we sometimes call such power laws 'Zipf' or 'Pareto' distributions. (for more see: George Kingsley Zipf and Vilfredo Pareto ).