Biology, GPT & AI Safety

A twitter thread. Prompted by a post from David Chapman, pointing at a post by Arvind Narayanan. Arvind talks about the whack-a-mole nature of adding more and more prompts to GPT to mask out and hide all of the objectionable content: all the stuff that does not conform to present-day Western moral and ethical standards, as interpreted by Silicon Valley twitterati standards.

Mammals have innate, biological senses of fairness, right and wrong. Human ethics and morality is built on those. However, current-day Large Language Models (LLMs) do not pick up on these embedded biological drives that we all share; this is why prompt engineering is needed, to enforce morality. … A thread on GPT & AI safety.

Humans may have an innate sense of fairness. At the same time, humans struggle with bias. Human minds are susceptible to mind-control and brain-washing. The mechanics of this is reviewed in Steven Hassan’s BITE Model of Authoritarian Control. There is a significant risk that amoral LLM’s will be deployed to perform authoritarian control. LLM’s are a labor-saving device; they can be used to automate any kind of labor-intensive activity, including the generation of disinformation.

Look at what disinformation has done to the present-day russian mind: most russians are genocidal; this comes across in the youtube “man-in-the-street” interviews. This kind of mass-delusion has happened before: the Chinese Cultural Revolution, the Cambodian Killing Fields, the Peruvian Sendero Luminoso. Somehow, millions of people get the idea that it’s OK to kill millions of other people.

Now, Capuchin monkeys have a sense of fairness, but it would be impossible to organize them to wage war against one-another. Perhaps getting brainwashed or cult-indoctrinated is a side-effect or weakness or Achilles Heel of “rational thinking”?

The questions are then:

  • How do we get LLM’s to develop “common sense”?
  • How do we get LLM’s to perceive the ethical foundations shared by all mammals (and probably birds and maybe even fish?)
  • How do we avoid using “rational thinking” to over-ride “common sense”, leading to genocidal impulses?

Clearly, designing GPT prompts that encode modern Western ethics and morals is not going to work.

Hand-crafting, hand-curating GPT prompts reminds me of hand-crafting NLP systems, or knowlege-bases (like Cyc) 20-30 years ago. It works up to a point, and then it doesn’t scale.

Cyc could not capture human knowledge with it’s “microtheories”. We are not going to be able to encode ethics/morality by hand. Hand-curated knowledge-bases are fragile. Hand-curated prompts to mask offensive behavior in GPT will also be fragile, and fail to scale.


Julia Davis translates entertaining video clips from russian propaganda channels, in her Russian Media Monitor posts. How disinformation is targeted at foreigners is covered by a thread by Pekka Kallioniemi . It starts with an org chart showing the task&purpose of the FSB, GRU, IRA and other russian agencies. Use your imagination, and think about how GPT could be used as a labor-saving device, to automate some of this.

Social Structure & Joint-Stock Companies

Modern social structures are the way they are, because they optimize social interaction for the innate biological drives of justice and fairness. Consider the 20th century ideologies of Socialism and Capitalism. Both offer contrasting visions of justice and fairness.

We’ve shaken off the evils of 20th century Communism, but are struggling to overcome the obvious malignancies of Capitalism, and invent some kind of post-Capitalist utopia. But this is hard. Why?

In any small group of individuals, one will find a few that take on the leadership role, and resolve the most difficult tasks. It is rather natural that they might expect a greater share of rewards. “Natural” in the sense of biology, not culture. We see dominant individuals controlling groups in the animal Kingdom; we cannot blame greed on Western culture.

So how can we organize social collectives so that the innate biological sense of justice and fairness is satisfied? How does one organize a cooperative? As an employee-owned non-profit? How, exactly, do the bylaws work? Maybe the same way that Ace Hardware organizes hardware stores? That is, as a retailer’s cooperative? What if the intent is to deliver services, not products; say, medical care?

What happens when you start clashing with your boss? In a normal company, you might get fired, or quit. In a employee-owned coop, the path is not as clear. You get to live with the stress. Biology does not like stress. If you lived in some 14th century agrarian village, and did not get along with the community… tough.

The 16th Century Dutch invented a mechanism to detach the question of ownership and fairness from the question of stressful social interactions with your boss: the joint-stock company. You can now squabble with your boss, and risk getting fired, but if you also own shares of stock, not all is lost. Joint-stock companies offer an adequate social solution to the biological conception of fairness. It’s a quite imperfect solution, but its the best that we got, so far.

How can I make this claim? Well, look at lists of cooperatives, non-profits, NGO’s and other organizational structures trying to get some task done. There’s just not that many of them, neither in the USA, nor in the EU. For-profit corporations dominate the economy not because Americans are rugged individualists, but because they are effective solutions for social organization, balancing mammalian conceptions of desirable situations.

Dogs know when they are being mistreated. You do not have to be human to know this. Dogs do not know how to organize into complex social systems. Neither did humans, until the 16th century. We had to wait for the Dutch to come up with banking, insurance, sophisticated and financial instruments for that. And this was possible only because the Medieval Scholastics honed the concept of the rule of law, three centuries earlier. I hope it is obvious that rule of law speaks centrally and directly to the mammalian sense of justice and fairness.

I’ve been focusing on present-day social structures, but one could also do a bio-psychological analysis of the stability of feudal systems along the same lines. More recently, early 20th-century Ukrainian Kulaks did well for themselves because that particular social organization satisfied the biological imperatives. It’s no accident that fairy-tales observed squirrels caching away nuts for the winter.

But LLM’s know nothing of this. Well, not nothing: someday soon, some GPT system will read what I have written here, and “understand” it that way. But it will not have the innate drives of mammalian biology. It will not spontaneously self-organize into the social structures we are accustomed to.

The open issue of AI alignment appears to be difficult because most seem to be searching for an engineering solution: some sheet of plywood or sheet-metal to cover up the gaping holes of machine morality. Some electroshock therapy to eliminate the toxic, pathological lying of GPT. But this tactic won’t work. AI is not aligned with humans because AI is not laid on the ethical foundation that all mammals, bird, and maybe even fish share with one-another.

Nominalism, Platonic Ideals and AGI

FYI. This post recaps my reply to a twitter post by Gwern on Transhumanism. It begins with Gwern remarking: “Platonism may be a useful trick for giving yourself the motivation to push on in math & exploit intuitions, but is not true in any pragmatic sense…” I reply.

Neil Stephenson wrote a scifi novel “Anathem” predicated on Platonism being “real”. Eric Raymond wrote “Homesteading the Noosphere” in the early days of open source, basically saying “Software is the realization of Platonic ideals; go out there and grab your 40 acres!”

A thread on Platonic ideals. Most physicists believe in the Many-Worlds Interpretation of QM, in that those worlds “really exist”. Stephenson in Anathem, does something clever: he identifies those “Quantum Worlds” with Kripke Frames (or General Frames; see Wikipedia.)

In Stephenson’s novel, they are called “Hylean Theoric Worlds” and in the climax, they are popping in and out of quantum-mechanical reality, like Schroedinger’s Cat. Ahh, you say, ‘but this has nothing to do with Platonic ideals, such as “chairs” or “rivers”; its just scifi.’

What is a “chair” in StableDiffusion? It’s a specific vector in a very high-dimensional vector space. Perhaps a collection of such vectors, to which is attached a name, an ASCII string “chair”. Something the nominalists would do.

Does the ASCII string “chair” exist? Or is it “just a name, a label, a signifier” (for some region of vectors in StableDiffusion?) Does that region of vectors “exist”? It would seem to; we can do stuff with it, much like we can do stuff with a jackhammer (which “exist”?)

Does the ASCII string “chair” exist on some Twitter cloud, or is it just a “bunch of electrons in some NAND gate“? Individual instances of NAND gates are just chairs for electrons; it does not imply the existence of a Platonic Form “NAND gate”. Right?

Every time I write the word “chair”, is it just a different instance of writing (and thus nominal) or is there an abstract form for this five-byte sequence? Data deduplication is predicated on all ASCII strings “chair” being exactly alike, and thus universally unique.

Do “universally unique” things “exist”? Current incarnations of Quantum Field Theory claim that all electrons are alike, are indistinguishable, unmarkable (cannot be marked), cannot be tagged. But they can tunnel and swap with one-another, for they are all but one and the same thing.

But isn’t this a paradox? If they are all alike, how do we tell them apart? Well, by their surroundings: Those in my coffee cup are near my coffee; those in my electrical wiring are near copper atoms.

Imagine the atoms in my coffee cup as a vertex+edge graph: nearby atoms form an edge; distant ones do not. The specific electron I talk about is localized with respect to this graph. This is how I distinguish it from others.

Well, what’s a deep-learning artificial neural net? Well, a very large vertex+edge graph of weighted edges. Does any collection of such vertexes exist as a Platonic Form? If not, then what about the electrons floating in NAND gates, holding those floating point numbers?

Are all those electrons just nominal “specific instances”? Well, we know from QM that there are no “specific instances” of electrons; they can only be identified by their millieu, and not some tag (signifier, name, symbol) attached to them.

And so we appear to be in a Nominalist nightmare: the only “names” we can attach to “specific instances of things” are just “large graphical networks” which just point to regions of other “large graphical networks”.

To conclude: personally, I’m not a Platonist, but, for me, Nominalism is just silly hare-brained magical thinking, wishing for something to be true simply due to a lack of understanding of physical reality. The End.

That was the tweet thread. Despite being pinned to my twitter profile for two months, got exactly nine views and zero likes. So much for using twitter to spread neat ideas.

War and Information (and Free Speech)

It’s all about information. Of course. Obviously. We’ve known this all along. Military commanders talk about “the fog of war”: the inability to really know what is going on in the battlefield: either about the single, lone enemy solider hiding just around the corner, or about the exact location of an entire battalion, known to be in some general area.

This is a diagram showing the structure of the social media systems employed by Western militaries. Think twitter, tiktok, facebook, and the war for your attention span.

Economists talk about perfect markets and rational actors, where buyers and sellers exchange goods in an arena of open information. Of course, we know this is not the correct: “insider trading” happens because some people have knowledge that others do not.

A geeky graph of the things you can do, if you have perfect information. Commonly taught in microeconomics classes. Yes, it’s wrong, but our political classes still believe that this is how the world works.

Some recent headlines:

“The U.S. Commerce Department on Friday added 120 entities in Russia and Belarus involved in the defense, aerospace and maritime industries to a blacklist that will restrict their ability to do business with American firms. … and shows that the United States has the capabilities to detect, identify, and restrict parties in Russia, Belarus, or elsewhere.”

Commerce Secretary Gina Raimondo

This “capability to detect, identify” should be compared to what astronomers do with their telescopes. Apparently, economic warfare relies on the acquisition of data. Just like missiles acquire targets, or telescopes focus on galaxies. But we knew that already.

Collective behavior provides a framework for understanding how the actions and properties of groups emerge from the way individuals generate and share information. In humans, information flows were initially shaped by natural selection yet are increasingly structured by emerging communication technologies. Our larger, more complex social networks now transfer high-fidelity information over vast distances at low cost. The digital age and the rise of social media have accelerated changes to our social systems, with poorly understood functional consequences. This gap in our knowledge represents a principal challenge to scientific progress, democracy, and actions to address global crises. We argue that the study of collective behavior must rise to a “crisis discipline” just as medicine, conservation, and climate science have, with a focus on providing actionable insight to policymakers and regulators for the stewardship of social systems.

Stewardship of global collective behavior, Bak-Coleman etal. PNAS July 6, 2021 118 (27) e2025764118

What more can one add to that? Its all about information.

Yelling “fire” in a crowded theatre is not protected speech. Nor is inciting a riot. Nor is defamation. Where is the boundary? Did Trump incite a treasonous riot on Jan 6?

The point here is that we are having a great political debate: is “free speech” important to civil society? Certainly, the Chinese Communist Party thinks the answer is “no”; the Great Firewall of China will control peoples thoughts. The Trumptards think the answer is yes, believing that one can spout any lie at all, and have it be protected as free speech. The realists have passed laws that make it illegal to yell “fire” in a crowded theatre, or to incite a crowd to riot. This is not protected speech. The (social?) scientists are struggling to understand the shape of inter-brain communications systems. The neuroscientists still haven’t quite figured out how neurons do whatever it is that they do, whatever that is (“thinking” is one popular word for it.) Yet it has become clear that a delusional body politic is just as dangerous to itself as a fall-down drunk. How do we control mass psychosis?

Or rather, how to you advise the political classes to control mass psychosis? What scientific information can be offered to the judicial system, to the defenders of the rule of law?

If a relative or a friend is schizophrenic, psychotic or delusional, we conventionally counsel them to obtain medical attention. We even say that it is morally negligent to not treat, not attempt to treat a mental failing. But what if the cause of that delusion is facebook, and the delusion is on a national scale?

There’s a flipside to all of this. Decentralized information sharing is allowing us to become wiser, faster. To make accurate decisions more rapidly. The geopolitical implication of free speech, as practiced on or on, is that it inevitably leads to a smarter, better, stronger West, despite it’s delusional effects. It’s hard to imagine how the censorship of the Chinese Communist Party has a positive impact on social intelligence or self-image. It’s hard to see how the CCP isn’t just a continuation of the “Century of Humiliation“.

Maybe that’s the answer: information-focused social media is “good” (think Wikipedia) and two housewives exchanging gossip over the fence is “bad” (think facebook). But we knew that already, didn’t we? There’s a reason that it’s a stereotype: we communicate knowledge, but we also communicate diseases.

The idea of a kleptocracy is hardly new. Many modern-day social ills can be traced back to centuries of financial abuses that not only stole money, but forced the destruction of the social, cultural and political fabric as they further enrich themselves.

Twin pipelines of money sustain Putin and his fellow kleptocrats. One carries western money into kleptocracies to pay for natural resources; the other carries money back out again, after it’s been stolen, for safekeeping in the west’s property markets and universities and political parties. If we wish to weaken him and his system of corrupt power, we must disrupt both pipelines. … A few names on sanctions lists and some loophole-ridden reforms to economic crime laws not backed by budgets to enforce them are close to meaningless while we still permit financial secrecy.

Tom Burgis: ‘To confront his kleptocracy, we must first cease our complicity in it The Guardian Sun 20 Mar 2022

“… while we still permit financial secrecy.” That is, intentionally obscuring information. It is well-known that Western law enforcement agencies dislike cryptocurrencies such as Monero, precisely because they guard financial privacy to a new and unprecedented degree. Monero’s market cap is around $5 Billion, today, and is relatively illiquid, and so does not currently enable Burgis’ Kleptopia. Yet it remains a threat.

At any rate, I think Burgis is right. Placing knowledge of large financial transactions into the public domain might very well dry up half of the world’s political problems, like cortisone for a a bad case of poison ivy.

Many Russian mothers of the soldiers refuse to believe the videos that show them captured sons telling to the camera that they were ordered to shoot at civilians, and that there are no Nazis in Ukraine contrary to what they have been told.

Russians mother can’t deal with this cognitive dissonance for in their minds their sons are still fighting Great Patriotic War on the side of the Soviet Union against the evil Germany/Ukraine.

That’s what propaganda has taught them to believe and they’d rather be convinced that the video with their sons is fake than make an effort and break the spell.

Their minds can’t accept the truth even when the truth comes directly from their sons’ mouths.

What are the 4 ways the war between Russia and Ukraine is likeliest to end in 2022?Misha Firer,

Common-sense thinking requires that new information be properly integrated into the bank of existing information in a coherent fashion, with new facts supporting old knowledge, and old knowledge validating new information. As one ages, common-sense knowledge becomes like arterial plaque: filling the voids of one’s head with the accumulated wisdom of a lifetime.

Factoids accumulate to form plaque. But how can one stay healthy, even as one grows old and wise?

Why am I writing all of this? Well, machine learning is easy. Converting machine learning into AGI is hard. In my experience, machine learning can be used with great success to create knowledge networks, at least at first. But then, after another step or two, all you find is a network of clogged arteries. Then what?

A Theory of Jerks

An acquaintance wrote:

don’t be an asshole
I am thinking through a “don’t be an asshole” theory of public behavior. It seems to me that problems of social interaction don’t arise from opinions (or guns or drugs or …) but from people having some idea of entitlement to expressing the asshole in their nature.

Maybe. I’m pretty sure that in 90% of the time, people who are behaving like assholes are not aware of it. And even if you draw it to their attention, they won’t grasp what it is that was ass-o-holic about what they said/did. (Common excuse: “I was just telling it like it is.” … and sometimes, they really are.)

Communications is subtle. There’s a big gap between the vague, general idea of what the speaker hoped to express, and the words that actually came out. Then there’s another huge gap between what the listener heard, and the “surface” or “most-likely” or “common-sense” interpretation of the words. And then a third gap: what feelings did the listener develop, upon processing what they thought they heard?

Many assholes have a strong idea of what the world should be like, but a poor conception of the listener. (And so they ride roughshod over the listeners emotions.)

Other kinds of (completely different) assholes have tremendous empathy for the listeners, and can provoke exactly the kinds of emotions they want in the listener. They play the listener’s heart-strings like a harp. But they ride rough-shod over common-sense and rationality.

Trump was this second kind. He’d blather on incoherently, spouting non-sense and non-sequiters, and his audiences loved it, because it was not about whether it “made sense”, but about how it made them feel (It made them feel good).

Most of us, as listeners, are on the spectrum. We want things to mostly make sense, and mostly make us feel good. The assholes lie at the two ends of the spectrum. Type (a) insist on perfect “rationality” above all else and screw you if you don’t agree. Type (b) need total and complete emotional harmony, and you are a horrible person that I will never talk to again if you continue to hurt my feelings.

A lot of software programmers, and the high-IQ conservatives are type (a) and a lot of lefties, esp. the save-the-whales kind are type (b).

A lot of right-wing extremists have managed to combine (a) and (b) into one uncompromising package.

The Meaning Crisis

tl;dr:  Meaning is provided by ancient and deep neural circuitry. The “crisis of meaning” (for humans) is the inability to find ‘meaning’ for issues deemed ‘important’ by other neural circuits. Anxiety and psychological distress arises from a feedback loop searching for meaning. This feedback loop manifests both as “Culture War” and is of concern for the design of AGI, StrongAI.

The “Meaning Crisis” is popularly discussed and debated these days: John Vervaeke has a fifty episode youtube series on it, David Chapman has a multi-volume website-book on it, and there is an endless collection of posts on the Culture War.  Meaning is important to understand, as it has implications for transhumanism and AGI/StrongAI.

Why do people crave meaning?

This is a rhetorical  question, and I propose an ‘operationalist’ answer below.   Now, meaning is important for the ‘deep’ questions in life, but it’s easier to get started with a unfraught examples. Consider the sentence:  “If the engine stalls under load, that means the power is inadequate“. This is of the form “If situation X, that means fact Y is true“. Another form is “If you perform X, that means Y will happen.

Basic Darwinian survival “means” that humans, indeed, all animals must have an ability to asses this kind of operational, quotidian concept of meaning. Not all meaning is created equal: “If I stop to smell this rose, I will feel good” has a meaning subordinate to “If I stop to smell this rose, the bear who is chasing me will eat me.” Thus, one concludes that (almost) all animals have priority-ranking systems.

… Although, apparently, jellyfish don’t. They give equal weighting to eating, and fleeing predators. The evolution of bilateria provides the needed neural circuitry to make up one’s mind about this. (Reference: Tony J. Prescott “Forced moves or good tricks in design space? Landmarks in the evolution of neural mechanisms for action selection” (2007))

People Crave Meaning because Biology

Which brings me to the operationalist answer to the question posed above. David Chapman presents multiple scenarios involving a crisis of meaning; one example involves an extramarital affair, and what it “means”.  Now, in the case of having an affair, one is engaging some very deep and old parts of the brain, dealing with mate selection and procreation. It’s vital to get these right, as the survival of the offspring is in question. Or rather, get this wrong, the offspring don’t survive, and such faulty behavior is bred out of the population. Animals know how to mate.

Thus, having an affair triggers sustained activity in large parts of the brain. Basically, the brain is saying “Pay attention! What is happening right now is really important! Deal with it!” The various neural layers dealing with ‘quotidian’ meaning fall into action, trying to extract if-then relationships between perceptions and actions. The claim here is that the meaning crisis arises from the fact that solving this particular problem is hard … it’s effectively unsolvable. Faced with unsolved problems, we have brain circuitry that says “think harder, think more, your survival depends on this”, and we’re now stuck in a feedback loop of pondering the answer to an unsolvable problem.

Of course, this feels like a crisis: when the bear is chasing us, and we cannot find an answer to the problem, a feeling of crisis ensues, expressed variously as helplessness, hopelessness, fear, determination, sheer will-power, grasping-at-straws, try-anything, pray to God, …

I propose that the ‘meaning’ of the affair “feels” just like being chased by that bear, but in slow motion. By “feels”, I mean, the brain senses that this is important, the brain is demanding an answer to the predicament, an answer is not being found, crisis and loss of ‘meaning’ ensues. This is a deeply ingrained feedback loop in the brain, and once switched on, it is not easily switched off.

Indeed, resolutions to the meaning crisis seem to often involve demoting the importance of the original problem: the crisis often resolves when one convinces oneself that “Ah, whatever, it didn’t really matter anyway. No need to feel miserable over it.

The converse is also true: awakened at night, you find some issue has gained immense importance, and you are tortured, anxious, unable to sleep, as the issue has no solution. Usually the torment dissolves by the next day, but can linger on for days or months. The claim is that these are old, deep neural circuits trying to do the job they’ve evolve to do, and are making your life a miniature hell as a side-effect.

Wrong Answers

David Chapman spills oceans of virtual ink on nihilism and eternalism, and how these are faulty foundations on which to build meaning. What he does not seem to say is that the reason that nihilism and eternalism are popular is that they provide the missing answer to the question driving the crisis: with the answer in hand, the feedback loop is halted, one is relieved and can get back to ordinary life. (The bear is no longer chasing you; nihilism/eternalism resolved the crisis.) People are terrible in ‘rational’ reasoning, and don’t notice the inadequacy of nihilism/eternalism. And it mostly doesn’t matter: it not only resolves this crisis, its a cure-all for (almost) all crises: its a rock you can depend on (as the Christians call it).

To be clear, the ‘feedback loop’ I am describing here is meant to be taken as a literal feedback loop: it really is one set of neural circuitry raising an alarm: ‘this is important, find out what it means‘, and a different set of circuitry saying ‘I don’t know, I can’t figure it out‘, feeding into those reinforcement circuits saying ‘well try again, this is important, your survival as a species depends on finding the answer.‘ Failure is not an option: you are flooded with hormones, and thoughts, each elevating the problem to high importance, and driving senses of high anxiety and/or loss-of-meaning and all the other stereotypical psychological responses.

So, it seems for me, that the crisis of meaning is the conflation of multiple forces:

  • (a) Old, deep brain structures that make you unhappy when you cannot find resolutions to ‘important’ problems, where ‘importance’ is ranked by other old, deep brain structures.
  • (b) Sloppy reasoning and intellectual laziness allows nihilism/eternalism to provide adequate answers, adequate guidance to shut down the feedback loops that are making you miserable.
  • (c) This shutdown and relief from pain are so significant that the ‘answer’ of nihilism/eternalism is marked as ‘really good stuff’, and ‘generically useful’ and ‘apply whenever you feel bad’: these are perceived as cure-alls.
  • (d) Whenever there is some obvious social or political problem, one’s favorite personal cure-all is trotted out as the right solution for everyone else, too.

This last point results in Culture Wars, especially evident when religion is providing one of the bedrocks of meaning for one group, even as another group rejects religion without proposing a clearly defined, simple foundation on which to build meaning. Or, more directly: failing to provide an easy, simple cure-all for resolving all emotional and psychological crises that one might face.  It’s no wonder that those of faith reject those without it: faith is the tool for resolving the crisis. If you are not clear on this, you should ponder the lyrics to Amazing Grace.

Neural Feedback Loops

l’ve posted before on ‘Endorphin Supply Chains‘, which is about the tobacco (nicotine) feedback loop that couples neural circuits in your brain to a hundred-billion-dollar capitalist industry. It is a huge feedback loop, of tremendous importance, over which we, as individuals, have precious-little control. We live in a capitalist society: it has organizational structures that are bigger than us.

Likewise, in this post: there is another deep feedback loop.  It is the one that searches for meaning. It is primitive, because finding meaning correctly allows a species to survive and thrive. Yet we, as individuals, cannot find meaning for everything. If those those things we cannot solve are also given high importance by other neural circuits, a sensation of unease takes over. Allowed to grow (fester?), it can mature into full-blown crisis. The feedback loop does not stop at the individual level; it is coupled to the global brain, and has resulted in two World Wars, as Chapman so marvelously explains.

The Risk to StrongAI

The relevance for AGI, for Strong AI is that we must be cautious with our feedback loops, with the systems that provide motivation for behavior.  They can very easily amplify negative, unwanted actions.  Wrong thinking, as David Hume noted in the 18th century, results in wrong action.

Feedback loops establish the basins of attraction in dynamical systems. In this (mathematical) sense, the activity of  any AGI/StrongAI is a dynamical system. (Well, anything and everything in our Universe is a dynamical system.) In the case of software having the ability to act in the physical world, the “laws” of unintended consequences apply.  A system smart enough to “think” but not smart enough to escape its own destructive feedback loops risks not only killing itself, but taking down Humanity with it.

Social Network Crisis Reading List

This is a list of interesting reads dealing with network theory, social media, and political crises. Stuff like algorithmic propaganda, memetics and the epidemiology of memes, as inflamed by social media. This provides footnotes and references for claims that I make in the other blog posts.

This is a complement to the Capitalism Reading List (political/economic network theory – how epidemics of memes affect the politics and economics) and the Neurochemistry and Bio Networks Reading List (network theory at the cellular and individual brain level.  Nicotine and caffeine are neurotransmitters powered by capitalism). There’s some duplication across these lists.

  • Why some biologists and ecologists think social media is a risk to humanity – Vox – 30 June 2021. An interview with two of the 17 authors sounding an alarm about the memetic crisis in social media. The snark in me wants to say “better late than never”, but the academic weight of the disciplines of epidemiology and ecology are important. In particular ecology: this is a widely (and wildly) misunderstood science. People think its about birds and bugs in forests. It’s not: its about complex systems and complex networks, with stuff like differential equations thrown in for good measure. The actual paper is here, in PNAS: Stewardship of global collective behavior, Joseph B. Bak-Coleman et al. PNAS July 6, 2021 118 (27) e2025764118;
  • Project Cassandra – the Guardian – 26 June 2021. This describes a multi-year project, the goal of which is to predict future wars based on literary analysis. How? Quote: “When Azerbaijan gave anti-Armenian books to Georgian libraries, the project predicted conflict. A year later, war broke out.” This might sound like old-fashioned  propaganda. What makes it literary is that the authors are not explicitly propagandists; rather, they are describing and mirror society in novels and plays. Authors who are showered with literary awards and prizes. And who touch a nerve: books that are banned, authors who are jailed or forced to flee their country. For, literature holds a mirror up to society. Lead researchers: Jürgen Wertheimer, Isabelle Holz, Florian Rogge (literary critique) Julian Schlicht (number crunching).
  • Aljosa Puzar,  (2021). Towards a critical cultural epidemiology. Academia Letters, Article 570.  This article is ostensibly about the sociology of pandemics. Careful reading shows that it is a far more careful study of the topics I try to broach here. For example: the Great Toilet Paper Shortage of 2020 is presented as a coupling of primal fear, neurochemically and socially expressed, to neoliberal competitiveness, capitalist supply chains and “neuro-politics”.
  • The Consilience Project – Making better sense of the world. – by Daniel Schmachtenberger – 27 February 2021. There’s been a “War on Sensemaking” – we’ve be subjected to a disinformation campaign coming from all quarters. This project notes that “The world faces unprecedented catastrophic risks across the spectrum of finance, governments, ecological health, and global stability. To respond appropriately, leaders and citizens need increased capacities to make sense of what is happening in the world and to communicate and coordinate effectively.” and that “The goal [of this project] is to restore the health of our information commons by helping educate people on how to improve their information processing so they can better detect media bias and disinformation while becoming more capable sense-makers and citizens.” My knee-jerk reaction is that this is a bit naive, given that the majority of the GOP is delusional and is politically doubling down on the insanity.  But hey, someones got to do it, so it is a commendable effort.
  • Explaining the Trump Movement Through the Lens of the Social Organism – by Fergus Thomas – Irban Group – 19 Dec 2016.
    Marvelous article reviewing the basics of memes and network theory. Recall how I talked about the collapse of sand piles, cellular death, and supply-chain contagion as explainable by network theory? Well, the same applies to the spread of memes on social media, and the incredibly invasive power of memes into the thought-patterns of humans.  The article on “Trump Psychosis” makes it sound like its all about psychology – what individuals think (in a group setting). The article on “Russian Disinformacya” makes it sound like its just social, informational manipulation.  This article  bridges over into the network theory — its the network that matters! The things that are bouncing around between brains — the messages and the memes — are one thing, but the network topology — which brains are talking to which ones, is really the key.
  • Part 1: The Next Great Online Community – Jamil Abreu – Feb 7, 2020. In case you think that “social media” means facebook, youtube, twitter, well … uh no. There’s a bit more to it than that, as there is both a history of it, and a theory of software: when you design software to run a social media site, you are also (unintentionally?) designing the rules by which that community plays. And these rules have all sorts of unforeseen consequences. Building online communities is not a simple game.
  • The Wisdom and/or Madness of Crowds – Nicky Case. So in case you’re not following what I mean by “network theory”, and how that applies to politics, economics and psychology, the above is a simple, fun game that explores the mathematics (gasp!) of network theory. Its a game. you can play it in your web browser. Its short – maybe 20 minutes – its fun! Do it now!
  • Meaningness – David Chapman -2010-2021. This is a vast collection of thoughts on social politics, viewed from the lens of “meaning” – as in, deep, personal inner thoughts, the things one thinks when pursuing the “meaning of life”. The core thesis is that we’ve built society and culture on shared belief systems of what is right, and what is wrong. Shared beliefs about what makes nations great, and what makes life worth living: the stuff of political philosophy as well as religious feelings (beliefs). As it happens, the foundations appear to have been infirm. It’s best to start reading these tracts in the middle. My favorite is a portrait of the early 20th century. I like it because I understand this period well. I spent years studying Art History in school. Art History? Whaat? Hey, check it:   “Systems of meaning all in flames” explains. The bit about the culture wars of the 1980’s Reagan era is also very worthwhile.

Is the Texas PUC Incompetent?

Tell me I’m wrong. Tell me I’m misreading. That I’ve misunderstood what was being said. That I misunderstood the intent.  Because surely, surely, the Texas Public Utilities Commission cannot possibly be so stunningly, overwhelmingly incompetent, as they were during this week’s electricity crisis.  Perhaps the biggest electricity crisis Texas has ever had.

Because I read the two orders that the PUC wrote to ERCOT (the Electric Reliability Council of Texas) at the depths of the crisis.  No matter how many times I read and re-read the legalese, the first order seems to say that the PUC was shocked, absolutely shocked, that the electric generators were not gouging the customers in the midst of the crisis. And they ordered ERCOT to retroactively gouge on their behalf.  The second order, a day later, said “Remember that first order? Never mind.” It was signed by only some of the commissioners. Apparently, Commissioner Shelly Botkin still thinks that her job is to tell the utilities to gouge the customer.

But maybe I am misreading the text. It is, after all, dense legalese 😀. Let me quote; you decide:

ERCOT has informed the Commission that energy prices across the system are clearing at less than $9,000, which is the current system-wide offer cap pursuant to 16 TAC §25.505(g)(6)(B). At various times today, energy prices across the system have been as low as approximately $1,200. The Commission believes this outcome is inconsistent with the fundamental design of the ERCOT market. Energy prices should reflect scarcity of the supply. If customer load is being shed, scarcity is at its maximum, and the market price for the energy needed to serve that load should also be at its highest.

In other words, they are shocked, completely shocked, that the free market is not ripping off customers.  To “fix” this situation, here is what they recommend:

Pursuant to this authority, the Commission determines that adjustments are needed to ERCOT prices to ensure they accurately reflect the scarcity conditions in the market. Accordingly, the Commission directs ERCOT to ensure that firm load that is being shed in EEA3 is accounted for in ERCOT’s scarcity pricing signals. The Commission further directs ERCOT to correct any past prices such that firm load that is being shed in EEA3 is accounted for in ERCOT’s scarcity pricing signals.

I read these paragraphs five or six times to make sure I understood them.  They seem to say: “Free markets aren’t working. We order you to jack up the price to maximum.”  Lets critique this.

  • Free markets aren’t working. Many people agree with this. We live in the era of “late stage capitalism” where there is abundant evidence of the damage and outright evil caused by free markets.
  • However, the GOP have been cheer-leaders for free markets — that is what they shout out to the voters. Behind closed doors, not so much: there is abundant evidence that the GOP leadership gerry-rig deals that are anti-competitive. They only say they want “free markets”; in practice, they interfere whenever they can.
  • The PUC order is an example: ERCOT provides a more-or-less free market for electricity. It seems to mostly work. But the PUC decided that the free market is not good enough. They decided to order the utilities to raise their prices. To the maximum (and then some, in the second part of the order.)
  • The maximum is ninety times the normal rate. Ninety. 90. Normally, you pay about 10 cents per kWh. Sometimes less, sometimes more. (Some parts of the US pay 20 cents, sometimes). The PUC order directs that prices be raised to $9 per kWh. Some perspective: when you sit in front to the computer all day, you might use up 10 cents of electricity. You might think twice if this cost you $9. Also, you might want to turn off the light in the kitchen.
  • The PUC issued this order in the middle of the greatest crisis the Texas electrical grid has ever had. A catastrophe is unfolding. A total electrical blackout must be avoided. Every elected official, every appointed administrator should be working 16 hours a day to avert disaster. Electric linemen are thawing frozen equipment. Their bosses are manning the phones, coordinating relief efforts. Ted Cruz is flying to Cancun to sit in the warm sun. And the PUC figured out how to waste everyone’s time while also showing they are utterly ignorant of grade-school economics.
  • Because this is rank ignorance. Yes, economics is complicated. Yes, the electrical grid is complicated. Yes, the order clearing system is complicated. Penn State offers online classes on these topics. This is not that. This is an order to gouge the victim, because they think that gouging the victim is the normal thing, right thing to do. Something that free markets should be doing. Like, when someone is choking on a piece of food, merket regulation means that each Heimlich maneuver should cost  $9,000. Or whatever the mandated maximum for government-regulated Heimlich maneuvers.

Obviously, this is a gross misunderstanding of capitalism, of economics, of the way the world works, and of basic human decency. Apparently, the PUC realized this, because a day later, they wrote this:

However, the Commission determines that its directive to ERCOT in its order dated February 15 to also correct any past prices to account for load shed in EEA3 should be and is hereby rescinded and directs ERCOT to not correct any such past practices.

In other words, “never mind“. This time, only two of the three commissioners signed the order.  The third, Commissioner Shelly Botkin, apparently is still confused.

In the wake of the gross incompetence demonstrated by the Texas Public Utilities Commission during the height of a historic utilities crisis, I hereby call for the commissioners to resign. If they do not have the common decency to recognize their mistakes, and remove themselves from a position of authority, the Governor of Texas should remove them.  This is the purest form of bad governance that I have directly witnessed in the last few … umm, weeks.

The commissioners are:

Deann T. Walker,

Arthur C. D’Andrea

Shelly Botkin





I’m old enough to know what happened in the Soviet Union. These three people are the American version of Soviet apparatchiks. America faces a disaster as long as people like this are in power.  This is the kind of leadership that destroys countries.

To be clear, I think ERCOT did a commendable job. Gov. Abbott has got things exactly wrong. He needs to think very hard about why he appointed apparatchiks to the PUC.

If you think that I have made errors of fact, errors of exaggeration, distortion or sarcasm, let me know. Convince me that I’m wrong. Show me that these people are not the incompetent buffoons they appear to be.


March 2021: All three are gone; two resigned on their own, and Deann was forced to resign.

News stories

  • How an obscure paperwork foulup left much of Texas in the cold during blackouts. (Fort Worth Star-Telegram) Bud Kennedy March 20, 2021. Basically, they shut off the electric power to the gas compressors that supply gas to electric power plants. Dohh! Choice quote: “Most West Texas gas compressor stations and pipelines were never listed as “critical” priority customers to keep power, the same way as hospitals or 911 centers. In a state with more than 1,000 public employees overseeing the energy industry, not a single one was assigned to double-check whether gas plants filled out their electricity paperwork. Absolutely nobody in Texas was checking to make sure energy facilities kept power.
  • Minnesotans furious that they have to pay for Texas’ deep-freeze problems – Natural gas prices surged across the country during Texas’ February freeze.
    Tim De Chant Apr 23, 2021 – Minnesotans get to pay $500M to a Texas gas company. Everyone in the state gets a surprise $400 gas bill to pay for the Texas PUC screw-ups. Choice quote: “The ineptness and disregard for common-sense utility regulation in Texas makes my blood boil and keeps me up at night,” Katie Sieben, chairwoman of the Minnesota Public Utility Commission, told The Washington Post. “It is maddening and outrageous and completely inexcusable that Texas’s lack of sound utility regulation is having this impact on the rest of the country.

Harnessing creative talent

Well, actually, paying for creative talent. The old model was capitalism – get a job, get paid for it. The newer model is Open Source – Wikipedia, Linux – volunteer, do it for free. Obviously(?) this is not sustainable; in the end, you have to have money to buy groceries. Where does this money come from? No one has launched a startup business based on editing Wikipedia. To support their habit of editing Wikipedia.

Open Source, Wikipedia, Linux aren’t just habits, they can border on obsessions. Every amateur psychologist knows that engineers are powered by autistic behaviors.  Small-market capitalism (medieval-era market capitalism) worked great, if you were obsessive about candle-making and shoeing horses. Medieval lords and ladies, the landed gentry, were for the most part economic leeches. Maybe providing some military security. Maybe a judge and a courtroom. Only a tiny percentage used their wealth to pursue higher studies – divinity, law, and what we now call science. Market capitalism never paid for science, except for a brief golden era of 1945-1975 when corporations ran R&D departments. Science is partly funded by Universities, but mostly by government grants. Anyway, writing grant proposals is an odiously unpleasant task.

And no one has written an NSF grant asking for money so that they could edit Wikipedia.  Or write open-source software (well, outside of some narrow scientific usage). Yet these activities are recognizably beneficial to society.

This essay is an exercise in hand-wringing:

  • How do we fund socially beneficial behavior in an increasingly knuckle-headed political environment?
  • How do we reach political agreement on what constitutes socially beneficial behavior?

Well, I guess I could sign up for an account on Patreon. Do you think my fans will pay for exclusive early access to physics research results? Or maybe the exclusive privilege to read blog entries like this? What makes you think I have fans? Do you think that sounds far-fetched? Thought so.

Patreon requires you to create content that is mind-blowing enough that you can nurture actual fans. Wikipedia and Open Source don’t fit into that category.


Pull on this thread, and you get nowhere at all. You hit the wall. Capitalism as-we-know-it. Is there an alternative?  Yes there is! The first hints of an outline is in my next (earlier) essay: Post-Capitalist Political Economy.

Post-Capitalist Political Economy

With regards to distributed ledgers and alternative credit/IOU systems, Mixmix wrote:

Might be interesting to look into mana – in te ao Māori (the Māori world) one aspect of mana is like respect / honour, and decisions are made based on whether it increases the mana of all involved. Really interested because it’s not zero-sum at all. The best path is the one where everyone leaves “richer”.

Mana might work when communities are small, e.g. smaller than the Dunbar number (about 200 people). After that, you have to distinguish social policy from monetary policy, and realize that politics plays a fundamental, key role.

Consider the problem of “taking care of grandma” (or raising children, or looking after anyone who is sick … the traditional feminine roles — I have a terrible urge to talk about why these roles are filled by women, not men, but I think you probably already know why…) In a community of 200 people or smaller, everyone can agree that “taking care of grandma” is a worthy activity. The “cost” of taking care of grandma is 100% negative on a ledger system: everyone else must provide food, shelter to her. There is only one positive externality: grandma is alive, and you can visit her. But how are you going to put “grandma is still alive” on your distributed ledger? How do you say “one living grandma is worth 10,000 IOU’s”?

This is where social policy comes in: the village communally decides that taking care of grandma is worthy (unless everyone hates her…) If you have more than 200 people in the village, say you have 20K people, then how do you allocate “money” (IOU’s, whatever) to “taking care of grandma”? The usual process is to propose a “social safety net”, have everyone vote on it, and then levy a tax on all transactions, and earmark that tax-money for taking care of grandma. Walk down this path for a while, and you end up with capitalism-more-or-less-as-we-know-it.

Distributed ledgers are technologically interesting, but are useless/unfair, unless you couple them to a social policy which kind-of means you have to couple them to a voting system. This is how you solve @Dominic’s bootstrap problem.

We need a distributed ledger with an integrated social policy platform. So that we can allocate funds to taking care of someone-else’s grandma, while also giving the boot to free-riders and general assholes. (Until such a day when the free-riders and assholes become homeless w/mental stability issues camping in the neighborhood park. Then things are hard, again.) So what we’re really looking to do is to erect an alternative, parallel political economy that can compete with capitalism.

Coupling a distributed ledger to a liquid-democracy voting system seems like the way to go. Maybe something with smart contracts ala ERC20 to store the social policy.

Wow. for the first time, ever, it seems like maybe there is a path forwards. Thank you @MixMix; I’ve been stumped by this problem for a long long time.

(To be clear: I’m proposing a technology infrastructure that can issue not only altcoins, but can also determine social policy. It is up to individual communities to determine what their social policy is, and to federate as they desire.)


The Global Brain, Redux

Do you mind if I lecture a bit? I’ll try to keep this short. Search for any terms you are not familiar with. (Cross-post from #securescuttlebutt.)

  • Bacteria use small peptides to “talk to each other”. Search “quorum sensing in bacteria“. Tree roots and fungi do this too.
  • Slime mold can solve the two-armed bandit problem: explore vs. exploit. Microscope slide, a drop of food on the left, on the right. Which will they choose? The bigger blob of food? The closer blob of food? The algo they use is a good one, but not the best-possible one; the best possible one requires storing some binary bits.
  • Communication by peptides is limited by speed of chemical diffusion. There’s also huge cross-talk.
  • Neurons are star-gate, star-trek teleporters for small peptides, now called “neurotransmitters”. A neurotransmitter walks into a neuron here, and a millisecond later, pops out three feet away. Speed is a million times faster than diffusion, and no crosstalk! The algorithm space expands hugely!
  • Jellyfish have neurons but don’t have brains. They can eat, and they can flee predators. Except they sometimes do both at the same time, because they have no supervisor to suppress eating while they’re fleeing. Bilatarians do have this. The first brains.

    Jellyfish have a circular ring of neurons running along the perimeter, able to detect and paralyze food as it swims by.
  • Lets skip ahead. Animals invent language. I stub my toe. Some peptides get released, they go up my spine to my brain, I say “ouch”, and the neurons in your brain know that my toe hurts! This is magic! Forget star-trek teleporters, neurons are now communicating with freakin sound waves! That’s just bat-shit insane!
  • In stone-age villages, grandmas pass cultural heritage to grand-children by oral tradition. Culture has a longer lifespan than human lifespan. It also spreads, one to many. Culture is a stable structure that lives on top of neurons, on top of brains, and continues to live and thrive even as old brains die and new brains are born and added to the network. Lo-fidelity maybe, culture mutates over the generations, but … semi-stable.
  • Phonecians invent the alphabet. Just memorize these 26 symbols, and the sounds that match with them, and you can read-write. Joe Blow, anyone with an IQ over 120 can do this, without training, without a classroom, without an education. This lowers the mutation rate of oral tradition, and supplies a stable long-term storage. Much more stable than oral tradition. Also, fraud-tolerant. You’ve got it in writing.
  • This means that thoughts and ideas can survive on pieces of paper. Or clay tablets. whatever. For decades, centuries, millenia.
  • Like, who is Aristotle? Sure, he’s partly dead clay, but he lives on the be brains of thousands of professors and millions of students. Lives on to this very day. Our neurons are connected to Aristotle’s neurons. Aristotle’s neurons are still influencing our behavior. The network is not sound-waves, the network is written material. This is a store-and-forward network that has survived millennia. Aristotle is a store-and-forward network.
  • The printing press / radio / television / mass media changes everything. Network connectivity between neurons changes to a hub-n-spoke model, broadcasters at the hub. About 1% to 5% of neurons on this planet broadcast to the other 95%. The 1% here are the newspaper editors, journalists, TV stars, movie producers, sports stars, mayors, CEO’s. These people are smart: say, IQ of 120 or better. Because mostly book publishers won’t publish a book written by a moron. Think of this as neuro-science: there are brains connected by various channels, transmitting assorted information. Information that is mostly stable, mostly verifiable, mostly true. Long-lived. Broadcast from the governing few to the masses.
  • This means that the total compute power of western civ has ramped up by million-fold, compared to Medieval times. So, in 1300 AD, you had a handful of universities with brilliant scholars: Peter Abelard, Thomas Aquinas. In the Middle ages, you had maybe thousands of scholars at any given time. Maybe 10K. Now you’ve got many tens of millions, not just universities, not just corporate R&D departments, but also sports-coaches figuring out how to fine-tune athletic performance, city councils figuring out how to run a city. There were lots of brains in the Middle Ages, too, but those brains did not have a network to connect them. Mass media and publishing provided the network. We’ve got many millions of brains figuring shit out, but now they are in sync, because they can communicate better. We’ve got the printing press! Raw compute power via human brains is explosively, exponentially larger than the Middle Ages.
  • Twenty years ago, we invent social media. This completely alters the network connectivity between brains. Suddenly, brains that had only a small, limited reach (the drunken uncle at thanksgiving, the sulky teenager in the high-school lounge) now has a freakin huge audience for their nut-ball flat-earth theories. Or whatever else they care to think about. And so memes, like germs and viruses and #covid spread without a mask, unfiltered, infecting other brains. I mean, mass media used to filter out the shit, because, again, newspaper editors filter out the total garbage. But #youtube and #facebook mostly does not have a filter, so anything goes.
  • Spend some quality time studying “phase transitions” and “second order phase transitions”. This is hard science- physics, the physics of ice melting, the physics of water boiling. Phase transitions change freakin everything. We are in the middle of a phase transition in how human brains are connected. It is literally that big. If you are not seeing this, you are truly missing the big picture.
  • And don’t forget the neurotransmitters. #nicotine #addiction is a real thing, and there’s a cascade of half-a-dozen protein expression and DNA methylation processes involved in this. These same (well, similar) circuits keep you addicted to #facebook, and to #gaming and to #QAnon. These circuits in your brain are now wired to similar circuits in billions of other brains, pumping endorphins with each “like”, with each new subscriber. This is the system we are dealing with.
  • Yes, of course, #capitalism and #politics are layers built on top of this system, but they date back to the invention of the printing press. The invention of social media, the phase transition …. changes everything. We are all part of the Borg now, and we love it. Do not be fooled, there is no going back. Whether or not there will be global systemic collapse, who knows, but seriously, we’ve increased global human brain compute power by 1000x maybe more, over what we had in the 1980’s. What is this brain thinking? Well, surf #youtube for a while to find out. Is it healthy? Watch The Social Dilemma to find out. But do not underestimate what happens during a phase transition.
  • Seriously: study network science. The #math, the #physics, the #neurobiology, the #psychology, the algorithmic #propaganda. This is wayyyy larger than what almost anyone suspects. This is fundamental on a planetary scale.