From broken hearts to Markov chains to Gaussian copulas to quant finance

From the Emerald Tablet:

As above so below;
as below so above;
as within so without;
as without so within

This probably sums part of the faith of most theoretical physicists and mathematicians: the belief in correspondence in the cosmos — that diverse phenomena are subtly interrelated. And for this, mathematics is ideal as it abstracts certain features from concrete phenomena that allow comparisons and correspondences to be drawn. Thus, to give one of the earliest examples, both the falling of an apple and the trajectory of the moon can be shown to be equivalent (with a bit of vector analysis) and thus it can be shown that one force is responsible for both — gravity. This was quite something in Newton’s time as it was thought that terrestrial phenomena were governed by laws differing from heavenly phenomena.

More recently, it’s been noticed that people often die soon after their spouse’s death — a phenomenon known as “broken-heart syndrome.” There’s an interesting piece in the FT on this:

In a March 2008 study, Jaap Spreeuw and Xu Wang of the Cass Business School observed that in the year following a loved one’s death, women were more than twice as likely to die than normal, and men more than six times as likely. “

The article goes on to explain how ideas from Markov chains (a part of a more general framework of ideas known as “stochastic processes”) can be used to model the broken-heart syndrome. Markov chains are fun, and a fair amount of the theory can be explained even to a well-prepared high school senior who knows how to play with matrices (indeed there are college texts on linear algebra that incorporate some elementary Markov chain theory as a supplementary topic). There’s a simple example of a Markov chain used to find the “steady state vector” with respect to a very simple weather model here. Markov chains allow all sorts of diverse phenomena to be modeled — economic, physical, meteorological. So the fact that Markov chain theory can be used to model broken hearts should not be cause for amazement. Though perhaps we would like to think that affairs of the heart are not really amenable to quantitative treatment. But perhaps they are statistically, “in the large.”

The FT piece goes on to discuss the math being used to model default probabilities:

And if he could apply the broken hearts maths to broken companies, he’d have a way of mathematically modelling the effect that one company’s default would have on the chance of default for others.

The problem with relying on Markov chains was that they painted a far too mechanical, physical – atomic, even – picture of the human lifespan. Li reasoned that with a copula that showed a probable distribution of outcomes, a more accurate, encompassing picture of the broken heart or, for that matter, the broken company, could be devised.

But of course, with 20/20 hindsight, we know that this modeling didn’t work (in fact Nassim Taleb had been telling us this all along). Economic and financial phenomena are not amenable to modeling — or at least not with the math toolkit we presently have. Attempts to do so (other than the Gaussian copula, the most famous example is the Black-Scholes PDE) have exploded in our faces, discrediting (rightfully so) the whole fledgling field of quant finance. We have to be careful in what correspondences we draw in the cosmos. The heat conduction PDE cannot be used — as the Black-Scholes equation — to successfully model option prices. As Mackenzie points out in his book, An Engine, Not a Camera, the primary purpose of the equations, of the quant approach, was not to model, but rather to create and shape the markets the quant approach was purportedly modeling.

The difference between physics and economics (and the limits of mathematics)

A mildly interesting essay in today’s FT:

That people respond rationally to incentives, and that market prices incorporate information about the world, are not terrible assumptions. But they are not universal truths either. Much of what creates profit opportunities and causes instability in the global economy results from the failure of these assumptions. Herd behaviour, asset mispricing and grossly imperfect information have led us to where we are today.

There is not, and never will be, an economic theory of everything. Physics may, or may not, be different. But the knowledge we can hope to have in economics is piecemeal and provisional, and different theories will illuminate different but particular situations. We should observe empirical regularities and – as in other applied subjects such as medicine and engineering – we will often find pragmatic solutions that work even though our understanding of why they work is incomplete.

Max Planck, the physicist, said he had eschewed economics because it was too difficult. Planck, Keynes observed, could have mastered the corpus of mathematical economics in a few days – it might now have taken him a few weeks. Keynes went on to explain that economic understanding required an amalgam of logic and intuition and a wide knowledge of facts, most of which are not precise: “a requirement overwhelmingly difficult for those whose gift mainly consists in the power to imagine and pursue to their furthest points the implications and prior conditions of comparatively simple facts which are known with a high degree of precision”. On this, as on much else, Keynes was right.

The question is: To what extent do mathematical models shed insight on economic phenomenon? My sober but humble opinion is: just about none. The problem is that economists envy physicists their prestige, their accuracy, and their esoteric mathematics. And in attempting to emulate physicists with their use of mathematics they make complate jackasses of themselves. Economists have been known to use the terminology (and perhaps the results) of topology. They are complete lunatics and should be chucked in some loony bin pronto.

A more general question might be posed: To what extent can even physical phenomena be mathematically modeled? When and where do we start to suffer from diminishing returns? When and where do we start creating models — like string theory — which are “not even wrong” (Peter Woit’s words)? Does the wholesale use of cohomology (for instance) shed any physical understanding? Where does the program of mathematisation start breaking down? Has it run its course? ODEs and PDEs are useful tools. So is Riemannian geometry (for GR). So is Hilbert space theory (for QM). But maybe it doesn’t go much further than this and the increasingly arcane instruments forged by mathematicians serve no useful purpose except aesthetic satisfaction.

There are more profound questions. Does mathematics “work” at all? Or is it a case of putting on rose-colored glasses and then exclaiming that the world looks red? In other words, the mathematical models may be self-referential in the subtle sense that they color how we look at the phenomena, how we make measurements, and what we look for. They are tautologically designed to work — and we fool ourselves into thinking some new insight has been shed.

Another aspect of modeling to think about is that it almost never happens that a “successful” model emerges ex nihilo. In practice, theory and experimental results converge slowly, with each helping the other. Again, it’s not the case that some ex nihilo mathematical model has shed miraculous insight on some are of physics.

Back in 1981, I read an article by R.W. Hamming, titled “The Unreasonable Effectiveness of Mathematics,” published in the February 1980 edition of the American Mathematical Monthly. I’ll come back to this post and comment on it, given time and inclination.

The Mathenauts

An interesting question for every mathematician is the ontological status of the various mathematical entities that he is obsessed with, that he plays with in his mind, that are the focus of his being. Most mathematicians, if pressed, will admit to being Platonists: their belief — if not conviction — that the mathematical entities have an objective existence of their own. Somewhere. And that this existence is not inferior to mundane day-to-day reality, which is often an unreal shadow world to many research mathematicians. Thus, a Riemann surface, a moduli space, a tensor category, or the monster group are just as real — if not considerably more so — than some material object or sentient being.

In 1964, Norman Kagan attempted to give flesh to these musings with an SF short story, “The Mathenauts.” It is unfortunately not available online. I read it about twenty years ago and it had quite an impact on me at the time. A mathematician looks at mathematical objects on their own terms, not looking for some raison d’etre in terms of applications; of course, if the ideas are any good and of enduring worth, they arise by abstracting some element or aspect of the “real world” — but once this is done, they develop a life and dynamic of their own, deriving their sustenance from the axioms that grant them immortal life. This is something physicists and engineers never understand, and their approach to mathematics is more mercenary, more slip-shod, more expedient. For them clarity, clear definitions, proved relationships, the architecture of a theory, the limits to what can (or cannot) be asserted are of no fundamental importance. They will not examine the mathematical universe and the objects therein on their own terms, as possessing an independent life of their own, and deserving of minute and careful study. They are Aristotelians, not Platonists.

The glass bead game

This is a provisional outline for a post I will flesh out in the fullness of time. I read Hesse’s “The Glass Bead Game” over thirty years ago and it has been one of my favorite books. I’ve noticed that it’s been also a favorite of quite a few mathematicians who recognise that the fictional ultra-aesthetic game corresponds in some fashion to mathematics, which is also aesthetic and driven in some sense by a romantic, aesthetic, and spiritual involvement with the abstract. Or to put it in a way that might make many mathematicians squirm with embarrassment, inchoate spiritual hankerings find an outlet in mathematics — mathematicians tacitly subscribe to the philosophy that the warp and woof of the cosmos is inherently mathematical, that the deity is fundamentally a mathematician. In this sense mathematics evokes a dedication and fervor that, say, chemistry or geography do not. But I get ahead of myself and the feelings I’ve described above are but a minor and often unconscious part of the outlook of a working mathematician.

I’ve noticed the expression, “glass bead game,” being employed in a couple of places. One was where someone writing about Hegel called the Hegelian structure of knowledge an elaborate glass bead game. Another was where a mathematician writing a high-level expository survey of the math behind the solution of Fermat’s Last Theorem also described Andrew Wiles’ work as a glass bead game. This fictional game is a symbolic intellectual game that unites different areas of human intellectual endeavor — music, philosophy, philology, astronomy, math — into an aesthetic unity. As such it provides a metaphor for much of what occurs in philosophy and math, as both philosophers and mathematicians — in different ways of course — attempt to unite different disciplines, different insights in overarching conceptual syntheses. Indeed, much of theoretical phyics can also be seen as an attempt at such conceptual synthesis. At the back of this is an inchoate demand for spiritual unity in man himself as he insists that knowledge must be one if he as man is to be one. If music and astronomy remain vastly different areas then man hiumself must be fragmented.

Hesse was an icon of the counterculture in the 1960s and books like “Siddhartha” were more or less required reading. “Steppenwolf” and “Narcissus and Goldmund” were also widely read. But “The Glass Bead Game” never achieved the same popularity — it is a more demanding book and considerably more pregnant with philosophical insight. This was the book that won Hesse the Nobel Prize for literature in 1946.

I’ll probably come back and clean this post up a bit, maybe flesh it out some more. At the moment I’m listening to Bach and feel sufficiently inspired to quickly put down what has been on my mind for quite some time about Hesse.

A Luddite argument

This post is provisional and I may come back and clean it up or amplify and extend some of the arguments.

I came across an interesting blog by one John Kozy (which I have added to my blogroll). In one of his essays there, he makes the claim that the hand-held calculator has destroyed Americans’ knowledge of arithmetic:

There is little doubt that technology has contributed to the dumbing-down of America. The first assault may have come with the hand-held calculator which has almost extinguished Americans’ knowledge of arithmetic, not to mention higher mathematical disciplines.

I couldn’t agree more. In my short stint as a schoolteacher, I abolished use of calculators on the very first day. Calculators and computers are crutches that prevent students from being able to do arithmetic. People need to be able to multiply 16 by 14 on a piece of paper. More importantly, only in doing lots of such calculations will they figure out shortcuts such as, for example, that 16 by 14 has to be 224 because 15 by 15 is 225 (and likewise 17 by 13 has to be 4 less than 225, i.e., 221). But they are not going to acquire this number sense if they use calculators. Likewise, almost no-one today — including most math teachers — can calculate square roots anymore. Ask for the square root of 5 calculated to three decimal places and even the teachers will look at you with a blank face.

Without the background in arithmetic, how are students ever going to learn algebra on a sound basis? For example, the fact that 19 by 21 = 399, which is 1 less than 20 by 20 is a particular example of a general identity: (x -1)(x+1) = (x^2) -1. How are they going to master the multiplication of polynomials when they never learnt how to multiply two two-digit numbers? And with non-existent foundations in both arithmetic and algebra, how are they expected to learn more advanced material (assuming they go on to it in the first place)?

An innumerate population can be hoodwinked by real estate salesmen, by credit card companies, by pharma companies making bogus statistical claims, by Wall Street firms, and by the government itself. And that is exactly what is happening. This is probably the principal reason why there is not more concerted resistance to the government’s bailout package: no-one understands the numbers involved and the significance these astronomical figures will have on people’s everyday lives. Americans do not even have enough arithmetical sense to calculate annual rates of interest on their credit card debt or other loans. Indeed, in a thought-provoking essay by Sasan Fayazmanesh, those who have known arithmetic have swindled those who have not over centuries:

… as I have argued in my book, Money and Exchange: Folktales and Reality, and in some other essays, in medieval trade merchants regularly tried to cheat one another in the market place. [3] In so doing they used other merchants’ ignorance of arithmetic to swindle them. Arithmetic—which at the time consisted mostly of knowledge of the Arab numerals, four basic mathematical operations and the “golden rule,” or the “rule of three,” where a missing fourth number in two equal ratios is found—had just reached Europe by way of Arab merchants. Between the 13th and 16th centuries a group of merchants in Europe, particularly in Italy, wrote manuscripts to teach merchants’ children, who attended special training schools, the newly received arithmetic. But what is perhaps most interesting about these manuscripts is that almost all of them teach how to use arithmetic, particularly in the act of barter, to cheat their trading opponents and increase what they called the “overprice.” As such, these medieval manuscripts taught that the rule of exchange was to come out ahead in transaction and that barter was “nothing but giving a good for another in order to get more.”

To make a long story short, in the medieval markets arithmetic became a tool, a “financial innovation” to use the language of the modern market, to make more money. The rule of the game was to take advantage of arithmetical ignorance of others to gain as much profit as possible. This was how capitalism was born. It was born not of honesty, equality, justice or fairness in exchange, but of deceit, swindle, inequality, injustice and unfairness.

An innumerate population that cannot cope with multiplication, division, percentages, averages, and so on is putty in the hands of a financial oligarchy in the same sense that an illiterate population that cannot cope with logic and sustained arguments and rebuttals is putty in the hands of that same financial oligarchy’s political puppets. We can’t have a healthy democracy without an informed and critical citizenry. And to be such, this citizenry has to be au fait with calculating square roots and multiplying polynomials.

I was intending to say a few words about Heidegger’s outlook on technology but to be frank it’s at a different level of seriousness and abstraction to the (relative) fluff I’ve written above. More can be found here and I’ll restrict myself to a couple of extracts:

Instead, technology enters the inmost recesses of human existence, transforming the way we know and think and will. Technology is, in essence, a mode of human existence, and we could not appreciate its mental infiltrations until the computer became a major cultural phenomenon.

and:

He (Heidegger) put forward an analysis that loaded technology with a determinate existence and an impetus of its own beyond any direct control of the “will to power.”

In short, technology develops a life, will, and purpose of its own like Frankenstein’s monster (or the replicants in Bladerunner) and as such is not amenable to individual and collective human will. But this kind of discussion goes way, way beyond my entreaty that tech should be kicked out of the classroom because it breeds innumeracy, illiteracy and the inability to think for oneself.

The Taliban and women’s rights in Pakistan

Ayesha Siddiqa came out with a book a couple of years back, titled Military, Inc.: Inside Pakistan’s Military Economy, describing how Pakistan’s military has hijacked not only the political system but also the corporate sector. The book’s been on my wish list for a couple of years but there’s only so much I can read, oft reminding myself that “much reading is a weariness of the flesh” (Ecclesiastes 12:12). She is also a columnist for the Dawn newspaper in Pakistan and she has something at the moment on how women are at the receiving end of the Taliban’s attentions:

Such representation of women in the media is also an indicator of the larger danger of a kind of radicalisation in society that has gradually begun to threaten life in the country, especially in urban centres. Given such representation, is it strange that families of working women should be threatened in Karachi, Islamabad and Lahore? The other day I was talking to a friend who was in a dilemma over not being able to send her four-year-old daughter to school as it had received threats from militants in Lahore. Taliban-type forces in all major cities threaten schools to stop educating girls else they will be bombarded.

I was a citizen of Pakistan myself for many years and this description — accurate in my opinion — resonates with me. The “Islamisation” of Pakistan started to become a reality when that scoundrel and hypocrite, Zia-ul-Haq, assumed power in 1977. But he was a darling of the USA, which needed him to help fight the war against the Soviets in Afghanistan and so whatever polite and muted objections the morally flaccid and self-serving West might have raised to some of his measures never got made. The Taliban today, mind, is composed of the same monkeys the USA was happy to befriend, train, and equip when it needed them.

By denying women education and keeping them subjugated, the country is headed even faster towards all-embracing castastrophe. Last I read, the average Pakistani woman had 6.2 children and the population doubling period was 22 years. But this will not change until and unless women are educated, get married later, have opportunities for being something more than domestic breeding machines, cooks, and washerwomen, and learn something about contraception.

Ideological bankruptcy

For me the single most distressing aspect of the current crisis (or depression, if you wish) has been the response of the Left. Or rather, the lack of response. For it has been clear for the last twenty years that socialists have had no rival program to propose as a serious alternative to the neoliberal project of unregulated global finance capitalism. And now that this capitalism has come a cropper (as was its inevitable destiny), there is no alternative vision waiting in the wings.

A big chunk of the reason for the lack of response has been the way Western Marxism has evolved. From Gramsci onwards, Marxists in Europe have shifted away from revolutionary ideas and solid economic analysis to matters more related to aesthetics and culture. Gramsci, Adorno, Marcuse, Althusser, and Jameson, all fit this mold. One couldn’t trust these people to run a corner shop. Sure, they wrote absorbing books (many of which I’ve bought over the years), but their practical analyses and prescriptions have been nil.

So Lenin’s old question resurfaces again: “What is to be done?” Each crisis is in a sense sui generis — it demands its own unique insights and solutions. The eminent Marxist historian, Eric Hobsbawm has a thought-provoking article in today’s Guardian:

Impotence therefore faces both those who believe in what amounts to a pure, stateless, market capitalism, a sort of international bourgeois anarchism, and those who believe in a planned socialism uncontaminated by private profit-seeking. Both are bankrupt. The future, like the present and the past, belongs to mixed economies in which public and private are braided together in one way or another. But how? That is the problem for everybody today, but especially for people on the left.

The dreaded “D” word

Six or seven years ago I used to constantly rack my brain trying to figure out how the world was going to get out of an economically unsustainable system. For it was clear that the USA couldn’t indefinitely continue living beyond its means and just paying its creditors worthless pieces of paper. At some stage those creditors (China, Japan, and OPEC being among the largest) would become reluctant to accept more fiat dollars. The last time this happened — in the late ’60s, when Johnson was trying to deficit finance both the Vietnam war and his Great Society programs — had USA’s major creditors feverishly exchanging their dollars for gold, until finally, in the early ’70s, Nixon was forced to go off the gold standard and the dollar became a free-floating fiat currency, ushering in the modern era of global finance.

It’s been clear for some years that a tectonic shift is long overdue but I could never quite envisage the mechanics of it: What exactly would be the trajectory from the old US-dominated world to a new multi-polar world where the emerging powers — China, Russia, India, and Brazil — would have more of a say? Today, with the benefit of hindsight, I have a clearer idea. The trajectory is through a global depression. For this is what we are now living in: let’s not pull our punches. And it’s now abundantly clear that the G7, which is in hock to BRIC (Brazil, Russia, India, China), has bleak economic prospects (see here for a more extended discussion).

There is a growing consensus that we are in a depression and not a recession. What, pray, is the fine semantic distinction? In a depression we have no idea of how to get out of the mess. As Morici argues here, a depression is not self-correcting and the economy shifts down to permanently lower levels of production, sales, and employment. A depression is always terra incognita, where the familiar tools we use in recessions and milder economic hiccups become useless and impotent. Two economists argue that the pace of decline is even more acute than that of the Great Depression (see here). The real level of joblessness in the USA is now nudging 23% (see here) — which is not that far from the Great Depression level of 25%.

Of course with an economic crisis of this dimension, civil society itself begins to unravel, to fracture. Violent crime, domestic abuse, alcoholism, and suicide all go through the roof. See here for a more extended discussion. Here in Europe I expect a summer of riots and demonstrations. We’ve already seen them in Latvia, Greece, and France.

Out of the rubble a new world will emerge. The contours of this world are still unclear to me — inchoate, vague and hazy. To paraphrase Heraclitus, depression is the father of all things. Meanwhile, and to conclude, a Marxist overview entirely congruent with my own outlook.

More on conspiracy theories, power, and Machiavelli

1974 was also the year when I first saw the film, “The Godfather.” It’s one of my favorites (and also a favorite for my son) and I’ve read the book so many times I can place any out-of-context sentence read to me aloud from it. It distresses me when it’s described as a novel (or film) about organised crime. It is really a novel about power — about the surreptitious and violent manner in which it is acquired and then exercised. And it accurately describes the political and social corruption that makes organised crime — or equivalently, conspiracies of all stripes — possible and expedient. I have consistently maintained over the years that it is the fictional equivalent of Machiavelli’s “The Prince.”

Machiavelli’s central premise is that conspiracies are ubiquitous: every prince has used it to acquire power, and every prince — once he has aquired power — has had to use conspiracies to thwart other ambitious upstarts who would take power from him. This is certainly true of the protagonists in “The Godfather,” none of whom can trust one another (save. perhaps, close kin).

Most of the characters say one thing and do something else; in our modern era, liquid modernity — to use an expression of Zygmunt Bauman — makes hypocrites of us all to the extent that we don’t know whether we have a core identity anymore.

The Gnomes of Zurich

Back in 1974, when I was living in Shiraz, Iran (Shiraz 1974 … the intoxicating air, the scent of the Spring flowers, the cute little brunette I lost my heart and soul to …), I came across a book by Taylor Caldwell, titled “Captains and the Kings.” A plot synopsis can be found at the wikipedia site here:

In The Captains and the Kings (1976) Caldwell takes on the global power brokers. In this book we find, running through the story line, a description of the way the international financiers and industrialists (all private consortiums owned by an elite of the world’s richest families and persons) hijack governments around the globe; instigating wars and gaining control over the warring countries through manipulation of the enormous debts incurred during a war. Mentioned too is the Council on Foreign Relations; and while a disclaimer states that all persons portrayed in the book are fictional, it is clear that the Council on Foreign Relations, as well as another major organization of the globalist}globalists]] are both very real organizations. Also described is the idea that political systems everywhere, and certainly in the US, are almost totally dominated by the ruling elite; and that no one even gets into the running for a major political office unless the elite believes the person is under their control. It is explained that this can be direct control; e.g., the candidate takes a solemn oath to be true to that organization above all others; or indirect control: the candidate is known to have done something illegal or scandalous, and then the threat of public exposure can be used to bend the person to the will of the elite. Politicians can also be compromised through a “set-up”. When necessary the elite will play that hand (conform or be ruined by the controlled media). It is further explained that there have been a few who were not under the control of the elite (back in the 40s and 50s) and who had some success on their own.

(Incidentally, 1976 is the wrong date: that was when the TV miniseries came out; the book itself was published in 1972)

I was a young fellow of 14 at the time and had neither the experience nor the analytical ability to be able to critically assess the novelist’s claims. The book made a great impression on me by way of explaining history in terms of conspiracies on the part of elites. Not all conspiracy theories need be assigned to the wastepaper basket. Some are credible and plausible. And we should bear in mind that ruling elites typically act conspirationally — this is what makes them an elite. Machiavelli teaches us that conspiracy is the most natural thing in the world of realpolitik as those who would be rulers attempt to supplant those in power and as those in power attempt to thwart those who would take it away from them.

The issue of a transnational ruling elite has again come to the fore during the last year or so as the world has been experiencing severe economic travail and people have been wondering how it came about and who has really been calling the shots. Here is Alex Jones’ take on the issue when he was interviewed by the Russia Today network: