The Priory Dispute II
The Priory Dispute II
The Case of the Missing Geometry
A mathematical farce in one act.
Setting: The Great Hall of the Priory of Applied Mathematics.
A large geometric diagram has been pinned to the monastery wall. It shows a cloud of points gradually rotating, stretching, folding and changing shape through a sequence of linear transformations.
Underneath is written:
GLOBAL GEOMETRIC GRADUALISM
Terence Tao and Jürgen Schmidhuber are standing on opposite sides of the diagram.
The Abbot enters, visibly exhausted.
The Abbot: Gentlemen! I have been informed that there is another dispute.
Tao: A serious matter, Your Reverence. The phrase Global Geometric Gradualism is mathematically misleading unless properly qualified.
Schmidhuber: That's not the dispute. He claims I didn't think of it first.
Tao: I claim no such thing. I merely observe that the underlying mathematics has been known for considerably longer than either of us.
Schmidhuber: Knowing the mathematics is not the same as recognising its significance for intelligence.
Tao: Recognising its significance is not the same as inventing it.
Schmidhuber: And inventing it is not the same as understanding it!
The Abbot: Excellent. We are making progress. Nobody agrees on anything.
The question of priority
Schmidhuber: Your Reverence, consider a neural network. Its layers progressively transform the representation. I have spent decades thinking about systems that learn, adapt and construct increasingly sophisticated representations.
Tao: Quite so. But the geometric interpretation follows naturally from the composition of transformations. The interesting mathematical questions concern the structure of those transformations.
Schmidhuber: You are describing the machinery. I am describing the intelligence.
Tao: And I am pointing out that the machinery has some rather interesting properties.
Schmidhuber: You mathematicians always arrive after the engineers have built the cathedral and then claim priority because you understand the arches.
Tao: And you engineers always claim to have invented geometry because you managed to build a wall.
Several monks quietly leave the room.
The Abbot examines the diagram
The Abbot: Let me see if I understand this new doctrine.
Instead of imagining every layer as performing some elaborate nonlinear manipulation, we consider the successive geometric transformations of the data.
Tao: Correct, with appropriate qualifications.
Schmidhuber: And the transformations can be selected according to context.
Tao: In the CCSLM formulation, yes.
The Abbot: And even ReLU networks can be understood in terms of changing effective linear mappings, according to activation patterns?
Tao: Precisely.
Schmidhuber: Which means that considerable global complexity can emerge from relatively simple local transformations.
The Abbot: So the important discovery is that the geometry gradually evolves.
Tao: That is a useful interpretation.
Schmidhuber: A very useful interpretation.
The Abbot: And which of you invented it?
A long silence.
The final judgment
The Abbot: I have consulted the historical records, the mathematical manuscripts and the engineering archives.
It appears that linear algebra is several centuries old.
The theory of neural networks has a substantial history.
Piecewise-linear representations are well established.
And the composition of transformations is hardly a recent discovery.
Tao: Exactly.
Schmidhuber: But the synthesis!
The Abbot: Ah, yes. The synthesis.
I hereby declare that neither of you may claim exclusive ownership of Global Geometric Gradualism.
Tao: A reasonable decision.
Schmidhuber: On what mathematical basis?
The Abbot: The principle of distributed intellectual responsibility.
Tao: That sounds suspiciously like an averaging operation.
Schmidhuber: With no guarantee of convergence!
Epilogue: A young monk speaks
A young monk, who has been taking notes throughout the proceedings, raises his hand.
Young Monk: Your Reverence, I have a question.
The Abbot: Yes, my son?
Young Monk: If neural networks achieve complexity by gradually accumulating simple geometric transformations, and if the context determines which transformations are applied, isn't the really interesting question how we can deliberately design and control those transformations?
Silence.
Tao: That's a legitimate mathematical question.
Schmidhuber: And an engineering question.
The Abbot: Good heavens. The boy has just started another research programme.
The Abbot orders the manuscript locked away in the deepest library vault.
The inscription on the vault reads:
CAUTION: CONTENTS MAY CAUSE PARADIGM SHIFTS.
A footnote from the monastery historian:
The dispute remains unresolved. Both scholars subsequently published papers explaining why the other had misunderstood the significance of the geometry.
Neither paper mentioned the young monk.
And somewhere in the basement, CCSLM quietly continued changing matrices.
Comments
Post a Comment