The Name of the Rose
The Priory Dispute: A Tragedy in Several Linear Mappings
A fictional mathematical comedy in one act.
Setting: A medieval priory. Two distinguished scholars have been summoned before the Abbot to settle a dispute concerning the ownership of a particularly interesting neural network architecture.
On the table lies a manuscript entitled:
Concatenated Context-Selected Linear Transformations: A New Foundation for Intelligence.
Tao: I believe we should begin by establishing precisely what is being claimed.
Schmidhuber: Naturally. The idea is that a sequence of linear mappings, selected according to context, can produce arbitrarily sophisticated nonlinear behaviour.
Tao: Yes, but that is rather obvious once one understands the composition of functions.
Schmidhuber: Composition is not the point. The point is that the mappings are selected by an intelligent mechanism.
Tao: Then you have introduced a nonlinear selection mechanism.
Schmidhuber: I have introduced context.
Tao: Context is a mathematical object now?
Schmidhuber: It is if it selects a matrix.
Tao: I see. And who first thought of selecting a matrix?
Schmidhuber: That is precisely the question.
Tao: I was doing linear algebra before you were doing neural networks.
Schmidhuber: And I was doing neural networks before you realised they could be linear algebra.
A long silence.
The Abbot: Gentlemen, perhaps the priory can settle this matter by determining who owns the underlying mathematical principle.
Tao: That would require a definition of ownership.
Schmidhuber: And a suitable measure of originality.
Tao: Which itself requires a metric.
Schmidhuber: Perhaps a learned metric?
Tao: Absolutely not.
The dispute escalates
Schmidhuber: Consider the following construction. We have matrices A1,A2,...,An, and a context mechanism selects which one to apply.
Tao: That is a piecewise linear mapping.
Schmidhuber: Exactly!
Tao: Which is not necessarily globally linear.
Schmidhuber: Exactly!
Tao: So the intelligence is in the selection.
Schmidhuber: Now you understand.
Tao: No. I understand that you have moved the nonlinearity from the transformation into the selection process.
Schmidhuber: That is called engineering.
Tao: That is called mathematics.
Schmidhuber: It is called both.
Tao: Then perhaps we should share the credit.
Schmidhuber: On what basis?
Tao: A concatenated sequence of context-selected linear mappings.
Schmidhuber: You cannot simply concatenate my intellectual property!
Tao: I was proposing a mathematical operation.
The tragic conclusion
The Abbot finally intervenes.
The Abbot: I have consulted the ancient manuscripts. It appears that both of you have independently arrived at the same conclusion: intelligence need not require a single complicated transformation if it can select and compose simpler transformations.
Tao: That is a reasonable summary.
Schmidhuber: But who gets the credit?
The Abbot: The priory has decided that the credit shall be distributed according to a singular value decomposition.
Tao: That doesn't make any sense.
Schmidhuber: What are the singular values?
The Abbot: Undisclosed.
Tao: And the eigenvectors?
The Abbot: Disputed.
Schmidhuber: Then the entire thing is ill-conditioned!
The two scholars storm out in opposite directions, each convinced that the other has misunderstood the fundamental theorem.
Epilogue
A young monk, who has been quietly observing the proceedings, picks up the manuscript.
He reads the title, considers the argument, and writes underneath:
"Perhaps the real discovery is that intelligence consists of knowing which linear mapping to use next."
He then adds:
"And perhaps the real tragedy is that nobody thought to switch matrices."
The Abbot orders the manuscript locked away.
It is classified as a dangerous work of heresy.
The priory has inadvertently invented a new field of mathematics: Contextual Linearism.
Comments
Post a Comment