The Name of the Rose

The Priory Dispute: A Tragedy in Several Linear Mappings

A fictional mathematical comedy in one act.

Setting: A medieval priory. Two distinguished scholars have been summoned before the Abbot to settle a dispute concerning the ownership of a particularly interesting neural network architecture.

On the table lies a manuscript entitled:

Concatenated Context-Selected Linear Transformations: A New Foundation for Intelligence.

Tao: I believe we should begin by establishing precisely what is being claimed.

Schmidhuber: Naturally. The idea is that a sequence of linear mappings, selected according to context, can produce arbitrarily sophisticated nonlinear behaviour.

Tao: Yes, but that is rather obvious once one understands the composition of functions.

Schmidhuber: Composition is not the point. The point is that the mappings are selected by an intelligent mechanism.

Tao: Then you have introduced a nonlinear selection mechanism.

Schmidhuber: I have introduced context.

Tao: Context is a mathematical object now?

Schmidhuber: It is if it selects a matrix.

Tao: I see. And who first thought of selecting a matrix?

Schmidhuber: That is precisely the question.

Tao: I was doing linear algebra before you were doing neural networks.

Schmidhuber: And I was doing neural networks before you realised they could be linear algebra.

A long silence.

The Abbot: Gentlemen, perhaps the priory can settle this matter by determining who owns the underlying mathematical principle.

Tao: That would require a definition of ownership.

Schmidhuber: And a suitable measure of originality.

Tao: Which itself requires a metric.

Schmidhuber: Perhaps a learned metric?

Tao: Absolutely not.

The dispute escalates

Schmidhuber: Consider the following construction. We have matrices A1,A2,...,An, and a context mechanism selects which one to apply.

y=Ai(x)x

Tao: That is a piecewise linear mapping.

Schmidhuber: Exactly!

Tao: Which is not necessarily globally linear.

Schmidhuber: Exactly!

Tao: So the intelligence is in the selection.

Schmidhuber: Now you understand.

Tao: No. I understand that you have moved the nonlinearity from the transformation into the selection process.

Schmidhuber: That is called engineering.

Tao: That is called mathematics.

Schmidhuber: It is called both.

Tao: Then perhaps we should share the credit.

Schmidhuber: On what basis?

Tao: A concatenated sequence of context-selected linear mappings.

Schmidhuber: You cannot simply concatenate my intellectual property!

Tao: I was proposing a mathematical operation.

The tragic conclusion

The Abbot finally intervenes.

The Abbot: I have consulted the ancient manuscripts. It appears that both of you have independently arrived at the same conclusion: intelligence need not require a single complicated transformation if it can select and compose simpler transformations.

Tao: That is a reasonable summary.

Schmidhuber: But who gets the credit?

The Abbot: The priory has decided that the credit shall be distributed according to a singular value decomposition.

Tao: That doesn't make any sense.

Schmidhuber: What are the singular values?

The Abbot: Undisclosed.

Tao: And the eigenvectors?

The Abbot: Disputed.

Schmidhuber: Then the entire thing is ill-conditioned!

The two scholars storm out in opposite directions, each convinced that the other has misunderstood the fundamental theorem.

Epilogue

A young monk, who has been quietly observing the proceedings, picks up the manuscript.

He reads the title, considers the argument, and writes underneath:

"Perhaps the real discovery is that intelligence consists of knowing which linear mapping to use next."

He then adds:

"And perhaps the real tragedy is that nobody thought to switch matrices."

The Abbot orders the manuscript locked away.

It is classified as a dangerous work of heresy.

The priory has inadvertently invented a new field of mathematics: Contextual Linearism.

Comments

Popular posts from this blog

Centralizing Neural Networks on Linear Algebra

The nonlinear defense

Sinbad and the Seven Seas of Linear Algebra