Introduction
This semester, I participated in the Functional Analysis Seminar set up by Mr. Liu Rui. The main content was to discuss some cutting-edge topics related to Functional Analysis.
And the fist part of this seminar is to study a paper called Expressivity of Spiking Neural Networks held by one of his graduate student.
the Paper
Abstract
The synergy between spiking neural networks and neuromorphic hardware holds promise for the development of energy-efficient AI applications. Inspired by this potential, we revisit the foundational aspects to study the capabilities of spiking neural networks where information is encoded in the firing time of neurons. Under the Spike Response Model as a mathematical model of a spiking neuron with a linear response function, we compare the expressive power of artificial and spiking neural networks, where we initially show that they realize piecewise linear mappings. In contrast to ReLU networks, we prove that spiking neural networks can realize both continuous and discontinuous functions. Moreover, we provide complexity bounds on the size of spiking neural networks to emulate multi-layer (ReLU) neural networks. Restricting to the continuous setting, we also establish complexity bounds in the reverse direction for one-layer spiking neural networks.
Keywords: Expressivity, Approximation Theory, Spiking Neural Networks, Deep (ReLU) Neural Networks, Temporal Coding, Linear Regions
Click Here to read the original paper.
the Notes
As you know, I’ve always been in the habit of taking notes.
But the size of my note is too large. So I I uploaded my notes to my Baidu Netdisk.
CLick Here to download my note.
Remark
About this Seminar, I have something to share.
At first, there were more than ten participants in this course, but I don’t know if it was because the content was too abstract or for some other reason. After three classes, only the graduate students and I were left (LOL).
But Neural Network Learning is really interesting
By the way, if you have any questions or need further assistance, feel free to ask! Here is my email: nkusherr1 at gmail.com
