The "Supersymmetric Artificial Neural Network"

[thoughtfuhk]

The “Supersymmetric Artificial Neural Network” (or “Edward Witten/String theory powered artificial neural network”) is a Lie Superalgebra aligned algorithmic learning model (created by myself 2 years ago), based on evidence pertaining to Supersymmetry in the biological brain.

 

A Deep Learning overview, by gauge group notation:

1. There has been a clear progression of “solution geometries”, ranging from those of the ancient Perceptron to complex valued neural nets, grassmann manifold artificial neural networks or unitaryRNNs. These models may be denoted by [math]\phi(x,\theta)^{\top}w [/math] parameterized by [math]\theta[/math], expressible as geometrical groups ranging from orthogonal to special unitary group based: [math]SO(n)[/math] to [math]SU(n)[/math]..., and they got better at representing input data i.e. representing richer weights, thus the learning models generated better hypotheses or guesses.
2. By “solution geometry” I mean simply the class of regions where an algorithm's weights may lie, when generating those weights to do some task.
3. As such, if one follows cognitive science, one would know that biological brains may be measured in terms of supersymmetric operations. (Perez et al, “Supersymmetry at brain scale”)
4. These supersymmetric biological brain representations can be represented by supercharge compatible special unitary notation [math]SU(m|n)[/math], or [math]\phi(x,\theta, \bar{{\theta}})^{\top}w[/math] parameterized by [math]\theta, \bar{{\theta}}[/math], which are supersymmetric directions, unlike [math]\theta[/math] seen in item (1). Notably, Supersymmetric values can encode or represent more information than the prior classes seen in (1), in terms of “partner potential” signals for example.
5. So, state of the art machine learning work forming [math]U(n)[/math] or [math]SU(n)[/math] based solution geometries, although non-supersymmetric, are already in the family of supersymmetric solution geometries that may be observed as occurring in biological brain or [math]SU(m|n)[/math] supergroup representation.

 

Psuedocode for the "Supersymmetric Artificial Neural Network": 

a. Initialize input Supercharge compatible special unitary matrix [math]SU(m|n)[/math]. [See source] (This is the atlas seen in b.)

b. Compute [math]\nabla C[/math] w.r.t. to [math]SU(m|n)[/math], where [math]C[/math] is some cost manifold.

• Weight space is reasonably some K¨ahler potential like form: [math]K(\phi,\phi^*)[/math], obtained on some initial projective space [math]CP^{n-1}[/math]. (source)
• It is feasible that [math]CP^{n-1}[/math] (a [math]C^{\infty}[/math] bound atlas) may be obtained from charts of grassmann manifold networks where there exists some invertible submatrix entailing matrix [math]A \in \phi_i (U_i \cap U_j)[/math] for [math]U_i = \pi(V_i)[/math] where [math]\pi[/math] is a submersion mapping enabling some differentiable grassmann manifold [math]GF_{k,n}[/math], and [math]V_i = u \in R^{n \times k} : det(u_i) \neq 0[/math]. (source)

c. Parameterize [math]SU(m|n)[/math] in [math]-\nabla C[/math] terms, by Darboux transformation.

d. Repeat until convergence.

References:

1. The “Supersymmetric Artificial Neural Network” on github.

2. “Thought Curvature” paper (2016).

3. Although not on supermanifolds/supersymmetry, but manifolds instead , here’s a talk by separate authors at Harvard University, regarding curvatures in Deep Learning (2017).

4. A relevant debate between Yann Lecun and Marcus Gary, along with my commentary, on the importance of priors in Machine learning.

5. Deepmind’s discussion regarding Neuroscience-Inspired Artificial Intelligence.

Edited January 10, 2018 by thoughtfuhk