NeuralBrew

Reduced computational complexity in hardware resulting in much lower energy consumption and silicon area because in log domains:

• Multiplication becomes addition: log_b(x · y) = log_b(x) + log_b(y)
• Division becomes subtraction: log_b(x / y) = log_b(x) - log_b(y)
• Exponentiation becomes multiply: log_b(x^k) = k · log_b(x)
• Squaring and square root become simple bit shifts.

---

Nature utilizes logarithmic and non-linear signal processing:

• Logarithmic nature of the brain
• The Logarithmic Scale of Sensory Experience
• Neural Networks Are Useless Without Non-Linearity

---

Higher dynamic ranges with uniform relative resolution:

• Reduces impact of vanishing and exploding gradients.
• Greatly reduces memory bandwidth for access of weights and other parameters.

---

Energy efficient algorithms that go beyond backpropagation:

• Correlation, covariance and other statistical analysis based training and inference.
• Energy based models have a natural log-probability interpretation.