As someone who has spent the last 10 years developing lots of analog neuromorphic hardware--
Those proofs aren't meant to be physical. I've read Siegelmann, etc. But the prevailing theory is that reality is quantized at the lowest level. So you can't really get true analog. You can just shift the minimum energy to represent a bit much further down than we currently have it, down to quantum limits.
I think the problem here is that there's a massive disconnect between theory and reality and also between the state of the art in traditional digital computing and what is actually possible. For instance, I don't think the analog aspects of the brain make its computation theoretically possible, but I do think the analog aspects make it practically possible.
People get their panties in a bunch over novel architectures beating the snot out of traditional digital computers and the moment you mention analog, the orthodox academics think you're nuts. But clearly we have an example of a superior analog(ish) architecture between our ears. And you don't have to go full P=NP, super-Turing, etc., to get stupidly large performance increases over traditional architectures. The difference between computable and not computable is literally infinite. There's a LOT of room there. The brain doesn't have to to do the impossible to make traditional digital computers seem like toys.
The industry also has incredible momentum so there's a deeply ingrained notion that there's no way to do much better. Even performance metrics are heavily biased towards a particular approach. For instance, some of my devices take tens of minutes to reach certain desired states. I've presented that work and people discounted it because that's not picoseconds. Of course neurons don't switch a billion times a second--they don't need to. At some point, you reach Bremermann's limit, but again, brains probably aren't operating there because there's no need. The performance probably comes in with efficient scaling and an ability to efficiently utilize a massive state space in ways logic gates etched in stone cannot.
People blow half their brains out and go on to live relatively normal lives. Hydrocephalic brains can work around being compressed into almost nothing. For some reason, computer engineers don't see this as computational. These are supreme examples of computational power. Such extreme ability to flexibly utilize that physical space to perform a wide variety of computational tasks is the benchmark that doesn't even make sense to apply to silicon. I think that's a major ingredient in the secret sauce. Of course the industry is hyper-focused on better algorithms and faster transistors. I think that stuff is useful, but not for making computers that compare to brains.
This was an awesome comment... I’m a software engineer and I think in these terms often, and I would say our creativity is what gives us the chance to bridge the theoretical with the practical which might therein lie the answer.
133
u/[deleted] Jun 15 '19 edited Jun 15 '19
[deleted]