When we are children, we lack patience. When we grow up, “we lack time”.
Not all is bad, of course. Once trained, the deep neural networks are very fast when used for prediction. However, that assumes that the model is actually* used in practice rather than for the incremental improvement of state-of-art, i.e. academic scores. Not to mention that, in the latter case, the published model usually leaves a lot to be desired in terms of generated scientific knowledge.
Apart from deep learning, Google is also famous for — well, their first product ever — the Google search engine. The latter must be incredibly fast as it has to work in real-time. Therefore, the time efficiency of the algorithms used here is of supreme importance, and the engineers developing it must swim without effort through the big O notation. Or, use Bubble-sort instead of Quick-sort and you are beaten by the competition.
Scarce resources, like time, in this case, stimulate creative and efficient usage of the same.
Thus, when using deep learning in science, perhaps we should instead search for inspiration in this corner of the Google Universe.
Thanks to Johan Hidding, Florian Huber, Tom Bakker and Pablo Rodriguez-Sanchez for the useful suggestions.
