I think this is no short of brilliant, and I immediately bookmarked it for further followup, because it's late. Missing piece.
I don't do deep learning, but have been recently thinking about paths to optimize DAG things, user specified functions in a completely different context, and have been wondering if there's enough off the shelf code for me to glue together...
I look forward to possibly standing on your shoulders to build something cool, and giving appropriate credit, should I post a "show HN" in 3mo-years. :)
Oh this is far too kind, but I hope it does inspire you to build little compilers everywhere! I am curious about your idea(s); please feel free to reach out via email.
I'm a systems programmer, with experience writing high performance code for mathy stuff (signal processing), network stuff, filesystem stuff, and most recently, I'm learning-by-doing "database stuff". I was joking at work the other day "if I can get compilers under my belt I'll have caught them all".
I see this applied to work (database) in a way I'll not elaborate until I can do it. :)
But in other areas, I've done a lot of signal processing, numerical algorithm development and optimization, lots of MATLAB (for improving the underlying math, but also using approximation theory for performance. Think: "replace exp(x) with a taylor series" except .... more. :)) and C/C++ (for turning those graphs of lists of equations) into machine code.
I've written dataflow style code starting from "paper of equations", to "runs on N nodes" many times.
Perhaps then you see how this looks (to me) like a "missing piece". :) I can now mix between mathematical optimizations (like replacing an expensive function with a pade approximant, using a lattice of symbolic identities, etc) and _compiler_ optimizations. \o/
