 |
Center for Nonlinear Studies |
Modeling practice seems to be partitioned into schientific models defined by mechanistic differential equations and machine learning models defined by parameterizations of neural networks. While the ability for interpretable mechanistic models to extrapolate from little information is seemingly at odds with the big data "model-free" approach of neural networks, the next step in scientific progress is to utilize these methodologies together in order to emphasize their strengths while mitigating weeaknesses.In this talk the audience will be introduced to how Julia's differentiable programming frameworks are bringing neural networks into differential equations and vice versa. The idea of prior structural informations in neural architectures will be explained via the relationship of between ResNet and ordinary differential equations (ODEs) and convolutional layers to partial differential equations (PDEs), and thus generalized to differential equation layers of neural networks.The ability for neural networks to learn nonlinear equations will be demonstrated through partial neural ODEs. DifferentialEquations.jl's GPU-compatible high-order adaptive methods for stiff ODEs, SDEs, DDEs, and PDEs will be demonstrated as an effective tool for handling these mixed models. The behavior of biochemical reaction networks can be better understood using a mixed neural differential equation approach, and their use in the context of systems biology will be shown.