 |
ML Newsnew | past | best | rss | submit | about |
| Functorial String Diagrams for Reverse-Mode Automatic Differentiation (2021) (arxiv.org) | |
| 1 point by gone35 on Jan 8, 2025 | hide | past | pdf | discuss on HN | |
|
In plain words: Box-and-wire drawings gain nested boxes to describe functions that take or return functions, giving a rule-based derivative calculator for simple typed function code. It is the first proof that the calculator's derivatives are correct, and the drawings match exactly a graph form that runs fast. Abstract · Functorial String Diagrams for Reverse-Mode Automatic DifferentiationWe enhance the calculus of string diagrams for monoidal categories with hierarchical features in order to capture closed monoidal (and cartesian closed) structure. Using this new syntax we formulate an automatic differentiation algorithm for (applied) simply typed lambda calculus in the style of [Pearlmutter and Siskind 2008] and we prove for the first time its soundness. To give an efficient yet principled implementation of the AD algorithm we define a sound and complete representation of hierarchical string diagrams as a class of hierarchical hypergraphs we call hypernets. | |
| add comment on HN |