PINNpoint
Physics Informed Neural Nets (PINNs) are an area of research that hold tremendous benefits. Complex PDEs can be solved with half of the necessary compute using a neural network instead of a supercomputer cluster. However, they like to lie; the loss drops to zero but the solution is useless. I looked to past literature--which is fairly recent--and we recreated the failures and characterize the effects of 3 hypotheses on PINN performance.
Reproduced verbatim from the project README:
"This repository's code, experiments, figures, and documentation were generated by AI systems — Claude Code (Anthropic) as the coordinating agent and OpenAI Codex for implementation — under the direction of the authors. It was produced as a project for the Engineering Summer Academy at Penn (ESAP), Artificial Intelligence course. The accompanying presentation is authored separately by the students and is not part of this AI-generated disclaimer."
Experimental design, equation groundings, and interpretation of results were directed and reviewed by the human authors.
Design
Three benchmark problems: one-dimensional convection at β = 30, a reaction–diffusion system with ρ = ν = 5 (both from Krishnapriyan et al., 2021), and Allen–Cahn with the standard Raissi et al. (2019) setup. The network is kept constant. 4 hidden layers of 50 tanh units, mapping (x, t) to u. Training is Adam for 5,000 iterations followed by L-BFGS.
We set the failure threshold at 0.5 relative L2 error against a trusted reference solution before running anything. We tested three factors: precision (FP32 or FP64), L2 regularization (1×10-4 or none), and collocation density.
Results
On Allen–Cahn, zero of sixty baseline runs passed the gate. The best of them reached a held-out residual of 7.1×10-5 while its relative L2 error was 0.526 The physics loss is five orders of magnitude down and the answer is still wrong by half. Neither precision nor L2 regularization fixed it either. FP32 and FP64 medians sat at 0.9934 and 0.9937, with and without L2 at 0.9935 and 0.9936.
On convection at β = 30, 59 of 60 runs failed. L2 regularization cut average error by 25.7%, but still, only one run passed. So residual overfitting contributes something, but it does not account for the failure on its own. More collocation points helped with diminishing returns: 41.6% improvement going from 100 to 1,000, then 4.5% from 1,000 to 10,000.
The rescue is the interesting part. Two interventions were tested separately and together: hard initial and boundary constraints, and double-backprop regularization penalizing residual gradients.
Hard constraints on their own reached a median of 0.492, which is technically below the threshold, but the solution isn't perfect, and one seed still failed. Double-backprop on its own reached 0.587 and failed. Together they reached 0.0056 and passed on every seed. In that condition the held-out residual dropped to 3.4×10-7 and the solution error came down with it instead of staying flat.
What this supports is that the two interventions are fixing different things. One enforces the conditions the loss was not pinning down; the other goes after residual overfitting. Neither is enough alone.
There's way more work to be done, by no means was this a comprehensive study, but it provides a foundation for future research. I wish we could have done this the entire time at ESAP, I think it would have been a much better use of time if we had been doing this from the start accompanied by learning the Linear Algebra in tandem. I might try to recreate a PINN for fluid flows next and actually get it to work. Sorry, I got sidetracked, the conclusions of the study were: there is an underlying seed dependence, there is a non-monotonic dependence between the various factors, and the loss is a liar. Let me break each of these down. 1. The underlying seed dependence. The randomness imposed by the seed and the collocation points alters model performance, the Allen-Cahn experienced a level of variance in the FP64 no regularization run with 8k collocation points, other PDEs experienced similar results. This means that the randomness affects the way that the model behaves when predicting the points in the plane. 2. The non monotonic dependence. The precision, regularization, and collocation points all had varying effects on each of the PDEs. No generalizable claim can be made, and it shouldn't be made. 3. The loss in every case drops but the solution stays wrong, that's why the rescue works. The rescue enforces where it starts with the constraints, and the double backprop prevents it from taking the easy way out in the loss gradient. Also, this project was an anecdotal support for the ability for AI to govern methodological operations. I have no idea how to code and test PDE-based neural nets from the ground up, but Claude does. I simply had to give it the methodological design framework from the literature, and it went at it, I also gave it the sources so it could see the exact methods they used and where it converges and diverges from our design. Humans are going to play a very interesting role in future research, maybe not experimental aerodynamics though, that's safe... for now (and hopefully forever).
Reproducibility
Configurations are locked and committed, as are aggregate result tables and all figures. Large per-run artifacts are excluded but regenerable from the configs. MIT licensed.
Tools
PyTorch, L-BFGS, LaTeX.