Sail by Night Physics

Machine learning, applying it requires very little knowledge, but understanding it can be frighteningly difficult.
This often gives a misleading sense of making progress, while it can make people unable to see that they’re still walking at the same place. I’ve always believed that machine learning can indeed offer breakthroughs, but I also believe it’s not that simple.
One should first study physics properly, truly understand what problem is being solved. If you just stare at the loss function all day to see if it is decreasing, it will be easy to forget that your original goal was to study nature.

  Reproducing results in ML-for-PDE-solving research using stronger baselines, by Nick McGreivy in 
    "I got fooled by AI-for-science hype — here's what it taught me"
  
  
    
        Article
        Number of citations
        Partial Differential Equation
        Weaker baseline
        Stronger baseline
        Old outcome
        New outcome
      

    Li et al.9412D incompressible Navier-StokesPseudo-spectral 64x64Discontinuous Galerkin Order 2 7x71,000-times faster7-times faster
Lu et al.9111D AdvectionFinite-difference nx=100Discontinuous Galerkin Order 2 nx=1324-times faster10-times slower
Kochkov et al.4292D incompressible Navier-StokesFinite VolumePseudo-spectral80-times fasterSlightly slower
Bar-Sinai et al.3821D Burgers’ equationWeighted Essentially Non-OscillatoryDiscontinuous Galerkin Order 2/34–8-fold fewer degrees of freedom2–4-fold fewer degrees of freedom
Wang et al.2301D Burgers’ equationSpectral nx=100Finite Volume nx=1001,000-times faster10-times slower
Li et al.1242D incompressible Navier-StokesPseudo-spectral 64x64Discontinuous Galerkin Order 2 3x31,000-times faster7-times faster
Hsieh et al.1012D PoissonMultigridLower–Upper decompositionFaster1,000-times slower
Brandstetter et al.871D Burgers’ equation, 1D WaveWeighted Essentially Non-Oscillatory, Pseudo-spectralWeighted Essentially Non-Oscillatory, Finite VolumeMuch faster1,000-times slower
de Lara and Ferrer122D PoissonDiscontinuous Galerkin Order 28 nx=1Discontinuous Galerkin Order 9 nx=122–75-times faster4–10-times slower
Tang et al.22D PoissonConjugate Gradient and MultigridLower–Upper decomposition12-times faster35–500-times slower

  

Article	Number of citations	Partial Differential Equation	Weaker baseline	Stronger baseline	Old outcome	New outcome
Li et al.	941	2D incompressible Navier-Stokes	Pseudo-spectral 64x64	Discontinuous Galerkin Order 2 7x7	1,000-times faster	7-times faster
Lu et al.	911	1D Advection	Finite-difference nx=100	Discontinuous Galerkin Order 2 nx=13	24-times faster	10-times slower
Kochkov et al.	429	2D incompressible Navier-Stokes	Finite Volume	Pseudo-spectral	80-times faster	Slightly slower
Bar-Sinai et al.	382	1D Burgers’ equation	Weighted Essentially Non-Oscillatory	Discontinuous Galerkin Order 2/3	4–8-fold fewer degrees of freedom	2–4-fold fewer degrees of freedom
Wang et al.	230	1D Burgers’ equation	Spectral nx=100	Finite Volume nx=100	1,000-times faster	10-times slower
Li et al.	124	2D incompressible Navier-Stokes	Pseudo-spectral 64x64	Discontinuous Galerkin Order 2 3x3	1,000-times faster	7-times faster
Hsieh et al.	101	2D Poisson	Multigrid	Lower–Upper decomposition	Faster	1,000-times slower
Brandstetter et al.	87	1D Burgers’ equation, 1D Wave	Weighted Essentially Non-Oscillatory, Pseudo-spectral	Weighted Essentially Non-Oscillatory, Finite Volume	Much faster	1,000-times slower
de Lara and Ferrer	12	2D Poisson	Discontinuous Galerkin Order 28 nx=1	Discontinuous Galerkin Order 9 nx=1	22–75-times faster	4–10-times slower
Tang et al.	2	2D Poisson	Conjugate Gradient and Multigrid	Lower–Upper decomposition	12-times faster	35–500-times slower

Is it too difficult? ? ?

Initiation and Guessing

Take the high road.

Next: To be continued