OFFICE OF ADVANCED SIMULATION AND COMPUTING AND INSTITUTIONAL R&D PROGRAMS (NA-114)

Welcome to the November 2024 issue of the NA-114 newsletter - published quarterly to socialize the impactful work being performed by the National Nuclear Security Administration (NNSA) laboratories and our other partners.  This edition begins with a highlight from Sandia National Laboratories (SNL) on their newly developed static gas model in RAMSES/Empire for design support for the W87-1 and W93 programs.  Other featured highlights include: 

• SNL’s new Distinguished Members of Staff.
• Lawrence Livermore National Laboratory’s (LLNL’s) acceptance of the El Capitan system from Hewlett Packard Enterprise (HPE) on October 30th.
• Los Alamos National Laboratory (LANL) analysis to definitively quantify the effect of geometry on jet formation in modeling ejecta from shocked metal - an area critical to stockpile stewardship.
• LANL’s oblique shock test problems to evaluate ASC codes in preserving shock structures in multi-physics, multi-material simulations.  

Please join me in thanking the professionals who delivered the achievements highlighted in this newsletter and on an ongoing basis, all in support of our national security mission.

Thuc Hoang
NA-114 Office Director

---

New Empire Static Gas Model Supports Needs in W87-1 and W93 Programs

The SNL ASC program has developed a static gas model in RAMSES/Empire to accelerate simulations of system-generated electromagnetic pulse (SGEMP) effects, source-region electromagnetic pulse (SREMP) effects, and atmospheric-pressure discharges in weapon components.  This development has been prioritized by anticipated needs for the W87-1 and W93 programs, especially for near-term design support when quick-turn analysis is critical.  The new static gas model in Empire allows for far more efficient simulation of SGEMP, SREMP, and discharge problems in this regime. Further development will extend the model to include additional physics and charged fluids for simulating cold plasmas, thereby achieving additional performance gains for these problems (SAND2024-09821M).

 

---

SNL ASC program announces new Distinguished Members of Staff

Sandia’s ASC Leadership team congratulates the following individuals on their promotions and thanks them for their hard work and dedication to the mission:

• Curtis Ober (Org 1446), support to Integrated Codes and Verification & Validation Subprograms for Trilinos and Large-Scale Calculations Initiative (LSCI).
• Denis Ridzal (Org 1463), support to Integrated Codes Subprogram for Rapid Optimization Library (ROL).
• Sivasankaran Rajamanickam (Org 1465), support to Computational Systems & Software Environment and Integrated Codes Subprograms for KOKKOS Kernels and Artificial Intelligence/Machine Learning (AI/ML).
• Christian Trott (Org 1465), support to Computational Systems & Software Environment and Integrated Codes Subprograms for Kokkos and ISO C++.
• Bill Scherzinger (Org 1558), support to Physics and Engineering Models Subprogram for development of the Library of Advanced Materials for Engineering (LAMÉ) for use in Sierra.
• Tracy Wilbur (Org 1985), embedded support to the ASC Program for Program Development and Strategy.
• Ernest Foss Friedman-Hill (Org 8753), support to Integrated Codes and Verification & Validation Subprograms for Sandia Analysis Workbench (SAW), Workflows, and Accelerated Digital Engineering (ADE).
• Jeff Ogden (Org 9327), support to Facility Operations & User Support Subprogram for High Performance Computing systems engineering, development, integration, and operations.

Promoted to Senior Scientists:

• William “Bill” Hart (1464), support to Computational Systems & Software Environment Subprogram to wrap up Exascale Computing Project activities.
• Mike Glass (1500), support to Integrated Codes and Computational Systems & Software Environment Subprograms for Computational Simulation, ADE Initiative, Production Simulation Initiative (PSI), and Computing as a Service (SAND2024-11578M).

 

---

LLNL officially accepted the El Capitan system from HPE on October 30, 2024, completing a significant milestone for the NNSA’s first exascale computer.

Work will continue over the next several months to bring the system into General Availability service by the end of FY25.

After completing prescribed full-system testing over two weeks, LLNL officially accepted the El Capitan exascale compute system from HPE on October 30, 2024.  El Capitan was exercised to demonstrate its functionality, performance, and stability and proved its usefulness and value.  El Capitan and several related systems will be included on the TOP500 list of the world’s most powerful computers that will be released November 18th at the SC24 conference in Atlanta, Georgia.  While contractual acceptance marks a major milestone for the system, work remains to be done to fully prepare the computer and NNSA applications for production use.

A limited set of initial tests using full applications have been run on the system and soon a small set of code teams from LLNL, SNL, and LANL will be given access.  The code teams will continue to stress-test the hardware and software for reliability, evaluate the system’s ability to run large jobs, and provide valuable input into performance tuning.  NNSA code developers will conduct porting, scaling, validation, and tuning of applications, system software, and configuration settings to ensure the system fully supports the stockpile stewardship mission throughout its anticipated lifetime.

The vendors, HPE and AMD, will continue to work closely with LLNL, the trilab application teams, and the ASC program over the next several months to help complete the final tasks required to bring the system into full production.  Projections are for the system to move to the secure network in the March 2025 timeframe, and to be in full-production use by the end of FY25 (LLNL-ABS-871239).

 

---

Modeling ejecta from shocked metal is a critical role for stockpile stewardship; however, existing models and experimental data are insufficient to constrain jet characteristics as a function of grove geometry.

LANL recommends novel experiments to definitively quantify the effect of geometry on jet formation.

Accurate modeling of fluid jets, which emerge from defects on metal surfaces, is essential to understanding the behavior of materials under extreme conditions, such as those encountered in nuclear weapons.  Ensuring the accuracy of predictive models for these phenomena is vital for maintaining the reliability and safety of the nation’s stockpile in the absence of nuclear weapons testing.

Meeting this need is particularly challenging because of the complex nature of the fluid dynamics involved.  Jet formation is driven by vorticity (rotational fluid flow) caused by pressure and density gradients in the material.  In a recent paper [1], LANL examined the most salient assumptions of two common predictive models of jet mass and jet-tip velocity.  Both classes of models have limitations due to geometric idealizations of the vorticity generated as shocks transit the groove defects.  The first class of models were adapted from shaped-charge jet theories originally conceived to predict the armor-penetrating, liquid-metal jets of WWII-era anti-tank weapons such as the American Bazooka and the German Panzerfaust.  These models approximate the vorticity along the metal-void interface in a discrete, piecewise approach.

The second class of models were derived from pioneering linear stability analyses of interface perturbation growth (Taylor and Richtmyer, among many others).  This modal approach predicts jets in terms of the most unstable modes.  As an extreme example of model limitations, for the special case of shocked square grooves, modal theory predicts jets while piecewise theory does not.

Existing experimental data does not provide the means to distinguish model predictions, especially concerning the relationship between groove geometry and jet characteristics.
Following the validation practices outlined in [2], LANL researchers identified key gaps in the experimental literature and conducted an exhaustive survey of published results spanning 48 years. Although preliminary evidence suggests that jet-tip velocity correlates positively with narrower and deeper grooves (high-aspect-ratio grooves) and that jet mass correlates with groove cross-sectional area (larger groove volumes), these claims are unproven due to the lack of experiments that independently control for aspect ratio and volume.  LANL’s FLAG simulations, which varied these parameters independently, revealed that jet-tip velocity is not correlated with groove volume and that jet mass behaves nonlinearly with respect to aspect ratio.  Consequently, LANL recommends novel experiments that isolate and control aspect ratio and volume independently to definitively quantify their effects on jet formation for the first time (LA-UR-24-29492).

References:
[1] Kaiser, B.E., Tregillis, I. L., Cherne, F. J., and Koskelo, A.C., Jets from shocked metal surfaces with grooves: Missing experiments. J. Appl. Phys., 2024. 135(17): p. 170903
[2] Wilson, B.M. and Koskelo, A.C., A Practical Validation Assessment Workflow, J. Verifi. Valid. Uncert. Quantif, 2020. 5(1): 011004.

---

Oblique shocks represent a verification challenge in the context of ensuring accurate predictions of shock structures involving multiple materials.  LANL has developed oblique shock test problems to evaluate ASC codes in preserving shock structures in multi-physics, multi-material simulations.

LANL plays a crucial role in supporting national security by conducting simulations that span a wide range of complex physics and engineering regimes.  These simulations must accurately preserve shock structures in multi-physics, multi-material simulations used for making informed decisions in critical scenarios.

Simulating oblique shocks in multi-material environments is challenging. The interactions between shock waves and different materials can lead to intricate shock structures, such as double-Mach reflections (DMRs), which are particularly difficult to predict and model.  Accurate simulations require advanced numerical algorithms that can handle these complexities without introducing significant errors.  The challenge is further compounded by the necessity of preserving shock dynamics across different materials, which demands highly precise and robust simulation tools.

To address this challenge, LANL has developed oblique shock test problems to evaluate whether the ASC codes can accurately preserve shock structures in multi-physics, multi-material simulations.  The Verification Test Suite (VTS) was also created to enable regular, automated testing of these codes.  This newly developed test suite allows for the evaluation of code accuracy in scenarios involving multiple shocks, which was previously not possible.  The test setup includes scenarios such as a planar shock traveling along a solid wedge, designed to induce DMRs.  LANL demonstrated a simple experimental configuration: a planar shock in air travels rightward up a solid wedge block.  For well-chosen values of shock strength and wedge inclination, experimental and theoretical results establish the existence of a DMR with Mach stem [1].  The DMR is due to multiple shock reflections off the wedge and shock interactions in air.  LANL can generate figures showing an algorithm that fails this test and an algorithm that passes this test.  With this information, users can improve their intuition with a better understanding of when numerical uncertainly is and is not negligible (LA-UR-24-29492).

[1] G. Ben-Dor. Shock Wave Reflection Phenomena.  2007, Springer

---

LLNL is developing predictive models that connect the microstructure of additive manufacturing samples with material strength and failure behaviors.

In dog-bone tension simulations the model predicts shear localization that resembles experiment and is distinct from isotropic models.

To be more responsive and cost effective, the complex is considering alternative manufacturing methods, and we must be able to rapidly evaluate new materials for national security applications.  Predictive models that connect as-manufactured microstructure and focused experiments to end applications offer a valuable path forward.  Material strength and failure behaviors are particularly sensitive to microstructure and thus to changes in manufacturing.  LLNL employs a multiscale framework where grain-scale simulations of polycrystal response determine a yield surface for as-manufactured microstructures.  For component-scale applications, a macroscopic yield surface model captures dependence on loading conditions and direction (anisotropy).  The as-calibrated model has been exercised in dog-bone tension simulations.  As shown in Figure 7, one obtains different stress-strain response for Ta-2.5W (wt%) samples produced by (i) additive-manufacturing (AM) and (ii) conventional wrought processing – and rather significant differences in ductility. Furthermore, the dog-bone samples (shown in the figure) display fundamental differences in strain patterning.  In particular, the shear localization behavior predicted by LLNL’s anisotropic multiscale model resembles experimentally observed behaviors and is distinct from the predictions of conventional isotropic yield surface models. Comparison to future experiments with digital image correlation for strain field mapping may allow for model discrimination and improved model calibration.  The new modeling capability is currently being evaluated for use in high-explosive-driven experimental conditions.  This work was supported by the ASC Production Simulation Initiative (PSI), with experiments funded by NA-115, Office of Engineering and Technology Maturation.  (LLNL-ABS-871000).

---

ASC experiments have shown that delamination and loss of structural integrity of carbon fiber epoxy composites is significant when exposed to high temperatures.  Qualification and safety assessments on all current and future Life Extension Programs (LEPs)/Alterations (ALTs)/Modifications (MODs) will be impacted, including the W80-4 and W87-1.

The thermal-mechanical response of carbon fiber epoxy composites is not well understood.  However, experiments funded by the ASC program have shown that delamination and loss of structural integrity is significant when these materials are exposed to relatively high temperatures, which can compromise safety.  As simulation capabilities around coupled thermal/mechanical problems are matured, qualification and safety assessments on all current and future LEPs/ALTs/MODs will be impacted, including the W80-4 and W87-1.  Although current efforts are focusing on composite materials, the material models can be leveraged for other materials of interest depending on shifting priorities (SAND2024-09821M).

---

LANL is developing automated approaches to calibrate and distribute models with associated uncertainties across multiple projects.  The team is applying its methods to a specific stockpile stewardship simulation with the goal of delivering this capability to the weapon design community.

The process of developing and propagating new constituent models into stockpile stewardship simulations is often long and arduous.  The complexity of the relevant physics makes updates a technical challenge, and the breadth of organizations required to provide model inputs is an equally difficult communication challenge.  Updates require the expertise and support of numerous vital, tightly integrated collaborations across multiple organizations.
To improve quantitative understanding of the uncertainty in stockpile stewardship simulations, researchers must first capture the uncertainty of the underlying constituent physics models and then propagate it through simulations.  This improvement requires coordination across multiple organizations and programs to provide physically and statistically meaningful populations of plausible models and parameters, rather than just a single best estimate, and to transmit these models to the design community in a form compatible with simulation requirements. 

Over the past year, LANL researchers have made significant progress on all fronts.  These teams developed more automated approaches to parameterize, or calibrate, models across multiple Physics and Engineering Models (PEM) projects.  Teams also refined workflows to automate distributing PEM-developed and PEM-calibrated models and corresponding parameter values, with associated uncertainties, to the design community, enabling uncertainty characterization in simulated outputs.  Importantly, this workflow also allows designers to give feedback to the teams performing calibration. 

Figure 9 demonstrates the emerging workflow applied to simulations of tin ejecta experiments [1].  Within this effort, individual physics models for strength, high explosive (HE), and equation of state were calibrated with a Bayesian approach to a multitude of respectively relevant characterization experiments.  For example, tin strength was calibrated with quasi-static and Hopkinson bar experiments, and HE with explosively driven cylinders and rate stick experiments.  Then, the posterior distributions—which represent the uncertainty in individual model parameters—were all sampled and propagated through simulations of an integral experimental output response distribution.  The individual curves in Figure 9 represent the simulation results of the experimental diagnostic at parameter values sampled from the posterior distributions.

The coordination required to tackle this exemplar problem resulted in forming a cross-organizational inter-project team that is now applying the developed methods to a specific stockpile stewardship simulation.  This more challenging application has highlighted several outstanding challenges for which the team will develop solutions, with the ultimate goal of enabling agile delivery of model calibrations to the weapons design community and rapid feedback to the ASC and PEM communities for further improvements (LA-UR-24-28053).

[1] Buttler, W.T., et al., “Second shock ejecta measurements with an explosively driven two-shockwave drive,” J. Appl. Phys. 116 (110), 103519 (2014)

---

LLNL researchers developed a new capability to predict nuclear scattering properties in data-starved regimes for nuclei across the isotopic chart, including actinides such as plutonium (Pu) and uranium (U).

Predictions show excellent agreement to experiment, without any parameter adjustments.

Developing a predictive theory of neutron scattering is key toward filling critical data needs both in the short and longer terms as neutron elastic and inelastic scattering plays a central role in a broad range of programmatic applications.  Our understanding of the underlying reaction processes is incomplete, making it difficult to resolve ambiguities in the existing data and fill gaps in nuclear databases.  Measurements, which require (a) a neutron beam, (b) neutron detection capabilities, and (c) suitable targets for each isotope of interest, are challenging and cannot address all needs. Remaining experimental difficulties include our inability to distinguish elastic and some inelastic processes, leaving uncertainties regarding the energies of the scattered neutrons. Neutron spectra are also not well-modeled with existing theoretical approaches, which rely on data for parameter adjustments and thus, have limited ability to predict outcomes where no measurements exist. 

To fill these gaps, LLNL developed a new capability that integrates state-of-the-art nuclear structure theory into a modern reaction description: LLNL’s structure code (HFT+QRPA*) predicts nuclear properties for the ground and excited states of nuclei across the isotopic chart, including actinides, such as Pu and U.  Through folding with an effective nucleon-nucleon interaction, LLNL generates optical and transition potentials, which are the key ingredients for predictive scatter calculations with the coupled-channels code, Fresco.  LLNL performed test calculations for spherical, medium-mass Zr isotopes, where elastic and inelastic data exist.  The elastic neutron scattering predictions agree with data and LLNL is able to compete with the best phenomenological approach, but unlike the latter, these predictions have not been adjusted to data.  Similarly, LLNL’s inelastic scattering predictions - tested for proton scattering, where more data exist – show excellent agreement, without any parameter adjustments.  Future work will apply the new computational capability to predict neutron scatter outcomes for isotopes of present programmatic interest, including actinides (LLNL-ABS-870999). 

*HFB+QRPA = Hartree-Fock Boguliubov Theory + Quasiparticle Random Phase Approximation

---

SNL’s Sierra Thermal/Fluids team deployed scalable, massively parallel mesh adaptivity capabilities in Sierra/Aria.  Aging and lifetime projects are using adaptive meshing to reduce computation for organic matter decomposition simulations.  The W80-4 is using it for cost-effective mesh sensitivity studies.

The Sierra Thermal/Fluids team has deployed scalable, massively parallel mesh adaptivity capabilities in Sierra/Aria and is now supporting adaptivity with the majority of Aria’s features.  Many problems of interest, such as fluid flow, chemistry, and heat transfer, exhibit localized physics at length scales much smaller than the overall domain which require fine meshes to resolve computationally.  Using dynamic adaptive meshing, Aria users can now have their simulation meshes automatically refined to capture these phenomena of interest.  This can dramatically reduce computational costs by lowering element counts, as well as save analysts’ time in creating highly specialized meshes.   Aging and lifetime projects are using adaptive meshing to reduce computational costs for organic matter decomposition simulations and the W80-4 program is using it for more cost-effective mesh sensitivity studies (SAND2024-11578M).

 

---

 

Welcome Aboard…

LLNL ASC program

Branson Stephens is the new High-Energy Density (HED) Project Coordination Council (PCC) Project Leader.  In this role, he will be working with the HED code project team to support users on their ongoing programmatic work.  The role will also involve coordinating code comparison work and working to improve the robustness and health of the HED code project.  Branson is also a co-leader of the Defense Science & Technology Internship (DSTI) program in LLNL’s Strategic Deterrence (SD) organization.  Branson joined LLNL in 2016 after working as a software developer for the Laser Interferometer Gravitational-wave Observatory (LIGO) Scientific Collaboration.  He holds a Ph.D. in Physics from the University of Illinois at Champaign-Urbana.

---

LANL ASC program

Kris Eriksen is the new Capabilities for Nuclear Intelligence (CNI) Program Manager at LANL.  Kris has spent his career at the Lab in the integrated design community, working across a range of projects in Verification & Validation (V&V), the Office of Experimental Sciences (OES), DOE’s National Technical Nuclear Forensics (NTNF), and Global Security portfolios.  Kris received his bachelor’s degree in Physics from Middlebury College, and his Ph.D. in Astronomy from the University of Arizona, where he studied the physics of supernova remnants and the interstellar medium.  Kris enjoys running, as well as hiking and skiing with his family in the mountains of Northern New Mexico.
Daniel Holladay is the new LAP Deputy Project Leader for Computer Science under the Integrated Codes subprogram at LANL.  

 

Daniel Holladay first joined the lab as a summer student in 2012 where he worked on the porting xRAGE routines to run on graphics processing units (GPUs) and multicore central processing units (CPUs) using the OpenCL portability framework.  He received a Ph.D. in Nuclear Engineering from Texas A&M University in 2018 while working as a LANL Graduate Research Assistant (GRA).  Daniel’s research interests include investigating the use of performance portability frameworks to relax simplifying assumptions and enable higher-fidelity, multi-physics simulations.  He enjoys the beauty and excellent weather northern New Mexico offers and likes spending time with his family.

 

---

SNL ASC program

Logan Meredith joined the RAMSES/Empire team at SNL in June 2024, following the completion of his Ph.D. at the University of Illinois at Urbana-Champaign, where he was a NNSA Laboratory Residence Graduate Fellow (LRGF).  He is now applying his expertise in plasmas and algorithms for high-performance computing to extend Empire’s capabilities.

---

 

NNSA LDRD/SDRD Quarterly Highlights

 

LLNL LDRD: Accelerating material characterization - machine learning meets X-ray absorption spectroscopy.

LLNL scientists have developed a new approach that can rapidly predict the structure and chemical composition of heterogeneous materials.  In a new study in ACS Chemistry of Materials, LLNL scientists Wonseok Jeong and Tuan Anh Pham developed a new approach that combines machine learning with X-ray absorption spectroscopy (XANES) to elucidate the chemical speciation of amorphous carbon nitrides.  The research offers profound new insights into the local atomic structure of the systems, and in a broader context, represents a critical step in establishing an automated framework for rapid characterization of heterogeneous materials with intricate structures (read more in this article on the LLNL website).

 

SNL LDRD: Quantum computers' unexpected advantage

Theoretical scientists at SNL and Boston University have discovered that quantum computers are unrivaled at solving an advanced math problem.  Unusually, they proved quantum computers are not faster than regular computers; instead, they use far less memory.  The revelation upends the conventional wisdom that the value of a quantum computer is that it can solve certain problems much faster than a normal one.  This could help researchers find more real-world uses for the rapidly advancing technology (read more in the SNL article).

 

SDRD: NNSS Technology Wins an R&D 100 Award

NNSS Senior Principal Scientist, Doug Seastrand, was recently recognized with an R&D 100 Award for the Electromagnetic Spectrum Management System that originated from his 2015 SDRD project (Read more in the article on the NNSS website).

---

Questions? Comments? Contact Us.

NA-114 Office Director: Thuc Hoang, 202-586-7050

• Integrated Codes: Jim Peltz, 202-586-7564
• Physics and Engineering Models/LDRD/SDRD: Anthony Lewis, 202-287-6367
• Verification and Validation/PSAAP/CSGF: David Etim, 202-586-8081
• Capabilities for Nuclear Intelligence: Anthony Lewis, 202-287-6367
• Computational Systems and Software Environment: Si Hammond, 202-586-5748
• Facility Operations and User Support: K. Mike Lang, 301-903-0240