OFFICE OF ADVANCED SIMULATION AND COMPUTING AND INSTITUTIONAL R&D PROGRAMS (NA-114)

Welcome to the fourth 2022 issue of the NA-114 newsletter - published quarterly to socialize the impactful work being performed by the NNSA laboratories and our other partners. This issue begins with a highlight from Lawrence Livermore National Laboratory (LLNL) in which two teams supported by NA-114 and NA-115 made substantial progress in developing and testing “Configuration Transfer” capabilities, specifically the ability to initialize a physics performance model from an engineering model by transferring both deformed geometry and simulation field variables from engineering codes to physics codes. This new capability will support the W87-1 Modification Program. Other highlights include:

• Acceleration of thermal battery design at Sandia National Laboratories (SNL) using digital engineering.
• LLNL’s acceptance of three new Commodity Technology Systems-2 (CTS-2) – “Mutt,” “Whippet,” and “Poodle” - Trilab resources that will serve computing needs for stockpile stewardship.
• Upgrades to Los Alamos National Laboratory’s (LANL’s) Rapid Iterative Feedback Tool which increases its speed, accuracy, and usability for nuclear weapons safety simulations.
• SNL’s development of a new coupled multi-physics simulation capability to characterize random vibration reentry environments (a warhead re-entry SPARC simulation is shown in the banner image above).

Please join me in thanking the professionals who delivered the achievements highlighted in this newsletter and on an ongoing basis, all in support of our national security mission.

Thuc Hoang
NA-114 Office Director

LLNL weapons program partnership supported by NA-114 and NA-115 develops transfer capability between engineering and physics models for simulations aiding W87-1 Modification Program.

Working in close partnership at LLNL, the Weapon Survivability (WS) team, supported by NA-115, and the Weapon Simulation and Computing (WSC) program, supported by NA-114, have made substantial progress in developing and testing “Configuration Transfer” capabilities. Configuration transfer is defined as the ability to initialize a physics performance model from an engineering model by transferring both deformed geometry and simulation field variables from engineering codes to physics codes. Until recently, the capability had only been demonstrated on a three-dimensional (3D) simplified engineering model to 3D physics model without transferring field variables. The WSC code development team recently created the capability to transfer from a 3D engineering wedge model into a two-dimensional (2D) physics model. 3D wedge models are commonly used in engineering to allow for fast throughput and analysis flexibility. The code development team is also developing techniques for de-featuring the geometrically complicated engineering models as well as robustly transferring field variables, such as density and temperature, between engineering and physics models. A successful initialization of a 2D physics performance calculation based on a pre-loaded engineering model was performed for the W87-1 Modification Program. Additionally, the capability has been handed off to a new user for a different system. The new user was able to learn the modeling technique and complete their simulation within a week, highlighting the capability’s ease of use and computational efficiency. (LLNL-ABS-841900)

SNL’s Power Sources Technology Group accelerates thermal battery design using Digital Engineering.

Thermal batteries are a key component in the normal function of weapon, providing power for components to perform critical functions at prescribed voltages and for a required duration. The design of a thermal battery is no small feat. It requires trade-off decisions to be made regarding battery mass, volume, and architecture while managing heat generation - all in the service of maintaining the desired voltage(s) over a duration of use (and with enough margin). Traditionally, battery design has been guided by years of expertise in the power sources team, relying on decades of successfully fielded war reserve (WR) designs. In the last decade, this historical knowledge captured in design guides was enhanced by advances in the Thermally Activated Battery Simulator (TABS) modeling tool, which is well suited to evaluating the thermal performance of specific designs. After a conceptual point design is defined, physical builds, functional (performance) tests, and thermal modeling help engineers to optimize the design. With the goal of accelerating the full design cycle, Sandia’s Power Source Technology Group has teamed with the ASC program and its Accelerated Digital Engineering (ADE) initiative to develop a more comprehensive digital approach to designing batteries. The digital approach combines two crucial elements: existing virtual test capability (TABS), powered by physics-based modeling and simulation developed by the ASC program, and a new and complementary trade-space exploration tool built using a set-based concurrent engineering commercial off-the-shelf (COTS) tool (Success Assured®). The combination of these digital design tools enables a much faster turn-around time to define all feasible battery architectures, explore the full trade-space, prioritize design attributes, and virtually assess function with modeling and simulation.

As a result, design teams can benefit from marked improvements in turn-around time for viable battery designs. A recent design study explored different thermal battery architectures, shown in Figure 1. In a matter of weeks, the Thermal Battery design team was able to assess four functionally performant options in a next-level system design. Building on this effort, these tools can even assess the trade-offs between next-level system architecture and thermal battery design before requirements are set or hardware builds are committed. (SAND2022-12715 O)

Livermore Computing accepts first computer systems for ASC Program’s Commodity Technology Systems-2 procurement (CTS-2) to serve computing needs for stockpile stewardship for NNSA.

The first systems for the CTS-2 procurement cycle were officially accepted on September 8th by Livermore Computing (LC) from the vendor Dell Technologies. This marks the fourth generation of joint procurements for the tri-Labs under the CTS model. Systems can be sized-to-need based on Scalable Unit (SU) building blocks hosting up to 192 individual 2-socket CPU nodes. The procurement is expected to deliver more than 21,000 central processing units (CPUs) and 65 peta floating-point operations/second (petaFLOPS) of computing in aggregate across the complex.

CTS-2 is a Tri-lab (LLNL, LANL, and SNL) joint procurement to provide a common hardware platform for robust capacity computing. The CTS model provides a common simulation and operating environment when coupled to the Tri-lab Operating System Stack (TOSS) and Tri-Lab Common Environment (TCE) user environment. The contract allows systems to be deployed at the NNSA labs and can be leveraged by other NNSA sites as well, for benefits including reduced total cost of ownership, rapid deployment of systems into production, and support for a spectrum of high-performance computing (HPC) demands.

The CTS-2 systems are based on Dell C6620 servers and Intel Sapphire Rapids Xeon CPUs with up to 256 Gigabytes of DDR5 memory per node. The systems use Cornelis Networks Omni-Path for the cluster interconnect. Other advanced features for power and cooling efficiency include CoolIT direct-to-chip liquid cooling. Options exist for graphics processing units (GPUs) from three major vendors to be procured for individual systems if desired by the sites. The initial systems deployed at LLNL are Mutt (32 nodes), Whippet (192 nodes), and Poodle (32 nodes) following a naming convention of dog breeds for systems deployed on the unclassified network. As part of the system acceptance activity, a simulation was performed on 12 nodes of the Mutt system using the ALE3D application code, which demonstrated performance gains on par with the expected speedups over previous generation Intel Xeon processors. (LLNL-ABS-841839)

Upgrades to LANL’s Rapid Iterative Feedback Tool increase its speed, accuracy, and usability for nuclear weapons safety simulations.

An ability to quickly address concerns about consequences of high explosive detonation is a principal challenge for the weapon safety community. Answering questions from a production facility commonly requires executing dozens of large 3D simulations. Thanks to ASC codes and high-performance computers, these simulations can now be completed in times as short as a few days.

Safety simulations generate terabytes of complex data that need to be evaluated by experts. To speed up and improve the accuracy of this analysis, LANL has developed the Rapid Iterative Feedback Tool (or RIFT). This capability assists safety modelers with the task of comparing simulation results with experimental results, such as for the Cyclops experiment shown in Figure 3.

Significant improvements have been made to RIFT over the last year. Better algorithms have increased its speed and accuracy for edge detection and overlays. Memory balancing has increased the speed when running in parallel across multiple processors. The graphical user interface (GUI) itself was also improved with overlay features to allow custom transparency values and color palettes. Through these improvements, RIFT is becoming an essential tool for the workflow within the safety community. (LA-UR-22-28533)

SNL researchers develop a new coupled multi-physics simulation capability to characterize random vibration reentry environments.

At hypersonic speeds, reentry vehicles face a harsh environment from aerodynamic heating and severe, unsteady, turbulence-induced pressure fluctuations on the vehicle body, termed the "random vibration" environment. The amplitude of these pressure fluctuations often induces a vibration response of the vehicle body that can propagate to the vehicle's internal components, thereby raising survivability concerns for all structural components of the vehicle. Engineers predict the structural response to the random vibration environment using an approximate model to characterize the behavior of the turbulence-induced pressure fluctuations in space and time. These models inherently fail to capture the complete physics of the turbulence, therefore improved representations of the random vibration environment are needed to support environmental specification and qualification of current and future stockpile reentry systems. A more general approach directly utilizes the turbulence-induced loading from a high-fidelity aerodynamics simulation to predict the structural response in a coupled fashion. Sandia successfully demonstrated this approach in predicting the structural response of a flexible panel on the surface of a representative conical reentry geometry subjected to hypersonic turbulence. Once validated, the suite of random vibration models at multiple fidelities will provide the means for augmenting limited flight test data and will be included in the design and qualification process of reentry systems.

The coupling is accomplished in two steps: first, a high-fidelity computational fluid dynamics simulation using the SPARC code is performed to predict the turbulence-induced forcing-function; then, the unsteady loading over the plate is passed to a Sierra/Structural Dynamics (Sierra/SD) solver to predict the panel dynamics. The simulation results, generated on ASC Astra and Sierra high-performance computing platforms, have initially been compared to experimental data from a Sandia hypersonic wind tunnel experiment. This capability provides an avenue to subject a flight vehicle/test specimen to the random-vibration environment it would encounter during flight (Figure 4, left graph), and to determine the structural response of an internal component to that environment (Figure 4, right graph). Initial comparisons of both the loading environment and structural response of the model to the experimental data show that the model accurately predicts the correct behavior over a wide frequency range of interest. A more rigorous validation study is underway, in which both the approximate and high-fidelity random vibration models will be compared to wind tunnel and flight test data, with uncertainties included. (SAND2022-11239 O)

KCNSC developed a high-fidelity computational fluid dynamics model for the flex cable electroplating tank, providing insight into the physical mixing process and positively impacting maintenance costs and schedules.

KCNSC developed and deployed a robust turbomachinery mixing model.

Through the use of simulation, the KCNSC reduced settling and fallout in an electrochemical plating tank for flex cable production by identifying a more optimal configuration for the impeller. This reduced maintenance costs and schedules on the tank by minimizing the need to add electroplating agent. KCNSC implemented an immersed mesh method in OpenFoam to overcome the difficulty in modeling the spinning impeller. This new method maintains numerical stability with acceptable levels of accuracy while modeling the moving boundary. The development of this method at KCNSC has enabled additional applications for production simulation where objects are moving through fluids and fluid boundaries are rapidly changing, including direct ink write nozzle mixing, bubbles in foams, and microfluidics.(NSC-614-4674 UUR)

LLNL Physics and Engineering Model’s newly developed wide-ranged multiphase equation of state for iron will benefit various computational studies ranging from manufacturing to planetary science.

A new 5-phase equation of state (EOS) model, L263, for elemental iron (Fe) has been created, by fitting to: 1) legacy static and dynamic ambient and high-pressure experimental data, 2) ramp compression data obtained at the National Ignition Facility (NIF), and 3) a collection of Density Functional Theory Molecular Dynamics (DFT-MD) predictions for pressure and temperature regimes where experimental data is absent. The construction made use of the MEOS code, which allowed for a treatment of the liquid free energy according to both the Purgatorio model (for electronic excitations) and the cell model (for the high-T ion-thermal contribution), and a liquid-vapor transition region was included to allow for a description of hot, under-dense situations.

Besides the application to new and old manufacturing studies where Fe is used as a component within various alloys, this EOS development is expected to have a major impact on the study of the interiors of Earth-like and super-Earth planets, as well as planetary collision phenomena. For this reason, the choice was made to construct two separate variations to the new Fe EOS model: version-1 is fit to the T_melt(P) inferences of Anzellini et al. (as well as others showing similar results), while version-2 is fit to the lower T_melt(P) measurements of Sinmyo et al. (and others). This will allow scientists to assess the effect of this uncertainty on their predictions of planetary phenomena which depend sensitively on these details and have yet to be fully resolved experimentally.

The use of the NIF ramp compression data (R.F. Smith et al., 2018, Nature Astronomy) as a constraint was crucial for addressing regimes of high compression (r > 15 g/cc) not otherwise visited by static and single-shock measurements. Both versions of the L263 EOS fit these data equally well; this is also the case for ambient, lower-pressure static, and dynamic compression data with the notable exception of shock-melting data which is only fit well by version-1.

The DFT-MD predictions were similarly crucial to obtain, as they provided validation for both the cold compressibility model and the choice of latent heat (and associated density change) between high-P solid and liquid phases. These computational predictions of P(ρ,T) and E(ρ,T) were obtained by performing calculations on 256 Fe atoms in a computational box over 1 picosecond, which was made possible by exploiting the large parallel computing resources of the ASC Quartz machine at LLNL. A detailed manuscript on the construction of L263, together with its initial application to studies pertaining to the Earth’s core-mantle boundary, is nearing completion. (LLNL-ABS-841899)

LANL researchers improve accuracy of material strength models for multiple metals in extreme conditions using large-scale quantum molecular dynamics.

The strength of solid metal components is important for the performance of both conventional and nuclear defense technologies. However, this aspect of material behavior can only be directly measured under a narrow range of laboratory conditions, and in some cases inferred from experiments under limited sets of more extreme environments. An accurate, simple analytic model of strength (for fast evaluation) is an essential part of numerical multi-physics simulations supporting stockpile stewardship.

Such an analytic model was introduced 20 years ago at LANL by Burakovsky, Greeff and Preston. It was recently generalized to be applicable to materials that have more than one solid phase, which is the case for our national security applications. Theoretical input for development of this strength model is developed with quantum molecular dynamics, which operates in terms of the so-called pseudopotentials that are based on the electronic structure of the material. Whereas calculations based on classical methods are usually unreliable beyond a compression of about two, the region of the validity of these quantum methods is much wider and typically extends to a compression of 10. While the quantum calculations are extremely time consuming, they are nevertheless reliable: simulations using contemporary, first principles-based codes are accurate to within 10% at all densities and temperatures.

Over the last five years, LANL’s new models have been systematically used for the construction of the melt-shear tables for: Ag, Be, Ce, Cr, Cu, Mo, Nb, Pb, Pt, Pu, Sn, Ta, Ti, U, V, W; LiF, Ti64 (Ti-6Al-4V), U₆Nb, and UO₂. This effort has required about 10 million CPU-hours, i.e., an average of 2 million CPU-hours per year and an average of 0.5 million CPU-hours per substance. The most challenging materials required 1-2 million CPU hours each. Newer heterogeneous architectures, which have GPUs that are very efficient for these types of calculations, offer the promise of speeding up calculations by a factor of five or more. (LA-UR-22-27097)

SNL demonstrates a new Cable System Generated Electromagnetic Pulse analysis tool to support hostile environment qualification for the W87-1.

Cable System Generated Electromagnetic Pulse (SGEMP) is a phenomenon of concern when a reentry system encounters a hostile radiation environment. When X-rays impact a cable, multiple physical phenomena interact to induce significant currents on the cable that can interfere with its proper functioning. Computational tools help provide evidence to system and component qualification teams that weapon system cables will meet survivability requirements. EMPIRE-Cable is a new tool in the EMPIRE suite that is being developed to support upcoming needs from the W87-1 system, replacing the legacy CABANA tool and expanding on its capability.

As a demonstration of the new capability, the team repeated a comparison against an experiment conducted at the Pithon facility that was originally conducted using CABANA. The results, shown in Figure 11, demonstrate that EMPIRE-Cable reproduces the CABANA solution, and shows good agreement with the experimental data. This was achieved despite significant differences in the underlying mathematical approach between CABANA and EMPIRE-Cable. The new approach is intended to support development of future capabilities needed to analyze the cables being designed for the W87-1. (SAND2022-10261 O)

LANL improves modeling capabilities for onset and evolution of failure in ductile materials.

A new capability for modeling damage in materials has been implemented in ASC codes at LANL. Accurate calculations for materials failure, and the details of how materials crack or fragment when subject to extreme conditions, have been a longstanding challenge for stockpile stewardship. Until recently, the solvers used for the complex set of coupled equations describing these processes were not robust enough for use in many important applications. Implementation of an improved implicit solver opens new possibilities for predictive design, assessments of aging phenomena, and exploration of other physics such as liquid cavitation. Figure 12 illustrates the new capability through a comparison with an interesting experiment that was performed to study effect of initial scratches in metal materials. A metal annulus (in this case, aluminum 6061) was scratched to varying depths to impose initial damage and filled with a polymer plug. A polymer projectile was then fired from a gas gun at the specimen, eventually impacting the polymer plug. The compression wave provides the traveling radial loading to split the annulus. Simulations carried out using the new ASC model show excellent agreement with the experimental results. (LA-UR-22-27097)

NNSA LDRD Quarterly Highlights

LLNL research finds mechanically driven chemistry accelerates reactions in explosives.

Scientists at the LLNL Energetic Materials Center and Purdue University Materials Engineering Department used simulations performed on the LLNL supercomputer, Quartz, to uncover a general mechanism that accelerates chemistry in detonating explosives critical to managing the nation’s nuclear stockpile. Their research is featured in the July 15 issue of the Journal of Physical Chemistry Letters. (LLNL-WEB-458451, see the full article)

Figure 14: Reactive molecular dynamics simulations predict that a complicated distribution of molecular states is generated in explosive hotspots formed under shock conditions. Distinct classes of molecular states are indicated by color. Deformations of molecule shape are found to accelerate chemical reactions in a similar way as increasing temperature, explaining why hotspots react faster than expected.

A nuclear hunter-killer for pathogens.

A new kind of treatment delivers a radioactive payload directly to the infectious bacteria within the body.

LANL researchers find a new way to battle drug-resistant bacterial infections by taking a cue from a type of cancer therapy that is currently in development. This new treatment delivers radioactive isotopes directly to the harmful bacteria within the body, thereby killing the bacteria with extremely localized irradiation. Other researchers at LANL and Lawrence Berkeley National Laboratories have already begun testing different aspects of the treatment against infections such as pneumonic plague. (LA-UR-21-32363, see the full article)

Figure 15: To find and kill dangerous pathogens (blue) within the body, an antibody (purple) that selectively binds to an antigen on the surface of the pathogen is linked to a chelator molecule (orange) designed to carry ions of a radioactive isotope with a brief half-life. Concentrated radiation from the attached isotope delivers a lethal dose to the pathogen before the isotope’s

SNL LDRD projects benefit from intern participation at Historically Black Colleges and Universities (HBCUs).

Sandia is sharpening its focus on select HBCUs with its Securing Top Academic Research and Talent (START) program. START builds academic partnerships that align with Sandia’s mission needs to fuel research collaboration and expose prospective underrepresented students to cutting-edge national laboratory work. In turn, Sandia creates an employee-recruiting pipeline for some of the strongest engineering talent in the U.S. in a competitive market. (SAND2022-8564L, see the full article)

Figure 16: Brooke Davis, a summer intern from University of Arkansas at Pine Bluff, prepares algae samples for a microbiome investigation at SNL. (Photo by Craig Fritz)

Questions? Comments? Contact Us.

NA-114 Office Director: Thuc Hoang, 202-586-7050

• Integrated Codes: Jim Peltz, 202-586-7564
• Physics and Engineering Models/LDRD: Anthony Lewis, 202-287-6367
• Verification and Validation: David Etim, 202-586-8081
• Computational Systems and Software Environment: Si Hammond, 202-586-5748
• Facility Operations and User Support: K. Mike Lang, 301-903-0240
• Advanced Technology Development and Mitigation: Thuc Hoang