1. Home
  2. CORAL-2 Benchmarks
  3. Change Log

Change Log

Updates on 4/5/18 

  • Updated Kripke source and summary to fix a bug in the particle count code.  This has no impact on performance, but does impact validation of code changes.
  • Updated the deep learning source to remove the I/O time from the P3B1 Candle problem.
  • Updated the deep learning summary to reflect the I/O change in P3B1 and explicitly state that the pre-processing time for Resnet is included in the FOM.
  • Updated the FOM spreadsheet to reflect the new FOM for P3B1.

Updates on 3/26/18

  • Updated IOR source to fix bug with verification when using the -W option
  • Updated IOR summary
  • Updated Deep Learning Suite summary
  • Updated MPI summary file to clarify how to run various tests
  • Updated Kripke source code to build on RAJA version 0.6.0
  • Updated Kripke summary file to reflect new FOM collected with new source version
  • Updated FOM spreadsheet with new Kripke and Quicksilver FOMs

Updates on 3/23/18

  • Updated IOR summary to switch from MPIIO to Posix.
  • Updated the Deep Learning summary file with new run rules for Resnet and a new FOM calculation formula.
  • Updated the CORAL Deep learning source to remove I/O from the Candle benchmarks (we missed this last time).
  • Updated the MPI benchmarks to fix bugs with where timers were placed.
  • Fixed a bug in HACC.  The gravity kernel has a value change from f to s2.
  • Performance bug in Quicksilver fixed.  Bounds checking removed from code.
  • Updated summary file to reflect new FOMs for updated code.  A new FOM Spreadsheet is coming soon.
  • Updated the ML/DL skeleton summary file to include sizes for the Deepbench kernels and FFT. 

Updates on 3/20/18

  • Updated FOM Spreadsheet to include deep-learning reference FOMs
  • Updated the deep learning summary to include clarifications for the base rules for Resnet and update Candle reference timings to exclude I/O
  • Updated the deep learning source code to remove I/O from the Candle FOM timing
  • Updated the ML/DL skeleton summary file to include sizes for the Deepbench kernels and FFT [CHANGE: This was not actually updated on 3/20, see 3/23 release]
  • Updated the MPI benchmarks summary to clarify and update the tests to be run
  • Updated RAJA source to include reported timer fixes
  • Updated the RAJA summary file to describe how to reasonably scale the problem size to parallel and accelerator architectures

Updates on 3/13/18

  • Updated Pynamic source configure file to look for libpython.so in addition to libpython.a.
  • Updated Pynamic summary file to make it more clear what to report.
  • Updated FOM spreadsheet to fix the number of nodes used for PCA in the big data suite.
  • Updated Deep Learning summary file with a run rule clarification for portability and to update the ResNet-50 FOM.

Updates on 3/2/18

  • Updated Quicksilver summary and source. Fluence threshold is increased to 6% and nBatches and batch size parameters are removed from all sample job files. These can now be set via command line and an explanation of the -b and -g flags to set them are in the summary file.
  • Updated the Deep learning source and summary file. Changes include fixing random seeds, updates to accuracy numbers and modified the run rules to take into account runtime variability. Baseline numbers are included in the summary and there is a new formula for calculating ResNet. The Candle benchmarks now print the numbers needed for the FOM.
  • Updated Pynamic summary and source. Includes a new parallel build options and makes the big-exe variant of Pynamic disabled by default.
  • Updated HACC source to make it easier to run without the BG/Q intrinsics enabled.
  • Updated the FOM spreadsheet to add 4 MPI rank per node data for Quicksilver and use these numbers as the reference FOM. This is to make our baselines consistent with the Quicksilver rules.

Updates on 2/21/18

  • Updated Deep Learning Suite to add timers to the P3B1 problem
  • Updated Deep Learning Summary file to clarify the FOM calculation of Resnet

Updates on 2/16/18

  • Updated HACC summary file to clarify compilation instructions
  • Updated the deep learning summary file to include Resnet FOM rules and descriptions
  • Updated the LAMMPS source code to change neighbor list build frequency from every 20 to every 10 time steps. This matches the DAT problem that was run.
  • Added benchmarking data for LAMMPS to the supplementary information page
  • Added QMCPack scaling data to the reference FOM spreadsheet

Updates on 2/13/18

  • Updated HACC summary file to clarify choice of problem size, VMAX modifications, and fix a typo about the number of particles.
  • Updated QMCPACK source and summary to reflect CORAL-2 problem size increase of 5x.
  • Updated Benchmark results spreadsheet to include QMCPack FOM.

Updates on 2/12/18

  • Updated AMG summary file to correct reference problem in section 3 of mechanics of building benchmarks
  • Added new I/O skeleton benchmark FTree. Both summary and source code
  • Updated NekBone summary to clarify correctness criteria
  • Added output script from Sequoia for LAMMPS to the supplementary information page

Updates on 2/8/18

  • Updated CORAL deep learning suite summary file to add more information.
  • Added DL/ML micro benchmark summary and source files
  • Updated RAJA Summary and source files. Changes in output from Pass/Fail and clarification of the information sought.
  • Updated AMG source to correct error of printing the wrong FOM.

Updates on 2/6/18

  • Add CLOMP summary file
  • Add updates MDTest summary to ask for extra metrics
  • Updated FOM Spreadsheet to fix issue with copy and paste error for AMG numbers, and add additional Pennant numbers
  • Updated AMG source and summary to correct for a bug in outputting the FOM
  • Update MPI benchmarks to include newest summary file and update README
  • Updated Pennant zip file to include outputs for smaller problem sizes
  • Updated LAMMPS summary file to clarify how to verify correctness.

Updates on 2/1/18

  • Added HACC supplementary information text
  • Updated HACC and Pennant baseline ports to No
  • Actually updated the version of Nekbone to fix the overflow issue
  • Actually updated the version of the RAJA suite to fix build issue
  • Added CLOMP source code
  • Added Draft Deep-learning summary file
  • Updated the MPI-Benchmark summary file to reduce the number of data points needed and make it more clear, which numbers need to be reported
  • Added to supplementary information a spreadsheet with estimates of the memory needs for the CORAL-2 problems.
  • Added supplementary information about LAMMPS
  • Added smaller problem size data for LAMMPS and clarifications about the data presented for the big data analytics suite to the FOM spreadsheet.

Updates on 1/25/18

  • Updated stream source to version 5.10
  • Added text to say whether a baseline GPU version exists or not for each scalable science and throughput benchmark
    NOTE: 2 are still TBD
  • Added in missing slides from Quicksilver supplementary information
  • Updated Quicksilver summary file and source. Corrected timing issues and FOM output bug
  • Added Nekbone GPU information to the Supplementary information Page
  • Update AMG summary and Source to add and describe new option -keepT
  • Updated Nekbone source to fix integer overflow issue
  • Updated FOM spreadsheet to add LAMPPS FOM and Big Data FOM (Corrected error: this previously said "QMCPack FOM;" it should have said "LAMPPS FOM"; it is now correct.)
  • Updated the big data summary file to add in FOM calculation and clarify how run problems

Updates on 1/18/18

  • Added LAMPPS source code (updated 1/18; added to change log 1/25)
  • IOR summary file update (updated 1/18; added to change log 1/25)
  • RAJA source and summary file update (updated 1/18; added to change log 1/25)
  • Added LAMMPS summary file; source code coming soon
  • Updated PENNANT source file; addition GPU source file in the sup info page
  • Updated Kripke summary file
  • Updated AMG summary file
  • Updated Deep Learning suite source code
  • Added IO source code and summary files for MDTest and Simul

Updates on 1/16/18

  • Updated AMR source which fixes reported timing bug
  • Updated IOR summary
  • Updated RAJA source
  • Fixed Kripke Supplementary download links

Updates on 1/10/18

  • Added IOR source
  • Added RAJA Performance Suite Source
  • Added link to RFP page
  • Added a supplementary information page with links to GPU versions and papers about application scaling.
  • Added a change log that will be used for this update forward
  • Moved GEMM micro-benchmark into ML/DL micro-benchmark suite
  • Removed the 1/24th problem FOM and problem from the Pennant summary file
  • Updated the reference FOM spreadsheet to reflect previous changes to benchmarks and add new data
  • Removed XGBoost from the BigData Suite
  • Added Stride and Stream tarballs