diff --git a/docs/source/benchmarking.rst b/docs/source/benchmarking.rst index 01cf210a..82c5f653 100644 --- a/docs/source/benchmarking.rst +++ b/docs/source/benchmarking.rst @@ -9,105 +9,12 @@ This section provides information and results from performance benchmarking of g How to benchmark? ================= -The following simple models (found in the ``tests/benchmarking`` sub-package) can be used to benchmark gprMax on your own system. The models feature different domain sizes (from 100^3 to 450^3 cells) and contain a simple Hertzian dipole source in free space. The following shows an example of the 100^3 cell model: +The following simple models (found in the ``tests/benchmarking`` sub-package) can be used to benchmark gprMax on your own system. The models feature different domain sizes (from 100^3 to 800^3 cells) and contain a simple Hertzian dipole source in free space. The following shows an example of the 100^3 cell model: - -.. literalinclude:: ../../tests/benchmarking/bench_100x100x100.in +.. literalinclude:: ../../testing/benchmarking/bench_100x100x100.in :language: none :linenos: - -Using the following steps to collect and report benchmarking results for each of the models: - -1. Run gprMax in benchmarking mode, e.g. ``python -m gprMax tests/benchmarking/bench_100x100x100.in -benchmark`` -2. Use the ``plot_benchmark`` module to create plots of the execution time and speed-up, e.g. ``python -m tests.benchmarking.plot_benchmark tests/benchmarking/bench_100x100x100.npz``. You can combine results into a single plot, e.g. e.g. ``python -m tests.benchmarking.plot_benchmark tests/benchmarking/bench_100x100x100.npz --otherresults tests/benchmarking/bench_150x150x150.npz``. -3. Share your data by emailing us your Numpy archives and plot files to info@gprmax.com - -Results: CPU -============ - -Mac OS X --------- - -iMac15,1 -^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.png - :width: 600px - - Execution time and speed-up factor plots for Python/Cython-based gprMax and previous (v.2) C-based code. - -.. note:: - - Zero threads indicates that the code was compiled serially, i.e. without using OpenMP. - -The results demonstrate that the Python/Cython-based code is faster, in these two benchmarks, than the previous version which was written in C. It also shows that the performance scaling with multiple OpenMP threads is better with the C-based code. Results from the C-based code show that when it is compiled serially the performance is approximately the same as when it is compiled with OpenMP and run with a single thread. With the Python/Cython-based code this is not the case. The overhead in setting up and tearing down the OpenMP threads means that for a single thread the performance is worse than the serially-compiled version. - -iMac15,1 -^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1.png - :width: 600px - -MacPro1,1 -^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.png - :width: 600px - - -MacPro3,1 -^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.png - :width: 600px - - -Linux ------ - -Dell PowerEdge R630 -^^^^^^^^^^^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/Dell_PowerEdge_R630.png - :width: 600px - -Lenovo System x3650 M5 -^^^^^^^^^^^^^^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/Lenovo_System_x3650_M5.png - :width: 600px - -SuperMicro SYS-7048GR-TR -^^^^^^^^^^^^^^^^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/Supermicro_SYS-7048GR-TR.png - :width: 600px - - -Windows -------- - -Lenovo T430 -^^^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/Windows7/Lenovo_T430/Lenovo_T430.png - :width: 600px - -Dell Z420 -^^^^^^^^^ - -.. figure:: ../../tests/benchmarking/results/cpu/Windows7/Dell_Z420/DellZ420.png - :width: 600px - - - -Results: GPU -============ - -NVIDIA GPUs ------------ - The performance metric used to measure the throughput of the solver is: .. math:: @@ -116,5 +23,5 @@ The performance metric used to measure the throughput of the solver is: where P is the throughput in millions of cells per second; NX, NY, and NZ are the number of cells in domain in the x, y, and z directions; NT is the number of time-steps in the simulation; and T is the runtime of the simulation in seconds. -.. figure:: ../../tests/benchmarking/results/gpu/NVIDIA.png +.. figure:: ../../testing/benchmarking/results/NVIDIA.png :width: 600px diff --git a/testing/benchmarking/results/NVIDIA.png b/testing/benchmarking/results/NVIDIA.png new file mode 100644 index 00000000..c371c9cf Binary files /dev/null and b/testing/benchmarking/results/NVIDIA.png differ diff --git a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/Dell_PowerEdge_R630.png b/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/Dell_PowerEdge_R630.png deleted file mode 100644 index d5750bae..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/Dell_PowerEdge_R630.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_100x100x100.npz b/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_100x100x100.npz deleted file mode 100755 index c5535486..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_150x150x150.npz b/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_150x150x150.npz deleted file mode 100755 index 29b411d1..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_200x200x200.npz b/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_200x200x200.npz deleted file mode 100755 index e556f78e..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Dell_PowerEdge_R630/bench_200x200x200.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/Lenovo_System_x3650_M5.png b/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/Lenovo_System_x3650_M5.png deleted file mode 100644 index 2e14ebd3..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/Lenovo_System_x3650_M5.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_100x100x100.npz b/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_100x100x100.npz deleted file mode 100644 index 69bd7c8c..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_150x150x150.npz b/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_150x150x150.npz deleted file mode 100644 index 47059503..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_200x200x200.npz b/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_200x200x200.npz deleted file mode 100644 index 0ffec1d9..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/Lenovo_System_x3650_M5/bench_200x200x200.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/Supermicro_SYS-7048GR-TR.png b/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/Supermicro_SYS-7048GR-TR.png deleted file mode 100644 index 75c84882..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/Supermicro_SYS-7048GR-TR.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_100x100x100.npz b/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_100x100x100.npz deleted file mode 100755 index 37abc1fa..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_150x150x150.npz b/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_150x150x150.npz deleted file mode 100755 index 90777e87..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_200x200x200.npz b/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_200x200x200.npz deleted file mode 100755 index 16612469..00000000 Binary files a/testing/benchmarking/results/cpu/Linux/SuperMicro_SYS-7048GR-TR/bench_200x200x200.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1-v.3.0.0b21.png b/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1-v.3.0.0b21.png deleted file mode 100644 index 3d0686bb..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1-v.3.0.0b21.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.npz b/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.npz deleted file mode 100644 index 12723e72..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.png b/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.png deleted file mode 100644 index 04e0222e..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/Apple_MacPro1,1.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/bench_100x100x100.npz b/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/bench_100x100x100.npz deleted file mode 100644 index 68360b7c..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/bench_150x150x150.npz b/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/bench_150x150x150.npz deleted file mode 100644 index 5691f7a2..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro1,1/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.npz b/testing/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.npz deleted file mode 100644 index 7da7e08a..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.png b/testing/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.png deleted file mode 100644 index fd93b16b..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/MacPro3,1/Apple_MacPro3,1.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.npz deleted file mode 100644 index b0c6d7ed..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.png b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.png deleted file mode 100644 index 8b909ff2..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1+Ccode.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1-v.3.0.5.png b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1-v.3.0.5.png deleted file mode 100644 index d41289da..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1-v.3.0.5.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1-v.3.0.7.png b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1-v.3.0.7.png deleted file mode 100644 index b67c82f1..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1-v.3.0.7.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1.png b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1.png deleted file mode 100644 index 2b5a5cfe..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/Apple_iMac15,1.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/bench_100x100x100.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/bench_100x100x100.npz deleted file mode 100644 index 3fdc9974..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/bench_150x150x150.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/bench_150x150x150.npz deleted file mode 100644 index 7fdae79c..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/Apple_iMac15,1.png b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/Apple_iMac15,1.png deleted file mode 100644 index 2b5a5cfe..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/Apple_iMac15,1.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_100x100x100.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_100x100x100.npz deleted file mode 100644 index f66db4fd..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_150x150x150.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_150x150x150.npz deleted file mode 100644 index 351acf20..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_200x200x200.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_200x200x200.npz deleted file mode 100644 index 51e49b90..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_200x200x200.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_300x300x300.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_300x300x300.npz deleted file mode 100644 index ce3baaf7..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_300x300x300.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_400x400x400.npz b/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_400x400x400.npz deleted file mode 100644 index e8becb38..00000000 Binary files a/testing/benchmarking/results/cpu/MacOSX/iMac15,1/v.3.1.0b2/bench_400x400x400.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Windows7/Dell_Z420/DellZ420.png b/testing/benchmarking/results/cpu/Windows7/Dell_Z420/DellZ420.png deleted file mode 100644 index 72b61344..00000000 Binary files a/testing/benchmarking/results/cpu/Windows7/Dell_Z420/DellZ420.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Windows7/Dell_Z420/bench_100x100x100.npz b/testing/benchmarking/results/cpu/Windows7/Dell_Z420/bench_100x100x100.npz deleted file mode 100644 index ada5cd56..00000000 Binary files a/testing/benchmarking/results/cpu/Windows7/Dell_Z420/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Windows7/Dell_Z420/bench_150x150x150.npz b/testing/benchmarking/results/cpu/Windows7/Dell_Z420/bench_150x150x150.npz deleted file mode 100644 index 2355c118..00000000 Binary files a/testing/benchmarking/results/cpu/Windows7/Dell_Z420/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/Lenovo_T430.png b/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/Lenovo_T430.png deleted file mode 100644 index ca471b7d..00000000 Binary files a/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/Lenovo_T430.png and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/bench_100x100x100.npz b/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/bench_100x100x100.npz deleted file mode 100644 index 6e3e8b06..00000000 Binary files a/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/bench_100x100x100.npz and /dev/null differ diff --git a/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/bench_150x150x150.npz b/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/bench_150x150x150.npz deleted file mode 100644 index 98633b7d..00000000 Binary files a/testing/benchmarking/results/cpu/Windows7/Lenovo_T430/bench_150x150x150.npz and /dev/null differ diff --git a/testing/benchmarking/results/gpu/NVIDIA.png b/testing/benchmarking/results/gpu/NVIDIA.png deleted file mode 100644 index 11f02456..00000000 Binary files a/testing/benchmarking/results/gpu/NVIDIA.png and /dev/null differ