Skip to content

Commit d956f82

Browse files
author
Albert Akhriev
committed
report and comments
1 parent c8af98a commit d956f82

6 files changed

Lines changed: 76 additions & 46 deletions

File tree

‎doc/amdados_report.tex‎

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
%opening
77
\title{Allscale AMDADOS Application \\ Techinal Report}
8-
\author{Albert Akhriev}
8+
\author{Albert Akhriev \\ Add Yourself}
99

1010
\begin{document}
1111
\maketitle
@@ -22,15 +22,19 @@ \section{Overview}
2222
\section{Working with Amdados application}
2323
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2424

25+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2526
\subsection{Building the application.}
27+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2628
\begin{enumerate}
2729
\item The application executable must be available before any test (even in Python) is run.
2830
\item We recommend the standard way for building the application presented in the script ``\textit{standard.build.sh}'' in the project root folder.
2931
\item Another useful script ``./scripts/download.sh'' downloads the latest Allscale API and the Armadillo library for unit tests.
3032
\item We do \textit{not} recommend the development script for building the application ``\textit{mybuild}'', which relies on ramdisk and other development specific features. Just for completeness, we provide script's options: \texttt{-f} clears any previous build and starts from scratch; \texttt{-r/-d} release/debug mode; \texttt{-t} runs tests after building the project. For example: \texttt{./mybuild -f -r -t}.
3133
\end{enumerate}
3234

35+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3336
\subsection{Application parameters.}
37+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3438
\begin{enumerate}
3539
\item The application is controlled by the configuration file. The default one can be found in the project root folder under the name ``\textit{amdados.conf}''. The most interesting parameters are: the integration period and the number of sub-domains in either dimension.
3640
\item Not everything can be controlled by configuration file. For example, the size of sub-domain is hard-coded because of using of templates in C++ grid implementation.
@@ -39,7 +43,9 @@ \subsection{Application parameters.}
3943
\item The default configuration file ``\textit{amdados.conf}'' contains brief description of each parameter sufficient to match it to the source code.
4044
\end{enumerate}
4145

46+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
4247
\subsection{The application structure}
48+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
4349
\begin{enumerate}
4450
\item In order to demonstrate a data assimilation process we need a ``true'' solution or, in other words, a ``true state of nature''. The reason is twofold. First, this ground-truth is used for the accuracy assessment of data assimilation. Second, the \textit{input data} ``measured'' at sensor locations across the domain govern the simulation by pushing it towards the ``true'' solution.
4551
\item In Amdados use-case scenario, the \textit{input data are simulated} by direct forward solver (see equation (\ref{?})) in the entire domain. The solver is written in Python. For simplicity, the Python forward solver does not implement any sub-domain partitioning, operating in the entire domain directly, but uses the same number of nodal points as the C++ Amdados application, the same integration period and the same flow model, see the functions \texttt{Flow()} in both C++ and Python implementations.
@@ -63,34 +69,52 @@ \subsection{The application structure}
6369
where \textit{analytic} is the \textsf{data type prefix}, $N_x=176$ is the total number of nodal points in $x$ dimension, $N_y=176$ is the total number of nodal points in $y$ dimension, $N_t=1225$ is number of time integration step, and \textit{txt/bin} is the extension of a text or binary file respectively. The number of time steps is omitted in the name of sensor locations file because sensors do not change their positions over time.
6470
\end{enumerate}
6571

72+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
6673
\subsection{Running the Amdados simulation}
74+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
6775
\subsubsection{Single run}
6876
\begin{enumerate}
6977
\item Step into the project root directory.
70-
\item Create a configuration file using whatever settings you need. It is not recommended to modify the default file ``\textit{amdados.conf}'' or tweak any parameter except the number of sub-domains in either direction and the integration time (unless you understand all the implications). Also, note that the size of a sub-domain must coincide with the hard-coded size in C++ code, where it is used for template instantiation. We advice to use ``\textit{amdados.conf}'' as a starting point.
78+
\item Create a configuration file using whatever settings you need. It is not recommended to modify the default file ``\textit{amdados.conf}'' or tweak any parameter, except for the number of sub-domains in either direction and the integration time, unless you understand all the implications. Also, note that the size of a sub-domain must coincide with the hard-coded size in C++ code, where it is used for template instantiation. We advice to use ``\textit{amdados.conf}'' as a starting point.
7179
\item Build \texttt{Amdados} executable in ``Release'' mode:
7280
$$
7381
\mbox{\small\texttt{bash standard.build.sh}}
7482
$$
75-
The building script extracts the output directory from ``\textit{amdados.conf}''. If your choice is different, please, modify the simple file ``\textit{standard.build.sh}'' accordingly.
83+
The building script extracts the output directory from ``\textit{amdados.conf}''. If your preferences are different, please, modify the self-explanatory file ``\textit{standard.build.sh}'' accordingly.
7684
\item If you want to get rid of any previous stuff in the output folder it is worth to do this right after building the executable. Note, all the simulation artefacts will go into the output directory.
7785
\item Run the Python script to generate a file of sensor locations (if it is missed), the file of observations and the file of full-field snapshots:
7886
$$
7987
\mbox{\small\texttt{python3 python/ObservationsGenerator.py --config your.conf}}
8088
$$
81-
If not specified, the configuration file ``\textit{amdados.conf}'' will be used. The command \texttt{--help} lists all the options. Upon completion the following files will be created in the output folder:
82-
$$\mbox{\small\texttt{sensors\_Nx*\_Ny*.txt}}$$
83-
$$\mbox{\small\texttt{analytic\_Nx*\_Ny*\_Nt*.txt}}$$
84-
$$\mbox{\small\texttt{true\_field\_Nx*\_Ny*\_Nt*.bin}}$$
89+
If not specified, the configuration file ``\textit{amdados.conf}'' will be used. The command \texttt{--help} lists all the options. Upon completion the following files will be created in the output folder: \newline
90+
\hspace*{4em}{\small\texttt{sensors\_Nx*\_Ny*.txt}} \newline
91+
\hspace*{4em}{\small\texttt{analytic\_Nx*\_Ny*\_Nt*.txt}} \newline
92+
\hspace*{4em}{\small\texttt{true\_field\_Nx*\_Ny*\_Nt*.bin}} \newline
8593
where the values under symbols '$*$' depend on current settings.
8694
\end{enumerate}
95+
8796
\subsubsection{Size scalability test}
8897
\begin{enumerate}
89-
\item fff
98+
\item The Python script ``\textit{python/ScalabilityTestSize.py}'' runs several simulations with increasing problem size. The doc-string at the beginning of the script provides further details.
99+
\item The script should be launched without any argument provided the \texttt{Amdados} application was compiled.
100+
\item There are two script's variables that user should modify according to his/her requirements: \texttt{GridSizes} and \texttt{IntegrationPeriod}. The former is a set of problem sizes (number of sub-domains in both dimensions). Note, the large sizes can result in weeks (if not months) of simulation. As a rule of thumb, a size should not exceed $100{\times}100$ sub-domains. Considering the default size of a sub-domain is $16{\times}16$ points, the whole domain size would be $1600{\times}1600$ nodal points --- quite large problem. The latter variable gives the integration time in seconds.
101+
\item Besides the aforementioned variables \texttt{GridSizes} and \texttt{IntegrationPeriod}, \newline which have to be selected by user, everything else is done automatically.
102+
\item Once the test had finished, the results, stored in the output directory, can be visualized, see Section~\ref{sec:visual} for further details.
90103
\end{enumerate}
104+
91105
\subsubsection{Multi-threading scalability test}
92106
\begin{enumerate}
93-
\item fff
107+
\item The Python script ``\textit{python/ScalabilityTestMT.py}'' mostly repeates the functionality of the previous one except runs several simulations with the same problem size but increasing the number of CPU cores (threads). The doc-string at the beginning of the script provides further details.
108+
\item The same variables \texttt{GridSizes} and \texttt{IntegrationPeriod} should be modified by user. No input arguments are required.
109+
\item Once the test had finished, the results, stored in the output directory, can be visualized, see Section~\ref{sec:visual} for further details.
110+
\item \textbf{Important}: we strongly recommend not to use the same size or integration period as in the test ``\textit{python/ScalabilityTestSize.py}'', otherwise some result files can be overwritten with the loss of test information.
111+
\end{enumerate}
112+
113+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
114+
\subsubsection{Visualization}\label{sec:visual}
115+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
116+
\begin{enumerate}
117+
\item fff
94118
\end{enumerate}
95119

96120
\end{document}

‎python/Configuration.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -----------------------------------------------------------------------------
22
# Author : Albert Akhriev, albert_akhriev@ie.ibm.com
3-
# Copyright : IBM Research Ireland, 2017
3+
# Copyright : IBM Research Ireland, 2017-2018
44
# -----------------------------------------------------------------------------
55

66
import re, os

‎python/ObservationsGenerator.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -----------------------------------------------------------------------------
22
# Author : Albert Akhriev, albert_akhriev@ie.ibm.com
3-
# Copyright : IBM Research Ireland, 2017
3+
# Copyright : IBM Research Ireland, 2017-2018
44
# -----------------------------------------------------------------------------
55

66
#import pdb; pdb.set_trace() # enables debugging

‎python/ScalabilityTestMT.py‎

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,28 +4,27 @@
44
# -----------------------------------------------------------------------------
55

66
""" This script runs several simulations with the same problem size but
7-
increasing the number of CPU cores. It saves the execution time of each
8-
simulation in a file that can be used to plot the scalability profile.
9-
One can modify the problem size and integration period, see the few global
10-
variables at the beginning of the script.
7+
increasing the number of CPU cores. One can modify the default problem
8+
size and integration period, see the few global variables at the beginning
9+
of the script.
1110
Each simulation is twofold. First, we run the Python forward solver that
12-
generates the ground-truth and observations. The Python code itself uses
13-
the C++ code running in the special mode for generating sensor locations
14-
(scenario "sensors"). Second, the C++ data assimilation application is run
15-
using the observations previously generated by the Python code (scenario
16-
"simulation").
11+
generates the ground-truth and observations ("ObservationsGenerator.py").
12+
The Python code itself uses the C++ code running in the special mode for
13+
generating sensor locations (scenario "sensors"). Second, the C++ data
14+
assimilation application is launched (scenario "simulation") with
15+
observations generated by "ObservationsGenerator.py".
1716
The results of all the simulations are accumulated in the output
1817
directory and can be visualized later on by the script "Visualize.py".
1918
The configuration file "amdados.conf" is used in all the simulations with
2019
modification of three parameters: grid sizes (number of subdomains) in both
21-
dimensions and integration time. The other parameters remain intact. It is
22-
not recommended to tweak parameters unless you understand what is done.
23-
If you modified the parameters, please, consider to rerun this script
24-
as the results in the output directory as not valid any longer.
20+
dimensions and integration time. Other parameters remain intact. It is not
21+
recommended to tweak parameters unless their meaning is absolutely clear.
22+
If you had modified the parameters, please, consider to rerun this script
23+
because the results in the output directory a not valid any longer.
2524
The script was designed to fulfil the formal requirements of the
2625
Allscale project.
27-
2826
"""
27+
print(__doc__)
2928

3029
#import pdb; pdb.set_trace() # enables debugging
3130
import matplotlib
@@ -91,8 +90,8 @@
9190
# Run C++ data assimilation application with different number
9291
# of working threads.
9392
print("##################################################")
94-
print("Simulation by 'amdados' application, which will be")
95-
print("silent if debugging & messaging were disabled ... ")
93+
print("Simulation by 'amdados' application ...")
94+
print("silent if debugging & messaging were disabled")
9695
print("##################################################")
9796
proc = subprocess.Popen([AMDADOS_EXE,
9897
"--scenario", "simulation",
@@ -109,11 +108,14 @@
109108
exe_time_profile)
110109

111110
# Plot and save the scalability profile.
111+
problem_size = (GridSize[0] * round(conf.subdomain_x),
112+
GridSize[1] * round(conf.subdomain_y))
112113
plt.plot(exe_time_profile[:,0], exe_time_profile[:,1])
113-
plt.xlabel("number of CPU cores")
114+
plt.xlabel("number of CPU cores (threads)")
114115
plt.ylabel("time in seconds")
115116
plt.grid()
116-
plt.title("Multi-threading scalability")
117+
plt.title("Multi-threading scalability, problem size: " +
118+
str(problem_size[0]) + "x" + str(problem_size[1]))
117119
plt.savefig(os.path.join(conf.output_dir, "scalability_mt.png"))
118120

119121
except subprocess.CalledProcessError as error:

‎python/ScalabilityTestSize.py‎

Lines changed: 19 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,28 @@
44
# -----------------------------------------------------------------------------
55

66
""" This script runs several simulations with increasing problem size
7-
utilizing all available CPU resources. It saves the execution time of each
7+
utilizing all available CPUs. It saves the execution time of each
88
simulation in a file that can be used to plot the scalability profile.
9-
One can modify the problem sizes and integration period, see the few global
10-
variables at the beginning of the script.
9+
Essential parameters, listed in the first lines, include the set of
10+
problem sizes and integration period.
1111
Each simulation is twofold. First, we run the Python forward solver that
12-
generates the ground-truth and observations. The Python code itself uses
13-
the C++ code running in the special mode for generating sensor locations
14-
(scenario "sensors"). Second, the C++ data assimilation application is run
15-
using the observations previously generated by the Python code (scenario
16-
"simulation").
12+
generates the ground-truth and observations ("ObservationsGenerator.py").
13+
The Python code itself uses the C++ code running in the special mode for
14+
generating sensor locations (scenario "sensors"). Second, the C++ data
15+
assimilation application is launched (scenario "simulation") with
16+
observations generated by "ObservationsGenerator.py".
1717
The results of all the simulations are accumulated in the output
1818
directory and can be visualized later on by the script "Visualize.py".
1919
The configuration file "amdados.conf" is used in all the simulations with
2020
modification of three parameters: grid sizes (number of subdomains) in both
21-
dimensions and integration time. The other parameters remain intact. It is
22-
not recommended to tweak parameters unless you understand what is done.
23-
If you modified the parameters, please, consider to rerun this script
24-
as the results in the output directory as not valid any longer.
21+
dimensions and integration time. Other parameters remain intact. It is not
22+
recommended to tweak parameters unless their meaning is absolutely clear.
23+
If you had modified the parameters, please, consider to rerun this script
24+
because the results in the output directory a not valid any longer.
2525
The script was designed to fulfil the formal requirements of the
2626
Allscale project.
2727
"""
28+
print(__doc__)
2829

2930
#import pdb; pdb.set_trace() # enables debugging
3031
import matplotlib
@@ -48,7 +49,8 @@
4849
# This is a reasonable set of problems, be patient for days to come ...
4950
#GridSizes = [(11,11), (19,17), (23,25), (37,31), (43,41), (83,89)]
5051
# Small problems for a relatively brief testing.
51-
GridSizes = [(13,11), (17,13), (19,15), (23,21), (25,23), (27,25)]
52+
#GridSizes = [(13,11), (17,13), (19,15), (23,21), (25,23), (27,25)]
53+
GridSizes = [(13,11), (18,16), (23,21), (29,27), (34,32), (39,37)]
5254

5355
# Integration period in seconds.
5456
IntegrationPeriod = 9000
@@ -62,7 +64,8 @@
6264
# Read configuration file.
6365
conf = Configuration("amdados.conf")
6466
# Create the output directory, if it does not exist.
65-
if not os.path.isdir(conf.output_dir): os.mkdir(conf.output_dir)
67+
if not os.path.isdir(conf.output_dir):
68+
os.mkdir(conf.output_dir)
6669
# Check existence of "amdados" application executable.
6770
assert os.path.isfile(AMDADOS_EXE), "amdados executable was not found"
6871
# Save some parameters that will be used for visualization.
@@ -91,8 +94,8 @@
9194

9295
# Run C++ data assimilation application.
9396
print("##################################################")
94-
print("Simulation by 'amdados' application, which will be")
95-
print("silent if debugging & messaging were disabled ... ")
97+
print("Simulation by 'amdados' application ...")
98+
print("silent if debugging & messaging were disabled")
9699
print("##################################################")
97100
subprocess.run([AMDADOS_EXE, "--scenario", "simulation",
98101
"--config", config_file], check=True)

‎python/Visualize.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -----------------------------------------------------------------------------
22
# Author : Albert Akhriev, albert_akhriev@ie.ibm.com
3-
# Copyright : IBM Research Ireland, 2018
3+
# Copyright : IBM Research Ireland, 2017-2018
44
# -----------------------------------------------------------------------------
55

66
""" Script visualizes simulation results given a path to field*.txt file.
@@ -12,6 +12,7 @@
1212
Note, the script visualizes results of one particular simulation.
1313
The reason for that is to have flexibility for debugging.
1414
"""
15+
print(__doc__)
1516

1617
#import pdb; pdb.set_trace() # enables debugging
1718
import numpy as np

0 commit comments

Comments
 (0)