From d8ca6f19e3d52875e013b9baac510b9574134082 Mon Sep 17 00:00:00 2001
From: Gennady Pospelov <g.pospelov@fz-juelich.de>
Date: Wed, 6 Nov 2013 18:02:30 +0100
Subject: [PATCH] Finalizing fitting manual
---
Doc/UserManual/AppendixListings.tex | 2 +-
Doc/UserManual/Fitting.tex | 3 +-
Doc/UserManual/FittingExamples.tex | 8 +++-
Doc/UserManual/FittingGentleIntroduction.tex | 46 ++++++++++++++++++--
Doc/UserManual/FittingImplementation.tex | 17 +++++++-
Doc/UserManual/FittingRightAnswers.tex | 40 +++++++++++++++--
6 files changed, 105 insertions(+), 11 deletions(-)
diff --git a/Doc/UserManual/AppendixListings.tex b/Doc/UserManual/AppendixListings.tex
index 2792ad3e6c7..1721d164b3d 100644
--- a/Doc/UserManual/AppendixListings.tex
+++ b/Doc/UserManual/AppendixListings.tex
@@ -157,7 +157,7 @@ def run_fitting():
simulation = get_simulation()
simulation.setSample(sample)
- real_data = OutputDataIOFactory.getOutputData('Refdata_fitcylinderprisms.txt')
+ real_data = OutputDataIOFactory.readIntensityData('Refdata_fitcylinderprisms.txt')
fit_suite = FitSuite()
fit_suite.addSimulationAndRealData(simulation, real_data)
diff --git a/Doc/UserManual/Fitting.tex b/Doc/UserManual/Fitting.tex
index d443f7242f0..0a4b119438f 100644
--- a/Doc/UserManual/Fitting.tex
+++ b/Doc/UserManual/Fitting.tex
@@ -23,9 +23,10 @@ Advanced fitting techniques, including fine tuning of minimization algorithms, s
\input{FittingExamples}
+\input{FittingAdvanced}
+
\input{FittingRightAnswers}
-\input{FittingAdvanced}
diff --git a/Doc/UserManual/FittingExamples.tex b/Doc/UserManual/FittingExamples.tex
index aed863074d2..8571bed9e03 100644
--- a/Doc/UserManual/FittingExamples.tex
+++ b/Doc/UserManual/FittingExamples.tex
@@ -108,7 +108,7 @@ def run_fitting(): @\label{script2::run_fitting}@
simulation = get_simulation()
simulation.setSample(sample) @\label{script2::setup_simulation2}@
- real_data = OutputDataIOFactory.getOutputData('Refdata_fitcylinderprisms.txt') @\label{script2::real_data}@
+ real_data = OutputDataIOFactory.readIntensityData('Refdata_fitcylinderprisms.txt') @\label{script2::real_data}@
\end{lstlisting}
Lines ~\ref{script2::setup_simulation1}-~\ref{script2::setup_simulation2} generate
sample and simulation description and assign the sample to the simulation.
@@ -172,3 +172,9 @@ During the fitting the progress will be displayed on the screen.
Lines ~\ref{script2::fitresults2}--~\ref{script2::fitresults3} shows different ways of
accessing to fit results.
+
+More details about fitting, access to its results and visualization of fit progress using matplotlib libraries can be learned from detailed example
+\begin{lstlisting}[language=shell, style=commandline]
+./Examples/python/fitting/ex002_FitCylindersAndPrisms/FitCylindersAndPrisms_detailed.py
+\end{lstlisting}
+
diff --git a/Doc/UserManual/FittingGentleIntroduction.tex b/Doc/UserManual/FittingGentleIntroduction.tex
index 594599527a5..d865406c630 100644
--- a/Doc/UserManual/FittingGentleIntroduction.tex
+++ b/Doc/UserManual/FittingGentleIntroduction.tex
@@ -42,7 +42,7 @@ to the detector noise. The rest of the formula represents our signal.
Lets define our model, namely specific mathematical function, to which we will fit our toy experimental data. By making an educated guess we assume that scattering intensity observed
in the experiment should be described with the help of $sinc$ function as follows
-$$ I(x,y) = p_0 + p_1 \cdot sinc(x - p_2) \cdot sinc(y - p_3) $$
+$$ f(x,y) = p_0 + p_1 \cdot sinc(x - p_2) \cdot sinc(y - p_3) $$
The model has four parameters: $p_0$ describing background, $p_{1}$ describing signal strength
and $p_2,p_3$ responsible for the peak position.
Fig.~\ref{fig:toyfit_data},right shows the intensity as a function (x,y) calculated according
@@ -83,9 +83,49 @@ $-\text{ln}(\mathcal{L})$.
\subsubsection*{$\chi^2$ or least squares minimization}
-A simple dataset consist of $n$ data pairs
+A dataset consist of $n$ data pairs $(\mathbf{x_{i}}, a_{i}), i=1,N$ where
+$\mathbf{x_{i}}$ is an independent variable and $a_{i}$ is dependent variable, whose
+value is found in the measurement $i$. The number $N$ denotes the total
+number of measurements.
+
+In the case of intensity map measured in our toy experiment
+and presented in Fig~\ref{fig:toyfit_data}, a variable $a_{i}$ denotes measured intensity, a variable $\mathbf{x_{i}}$ is a vector and correspond to the
+$(x_{i}, y_{i})$ coordinates of pixels in our detector while number $N$ corresponds
+to total number of detector pixels.
+
+The model function has the form
+$f(\mathbf{x_{i}},\mathbf{p})$
+where adjustable parameters are held in the vector $\mathbf{p}$.
+The least squared method finds the optimum of model function which
+fit the data in the best way by searching for the minimum of the sum of squared
+residuals
+$$ \chi^{2}(\mathbf{p}) = \sum_{i=1}^{N}r_{i}^{2}$$
+where residual is defined as the difference between measured value and the value predicted by the model.
+$$r = a_{i} - f(\mathbf{x_{i}},\mathbf{p})$$
+
+In the case of normally distributed variables with the $\sigma^2$ variance
+the quantity to minimize becomes
+$$ \chi^{2}(\mathbf{p}) =
+\frac{1}{d}
+\sum_{i=1}^{N}
+\frac{ (a_{i} - f(\mathbf{x_{i}},\mathbf{p}))^2}{\sigma^2} $$
+where $d=N-k$ is number of degree of freedom ($k$ number of free fit parameters).
+
+
+\subsubsection*{Minimization algorithms}
+There are a large number of minimization algorithms providing a solution to the problem
+of minimizing the objective function over the space of parameters of the function.
+The minimization starts from initial guess for the parameters provided by the user,
+and then evolves iteratively under control of minimization algorithm. The procedural
+modifications on the parameters, the objective function, as well as convergence
+criterion depend on the method implemented.
+Details of particular implementation is beyond the scope of this manual and
+ interested reader is encouraged to look at outside resources.
+
+
+
+\subsubsection*{Local minima trap}
-\subsubsection*{Main features of minimization algorithm}
diff --git a/Doc/UserManual/FittingImplementation.tex b/Doc/UserManual/FittingImplementation.tex
index ebff1c17651..4a32b9082de 100644
--- a/Doc/UserManual/FittingImplementation.tex
+++ b/Doc/UserManual/FittingImplementation.tex
@@ -243,7 +243,7 @@ before minimization starts. They are handled by \Code{MinimizerOptions} class:
\begin{lstlisting}[language=python, style=eclipseboxed, numbers = none]
options = MinimizerOptions()
options.setMaxFunctionCalls(10)
-FitSuite().getMinimizer().setOptions(options)
+fit_suite.getMinimizer().setOptions(options)
\end{lstlisting}
In given code snippet a number of ``maximum function calls'', namely a number of times the minimizer is allowed to call the simulation, is limited to the 10. The minimizer will take that number into consideration and will try to limit number of iterations by that value.
@@ -257,3 +257,18 @@ They will be explained in \SecRef{FittingAdvanced}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Running the fitting ant retrieving the results.}
+After the initial configuration of \Code{FitSuite} has been performed, the fitting
+can be started using the command
+\begin{lstlisting}[language=python, style=eclipseboxed, numbers = none]
+fit_suite.runFit()
+\end{lstlisting}
+
+Depending on complexity of the sample and number of free sample parameters the fitting
+process can go from tenths to thousands of iterations. The results of the fit can
+be printed on the screen using the command
+\begin{lstlisting}[language=python, style=eclipseboxed, numbers = none]
+fit_suite.printResults()
+\end{lstlisting}
+\SecRef{FittingExamples} gives more details about access to fitting results.
+
+
diff --git a/Doc/UserManual/FittingRightAnswers.tex b/Doc/UserManual/FittingRightAnswers.tex
index 55e1885c516..c2767255eac 100644
--- a/Doc/UserManual/FittingRightAnswers.tex
+++ b/Doc/UserManual/FittingRightAnswers.tex
@@ -1,9 +1,41 @@
-\section {How to get right answer from \BornAgain\ fitting.}
+\section {How to get right answer from fitting.}
\SecLabel{FittingRightAnswers}
+As it was already mentioned in \SecRef{FittingGentleIntroducion},
+one of the main difficulties in fitting the data with the model
+is the presence of multiple
+local minima in the objective function. The extended list of problems causing fit to failure includes
\begin{itemize}
-\item It is recommended to start from default minimizer settings and turn to the fine tunings
-only after some experience has been acquired.
-\item error interpretation
+\item unreliable physical model
+\item multiple local minima
+\item unphysical behavior of objective function, unphysical regions in parameter space
+\item unreliable parameter error calculation in the presence of limits on parameter value
+\item often exponential behavior of objective function and corresponding numerical inaccuracies and excessive numerical roundoff in calculation of its value and derivatives
+\item large correlations between parameters
+\item very different scale of parameters involved in calculation
+\item not positive definite error matrix even at minimum
+\end{itemize}
+
+
+Given list, of course, is unrelated only to \BornAgain\ fitting. It remains the same
+while fitting the data with any fitting program and any kind of theoretical model.
+To address all these difficulties some amount of hand work might be necessary. Below we give some recommendations which might help the user to achieve reliable fit results.
+
+\subsection*{General recommendation}
+\begin{itemize}
+\item initially choose small number of free fitting parameters
+\item provide a good initial guess for fit parameters
+\item start from default minimizer settings and turn to the fine tuning after some experience has been acquired.
+\item repeat fit using different starting values for parameters or their limits
+\item repeat fit fixing and releasing different groups of parameters
+\item use \Code{Minuit2} minimizer with \Code{Migrad} algorithm (default) to get most reliable parameter error estimation
+\item use \Code{GSLMultiFit} minimizer or \Code{Minuit2} minimizer with \Code{Fumili} algorithm to get less interation as possible
+
+
+%\subsection*{Interpretation of errors.}
+
+
+{\bf to be continued... }
+
\end{itemize}
--
GitLab