diff --git a/Doc/UserManual/AppendixListings.tex b/Doc/UserManual/AppendixListings.tex index 2792ad3e6c73a8db918e86e7ce7cc990f7dbfbed..1721d164b3dc30dac141fa72962ac309d66568f2 100644 --- a/Doc/UserManual/AppendixListings.tex +++ b/Doc/UserManual/AppendixListings.tex @@ -157,7 +157,7 @@ def run_fitting(): simulation = get_simulation() simulation.setSample(sample) - real_data = OutputDataIOFactory.getOutputData('Refdata_fitcylinderprisms.txt') + real_data = OutputDataIOFactory.readIntensityData('Refdata_fitcylinderprisms.txt') fit_suite = FitSuite() fit_suite.addSimulationAndRealData(simulation, real_data) diff --git a/Doc/UserManual/Fitting.tex b/Doc/UserManual/Fitting.tex index d443f7242f08bc3f02f7048d602d351ab840e532..0a4b119438f37b42a5448e518e2643e1c7a26e22 100644 --- a/Doc/UserManual/Fitting.tex +++ b/Doc/UserManual/Fitting.tex @@ -23,9 +23,10 @@ Advanced fitting techniques, including fine tuning of minimization algorithms, s \input{FittingExamples} +\input{FittingAdvanced} + \input{FittingRightAnswers} -\input{FittingAdvanced} diff --git a/Doc/UserManual/FittingExamples.tex b/Doc/UserManual/FittingExamples.tex index aed863074d28dcdb0d9b20246e582ad61b82d547..8571bed9e03806c4966d1cd4f728dead295ba09d 100644 --- a/Doc/UserManual/FittingExamples.tex +++ b/Doc/UserManual/FittingExamples.tex @@ -108,7 +108,7 @@ def run_fitting(): @\label{script2::run_fitting}@ simulation = get_simulation() simulation.setSample(sample) @\label{script2::setup_simulation2}@ - real_data = OutputDataIOFactory.getOutputData('Refdata_fitcylinderprisms.txt') @\label{script2::real_data}@ + real_data = OutputDataIOFactory.readIntensityData('Refdata_fitcylinderprisms.txt') @\label{script2::real_data}@ \end{lstlisting} Lines ~\ref{script2::setup_simulation1}-~\ref{script2::setup_simulation2} generate sample and simulation description and assign the sample to the simulation. @@ -172,3 +172,9 @@ During the fitting the progress will be displayed on the screen. Lines ~\ref{script2::fitresults2}--~\ref{script2::fitresults3} shows different ways of accessing to fit results. + +More details about fitting, access to its results and visualization of fit progress using matplotlib libraries can be learned from detailed example +\begin{lstlisting}[language=shell, style=commandline] +./Examples/python/fitting/ex002_FitCylindersAndPrisms/FitCylindersAndPrisms_detailed.py +\end{lstlisting} + diff --git a/Doc/UserManual/FittingGentleIntroduction.tex b/Doc/UserManual/FittingGentleIntroduction.tex index 594599527a535dd50bdb421593f3ed93f3e0397a..d865406c630434db7cdba2a2f0758d29ec7dc7a6 100644 --- a/Doc/UserManual/FittingGentleIntroduction.tex +++ b/Doc/UserManual/FittingGentleIntroduction.tex @@ -42,7 +42,7 @@ to the detector noise. The rest of the formula represents our signal. Lets define our model, namely specific mathematical function, to which we will fit our toy experimental data. By making an educated guess we assume that scattering intensity observed in the experiment should be described with the help of $sinc$ function as follows -$$ I(x,y) = p_0 + p_1 \cdot sinc(x - p_2) \cdot sinc(y - p_3) $$ +$$ f(x,y) = p_0 + p_1 \cdot sinc(x - p_2) \cdot sinc(y - p_3) $$ The model has four parameters: $p_0$ describing background, $p_{1}$ describing signal strength and $p_2,p_3$ responsible for the peak position. Fig.~\ref{fig:toyfit_data},right shows the intensity as a function (x,y) calculated according @@ -83,9 +83,49 @@ $-\text{ln}(\mathcal{L})$. \subsubsection*{$\chi^2$ or least squares minimization} -A simple dataset consist of $n$ data pairs +A dataset consist of $n$ data pairs $(\mathbf{x_{i}}, a_{i}), i=1,N$ where +$\mathbf{x_{i}}$ is an independent variable and $a_{i}$ is dependent variable, whose +value is found in the measurement $i$. The number $N$ denotes the total +number of measurements. + +In the case of intensity map measured in our toy experiment +and presented in Fig~\ref{fig:toyfit_data}, a variable $a_{i}$ denotes measured intensity, a variable $\mathbf{x_{i}}$ is a vector and correspond to the +$(x_{i}, y_{i})$ coordinates of pixels in our detector while number $N$ corresponds +to total number of detector pixels. + +The model function has the form +$f(\mathbf{x_{i}},\mathbf{p})$ +where adjustable parameters are held in the vector $\mathbf{p}$. +The least squared method finds the optimum of model function which +fit the data in the best way by searching for the minimum of the sum of squared +residuals +$$ \chi^{2}(\mathbf{p}) = \sum_{i=1}^{N}r_{i}^{2}$$ +where residual is defined as the difference between measured value and the value predicted by the model. +$$r = a_{i} - f(\mathbf{x_{i}},\mathbf{p})$$ + +In the case of normally distributed variables with the $\sigma^2$ variance +the quantity to minimize becomes +$$ \chi^{2}(\mathbf{p}) = +\frac{1}{d} +\sum_{i=1}^{N} +\frac{ (a_{i} - f(\mathbf{x_{i}},\mathbf{p}))^2}{\sigma^2} $$ +where $d=N-k$ is number of degree of freedom ($k$ number of free fit parameters). + + +\subsubsection*{Minimization algorithms} +There are a large number of minimization algorithms providing a solution to the problem +of minimizing the objective function over the space of parameters of the function. +The minimization starts from initial guess for the parameters provided by the user, +and then evolves iteratively under control of minimization algorithm. The procedural +modifications on the parameters, the objective function, as well as convergence +criterion depend on the method implemented. +Details of particular implementation is beyond the scope of this manual and + interested reader is encouraged to look at outside resources. + + + +\subsubsection*{Local minima trap} -\subsubsection*{Main features of minimization algorithm} diff --git a/Doc/UserManual/FittingImplementation.tex b/Doc/UserManual/FittingImplementation.tex index ebff1c1765143953c3ac92d3582c70a695d6b57f..4a32b9082dee0be7bad2d7e9b5dd6d394b36e230 100644 --- a/Doc/UserManual/FittingImplementation.tex +++ b/Doc/UserManual/FittingImplementation.tex @@ -243,7 +243,7 @@ before minimization starts. They are handled by \Code{MinimizerOptions} class: \begin{lstlisting}[language=python, style=eclipseboxed, numbers = none] options = MinimizerOptions() options.setMaxFunctionCalls(10) -FitSuite().getMinimizer().setOptions(options) +fit_suite.getMinimizer().setOptions(options) \end{lstlisting} In given code snippet a number of ``maximum function calls'', namely a number of times the minimizer is allowed to call the simulation, is limited to the 10. The minimizer will take that number into consideration and will try to limit number of iterations by that value. @@ -257,3 +257,18 @@ They will be explained in \SecRef{FittingAdvanced}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsection{Running the fitting ant retrieving the results.} +After the initial configuration of \Code{FitSuite} has been performed, the fitting +can be started using the command +\begin{lstlisting}[language=python, style=eclipseboxed, numbers = none] +fit_suite.runFit() +\end{lstlisting} + +Depending on complexity of the sample and number of free sample parameters the fitting +process can go from tenths to thousands of iterations. The results of the fit can +be printed on the screen using the command +\begin{lstlisting}[language=python, style=eclipseboxed, numbers = none] +fit_suite.printResults() +\end{lstlisting} +\SecRef{FittingExamples} gives more details about access to fitting results. + + diff --git a/Doc/UserManual/FittingRightAnswers.tex b/Doc/UserManual/FittingRightAnswers.tex index 55e1885c516c06bdad093630be1c4b2515a66f53..c2767255eac160ebdf49b910c194412f0d5dc4be 100644 --- a/Doc/UserManual/FittingRightAnswers.tex +++ b/Doc/UserManual/FittingRightAnswers.tex @@ -1,9 +1,41 @@ -\section {How to get right answer from \BornAgain\ fitting.} +\section {How to get right answer from fitting.} \SecLabel{FittingRightAnswers} +As it was already mentioned in \SecRef{FittingGentleIntroducion}, +one of the main difficulties in fitting the data with the model +is the presence of multiple +local minima in the objective function. The extended list of problems causing fit to failure includes \begin{itemize} -\item It is recommended to start from default minimizer settings and turn to the fine tunings -only after some experience has been acquired. -\item error interpretation +\item unreliable physical model +\item multiple local minima +\item unphysical behavior of objective function, unphysical regions in parameter space +\item unreliable parameter error calculation in the presence of limits on parameter value +\item often exponential behavior of objective function and corresponding numerical inaccuracies and excessive numerical roundoff in calculation of its value and derivatives +\item large correlations between parameters +\item very different scale of parameters involved in calculation +\item not positive definite error matrix even at minimum +\end{itemize} + + +Given list, of course, is unrelated only to \BornAgain\ fitting. It remains the same +while fitting the data with any fitting program and any kind of theoretical model. +To address all these difficulties some amount of hand work might be necessary. Below we give some recommendations which might help the user to achieve reliable fit results. + +\subsection*{General recommendation} +\begin{itemize} +\item initially choose small number of free fitting parameters +\item provide a good initial guess for fit parameters +\item start from default minimizer settings and turn to the fine tuning after some experience has been acquired. +\item repeat fit using different starting values for parameters or their limits +\item repeat fit fixing and releasing different groups of parameters +\item use \Code{Minuit2} minimizer with \Code{Migrad} algorithm (default) to get most reliable parameter error estimation +\item use \Code{GSLMultiFit} minimizer or \Code{Minuit2} minimizer with \Code{Fumili} algorithm to get less interation as possible + + +%\subsection*{Interpretation of errors.} + + +{\bf to be continued... } + \end{itemize}