Computing the Result. At first the distinction is made that the factorial part of the equation is going to be a large number, possibly outside of the range of an int type and causing overflow. The second part of the equation involves taking the power of two numbers below 1.0 which will yield a very small number and here significant digits may be truncated. We know a probability MUST be between zero and one so the general algorithm is to interleave the computing of the factorial side and the fractional power side of the equation so that the running result never strays far from being between zero and one. There are a couple optimizations for when the number of successes is zero or equal to the number of trials. Also of note is an optimization on the division of the factorial by factorial. All of the digits in either N-Y or Y are reproduced in the numerator. The greatest number of cancellations is made depending on which is bigger, N-Y or Y, thus reducing significantly the number of multiplications.
the computed result
NuGenBinomialProbability Class | Genetibase.MathX Namespace