Examples of com.numb3r3.common.opt.QNMinimizer

com.numb3r3.common.math.Matrix
This code is part of the Stanford NLP Toolkit.

An implementation of L-BFGS for Quasi Newton unconstrained minimization.
The general outline of the algorithm is taken from:
Numerical Optimization (second edition) 2006 Jorge Nocedal and Stephen J. Wright
A variety of different options are available.

LINESEARCHES

BACKTRACKING: This routine simply starts with a guess for step size of 1. If the step size doesn't supply a sufficient decrease in the function value the step is updated through step = 0.1*step. This method is certainly simpler, but doesn't allow for an increase in step size, and isn't well suited for Quasi Newton methods.
MINPACK: This routine is based off of the implementation used in MINPACK. This routine finds a point satisfying the Wolfe conditions, which state that a point must have a sufficiently smaller function value, and a gradient of smaller magnitude. This provides enough to prove theoretically quadratic convergence. In order to find such a point the linesearch first finds an interval which must contain a satisfying point, and then progressively reduces that interval all using cubic or quadratic interpolation.

SCALING: L-BFGS allows the initial guess at the hessian to be updated at each step. Standard BFGS does this by approximating the hessian as a scaled identity matrix. To use this method set the scaleOpt to SCALAR. A better way of approximate the hessian is by using a scaling diagonal matrix. The diagonal can then be updated as more information comes in. This method can be used by setting scaleOpt to DIAGONAL.

CONVERGENCE: Previously convergence was gauged by looking at the average decrease per step dividing that by the current value and terminating when that value because smaller than TOL. This method fails when the function value approaches zero, so two other convergence criteria are used. The first stores the initial gradient norm |g0|, then terminates when the new gradient norm, |g| is sufficiently smaller: i.e., |g| < eps*|g0| the second checks if |g| < eps*max( 1 , |x| ) which is essentially checking to see if the gradient is numerically zero.
Each of these convergence criteria can be turned on or off by setting the flags:
private boolean useAveImprovement = true; private boolean useRelativeNorm = true; private boolean useNumericalZero = true;

To use the QNMinimizer first construct it using
QNMinimizer qn = new QNMinimizer(mem, true)
mem - the number of previous estimate vector pairs to store, generally 15 is plenty. true - this tells the QN to use the MINPACK linesearch with DIAGONAL scaling. false would lead to the use of the criteria used in the old QNMinimizer class.
Then call:
qn.minimize(dfunction,convergenceTolerance,initialGuess,maxFunctionEvaluations);
@author akleeman

        return new InMemoryJBlasMatrix(DoubleMatrix.eye(n), null);
    }


    @Override
    public Pair getDims() {
        return new Pair(this.getRowsNum(), this.getColumnsNum());
    }

View Full Code Here

    public InMemoryJBlasMatrix(final int rowNum, final int colNum,
                               final ErrorProcessor errorProcessor) {
        if (errorProcessor != null) {
            this.errorProcessor = errorProcessor;
        } else {
            this.errorProcessor = new SystemErrorProcessor();
        }


        if (rowNum == 0 && colNum == 0) {
            throw new UnsupportedOperationException(
                    "Matrix should have more than zero elements.");

View Full Code Here

    public InMemoryJBlasMatrix(DoubleMatrix matrix,
                               final ErrorProcessor errorProcessor) {
        if (errorProcessor != null) {
            this.errorProcessor = errorProcessor;
        } else {
            this.errorProcessor = new SystemErrorProcessor();
        }
        this.doubleMatrix = matrix;
    }

View Full Code Here


        double part1 = Math.exp(-0.5 * k * Math.log(2 * Math.PI));


        double part2 = Math.pow(det, -0.5);


        Matrix dev = value.sub(mu);


        double part3 = Math.exp(-0.5
                * (dev.transpose().product(cox.inverse())).dot(dev));
        return part1 * part2 * part3;


    }

View Full Code Here


        double part1 = Math.exp(-0.5 * k * Math.log(2 * Math.PI));


        double part2 = Math.pow(det, -0.5);


        Matrix dev = value.sub(mu);


        double part3 = Math.exp(-0.5 * (dev.transpose().product(inv)).dot(dev));
        return part1 * part2 * part3;


    }

View Full Code Here

        else
            return PhiInverse(y, delta, mid, hi);
    }


    public static Matrix oneCox(int k) {
        Matrix cox = InMemoryJBlasMatrix.eye(k);


        return null;
    }

View Full Code Here


        return null;
    }


    public static Matrix sample(Matrix mu, Matrix Sigma) {
        Matrix values = InMemoryJBlasMatrix.randn(mu.getRowsNum());
        return mu.add(Sigma.product(values));
    }

View Full Code Here

        return distance;
    }


    public static void main(String[] args) {
        // 2.1, 3.5, 8. , 9.5
        Matrix x = InMemoryJBlasMatrix.randn(4);
        x.put(0, 0, 2.1);
        x.put(1, 0, 3.5);
        x.put(2, 0, 8.0);
        x.put(3, 0, 9.5);


        Matrix mu = InMemoryJBlasMatrix.zeros(4);
        mu.put(0, 0, 2.0);
        mu.put(1, 0, 3.0);
        mu.put(2, 0, 8.0);
        mu.put(3, 0, 10.0);


        Matrix Sigma = InMemoryJBlasMatrix.eye(4);
        Sigma.put(0, 0, 2.3);
        Sigma.put(1, 1, 1.5);
        Sigma.put(2, 2, 1.7);
        Sigma.put(3, 3, 2.0);


        System.out.println("Det: " + Sigma.det());
        System.out.println("Normal density: " + Gaussian.mphi(x, mu, Sigma));
        // System.out.println("PDF: " + Gaussian.pdf());


        double y = 0.1;
        double mean = 1.5;

View Full Code Here

    public InMemoryJBlasMatrix() {


    }


    public static Matrix arrayMatrix(double[] arrays) {
        Matrix m = InMemoryJBlasMatrix.zeros(arrays.length);
        for (int i = 0; i < arrays.length; i++) {
            m.put(i, 0, arrays[i]);
        }
        return m;
    }

View Full Code Here

        else
            return -1;
    }


    private Matrix createSubMatrix(int excluding_row, int excluding_col) {
        Matrix mat = new InMemoryJBlasMatrix(this.getRowsNum() - 1,
                this.getColumnsNum() - 1, this.errorProcessor);
        int r = -1;
        for (int i = 0; i < this.getRowsNum(); i++) {
            if (i == excluding_row)
                continue;
            r++;
            int c = -1;
            for (int j = 0; j < this.getColumnsNum(); j++) {
                if (j == excluding_col)
                    continue;
                mat.put(r, ++c, this.get(i, j));
            }
        }
        return mat;
    }

View Full Code Here

0 1 2 3

TOP

Related Classes of com.numb3r3.common.opt.QNMinimizer

com.google.common.io.PatternFilenameFilter

com.numb3r3.common.math.local.InMemoryJBlasMatrix

com.numb3r3.common.math.Matrix

com.numb3r3.common.math.stat.Gaussian

com.numb3r3.common.MathUtil

com.numb3r3.common.Pair

com.numb3r3.common.SystemErrorProcessor

com.sleepycat.bind.serial.SerialBinding

com.sleepycat.bind.serial.StoredClassCatalog

com.sleepycat.je.Cursor

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of com.numb3r3.common.opt.QNMinimizer

LINESEARCHES

Related Classes of com.numb3r3.common.opt.QNMinimizer