Examples of weka.classifiers.functions.Logistic$OptEng

weka.classifiers.functions.Logistic
Class for building and using a multinomial logistic regression model with a ridge estimator.

There are some modifications, however, compared to the paper of leCessie and van Houwelingen(1992):

If there are k classes for n instances with m attributes, the parameter matrix B to be calculated will be an m*(k-1) matrix.

The probability for class j with the exception of the last class is

Pj(Xi) = exp(XiBj)/((sum[j=1..(k-1)]exp(Xi*Bj))+1)

The last class has probability

1-(sum[j=1..(k-1)]Pj(Xi))
= 1/((sum[j=1..(k-1)]exp(Xi*Bj))+1)

The (negative) multinomial log-likelihood is thus:

L = -sum[i=1..n]{
sum[j=1..(k-1)](Yij * ln(Pj(Xi)))
+(1 - (sum[j=1..(k-1)]Yij))
* ln(1 - sum[j=1..(k-1)]Pj(Xi))
} + ridge * (B^2)

In order to find the matrix B for which L is minimised, a Quasi-Newton Method is used to search for the optimized values of the m*(k-1) variables. Note that before we use the optimization procedure, we 'squeeze' the matrix B into a m*(k-1) vector. For details of the optimization procedure, please check weka.core.Optimization class.

Although original Logistic Regression does not deal with instance weights, we modify the algorithm a little bit to handle the instance weights.

For more information see:

le Cessie, S., van Houwelingen, J.C. (1992). Ridge Estimators in Logistic Regression. Applied Statistics. 41(1):191-201.

Note: Missing values are replaced using a ReplaceMissingValuesFilter, and nominal attributes are transformed into numeric attributes using a NominalToBinaryFilter.
BibTeX:
```
 @article{leCessie1992, author = {le Cessie, S. and van Houwelingen, J.C.}, journal = {Applied Statistics}, number = {1}, pages = {191-201}, title = {Ridge Estimators in Logistic Regression}, volume = {41}, year = {1992} } 
```
Valid options are:
```
 -D Turn on debugging output.
```
```
 -R <ridge> Set the ridge in the log-likelihood.
```
```
 -M <number> Set the maximum number of iterations (default -1, until convergence).
```
@author Xin Xu (xx5@cs.waikato.ac.nz) @version $Revision: 5523 $

          }
        }
      }


      // Build logistic regression model
      m_logistic = new Logistic();
      m_logistic.buildClassifier(data);
    }

View Full Code Here

          }
        }
      }


      // Build logistic regression model
      m_logistic = new Logistic();
      m_logistic.buildClassifier(data);
    }

View Full Code Here

        insample_ROC = new double[2];
        validation_ACC = new double[2];
        validation_ROC = new double[2];
            
        //LOGISTIC REGRESSION   
        Classifier loggy_class = new Logistic();
        Evaluation loggy_eval = new Evaluation(learn);
        loggy_eval.crossValidateModel(Classifier.makeCopy(loggy_class), learn, 5, new Random(42));
            insample_ACC[0] = loggy_eval.pctCorrect();
            insample_ROC[0] = loggy_eval.weightedAreaUnderROC();

View Full Code Here

        case SMO:
          SMO smo = new SMO();
          smo.setOptions(Utils.splitOptions("-C 1.0 -L 0.001 -P 1.0E-12 -N 0 -V -1 -W 1 -K \"weka.classifiers.functions.supportVector.PolyKernel -C 250007 -E 1.0\""));
          return smo;
        case LOGISTIC:
          Logistic logistic = new Logistic();
          logistic.setOptions(Utils.splitOptions("-R 1.0E-8 -M -1"));
          return logistic;
        default:
          throw new IllegalArgumentException("Classifier " + classifier + " not found!");
      }
    }

View Full Code Here

        rbf.setGamma(0.5);
        smo.setKernel(rbf);
        smo.setC(1.5);
        
        //These also work pretty ok
        Logistic log = new Logistic();
        log.setRidge(100);
        
        Classifier classifier = log;


        try {
            System.out.print("Training on " + examples.size() + " examples... ");

View Full Code Here

TOP

Related Classes of weka.classifiers.functions.Logistic$OptEng

bnGUI.Thread_competitors

etc.aloe.oilspill2010.TrainingImpl

weka.classifiers.mi.MISMO$BinaryMISMO

weka.core.Capabilities

weka.core.Instance

weka.core.Instances

weka.core.Option

weka.core.TechnicalInformation

weka.filters.unsupervised.attribute.NominalToBinary

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.