Package statechum.analysis.learning.rpnicore

Examples of statechum.analysis.learning.rpnicore.LSolver

statechum.analysis.learning.rpnicore.LSolver

atlas.sourceforge.net}. After uncompressing it, create a build directory builddir in its top folder, enter it

 cd ATLAS/builddir  and run (replacing the prefix path with the one where you'd like Atlas to be installed)    ../configure --prefix=/usr/local/soft/atlas-3.10.1 -Fa alg -fPIC 
 Note: on Win32 with cygwin,  make sure you include -b 32 on the command line of configure to ensure that it does not attempt to build a 64-bit library and fail. 
 the configure script may fail to detect the number of cores and decide to abort. Forcing this number to the actual number of processors by modifying the appropriate file would help.  ATLAS/CONFIG/src/config.c 
 To correct the problem, you need to add one line after the probe "GetIntProbe", as follows:  iret = GetIntProbe(verb, ln, "arch", "NCPU", 2048); if (iret == 0) { printf("\nKIRR: ugly hack: forcing NCPU to 2");iret=2;} 
 
 After configure succeeds, you need to run "make" to build Atlas, then run  make check 
 to verify that it works and "make install" to install it. Note that when building starts, you need an unloaded PC for it to collect the relevant times for self-optimising. For this reason, 
 cpufreq-selector -g performance 
 might help to disable power-saving (Atlas calls it "CPU throttling" and attempts to detect it).

You need UMFPACK, {@link http://www.cise.ufl.edu/research/sparse/umfpack/}. Packages to download are AMD, COLAMD, CHOLMOD and SuiteSparse_config. These need to be uncompressed starting from the same directory. Subsequently, configuration needs to be set. The file to edit is
```
  SuiteSparse_config/SuiteSparse_config.mk 
```
(or any other depending on whether you use a Mac or GPU. It has to have the following two lines included (which would usually replace the appropriate lines in the file rather than be added at the end):
```
 UMFPACK_CONFIG = -fPIC BLAS = -L/usr/local/soft/atlas-3.10.1/lib -lf77blas -latlas -lgfortran 
```
Note the reference to the Atlas installation directory. You may also wish to replace -llf77blas with -lptf77blas, but it should not affect anything.
You might also want to
```
 CC = gcc CFLAGS = -O3 
```
or even
```
 CC = gcc -fPIC 
```
to ensure that AMD is built as a relocable library - without it Statechum's .so library may fail to build.
Win32/cygwin note: if using gcc version 3 rather than 4, use the following line
```
 BLAS = -L/usr/local/soft/atlas-3.8.1/lib -lf77blas -latlas -lg2c 
```
Subsequently, running make in the UMFPACK directory will build UMFPACK (and other libraries). Most likely, you will have to run
```
 make install  
```
in all of AMD/CHOLMOD/COLAMD/SuiteSparse_config/UMFPACK in order to get all the files installed. Without them, it will not be possible to build libStatechumsolver The best way to do this is via
```
 for D in AMD CHOLMOD COLAMD SuiteSparse_config UMFPACK;do (cd $D;make install );done 
```
If files linear/configure or linear/Makefile.in are not present, you need to build them as follows:
```
 cd linear ./bootstrap 
```
Building the interface involves running
```
 cd linear ./configure --with-blasdir=/usr/local/soft/atlas-3.10.1/lib --with-umfpack=/usr/local/src/umfpack && make 
```
Note that the above includes both Atlas directory and the one into which UMFPACK was extracted. Subsequently, running make should build the library.
If gcc has gcj installed, Sun JDK may include a jni_md.h which will pick gcc's jni_md.h rather than Sun's one and give an error about a boolean. The solution is to do something like
```
 CC='gcc -I/usr/local/soft/jdk1.6.0_16/include/linux/' sh ./configure --with-blasdir=/usr/local/soft/atlas-3.10.1 --with-umfpack=/usr/local/src/umfpack/ 
```
On Win32 with cygwin's gcc version 3, this will not work because libtool will not link to static libraries (.a import libraries from .dlls are ok). Linking to all the object files from atlas, amd and umfpack will not work because these .o files have not been built with libtool. The only option is hence to build a .dll from atlas, amd and umfpack and link to that dll. Multi-threaded support does not work because libptf77blas links to cygwin1.dll which we cannot use - this dll does not support being dynamically loaded (a well-documented problem) so the library we build with it will not work with Java (jvm will lock up on LoadLibrary).
The easiest way to build a library is to edit fullrebuild.sh by adding the relevant paths to it and then issue
```
 cd linear ./fullrebuild.sh 
```
On Linux, this runs bootstrap and make. On Win32, this script takes all .o files from atlas, umfpack and amd, builds a library out of them, converts it to .dll (following {@link http://www.kevinsheppard.org/research/matlabatlas}) and subsequently links the interface module to that .dll. If any changes are made to the .c or .h file, the library can be rebuilt with
```
 make -f Makefile_win32 
```
where the Makefile_win32 is built by fullrebuild.sh.
Note that the way the script builds a dll is very ugly but since it seems that cygwin does not do what one would expect it to (build a dll out of static libraries built with -fPIC), I think it is not appropriate to "fix" configure scripts to work around these problems.
In order for the just built library to be picked by Java, you need to pass the following as a JVM option:
```
 -Djava.library.path=linear/.libs 
```

MacOS X - specific instructions

It may be advisable to build gcc first (gmp web page suggests it will not work when built with XCode so using a recent gcc may be advisable),

Using Applications/Utilities/Java/Java Preferences, set the default JVM to be 64-bit version 6.0.

build gmp

 CC='gcc-4.2 -m64' ./configure --prefix=/usr/local/soft_a/gmp make make install

build mpfr

 CC='gcc-4.2 -m64' ./configure --with-gmp=/usr/local/soft_a/gmp --prefix=/usr/local/soft_a/mpfr make make install

Build gcc,

 CC='gcc-4.2 -m64' ./configure --prefix=/usr/local/soft_a/gcc-4.4.1 --with-gmp=/usr/local/soft_a/gmp-4.3.1 --with-mpfr=/usr/local/soft_a/mpfr-2.4.1 --host=x86_64-apple-darwin9.8.0 --build=x86_64-apple-darwin9.8.0 --enable-languages="c c++"

The -m64 ensures all of gcc is built for 64-bit. --host and --build make sure that gcc rebuilds itself for 64-bit.

Now repeat the above but build gmp,mpfr and gcc with the just-built gcc, replacing soft_a with soft above. The distinction between soft_a and soft is whether Apple's gcc (gcc-4.2) has been used to build a library or not.

MacOS X or XCode come with blas libraries, hence one does not need to install Atlas (which can be built using instructions for building Atlas on Linux). If Atlas is installed into somewhere like /usr/local/soft/atlas-3.8.3, in the course of building UMFPACK gcc may choose to pick /usr/lib/libatlas.dylib instead of the newly-installed Atlas library /usr/local/soft/atlas-3.8.3/lib/libatlas.dylib, leading to build errors for umfpack's examples because Atlas's symbols have different names to that of Apples's Atlas.

After unpacking all three parts of UMFPACK, options to be included in UFconfig/UFconfig.mk are :

 UMFPACK_CONFIG = -DNO_TIMER -fPIC BLAS = -L/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A -lblas

For building Statechum library, the following is done:

 cd linear ./configure --with-blasdir=/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A --with-umfpack=/Users/Shared/experiment/umfpack && make

Where /Users/Shared/experiment/umfpack is where UMFPACK was extracted, it should have subdirectories UMFPACK, AMD and UFconfig.

For testing, providing the -Djava.awt.headless=true option to JVM is necessary on MacOS to stop loading of Statechum's properties from making a connection to a monitor and triggering a bouncing coffee icon in the dock as well as a display of application name in the menu bar. The problem is caused by static data in, for instance, java.awt.Rectangle making the said connection upon classloading of java.awt.Rectangle.

It is worth noting that the full path of the interface is encoded in the shared libraries hence moving this interface is only possible if libraries are rebuilt. For this reason, LSolver is stuck in the rpnicore rather than in learner/linear. Instructions for building OpenBLAS on Linux, Uncompress the .zip, then from the main directory run

 make PREFIX=/usr/local/soft/OpenBLAS-be853da DYNAMIC_ARCH=1 UTEST_CHECK=1 NO_LAPACK=1	NO_AFFINITY=1

The first parameter is responsible for building in support for all CPU architectures, otherwise only the current one will This will attempt to download CUnit and then fail.

 cd utest;make

should build and install CUnit into utest. Subsequently doing the following command from the main directory (the one just above utest)

  ln -s utest/CUnit-2.1-2/include/CUnit .

would fix unit tests that ship with OpenBLAS and they will run when the above make command is run. After doing

  make PREFIX=/usr/local/soft/OpenBLAS-be853da DYNAMIC_ARCH=1 UTEST_CHECK=1 NO_LAPACK=1 install

OpenBLAS is available. SuiteSparse_config.mk should have

 CC=x86_64-w64-mingw32-gcc AR=x86_64-w64-mingw32-ar RANLIB = x86_64-w64-mingw32-ranlib F77 = x86_64-w64-mingw32-gfortran BLAS = /usr/local/soft/OpenBLAS-be853da/lib/libopenblas.lib INSTALL_LIB=/usr/local/soft/umfpack-5.6.1-openblas/lib INSTALL_INCLUDE=/usr/local/soft/umfpack-5.6.1-openblas/include

OpenBLAS on Windows7-x86_64 First, build OpenBLAS with both options for Unix (above) and those mentioned in quickbuild.64bit.

When building UMFPACK, most options are the same as for Unix but there are a few changes,

BLAS = /usr/local/soft/OpenBLAS-be853da/lib/libopenblas.lib
It is worth removing -fPIC from the definition of CF since on Windows all code is position-independent and this eliminates the warning.
It is worth removing -lrt from definition LIB because the library is missing.

Finally, Makefile in the UMFPACK/Demo directory has the "run" target, where it runs demos and records the results. Every time it runs a demo, the output has to be piped through d2u to correct line endings on Windows and through sed 's/e$[+-]$0$[0-9]\+$/e\1\2/g' to convert the differences in which exponents are output on Windows. Hence the extra content is like

 ... _demo | d2u |  sed  's/e\([+-]\)0\([0-9]\+\)/e\1\2/g' > my_ ... .out

GDLearnerGraph ndGraph = new GDLearnerGraph(new LearnerGraph(buildGraph("A-a->B-a->B-b->A / B-c->C / E-a->F-a->F-d->F-b->E-c->F", "dumpEquations"),config),LearnerGraphND.ignoreRejectStates, false);

final int [] incompatiblePairs = new int[ndGraph.getPairNumber()];

final int pairsNumber = incompatiblePairs.length;

for(int i=0;i<incompatiblePairs.length;++i) incompatiblePairs[i]=i;// emulate the behaviour or non-public part for(int i=0;i<incompatiblePairs.length;++i) incompatiblePairs[i]=PAIR_OK;final int pairsNumber = ndGraph.findIncompatiblePairs(incompatiblePairs,1);

LSolver solver = ndGraph.buildMatrix_internal(incompatiblePairs, pairsNumber, 1,null);

System.out.println(ndGraph.dumpEquations(solver, incompatiblePairs, null).toString()); }

performRowTasks(handlerList, ThreadNumber, matrixForward.transitionMatrix,filter,

GDLearnerGraph.partitionWorkLoadTriangular(ThreadNumber,matrixForward.transitionMatrix.size()));

// At this point, we are finished building the matrices, it's time to populate the main arrays if needed - we do not have to do this if we only aim to compute the right-hand side.

LSolver result = null;

if (config.getGdScoreComputation() == GDScoreComputationEnum.GD_RH)

// First, we compute the number of non-zero elements.

int size = 0;for(int thread=0;thread<ThreadNumber;++thread) size+=currentPosition[thread];

// Second, allocate arrays.

int Ai[]=new int[size];double Ax[]=new double[size];

// In a number of cases we'll have more threads than rows to handle, hence threads will have nothing to do

// and Ap_threadStart will be -1 for those threads. In order to ensure that Ap_threadStart contains

// continuous values, we set those -1 elements to Ap_threadStart of the subsequent thread (or pairsNumber if there is none).

int currentLastStatePair = pairsNumber;

for(int th=ThreadNumber;th>=0;--th) if (Ap_threadStart[th] < 0) Ap_threadStart[th]=currentLastStatePair;else currentLastStatePair=Ap_threadStart[th];

int prevLastPos = 0;

// Finally, populate them.

for(int thread=0;thread<ThreadNumber;++thread)

/* I'll try to draw a picture which shows the relation between arrays computed by different threads,

StatePair values : 0 1 2 | 3 4 5 | 6 7 8

Ap: 0 8 12 | 0 11 16 | 0 4 12

currentPosition : 20 18 19

The above picture with three threads shows the following relation between columns

and the number of elements in them:

When we copy the outcome into Ai and Ax, we have to introduce sequential numbers,

so column 3 will start where column 2 finished in Ai populated by thread 0,

i.e. at position 20 (last element filled in by thread 0),

column 4 will start at 11+20 and so on until data filled in by thread 2,

hence column 6 will start at position 20+18.

With the above illustration in mind, the code below updates Ap and fills in Ax and Ai.

int lastPair = Ap_threadStart[thread+1];// the position where the next thread has started (or would've started if it was not the last thread).

for(int i=Ap_threadStart[thread];i<lastPair;++i)

// If a thread had no work to do, currentPosition[thread] would stay at zero so we shall not corrupt anything,

// but Ai_array[thread] may be null, so we'd best check for "no work".

if (currentPosition[thread]>0)

System.arraycopy(Ai_array[thread].elements(), 0, Ai, prevLastPos, currentPosition[thread]);

System.arraycopy(Ax_array[thread].elements(), 0, Ax, prevLastPos, currentPosition[thread]);

prevLastPos+=currentPosition[thread];

Ap[pairsNumber]=prevLastPos;

result = new LSolver(Ap,Ai,Ax,b,new double[pairsNumber]);

{// we are here to compute the right-hand side only, hence build a dummy solver.

result = new LSolver(pairsNumber)

{// constructor: copy b to the result column

System.arraycopy(b, 0, j_x, 0, pairsNumber);

public double [] computeStateCompatibility(int ThreadNumber,final Class<? extends DetermineDiagonalAndRightHandSideInterface> ddrh)

final int [] incompatiblePairs = new int[getPairNumber()];for(int i=0;i<incompatiblePairs.length;++i) incompatiblePairs[i]=PAIR_OK;

final int pairsNumber = findIncompatiblePairs(incompatiblePairs,ThreadNumber);

LSolver solver = buildMatrix_internal(incompatiblePairs, pairsNumber, ThreadNumber,ddrh);

solver.solve(ThreadNumber);

solver.freeAllButResult();// deallocate memory before creating a large array.

double statePairScores[] = new double[incompatiblePairs.length];

// now fill in the scores in the array.

for(int i=0;i<incompatiblePairs.length;++i)

if (incompatiblePairs[i] >=0) statePairScores[i]=solver.j_x[incompatiblePairs[i]];

else statePairScores[i]=incompatiblePairs[i];// PAIR_INCOMPATIBLE

* Since it messes up the configuration of the graph, it has to be run at the end of every test method rather than multiple times.

protected void checkBuildMatrix(LearnerGraph gr, DoubleMatrix2D expectedAx, DoubleMatrix1D expectedB)

GDLearnerGraph ndGraph = new GDLearnerGraph(gr,LearnerGraphND.ignoreRejectStates, false);

LSolver solver = ndGraph.buildMatrix(ThreadNumber);

DoubleMatrix2D Ax=solver.toDoubleMatrix2D();

Assert.assertEquals(getExpectedMatrix2DSlowly(gr),Ax);

if (expectedAx != null) Assert.assertEquals(expectedAx, Ax);

DoubleMatrix1D b=solver.toDoubleMatrix1D();

if (expectedB != null) Assert.assertEquals(expectedB, b);Assert.assertEquals(getExpectedMatrix1DSlowly(gr),b);

solver.solveExternally(1);// check if we have a solution, just in case it fails.

// Now check consistency.

gr.config.setAttenuationK_testOnly(1);DoubleMatrix2D Ax1 = ndGraph.buildMatrix(ThreadNumber).toDoubleMatrix2D();

gr.config.setAttenuationK(0);DoubleMatrix2D Ax0 = ndGraph.buildMatrix(ThreadNumber).toDoubleMatrix2D();

DoubleMatrix1D one = DoubleFactory1D.dense.make(Ax1.rows(), 1), a=DoubleFactory1D.dense.make(Ax.rows(), 0);

public final void testDumpEquations1()

GDLearnerGraph ndGraph = new GDLearnerGraph(buildLearnerGraph("A-a->B-a->B-b->A", "testBuildMatrix1",config),LearnerGraphND.ignoreRejectStates, false);

final int [] incompatiblePairs = new int[ndGraph.getPairNumber()];for(int i=0;i<incompatiblePairs.length;++i) incompatiblePairs[i]=PAIR_OK;

final int pairsNumber = ndGraph.findIncompatiblePairs(incompatiblePairs,ThreadNumber);

LSolver solver = ndGraph.buildMatrix_internal(incompatiblePairs, pairsNumber, ThreadNumber,null);

String outcome = ndGraph.dumpEquations(solver, incompatiblePairs, null).toString();

Assert.assertEquals("2.0([A,A]:[A,A]) + -"+k+"([A,A]:[B,B]) = 1.0\n"+

"4.0([B,A]:[B,A]) + -"+k+"([B,A]:[B,B]) = 1.0\n"+

"-"+k+"([B,B]:[A,A]) + "+(4-k)+"([B,B]:[B,B]) = 2.0\n",outcome);

LearnerGraph graph = buildLearnerGraph("A-a->B-a->B-b->A", "testBuildMatrix1",config);

GDLearnerGraph ndGraph = new GDLearnerGraph(graph,LearnerGraphND.ignoreRejectStates, false);

final int [] incompatiblePairs = new int[ndGraph.getPairNumber()];for(int i=0;i<incompatiblePairs.length;++i) incompatiblePairs[i]=PAIR_OK;

final int pairsNumber = ndGraph.findIncompatiblePairs(incompatiblePairs,ThreadNumber);

LSolver solver = ndGraph.buildMatrix_internal(incompatiblePairs, pairsNumber, ThreadNumber,null);

Map<CmpVertex,CmpVertex> newToOrig = new TreeMap<CmpVertex,CmpVertex>();

String P67="P67",P99="B";

newToOrig.put(graph.findVertex("A"),AbstractLearnerGraph.generateNewCmpVertex(VertexID.parseID(P67), graph.config));

String outcome = ndGraph.dumpEquations(solver, incompatiblePairs, newToOrig).toString();

Assert.assertEquals("2.0(["+P67+","+P67+"]:["+P67+","+P67+"]) + -"+k+"(["+P67+","+P67+"]:["+P99+","+P99+"]) = 1.0\n"+

LearnerGraph graph = buildLearnerGraph("A-a->B-a->B-b->A", "testBuildMatrix1",config);

GDLearnerGraph ndGraph = new GDLearnerGraph(graph,LearnerGraphND.ignoreRejectStates, false);

final int [] incompatiblePairs = new int[ndGraph.getPairNumber()];for(int i=0;i<incompatiblePairs.length;++i) incompatiblePairs[i]=PAIR_OK;

final int pairsNumber = ndGraph.findIncompatiblePairs(incompatiblePairs,ThreadNumber);

LSolver solver = ndGraph.buildMatrix_internal(incompatiblePairs, pairsNumber, ThreadNumber,null);

Map<CmpVertex,CmpVertex> newToOrig = new TreeMap<CmpVertex,CmpVertex>();

String P67="P67",P99="P99";

newToOrig.put(graph.findVertex("A"),AbstractLearnerGraph.generateNewCmpVertex(VertexID.parseID(P67), graph.config));

newToOrig.put(graph.findVertex("B"),AbstractLearnerGraph.generateNewCmpVertex(VertexID.parseID(P99), graph.config));

String outcome = ndGraph.dumpEquations(solver, incompatiblePairs, newToOrig).toString();

"\nB-b-#G\n","TestComputeStateCompatibility_checking_filtering2",config);

StatesToConsider filter = TestLearnerGraphND.createInstanceOfFilter(filterClass, gr);

double score_CC=1./2,score_BB=(2+score_CC*k)/4, score_AA=(1+k*score_BB)/2;

GDLearnerGraph ndGraph = new GDLearnerGraph(gr,LearnerGraphND.ignoreNone, false);

LSolver solver = ndGraph.buildMatrix(ThreadNumber);

DoubleMatrix2D Ax=solver.toDoubleMatrix2D();

Assert.assertEquals(6,Ax.columns());

Set<PairScore> pairsSet = addAllPermutations(gr.pairscores.chooseStatePairs(PAIR_INCOMPATIBLE*2,10,ThreadNumber,null,filter, new NonRandomRandom()));

Set<PairScore> exp = addAllPermutations(Arrays.asList(new PairScore[]{

new PairScore(gr.findVertex("A"),gr.findVertex("A"),(int)(10*score_AA),1),

public final void testExternalSolver1()

LSolver s = new LSolver(testMatrix,DoubleFactory1D.dense.make(new double[]{8., 45., -3., 3., 19.}));

DoubleMatrix1D x = DoubleFactory1D.dense.make(testMatrix.rows());

for(int i=0;i<s.j_b.length;++i) x.setQuick(i, s.j_b[i]);

s.solveExternally(1);

for(int i=0;i<testMatrix.rows();++i)

Assert.assertEquals(i+1, s.j_x[i],Configuration.fpAccuracy);

LUDecompositionQuick solver = new LUDecompositionQuick();

solver.decompose(testMatrix);solver.setLU(testMatrix);

for(int i=0;i<testMatrix.rows();++i)

Assert.assertEquals(i+1, x.getQuick(i),Configuration.fpAccuracy);

for(int i=0;i<testMatrix.rows();++i) s.j_x[i]=0;

for(int i=0;i<testMatrix.rows();++i)

Assert.assertEquals(i+1, s.j_x[i],Configuration.fpAccuracy);

}, IllegalArgumentException.class,"zero-sized problem");

checkForCorrectException(new whatToRun() {

@SuppressWarnings("unused")

public @Override void run() throws NumberFormatException {

new LSolver(Ap, Ai, Ax, b, x);

}, IllegalArgumentException.class,"zero-sized problem");

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of statechum.analysis.learning.rpnicore.LSolver

MacOS X - specific instructions

Related Classes of statechum.analysis.learning.rpnicore.LSolver