natas_hw
/
jidt
mirror of https://github.com/jlizier/jidt
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'ghub/master'. This merges Joe's v1.3.1 master into Pedro's master.

This commit is contained in:
Pedro Martinez Mediano 2017-05-24 20:24:32 +10:00
parent 2654e848ae 4e54fc6ce4
commit 09548028e0
48 changed files with 6549 additions and 1003 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@ -1,6 +1,6 @@
# Java Information Dynamics Toolkit (JIDT)
Copyright (C) 2012-2014 [Joseph T. Lizier](http://lizier.me/joseph/); 2014-2015 [Joseph T. Lizier](http://lizier.me/joseph/) and Ipek Özdemir
Copyright (C) 2012-2014 [Joseph T. Lizier](http://lizier.me/joseph/); 2014-2016 [Joseph T. Lizier](http://lizier.me/joseph/) and Ipek Özdemir
*JIDT* provides a stand-alone, open-source code Java implementation (also usable in [Matlab, Octave](../../wiki/UseInOctaveMatlab), [Python](../../wiki/UseInPython), [R](../../wiki/UseInR), [Julia](../../wiki/UseInJulia) and [Clojure](../../wiki/UseInClojure)) of information-theoretic measures of distributed computation in complex systems: i.e. information storage, transfer and modification.
@ -15,11 +15,13 @@ JIDT is distributed under the [GNU GPL v3 license](http://www.gnu.org/licenses/g
# Getting started
1. [Download](../../wiki/Downloads) and [Installation](../../wiki/Installation) is very easy!
1. _Quick start_: download the latest [v1.3 full distribution](http://lizier.me/joseph/software/jidt/download.php?file=infodynamics-dist-1.3.zip) (suitable for all platforms) and see the readme.txt file therein.
1. [Documentation](../../wiki/Documentation) including: the paper describing JIDT at [arXiv:1408.3270](http://arxiv.org/abs/1408.3270) (distributed with the toolkit), a [Tutorial](../../wiki/Tutorial), and [Javadocs (v1.3 here)](http://lizier.me/joseph/software/jidt/javadocs/v1.3/);
1. [Demos](../../wiki/Demos) are included with the full distribution, including a [GUI app](../../wiki/AutoAnalyser) for automatic analysis and code generation, [simple java demos](../../wiki/SimpleJavaExamples) and [cellular automata (CA) demos](../../wiki/CellularAutomataDemos).
1. _Quick start_: download the latest [v1.3.1 full distribution](http://lizier.me/joseph/software/jidt/download.php?file=infodynamics-dist-1.3.1.zip) (suitable for all platforms) and see the readme.txt file therein.
1. [Documentation](../../wiki/Documentation) including: the paper describing JIDT at [arXiv:1408.3270](http://arxiv.org/abs/1408.3270) (distributed with the toolkit), a [Tutorial](../../wiki/Tutorial), and [Javadocs (v1.3.1 here)](http://lizier.me/joseph/software/jidt/javadocs/v1.3.1/);
1. [Demos](../../wiki/Demos) are included with the full distribution, including a [GUI app](../../wiki/AutoAnalyser) for automatic analysis and code generation (see picture below), [simple java demos](../../wiki/SimpleJavaExamples) and [cellular automata (CA) demos](../../wiki/CellularAutomataDemos).
1. These Java tools can easily be used in [Matlab/Octave](../../wiki/OctaveMatlabExamples), [Python](../../wiki/PythonExamples), [R](../../wiki/R_Examples), [Julia](../../wiki/JuliaExamples) and [Clojure](../../wiki/Clojure_Examples)! (click on each language here for examples)
![Computing in the GUI app image](https://raw.githubusercontent.com/jlizier/jidt/master/web/AutoAnalyserGUI-2-Compute.png)
For further information or announcements:
* Join our discussion group: http://groups.google.com/d/forum/jidt-discuss
* See also the [FAQs](../../wiki/FAQs)
@ -37,6 +39,23 @@ See other [PublicationsUsingThisToolkit](../../wiki/PublicationsUsingThisToolkit
# News
_17/12/2016_ - New book out from J. Lizier et al., ["An Introduction to Transfer Entropy: Information Flow in Complex Systems"](http://bit.ly/te-book-2016) published by Springer, which contains various examples using JIDT (distributed in our releases)
_21/10/2016_ - New jar and full distribution files available for **release v1.3.1**; Changes for v1.3.1 include:
Major update to TransferEntropyCalculatorDiscrete so as to implement arbitrary source and dest embeddings and source-dest delay;
Conditional TE calculators (continuous) handle empty conditional variables;
Added new auto-embedding method for AIS and TE which maximises bias corrected AIS;
Added getNumSeparateObservations() method to TE calculators to make reconstructing/separating local values easier after multiple addObservations() calls;
Fixed kernel estimator classes to return proper densities, not probabilities;
Bug fix in mixed discrete-continuous MI (Kraskov) implementation;
Added simple interface for adding joint observations for MultiInfoCalculatorDiscrete
Including compiled class files for the AutoAnalyser demo in distribution;
Updated Python demo 1 to show use of numpy arrays with ints;
Added Python demo 7 and 9 for TE Kraskov with ensemble method and auto-embedding respectively;
Added Matlab/Octave example 10 for conditional TE via Kraskov (KSG) algorithm;
Added utilities to prepare for enhancing surrogate calculations with fast nearest neighbour search;
Minor bug patch to Python readFloatsFile utility.
_19/7/2015_ - New jar and full distribution files available for **release v1.3**; Changes for v1.3 include:
Added AutoAnalyser (Code Generator) GUI demo for MI and TE;
Added auto-embedding capability via Ragwitz criteria for AIS and TE calculators (KSG estimators);

25
build.xml
View File

@ -5,7 +5,7 @@
</description>
<!-- set global properties for this build -->
<property name="version" value="1.3"/>
<property name="version" value="1.3.1"/>
<property name="mainfilename" value="infodynamics"/>
<property name="jarplainname" value="${mainfilename}.jar" />
<property name="jarversiondistnamezip" value="${mainfilename}-jar-${version}.zip" />
@ -16,6 +16,7 @@
<property name="unittestsouttoplevel" location="unittests"/>
<property name="unittestssrc" location="java/unittests"/>
<property name="unittestsbin" location="${unittestsouttoplevel}/bin"/>
<property name="autoanalyserdemo" location="demos/java"/>
<property name="reports.tests" location="${unittestsouttoplevel}/reports"/>
<property name="javadocsdir" location="javadocs"/>
<property name="versionfile" value="version-${version}.txt"/>
@ -79,6 +80,23 @@
</junit>
</target>
<!-- Compile the autoanalyser demo -->
<target name="autoanalyser" depends="jar" description="compile the autoanalyser demo">
<!-- Compile to Java 6 to provide compatibility for users with older JREs.
Caveat: The flags here only check the language compatibility, but
may still use newer libraries which may cause issues for users with JDK 6.
Indeed, one gets the warning: "bootstrap class path not set in conjunction with -source 1.6"
To fix this, one would use the bootstrap classpath to point our JDK to an rt.jar
for Java 6.
At this stage, I'm sure I'm not using new library calls from Java 7, so we can
ignore the warning, and I don't want to bother installing Java 6 just to compile
like this. I'll endeavour not to use JDK 7 libraries so as not to cause
any issues here ... -->
<javac srcdir="${autoanalyserdemo}" includes="infodynamics/demos/autoanalysis/*.java" excludes="infodynamics/demos/autoanalysis/GeneratedCalculator.java" includeAntRuntime="false" target="1.6" source="1.6">
<classpath refid="project.classpath"/>
</javac>
</target>
<!-- Make javadocs, excluding the classes we've derived from Apache Commons Math -->
<target name="javadocs" depends="compile" description="Make the javadocs for the toolkit">
<delete dir="${javadocsdir}"/>
@ -190,7 +208,7 @@
</target>
<!-- Developer build - builds everything and makes the full distribution file -->
<target name="dist" depends="jar,junit,javadocs,readmefiles" description="developer: generate the full distribution">
<target name="dist" depends="jar,junit,autoanalyser,javadocs,readmefiles" description="developer: generate the full distribution">
<echo message="${ant.project.name}: ${ant.file}"/>
<zip destfile="${distnamezip}">
<fileset file="build.xml"/>
@ -198,13 +216,14 @@
<fileset file="license-gplv3.txt"/>
<fileset file="readme.txt"/>
<fileset file="InfoDynamicsToolkit.pdf"/>
<fileset file="JIDT-logo.png" erroronmissingdir="false"/> <!-- This file is missing in full repository versions -->
<fileset file="${versionfile}"/>
<zipfileset dir="java" includes="**/*.java" prefix="java"/>
<zipfileset dir="demos" includes="**/*.*,**/*" excludes="clojure/deploy,clojure/deploy/*.*,python/*.pyc" prefix="demos"/>
<zipfileset dir="javadocs" includes="**/*.*,**/*" prefix="javadocs"/>
<zipfileset dir="notices" includes="**/*.*,**/*" prefix="notices"/>
<zipfileset dir="tutorial" prefix="tutorial"/>
<zipfileset dir="web" includes="JIDT-logo.png" prefix=""/>
<zipfileset dir="web" includes="JIDT-logo.png" prefix="" erroronmissingdir="false"/> <!-- This file is missing in zip dist versions -->
</zip>
</target>
</project>

BIN
demos/AutoAnalyser/README-AutoAnalyserDemo.pdf
View File

Binary file not shown.

BIN
demos/java/README-SimpleJavaDemos.pdf
View File

Binary file not shown.

594
demos/java/infodynamics/demos/autoanalysis/AutoAnalyser.java
View File

@ -20,7 +20,10 @@ package infodynamics.demos.autoanalysis;
import infodynamics.measures.continuous.ChannelCalculatorCommon;
import infodynamics.measures.discrete.ChannelCalculatorDiscrete;
import infodynamics.utils.AnalyticMeasurementDistribution;
import infodynamics.utils.AnalyticNullDistributionComputer;
import infodynamics.utils.ArrayFileReader;
import infodynamics.utils.EmpiricalMeasurementDistribution;
import infodynamics.utils.MatrixUtils;
import javax.swing.BorderFactory;
@ -30,6 +33,7 @@ import javax.swing.JFrame;
import javax.swing.JOptionPane;
import javax.swing.JPanel;
import javax.swing.JComboBox;
import javax.swing.JCheckBox;
import javax.swing.JButton;
import javax.swing.JLabel;
import javax.swing.JFileChooser;
@ -39,6 +43,8 @@ import javax.swing.JTable;
import javax.swing.JTextArea;
import javax.swing.JTextField;
import javax.swing.ToolTipManager;
import javax.swing.event.ChangeEvent;
import javax.swing.event.ChangeListener;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.table.AbstractTableModel;
@ -48,6 +54,7 @@ import javax.swing.table.TableColumn;
import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.FlowLayout;
import java.awt.Font;
import java.awt.Graphics;
import java.awt.GridBagConstraints;
@ -73,7 +80,7 @@ import java.util.Vector;
*
*/
public abstract class AutoAnalyser extends JFrame
implements ActionListener, DocumentListener, MouseListener {
implements ActionListener, DocumentListener, MouseListener, ChangeListener {
/**
* Need serialVersionUID to be serializable
@ -142,6 +149,14 @@ public abstract class AutoAnalyser extends JFrame
protected JTextField sourceColTextField;
// Destination column field
protected JTextField destColTextField;
// CheckBox for "all pairs?"
protected JCheckBox allPairsCheckBox;
// CheckBox for statistical significance
protected JCheckBox statSigCheckBox;
// CheckBox for statistical significance analytically
protected JCheckBox statSigAnalyticCheckBox;
// Number of permutations to use for statistical significance
protected int numPermutationsToCheck = 100;
// Table for the properties
protected JTable propertiesTable;
// Table model (local class) for the table
@ -154,6 +169,8 @@ public abstract class AutoAnalyser extends JFrame
protected Vector<String> propertyDescriptions;
// Values of the properties
protected HashMap<String,String> propertyValues;
// CheckBox for "Compute result?"
protected JCheckBox computeResultCheckBox;
// Results text
protected JLabel resultsLabel;
// Text area for the generated Java code
@ -165,6 +182,9 @@ public abstract class AutoAnalyser extends JFrame
protected String codeDefaultText = " ... Awaiting new parameter selection (press compute) ...";
protected String buttonTextCodeAndCompute = "Generate code and Compute";
protected String buttonTextCodeOnly = "Generate code";
protected String appletTitle;
public class TextAreaWithImage extends JTextArea {
@ -232,7 +252,7 @@ public abstract class AutoAnalyser extends JFrame
Image watermarkImage = (new ImageIcon("JIDT-logo-watermark.png")).getImage();
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setSize(1100,530);
setSize(1100,640);
setTitle(appletTitle);
// Centre in the middle of the screen
setLocationRelativeTo(null);
@ -274,11 +294,26 @@ public abstract class AutoAnalyser extends JFrame
// To column:
JLabel destLabel = new JLabel("Destination column:");
destLabel.setToolTipText("First column is 0.");
destLabel.setBorder(BorderFactory.createEmptyBorder(0,0,10,0));
destColTextField = new JTextField(5);
destColTextField.setEnabled(true);
destColTextField.setText("1");
destColTextField.getDocument().addDocumentListener(this);
// Checkbox for all pairs
allPairsCheckBox = new JCheckBox("All pairs?");
allPairsCheckBox.setToolTipText("Compute for all pairs of columns?");
allPairsCheckBox.addChangeListener(this);
// CheckBox for statistical signficance
statSigCheckBox = new JCheckBox("Add stat. signif.?");
statSigCheckBox.setToolTipText("Compute the null surrogate distribution under the null hypothesis that source and target are not related?");
statSigCheckBox.addChangeListener(this);
statSigAnalyticCheckBox = new JCheckBox("analytically?");
statSigAnalyticCheckBox.setToolTipText("Compute the null surrogate distribution analytically?");
statSigAnalyticCheckBox.addChangeListener(this);
statSigAnalyticCheckBox.setEnabled(false);
JLabel dummyLabel1 = new JLabel(" ");
dummyLabel1.setSize(10, 10);
JLabel dummyLabel2 = new JLabel(" ");
@ -307,9 +342,14 @@ public abstract class AutoAnalyser extends JFrame
new Dimension(d.width,rowHeight*17+6));
System.out.println("Row height was " + rowHeight);
// Checkbox for compute result
computeResultCheckBox = new JCheckBox("Compute result?");
computeResultCheckBox.setToolTipText("Compute result or only generate code?");
computeResultCheckBox.setSelected(true);
computeResultCheckBox.addChangeListener(this);
// Button to compute
computeButton = new JButton("Compute");
computeButton = new JButton(buttonTextCodeAndCompute);
computeButton.addActionListener(this);
// Results label
@ -336,7 +376,7 @@ public abstract class AutoAnalyser extends JFrame
javaAreaScrollPane.setHorizontalScrollBarPolicy(
JScrollPane.HORIZONTAL_SCROLLBAR_AS_NEEDED);
int codeTextAreaWidth = 560;
int codeTextAreaHeight = 460;
int codeTextAreaHeight = 480;
Dimension codeTextAreaDimension =
new Dimension(codeTextAreaWidth, codeTextAreaHeight);
javaAreaScrollPane.setPreferredSize(codeTextAreaDimension);
@ -425,6 +465,14 @@ public abstract class AutoAnalyser extends JFrame
paramsPanel.add(dataFileDescriptorLabel, c);
// Add the all pairs label and checkbox
c.gridwidth = GridBagConstraints.REMAINDER; //end row
c.gridx = 1;
c.fill = GridBagConstraints.HORIZONTAL;
c.weightx = 1.0;
paramsPanel.add(allPairsCheckBox, c);
c.gridx = -1; // reset to no indication
// Add the source selector
c.gridwidth = GridBagConstraints.RELATIVE; //next-to-last
c.fill = GridBagConstraints.NONE; //reset to default
@ -444,20 +492,46 @@ public abstract class AutoAnalyser extends JFrame
c.weightx = 1.0;
paramsPanel.add(destColTextField, c);
// Add the CheckBoxes for statistical significance checks:
c.gridwidth = GridBagConstraints.RELATIVE; //next-to-last
c.fill = GridBagConstraints.NONE; //reset to default
c.weightx = 0.0; //reset to default
paramsPanel.add(statSigCheckBox, c);
c.gridwidth = GridBagConstraints.REMAINDER; //end row
c.fill = GridBagConstraints.HORIZONTAL;
c.weightx = 1.0;
paramsPanel.add(statSigAnalyticCheckBox, c);
// Add dummy label for spacing
paramsPanel.add(dummyLabel1, c);
// Add the properties table
paramsPanel.add(propsTableScrollPane, c);
// Add dummy label for spacing
paramsPanel.add(dummyLabel2, c);
// Add the "compute result" checkbox:
c.gridwidth = GridBagConstraints.RELATIVE; //next-to-last
c.fill = GridBagConstraints.NONE; //reset to default
c.weightx = 0.0; //reset to default
paramsPanel.add(computeResultCheckBox);
// Add the compute button
c.gridwidth = GridBagConstraints.REMAINDER; //end row
c.fill = GridBagConstraints.HORIZONTAL;
c.weightx = 1.0;
paramsPanel.add(computeButton, c);
// Status panel
JPanel statusPanel = new JPanel();
statusPanel.setBorder(BorderFactory.createCompoundBorder(
BorderFactory.createTitledBorder("Status"),
BorderFactory.createEmptyBorder(5,5,5,5)));
statusPanel.setLayout(new FlowLayout(FlowLayout.LEFT));
// Add the results text
paramsPanel.add(resultsLabel, c);
// Add both panels into the frame with Border layout
statusPanel.add(resultsLabel);
// Add all panels into the frame with Border layout
add(paramsPanel, BorderLayout.WEST);
add(codePanel);
add(codePanel, BorderLayout.EAST);
add(statusPanel, BorderLayout.SOUTH);
setVisible(true);
@ -492,6 +566,20 @@ public abstract class AutoAnalyser extends JFrame
putCalcPropertiesInTable();
propertiesTableModel.fireTableDataChanged(); // Alerts to refresh the table contents
System.out.println("Added properties for new calculator");
if (statSigCheckBox.isSelected()) {
// We might allow the analytic box to be checked if the selected
// calculator allows this:
if (calcImplementsAnalyticStatSig()) {
statSigAnalyticCheckBox.setEnabled(true);
} else {
statSigAnalyticCheckBox.setSelected(false);
statSigAnalyticCheckBox.setEnabled(false);
}
} else {
statSigAnalyticCheckBox.setSelected(false);
statSigAnalyticCheckBox.setEnabled(false);
}
}
// Else nothing extra to do
}
@ -550,7 +638,8 @@ public abstract class AutoAnalyser extends JFrame
dataRows, dataColumns);
} catch (Exception ex) {
ex.printStackTrace(System.err);
JOptionPane.showMessageDialog(this, ex.getMessage());
JOptionPane.showMessageDialog(this, ex.getMessage() +
(isInts ? ".\nFor Discrete calculator make sure you load a file with integer data!" : ""));
dataFileDescriptorLabel.setText("Invalid data file, please load another");
if (isInts) {
dataDiscrete = null;
@ -574,26 +663,44 @@ public abstract class AutoAnalyser extends JFrame
loadData(true);
if (dataDiscrete == null) {
// An error message will have been shown by loadData()
resultsLabel.setText(" ");
return;
}
} else if (data == null) {
// We need a file of continuous data but no file has been selected
JOptionPane.showMessageDialog(this, "No valid data source selected");
resultsLabel.setText(" ");
return;
}
int sourceColumn = Integer.parseInt(sourceColTextField.getText());
int destColumn = Integer.parseInt(destColTextField.getText());
if ((sourceColumn < 0) || (sourceColumn >= dataColumns)) {
JOptionPane.showMessageDialog(this,
String.format("Source column must be between 0 and %d for this data set",
dataColumns-1));
return;
}
if ((destColumn < 0) || (destColumn >= dataColumns)) {
JOptionPane.showMessageDialog(this,
String.format("Destination column must be between 0 and %d for this data set",
dataColumns-1));
return;
int singleSourceColumn = -1, singleDestColumn = -1;
Vector<int[]> sourceDestPairs = new Vector<int[]>();
if (allPairsCheckBox.isSelected()) {
// We're doing all pairs
for (int s = 0; s < dataColumns; s++) {
for (int d = 0; d < dataColumns; d++) {
sourceDestPairs.add(new int[] {s, d});
}
}
} else {
// we're doing a single source-destination pair
singleSourceColumn = Integer.parseInt(sourceColTextField.getText());
singleDestColumn = Integer.parseInt(destColTextField.getText());
sourceDestPairs.add(new int[] {singleSourceColumn, singleDestColumn});
if ((singleSourceColumn < 0) || (singleSourceColumn >= dataColumns)) {
JOptionPane.showMessageDialog(this,
String.format("Source column must be between 0 and %d for this data set",
dataColumns-1));
resultsLabel.setText(" ");
return;
}
if ((singleDestColumn < 0) || (singleDestColumn >= dataColumns)) {
JOptionPane.showMessageDialog(this,
String.format("Destination column must be between 0 and %d for this data set",
dataColumns-1));
resultsLabel.setText(" ");
return;
}
}
// Generate headers:
@ -601,6 +708,9 @@ public abstract class AutoAnalyser extends JFrame
StringBuffer javaCode = new StringBuffer();
javaCode.append("package infodynamics.demos.autoanalysis;\n\n");
javaCode.append("import infodynamics.utils.ArrayFileReader;\n");
if (statSigCheckBox.isSelected()) {
javaCode.append("import infodynamics.utils.EmpiricalMeasurementDistribution;\n");
}
javaCode.append("import infodynamics.utils.MatrixUtils;\n\n");
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
// Cover all children:
@ -688,12 +798,16 @@ public abstract class AutoAnalyser extends JFrame
javaCode.append(" ArrayFileReader afr = new ArrayFileReader(dataFile);\n");
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
javaCode.append(" int[][] data = afr.getInt2DMatrix();\n");
javaCode.append(" int[] source = MatrixUtils.selectColumn(data, " + sourceColumn + ");\n");
javaCode.append(" int[] dest = MatrixUtils.selectColumn(data, " + destColumn + ");\n\n");
if (! allPairsCheckBox.isSelected()) {
javaCode.append(" int[] source = MatrixUtils.selectColumn(data, " + singleSourceColumn + ");\n");
javaCode.append(" int[] dest = MatrixUtils.selectColumn(data, " + singleDestColumn + ");\n\n");
}
} else {
javaCode.append(" double[][] data = afr.getDouble2DMatrix();\n");
javaCode.append(" double[] source = MatrixUtils.selectColumn(data, " + sourceColumn + ");\n");
javaCode.append(" double[] dest = MatrixUtils.selectColumn(data, " + destColumn + ");\n\n");
if (! allPairsCheckBox.isSelected()) {
javaCode.append(" double[] source = MatrixUtils.selectColumn(data, " + singleSourceColumn + ");\n");
javaCode.append(" double[] dest = MatrixUtils.selectColumn(data, " + singleDestColumn + ");\n\n");
}
}
// 2. Python
pythonCode.append("# " + loadDataComment);
@ -704,18 +818,22 @@ public abstract class AutoAnalyser extends JFrame
}
pythonCode.append("# As numpy array:\n");
pythonCode.append("data = numpy.array(dataRaw)\n");
pythonCode.append("source = data[:," + sourceColumn + "]\n");
pythonCode.append("dest = data[:," + destColumn + "]\n\n");
if (! allPairsCheckBox.isSelected()) {
pythonCode.append("source = JArray(JInt, 1)(data[:," + singleSourceColumn + "].tolist())\n");
pythonCode.append("dest = JArray(JInt, 1)(data[:," + singleDestColumn + "].tolist())\n\n");
}
// 3. Matlab
matlabCode.append("% " + loadDataComment);
matlabCode.append("data = load('" + dataFile.getAbsolutePath() + "');\n");
matlabCode.append("% Column indices start from 1 in Matlab:\n");
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
matlabCode.append("source = octaveToJavaIntArray(data(:," + (sourceColumn+1) + "));\n");
matlabCode.append("dest = octaveToJavaIntArray(data(:," + (destColumn+1) + "));\n\n");
} else {
matlabCode.append("source = octaveToJavaDoubleArray(data(:," + (sourceColumn+1) + "));\n");
matlabCode.append("dest = octaveToJavaDoubleArray(data(:," + (destColumn+1) + "));\n\n");
if (! allPairsCheckBox.isSelected()) {
matlabCode.append("% Column indices start from 1 in Matlab:\n");
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
matlabCode.append("source = octaveToJavaIntArray(data(:," + (singleSourceColumn+1) + "));\n");
matlabCode.append("dest = octaveToJavaIntArray(data(:," + (singleDestColumn+1) + "));\n\n");
} else {
matlabCode.append("source = octaveToJavaDoubleArray(data(:," + (singleSourceColumn+1) + "));\n");
matlabCode.append("dest = octaveToJavaDoubleArray(data(:," + (singleDestColumn+1) + "));\n\n");
}
}
// Construct the calculator and set relevant properties:
@ -736,9 +854,12 @@ public abstract class AutoAnalyser extends JFrame
int minInData = MatrixUtils.min(dataDiscrete);
int maxInData = MatrixUtils.max(dataDiscrete);
if ((minInData < 0) || (maxInData >= base)) {
throw new Exception("Values in data file (in range " + minInData +
JOptionPane.showMessageDialog(this,
"Values in data file (in range " + minInData +
":" + maxInData + ") lie outside the expected range 0:" +
(base-1) + " for base " + base);
resultsLabel.setText(" ");
return;
}
// 1. Java
@ -774,6 +895,7 @@ public abstract class AutoAnalyser extends JFrame
pythonCode.append("# " + setPropertiesComment);
matlabCode.append("% " + setPropertiesComment);
int i = 0;
boolean anyPropValueChanged = false;
for (String propName : propertyNames) {
String propValue = null;
String propFieldName = propertyFieldNames.get(i++);
@ -789,6 +911,7 @@ public abstract class AutoAnalyser extends JFrame
// this calculator. This is more for generating the minimal code.
if (!propValue.equalsIgnoreCase(calcContinuous.getProperty(propName))) {
// We need to set this property:
anyPropValueChanged = true;
calcContinuous.setProperty(propName, propValue);
// 1. Java Code -- use full field name here
javaCode.append(" calc.setProperty(" + propFieldName +
@ -802,80 +925,180 @@ public abstract class AutoAnalyser extends JFrame
propValue + "');\n");
}
}
if (!anyPropValueChanged) {
String noPropValueChangedComment = "No properties were set to non-default values\n";
javaCode.append(" // " + noPropValueChangedComment);
pythonCode.append("# " + noPropValueChangedComment);
matlabCode.append("% " + noPropValueChangedComment);
}
}
String javaPrefix = " ";
String pythonPrefix = "";
String matlabPrefix = "";
if (sourceDestPairs.size() > 1) {
// We're doing all pairs
// Set up loops in the code:
// 1. Java code
javaCode.append(" \n");
javaCode.append(" // Compute for all pairs:\n");
javaCode.append(" for (int s = 0; s < " + dataColumns +
"; s++) {\n");
javaCode.append(" for (int d = 0; d < " + dataColumns +
"; d++) {\n");
javaPrefix = " ";
javaCode.append(javaPrefix + "// For each source-dest pair:\n");
javaCode.append(javaPrefix + "if (s == d) {\n");
javaCode.append(javaPrefix + " continue;\n");
javaCode.append(javaPrefix + "}\n");
// 2. Python code
pythonCode.append("\n");
pythonCode.append("# Compute for all pairs:\n");
pythonCode.append("for s in range(" + dataColumns + "):\n");
pythonCode.append("\tfor d in range(" + dataColumns + "):\n");
pythonPrefix = "\t\t";
pythonCode.append(pythonPrefix+ "# For each source-dest pair:\n");
pythonCode.append(pythonPrefix + "if (s == d):\n");
pythonCode.append(pythonPrefix + "\tcontinue\n");
// 3. Matlab code
matlabCode.append("\n");
matlabCode.append("% Compute for all pairs:\n");
matlabCode.append("for s = 1:" + dataColumns + "\n");
matlabCode.append("\tfor d = 1:" + dataColumns + "\n");
matlabPrefix = "\t\t";
matlabCode.append(matlabPrefix + "% For each source-dest pair:\n");
matlabCode.append(matlabPrefix + "if (s == d)\n");
matlabCode.append(matlabPrefix + "\tcontinue;\n");
matlabCode.append(matlabPrefix + "end\n");
// Read in the data for these columns for all pairs
matlabCode.append(matlabPrefix + "% Column indices start from 1 in Matlab:\n");
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
javaCode.append(javaPrefix + "int[] source = MatrixUtils.selectColumn(data, s);\n");
javaCode.append(javaPrefix + "int[] dest = MatrixUtils.selectColumn(data, d);\n\n");
matlabCode.append(matlabPrefix + "source = octaveToJavaIntArray(data(:, s));\n");
matlabCode.append(matlabPrefix + "dest = octaveToJavaIntArray(data(:, d));\n\n");
pythonCode.append(pythonPrefix + "source = JArray(JInt, 1)(data[:, s].tolist())\n");
pythonCode.append(pythonPrefix + "dest = JArray(JInt, 1)(data[:, d].tolist())\n\n");
} else {
javaCode.append(javaPrefix + "double[] source = MatrixUtils.selectColumn(data, s);\n");
javaCode.append(javaPrefix + "double[] dest = MatrixUtils.selectColumn(data, d);\n\n");
matlabCode.append(matlabPrefix + "source = octaveToJavaDoubleArray(data(:, s));\n");
matlabCode.append(matlabPrefix + "dest = octaveToJavaDoubleArray(data(:, d));\n\n");
pythonCode.append(pythonPrefix + "source = data[:, s]\n");
pythonCode.append(pythonPrefix + "dest = data[:, d]\n\n");
}
} else {
// For a single pair, we don't need to set up loops, and
// we've already read in the data to source and dest variables above
}
// Initialise
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
calcDiscrete.initialise();
} else {
calcContinuous.initialise();
}
String initialiseComment = "3. Initialise the calculator for (re-)use:\n";
javaCode.append(" // " + initialiseComment);
javaCode.append(" calc.initialise();\n");
pythonCode.append("# " + initialiseComment);
pythonCode.append("calc.initialise()\n");
matlabCode.append("% " + initialiseComment);
matlabCode.append("calc.initialise();\n");
javaCode.append(javaPrefix + "// " + initialiseComment);
javaCode.append(javaPrefix + "calc.initialise();\n");
pythonCode.append(pythonPrefix + "# " + initialiseComment);
pythonCode.append(pythonPrefix + "calc.initialise()\n");
matlabCode.append(matlabPrefix + "% " + initialiseComment);
matlabCode.append(matlabPrefix + "calc.initialise();\n");
// Set observations
String supplyDataComment = "4. Supply the sample data:\n";
String setObservationsMethod;
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
setObservationsMethod = "addObservations";
calcDiscrete.addObservations(
MatrixUtils.selectColumn(dataDiscrete, sourceColumn),
MatrixUtils.selectColumn(dataDiscrete, destColumn));
} else {
setObservationsMethod = "setObservations";
// TODO We should be able to directly call setObservations(double[], double[])
// here but we can't right now because ChannelCalculator does not
// define this method. It would be complicated to fix this
// because it involves Conditional MI calculator it seems.
// Revisit this later -- for now fix by deferring to child classes
// calcContinuous.setObservations(
// MatrixUtils.selectColumn(data, sourceColumn),
// MatrixUtils.selectColumn(data, destColumn));
setObservations(calcContinuous,
MatrixUtils.selectColumn(data, sourceColumn),
MatrixUtils.selectColumn(data, destColumn));
}
// 1. Java
javaCode.append(" // " + supplyDataComment);
javaCode.append(" calc." + setObservationsMethod + "(source, dest);\n");
javaCode.append(javaPrefix + "// " + supplyDataComment);
javaCode.append(javaPrefix + "calc." + setObservationsMethod + "(source, dest);\n");
// 2. Python
pythonCode.append("# " + supplyDataComment);
pythonCode.append("calc." + setObservationsMethod + "(source, dest)\n");
pythonCode.append(pythonPrefix + "# " + supplyDataComment);
pythonCode.append(pythonPrefix + "calc." + setObservationsMethod + "(source, dest)\n");
// 3. Matlab
matlabCode.append("% " + supplyDataComment);
matlabCode.append("calc." + setObservationsMethod + "(source, dest);\n");
matlabCode.append(matlabPrefix + "% " + supplyDataComment);
matlabCode.append(matlabPrefix + "calc." + setObservationsMethod + "(source, dest);\n");
// Compute the result
double result;
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
result = calcDiscrete.computeAverageLocalOfObservations();
} else {
result = calcContinuous.computeAverageLocalOfObservations();
}
String computeComment = "5. Compute the estimate:\n";
javaCode.append(" // " + computeComment);
javaCode.append(" double result = calc.computeAverageLocalOfObservations();\n");
pythonCode.append("# " + computeComment);
pythonCode.append("result = calc.computeAverageLocalOfObservations()\n");
matlabCode.append("% " + computeComment);
matlabCode.append("result = calc.computeAverageLocalOfObservations();\n");
String resultsPrefixString = String.format(measureAcronym + "_%s(col_%d -> col_%d) = ",
selectedCalcType, sourceColumn, destColumn);
resultsLabel.setText(String.format(resultsPrefixString + "%.4f %s", result, units));
javaCode.append(javaPrefix + "// " + computeComment);
javaCode.append(javaPrefix + "double result = calc.computeAverageLocalOfObservations();\n");
pythonCode.append(pythonPrefix + "# " + computeComment);
pythonCode.append(pythonPrefix + "result = calc.computeAverageLocalOfObservations()\n");
matlabCode.append(matlabPrefix + "% " + computeComment);
matlabCode.append(matlabPrefix + "result = calc.computeAverageLocalOfObservations();\n");
String resultsPrefixString, extraFormatTerms;
if (sourceDestPairs.size() > 1) {
resultsPrefixString = String.format(measureAcronym + "_%s(col_%%d -> col_%%d) = ",
selectedCalcType);
extraFormatTerms = "s, d, ";
} else {
resultsPrefixString = String.format(measureAcronym + "_%s(col_%d -> col_%d) = ",
selectedCalcType, singleSourceColumn, singleDestColumn);
extraFormatTerms = "";
}
String resultsSuffixString = "";
String statSigFormatTerms = "";
if (statSigCheckBox.isSelected()) {
// Compute statistical significance
String statSigComment = "6. Compute the (statistical significance via) null distribution (e.g. 100 permutations):\n";
javaCode.append(javaPrefix + "// " + statSigComment);
javaCode.append(javaPrefix + "EmpiricalMeasurementDistribution measDist = calc.computeSignificance(100);\n");
pythonCode.append(pythonPrefix + "# " + statSigComment);
pythonCode.append(pythonPrefix + "measDist = calc.computeSignificance(100)\n");
matlabCode.append(matlabPrefix + "% " + statSigComment);
matlabCode.append(matlabPrefix + "measDist = calc.computeSignificance(100);\n");
if (statSigAnalyticCheckBox.isSelected()) {
resultsSuffixString = " (analytic p(surrogate > measured)=%.5f)";
statSigFormatTerms = ", measDist.pValue";
} else {
resultsSuffixString = " (null: %.4f +/- %.4f std dev.; p(surrogate > measured)=%.5f from %d surrogates)";
statSigFormatTerms = String.format(", measDist.getMeanOfDistribution(), measDist.getStdOfDistribution(), measDist.pValue, %d",
numPermutationsToCheck);
}
}
// And generate code to write the results and finalise:
// 1. Java
javaCode.append(" System.out.printf(\"" + resultsPrefixString + "%.4f " + units + "\\n\", result);\n");
javaCode.append("\n" + javaPrefix + "System.out.printf(\"" + resultsPrefixString +
"%.4f " + units + resultsSuffixString + "\\n\",\n " + javaPrefix +
extraFormatTerms + "result" + statSigFormatTerms + ");\n");
// 2. Python
pythonCode.append("\n" + pythonPrefix + "print(\"" + resultsPrefixString +
"%.4f " + units + resultsSuffixString + "\" %\n\t" + pythonPrefix + "(" +
extraFormatTerms + "result" + statSigFormatTerms + "))\n");
// 3. Matlab
matlabCode.append("\n" + matlabPrefix + "fprintf('" + resultsPrefixString +
"%.4f " + units + resultsSuffixString + "\\n', ...\n\t" + matlabPrefix +
extraFormatTerms + "result" + statSigFormatTerms + ");\n");
if (sourceDestPairs.size() > 1) {
// We're doing all pairs
// Finalise the loops in the code:
// 1. Java code
javaCode.append(" }\n");
javaCode.append(" }\n");
// 2. Python code
// Nothing to do
// 3. Matlab code
matlabCode.append("\tend\n");
matlabCode.append("end\n");
// And tell the user to see results in the console:
resultsLabel.setText("See console for all pairs calculation results");
if (!selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
System.out.println("Property values not read back into GUI for all pairs calculation");
}
}
// Finalise the Java code:
javaCode.append(" }\n");
javaCode.append("}\n\n");
// 2. Python
pythonCode.append("print(\"" + resultsPrefixString + "%.4f " + units + "\\n\" % result)\n");
// 3. Matlab
matlabCode.append("fprintf('" + resultsPrefixString + "%.4f " + units + "\\n', result);\n");
// Now set the code in the panel for the user
javaCodeTextArea.setText(javaCode.toString());
@ -899,22 +1122,123 @@ public abstract class AutoAnalyser extends JFrame
codeFileWriter.write(matlabCode.toString());
codeFileWriter.close();
if (!selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
// Read the current property values back out (in case of
// automated parameter assignment)
for (String propName : propertyNames) {
String propValue = null;
try {
propValue = calcContinuous.getProperty(propName);
propertyValues.put(propName, propValue);
} catch (Exception ex) {
ex.printStackTrace(System.err);
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot find a value for property " + propName);
if (computeResultCheckBox.isSelected()) {
// Now make our own calculations here:
// TODO make the calculation in a separate thread, and have a
// button to cancel the calculation whilst it's running
for (int[] pair : sourceDestPairs) {
int sourceColumn = pair[0];
int destColumn = pair[1];
if (sourceColumn == destColumn) {
continue;
}
// Initialise
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
calcDiscrete.initialise();
} else {
calcContinuous.initialise();
}
// Set observations
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
calcDiscrete.addObservations(
MatrixUtils.selectColumn(dataDiscrete, sourceColumn),
MatrixUtils.selectColumn(dataDiscrete, destColumn));
} else {
// TODO We should be able to directly call setObservations(double[], double[])
// here but we can't right now because ChannelCalculator does not
// define this method. It would be complicated to fix this
// because it involves Conditional MI calculator it seems.
// Revisit this later -- for now fix by deferring to child classes
// calcContinuous.setObservations(
// MatrixUtils.selectColumn(data, sourceColumn),
// MatrixUtils.selectColumn(data, destColumn));
setObservations(calcContinuous,
MatrixUtils.selectColumn(data, sourceColumn),
MatrixUtils.selectColumn(data, destColumn));
}
// Compute the result:
double result;
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
result = calcDiscrete.computeAverageLocalOfObservations();
} else {
result = calcContinuous.computeAverageLocalOfObservations();
}
String resultsSuffixStringFormatted = "";
if (statSigCheckBox.isSelected()) {
// Check statistical significance:
if (statSigAnalyticCheckBox.isSelected()) {
// analytic check of statistical significance:
AnalyticNullDistributionComputer analyticCalc;
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
analyticCalc =
(AnalyticNullDistributionComputer) calcDiscrete;
} else {
analyticCalc =
(AnalyticNullDistributionComputer) calcContinuous;
}
AnalyticMeasurementDistribution measDist =
analyticCalc.computeSignificance();
resultsSuffixStringFormatted = String.format(
resultsSuffixString,
measDist.pValue);
} else {
// permutation test of statistical significance:
EmpiricalMeasurementDistribution measDist;
if (selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
measDist = calcDiscrete.computeSignificance(numPermutationsToCheck);
} else {
measDist = calcContinuous.computeSignificance(numPermutationsToCheck);
}
resultsSuffixStringFormatted = String.format(
resultsSuffixString,
measDist.getMeanOfDistribution(),
measDist.getStdOfDistribution(),
measDist.pValue,
numPermutationsToCheck);
}
}
String resultsText;
if (sourceDestPairs.size() > 1) {
// We're doing all pairs
resultsText = String.format(
resultsPrefixString + "%.4f %s" + resultsSuffixStringFormatted,
sourceColumn, destColumn, result, units);
} else {
resultsText = String.format(
resultsPrefixString + "%.4f %s" + resultsSuffixStringFormatted,
result, units);
resultsLabel.setText(resultsText);
}
// Write the results to the console:
System.out.println(resultsText);
}
propertiesTableModel.fireTableDataChanged(); // Alerts to refresh the table contents
if ((sourceDestPairs.size() == 1) &&
!selectedCalcType.equalsIgnoreCase(CALC_TYPE_DISCRETE)) {
// Read the current property values back out (in case of
// automated parameter assignment)
for (String propName : propertyNames) {
String propValue = null;
try {
propValue = calcContinuous.getProperty(propName);
propertyValues.put(propName, propValue);
} catch (Exception ex) {
ex.printStackTrace(System.err);
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot find a value for property " + propName);
}
}
propertiesTableModel.fireTableDataChanged(); // Alerts to refresh the table contents
}
} else {
resultsLabel.setText("Code generated (no computation performed)");
}
} catch (Exception ex) {
ex.printStackTrace(System.err);
@ -1263,4 +1587,68 @@ public abstract class AutoAnalyser extends JFrame
public void mouseReleased(MouseEvent me) {
// Do nothing
}
@Override
public void stateChanged(ChangeEvent e) {
// I'm not sure how to check which checkbox changed,
// so we'll just act on potential changes from both.
// This won't cause a problem here
if (e.getSource() == allPairsCheckBox) {
// "All pairs" checkbox -- update in case changed:
if (allPairsCheckBox.isSelected()) {
sourceColTextField.setEnabled(false);
destColTextField.setEnabled(false);
} else {
sourceColTextField.setEnabled(true);
destColTextField.setEnabled(true);
}
} else if (e.getSource() == computeResultCheckBox) {
// Compute checkbox -- update in case changed:
if (computeResultCheckBox.isSelected()) {
computeButton.setText(buttonTextCodeAndCompute);
} else {
computeButton.setText(buttonTextCodeOnly);
}
} else if (e.getSource() == statSigCheckBox) {
System.out.println("StatSigCheckBox state changed to " + statSigCheckBox.isSelected());
// statSigCheckBox -- update statSigAnalyticCheckBox in case changed:
if (statSigCheckBox.isSelected()) {
// We might allow the analytic box to be checked if the selected
// calculator allows this:
if (calcImplementsAnalyticStatSig()) {
statSigAnalyticCheckBox.setEnabled(true);
} else {
statSigAnalyticCheckBox.setSelected(false);
statSigAnalyticCheckBox.setEnabled(false);
}
} else {
statSigAnalyticCheckBox.setSelected(false);
statSigAnalyticCheckBox.setEnabled(false);
}
}
}
/**
* Checks whether the current calculator class implements the
* AnalyticNullDistributionComputer interface
*
* @return
*/
protected boolean calcImplementsAnalyticStatSig() {
Class<?> currentClass = calcClass;
while (currentClass != null) {
// This getInterfaces call doesn't seem to return the interfaces
// implemented by the superclass, so we need to go up the tree
// to check all of those. That seems odd, surely there is a simpler way?
Class<?>[] interfacesOfThisClass = currentClass.getInterfaces();
for (Class<?> theInterface : interfacesOfThisClass) {
if (theInterface == AnalyticNullDistributionComputer.class) {
return true;
}
}
currentClass = currentClass.getSuperclass();
}
return false;
}
}

83
demos/java/infodynamics/demos/autoanalysis/AutoAnalyserTE.java
View File

@ -50,6 +50,10 @@ public class AutoAnalyserTE extends AutoAnalyser
private static final long serialVersionUID = 1L;
protected static final String DISCRETE_PROPNAME_K = "k_HISTORY";
protected static final String DISCRETE_PROPNAME_K_TAU = "k_TAU";
protected static final String DISCRETE_PROPNAME_L = "l_HISTORY";
protected static final String DISCRETE_PROPNAME_L_TAU = "l_TAU";
protected static final String DISCRETE_PROPNAME_DELAY = "DELAY";
/**
* Constructor to initialise the GUI for TE
@ -63,16 +67,28 @@ public class AutoAnalyserTE extends AutoAnalyser
// Discrete:
discreteClass = TransferEntropyCalculatorDiscrete.class;
discreteProperties = new String[] {
DISCRETE_PROPNAME_BASE,
DISCRETE_PROPNAME_K,
DISCRETE_PROPNAME_BASE
DISCRETE_PROPNAME_K_TAU,
DISCRETE_PROPNAME_L,
DISCRETE_PROPNAME_L_TAU,
DISCRETE_PROPNAME_DELAY
};
discretePropertyDefaultValues = new String[] {
"2",
"1",
"2"
"1",
"1",
"1",
"1"
};
discretePropertyDescriptions = new String[] {
"Destination history embedding length",
"Number of discrete states available for each variable (i.e. 2 for binary)"
"Number of discrete states available for each variable (i.e. 2 for binary)",
"Destination history embedding length (k_HISTORY)",
"Destination history embedding delay (k_TAU)",
"Source history embedding length (l_HISTORY)",
"Source history embeding delay (l_TAU)",
"Delay from source to destination (in time steps)",
};
// Continuous:
@ -183,7 +199,10 @@ public class AutoAnalyserTE extends AutoAnalyser
"\" meaning values are set manually; other values include: <br/> -- \"" + TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_RAGWITZ +
"\" for use of the Ragwitz criteria for both source and destination (searching up to \"" + TransferEntropyCalculatorKraskov.PROP_K_SEARCH_MAX +
"\" and \"" + TransferEntropyCalculatorKraskov.PROP_TAU_SEARCH_MAX + "\"); <br/> -- \"" + TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY +
"\" for use of the Ragwitz criteria for the destination only. <br/>Use of values other than \"" + TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_NONE +
"\" for use of the Ragwitz criteria for the destination only. <br/> -- \"" + TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_MAX_CORR_AIS +
"\" for maximising the (bias corrected) Active Info Storage for both source and destination (searching up to \"" + TransferEntropyCalculatorKraskov.PROP_K_SEARCH_MAX +
"\" and \"" + TransferEntropyCalculatorKraskov.PROP_TAU_SEARCH_MAX + "\"); <br/> -- \"" + TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_MAX_CORR_AIS_DEST_ONLY +
"\" for maximising the (bias corrected) Active Info Storage for the destination only. <br/>Use of values other than \"" + TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_NONE +
"\" leads to any previous settings for embedding lengths and delays for the destination and perhaps source to be overwritten after observations are supplied",
"Max. embedding length to search to <br/>if auto embedding (as determined by " + TransferEntropyCalculatorKraskov.PROP_AUTO_EMBED_METHOD + ")",
"Max. embedding delay to search to <br/>if auto embedding (as determined by " + TransferEntropyCalculatorKraskov.PROP_AUTO_EMBED_METHOD + ")",
@ -213,30 +232,66 @@ public class AutoAnalyserTE extends AutoAnalyser
* Method to assign and initialise our discrete calculator class
*/
protected DiscreteCalcAndArguments assignCalcObjectDiscrete() throws Exception {
String kPropValueStr, basePropValueStr;
int base, k, k_tau, l, l_tau, delay;
try {
kPropValueStr = propertyValues.get(DISCRETE_PROPNAME_K);
String basePropValueStr = propertyValues.get(DISCRETE_PROPNAME_BASE);
base = Integer.parseInt(basePropValueStr);
} catch (Exception ex) {
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot find a value for property " + DISCRETE_PROPNAME_K);
resultsLabel.setText("Cannot read a value for property " + DISCRETE_PROPNAME_BASE);
return null;
}
try {
basePropValueStr = propertyValues.get(DISCRETE_PROPNAME_BASE);
String kPropValueStr = propertyValues.get(DISCRETE_PROPNAME_K);
k = Integer.parseInt(kPropValueStr);
} catch (Exception ex) {
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot find a value for property " + DISCRETE_PROPNAME_BASE);
resultsLabel.setText("Cannot read a value for property " + DISCRETE_PROPNAME_K);
return null;
}
try {
String kTauPropValueStr = propertyValues.get(DISCRETE_PROPNAME_K_TAU);
k_tau = Integer.parseInt(kTauPropValueStr);
} catch (Exception ex) {
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot read a value for property " + DISCRETE_PROPNAME_K_TAU);
return null;
}
try {
String lPropValueStr = propertyValues.get(DISCRETE_PROPNAME_L);
l = Integer.parseInt(lPropValueStr);
} catch (Exception ex) {
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot read a value for property " + DISCRETE_PROPNAME_L);
return null;
}
try {
String lTauPropValueStr = propertyValues.get(DISCRETE_PROPNAME_L_TAU);
l_tau = Integer.parseInt(lTauPropValueStr);
} catch (Exception ex) {
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot read a value for property " + DISCRETE_PROPNAME_L_TAU);
return null;
}
try {
String delayPropValueStr = propertyValues.get(DISCRETE_PROPNAME_DELAY);
delay = Integer.parseInt(delayPropValueStr);
} catch (Exception ex) {
JOptionPane.showMessageDialog(this,
ex.getMessage());
resultsLabel.setText("Cannot read a value for property " + DISCRETE_PROPNAME_DELAY);
return null;
}
int k = Integer.parseInt(kPropValueStr);
int base = Integer.parseInt(basePropValueStr);
return new DiscreteCalcAndArguments(
new TransferEntropyCalculatorDiscrete(base, k),
new TransferEntropyCalculatorDiscrete(base, k, k_tau, l, l_tau, delay),
base,
base + ", " + k);
base + ", " + k + ", " + k_tau + ", " + l + ", " + l_tau + ", " + delay);
}
protected void setObservations(ChannelCalculatorCommon calc,

BIN
demos/octave/README-OctaveMatlabDemos.pdf
View File

Binary file not shown.

76
demos/octave/example10ConditionalTeKraskov.m Normal file
View File

@ -0,0 +1,76 @@
%%
%% Java Information Dynamics Toolkit (JIDT)
%% Copyright (C) 2016 Joseph T. Lizier
%%
%% This program is free software: you can redistribute it and/or modify
%% it under the terms of the GNU General Public License as published by
%% the Free Software Foundation, either version 3 of the License, or
%% (at your option) any later version.
%%
%% This program is distributed in the hope that it will be useful,
%% but WITHOUT ANY WARRANTY; without even the implied warranty of
%% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
%% GNU General Public License for more details.
%%
%% You should have received a copy of the GNU General Public License
%% along with this program. If not, see <http://www.gnu.org/licenses/>.
%%
% = Example 10 - Conditional Transfer entropy on continuous multivariate data using Kraskov estimators =
% Conditional Transfer entropy (TE) calculation on multivariate continuous-valued data using the Kraskov-estimator TE calculator.
% Change location of jar to match yours:
javaaddpath('../../infodynamics.jar')
% Generate some random normalised data.
numObservations = 100000;
% Keep the sum of squares of these covariances below 1 to allow proper calculation of noise term.
covarianceToSource = 0.4;
covarianceToConds = [0.3,0.3];
if (sumsq([covarianceToSource, covarianceToConds]) >= 1)
error('Sum of squares of the covariances must be < 1 here');
end
noiseCovar = sqrt(1 - sumsq([covarianceToSource, covarianceToConds]));
% Generate the random variables
sourceArray = randn(numObservations, 1);
condArray = randn(numObservations, length(covarianceToConds));
destArray = [0; covarianceToSource*sourceArray(1:numObservations-1,:) + covarianceToConds(1)*condArray(1:numObservations-1,1) + covarianceToConds(2)*condArray(1:numObservations-1,2) + noiseCovar*randn(numObservations-1, 1)];
% Expected results:
expectedConditional = -0.5 * log(noiseCovar .* noiseCovar ./ (1 - sumsq(covarianceToConds)));
expectedPairwise = -0.5 * log(1-covarianceToSource.^2);
% Create a conditional TE calculator and run it:
teCalc=javaObject('infodynamics.measures.continuous.kraskov.ConditionalTransferEntropyCalculatorKraskov');
teCalc.initialise(1,1, ... % Destination embedding length (Schreiber k=1) and delays
1,1, ... % Source embedding length (Schreiber l=1) and delays
1, ... % Source-destination delay of 1 (default)
octaveToJavaDoubleArray([1,1]), ... % Embedding lengths for each conditional variable
octaveToJavaDoubleArray([1,1]), ... % Embedding delays for each conditional variable
octaveToJavaDoubleArray([1,1]) ... % conditional-destination delays for each conditional variable
);
teCalc.setObservations(octaveToJavaDoubleArray(sourceArray), ...
octaveToJavaDoubleArray(destArray), ...
octaveToJavaDoubleMatrix(condArray));
% Perform calculation with correlated source, but no conditioning on other sources:
conditionalResult = teCalc.computeAverageLocalOfObservations();
% Create a pairwise TE calculator and run it:
teCalc=javaObject('infodynamics.measures.continuous.kraskov.TransferEntropyCalculatorKraskov');
teCalc.initialise(); % Use default embeddings of 1 (e.g. Schreiber k=1 and l=1)
teCalc.setObservations(octaveToJavaDoubleArray(sourceArray), octaveToJavaDoubleArray(destArray));
% Perform calculation with correlated source, but no conditioning on other sources:
pairwiseResult = teCalc.computeAverageLocalOfObservations();
% Note that the calculation is a random variable (because the generated
% data is a set of random variables) - the result will be of the order
% of what we expect, but not exactly equal to it; in fact, there will
% be some variance around it. It will probably be biased down here
% due to small correlations between the supposedly uncorrelated variables.
fprintf('From %d samples:\nTE result conditional result = %.4f nats, pairwise = %.4f nats;\nexpected around %.4f nats (conditional) and %.4f nats (pairwise) for the correlated Gaussians\n', ...
numObservations, conditionalResult, pairwiseResult, expectedConditional, expectedPairwise);
clear teCalc

BIN
demos/python/README-PythonDemos.pdf
View File

Binary file not shown.

BIN
demos/python/README-UseInPython.pdf
View File

Binary file not shown.

25
demos/python/example1TeBinaryData.py
View File

@ -20,13 +20,14 @@
# Simple transfer entropy (TE) calculation on binary data using the discrete TE calculator:
from jpype import *
import jpype
import random
import numpy
# Change location of jar to match yours:
jarLocation = "../../infodynamics.jar"
# Start the JVM (add the "-Xmx" option with say 1024M if you get crashes due to not enough memory space)
startJVM(getDefaultJVMPath(), "-ea", "-Djava.class.path=" + jarLocation)
jpype.startJVM(jpype.getDefaultJVMPath(), "-ea", "-Djava.class.path=" + jarLocation)
# Generate some random binary data.
sourceArray = [random.randint(0,1) for r in range(100)]
@ -34,15 +35,29 @@ destArray = [0] + sourceArray[0:99]
sourceArray2 = [random.randint(0,1) for r in range(100)]
# Create a TE calculator and run it:
teCalcClass = JPackage("infodynamics.measures.discrete").TransferEntropyCalculatorDiscrete
teCalcClass = jpype.JPackage("infodynamics.measures.discrete").TransferEntropyCalculatorDiscrete
teCalc = teCalcClass(2,1)
teCalc.initialise()
# Since we have simple arrays of ints, we can directly pass these in:
# First use simple arrays of ints, which we can directly pass in:
teCalc.addObservations(sourceArray, destArray)
print("For copied source, result should be close to 1 bit : %.4f" % teCalc.computeAverageLocalOfObservations())
teCalc.initialise()
teCalc.addObservations(sourceArray2, destArray)
print("For random source, result should be close to 0 bits: %.4f" % teCalc.computeAverageLocalOfObservations())
shutdownJVM()
# Next, demonstrate how to do this with a numpy array
teCalc.initialise()
# Create the numpy arrays:
sourceNumpy = numpy.array(sourceArray, dtype=numpy.int)
destNumpy = numpy.array(destArray, dtype=numpy.int)
# The above can be passed straight through to JIDT in python 2:
# teCalc.addObservations(sourceNumpy, destNumpy)
# But you need to do this in python 3:
sourceNumpyJArray = jpype.JArray(jpype.JInt, 1)(sourceNumpy.tolist())
destNumpyJArray = jpype.JArray(jpype.JInt, 1)(destNumpy.tolist())
teCalc.addObservations(sourceNumpyJArray, destNumpyJArray)
print("Using numpy array for copied source, result confirmed as: %.4f" % teCalc.computeAverageLocalOfObservations())
jpype.shutdownJVM()

2
demos/python/example6DynamicCallingMutualInfo.py Executable file → Normal file
View File

@ -75,7 +75,7 @@ jointVariable2 = A[:,jointVariable2Columns]
# (in fact, all java object creation in python is dynamic - it has to be,
# since the languages are interpreted. This makes our life slightly easier at this
# point than it is in demos/java/example6 where we have to handle this manually)
indexOfLastDot = string.rfind(implementingClass, ".")
indexOfLastDot = implementingClass.rfind(".")
implementingPackage = implementingClass[:indexOfLastDot]
implementingBaseName = implementingClass[indexOfLastDot+1:]
miCalcClass = eval('JPackage(\'%s\').%s' % (implementingPackage, implementingBaseName))

86
demos/python/example7EnsembleMethodTeContinuousDataKraskov.py Normal file
View File

@ -0,0 +1,86 @@
##
## Java Information Dynamics Toolkit (JIDT)
## Copyright (C) 2012, Joseph T. Lizier
##
## This program is free software: you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 3 of the License, or
## (at your option) any later version.
##
## This program is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
## GNU General Public License for more details.
##
## You should have received a copy of the GNU General Public License
## along with this program. If not, see <http://www.gnu.org/licenses/>.
##
# = Example 7 - Ensemble method with transfer entropy on continuous data using Kraskov estimators =
# Calculation of transfer entropy (TE) by supplying an ensemble of samples from multiple time series.
# We use continuous-valued data using the Kraskov-estimator TE calculator here.
from jpype import *
import random
import math
# Change location of jar to match yours:
jarLocation = "../../infodynamics.jar"
# Start the JVM (add the "-Xmx" option with say 1024M if you get crashes due to not enough memory space)
startJVM(getDefaultJVMPath(), "-ea", "-Djava.class.path=" + jarLocation)
# Generate some random normalised data.
numObservations = 1000
covariance=0.4
numTrials=10
kHistoryLength=1
# Create a TE calculator and run it:
teCalcClass = JPackage("infodynamics.measures.continuous.kraskov").TransferEntropyCalculatorKraskov
teCalc = teCalcClass()
teCalc.setProperty("k", "4") # Use Kraskov parameter K=4 for 4 nearest points
teCalc.initialise(kHistoryLength) # Use target history length of kHistoryLength (Schreiber k)
teCalc.startAddObservations()
for trial in range(0,numTrials):
# Create a new trial, with destArray correlated to
# previous value of sourceArray:
sourceArray = [random.normalvariate(0,1) for r in range(numObservations)]
destArray = [0] + [sum(pair) for pair in zip([covariance*y for y in sourceArray[0:numObservations-1]], \
[(1-covariance)*y for y in [random.normalvariate(0,1) for r in range(numObservations-1)]] ) ]
# Add observations for this trial:
print("Adding samples from trial %d ..." % trial)
teCalc.addObservations(JArray(JDouble, 1)(sourceArray), JArray(JDouble, 1)(destArray))
# We've finished adding trials:
print("Finished adding trials")
teCalc.finaliseAddObservations()
# Compute the result:
print("Computing TE ...")
result = teCalc.computeAverageLocalOfObservations()
# Note that the calculation is a random variable (because the generated
# data is a set of random variables) - the result will be of the order
# of what we expect, but not exactly equal to it; in fact, there will
# be some variance around it (smaller than example 4 since we have more samples).
print("TE result %.4f nats; expected to be close to %.4f nats for these correlated Gaussians " % \
(result, math.log(1.0/(1-math.pow(covariance,2)))))
# And here's how to pull the local TEs out corresponding to each input time
# series under the ensemble method (i.e. for multiple trials).
localTEs=teCalc.computeLocalOfPreviousObservations()
localValuesPerTrial = teCalc.getSeparateNumObservations() # Need to convert to int for indices later
startIndex = 0
for localValuesInThisTrial in localValuesPerTrial:
endIndex = startIndex + localValuesInThisTrial - 1
print("Local TEs for trial %d go from array index %d to %d" % (trial, startIndex, endIndex))
print(" corresponding to time points %d:%d (indexed from 0) of that trial" % (kHistoryLength, numObservations-1))
# Access the local TEs for this trial as:
localTEForThisTrial = localTEs[startIndex:endIndex]
# Now update the startIndex before we go to the next trial
startIndex = endIndex + 1
# And make a sanity check that we've looked at all of the local values here:
print("We've looked at %d local values in total, matching the number of samples we have (%d)" % (startIndex, teCalc.getNumObservations()))

93
demos/python/example9TeKraskovAutoEmbedding.py Normal file
View File

@ -0,0 +1,93 @@
##
## Java Information Dynamics Toolkit (JIDT)
## Copyright (C) 2012, Joseph T. Lizier
##
## This program is free software: you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 3 of the License, or
## (at your option) any later version.
##
## This program is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
## GNU General Public License for more details.
##
## You should have received a copy of the GNU General Public License
## along with this program. If not, see <http://www.gnu.org/licenses/>.
##
# = Example 9 - Transfer entropy on continuous data using Kraskov estimators with auto-embedding =
# Transfer entropy (TE) calculation on continuous-valued data using the Kraskov-estimator TE calculator,
# with automatic selection of embedding parameters
from jpype import *
import random
import math
import numpy
import readFloatsFile
# Change location of jar to match yours:
jarLocation = "../../infodynamics.jar"
# Start the JVM (add the "-Xmx" option with say 1024M if you get crashes due to not enough memory space)
startJVM(getDefaultJVMPath(), "-ea", "-Djava.class.path=" + jarLocation)
# Examine the heart-breath interaction that Schreiber originally looked at:
datafile = '../data/SFI-heartRate_breathVol_bloodOx.txt'
data = readFloatsFile.readFloatsFile(datafile)
# As numpy array:
A = numpy.array(data)
# Select data points 2350:3550, pulling out the relevant columns:
breathRate = A[2350:3551,1];
heartRate = A[2350:3551,0];
# Create a Kraskov TE calculator:
teCalcClass = JPackage("infodynamics.measures.continuous.kraskov").TransferEntropyCalculatorKraskov
teCalc = teCalcClass()
# Set properties for auto-embedding of both source and destination
# using the Ragwitz criteria:
# a. Auto-embedding method
teCalc.setProperty(teCalcClass.PROP_AUTO_EMBED_METHOD,
teCalcClass.AUTO_EMBED_METHOD_RAGWITZ)
# b. Search range for embedding dimension (k) and delay (tau)
teCalc.setProperty(teCalcClass.PROP_K_SEARCH_MAX, "6")
teCalc.setProperty(teCalcClass.PROP_TAU_SEARCH_MAX, "6")
# Since we're auto-embedding, no need to supply k, l, k_tau, l_tau here:
teCalc.initialise()
# Compute TE from breath (column 1) to heart (column 0)
teCalc.setObservations(breathRate, heartRate)
teBreathToHeart = teCalc.computeAverageLocalOfObservations()
# Check the auto-selected parameters and print out the result:
optimisedK = int(teCalc.getProperty(teCalcClass.K_PROP_NAME))
optimisedKTau = int(teCalc.getProperty(teCalcClass.K_TAU_PROP_NAME))
optimisedL = int(teCalc.getProperty(teCalcClass.L_PROP_NAME))
optimisedLTau = int(teCalc.getProperty(teCalcClass.L_TAU_PROP_NAME))
print(("TE(breath->heart) was %.3f nats for (heart embedding:) k=%d," + \
"k_tau=%d, (breath embedding:) l=%d,l_tau=%d optimised via Ragwitz criteria") % \
(teBreathToHeart, optimisedK, optimisedKTau, optimisedL, optimisedLTau))
# Next, embed the destination only using the Ragwitz criteria:
teCalc.setProperty(teCalcClass.PROP_AUTO_EMBED_METHOD,
teCalcClass.AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY)
teCalc.setProperty(teCalcClass.PROP_K_SEARCH_MAX, "6")
teCalc.setProperty(teCalcClass.PROP_TAU_SEARCH_MAX, "6")
# Since we're only auto-embedding the destination, we supply
# source embedding here (to overwrite the auto embeddings from above):
teCalc.setProperty(teCalcClass.L_PROP_NAME, "1")
teCalc.setProperty(teCalcClass.L_TAU_PROP_NAME, "1")
# Since we're auto-embedding, no need to supply k and k_tau here:
teCalc.initialise()
# Compute TE from breath (column 1) to heart (column 0)
teCalc.setObservations(breathRate, heartRate)
teBreathToHeartDestEmbedding = teCalc.computeAverageLocalOfObservations()
# Check the auto-selected parameters and print out the result:
optimisedK = int(teCalc.getProperty(teCalcClass.K_PROP_NAME))
optimisedKTau = int(teCalc.getProperty(teCalcClass.K_TAU_PROP_NAME))
print(("TE(breath->heart) was %.3f nats for (heart embedding:) k=%d," + \
"k_tau=%d, optimised via Ragwitz criteria, plus (breath embedding:) l=1,l_tau=1") % \
(teBreathToHeartDestEmbedding, optimisedK, optimisedKTau))

25
demos/python/readFloatsFile.py Executable file → Normal file
View File

@ -19,15 +19,16 @@
def readFloatsFile(filename):
"Read a 2D array of floats from a given file"
with open(filename) as f:
# Space separate numbers, one time step per line, each column is a variable
array = []
for line in f: # read all lines
if (line.startswith("%") or line.startswith("#")):
# Assume this is a comment line
continue
if (len(line.split()) == 0):
# Line is empty
continue
array.append([float(x) for x in line.split()])
return array
# Space separate numbers, one time step per line, each column is a variable
array = []
for line in f:
# read all lines
if (line.startswith("%") or line.startswith("#")):
# Assume this is a comment line
continue
if (len(line.split()) == 0):
# Line is empty
continue
array.append([float(x) for x in line.split()])
return array

25
demos/python/readIntsFile.py
View File

@ -19,15 +19,16 @@
def readIntsFile(filename):
"Read a 2D array of int from a given file"
with open(filename) as f:
# Space separate numbers, one time step per line, each column is a variable
array = []
for line in f: # read all lines
if (line.startswith("%") or line.startswith("#")):
# Assume this is a comment line
continue
if (len(line.split()) == 0):
# Line is empty
continue
array.append([int(x) for x in line.split()])
return array
# Space separate numbers, one time step per line, each column is a variable
array = []
for line in f:
# read all lines
if (line.startswith("%") or line.startswith("#")):
# Assume this is a comment line
continue
if (len(line.split()) == 0):
# Line is empty
continue
array.append([int(x) for x in line.split()])
return array

17
java/source/infodynamics/measures/continuous/ActiveInfoStorageCalculator.java Executable file → Normal file
View File

@ -231,6 +231,22 @@ public interface ActiveInfoStorageCalculator {
public void setObservations(double[] observations,
boolean[] valid) throws Exception;
/**
* Add more time-series for the computation of the PDF,
* subject to the validity of each sample in that series.
* The array observations must not be over-written by the user
* until after finaliseAddObservations() has been called.
*
* @param observations time-series array of (univariate) samples,
* where the array index is time.
* @param valid a time series (with indices the same as observations)
* indicating whether the entry in observations at that index is valid;
* we only take vectors as samples to add to the observation set where
* all points in the time series (even between points in
* the embedded k-vector with embedding delays) are valid.
*/
public void addObservations(double[] observations, boolean[] valid) throws Exception;
/**
* Compute the AIS from the previously-supplied samples.
*
@ -377,4 +393,5 @@ public interface ActiveInfoStorageCalculator {
* @throws Exception
*/
public int getNumObservations() throws Exception;
}

10
java/source/infodynamics/measures/continuous/ActiveInfoStorageCalculatorViaMutualInfo.java Executable file → Normal file
View File

@ -268,6 +268,16 @@ public class ActiveInfoStorageCalculatorViaMutualInfo implements
vectorOfObservationTimeSeries.add(observations);
vectorOfValidityOfObservations.add(null); // All observations were valid
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ActiveInfoStorageCalculator#addObservations(double[])
*/
@Override
public void addObservations(double[] observations, boolean[] valid) throws Exception {
// Store these observations in our vector for now
vectorOfObservationTimeSeries.add(observations);
vectorOfValidityOfObservations.add(valid); // All observations were valid
}
/**
* Protected method to internally parse and submit observations through

72
java/source/infodynamics/measures/continuous/ConditionalMutualInfoMultiVariateCommon.java
View File

@ -134,6 +134,39 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
public static final String PROP_NORMALISE = "NORMALISE";
protected boolean normalise = true;
/**
* Cache for the means of each dimension in variable 1, in case we need to normalise
* new observations later
*/
protected double[] var1Means = null;
/**
* Cache for the standard deviations of each dimension in variable 1, in case we need to normalise
* new observations later
*/
protected double[] var1Stds = null;
/**
* Cache for the means of each dimension in variable 2, in case we need to normalise
* new observations later
*/
protected double[] var2Means = null;
/**
* Cache for the standard deviations of each dimension in variable 2, in case we need to normalise
* new observations later
*/
protected double[] var2Stds = null;
/**
* Cache for the means of each dimension in conditional variable, in case we need to normalise
* new observations later
*/
protected double[] condMeans = null;
/**
* Cache for the standard deviations of each dimension in conditional variable, in case we need to normalise
* new observations later
*/
protected double[] condStds = null;
@Override
public void initialise(int var1Dimensions, int var2Dimensions, int condDimensions) {
dimensionsVar1 = var1Dimensions;
@ -146,6 +179,12 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
var2Observations = null;
condObservations = null;
addedMoreThanOneObservationSet = false;
var1Means = null;
var1Stds = null;
var2Means = null;
var2Stds = null;
condMeans = null;
condStds = null;
}
/**
@ -159,7 +198,8 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
* <li>{@link #PROP_NORMALISE} - whether to normalise the individual
* variables to mean 0, standard deviation 1
* (true by default, except for child class
* {@link infodynamics.measures.continuous.gaussian.ConditionalMutualInfoCalculatorMultiVariateGaussian})</li>
* {@link infodynamics.measures.continuous.gaussian.ConditionalMutualInfoCalculatorMultiVariateGaussian}
* for which this property is false and cannot be altered)</li>
* </ul>
*
* <p>Unknown property values are ignored.</p>
@ -208,7 +248,8 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
// startAddObservations was not called first
throw new RuntimeException("User did not call startAddObservations before addObservations");
}
if ((var1.length != var2.length) || (var1.length != cond.length)) {
if ((var1.length != var2.length) ||
((dimensionsCond != 0) && (var1.length != cond.length))) {
throw new Exception(String.format("Observation vector lengths (%d, %d and %d) must match!",
var1.length, var2.length, cond.length));
}
@ -220,7 +261,7 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
throw new Exception("Number of joint variables in var2 data " +
"does not match the initialised value");
}
if (cond[0].length != dimensionsCond) {
if ((dimensionsCond != 0) && (cond[0].length != dimensionsCond)) {
throw new Exception("Number of joint variables in cond data " +
"does not match the initialised value");
}
@ -244,8 +285,11 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
System.arraycopy(var1, startTime, var1ToAdd, 0, numTimeSteps);
double[][] var2ToAdd = new double[numTimeSteps][];
System.arraycopy(var2, startTime, var2ToAdd, 0, numTimeSteps);
double[][] condToAdd = new double[numTimeSteps][];
System.arraycopy(cond, startTime, condToAdd, 0, numTimeSteps);
double[][] condToAdd = null;
if (dimensionsCond != 0) {
condToAdd = new double[numTimeSteps][];
System.arraycopy(cond, startTime, condToAdd, 0, numTimeSteps);
}
addObservations(var1ToAdd, var2ToAdd, condToAdd);
}
@ -317,17 +361,27 @@ public abstract class ConditionalMutualInfoMultiVariateCommon implements
MatrixUtils.arrayCopy(var2, 0, 0,
var2Observations, startObservation, 0,
var2.length, dimensionsVar2);
MatrixUtils.arrayCopy(cond, 0, 0,
if (dimensionsCond != 0) {
MatrixUtils.arrayCopy(cond, 0, 0,
condObservations, startObservation, 0,
cond.length, dimensionsCond);
} // else we can do nothing there
startObservation += var2.length;
}
// Normalise the data if required
if (normalise) {
MatrixUtils.normalise(var1Observations);
MatrixUtils.normalise(var2Observations);
MatrixUtils.normalise(condObservations);
var1Means = MatrixUtils.means(var1Observations);
var1Stds = MatrixUtils.stdDevs(var1Observations, var1Means);
MatrixUtils.normalise(var1Observations, var1Means, var1Stds);
var2Means = MatrixUtils.means(var2Observations);
var2Stds = MatrixUtils.stdDevs(var2Observations, var2Means);
MatrixUtils.normalise(var2Observations, var2Means, var2Stds);
if (dimensionsCond != 0) {
condMeans = MatrixUtils.means(condObservations);
condStds = MatrixUtils.stdDevs(condObservations, condMeans);
MatrixUtils.normalise(condObservations, condMeans, condStds);
}
}
// We don't need to keep the vectors of observation sets anymore:

448
java/source/infodynamics/measures/continuous/TransferEntropyCalculatorMultiVariateViaCondMutualInfo.java
View File

@ -196,222 +196,238 @@ public class TransferEntropyCalculatorMultiVariateViaCondMutualInfo
condMiCalc.initialise(l*sourceDimensions, destDimensions, k*destDimensions);
}
/**
* <p>Sets a single set of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
*/
@Override
public void setObservations(double[] source, double[] destination) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate setObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.setObservations(source, destination);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#setObservations(double[][], double[][])
*/
@Override
public void setObservations(double[][] source, double[][] destination)
throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
}
if (source.length < startTimeForFirstDestEmbedding + 2) {
// There are no observations to add here, the time series is too short
throw new Exception("Not enough observations to set here given k, k_tau, l, l_tau and delay parameters");
}
double[][] currentDestPastVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, k, k_tau,
startTimeForFirstDestEmbedding,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentDestNextVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, 1,
startTimeForFirstDestEmbedding + 1,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentSourcePastVectors =
MatrixUtils.makeDelayEmbeddingVector(source, l, l_tau,
startTimeForFirstDestEmbedding + 1 - delay,
source.length - startTimeForFirstDestEmbedding - 1);
condMiCalc.setObservations(currentSourcePastVectors, currentDestNextVectors, currentDestPastVectors);
}
@Override
public void startAddObservations() {
if ((sourceDimensions == 1) && (destDimensions == 1)) {
super.startAddObservations();
} else {
condMiCalc.startAddObservations();
}
}
@Override
public void finaliseAddObservations() throws Exception {
if ((sourceDimensions == 1) && (destDimensions == 1)) {
super.finaliseAddObservations();
} else {
condMiCalc.finaliseAddObservations();
}
}
/**
* <p>Adds a new set of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
* @see {@link ChannelCalculator#addObservations(double[], double[])}
*/
@Override
public void addObservations(double[] source, double[] destination) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate addObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.addObservations(source, destination);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#addObservations(double[][], double[][])
*/
@Override
public void addObservations(double[][] source, double[][] destination)
throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
}
if (source.length < startTimeForFirstDestEmbedding + 2) {
// There are no observations to add here, the time series is too short
// Don't throw an exception, do nothing since more observations
// can be added later.
return;
}
double[][] currentDestPastVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, k, k_tau,
startTimeForFirstDestEmbedding,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentDestNextVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, 1,
startTimeForFirstDestEmbedding + 1,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentSourcePastVectors =
MatrixUtils.makeDelayEmbeddingVector(source, l, l_tau,
startTimeForFirstDestEmbedding + 1 - delay,
source.length - startTimeForFirstDestEmbedding - 1);
condMiCalc.addObservations(currentSourcePastVectors, currentDestNextVectors, currentDestPastVectors);
}
/**
* <p>Adds a new sub-series of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
* @see {@link ChannelCalculator#addObservations(double[], double[], int, int)}
*/
@Override
public void addObservations(double[] source, double[] destination,
int startTime, int numTimeSteps) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate addObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.addObservations(source, destination, startTime, numTimeSteps);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#addObservations(double[][], double[][], int, int)
*/
@Override
public void addObservations(double[][] source, double[][] destination,
int startTime, int numTimeSteps) throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
}
if (source.length < startTime + numTimeSteps) {
// There are not enough observations given the arguments here
throw new Exception("Not enough observations to set here given startTime and numTimeSteps parameters");
}
addObservations(MatrixUtils.selectRows(source, startTime, numTimeSteps),
MatrixUtils.selectRows(destination, startTime, numTimeSteps));
}
/**
* <p>Adds a new sub-series of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
* @see {@link ChannelCalculator#setObservations(double[], double[], boolean[], boolean[])}
*/
public void setObservations(double[] source, double[] destination,
boolean[] sourceValid, boolean[] destValid) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate setObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.setObservations(source, destination, sourceValid, destValid);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#setObservations(double[][], double[][], boolean[], boolean[])
*/
@Override
public void setObservations(double[][] source, double[][] destination,
boolean[] sourceValid, boolean[] destValid) throws Exception {
Vector<int[]> startAndEndTimePairs = computeStartAndEndTimePairs(sourceValid, destValid);
// We've found the set of start and end times for this pair
startAddObservations();
for (int[] timePair : startAndEndTimePairs) {
int startTime = timePair[0];
int endTime = timePair[1];
addObservations(source, destination, startTime, endTime - startTime + 1);
}
finaliseAddObservations();
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#setObservations(double[][], double[][], boolean[][], boolean[][])
*/
@Override
public void setObservations(double[][] source, double[][] destination,
boolean[][] sourceValid, boolean[][] destValid) throws Exception {
boolean[] jointSourceValid = MatrixUtils.andRows(sourceValid);
boolean[] jointDestValid = MatrixUtils.andRows(destValid);
setObservations(source, destination, jointSourceValid, jointDestValid);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#setObservations(double[][], double[][])
*/
@Override
public void setObservations(double[][] source, double[][] destination)
throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
}
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.setObservations(MatrixUtils.selectColumn(source, 0),
MatrixUtils.selectColumn(destination, 0));
return;
}
if (source.length < startTimeForFirstDestEmbedding + 2) {
// There are no observations to add here, the time series is too short
throw new Exception("Not enough observations to set here given k, k_tau, l, l_tau and delay parameters");
}
double[][] currentDestPastVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, k, k_tau,
startTimeForFirstDestEmbedding,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentDestNextVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, 1,
startTimeForFirstDestEmbedding + 1,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentSourcePastVectors =
MatrixUtils.makeDelayEmbeddingVector(source, l, l_tau,
startTimeForFirstDestEmbedding + 1 - delay,
source.length - startTimeForFirstDestEmbedding - 1);
condMiCalc.setObservations(currentSourcePastVectors, currentDestNextVectors, currentDestPastVectors);
}
@Override
public void startAddObservations() {
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.startAddObservations();
return;
}
// Otherwise initialise ourselves:
condMiCalc.startAddObservations();
}
/**
* <p>Adds a new set of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
* @see {@link ChannelCalculator#addObservations(double[], double[])}
*/
@Override
public void addObservations(double[] source, double[] destination) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate addObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.addObservations(source, destination);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#addObservations(double[][], double[][])
*/
@Override
public void addObservations(double[][] source, double[][] destination)
throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
}
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.addObservations(MatrixUtils.selectColumn(source, 0),
MatrixUtils.selectColumn(destination, 0));
return;
}
if (source.length < startTimeForFirstDestEmbedding + 2) {
// There are no observations to add here, the time series is too short
// Don't throw an exception, do nothing since more observations
// can be added later.
return;
}
double[][] currentDestPastVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, k, k_tau,
startTimeForFirstDestEmbedding,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentDestNextVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, 1,
startTimeForFirstDestEmbedding + 1,
destination.length - startTimeForFirstDestEmbedding - 1);
double[][] currentSourcePastVectors =
MatrixUtils.makeDelayEmbeddingVector(source, l, l_tau,
startTimeForFirstDestEmbedding + 1 - delay,
source.length - startTimeForFirstDestEmbedding - 1);
condMiCalc.addObservations(currentSourcePastVectors, currentDestNextVectors, currentDestPastVectors);
}
/**
* <p>Adds a new sub-series of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
* @see {@link ChannelCalculator#addObservations(double[], double[], int, int)}
*/
@Override
public void addObservations(double[] source, double[] destination,
int startTime, int numTimeSteps) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate addObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.addObservations(source, destination, startTime, numTimeSteps);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#addObservations(double[][], double[][], int, int)
*/
@Override
public void addObservations(double[][] source, double[][] destination,
int startTime, int numTimeSteps) throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
}
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.addObservations(MatrixUtils.selectColumn(source, 0),
MatrixUtils.selectColumn(destination, 0),
startTime, numTimeSteps);
return;
}
if (source.length < startTime + numTimeSteps) {
// There are not enough observations given the arguments here
throw new Exception("Not enough observations to set here given startTime and numTimeSteps parameters");
}
addObservations(MatrixUtils.selectRows(source, startTime, numTimeSteps),
MatrixUtils.selectRows(destination, startTime, numTimeSteps));
}
/**
* <p>Adds a new sub-series of <b>univariate</b> observations to compute the PDFs from.
* Can only be called on this multivariate calculator if the dimensions
* of both source and destination are 1, otherwise throws an exception</p>
*
* {@inheritDoc}
*
* @param source univariate observations for the source variable
* @param destination univariate observations for the destination variable
* @throws Exception if initialised dimensions were not 1
* @see {@link ChannelCalculator#setObservations(double[], double[], boolean[], boolean[])}
*/
public void setObservations(double[] source, double[] destination,
boolean[] sourceValid, boolean[] destValid) throws Exception {
if ((sourceDimensions != 1) || (destDimensions != 1)) {
throw new Exception("Cannot call the univariate setObservations if you " +
"have initialised with dimension > 1 for either source or destination");
}
super.setObservations(source, destination, sourceValid, destValid);
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#setObservations(double[][], double[][], boolean[], boolean[])
*/
@Override
public void setObservations(double[][] source, double[][] destination,
boolean[] sourceValid, boolean[] destValid) throws Exception {
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.setObservations(MatrixUtils.selectColumn(source, 0),
MatrixUtils.selectColumn(destination, 0),
sourceValid, destValid);
return;
}
Vector<int[]> startAndEndTimePairs = computeStartAndEndTimePairs(sourceValid, destValid);
// We've found the set of start and end times for this pair
startAddObservations();
for (int[] timePair : startAndEndTimePairs) {
int startTime = timePair[0];
int endTime = timePair[1];
addObservations(source, destination, startTime, endTime - startTime + 1);
}
finaliseAddObservations();
}
/* (non-Javadoc)
* @see infodynamics.measures.continuous.ChannelCalculatorMultiVariate#setObservations(double[][], double[][], boolean[][], boolean[][])
*/
@Override
public void setObservations(double[][] source, double[][] destination,
boolean[][] sourceValid, boolean[][] destValid) throws Exception {
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.setObservations(MatrixUtils.selectColumn(source, 0),
MatrixUtils.selectColumn(destination, 0),
MatrixUtils.selectColumn(sourceValid, 0),
MatrixUtils.selectColumn(destValid, 0));
return;
}
boolean[] jointSourceValid = MatrixUtils.andRows(sourceValid);
boolean[] jointDestValid = MatrixUtils.andRows(destValid);
setObservations(source, destination, jointSourceValid, jointDestValid);
}
@Override
public void finaliseAddObservations() throws Exception {
if ((sourceDimensions == 1) && (destDimensions == 1)) {
// We'll be using the superclass for the computation
super.finaliseAddObservations();
return;
}
// Otherwise finalise ourselves:
condMiCalc.finaliseAddObservations();
}
/**
* <p>Computes the local values of the transfer entropy
* for each valid observation in the supplied univariate observations

42
java/source/infodynamics/measures/continuous/TransferEntropyCalculatorViaCondMutualInfo.java
View File

@ -125,6 +125,13 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
*/
protected Vector<boolean[]> vectorOfValidityOfDestination;
/**
* Array of the number of observations added by each separate call to
* {@link #addObservations(double[], double[])}, in order of which those calls
* were made. This is returned by {@link #getSeparateNumObservations()}
*/
protected int[] separateNumObservations;
/**
* Construct a transfer entropy calculator using an instance of
* condMiCalculatorClassName as the underlying conditional mutual information calculator.
@ -228,6 +235,7 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
vectorOfDestinationTimeSeries = null;
vectorOfValidityOfSource = null;
vectorOfValidityOfDestination = null;
separateNumObservations = new int[] {};
}
/**
@ -342,9 +350,10 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
*
* @param source time series of source observations
* @param destination time series of destination observations
* @return the number of observations added
* @throws Exception
*/
protected void addObservationsAfterParamsDetermined(double[] source, double[] destination) throws Exception {
protected int addObservationsAfterParamsDetermined(double[] source, double[] destination) throws Exception {
if (source.length != destination.length) {
throw new Exception(String.format("Source and destination lengths (%d and %d) must match!",
source.length, destination.length));
@ -353,7 +362,7 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
// There are no observations to add here, the time series is too short
// Don't throw an exception, do nothing since more observations
// can be added later.
return;
return 0;
}
double[][] currentDestPastVectors =
MatrixUtils.makeDelayEmbeddingVector(destination, k, k_tau,
@ -368,6 +377,7 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
startTimeForFirstDestEmbedding + 1 - delay,
source.length - startTimeForFirstDestEmbedding - 1);
condMiCalc.addObservations(currentSourcePastVectors, currentDestNextVectors, currentDestPastVectors);
return destination.length - startTimeForFirstDestEmbedding - 1;
}
/**
@ -384,21 +394,24 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
* the source at that index is valid.
* @param destValid array (with indices the same as destination) indicating whether
* the destination at that index is valid.
* @return total number of observations added
* @throws Exception
*/
protected void addObservationsAfterParamsDetermined(double[] source, double[] destination,
protected int addObservationsAfterParamsDetermined(double[] source, double[] destination,
boolean[] sourceValid, boolean[] destValid) throws Exception {
// Compute the start and end time pairs using our embedding parameters:
Vector<int[]> startAndEndTimePairs = computeStartAndEndTimePairs(sourceValid, destValid);
int totalObservationsAdded = 0;
for (int[] timePair : startAndEndTimePairs) {
int startTime = timePair[0];
int endTime = timePair[1];
addObservationsAfterParamsDetermined(
totalObservationsAdded += addObservationsAfterParamsDetermined(
MatrixUtils.select(source, startTime, endTime - startTime + 1),
MatrixUtils.select(destination, startTime, endTime - startTime + 1));
}
return totalObservationsAdded;
}
@Override
@ -441,17 +454,21 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
Iterator<double[]> destIterator = vectorOfDestinationTimeSeries.iterator();
Iterator<boolean[]> sourceValidityIterator = vectorOfValidityOfSource.iterator();
Iterator<boolean[]> destValidityIterator = vectorOfValidityOfDestination.iterator();
separateNumObservations = new int[vectorOfDestinationTimeSeries.size()];
int setNum = 0;
for (double[] source : vectorOfSourceTimeSeries) {
double[] destination = destIterator.next();
boolean[] sourceValidity = sourceValidityIterator.next();
boolean[] destValidity = destValidityIterator.next();
int observationsAddedThisTime = 0;
if (sourceValidity == null) {
// Add the whole time-series
addObservationsAfterParamsDetermined(source, destination);
observationsAddedThisTime = addObservationsAfterParamsDetermined(source, destination);
} else {
addObservationsAfterParamsDetermined(source, destination,
observationsAddedThisTime = addObservationsAfterParamsDetermined(source, destination,
sourceValidity, destValidity);
}
separateNumObservations[setNum++] = observationsAddedThisTime;
}
vectorOfSourceTimeSeries = null; // No longer required
vectorOfDestinationTimeSeries = null; // No longer required
@ -642,6 +659,19 @@ public class TransferEntropyCalculatorViaCondMutualInfo implements
return condMiCalc.getNumObservations();
}
/**
* Retrieve an array of the number of observations that
* were added by each call to {@link #addObservations(double[], double[])} etc.,
* in order of them being called.
* The actual number of observations for each call is computed <b>after</b>
* any auto-embedding is performed.
*
* @return
*/
public int[] getSeparateNumObservations() {
return separateNumObservations;
}
@Override
public boolean getAddedMoreThanOneObservationSet() {
return condMiCalc.getAddedMoreThanOneObservationSet();

32
java/source/infodynamics/measures/continuous/gaussian/ActiveInfoStorageCalculatorGaussian.java
View File

@ -21,6 +21,8 @@ package infodynamics.measures.continuous.gaussian;
import infodynamics.measures.continuous.ActiveInfoStorageCalculator;
import infodynamics.measures.continuous.ActiveInfoStorageCalculatorViaMutualInfo;
import infodynamics.measures.continuous.MutualInfoCalculatorMultiVariate;
import infodynamics.utils.AnalyticNullDistributionComputer;
import infodynamics.utils.ChiSquareMeasurementDistribution;
/**
* An Active Information Storage (AIS) calculator (implementing {@link ActiveInfoStorageCalculator})
@ -62,7 +64,8 @@ import infodynamics.measures.continuous.MutualInfoCalculatorMultiVariate;
* @see MutualInfoCalculatorMultiVariateGaussian
*/
public class ActiveInfoStorageCalculatorGaussian
extends ActiveInfoStorageCalculatorViaMutualInfo {
extends ActiveInfoStorageCalculatorViaMutualInfo
implements AnalyticNullDistributionComputer {
public static final String MI_CALCULATOR_GAUSSIAN = MutualInfoCalculatorMultiVariateGaussian.class.getName();
@ -76,4 +79,31 @@ public class ActiveInfoStorageCalculatorGaussian
public ActiveInfoStorageCalculatorGaussian() throws InstantiationException, IllegalAccessException, ClassNotFoundException {
super(MI_CALCULATOR_GAUSSIAN);
}
/**
* Generate an <b>analytic</b> distribution of what the AIS would look like,
* under a null hypothesis that our variables had no relation.
* This is performed without bootstrapping (which is done in
* {@link #computeSignificance(int, int)} and {@link #computeSignificance(int, int[][])}).
* The method is implemented using the corresponding method of the
* underlying {@link MutualInfoCalculatorMultiVariateGaussian}
*
* <p>See Section II.E "Statistical significance testing" of
* the JIDT paper below, and the other papers referenced in
* {@link AnalyticNullDistributionComputer#computeSignificance()}
* (in particular Brillinger and Geweke),
* for a description of how this is done for MI.
* Basically, the null distribution is a chi-square distribution.
* </p>
*
* @return ChiSquareMeasurementDistribution object which describes
* the proportion of AIS scores from the null distribution
* which have higher or equal AISs to our actual value.
* @see "J.T. Lizier, 'JIDT: An information-theoretic
* toolkit for studying the dynamics of complex systems', 2014."
* @throws Exception
*/
public ChiSquareMeasurementDistribution computeSignificance() {
return ((MutualInfoCalculatorMultiVariateGaussian) miCalc).computeSignificance();
}
}

16
java/source/infodynamics/measures/continuous/gaussian/ConditionalMutualInfoCalculatorMultiVariateGaussian.java
View File

@ -158,6 +158,22 @@ public class ConditionalMutualInfoCalculatorMultiVariateGaussian
var2IndicesInCovariance = null;
}
@Override
public void setProperty(String propertyName, String propertyValue) {
if (propertyName.equalsIgnoreCase(PROP_NORMALISE)) {
// We do not allow the user to alter this property away from false,
// otherwise we would need to alter the functionality of the
// local calculations, etc.
return;
}
super.setProperty(propertyName, propertyValue);
}
@Override
protected Object clone() throws CloneNotSupportedException {
return super.clone();
}
/**
* @throws Exception if the observation variables are not linearly independent
* (leading to a non-positive definite covariance matrix).

18
java/source/infodynamics/measures/continuous/kernel/KernelEstimatorMultiVariate.java
View File

@ -52,6 +52,7 @@ public class KernelEstimatorMultiVariate implements Cloneable {
protected double[] suppliedKernelWidths = null;
protected double[] kernelWidthsInUse = null;
protected double kernelVolumeInUse = 0;
protected int dimensions = 1;
private int[] bins = null;
private boolean usingIntegerIndexBins = true;
@ -196,6 +197,7 @@ public class KernelEstimatorMultiVariate implements Cloneable {
// many bins:
usingIntegerIndexBins = !forceCompareToAll;
useBins = true;
kernelVolumeInUse = 0;
// Work out the max and min for each bin
for (int d = 0; d < dimensions; d++) {
mins[d] = MatrixUtils.min(data, d);
@ -210,6 +212,11 @@ public class KernelEstimatorMultiVariate implements Cloneable {
} else {
kernelWidthsInUse[d] = suppliedKernelWidths[d];
}
if (d == 0) {
kernelVolumeInUse = 2.0 * kernelWidthsInUse[d];
} else {
kernelVolumeInUse *= 2.0 * kernelWidthsInUse[d];
}
bins[d] = (int) Math.ceil((max - mins[d]) / kernelWidthsInUse[d]);
if (bins[d] == 0) {
// This means the min and max are exactly the same:
@ -711,7 +718,7 @@ public class KernelEstimatorMultiVariate implements Cloneable {
}
/**
* Give kernel estimated probability for this observation.
* Give kernel estimated probability density for this observation.
* Do this using bins, so we only need to compare to neighbouring bins
* and of course add in the observations in this bin.
* Achieves this by trawling through neighbouring bins in a recursive fashion.
@ -725,8 +732,8 @@ public class KernelEstimatorMultiVariate implements Cloneable {
boolean dynCorrExclusion) {
KernelCount kernelCount = getCountFromBins(observation, timeStep, dynCorrExclusion);
return (double) kernelCount.count / (double) kernelCount.totalObservationsForCount;
// Dividing by kernelVolume also to get a density
return (double) kernelCount.count / (double) kernelCount.totalObservationsForCount / kernelVolumeInUse;
}
/**
@ -792,7 +799,7 @@ public class KernelEstimatorMultiVariate implements Cloneable {
}
/**
* Give kernel estimated probability for this observation, by comparing to all
* Give kernel estimated probability density for this observation, by comparing to all
* given observations.
*
* @param observation
@ -804,7 +811,8 @@ public class KernelEstimatorMultiVariate implements Cloneable {
boolean dynCorrExclusion) {
KernelCount kernelCount = getCountComparingToAll(observation, timeStep, dynCorrExclusion);
return (double) kernelCount.count / (double) kernelCount.totalObservationsForCount;
// Dividing by kernelVolume also to get a density
return (double) kernelCount.count / (double) kernelCount.totalObservationsForCount / kernelVolumeInUse;
}
private int getBinIndex(double value, int dimension) {

10
java/source/infodynamics/measures/continuous/kernel/KernelEstimatorUniVariate.java
View File

@ -284,11 +284,17 @@ public class KernelEstimatorUniVariate {
System.out.println("Count after upper bin " + (bin + 1) + " = " + count);
}
// TODO Should we divide this result by the kernel width (in use)?
// Should we divide this result by the kernel width (in use)?
// Schreiber doesn't do this in the TE paper, but it is done in
// the Kaiser and Schreiber paper. It won't matter to TE, but
// will make a difference to individual entropy estimates.
return (double) count / (double) totalTimePointsCompared;
// Answer: YES we should. this makes it a proper density (for differential entropies)
// so this has been included below.
// (If you want to retain a (discrete) Shannon entropy, based on counts within a defined width then no)
// ACTUALLY you divide by twice the kernel width I think (since this is the total range)
// At least, I've validated that this gives the correct diff entropy for
// the uniform distribution
return (double) count / (double) totalTimePointsCompared / (double) (2.0 * kernelWidthInUse);
}
private int getBinIndex(double value) {

81
java/source/infodynamics/measures/continuous/kraskov/ActiveInfoStorageCalculatorKraskov.java
View File

@ -46,8 +46,10 @@ import infodynamics.utils.MatrixUtils;
* (except {@link MutualInfoCalculatorMultiVariate#PROP_TIME_DIFF} as outlined
* in {@link ActiveInfoStorageCalculatorViaMutualInfo#setProperty(String, String)}).
* Embedding parameters may be automatically determined as per the Ragwitz criteria
* by setting the property {@link #PROP_AUTO_EMBED_METHOD} to {@link #AUTO_EMBED_METHOD_RAGWITZ}
* (plus additional parameter settings for this).</li>
* by setting the property {@link #PROP_AUTO_EMBED_METHOD} to {@link #AUTO_EMBED_METHOD_RAGWITZ},
* or as per the max. bias-corrected AIS criteria by
* setting the property {@link #PROP_AUTO_EMBED_METHOD} to {@link #AUTO_EMBED_METHOD_MAX_CORR_AIS}
* (plus additional parameter settings for these).</li>
* <li>Computed values are in <b>nats</b>, not bits!</li>
* </ul>
* </p>
@ -60,6 +62,9 @@ import infodynamics.utils.MatrixUtils;
* Information Sciences, vol. 208, pp. 39-54, 2012.</li>
* <li>Ragwitz and Kantz, "Markov models from data by simple nonlinear time series
* predictors in delay embedding spaces", Physical Review E, vol 65, 056201 (2002).</li>
* <li>J. Garland, R. G. James, E. Bradley, <a href="http://dx.doi.org/10.1103/physreve.93.022221">
* "Leveraging information storage to select forecast-optimal parameters for delay-coordinate reconstructions"</a>,
* Physical Review E, Vol. 93 (2016), 022221, doi:</li>
* </ul>
*
* @author Joseph Lizier (<a href="joseph.lizier at gmail.com">email</a>,
@ -82,7 +87,9 @@ public class ActiveInfoStorageCalculatorKraskov
public static final String MI_CALCULATOR_KRASKOV2 = MutualInfoCalculatorMultiVariateKraskov2.class.getName();
/**
* Property name for the auto-embedding method. Defaults to {@link #AUTO_EMBED_METHOD_NONE}
* Property name for the auto-embedding method. Defaults to {@link #AUTO_EMBED_METHOD_NONE}.
* Other valid values are {@link #AUTO_EMBED_METHOD_RAGWITZ} or
* {@link #AUTO_EMBED_METHOD_MAX_CORR_AIS}
*/
public static final String PROP_AUTO_EMBED_METHOD = "AUTO_EMBED_METHOD";
/**
@ -95,6 +102,12 @@ public class ActiveInfoStorageCalculatorKraskov
* the Ragwitz optimisation technique should be used for automatic embedding
*/
public static final String AUTO_EMBED_METHOD_RAGWITZ = "RAGWITZ";
/**
* Valid value for the property {@link #PROP_AUTO_EMBED_METHOD} indicating that
* the automatic embedding should be done by maximising the bias corrected
* AIS (as per Garland et al. in the references above).
*/
public static final String AUTO_EMBED_METHOD_MAX_CORR_AIS = "MAX_CORR_AIS";
/**
* Internal variable tracking what type of auto embedding (if any)
* we are using
@ -201,8 +214,10 @@ public class ActiveInfoStorageCalculatorKraskov
* automatically determines the embedding history length ({@link #K_PROP_NAME})
* and embedding delay ({@link #TAU_PROP_NAME}). Default is {@link #AUTO_EMBED_METHOD_NONE} meaning
* values are set manually; other accepted values include: {@link #AUTO_EMBED_METHOD_RAGWITZ} for use
* of the Ragwitz criteria (searching up to {@link #PROP_K_SEARCH_MAX} and
* {@link #PROP_TAU_SEARCH_MAX}). Use of any value other than {@link #AUTO_EMBED_METHOD_NONE}
* of the Ragwitz criteria and {@link #AUTO_EMBED_METHOD_MAX_CORR_AIS} for using
* the maz bias-corrected AIS criteria (both searching up to {@link #PROP_K_SEARCH_MAX} and
* {@link #PROP_TAU_SEARCH_MAX}, as outlined by Garland et al. in the references list above).
* Use of any value other than {@link #AUTO_EMBED_METHOD_NONE}
* will lead to any previous settings for k and tau (via e.g. {@link #initialise(int, int)} or
* auto-embedding during previous calculations) will be overwritten after observations
* are supplied.</li>
@ -297,11 +312,13 @@ public class ActiveInfoStorageCalculatorKraskov
}
// Else we need to auto embed
double bestPredictionError = Double.POSITIVE_INFINITY;
// TODO Should make sure the rest of the code could handle k=0
// as default if nothing can improve on this
int k_candidate_best = 1;
int tau_candidate_best = 1;
if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ)) {
double bestPredictionError = Double.POSITIVE_INFINITY;
if (debug) {
System.out.printf("Beginning Ragwitz auto-embedding with k_max=%d, tau_max=%d\n",
k_search_max, tau_search_max);
@ -358,13 +375,61 @@ public class ActiveInfoStorageCalculatorKraskov
}
}
}
} else if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_MAX_CORR_AIS)) {
double bestAIS = Double.NEGATIVE_INFINITY;
if (debug) {
System.out.printf("Beginning max bias corrected AIS auto-embedding with k_max=%d, tau_max=%d\n",
k_search_max, tau_search_max);
}
for (int k_candidate = 1; k_candidate <= k_search_max; k_candidate++) {
for (int tau_candidate = 1; tau_candidate <= tau_search_max; tau_candidate++) {
// Use our internal MI calculator in case it has any particular
// properties we need to have been set already
miCalc.initialise(k_candidate, 1);
miCalc.startAddObservations();
// Send all of the observations through:
for (double[] observations : vectorOfObservationTimeSeries) {
double[][] currentDestPastVectors =
MatrixUtils.makeDelayEmbeddingVector(observations, k_candidate,
tau_candidate, (k_candidate-1)*tau_candidate,
observations.length - (k_candidate-1)*tau_candidate - 1);
double[][] currentDestNextVectors =
MatrixUtils.makeDelayEmbeddingVector(observations, 1,
(k_candidate-1)*tau_candidate + 1,
observations.length - (k_candidate-1)*tau_candidate - 1);
miCalc.addObservations(currentDestPastVectors, currentDestNextVectors);
}
miCalc.finaliseAddObservations();
// Now grab the AIS estimate here -- it's already bias corrected for
double thisAIS = miCalc.computeAverageLocalOfObservations();
if (debug) {
System.out.printf("AIS for k=%d,tau=%d is %.3f\n",
k_candidate, tau_candidate, thisAIS);
}
if (thisAIS > bestAIS) {
// This parameter setting is the best so far:
bestAIS = thisAIS;
k_candidate_best = k_candidate;
tau_candidate_best = tau_candidate;
}
if (k_candidate == 1) {
// tau is irrelevant, so no point testing other values
break;
}
}
}
} else {
throw new RuntimeException("Unexpected value " + autoEmbeddingMethod +
" for property " + PROP_AUTO_EMBED_METHOD);
}
// Make sure the embedding length and delay are set here
k = k_candidate_best;
tau = tau_candidate_best;
if (debug) {
System.out.printf("Embedding parameters set to k=%d,tau=%d (for prediction error %.3f)\n",
k, tau, bestPredictionError);
System.out.printf("Embedding parameters set to k=%d,tau=%d\n",
k, tau);
}
}

141
java/source/infodynamics/measures/continuous/kraskov/ConditionalMutualInfoCalculatorMultiVariateKraskov.java
View File

@ -46,7 +46,8 @@ import infodynamics.utils.UnivariateNearestNeighbourSearcher;
* actually implement the two KSG algorithms.</p>
*
* <p>Crucially, the calculation is performed by examining
* neighbours in the full joint space (as specified by Frenzel and Pompe)
* neighbours in the full joint space (as specified by Frenzel and Pompe,
* and later by Vejmelka and Paluš, and Vlachos and Kugiumtzis)
* rather than two MI calculators.</p>
*
* <p>Usage is as per the paradigm outlined for {@link ConditionalMutualInfoCalculatorMultiVariate},
@ -69,6 +70,12 @@ import infodynamics.utils.UnivariateNearestNeighbourSearcher;
* <li>Frenzel and Pompe, <a href="http://dx.doi.org/10.1103/physrevlett.99.204101">
* "Partial Mutual Information for Coupling Analysis of Multivariate Time Series"</a>,
* Physical Review Letters, <b>99</b>, p. 204101+ (2007).</li>
* <li>Vejmelka and Paluš, <a href="http://dx.doi.org/10.1103/physreve.77.026214">
* "Inferring the directionality of coupling with conditional mutual information"</a>,
* Physical Review E, <b>77</b>, 026214, (2008)</li>
* <li>I. Vlachos and D. Kugiumtzis, <a href="http://dx.doi.org/10.1103/physreve.82.016207">
* "Nonuniform state-space reconstruction and coupling detection"</a>,
* Physical Review E, <b>82</b>, 016207, (2010)</li>
* <li>Kraskov, A., Stoegbauer, H., Grassberger, P.,
* <a href="http://dx.doi.org/10.1103/PhysRevE.69.066138">"Estimating mutual information"</a>,
* Physical Review E 69, (2004) 066138.</li>
@ -331,6 +338,7 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
var2Observations[r][c] +=
random.nextGaussian()*noiseLevel;
}
// This next loop will only execute if dimensionsCond > 0
for (int c = 0; c < dimensionsCond; c++) {
condObservations[r][c] +=
random.nextGaussian()*noiseLevel;
@ -351,7 +359,7 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
public double computeAverageLocalOfObservations() throws Exception {
// Compute the conditional MI
double startTime = Calendar.getInstance().getTimeInMillis();
lastAverage = computeFromObservations(false)[0];
lastAverage = computeFromObservations(false, null)[0];
condMiComputed = true;
if (debug) {
Calendar rightNow2 = Calendar.getInstance();
@ -395,7 +403,7 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
var2Observations = MatrixUtils.extractSelectedTimePointsReusingArrays(originalData, reordering);
}
// Compute the conditional MI
double newCondMI = computeFromObservations(false)[0];
double newCondMI = computeFromObservations(false, null)[0];
// restore original data
kdTreeJoint = originalKdTreeJoint;
if (variableToReorder == 1) {
@ -412,7 +420,7 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
@Override
public double[] computeLocalOfPreviousObservations() throws Exception {
double[] localValues = computeFromObservations(true);
double[] localValues = computeFromObservations(true, null);
lastAverage = MatrixUtils.mean(localValues);
condMiComputed = true;
return localValues;
@ -424,10 +432,25 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
@Override
public double[] computeLocalUsingPreviousObservations(double[][] states1,
double[][] states2, double[][] condStates) throws Exception {
// If implemented, will need to incorporate any normalisation here
// (normalising the incoming data the same way the previously
// supplied observations were normalised).
throw new Exception("Local method not implemented yet");
// Do normalisation of the incoming data if required:
double[][] states1ToUse, states2ToUse, condStatesToUse;
if (normalise) {
states1ToUse = MatrixUtils.normaliseIntoNewArray(states1, var1Means, var1Stds);
states2ToUse = MatrixUtils.normaliseIntoNewArray(states2, var2Means, var2Stds);
if (dimensionsCond != 0) {
condStatesToUse = MatrixUtils.normaliseIntoNewArray(condStates, condMeans, condStds);
} else {
condStatesToUse = null;
}
} else {
states1ToUse = states1;
states2ToUse = states2;
condStatesToUse = condStates;
}
// And call the algorithm:
double[] localValues = computeFromObservations(true,
new double[][][]{states1ToUse, states2ToUse, condStatesToUse});
return localValues;
}
/**
@ -444,10 +467,12 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
*
* @param returnLocals whether to return an array or local values, or else
* sums of these values
* @param newObservations set to null for computing for the observation set for the PDF, or pass in a new set
* of observations to compute the average/locals for (using the existing observations to construct the PDF)
* @return either the average conditional MI, or array of local conditional MI value, in nats not bits
* @throws Exception
*/
protected double[] computeFromObservations(boolean returnLocals) throws Exception {
protected double[] computeFromObservations(boolean returnLocals, double[][][] newObservations) throws Exception {
int N = var1Observations.length; // number of observations
double[] returnValues = null;
@ -488,28 +513,36 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
uniNNSearcherVar2 = new UnivariateNearestNeighbourSearcher(var2Observations);
}
}
if (nnSearcherConditional == null) {
if ((nnSearcherConditional == null) && (dimensionsCond > 0)) {
nnSearcherConditional = NearestNeighbourSearcher.create(condObservations);
nnSearcherConditional.setNormType(normType);
}
// How many time points are we averaging over?
int numTimePointsToComputeFor = (newObservations == null) ?
N : newObservations[0].length;
if (numThreads == 1) {
// Single-threaded implementation:
returnValues = partialComputeFromObservations(0, N, returnLocals);
if (newObservations == null) {
returnValues = partialComputeFromObservations(0, numTimePointsToComputeFor, returnLocals);
} else {
returnValues = partialComputeFromNewObservations(0, numTimePointsToComputeFor,
newObservations[0], newObservations[1], newObservations[2], returnLocals);
}
} else {
// We're going multithreaded:
if (returnLocals) {
// We're computing local conditional MI
returnValues = new double[N];
returnValues = new double[numTimePointsToComputeFor];
} else {
// We're computing average conditional MI
returnValues = new double[CondMiKraskovThreadRunner.RETURN_ARRAY_LENGTH];
}
// Distribute the observations to the threads for the parallel processing
int lTimesteps = N / numThreads; // each thread gets the same amount of data
int res = N % numThreads; // the first thread gets the residual data
int lTimesteps = numTimePointsToComputeFor / numThreads; // each thread gets the same amount of data
int res = numTimePointsToComputeFor % numThreads; // the first thread gets the residual data
if (debug) {
System.out.printf("Computing Kraskov conditional MI with %d threads (%d timesteps each, plus %d residual)\n",
numThreads, lTimesteps, res);
@ -523,7 +556,7 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
System.out.println(t + ".Thread: from " + startTime +
" to " + (startTime + numTimesteps)); // Trace Message
}
runners[t] = new CondMiKraskovThreadRunner(this, startTime, numTimesteps, returnLocals);
runners[t] = new CondMiKraskovThreadRunner(this, startTime, numTimesteps, newObservations, returnLocals);
tCalculators[t] = new Thread(runners[t]);
tCalculators[t].start();
}
@ -552,10 +585,10 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
return returnValues;
} else {
// Average out the components for the final equation(s) and for debugging:
double averageDiGammas = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_DIGAMMAS] / (double) N;
double avNxz = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_NXZ] / (double) N;
double avNyz = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_NYZ] / (double) N;
double avNz = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_NZ] / (double) N;
double averageDiGammas = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_DIGAMMAS] / (double) numTimePointsToComputeFor;
double avNxz = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_NXZ] / (double) numTimePointsToComputeFor;
double avNyz = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_NYZ] / (double) numTimePointsToComputeFor;
double avNz = returnValues[CondMiKraskovThreadRunner.INDEX_SUM_NZ] / (double) numTimePointsToComputeFor;
if (debug) {
System.out.printf("<n_xz>=%.3f, <n_yz>=%.3f, <n_z>=%.3f\n",
avNxz, avNyz, avNz);
@ -573,17 +606,27 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
// Algorithm 2:
// We also retrieve the sums of inverses for debugging purposes:
double averageInverseCountInJointXZ =
returnValues[CondMiKraskovThreadRunner.INDEX_SUM_INV_NXZ] / (double) N;
returnValues[CondMiKraskovThreadRunner.INDEX_SUM_INV_NXZ] / (double) numTimePointsToComputeFor;
double averageInverseCountInJointYZ =
returnValues[CondMiKraskovThreadRunner.INDEX_SUM_INV_NYZ] / (double) N;
double averageMeasure = MathsUtils.digamma(k) - (2.0 / (double)k) + averageDiGammas +
returnValues[CondMiKraskovThreadRunner.INDEX_SUM_INV_NYZ] / (double) numTimePointsToComputeFor;
double inverseKTerm;
if (dimensionsCond > 0) {
// We will add in the 2/K term as usual
inverseKTerm = 2.0 / (double) k;
} else {
// We will only add in 1/K term since without a conditional there is
// technically one less variable in the full joint space
inverseKTerm = 1.0 / (double) k;
}
double averageMeasure = MathsUtils.digamma(k) - inverseKTerm + averageDiGammas +
averageInverseCountInJointXZ + averageInverseCountInJointYZ;
if (debug) {
System.out.printf("Av = digamma(k)=%.3f + <digammas>=%.3f +<inverses>=%.3f - 2/k=%.3f = %.3f" +
System.out.printf("Av = digamma(k)=%.3f + <digammas>=%.3f +<inverses>=%.3f - $d/k=%.3f = %.3f" +
" (<1/n_yz>=%.3f, <1/n_xz>=%.3f)\n",
MathsUtils.digamma(k), averageDiGammas,
averageInverseCountInJointXZ + averageInverseCountInJointYZ,
(2.0 / (double)k), averageMeasure,
dimensionsCond > 0 ? 2 : 1,
inverseKTerm, averageMeasure,
averageInverseCountInJointYZ, averageInverseCountInJointXZ);
}
double[] result = new double[1];
@ -619,6 +662,40 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
protected abstract double[] partialComputeFromObservations(
int startTimePoint, int numTimePoints, boolean returnLocals) throws Exception;
/**
* Protected method to be used internally for threaded implementations.
* This method implements the guts of each Kraskov algorithm, computing the number of
* nearest neighbours in each dimension for a sub-set of the data points.
* It is intended to be called by one thread to work on that specific
* sub-set of the data.
* In particular, this method differs from {@link #partialComputeFromObservations(int, int, boolean)}
* because it operates on a new set of observations (using the old set of observations for
* constructing the search spaces and PDFs)
*
* <p>The method returns:<ol>
* <li>for average conditional MIs (returnLocals == false), the relevant sums of digamma(n_{xz}), digamma(n_{yz})
* and digamma(n_z)
* for a partial set of the observations</li>
* <li>for local conditional MIs (returnLocals == true), the array of local conditional MI values</li>
* </ol>
*
* @param startTimePoint start time for the partial set we examine
* @param numTimePoints number of time points (including startTimePoint to examine)
* @param newVar1Observations new time series of observations for variable 1
* @param newVar2Observations new time series of observations for variable 1
* @param newCondObservations new time series of observations for variable 1
* @param returnLocals whether to return an array or local values, or else
* sums of these values
* @return an array of the relevant sum of digamma(n_xz+1) and digamma(n_yz+1) and digamma(n_z), then
* sum of n_xz, n_yz, n_z and for algorithm 2 only, sum of 1/n_xz and 1/n_yz
* (these latter five are only for debugging purposes).
* @throws Exception
*/
protected abstract double[] partialComputeFromNewObservations(
int startTimePoint, int numTimePoints,
double[][] newVar1Observations, double[][] newVar2Observations, double[][] newCondObservations,
boolean returnLocals) throws Exception;
/**
* Private class to handle multi-threading of the Kraskov algorithms.
* Each instance calls partialComputeFromObservations()
@ -633,6 +710,7 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
protected ConditionalMutualInfoCalculatorMultiVariateKraskov condMiCalc;
protected int myStartTimePoint;
protected int numberOfTimePoints;
protected double[][][] newObservations;
protected boolean computeLocals;
protected double[] returnValues = null;
@ -649,11 +727,13 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
public CondMiKraskovThreadRunner(
ConditionalMutualInfoCalculatorMultiVariateKraskov condMiCalc,
int myStartTimePoint, int numberOfTimePoints,
double[][][] newObservations,
boolean computeLocals) {
this.condMiCalc = condMiCalc;
this.myStartTimePoint = myStartTimePoint;
this.numberOfTimePoints = numberOfTimePoints;
this.computeLocals = computeLocals;
this.newObservations = newObservations;
}
/**
@ -676,7 +756,16 @@ public abstract class ConditionalMutualInfoCalculatorMultiVariateKraskov
*/
public void run() {
try {
returnValues = condMiCalc.partialComputeFromObservations(myStartTimePoint, numberOfTimePoints, computeLocals);
if (newObservations == null) {
// Computing on existing observations
returnValues = condMiCalc.partialComputeFromObservations(myStartTimePoint, numberOfTimePoints, computeLocals);
} else {
// Computing on new observations
returnValues = condMiCalc.partialComputeFromNewObservations(
myStartTimePoint, numberOfTimePoints,
newObservations[0], newObservations[1],
newObservations[2], computeLocals);
}
} catch (Exception e) {
// Store the exception for later retrieval
problem = e;

305
java/source/infodynamics/measures/continuous/kraskov/ConditionalMutualInfoCalculatorMultiVariateKraskov1.java
View File

@ -18,12 +18,18 @@
package infodynamics.measures.continuous.kraskov;
import java.util.Arrays;
import java.util.Calendar;
import java.util.PriorityQueue;
import infodynamics.measures.continuous.ConditionalMutualInfoCalculatorMultiVariate;
import infodynamics.utils.EmpiricalMeasurementDistribution;
import infodynamics.utils.FirstIndexComparatorDouble;
import infodynamics.utils.KdTree;
import infodynamics.utils.MathsUtils;
import infodynamics.utils.MatrixUtils;
import infodynamics.utils.NeighbourNodeData;
import infodynamics.utils.UnivariateNearestNeighbourSearcher;
/**
* <p>Computes the differential conditional mutual information of two multivariate
@ -99,7 +105,20 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
// First element in the PQ is the kth NN,
// and epsilon = kthNnData.distance
NeighbourNodeData kthNnData = nnPQ.poll();
/*
if (debug) {
System.out.print("t = " + t + " : for data point: ");
MatrixUtils.printArray(System.out, var1Observations[t]);
MatrixUtils.printArray(System.out, var2Observations[t]);
if (dimensionsCond > 0) {
MatrixUtils.printArray(System.out, condObservations[t]);
} else {
System.out.print("[]");
}
System.out.printf("t=%d: K=%d NNs found at range %.5f (point %d)\n", t, k, kthNnData.distance, kthNnData.sampleIndex);
}
*/
// Now count the points in the conditional space, and
// the var1-conditional and var2-conditional spaces.
// We have 3 coded options for how to do this:
@ -112,7 +131,7 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
int n_yz = kdTreeVar2Conditional.countPointsStrictlyWithinR(
t, kthNnData.distance, dynCorrExclTime);
int n_z = nnSearcherConditional.countPointsStrictlyWithinR(
t, kthNnData.distance, dynCorrExclTime);
t, kthNnData.distance, dynCorrExclTime);
*/ // end option A
/* Option B --
@ -151,8 +170,10 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
if (debug) {
methodStartTime = Calendar.getInstance().getTimeInMillis();
}
nnSearcherConditional.findPointsWithinR(t, kthNnData.distance, dynCorrExclTime,
if (dimensionsCond > 0) {
nnSearcherConditional.findPointsWithinR(t, kthNnData.distance, dynCorrExclTime,
false, isWithinRForConditionals, indicesWithinRForConditionals);
}
if (debug) {
conditionalTime += Calendar.getInstance().getTimeInMillis() -
methodStartTime;
@ -160,15 +181,26 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
}
// 2. Then compute n_xz and n_yz harnessing our knowledge of
// which points qualified for the conditional already:
// Don't need to supply dynCorrExclTime in the following, because only
// Don't need to supply dynCorrExclTime in most of the following, because only
// points outside of it have been included in isWithinRForConditionals
int n_xz;
if (dimensionsVar1 > 1) {
n_xz = kdTreeVar1Conditional.countPointsWithinR(t, kthNnData.distance,
false, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_xz = uniNNSearcherVar1.countPointsWithinR(t, kthNnData.distance,
false, isWithinRForConditionals);
if (dimensionsCond == 0) {
// We're really only counting whether the x space qualifies
if (dimensionsVar1 > 1) {
n_xz = kdTreeVar1Conditional.countPointsStrictlyWithinR(
t, kthNnData.distance, dynCorrExclTime);
} else {
n_xz = uniNNSearcherVar1.countPointsWithinR(t, kthNnData.distance,
dynCorrExclTime, false);
}
} else {
if (dimensionsVar1 > 1) {
n_xz = kdTreeVar1Conditional.countPointsWithinR(t, kthNnData.distance,
false, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_xz = uniNNSearcherVar1.countPointsWithinR(t, kthNnData.distance,
false, isWithinRForConditionals);
}
}
if (debug) {
conditionalXTime += Calendar.getInstance().getTimeInMillis() -
@ -176,12 +208,23 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
methodStartTime = Calendar.getInstance().getTimeInMillis();
}
int n_yz;
if (dimensionsVar2 > 1) {
n_yz = kdTreeVar2Conditional.countPointsWithinR(t, kthNnData.distance,
false, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_yz = uniNNSearcherVar2.countPointsWithinR(t, kthNnData.distance,
false, isWithinRForConditionals);
if (dimensionsCond == 0) {
// We're really only counting whether the y space qualifies
if (dimensionsVar2 > 1) {
n_yz = kdTreeVar2Conditional.countPointsStrictlyWithinR(
t, kthNnData.distance, dynCorrExclTime);
} else {
n_yz = uniNNSearcherVar2.countPointsWithinR(t, kthNnData.distance,
dynCorrExclTime, false);
}
} else {
if (dimensionsVar2 > 1) {
n_yz = kdTreeVar2Conditional.countPointsWithinR(t, kthNnData.distance,
false, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_yz = uniNNSearcherVar2.countPointsWithinR(t, kthNnData.distance,
false, isWithinRForConditionals);
}
}
if (debug) {
conditionalYTime += Calendar.getInstance().getTimeInMillis() -
@ -189,8 +232,15 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
}
// 3. Finally, reset our boolean array for its next use while we count n_z:
int n_z;
for (n_z = 0; indicesWithinRForConditionals[n_z] != -1; n_z++) {
isWithinRForConditionals[indicesWithinRForConditionals[n_z]] = false;
if (dimensionsCond == 0) {
n_z = totalObservations - 1; // - 1 to remove the point itself.
// Note: This doesn't respect the dynamic correlation exclusion, but
// does align with a mutual information calculation here (to give the digamma(N) term).
// No need to reset the boolean array
} else {
for (n_z = 0; indicesWithinRForConditionals[n_z] != -1; n_z++) {
isWithinRForConditionals[indicesWithinRForConditionals[n_z]] = false;
}
}
// end option C
@ -206,11 +256,16 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
if (returnLocals) {
localCondMi[t-startTimePoint] = digammaK - digammaNxzPlusOne - digammaNyzPlusOne + digammaNzPlusOne;
if (debug) {
System.out.printf("t=%d, n_xz=%d, n_yz=%d, n_z=%d, local=%.4f," +
System.out.printf("t=%d, r=%.5f, n_xz=%d, n_yz=%d, n_z=%d, local=%.4f," +
" digamma(n_xz+1)=%.5f, digamma(n_yz+1)=%.5f, digamma(n_z+1)=%.5f, \n",
t, n_xz, n_yz, n_z, localCondMi[t-startTimePoint],
t, kthNnData.distance, n_xz, n_yz, n_z, localCondMi[t-startTimePoint],
digammaNxzPlusOne, digammaNyzPlusOne, digammaNzPlusOne);
}
} else if (debug) {
System.out.printf("t=%d, n_xz=%d, n_yz=%d, n_z=%d," +
" digamma(n_xz+1)=%.5f, digamma(n_yz+1)=%.5f, digamma(n_z+1)=%.5f, \n",
t, n_xz, n_yz, n_z,
digammaNxzPlusOne, digammaNyzPlusOne, digammaNzPlusOne);
}
}
@ -244,4 +299,214 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov1
return results;
}
}
/**
* Compute the local conditional mutual information values for a new set of
* observations, where the PDFs are constructed from the observations added earlier.
*
* @param startTimePoint
* @param numTimePoints
* @param newStates1
* @param newStates2
* @param newCondStates
* @return
* @throws Exception
*/
@Override
protected double[] partialComputeFromNewObservations(int startTimePoint,
int numTimePoints, double[][] newStates1,
double[][] newStates2, double[][] newCondStates, boolean returnLocals) throws Exception {
double startTime = Calendar.getInstance().getTimeInMillis();
double[] localCondMi = null;
if (returnLocals) {
localCondMi = new double[numTimePoints];
}
// Count the average number of points within eps_xz and eps_yz and eps_z of each point
double sumDiGammas = 0;
double sumNxz = 0;
double sumNyz = 0;
double sumNz = 0;
long knnTime = 0, conditionalTime = 0,
conditionalXTime = 0, conditionalYTime = 0;
// Arrays used for fast searching on conditionals with a marginal:
boolean[] isWithinRForConditionals = new boolean[totalObservations];
int[] indicesWithinRForConditionals = new int[totalObservations+1];
for (int t = startTimePoint; t < startTimePoint + numTimePoints; t++) {
// Compute eps for this time step by
// finding the kth closest neighbour for point t:
long methodStartTime = Calendar.getInstance().getTimeInMillis();
PriorityQueue<NeighbourNodeData> nnPQ =
kdTreeJoint.findKNearestNeighbours(k,
new double[][] {newStates1[t], newStates2[t], newCondStates[t]});
knnTime += Calendar.getInstance().getTimeInMillis() -
methodStartTime;
// First element in the PQ is the kth NN,
// and epsilon = kthNnData.distance
NeighbourNodeData kthNnData = nnPQ.poll();
if (debug) {
System.out.print("t = " + t + " : for data point: ");
MatrixUtils.printArray(System.out, newStates1[t]);
MatrixUtils.printArray(System.out, newStates2[t]);
if (dimensionsCond > 0) {
MatrixUtils.printArray(System.out, newCondStates[t]);
} else {
System.out.print("[]");
}
System.out.printf("t=%d : K=%d NNs found at range %.5f (point %d)\n", t, k, kthNnData.distance, kthNnData.sampleIndex);
}
// Now count the points in the conditional space, and
// the var1-conditional and var2-conditional spaces.
// We use Option C determined above to do this --
// Identify the points satisfying the conditional criteria, then use
// the knowledge of which points made this cut to speed up the searching
// in the conditional-marginal spaces:
// 1. Identify the n_z points within the conditional boundaries:
if (debug) {
methodStartTime = Calendar.getInstance().getTimeInMillis();
}
if (dimensionsCond > 0) {
nnSearcherConditional.findPointsWithinR(kthNnData.distance,
new double[][] {newCondStates[t]},
false, isWithinRForConditionals, indicesWithinRForConditionals);
}
if (debug) {
conditionalTime += Calendar.getInstance().getTimeInMillis() -
methodStartTime;
methodStartTime = Calendar.getInstance().getTimeInMillis();
}
// 2. Then compute n_xz and n_yz harnessing our knowledge of
// which points qualified for the conditional already:
// Don't need to supply dynCorrExclTime in the following, because
// it's not relevant for the new points
int n_xz;
if (dimensionsCond == 0) {
// We're really only counting whether the x space qualifies
if (dimensionsVar1 > 1) {
n_xz = kdTreeVar1Conditional.countPointsWithinR(
new double[][] {newStates1[t], newCondStates[t]},
kthNnData.distance, false);
} else {
n_xz = uniNNSearcherVar1.countPointsWithinR(
new double[][] {newStates1[t]}, kthNnData.distance, false);
}
} else {
if (dimensionsVar1 > 1) {
n_xz = kdTreeVar1Conditional.countPointsWithinR(kthNnData.distance,
new double[][] {newStates1[t], newCondStates[t]},
false, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_xz = uniNNSearcherVar1.countPointsWithinR(
new double[][] {newStates1[t]}, kthNnData.distance,
false, isWithinRForConditionals);
}
}
if (debug) {
conditionalXTime += Calendar.getInstance().getTimeInMillis() -
methodStartTime;
methodStartTime = Calendar.getInstance().getTimeInMillis();
}
int n_yz;
if (dimensionsCond == 0) {
// We're really only counting whether the y space qualifies
if (dimensionsVar2 > 1) {
n_yz = kdTreeVar2Conditional.countPointsWithinR(
new double[][] {newStates2[t], newCondStates[t]},
kthNnData.distance, false);
} else {
n_yz = uniNNSearcherVar2.countPointsWithinR(
new double[][] {newStates2[t]},
kthNnData.distance, false);
}
} else {
if (dimensionsVar2 > 1) {
n_yz = kdTreeVar2Conditional.countPointsWithinR(kthNnData.distance,
new double[][] {newStates2[t], newCondStates[t]},
false, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_yz = uniNNSearcherVar2.countPointsWithinR(
new double[][] {newStates2[t]}, kthNnData.distance,
false, isWithinRForConditionals);
}
}
if (debug) {
conditionalYTime += Calendar.getInstance().getTimeInMillis() -
methodStartTime;
}
// 3. Finally, reset our boolean array for its next use while we count n_z:
int n_z;
if (dimensionsCond == 0) {
n_z = totalObservations - 1; // - 1 to remove the point itself.
// no need to reset the boolean array
} else {
for (n_z = 0; indicesWithinRForConditionals[n_z] != -1; n_z++) {
isWithinRForConditionals[indicesWithinRForConditionals[n_z]] = false;
}
}
// end option C
sumNxz += n_xz;
sumNyz += n_yz;
sumNz += n_z;
// And take the digammas:
double digammaNxzPlusOne = MathsUtils.digamma(n_xz+1);
double digammaNyzPlusOne = MathsUtils.digamma(n_yz+1);
double digammaNzPlusOne = MathsUtils.digamma(n_z+1);
sumDiGammas += digammaNzPlusOne - digammaNxzPlusOne - digammaNyzPlusOne;
if (returnLocals) {
localCondMi[t-startTimePoint] = digammaK - digammaNxzPlusOne - digammaNyzPlusOne + digammaNzPlusOne;
if (debug) {
System.out.printf("t=%d, n_xz=%d, n_yz=%d, n_z=%d, local=%.4f," +
" digamma(n_xz+1)=%.5f, digamma(n_yz+1)=%.5f, digamma(n_z+1)=%.5f, \n",
t, n_xz, n_yz, n_z, localCondMi[t-startTimePoint],
digammaNxzPlusOne, digammaNyzPlusOne, digammaNzPlusOne);
}
} else if (debug) {
double localValue = digammaK - digammaNxzPlusOne - digammaNyzPlusOne + digammaNzPlusOne;
System.out.printf("t=%d, n_xz=%d, n_yz=%d, n_z=%d, local=%.4f," +
" digamma(n_xz+1)=%.5f, digamma(n_yz+1)=%.5f, digamma(n_z+1)=%.5f, \n",
t, n_xz, n_yz, n_z, localValue,
digammaNxzPlusOne, digammaNyzPlusOne, digammaNzPlusOne);
}
}
if (debug) {
Calendar rightNow2 = Calendar.getInstance();
long endTime = rightNow2.getTimeInMillis();
System.out.println("Subset " + startTimePoint + ":" +
(startTimePoint + numTimePoints) + " Calculation time: " +
((endTime - startTime)/1000.0) + " sec" );
System.out.println("Total exec times for: ");
System.out.println("\tknn search: " + (knnTime/1000.0));
System.out.println("\tz search: " + (conditionalTime/1000.0));
System.out.println("\tzx search: " + (conditionalXTime/1000.0));
System.out.println("\tzy search: " + (conditionalYTime/1000.0));
System.out.printf("%d:%d -- Returning: %.4f, %.4f, %.4f, %.4f\n",
startTimePoint, (startTimePoint + numTimePoints),
sumDiGammas, sumNxz, sumNyz, sumNz);
}
// Select what to return:
if (returnLocals) {
return localCondMi;
} else {
// Pad return array with two values, to allow compatibility in
// return length with algorithm 2
double[] results = new double[6];
results[0] = sumDiGammas;
results[1] = sumNxz;
results[2] = sumNyz;
results[3] = sumNz;
return results;
}
}
}

105
java/source/infodynamics/measures/continuous/kraskov/ConditionalMutualInfoCalculatorMultiVariateKraskov2.java
View File

@ -79,7 +79,15 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov2
}
// Constants:
double twoOnK = 2.0 / (double) k;
double inverseKTerm;
if (dimensionsCond > 0) {
// We will add in the 2/K term as usual
inverseKTerm = 2.0 / (double) k;
} else {
// We will only add in 1/K term since without a conditional there is
// technically one less variable in the full joint space
inverseKTerm = 1.0 / (double) k;
}
// Count the average number of points within eps_xz, eps_yz and eps_z
double sumDiGammas = 0;
@ -136,36 +144,68 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov2
// the knowledge of which points made this cut to speed up the searching
// in the conditional-marginal spaces:
// 1. Identify the n_z points within the conditional boundaries:
nnSearcherConditional.findPointsWithinR(t, eps_z, dynCorrExclTime,
if (dimensionsCond > 0) {
nnSearcherConditional.findPointsWithinR(t, eps_z, dynCorrExclTime,
true, isWithinRForConditionals, indicesWithinRForConditionals);
}
// 2. Then compute n_xz and n_yz harnessing our knowledge of
// which points qualified for the conditional already:
// Don't need to supply dynCorrExclTime in the following, because only
// points outside of it have been included in isWithinRForConditionals
int n_xz;
if (dimensionsVar1 > 1) {
// Check only the x variable against eps_x, use existing results for z
n_xz = kdTreeVar1Conditional.countPointsWithinRs(t,
new double[] {eps_x, eps_z},
true, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_xz = uniNNSearcherVar1.countPointsWithinR(t, eps_x,
true, isWithinRForConditionals);
if (dimensionsCond == 0) {
// We're really only counting whether the x space qualifies
if (dimensionsVar1 > 1) {
n_xz = kdTreeVar1Conditional.countPointsWithinOrOnR(
t, eps_x, dynCorrExclTime); // No need to pass eps_z in
} else { // Generally faster to search only the marginal space if it is univariate
n_xz = uniNNSearcherVar1.countPointsWithinOrOnR(t, eps_x,
dynCorrExclTime);
}
} else {
if (dimensionsVar1 > 1) {
// Check only the x variable against eps_x, use existing results for z
n_xz = kdTreeVar1Conditional.countPointsWithinRs(t,
new double[] {eps_x, eps_z},
true, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_xz = uniNNSearcherVar1.countPointsWithinR(t, eps_x,
true, isWithinRForConditionals);
}
}
int n_yz;
if (dimensionsVar2 > 1) {
// Check only the y variable against eps_y, use existing results for z
n_yz = kdTreeVar2Conditional.countPointsWithinRs(t,
new double[] {eps_y, eps_z},
true, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_yz = uniNNSearcherVar2.countPointsWithinR(t, eps_y,
true, isWithinRForConditionals);
if (dimensionsCond == 0) {
if (dimensionsVar2 > 1) {
// Check only the y variable against eps_y, use existing results for z
n_yz = kdTreeVar2Conditional.countPointsWithinOrOnR(
t, eps_y, dynCorrExclTime); // No need to pass eps_z in
} else { // Generally faster to search only the marginal space if it is univariate
n_yz = uniNNSearcherVar2.countPointsWithinOrOnR(t, eps_y,
dynCorrExclTime);
}
} else {
// We're really only counting whether the y space qualifies
if (dimensionsVar2 > 1) {
// Check only the y variable against eps_y, use existing results for z
n_yz = kdTreeVar2Conditional.countPointsWithinRs(t,
new double[] {eps_y, eps_z},
true, 1, isWithinRForConditionals);
} else { // Generally faster to search only the marginal space if it is univariate
n_yz = uniNNSearcherVar2.countPointsWithinR(t, eps_y,
true, isWithinRForConditionals);
}
}
// 3. Finally, reset our boolean array for its next use while we count n_z:
int n_z;
for (n_z = 0; indicesWithinRForConditionals[n_z] != -1; n_z++) {
isWithinRForConditionals[indicesWithinRForConditionals[n_z]] = false;
if (dimensionsCond == 0) {
n_z = totalObservations; // Dropping - 1 to remove the point itself, and
// Note: This doesn't respect the dynamic correlation exclusion, but this
// does align with a mutual information calculation here (to give the digamma(N) term).
// No need to reset the boolean array
} else {
for (n_z = 0; indicesWithinRForConditionals[n_z] != -1; n_z++) {
isWithinRForConditionals[indicesWithinRForConditionals[n_z]] = false;
}
}
// end option C
@ -177,15 +217,25 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov2
double digammaNxz = MathsUtils.digamma(n_xz);
double digammaNyz = MathsUtils.digamma(n_yz);
double digammaNz = MathsUtils.digamma(n_z);
double invN_xz = 1.0/(double) n_xz;
double invN_yz = 1.0/(double) n_yz;
double invN_xz, invN_yz;
if (dimensionsCond > 0) {
// We will add in the inverse neighbour counts as usual
invN_xz = 1.0/(double) n_xz;
invN_yz = 1.0/(double) n_yz;
} else {
// We do not add in the inverse neighbour counts, because they
// are technically coming from a single variable neighbour search,
// and so there is a 0/n_xz and 0/n_yz contribution
invN_xz = 0.0;
invN_yz = 0.0;
}
sumInverseCountInJointXZ += invN_xz;
sumInverseCountInJointYZ += invN_yz;
double contributionDigammas = digammaNz - digammaNxz - digammaNyz;
sumDiGammas += contributionDigammas;
if (returnLocals) {
localCondMi[t-startTimePoint] = digammaK - twoOnK + contributionDigammas + invN_xz + invN_yz;
localCondMi[t-startTimePoint] = digammaK - inverseKTerm + contributionDigammas + invN_xz + invN_yz;
if (debug) {
// Only tracking this for debugging purposes:
System.out.printf("t=%d, n_xz=%d, n_yz=%d, n_z=%d, 1/n_yz=%.3f, 1/n_xz=%.3f, local=%.4f\n",
@ -216,4 +266,13 @@ public class ConditionalMutualInfoCalculatorMultiVariateKraskov2
return returnValues;
}
}
@Override
protected double[] partialComputeFromNewObservations(int startTimePoint,
int numTimePoints, double[][] newVar1Observations,
double[][] newVar2Observations, double[][] newCondObservations,
boolean returnLocals) throws Exception {
// TODO implement me
throw new RuntimeException("Not implemented yet");
}
}

145
java/source/infodynamics/measures/continuous/kraskov/TransferEntropyCalculatorKraskov.java
View File

@ -53,8 +53,11 @@ import infodynamics.measures.continuous.TransferEntropyCalculatorViaCondMutualIn
* as well as for {@link #PROP_KRASKOV_ALG_NUM}.
* Embedding parameters may be automatically determined as per the Ragwitz criteria
* by setting the property {@link #PROP_AUTO_EMBED_METHOD} to {@link #AUTO_EMBED_METHOD_RAGWITZ}
* or {@link #AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY}
* (plus additional parameter settings for this).</li>
* or {@link #AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY},
* or as per the max. bias-corrected AIS criteria by
* setting the property {@link #PROP_AUTO_EMBED_METHOD} to {@link #AUTO_EMBED_METHOD_MAX_CORR_AIS}
* (as per Garland et al. in the reference list)
* plus additional parameter settings for these.</li>
* </li>
* <li>Computed values are in <b>nats</b>, not bits!</li>
* </ul>
@ -81,6 +84,9 @@ import infodynamics.measures.continuous.TransferEntropyCalculatorViaCondMutualIn
* Physical Review E 77, 026110, 2008.</li>
* <li>Ragwitz and Kantz, "Markov models from data by simple nonlinear time series
* predictors in delay embedding spaces", Physical Review E, vol 65, 056201 (2002).</li>
* <li>J. Garland, R. G. James, E. Bradley, <a href="http://dx.doi.org/10.1103/physreve.93.022221">
* "Leveraging information storage to select forecast-optimal parameters for delay-coordinate reconstructions"</a>,
* Physical Review E, Vol. 93 (2016), 022221, doi:</li>
* </ul>
*
* @author Joseph Lizier (<a href="joseph.lizier at gmail.com">email</a>,
@ -118,7 +124,8 @@ public class TransferEntropyCalculatorKraskov
/**
* Property name for specifying which (if any) auto-embedding method to use.
* Valid values include {@link #AUTO_EMBED_METHOD_RAGWITZ}, {@link #AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY}
* Valid values include {@link #AUTO_EMBED_METHOD_RAGWITZ}, {@link #AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY},
* {@link #AUTO_EMBED_METHOD_MAX_CORR_AIS}, {@link #AUTO_EMBED_METHOD_MAX_CORR_AIS_DEST_ONLY}
* and {@link #AUTO_EMBED_METHOD_NONE}.
* Defaults to {@link #AUTO_EMBED_METHOD_NONE}
*/
@ -140,6 +147,18 @@ public class TransferEntropyCalculatorKraskov
* for the destination time-series only
*/
public static final String AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY = "RAGWITZ_DEST_ONLY";
/**
* Valid value for the property {@link #PROP_AUTO_EMBED_METHOD} indicating that
* the automatic embedding should be done by maximising the bias corrected
* AIS, for both source and destination time series
*/
public static final String AUTO_EMBED_METHOD_MAX_CORR_AIS = "MAX_CORR_AIS";
/**
* Valid value for the property {@link #PROP_AUTO_EMBED_METHOD} indicating that
* the automatic embedding should be done by maximising the bias corrected
* AIS, for destination time series only
*/
public static final String AUTO_EMBED_METHOD_MAX_CORR_AIS_DEST_ONLY = "MAX_CORR_AIS_DEST_ONLY";
/**
* Internal variable tracking what type of auto embedding (if any)
* we are using
@ -268,7 +287,11 @@ public class TransferEntropyCalculatorKraskov
* values are set manually; other accepted values include: {@link #AUTO_EMBED_METHOD_RAGWITZ} for use
* of the Ragwitz criteria for both source and destination (searching up to {@link #PROP_K_SEARCH_MAX} and
* {@link #PROP_TAU_SEARCH_MAX}), and {@link #AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY} for use
* of the Ragwitz criteria for the destination only.
* of the Ragwitz criteria for the destination only;
* {@link AUTO_EMBED_METHOD_MAX_CORR_AIS} for use of the max bias corrected AIS criteria
* for both source and destination (searching up to {@link #PROP_K_SEARCH_MAX} and
* {@link #PROP_TAU_SEARCH_MAX}) and {@link AUTO_EMBED_METHOD_MAX_CORR_AIS_DEST_ONLY} for use of
* this criteria for the destination only.
* Use of any value other than {@link #AUTO_EMBED_METHOD_NONE}
* will lead to previous settings for embedding lengths and delays (via e.g. {@link #initialise(int, int)} or
* auto-embedding during previous calculations) for the destination and perhaps source to
@ -366,66 +389,78 @@ public class TransferEntropyCalculatorKraskov
// Else we need to auto embed
// If we need to check which embedding method later:
// if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ) ||
// autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY)) {
if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ) ||
autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY) ||
autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_MAX_CORR_AIS) ||
autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_MAX_CORR_AIS_DEST_ONLY)) {
// Use a Kraskov AIS calculator to embed both time-series individually:
ActiveInfoStorageCalculatorKraskov aisCalc = new ActiveInfoStorageCalculatorKraskov();
// Set the properties for the underlying MI Kraskov calculator here to match ours:
for (String key : props.keySet()) {
aisCalc.setProperty(key, props.get(key));
}
// Set the auto-embedding properties as we require:
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_AUTO_EMBED_METHOD,
ActiveInfoStorageCalculatorKraskov.AUTO_EMBED_METHOD_RAGWITZ);
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_K_SEARCH_MAX,
Integer.toString(k_search_max));
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_TAU_SEARCH_MAX,
Integer.toString(tau_search_max));
// In case !ragwitz_num_nns_set and our condMiCalc has a different default number of
// kNNs for Kraskov search than miCalc, we had best supply the number directly here:
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_RAGWITZ_NUM_NNS,
getProperty(PROP_RAGWITZ_NUM_NNS));
// Embed the destination with the Ragwitz criteria:
if (debug) {
System.out.println("Starting embedding of destination:");
}
aisCalc.initialise();
aisCalc.startAddObservations();
for (double[] destination : vectorOfDestinationTimeSeries) {
aisCalc.addObservations(destination);
}
aisCalc.finaliseAddObservations();
// Set the auto-embedding parameters for the destination:
k = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.K_PROP_NAME));
k_tau = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.TAU_PROP_NAME));
if (debug) {
System.out.printf("Embedding parameters for destination set to k=%d,k_tau=%d\n",
k, k_tau);
}
if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ)) {
// Embed the source also with the Ragwitz criteria:
// Use a Kraskov AIS calculator to embed both time-series individually:
ActiveInfoStorageCalculatorKraskov aisCalc = new ActiveInfoStorageCalculatorKraskov();
// Set the properties for the underlying MI Kraskov calculator here to match ours:
for (String key : props.keySet()) {
aisCalc.setProperty(key, props.get(key));
}
// Set the auto-embedding properties as we require:
if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ) ||
autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY)) {
// We're doing Ragwitz auto-embedding
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_AUTO_EMBED_METHOD,
ActiveInfoStorageCalculatorKraskov.AUTO_EMBED_METHOD_RAGWITZ);
// In case !ragwitz_num_nns_set and our condMiCalc has a different default number of
// kNNs for Kraskov search than miCalc, we had best supply the number directly here:
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_RAGWITZ_NUM_NNS,
getProperty(PROP_RAGWITZ_NUM_NNS));
} else {
// We're doing max bias-corrected AIS embeding:
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_AUTO_EMBED_METHOD,
ActiveInfoStorageCalculatorKraskov.AUTO_EMBED_METHOD_MAX_CORR_AIS);
}
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_K_SEARCH_MAX,
Integer.toString(k_search_max));
aisCalc.setProperty(ActiveInfoStorageCalculatorKraskov.PROP_TAU_SEARCH_MAX,
Integer.toString(tau_search_max));
// Embed the destination:
if (debug) {
System.out.println("Starting embedding of source:");
System.out.println("Starting embedding of destination:");
}
aisCalc.initialise();
aisCalc.startAddObservations();
for (double[] source : vectorOfSourceTimeSeries) {
aisCalc.addObservations(source);
for (double[] destination : vectorOfDestinationTimeSeries) {
aisCalc.addObservations(destination);
}
aisCalc.finaliseAddObservations();
// Set the auto-embedding parameters for the source:
l = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.K_PROP_NAME));
l_tau = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.TAU_PROP_NAME));
// Set the auto-embedding parameters for the destination:
k = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.K_PROP_NAME));
k_tau = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.TAU_PROP_NAME));
if (debug) {
System.out.printf("Embedding parameters for source set to l=%d,l_tau=%d\n",
l, l_tau);
System.out.printf("Embedding parameters for destination set to k=%d,k_tau=%d\n",
k, k_tau);
}
if (autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_RAGWITZ) ||
autoEmbeddingMethod.equalsIgnoreCase(AUTO_EMBED_METHOD_MAX_CORR_AIS)) {
// Embed the source also:
if (debug) {
System.out.println("Starting embedding of source:");
}
aisCalc.initialise();
aisCalc.startAddObservations();
for (double[] source : vectorOfSourceTimeSeries) {
aisCalc.addObservations(source);
}
aisCalc.finaliseAddObservations();
// Set the auto-embedding parameters for the source:
l = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.K_PROP_NAME));
l_tau = Integer.parseInt(aisCalc.getProperty(ActiveInfoStorageCalculator.TAU_PROP_NAME));
if (debug) {
System.out.printf("Embedding parameters for source set to l=%d,l_tau=%d\n",
l, l_tau);
}
}
// Now that embedding parameters are finalised:
setStartTimeForFirstDestEmbedding();
}
// Now that embedding parameters are finalised:
setStartTimeForFirstDestEmbedding();
}
}

132
java/source/infodynamics/measures/discrete/ConditionalTransferEntropyCalculatorDiscrete.java Executable file → Normal file
View File

@ -498,7 +498,7 @@ public class ConditionalTransferEntropyCalculatorDiscrete extends InfoMeasureCal
* multivariate time series.
* to our estimates of the pdfs.
* This call should be made as opposed to addObservations(int states[][])
* for computing active info for heterogeneous agents.
* for computing conditional TE for heterogeneous agents.
*
* @param states multivariate time series, indexed first by time
* then by variable number.
@ -843,7 +843,7 @@ public class ConditionalTransferEntropyCalculatorDiscrete extends InfoMeasureCal
}
/**
* Computes local active information storage for the given
* Computes local conditional transfer entropy for the given
* states, using pdfs built up from observations previously
* sent in via the addObservations method.
* This method is suitable for heterogeneous agents
@ -939,6 +939,134 @@ public class ConditionalTransferEntropyCalculatorDiscrete extends InfoMeasureCal
return localTE;
}
/**
* Computes local conditional transfer entropy for the given
* samples, using pdfs built up from observations previously
* sent in via the addObservations method.
*
* <p>This method takes in a univariate conditionals time series - it is assumed
* that either numOtherInfoContributors == 1 or the user has combined the
* multivariate tuples into a single value for each observation, e.g. by calling
* {@link MatrixUtils#computeCombinedValues(int[][], int)}. This cannot be
* checked here however, so use at your own risk!
* </p>
*
* @param source source time series samples
* @param dest dest time series samples
* @param conditional time series for conditionals
* (indexed only by time step)
* @return time-series of local conditional TE values
*/
private double[] computeLocalFromPreviousObservations
(int source[], int dest[], int[] conditional){
int rows = source.length;
// Allocate for all rows even though we'll leave the first ones as zeros
double[] localTE = new double[rows];
average = 0;
max = 0;
min = 0;
// Initialise and store the current previous value for each column
int pastVal = 0;
pastVal = 0;
for (int p = 0; p < k; p++) {
pastVal *= base;
pastVal += dest[p];
}
int destVal, sourceVal, conditionalVal;
double logTerm;
for (int r = startObservationTime; r < rows; r++) {
destVal = dest[r];
sourceVal = source[r-1];
conditionalVal = conditional[r-1];
// Now compute the local value
logTerm = ((double) sourceDestPastOthersCount[sourceVal][destVal][pastVal][conditionalVal] / (double) sourcePastOthersCount[sourceVal][pastVal][conditionalVal]) /
((double) destPastOthersCount[destVal][pastVal][conditionalVal] / (double) pastOthersCount[pastVal][conditionalVal]);
localTE[r] = Math.log(logTerm) / log_2;
average += localTE[r];
if (localTE[r] > max) {
max = localTE[r];
} else if (localTE[r] < min) {
min = localTE[r];
}
// Update the previous value:
if (k > 0) {
pastVal -= maxShiftedValue[dest[r-k]];
pastVal *= base;
pastVal += dest[r];
}
}
average = average/(double) (rows - startObservationTime);
return localTE;
}
/**
* Computes local conditional transfer entropy for the given
* samples, using pdfs built up from observations previously
* sent in via the addObservations method.
*
* @param source source time series samples
* @param dest dest time series samples
* @param conditionals multivariate time series for conditionals
* (indexed first by time step then by variable number)
* @return time-series of local conditional TE values
*/
private double[] computeLocalFromPreviousObservations
(int source[], int dest[], int[][] conditionals){
int rows = source.length;
// Allocate for all rows even though we'll leave the first ones as zeros
double[] localTE = new double[rows];
average = 0;
max = 0;
min = 0;
// Initialise and store the current previous value for each column
int pastVal = 0;
pastVal = 0;
for (int p = 0; p < k; p++) {
pastVal *= base;
pastVal += dest[p];
}
int destVal, sourceVal, conditionalsVal;
double logTerm;
for (int r = startObservationTime; r < rows; r++) {
destVal = dest[r];
sourceVal = source[r-1];
conditionalsVal = 0;
for (int o = 0; o < conditionals[r-1].length; o++) {
// Include this other contributor
conditionalsVal *= base;
conditionalsVal += conditionals[r-1][o];
}
// Now compute the local value
logTerm = ((double) sourceDestPastOthersCount[sourceVal][destVal][pastVal][conditionalsVal] / (double) sourcePastOthersCount[sourceVal][pastVal][conditionalsVal]) /
((double) destPastOthersCount[destVal][pastVal][conditionalsVal] / (double) pastOthersCount[pastVal][conditionalsVal]);
localTE[r] = Math.log(logTerm) / log_2;
average += localTE[r];
if (localTE[r] > max) {
max = localTE[r];
} else if (localTE[r] < min) {
min = localTE[r];
}
// Update the previous value:
if (k > 0) {
pastVal -= maxShiftedValue[dest[r-k]];
pastVal *= base;
pastVal += dest[r];
}
}
average = average/(double) (rows - startObservationTime);
return localTE;
}
/**
* Standalone routine to
* compute local conditional transfer entropy across a 2D spatiotemporal

25
java/source/infodynamics/measures/discrete/MultiInformationCalculatorDiscrete.java
View File

@ -91,8 +91,29 @@ public class MultiInformationCalculatorDiscrete extends InfoMeasureCalculatorDis
MatrixUtils.fill(marginalCounts, 0);
}
// TODO Define just a simple addObservations for
// int[][] states where there are just numVars variables
/**
* Given multiple time samples of a homogeneous array of variables (states),
* add the observations of all sets of numVars of these
* Do this for every time point
*
* @param states 2D array of values of an array of variables
* at many observations (first index is time, second is variable index)
*/
public void addObservations(int[][] states) {
for (int t = 0; t < states.length; t++) {
// Add the marginal observations in, and compute the joint state value
// for this sample at time t
int jointValue = 0;
for (int i = 0; i < numVars; i++) {
int thisValue = states[t][i];
marginalCounts[i][thisValue]++;
jointValue *= base;
jointValue += thisValue;
}
jointCount[jointValue]++;
observations++;
}
}
/**
* Given one time sample of a homogeneous array of variables (states),

1322
java/source/infodynamics/measures/discrete/TransferEntropyCalculatorDiscrete.java
View File

File diff suppressed because it is too large Load Diff

31
java/source/infodynamics/measures/mixed/kraskov/MutualInfoCalculatorMultiVariateWithDiscreteKraskov.java
View File

@ -41,6 +41,20 @@ import infodynamics.utils.RandomGenerator;
* {@link MutualInfoCalculatorMultiVariate}
* </p>
*
* <p>This calculator effectively implements the nearest neighbour method
* laid out <a href="http://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0087357">here</a>
* (see Ross reference below),
* but we have subtract out the sample point from the count in the discrete space,
* following the original KSG method.
* </p>
*
* <p><b>References:</b><br/>
* <ul>
* <li> B.C. Ross, "Mutual Information between Discrete and Continuous Data Sets",
* PLoS ONE 9(2): e87357. doi:
* <a href="http://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0087357">10.1371/journal.pone.0087357</a></li>
* </ul>
*
* @author Joseph Lizier (<a href="joseph.lizier at gmail.com">email</a>,
* <a href="http://lizier.me/joseph/">www</a>)
*/
@ -571,8 +585,11 @@ public class MutualInfoCalculatorMultiVariateWithDiscreteKraskov implements Mutu
continuousNewStates, means, stds);
}
int N = continuousNewStates.length; // number of observations
double[] locals = new double[N];
// Thanks to Rosalind Wang for picking up the bug in not using two different N's here
int N_newObservations = continuousNewStates.length; // number of new observations we're evaluating
int N_samplesForPdfs = continuousData.length; // number of observations we're using to compute the PDFs
double[] locals = new double[N_newObservations];
// Don't need the 1/k correction here because the conditional entropy term
// is taken over the continuous space only. The correction is (m-1)/k
@ -580,16 +597,16 @@ public class MutualInfoCalculatorMultiVariateWithDiscreteKraskov implements Mutu
// double fixedPartOfLocals = MathsUtils.digamma(k) - 1.0/(double)k +
// MathsUtils.digamma(N);
// Instead do:
double fixedPartOfLocals = MathsUtils.digamma(k) + MathsUtils.digamma(N);
double fixedPartOfLocals = MathsUtils.digamma(k) + MathsUtils.digamma(N_samplesForPdfs); // Use N_samplesForPdfs here because that's what would be in denominator of probability functions
double testSum = 0.0;
if (debug) {
System.out.printf("digamma(k)=%.3f + digamma(N)=%.3f\n",
MathsUtils.digamma(k), MathsUtils.digamma(N));
MathsUtils.digamma(k), MathsUtils.digamma(N_samplesForPdfs));
}
double avNx = 0;
double avNy = 0;
for (int t = 0; t < N; t++) {
for (int t = 0; t < N_newObservations; t++) {
// Compute eps_x and eps_y for this time step:
// First get x norms to all points in the previously
// given observations.
@ -629,8 +646,8 @@ public class MutualInfoCalculatorMultiVariateWithDiscreteKraskov implements Mutu
}
}
if (debug) {
avNx /= (double)N;
avNy /= (double)N;
avNx /= (double)N_newObservations;
avNy /= (double)N_newObservations;
System.out.printf("Average n_x=%.3f, Average n_y=%.3f\n", avNx, avNy);
}

1321
java/source/infodynamics/utils/KdTree.java
View File

File diff suppressed because it is too large Load Diff

204
java/source/infodynamics/utils/MatrixUtils.java
View File

@ -137,6 +137,18 @@ public class MatrixUtils {
return total;
}
public static double sum(double[][][] input) {
double total = 0;
for (int i = 0; i < input.length; i++) {
for (int j = 0; j < input[i].length; j++) {
for (int k = 0; k < input[i][j].length; k++) {
total += input[i][j][k];
}
}
}
return total;
}
public static int sum(int[] input) {
int total = 0;
for (int i = 0; i < input.length; i++) {
@ -155,6 +167,18 @@ public class MatrixUtils {
return total;
}
public static int sum(int[][][] input) {
int total = 0;
for (int i = 0; i < input.length; i++) {
for (int j = 0; j < input[i].length; j++) {
for (int k = 0; k < input[i][j].length; k++) {
total += input[i][j][k];
}
}
}
return total;
}
/**
* Return an array of the sums for each column in the 2D input
*
@ -979,6 +1003,23 @@ public class MatrixUtils {
}
}
/**
* Copies the given source array into the required column number of the destination
* @param destination
* @param index1
* @param index2
* @param source
*/
public static void copyIntoColumn3D(int[][][] destination, int index1, int index2, int[] source) throws Exception {
if (source.length != destination.length) {
throw new Exception("Destination column is not of the same length as the source (" +
destination.length + " vs " + source.length + ")");
}
for (int r = 0; r < destination.length; r++) {
destination[r][index1][index2] = source[r];
}
}
/**
* Return a new matrix with the columns of matrix1 joined on the back of matrix2
*
@ -1258,6 +1299,14 @@ public class MatrixUtils {
return returnData;
}
public static boolean[] selectColumn(boolean matrix[][], int columnNo) {
boolean[] column = new boolean[matrix.length];
for (int r = 0; r < matrix.length; r++) {
column[r] = matrix[r][columnNo];
}
return column;
}
public static int[] selectColumn(int matrix[][], int columnNo) {
int[] column = new int[matrix.length];
for (int r = 0; r < matrix.length; r++) {
@ -1301,6 +1350,25 @@ public class MatrixUtils {
return column;
}
/**
* Extract the required columns from the matrix
*
* @param matrix
* @param fromCol
* @param cols
* @return
*/
public static double[][] selectColumns(double matrix[][],
int fromCol, int cols) {
double[][] data = new double[matrix.length][cols];
for (int r = 0; r < matrix.length; r++) {
for (int cIndex = 0; cIndex < cols; cIndex++) {
data[r][cIndex] = matrix[r][cIndex + fromCol];
}
}
return data;
}
/**
* Extract the required columns from the matrix
*
@ -1394,6 +1462,27 @@ public class MatrixUtils {
return data;
}
/**
* Extract the required rows and columns from the matrix
*
* @param matrix 2D data array
* @param fromRow index of the first row to return
* @param rows number of rows (including the first) to return
* @param fromCol index of the first column to return
* @param cols number of columns (including the first) to return
* @return a 2D data array of the selected rows and columns
*/
public static double[][] selectRowsAndColumns(double matrix[][],
int fromRow, int rows, int fromCol, int cols) {
double[][] data = new double[rows][cols];
for (int rIndex = 0; rIndex < rows; rIndex++) {
for (int cIndex = 0; cIndex < cols; cIndex++) {
data[rIndex][cIndex] = matrix[rIndex + fromRow][cIndex + fromCol];
}
}
return data;
}
public static double[][] selectFirstTwoDimenions(double[][][][] matrix, int d2, int d3) {
double[][] newMatrix = new double[matrix.length][];
for (int i = 0; i < matrix.length; i++) {
@ -1654,6 +1743,24 @@ public class MatrixUtils {
return matrix;
}
/**
* Constructs all embedding vectors of size k for the data.
* There will be (data.length - k + 1) of these vectors returned.
*
* @param data time series data
* @param k embedding length
* @return An array of k-length embedding vectors
*/
public static int[][] makeDelayEmbeddingVector(int[] data, int k) {
try {
return makeDelayEmbeddingVector(data, k, k - 1, data.length - k + 1);
} catch (Exception e) {
// The above call should not throw an Exception, handle here
// in a RuntimeException so this method doesn't throw one
throw new RuntimeException(e);
}
}
/**
* Constructs all embedding vectors of size k for the data.
* There will be (data.length - k + 1) of these vectors returned.
@ -1672,6 +1779,39 @@ public class MatrixUtils {
}
}
/**
* Constructs numEmbeddingVectors embedding vectors of size k for the data,
* with the first embedding vector having it's last time point at t=startKthPoint
*
* @param data time series data
* @param k embedding length
* @param startKthPoint last time point of the first embedding vector
* (i.e. use k-1 if you want to go from the start)
* @param numEmbeddingVectors the number of embedding vectors to return
* (i.e. use data.length-k+1 if you go from the start and want all
* of them extracted)
* @return a 2D array of k-length embedding vectors.
*/
public static int[][] makeDelayEmbeddingVector(int[] data, int k,
int startKthPoint, int numEmbeddingVectors) throws Exception {
if (startKthPoint < k - 1) {
throw new Exception("Start point t=" + startKthPoint + " is too early for a " +
k + " length embedding vector");
}
if (numEmbeddingVectors + startKthPoint > data.length) {
throw new Exception("Too many embedding vectors " + numEmbeddingVectors +
" requested for the given startPoint " + startKthPoint +
" and time series length " + data.length);
}
int[][] embeddingVectors = new int[numEmbeddingVectors][k];
for (int t = startKthPoint; t < numEmbeddingVectors + startKthPoint; t++) {
for (int i = 0; i < k; i++) {
embeddingVectors[t - startKthPoint][i] = data[t - i];
}
}
return embeddingVectors;
}
/**
* Constructs numEmbeddingVectors embedding vectors of size k for the data,
* with the first embedding vector having it's last time point at t=startKthPoint
@ -1705,6 +1845,42 @@ public class MatrixUtils {
return embeddingVectors;
}
/**
* Constructs numEmbeddingVectors embedding vectors of size k for the data,
* with embedding delay tau between each time sample for the vectors,
* with the first embedding vector having it's last time point at t=startKthPoint
*
* @param data time series data
* @param k embedding length
* @param tau embedding delay between each point in the original time series
* selected into each embedding vector
* @param startKthPoint last time point of the first embedding vector
* (i.e. use (k-1)*tau if you want to go from the start)
* @param numEmbeddingVectors the number of embedding vectors to return
* (i.e. use data.length-(k-1)*tau if you go from the start and want all
* of them extracted)
* @return a 2D array of k-length embedding vectors.
*/
public static int[][] makeDelayEmbeddingVector(int[] data, int k, int tau,
int startKthPoint, int numEmbeddingVectors) throws Exception {
if (startKthPoint < (k - 1)*tau) {
throw new Exception("Start point t=" + startKthPoint + " is too early for a " +
k + " length embedding vector with delay " + tau);
}
if (numEmbeddingVectors + startKthPoint > data.length) {
throw new Exception("Too many embedding vectors " + numEmbeddingVectors +
" requested for the given startPoint " + startKthPoint +
" and time series length " + data.length);
}
int[][] embeddingVectors = new int[numEmbeddingVectors][k];
for (int t = startKthPoint; t < numEmbeddingVectors + startKthPoint; t++) {
for (int i = 0; i < k; i++) {
embeddingVectors[t - startKthPoint][i] = data[t - i*tau];
}
}
return embeddingVectors;
}
/**
* Constructs numEmbeddingVectors embedding vectors of size k for the data,
* with embedding delay tau between each time sample for the vectors,
@ -2163,6 +2339,25 @@ public class MatrixUtils {
}
}
/**
* Normalises the elements along each column of the matrix
*
* @param matrix 2D matrix of doubles
* @param means means to normalise to for each column of data
* @param stds std deviations to normalise to for each column of data
*/
public static void normalise(double[][] matrix, double[] means, double[] stds) {
for (int r = 0; r < matrix.length; r++) {
for (int c = 0; c < matrix[r].length; c++) {
matrix[r][c] = matrix[r][c] - means[c];
if (!Double.isInfinite(1.0 / stds[c])) {
matrix[r][c] /= stds[c];
} // else we just subtract off the mean
}
}
}
/**
* Normalises the elements along each column of the matrix
*
@ -3706,9 +3901,16 @@ public class MatrixUtils {
for (int r = 0; r < array.length; r++) {
out.print(array[r] + " ");
}
out.println();
// out.println();
}
public static void printArray(PrintStream out, double[] array, int numDp) {
for (int r = 0; r < array.length; r++) {
out.printf(String.format("%%.%df ", numDp), array[r]);
}
// out.println();
}
public static String arrayToString(double[] array) {
StringBuffer sb = new StringBuffer();
for (int r = 0; r < array.length; r++) {

93
java/source/infodynamics/utils/NearestNeighbourSearcher.java
View File

@ -49,7 +49,10 @@ public abstract class NearestNeighbourSearcher {
public static NearestNeighbourSearcher create(double[][] data)
throws Exception {
if (data[0].length == 1) {
if ((data == null) || (data[0].length == 0)) {
// We have null data:
return null;
} else if (data[0].length == 1) {
// We have univariate data:
return new UnivariateNearestNeighbourSearcher(MatrixUtils.selectColumn(data, 0));
} else {
@ -271,6 +274,23 @@ public abstract class NearestNeighbourSearcher {
public abstract int countPointsWithinR(int sampleIndex, double r,
int dynCorrExclTime, boolean allowEqualToR);
/**
* As per {@link #countPointsWithinR(int, double, boolean)}
* however the search is to match a specified sample point (not a sample
* point within the search space itself).
*
* @param sampleVectors sample vectors to find the neighbours within r
* for
* @param r radius within which to count points
* @param dynCorrExclTime time window within which to exclude
* points to be counted. Is >= 0. 0 means only exclude sampleIndex.
* @param allowEqualToR if true, then count points at radius r also,
* otherwise only those strictly within r
* @return the count of points within r.
*/
public abstract int countPointsWithinR(double[][] sampleVectors, double r,
boolean allowEqualToR);
/**
* As per {@link #countPointsWithinR(int, double, boolean)}
* however returns a collection rather than a count.
@ -331,6 +351,60 @@ public abstract class NearestNeighbourSearcher {
int sampleIndex, double r, int dynCorrExclTime,
boolean allowEqualToR, boolean[] isWithinR, int[] indicesWithinR);
/**
* As per {@link #findPointsWithinR(int, double, boolean, boolean[], int[])}
* however incorporates dynamic correlation exclusion.
* </p>
*
* @param sampleIndex sample index in the data to find a nearest neighbour
* for
* @param r radius within which to count points
* @param dynCorrExclTime time window within which to exclude
* points to be counted. Is >= 0. 0 means only exclude sampleIndex.
* @param allowEqualToR if true, then count points at radius r also,
* otherwise only those strictly within r
* @param isWithinR the array MUST be passed in with all points set to
* false initially, and is returned indicating whether each sample was
* found to be within r of that at sampleIndex.
* @param distancesWithinRInOrder the array must be passed in and is
* returned with distances for each point found to be within r. Values
* at other indices are not defined.
* @param distancesAndIndicesWithinR is returned as
* a list of distances (in column index 0)
* and array indices (in column index 1)
* for points marked as true in isWithinR, terminated with a -1 value on the index.
* @return the point count
*/
public abstract int findPointsWithinR(
int sampleIndex, double r, int dynCorrExclTime,
boolean allowEqualToR, boolean[] isWithinR,
double[] distancesWithinRInOrder,
double[][] distancesAndIndicesWithinR);
/**
* As per {@link #countPointsWithinR(int, double, boolean)}
* however records the nearest neighbours for a sample data point
* (which may not be in the search tree), within the isWithinR
* and indicesWithinR arrays, which must be constructed before
* calling this method, with length at or exceeding the total
* number of data points. indicesWithinR is
* </p>
*
* @param r radius within which to count points
* @param sampleVectors sample vectors to find the neighbours within r
* for
* @param allowEqualToR if true, then count points at radius r also,
* otherwise only those strictly within r
* @param isWithinR the array MUST be passed in with all points set to
* false initially, and is returned indicating whether each sample was
* found to be within r of that at sampleIndex.
* @param indicesWithinR a list of array indices
* for points marked as true in isWithinR, terminated with a -1 value.
*/
public abstract void findPointsWithinR(
double r, double[][] sampleVectors,
boolean allowEqualToR, boolean[] isWithinR, int[] indicesWithinR);
/**
* As per {@link #countPointsWithinR(int, double, int, boolean)}
* with allowEqualToR == false
@ -472,5 +546,20 @@ public abstract class NearestNeighbourSearcher {
public abstract int countPointsWithinR(int sampleIndex, double r,
boolean allowEqualToR, boolean[] additionalCriteria);
/**
* As per {@link #countPointsWithinR(double[][], double, boolean)}
* however each point is subject to also meeting the additional
* criteria of being true in additionalCriteria.
*
* @param sampleVectors sample vectors to find the neighbours within r
* for
* @param r radius within which to count points
* @param allowEqualToR if true, then count points at radius r also,
* otherwise only those strictly within r
* @param additionalCriteria array of booleans. Only count a point if it
* is within r and is true in additionalCrtieria.
* @return the count of points within r.
*/
public abstract int countPointsWithinR(double[][] sampleVectors, double r,
boolean allowEqualToR, boolean[] additionalCriteria);
}

480
java/source/infodynamics/utils/UnivariateNearestNeighbourSearcher.java
View File

@ -18,6 +18,7 @@
package infodynamics.utils;
import java.util.Arrays;
import java.util.Collection;
import java.util.PriorityQueue;
import java.util.Vector;
@ -43,6 +44,7 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
* Number of samples in the data
*/
protected int numObservations = 0;
/**
* An array of indices to the data in originalDataSet,
* sorted in order (min to max).
@ -55,6 +57,11 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
*/
protected int[] indicesInSortedArray = null;
/**
* An array of the original values sorted into ascending order.
*/
protected double[] sortedValues = null;
public UnivariateNearestNeighbourSearcher(double[][] data) throws Exception {
// Ideally we would not call the constructor until after the following check,
// but the constructor must come first in Java.
@ -88,6 +95,13 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
sortedArrayIndices[t] = (int) dataWithIndices[t][1];
indicesInSortedArray[(int) dataWithIndices[t][1]] = t;
}
// Finally, create the sorted array of the values:
// (these are only used by a few methods, but better to create the array now
// rather than hit concurrency issues trying to create it when required
sortedValues = new double[numObservations];
for (int i = 0; i < numObservations; i++) {
sortedValues[i] = originalDataSet[sortedArrayIndices[i]];
}
}
/**
@ -359,6 +373,47 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
return count;
}
/**
* Count the number of points within norms r_upper and r_lower for a given
* sample index in the data set. The node itself is
* excluded from the search.
* Nearest neighbour function to compare to radii is the specified norm.
* (If {@link EuclideanUtils#NORM_EUCLIDEAN} was selected, then the supplied
* r should be the required Euclidean norm <b>squared</b>, since we switch it
* to {@link EuclideanUtils#NORM_EUCLIDEAN_SQUARED} internally).
*/
public int countPointsWithinRs(int sampleIndex, double r_upper,
double r_lower, boolean allowEqualToR) {
int count = 0;
// Find where this node sits in the sorted array:
int indexInSortedArray = indicesInSortedArray[sampleIndex];
// Check the points with smaller data values first:
for (int i = indexInSortedArray - 1; i >= 0; i--) {
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r_lower)) ||
(!allowEqualToR && (theNorm < r_lower))) {
count++;
continue;
}
// Else no point checking further points
break;
}
// Next check the points with larger data values:
for (int i = indexInSortedArray + 1; i < numObservations; i++) {
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r_upper)) ||
(!allowEqualToR && (theNorm < r_upper))) {
count++;
continue;
}
// Else no point checking further points
break;
}
return count;
}
/**
* Count the number of points within norm r for a given
* sample index in the data set. Nodes within dynCorrExclTime
@ -408,6 +463,124 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
return count;
}
/**
* Count the number of points within norm r for a given
* sample index in the data set, or larger than r. The node itself is
* excluded from the search.
* Nearest neighbour function to compare to r is the specified norm.
* (If {@link EuclideanUtils#NORM_EUCLIDEAN} was selected, then the supplied
* r should be the required Euclidean norm <b>squared</b>, since we switch it
* to {@link EuclideanUtils#NORM_EUCLIDEAN_SQUARED} internally).
*/
public int countPointsWithinROrLarger(int sampleIndex, double r, boolean allowEqualToR) {
int count = 0;
// Find where this node sits in the sorted array:
int indexInSortedArray = indicesInSortedArray[sampleIndex];
// Check the points with smaller data values first:
for (int i = indexInSortedArray - 1; i >= 0; i--) {
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
count++;
continue;
}
// Else no point checking further points
break;
}
count += numObservations - indexInSortedArray - 1;
return count;
}
/**
* Count the number of points less than the sample at the given index,
* and outside norm r. allowEqualToR means to be considered same as the sample.
* The node itself is
* excluded from the search.
* Nearest neighbour function to compare to r is the specified norm.
* (If {@link EuclideanUtils#NORM_EUCLIDEAN} was selected, then the supplied
* r should be the required Euclidean norm <b>squared</b>, since we switch it
* to {@link EuclideanUtils#NORM_EUCLIDEAN_SQUARED} internally).
*/
public int countPointsSmallerAndOutsideR(int sampleIndex, double r, boolean allowEqualToR) {
// Find where this node sits in the sorted array:
int indexInSortedArray = indicesInSortedArray[sampleIndex];
// Check the points with smaller data values first:
int i;
for (i = indexInSortedArray - 1; i >= 0; i--) {
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
continue;
}
// Else no point checking further points
break;
}
return i + 1;
}
@Override
public int countPointsWithinR(double[][] sampleVectors, double r, boolean allowEqualToR) {
if (sampleVectors.length > 1) {
throw new RuntimeException("sampleVectors[][] argument must have length 1 on each dimension when calling a UnivariateNearstNeighbourSearcher");
}
if (sampleVectors[0].length > 1) {
throw new RuntimeException("sampleVectors[][] argument must have length 1 on each dimension when calling a UnivariateNearstNeighbourSearcher");
}
return countPointsWithinR(sampleVectors[0][0], r, allowEqualToR);
}
/**
* As per {@link #countPointsWithinR(double, double, boolean)} however
* for a specific sample value (not one within the data set itself).
*
* @param sampleValue the specific value to search for
*/
public int countPointsWithinR(double sampleValue, double r, boolean allowEqualToR) {
int count = 0;
// Find where this value would sit in the sorted array.
int potentialArrayIndex = Arrays.binarySearch(sortedValues, sampleValue);
if (potentialArrayIndex < 0) {
// The sampleValue was not found because it wasn't in the original set of values:
// invert the return value of binarySearch to find where the sampleValue would be inserted into the array:
potentialArrayIndex = -potentialArrayIndex - 1;
// This means at potentialArrayIndex and higher are values >= sampleValue, while
// at potentialArrayIndex-1 and lower are values <= sampleValue.
// potentialArrayIndex will be array.length if all values are smaller than it.
} // else potentialArrayIndex is pointing to one instance of this value in the sorted array
// Check the points with smaller data values first
for (int i = potentialArrayIndex - 1; i >= 0; i--) {
double theNorm = norm(sampleValue,
sortedValues[i], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
count++;
continue;
}
// Else no point checking further points
break;
}
// Next check the points with larger data values: (and including potentialArrayIndex, since we're
// not excluding it since it's not nominally part of the underlying data set)
for (int i = potentialArrayIndex; i < numObservations; i++) {
double theNorm = norm(sampleValue,
sortedValues[i], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
count++;
continue;
}
// Else no point checking further points
break;
}
return count;
}
/**
* Find the collection of points within norm r for a given
* sample index in the data set. The node itself is
@ -565,6 +738,214 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
indicesWithinR[indexInIndicesWithinR++] = -1;
}
@Override
public int findPointsWithinR(int sampleIndex, double r,
int dynCorrExclTime, boolean allowEqualToR, boolean[] isWithinR,
double[] distancesWithinRInOrder,
double[][] distancesAndIndicesWithinR) {
int indexInIndicesWithinR = 0;
// Find where this node sits in the sorted array:
int indexInSortedArray = indicesInSortedArray[sampleIndex];
// Check the points with smaller data values first:
for (int i = indexInSortedArray - 1; i >= 0; i--) {
if (Math.abs(sampleIndex - sortedArrayIndices[i]) <= dynCorrExclTime) {
// Can't count this point, but keep checking:
continue;
}
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
isWithinR[sortedArrayIndices[i]] = true;
distancesWithinRInOrder[sortedArrayIndices[i]] = theNorm;
distancesAndIndicesWithinR[indexInIndicesWithinR][0] = theNorm;
distancesAndIndicesWithinR[indexInIndicesWithinR++][1] = sortedArrayIndices[i];
continue;
}
// Else no point checking further points
break;
}
// Next check the points with larger data values:
for (int i = indexInSortedArray + 1; i < numObservations; i++) {
if (Math.abs(sampleIndex - sortedArrayIndices[i]) <= dynCorrExclTime) {
// Can't count this point, but keep checking:
continue;
}
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
isWithinR[sortedArrayIndices[i]] = true;
distancesWithinRInOrder[sortedArrayIndices[i]] = theNorm;
distancesAndIndicesWithinR[indexInIndicesWithinR][0] = theNorm;
distancesAndIndicesWithinR[indexInIndicesWithinR++][1] = sortedArrayIndices[i];
continue;
}
// Else no point checking further points
break;
}
// Write the terminating integer into the indicesWithinR array:
distancesAndIndicesWithinR[indexInIndicesWithinR++][1] = -1;
return indexInIndicesWithinR - 1;
}
/**
* As per {@link #findPointsWithinR(int, double, int, boolean, boolean[], double[], double[][])}
* only we have additional criteria, and we report max values taking that additional
* criteria into account.
*
* @param sampleIndex sample index in the data to find a nearest neighbour
* for
* @param r radius within which to count points
* @param dynCorrExclTime time window within which to exclude
* points to be counted. Is >= 0. 0 means only exclude sampleIndex.
* @param allowEqualToR if true, then count points at radius r also,
* otherwise only those strictly within r
* @param additionalCriteria an array indicating whether each sample for
* should be tested for this search point
* @param reportDistancesAsMaxWith an array of distances for
* points flagged in additionalCriteria: distancesAndIndicesWithinR should
* return the max of this value or the distances within r computed here.
* Values at indices where distances are not within r are not defined.
* @param distancesAndIndicesWithinR a list of distances (in column index 0)
* and array indices (in column index 1)
* for points found to be within r, terminated with a -1 value on the index.
* @return the next available index in distancesAndIndicesWithinR after the method is complete
*/
public int findPointsWithinRAndTakeRadiusMax(int sampleIndex, double r,
int dynCorrExclTime, boolean allowEqualToR,
boolean[] additionalCriteria,
double[] reportDistancesAsMaxWith,
double[][] distancesAndIndicesWithinR) {
int indexInIndicesWithinR = 0;
// Find where this node sits in the sorted array:
int indexInSortedArray = indicesInSortedArray[sampleIndex];
// Check the points with smaller data values first:
for (int i = indexInSortedArray - 1; i >= 0; i--) {
if (!additionalCriteria[sortedArrayIndices[i]] ||
(Math.abs(sampleIndex - sortedArrayIndices[i]) <= dynCorrExclTime)) {
// Can't count this point, but keep checking:
continue;
}
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
if (theNorm < reportDistancesAsMaxWith[sortedArrayIndices[i]]) {
theNorm = reportDistancesAsMaxWith[sortedArrayIndices[i]];
}
distancesAndIndicesWithinR[indexInIndicesWithinR][0] = theNorm;
distancesAndIndicesWithinR[indexInIndicesWithinR++][1] = sortedArrayIndices[i];
continue;
}
// Else no point checking further points
break;
}
// Next check the points with larger data values:
for (int i = indexInSortedArray + 1; i < numObservations; i++) {
if (!additionalCriteria[sortedArrayIndices[i]] ||
(Math.abs(sampleIndex - sortedArrayIndices[i]) <= dynCorrExclTime)) {
// Can't count this point, but keep checking:
continue;
}
double theNorm = norm(originalDataSet[sampleIndex],
originalDataSet[sortedArrayIndices[i]], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
if (theNorm < reportDistancesAsMaxWith[sortedArrayIndices[i]]) {
theNorm = reportDistancesAsMaxWith[sortedArrayIndices[i]];
}
distancesAndIndicesWithinR[indexInIndicesWithinR][0] = theNorm;
distancesAndIndicesWithinR[indexInIndicesWithinR++][1] = sortedArrayIndices[i];
continue;
}
// Else no point checking further points
break;
}
// Write the terminating integer into the indicesWithinR array:
distancesAndIndicesWithinR[indexInIndicesWithinR++][1] = -1;
return indexInIndicesWithinR - 1;
}
/**
* As per {@link #findPointsWithinR(int, double, int, boolean, boolean[], int[])}
* only for a specific data point.
*
* Note that the method signature takes a 2D sample vector, however since this is class
* only searches univariate spaces, then there must only be one value in #sampleVector
* (i.e. length one on both array indices).
*
*
* @see super{@link #findPointsWithinR(double, double[][], boolean, boolean[], int[])}
*/
@Override
public void findPointsWithinR(double r, double[][] sampleVectors,
boolean allowEqualToR, boolean[] isWithinR, int[] indicesWithinR) {
if (sampleVectors.length > 1) {
throw new RuntimeException("sampleVectors[][] argument must have length 1 on each dimension when calling a UnivariateNearstNeighbourSearcher");
}
if (sampleVectors[0].length > 1) {
throw new RuntimeException("sampleVectors[][] argument must have length 1 on each dimension when calling a UnivariateNearstNeighbourSearcher");
}
findPointsWithinR(r, sampleVectors[0][0], allowEqualToR, isWithinR, indicesWithinR);
}
/**
* As per {@link #findPointsWithinR(int, double, int, boolean, boolean[], int[])}
* only for a specific data point.
*
* @see super{@link #findPointsWithinR(double, double[][], boolean, boolean[], int[])}
*
* @param sampleValue the specific point being searched (not a data point in the search structure)
*/
public void findPointsWithinR(double r, double sampleValue,
boolean allowEqualToR, boolean[] isWithinR, int[] indicesWithinR) {
int indexInIndicesWithinR = 0;
// Find where this value would sit in the sorted array.
int potentialArrayIndex = Arrays.binarySearch(sortedValues, sampleValue);
if (potentialArrayIndex < 0) {
// The sampleValue was not found because it wasn't in the original set of values:
// invert the return value of binarySearch to find where the sampleValue would be inserted into the array:
potentialArrayIndex = -potentialArrayIndex - 1;
// This means at potentialArrayIndex and higher are values >= sampleValue, while
// at potentialArrayIndex-1 and lower are values <= sampleValue.
// potentialArrayIndex will be array.length if all values are smaller than it.
} // else potentialArrayIndex is pointing to one instance of this value in the sorted array
// Check the points with smaller data values first:
for (int i = potentialArrayIndex - 1; i >= 0; i--) {
double theNorm = norm(sampleValue,
sortedValues[i], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
isWithinR[sortedArrayIndices[i]] = true;
indicesWithinR[indexInIndicesWithinR++] = sortedArrayIndices[i];
continue;
}
// Else no point checking further points
break;
}
// Next check the points with larger data values: (and including potentialArrayIndex, since we're
// not excluding it since it's not nominally part of the underlying data set)
for (int i = potentialArrayIndex; i < numObservations; i++) {
double theNorm = norm(sampleValue,
sortedValues[i], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
isWithinR[sortedArrayIndices[i]] = true;
indicesWithinR[indexInIndicesWithinR++] = sortedArrayIndices[i];
continue;
}
// Else no point checking further points
break;
}
// Write the terminating integer into the indicesWithinR array:
indicesWithinR[indexInIndicesWithinR++] = -1;
}
@Override
public int countPointsWithinR(int sampleIndex, double r, boolean allowEqualToR,
boolean[] additionalCriteria) {
@ -618,4 +999,103 @@ public class UnivariateNearestNeighbourSearcher extends NearestNeighbourSearcher
}
return count;
}
/**
* As per {@link #countPointsWithinR(int, double, boolean, boolean[])}
* only for a specific data point.
*
* Note that the method signature takes a 2D sample vector, however since this is class
* only searches univariate spaces, then there must only be one value in #sampleVector
* (i.e. length one on both array indices).
*
*
* @see super{@link #findPointsWithinR(double, double[][], boolean, boolean[], int[])}
*/
@Override
public int countPointsWithinR(double[][] sampleVectors, double r, boolean allowEqualToR,
boolean[] additionalCriteria) {
if (sampleVectors.length > 1) {
throw new RuntimeException("sampleVectors[][] argument must have length 1 on each dimension when calling a UnivariateNearstNeighbourSearcher");
}
if (sampleVectors[0].length > 1) {
throw new RuntimeException("sampleVectors[][] argument must have length 1 on each dimension when calling a UnivariateNearstNeighbourSearcher");
}
return countPointsWithinR(sampleVectors[0][0], r, allowEqualToR, additionalCriteria);
}
/**
* As per {@link #countPointsWithinR(int, double, boolean, boolean[])}
* only for a specific data point.
*
* @param sampleValue the specific point being searched (not a data point in the search structure)
*/
public int countPointsWithinR(double sampleValue, double r, boolean allowEqualToR,
boolean[] additionalCriteria) {
int count = 0;
// Find where this value would sit in the sorted array.
int potentialArrayIndex = Arrays.binarySearch(sortedValues, sampleValue);
if (potentialArrayIndex < 0) {
// The sampleValue was not found because it wasn't in the original set of values:
// invert the return value of binarySearch to find where the sampleValue would be inserted into the array:
potentialArrayIndex = -potentialArrayIndex - 1;
// This means at potentialArrayIndex and higher are values >= sampleValue, while
// at potentialArrayIndex-1 and lower are values <= sampleValue.
// potentialArrayIndex will be array.length if all values are smaller than it.
} // else potentialArrayIndex is pointing to one instance of this value in the sorted array
// Check the points with smaller data values first:
for (int i = potentialArrayIndex - 1; i >= 0; i--) {
if (!additionalCriteria[sortedArrayIndices[i]]) {
// This point failed the additional criteria,
// so skip checking it.
// We check this first, even though it's the norm
// that really determines whether we need to continue
// checking or not. In some circumstances this may be
// slower, but for our main application - KSG conditional MI -
// this should be faster.
continue;
}
double theNorm = norm(sampleValue,
sortedValues[i], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
count++;
continue;
}
// Else no point checking further points
break;
}
// Next check the points with larger data values: (and including potentialArrayIndex, since we're
// not excluding it since it's not nominally part of the underlying data set)
for (int i = potentialArrayIndex; i < numObservations; i++) {
if (!additionalCriteria[sortedArrayIndices[i]]) {
// This point failed the additional criteria,
// so skip checking it.
// We check this first, even though it's the norm
// that really determines whether we need to continue
// checking or not. In some circumstances this may be
// slower, but for our main application - KSG conditional MI -
// this should be faster.
continue;
}
double theNorm = norm(sampleValue,
sortedValues[i], normTypeToUse);
if ((allowEqualToR && (theNorm <= r)) ||
(!allowEqualToR && (theNorm < r))) {
count++;
continue;
}
// Else no point checking further points
break;
}
return count;
}
public int getNumObservations() {
return numObservations;
}
}

6
java/unittests/infodynamics/measures/continuous/kernel/ActiveInfoStorageCalculatorKernelDirect.java
View File

@ -272,4 +272,10 @@ public class ActiveInfoStorageCalculatorKernelDirect
double[] newObservations) throws Exception {
throw new Exception("Not yet implemented");
}
public void addObservations(double[] observations, boolean[] valid)
throws Exception {
// Probably won't need to implement this, as we're only using this class for testing
throw new Exception("Not implemented yet");
}
}

238
java/unittests/infodynamics/measures/continuous/kraskov/ConditionalMutualInfoMultiVariateTester.java
View File

@ -18,8 +18,13 @@
package infodynamics.measures.continuous.kraskov;
import java.util.Calendar;
import infodynamics.utils.ArrayFileReader;
import infodynamics.utils.EmpiricalMeasurementDistribution;
import infodynamics.utils.MathsUtils;
import infodynamics.utils.MatrixUtils;
import infodynamics.utils.RandomGenerator;
public class ConditionalMutualInfoMultiVariateTester
extends infodynamics.measures.continuous.ConditionalMutualInfoMultiVariateAbstractTester {
@ -423,7 +428,109 @@ public class ConditionalMutualInfoMultiVariateTester
1, 1,
kNNs, expectedFromTRENTOOL);
}
/**
* Unit test for conditional MI on new observations.
* We can test this against the calculator itself. If we send in the original data set as new observations,
* we can recreate the neighbour counts (plus one) by setting K to 1 larger (to account for the data point itself), and
* account for the change in bias.
*
* @throws Exception
*/
public void testMultivariateCondMIForNewObservations() throws Exception {
ArrayFileReader afr = new ArrayFileReader("demos/data/4ColsPairedOneStepNoisyDependence-1.txt");
double[][] data = afr.getDouble2DMatrix();
// RandomGenerator rg = new RandomGenerator();
//double[][] data = rg.generateNormalData(50, 4, 0, 1);
// Use various Kraskov k nearest neighbours parameter
int[] kNNs = {4};
System.out.println("Kraskov Cond MI testing new Observations:");
for (int ki = 0; ki < kNNs.length; ki++) {
ConditionalMutualInfoCalculatorMultiVariateKraskov condMiCalc = getNewCalc(1);
ConditionalMutualInfoCalculatorMultiVariateKraskov condMiCalcForNew = getNewCalc(1);
// Let it normalise by default
// And no noise addition to protect the integrity of our neighbour counts under both techniques here:
condMiCalc.setProperty(ConditionalMutualInfoCalculatorMultiVariateKraskov.PROP_ADD_NOISE, "0");
condMiCalcForNew.setProperty(ConditionalMutualInfoCalculatorMultiVariateKraskov.PROP_ADD_NOISE, "0");
double[][] var1 = MatrixUtils.selectColumns(data, new int[] {0});
double[][] var2 = MatrixUtils.selectColumns(data, new int[] {1});
double[][] condVar = MatrixUtils.selectColumns(data, new int[] {2, 3});
// Compute MI(0;1|2,3) :
condMiCalc.setProperty(
ConditionalMutualInfoCalculatorMultiVariateKraskov.PROP_K,
Integer.toString(kNNs[ki]));
System.out.println("Main calc normalisation is " + condMiCalc.getProperty(condMiCalc.PROP_NORMALISE));
condMiCalc.initialise(var1[0].length, var2[0].length, condVar[0].length);
condMiCalc.setObservations(var1, var2, condVar);
double miAverage = condMiCalc.computeAverageLocalOfObservations();
// Now compute as new observations:
condMiCalcForNew.setProperty(
ConditionalMutualInfoCalculatorMultiVariateKraskov.PROP_K,
Integer.toString(kNNs[ki] + 1)); // Using K = K + 1
// condMiCalcForNew.setProperty(
// ConditionalMutualInfoCalculatorMultiVariateKraskov.PROP_NUM_THREADS,
// "1");
System.out.println("New obs calc normalisation is " + condMiCalcForNew.getProperty(condMiCalc.PROP_NORMALISE));
condMiCalcForNew.initialise(var1[0].length, var2[0].length, condVar[0].length);
condMiCalcForNew.setObservations(var1, var2, condVar);
// condMiCalc.setDebug(true);
//condMiCalcForNew.setDebug(true);
double[] newLocals = condMiCalcForNew.computeLocalUsingPreviousObservations(var1, var2, condVar);
//condMiCalcForNew.setDebug(false);
double averageFromNewObservations = MatrixUtils.mean(newLocals);
// We can't check this directly, so test each point individually:
for (int t = 0; t < data.length; t++) {
double[] originalNeighbourCounts = condMiCalc.partialComputeFromObservations(t, 1, false);
// Need to normalise the data before passing it in here -- this is
// what is happening inside computeLocalUsingPreviousObservations above
double[] newObsNeighbourCounts = condMiCalcForNew.partialComputeFromNewObservations(
t, 1,
MatrixUtils.normaliseIntoNewArray(var1),
MatrixUtils.normaliseIntoNewArray(var2),
MatrixUtils.normaliseIntoNewArray(condVar), false);
// Now check each return count in the array:
if (originalNeighbourCounts[1] != newObsNeighbourCounts[1] - 1) {
System.out.println("Assertion failure for t=" + t + ": expected " + originalNeighbourCounts[1] +
" from original, plus 1, but got " + newObsNeighbourCounts[1]);
System.out.print("Actual raw data was: ");
MatrixUtils.printArray(System.out, data[0]);
}
assertEquals(originalNeighbourCounts[1], newObsNeighbourCounts[1] - 1); // Nxz should be 1 higher
assertEquals(originalNeighbourCounts[2], newObsNeighbourCounts[2] - 1); // Nyz should be 1 higher
assertEquals(originalNeighbourCounts[3], newObsNeighbourCounts[3] - 1); // Nz should be 1 higher
// Now check the local value at each point using these verified counts:
double newLocalValue = condMiCalcForNew.digammaK -
MathsUtils.digamma((int) newObsNeighbourCounts[1] + 1) -
MathsUtils.digamma((int) newObsNeighbourCounts[2] + 1) +
MathsUtils.digamma((int) newObsNeighbourCounts[3] + 1);
/* System.out.printf("n=%d, n_xz=%d, n_yz=%d, n_z=%d, local=%.4f," +
" digamma(n_xz+1)=%.5f, digamma(n_yz+1)=%.5f, digamma(n_z+1)=%.5f, \n",
t, (int) newObsNeighbourCounts[1], (int) newObsNeighbourCounts[2],
(int) newObsNeighbourCounts[3], newLocalValue,
MathsUtils.digamma((int) newObsNeighbourCounts[1] + 1),
MathsUtils.digamma((int) newObsNeighbourCounts[2] + 1),
MathsUtils.digamma((int) newObsNeighbourCounts[3] + 1)); */
if (Math.abs(newLocalValue - newLocals[t]) > 0.00000001) {
System.out.printf("t=%d: Assertion failed: computed local was %.5f, local from nn counts was %.5f\n",
t, newLocals[t], newLocalValue);
}
assertEquals(newLocalValue, newLocals[t], 0.00000001);
}
// And check that the two methods are equivalent, after correcting for the different K biases:
// CAN'T EXPECT THIS TO WORK though, because different neighbour counts (by one in each case) puts
// different inputs to the digamma functions, and we have no way to check that for the average here;
// Our above checks on the counts for each point suffice though.
// assertEquals(miAverage, averageFromNewObservations - MathsUtils.digamma(kNNs[ki]+1) + MathsUtils.digamma(kNNs[ki]),
// 0.00000001);
}
}
/**
* Test the computed univariate TE as a conditional MI
* using various numbers of threads.
@ -502,4 +609,133 @@ public class ConditionalMutualInfoMultiVariateTester
kNNs, expectedValue);
NUM_THREADS_TO_USE = NUM_THREADS_TO_USE_DEFAULT;
}
/**
* Test the conditional MI without any conditionals
* against the MI calculated by Kraskov's own MILCA
* tool on the same data.
*
* To run Kraskov's tool (http://www.klab.caltech.edu/~kraskov/MILCA/) for this
* data, run:
* ./MIxnyn <dataFile> 1 1 3000 <kNearestNeighbours> 0
*
* @throws Exception if file not found
*
*/
public void testNoConditionalsIsMIforRandomVariablesFromFile() throws Exception {
// Test set 1:
ArrayFileReader afr = new ArrayFileReader("demos/data/2randomCols-1.txt");
double[][] data = afr.getDouble2DMatrix();
// Use various Kraskov k nearest neighbours parameter
int[] kNNs = {1, 2, 3, 4, 5, 6, 10, 15};
// Expected values from Kraskov's MILCA toolkit:
double[] expectedFromMILCA = {-0.05294175, -0.03944338, -0.02190217,
0.00120807, -0.00924771, -0.00316402, -0.00778205, -0.00565778};
System.out.println("Kraskov comparison 1 - univariate random data 1 --");
// Check for algorithm 1 first (no expected results from MILCA included):
System.out.println("Algorithm 1:");
checkMIForGivenData(MatrixUtils.selectColumns(data, new int[] {0}),
MatrixUtils.selectColumns(data, new int[] {1}),
kNNs, null);
// then check for algorithm 2 with expected results from MILCA
System.out.println("Algorithm 2:");
checkMIForGivenData(MatrixUtils.selectColumns(data, new int[] {0}),
MatrixUtils.selectColumns(data, new int[] {1}),
kNNs, expectedFromMILCA);
//------------------
// Test set 2:
// We'll just take the first two columns from this data set
afr = new ArrayFileReader("demos/data/4randomCols-1.txt");
data = afr.getDouble2DMatrix();
// Expected values from Kraskov's MILCA toolkit:
double[] expectedFromMILCA_2 = {-0.04614525, -0.00861460, -0.00164540,
-0.01130354, -0.01339670, -0.00964035, -0.00237072, -0.00096891};
System.out.println("Kraskov comparison 2 - univariate random data 2 --");
// Check for algorithm 1 first (no expected results from MILCA included):
System.out.println("Algorithm 1:");
checkMIForGivenData(MatrixUtils.selectColumns(data, new int[] {0}),
MatrixUtils.selectColumns(data, new int[] {1}),
kNNs, null);
// then check for algorithm 2 with expected results from MILCA
System.out.println("Algorithm 2:");
checkMIForGivenData(MatrixUtils.selectColumns(data, new int[] {0}),
MatrixUtils.selectColumns(data, new int[] {1}),
kNNs, expectedFromMILCA_2);
}
/**
* Utility function to run Kraskov MI for data with known results
*
* @param var1
* @param var2
* @param kNNs array of Kraskov k nearest neighbours parameter to check
* @param expectedResults array of expected results for each k
*/
protected void checkMIForGivenData(double[][] var1, double[][] var2,
int[] kNNs, double[] expectedResults) throws Exception {
// The Kraskov MILCA toolkit MIhigherdim executable
// uses algorithm 2 by default (this is what it means by rectangular);
// These are the only ones we have expected results for
ConditionalMutualInfoCalculatorMultiVariateKraskov condMiCalc = getNewCalc(expectedResults == null ? 1 : 2);
for (int kIndex = 0; kIndex < kNNs.length; kIndex++) {
int k = kNNs[kIndex];
condMiCalc.setProperty(
MutualInfoCalculatorMultiVariateKraskov.PROP_K,
Integer.toString(k));
condMiCalc.setProperty(
MutualInfoCalculatorMultiVariateKraskov.PROP_NUM_THREADS,
NUM_THREADS_TO_USE);
// No longer need to set this property as it's set by default:
//miCalc.setProperty(MutualInfoCalculatorMultiVariateKraskov.PROP_NORM_TYPE,
// EuclideanUtils.NORM_MAX_NORM_STRING);
condMiCalc.setProperty(MutualInfoCalculatorMultiVariateKraskov.PROP_ADD_NOISE, "0"); // Need consistency for unit tests
condMiCalc.initialise(var1[0].length, var2[0].length, 0);
condMiCalc.setObservations(var1, var2, null);
// condMiCalc.setDebug(true);
double mi = condMiCalc.computeAverageLocalOfObservations();
// condMiCalc.setDebug(false);
// And compute also passing in an array of vectors of length 0
condMiCalc.initialise(var1[0].length, var2[0].length, 0);
condMiCalc.setObservations(var1, var2, new double[var1.length][]);
// condMiCalc.setDebug(true);
double miFromEmptyVectors = condMiCalc.computeAverageLocalOfObservations();
double expectedMi = 0.0;
if (expectedResults == null) {
// We don't have expected results from MILCA toolkit for algorithm 1.
// In this case, we'll generate our own expectations:
MutualInfoCalculatorMultiVariateKraskov1 miCalc = new MutualInfoCalculatorMultiVariateKraskov1();
miCalc.setProperty(
MutualInfoCalculatorMultiVariateKraskov.PROP_K,
Integer.toString(k));
miCalc.setProperty(
MutualInfoCalculatorMultiVariateKraskov.PROP_NUM_THREADS,
NUM_THREADS_TO_USE);
miCalc.setProperty(MutualInfoCalculatorMultiVariateKraskov.PROP_ADD_NOISE, "0"); // Need consistency for unit tests
miCalc.initialise(var1[0].length, var2[0].length);
miCalc.setObservations(var1, var2);
expectedMi = miCalc.computeAverageLocalOfObservations();
} else {
expectedMi = expectedResults[kIndex];
}
System.out.printf("k=%d: Average MI %.8f (expected %.8f)\n",
k, mi, expectedMi);
// Dropping required accuracy by one order of magnitude, due
// to faster but slightly less accurate digamma estimator change
assertEquals(expectedMi, mi, 0.0000001);
assertEquals(expectedMi, miFromEmptyVectors, 0.0000001);
}
}
}

37
java/unittests/infodynamics/measures/continuous/kraskov/TransferEntropyMultiVariateTester.java
View File

@ -20,6 +20,7 @@ package infodynamics.measures.continuous.kraskov;
import infodynamics.utils.ArrayFileReader;
import infodynamics.utils.MatrixUtils;
import infodynamics.utils.RandomGenerator;
public class TransferEntropyMultiVariateTester
extends infodynamics.measures.continuous.TransferEntropyMultiVariateAbstractTester {
@ -441,4 +442,40 @@ public class TransferEntropyMultiVariateTester
kNNs, expectedValue);
NUM_THREADS_TO_USE = NUM_THREADS_TO_USE_DEFAULT;
}
public void testMulitvariateAddObservations() throws Exception {
// Just make sure the code runs first (we had an execution error earlier)
TransferEntropyCalculatorMultiVariateKraskov teCalc =
new TransferEntropyCalculatorMultiVariateKraskov();
teCalc.setProperty("k", "4");
teCalc.initialise(1,3,3);
teCalc.startAddObservations();
RandomGenerator rg = new RandomGenerator();
for (int i = 0; i < 5; i++) {
double[][] source = rg.generateNormalData(100, 3, 0, 1);
double[][] target = rg.generateNormalData(100, 3, 0, 1);
teCalc.addObservations(source, target);
}
teCalc.finaliseAddObservations();
teCalc.computeAverageLocalOfObservations();
// Now make sure the ensemble method returns the same value as for
// a single time series
teCalc.initialise(1,3,3);
double[][] source = rg.generateNormalData(100, 3, 0, 1);
double[][] target = rg.generateNormalData(100, 3, 0, 1);
teCalc.setObservations(source, target);
int numObervationsSingle = teCalc.getNumObservations();
double resultSingle = teCalc.computeAverageLocalOfObservations();
teCalc.initialise(1,3,3);
teCalc.startAddObservations();
teCalc.addObservations(source, target, 0, 50); // Give first 50 time steps
teCalc.addObservations(source, target, 49, 51); // Give next 50 with
// one previous as the embedded history
teCalc.finaliseAddObservations();
int numObervationsEnsemble = teCalc.getNumObservations();
double resultEnsemble = teCalc.computeAverageLocalOfObservations();
assertEquals(numObervationsSingle, numObervationsEnsemble);
assertEquals(resultSingle, resultEnsemble, 0.00001);
}
}

66
java/unittests/infodynamics/measures/continuous/kraskov/TransferEntropyTester.java
View File

@ -537,4 +537,70 @@ public class TransferEntropyTester
assertEquals(teOptimisedSingleThread, teOptimisedWithValidity, 0.00000001);
System.out.println("Answer unchanged by setting validity");
}
public void testGetSeparateNumObservations() throws Exception {
ArrayFileReader afr = new ArrayFileReader("demos/data/SFI-heartRate_breathVol_bloodOx.txt");
double[][] data = afr.getDouble2DMatrix();
TransferEntropyCalculatorKraskov teCalc = new TransferEntropyCalculatorKraskov();
teCalc.initialise();
teCalc.startAddObservations();
int timeStepsPerCall = 100;
int calls = 10;
for (int i = 0; i < calls; i++) {
// Add more samples
teCalc.addObservations(MatrixUtils.selectColumn(data, 0, i*timeStepsPerCall, timeStepsPerCall),
MatrixUtils.selectColumn(data, 1, i*timeStepsPerCall, timeStepsPerCall));
}
teCalc.finaliseAddObservations();
@SuppressWarnings("unused")
double result = teCalc.computeAverageLocalOfObservations();
// Now we want to check how many observations were added at each call:
int[] samplesPerCall = teCalc.getSeparateNumObservations();
assertEquals(calls, samplesPerCall.length);
for (int i = 0; i < calls; i++) {
// For k = l = 1, we should have timeStepsPerCall - 1 samples per addObservations() call:
assertEquals(timeStepsPerCall - 1, samplesPerCall[i]);
}
// =====================
// Now run it again with different k and l and embedding lags, etc:
teCalc.initialise();
teCalc.startAddObservations();
// auto embed destination only
teCalc.setProperty(TransferEntropyCalculatorKraskov.PROP_AUTO_EMBED_METHOD,
TransferEntropyCalculatorKraskov.AUTO_EMBED_METHOD_RAGWITZ_DEST_ONLY);
teCalc.setProperty(TransferEntropyCalculatorKraskov.PROP_K_SEARCH_MAX, "5");
teCalc.setProperty(TransferEntropyCalculatorKraskov.PROP_TAU_SEARCH_MAX, "5");
// Explicitly set the source embedding params
teCalc.setProperty(TransferEntropyCalculatorKraskov.L_PROP_NAME, "1");
teCalc.setProperty(TransferEntropyCalculatorKraskov.L_TAU_PROP_NAME, "1");
int[] timeStepsPerCallArray = new int[] {100, 200, 150, 300, 99, 54};
int startTime = 0;
for (int i = 0; i < timeStepsPerCallArray.length; i++) {
// Add more samples
teCalc.addObservations(MatrixUtils.selectColumn(data, 0, startTime, timeStepsPerCallArray[i]),
MatrixUtils.selectColumn(data, 1, startTime, timeStepsPerCallArray[i]));
startTime += timeStepsPerCallArray[i];
}
teCalc.finaliseAddObservations();
result = teCalc.computeAverageLocalOfObservations();
int optimisedK = Integer.parseInt(teCalc.getProperty(TransferEntropyCalculatorKraskov.K_PROP_NAME));
int optimisedKTau = Integer.parseInt(teCalc.getProperty(TransferEntropyCalculatorKraskov.K_TAU_PROP_NAME));
int timeOfFirstObservationPerSet = (optimisedK - 1)*optimisedKTau + 1;
System.out.printf("In testing tracking of observations per addObservations() call" +
" we have auto-embedding dimension %d and lag %d, timeOfFirstObservationPerSet %d\n",
optimisedK, optimisedKTau, timeOfFirstObservationPerSet);
// Now we want to check how many observations were added at each call:
samplesPerCall = teCalc.getSeparateNumObservations();
assertEquals(timeStepsPerCallArray.length, samplesPerCall.length);
for (int i = 0; i < timeStepsPerCallArray.length; i++) {
// For timeOfFirstObservationPerSet, we should have timeStepsPerCall - timeOfFirstObservationPerSet
// samples per addObservations() call:
assertEquals(timeStepsPerCallArray[i] - timeOfFirstObservationPerSet, samplesPerCall[i]);
}
}
}

15
java/unittests/infodynamics/measures/discrete/MutualInformationTester.java
View File

@ -50,6 +50,21 @@ public class MutualInformationTester extends TestCase {
assertEquals(0.0, miRand, 0.000001);
}
public void testAnd() throws Exception {
MutualInformationCalculatorDiscrete miCalc = new MutualInformationCalculatorDiscrete(2, 0);
int[] X1 = new int[] {0, 0, 1, 1};
int[] X2 = new int[] {0, 1, 0, 1};
int[] Y = new int[] {0, 0, 0, 1};
// Y is dependent on X1 - MI should be 0.311 bits
miCalc.initialise();
miCalc.addObservations(X1, Y);
double miX1Y = miCalc.computeAverageLocalOfObservations();
assertEquals(0.311, miX1Y, 0.001);
assertEquals(4, miCalc.getNumObservations());
}
public void testXor() throws Exception {
MutualInformationCalculatorDiscrete miCalc = new MutualInformationCalculatorDiscrete(2, 0);

479
java/unittests/infodynamics/measures/discrete/TransferEntropyTester.java
View File

@ -18,6 +18,12 @@
package infodynamics.measures.discrete;
import java.util.Arrays;
import java.util.Random;
import infodynamics.utils.AnalyticMeasurementDistribution;
import infodynamics.utils.EmpiricalMeasurementDistribution;
import infodynamics.utils.MatrixUtils;
import infodynamics.utils.RandomGenerator;
import junit.framework.TestCase;
@ -84,4 +90,477 @@ public class TransferEntropyTester extends TestCase {
result = teCalc.computeAverageLocalOfObservations();
assertEquals(1, result, 0.0000001);
}
public void testEmbeddings() throws Exception {
// Use a short length so that if anything is mis-embedded we'll
// definitely see a change in the result
int length = 50;
for (int type = 0; type < NUM_ORDINARY_TESTS; type++) {
System.out.println("Testing operation " + type + " for TE discrete embeddings...");
for (int delay = 1; delay < 4; delay++) {
for (int selfLag = 1; selfLag < 4; selfLag++) {
for (int k = 1; k < 4; k++) {
for (int l = 1; l < 4; l++) {
for (int k_tau = 1; k_tau < 4; k_tau++) {
for (int l_tau = 1; l_tau < 4; l_tau++) {
for (int cDelay = 1; cDelay < 4; cDelay++) {
for (int startOffset = 0; startOffset < 10; startOffset += 3) {
int[][] data = generateData(length, type, delay, selfLag);
int[] source = data[0];
int[] dest = data[1];
TransferEntropyCalculatorDiscrete teCalc =
new TransferEntropyCalculatorDiscrete(2, k, k_tau, l, l_tau, cDelay);
teCalc.initialise();
teCalc.addObservations(source, dest, startOffset, dest.length - 1);
double result = teCalc.computeAverageLocalOfObservations();
// System.out.printf("Result = %.4f\n", result);
// Now check that against our own embeddings with conditional MI:
int startTimeBasedOnDestPast = (k-1)*k_tau + 1;
int startTimeBasedOnSourcePast = (l-1)*l_tau + cDelay;
int startObservationTime = Math.max(startTimeBasedOnDestPast, startTimeBasedOnSourcePast) + startOffset;
int[][] destPastVectors =
MatrixUtils.makeDelayEmbeddingVector(dest, k, k_tau,
startObservationTime-1,
dest.length - startObservationTime);
int[] destPastCombined = MatrixUtils.computeCombinedValues(destPastVectors, 2);
int[][] sourcePastVectors =
MatrixUtils.makeDelayEmbeddingVector(source, l, l_tau,
startObservationTime - cDelay,
source.length - startObservationTime);
int[] sourcePastCombined = MatrixUtils.computeCombinedValues(sourcePastVectors, 2);
int[] destNext = MatrixUtils.select(dest, startObservationTime, dest.length - startObservationTime);
ConditionalMutualInformationCalculatorDiscrete cmiCalc =
new ConditionalMutualInformationCalculatorDiscrete((int) Math.pow(2, l), 2, (int) Math.pow(2, k));
cmiCalc.initialise();
cmiCalc.addObservations(sourcePastCombined, destNext, destPastCombined);
double resultCmi = cmiCalc.computeAverageLocalOfObservations();
assertEquals(resultCmi, result, 0.000001);
}
}
}
}
}
}
}
}
}
}
public final static int TYPE_COPY = 0;
public final static int TYPE_OR = 1;
public final static int TYPE_AND = 2;
public final static int TYPE_XOR = 3;
public final static int TYPE_NOISY_XOR = 4;
public final static int NUM_ORDINARY_TESTS = 5;
public int[][] generateData(int length, int type, int delay, int selfLag) {
int[] source = rand.generateRandomInts(length, 2);
int[] dest = new int[length];
int startTime = Math.max(delay, selfLag);
Random rng = new Random();
for (int t = 0; t < startTime; t++) {
dest[t] = rng.nextInt(2);
}
for (int t = startTime; t < length; t++) {
switch(type){
case TYPE_COPY:
dest[t] = source[t-delay];
break;
case TYPE_OR:
dest[t] = source[t-delay] | dest[t-selfLag];
break;
case TYPE_AND:
dest[t] = source[t-delay] & dest[t-selfLag];
break;
case TYPE_XOR:
dest[t] = (source[t-delay] ^ dest[t-selfLag]) & 1; // Mask out the upper bits
break;
case TYPE_NOISY_XOR:
dest[t] = (source[t-delay] ^ dest[t-selfLag]) & 1; // Mask out the upper bits
if (rng.nextInt(4) == 1) {
// 1 in 4 chance we flip the output:
dest[t] = (dest[t] ^ 1) & 1; // Mask out the upper bits
}
break;
}
}
int[][] retVal = new int[2][];
retVal[0] = source;
retVal[1] = dest;
return retVal;
}
public void testValidObservations() {
int length = 100;
int[][] data = generateData(100, 1, 1, 1);
int[] source = data[0];
int[] dest = data[1];
TransferEntropyCalculatorDiscrete teCalc =
new TransferEntropyCalculatorDiscrete(2, 1, 1, 1, 1, 1);
teCalc.initialise();
boolean[] validity = new boolean[length];
boolean[] allValid = new boolean[length];
// 1. check that all samples are counted if validity is always true
Arrays.fill(allValid, true);
teCalc.addObservations(source, dest, allValid);
assertEquals(length - 1, teCalc.getNumObservations());
// 2. check that only one samples is left out if first or last entry is invalid:
Arrays.fill(validity, true);
validity[0] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, 1, 1, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - 2, teCalc.getNumObservations());
validity[0] = true;
validity[length - 1] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, 1, 1, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - 2, teCalc.getNumObservations());
// 3. check that only two samples are left out if an entry in middle invalid:
Arrays.fill(validity, true);
validity[10] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, 1, 1, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - 3, teCalc.getNumObservations());
// 4. check that only three samples are left out if two consecutive entries in middle invalid:
Arrays.fill(validity, true);
validity[10] = false;
validity[11] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, 1, 1, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - 4, teCalc.getNumObservations());
// 5. check that four samples are left out if two non-consecutive entries in middle invalid:
Arrays.fill(validity, true);
validity[10] = false;
validity[12] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, 1, 1, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - 5, teCalc.getNumObservations());
// 6. check that only k+1 extra samples are left out if an entry in middle invalid:
Arrays.fill(validity, true);
validity[50] = false;
for (int k = 1; k < 5; k++) {
for (int k_tau = 1; k_tau < 5; k_tau++) {
int startTimeBasedOnDestPast = (k-1)*k_tau + 1;
// First check the number of observations when all valid:
System.out.printf("Testing addObservations(valid) for k=%d, k_tau=%d\n", k, k_tau);
teCalc = new TransferEntropyCalculatorDiscrete(2, k, k_tau, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, allValid);
assertEquals(length - startTimeBasedOnDestPast, teCalc.getNumObservations());
// Now check with an entry left out in the middle:
teCalc = new TransferEntropyCalculatorDiscrete(2, k, k_tau, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - startTimeBasedOnDestPast - ((k-1)*k_tau+2), teCalc.getNumObservations());
}
}
// 7. check correct number of extra samples left out
// for two nearby entries in middle being invalid:
Arrays.fill(validity, true);
int invalidtime = 50;
validity[invalidtime] = false;
for (int k = 1; k < 5; k++) {
for (int k_tau = 1; k_tau < 5; k_tau++) {
int startTimeBasedOnDestPast = (k-1)*k_tau + 1;
for (int t = 1; t < 20; t++) {
validity[invalidtime+t] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, k, k_tau, 1, 1, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
int expectedInvalidTimes = 0;
if (t > startTimeBasedOnDestPast) {
// Both invalid entries cause independent invalid periods
expectedInvalidTimes = (startTimeBasedOnDestPast + 1)*2;
} else {
// Their invalid times overlap
expectedInvalidTimes = (startTimeBasedOnDestPast + 1) + t;
}
assertEquals(length - startTimeBasedOnDestPast - expectedInvalidTimes,
teCalc.getNumObservations());
validity[invalidtime+t] = true;
}
}
}
// 8. check correct number of extra samples left out
// for two nearby entries in middle being invalid,
// but using a long source embedding
Arrays.fill(validity, true);
validity[invalidtime] = false;
for (int l = 1; l < 5; l++) {
for (int l_tau = 1; l_tau < 5; l_tau++) {
int startTimeBasedOnDestPast = (l-1)*l_tau + 1;
for (int t = 1; t < 20; t++) {
validity[invalidtime+t] = false;
teCalc = new TransferEntropyCalculatorDiscrete(2, 1, 1, l, l_tau, 1);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
int expectedInvalidTimes = 0;
if (t > startTimeBasedOnDestPast) {
// Both invalid entries cause independent invalid periods
// Need the +1 here because when the invalid is next state
// it still adds 1 in addition to the source
// invalidities!
expectedInvalidTimes = (startTimeBasedOnDestPast + 1)*2;
} else {
// Their invalid times overlap
expectedInvalidTimes = (startTimeBasedOnDestPast + 1) + t;
}
assertEquals(length - startTimeBasedOnDestPast - expectedInvalidTimes,
teCalc.getNumObservations());
validity[invalidtime+t] = true;
}
}
}
// 9. check correct number of extra samples left out
// for two nearby entries in middle being invalid,
// but using a long source embedding AND a source-dest delay
for (int l = 1; l < 5; l++) {
for (int l_tau = 1; l_tau < 5; l_tau++) {
for (int k = 1; k < 5; k++) {
for (int k_tau = 1; k_tau < 5; k_tau++) {
for (int delay = 1; delay < 5; delay++) {
int startTimeBasedOnDestPast = (k-1)*k_tau + 1;
int startTimeBasedOnSourcePast = (l-1)*l_tau + delay;
int startObservationTime = Math.max(startTimeBasedOnDestPast, startTimeBasedOnSourcePast);
int numPointsRemoved = 0;
Arrays.fill(validity, true);
validity[startObservationTime+2] = false;
boolean overlap = false;
if (startTimeBasedOnDestPast >= delay) {
// The embeddings overlap, so number of points removed
// is solely based on startObservationTime
numPointsRemoved = (startObservationTime + 1);
overlap = true;
} else {
// The embeddings do not overlap, so will contribute
// to point removal separately
numPointsRemoved = (l-1)*l_tau + 1 + startTimeBasedOnDestPast + 1;
overlap = false;
}
System.out.printf("k=%d, k_tau=%d, l=%d, l_tau=%d, delay=%d, overlap=%b; expecting removal of %d points\n",
k, k_tau, l, l_tau, delay, overlap, numPointsRemoved);
teCalc = new TransferEntropyCalculatorDiscrete(2, k, k_tau, l, l_tau, delay);
teCalc.initialise();
teCalc.addObservations(source, dest, validity);
assertEquals(length - startObservationTime - numPointsRemoved,
teCalc.getNumObservations());
}
}
}
}
}
}
public void testSignificance() throws Exception {
int length = 1000;
int[][] data = generateData(length, 2, 2, 3);
int[] source = data[0];
int[] dest = data[1];
TransferEntropyCalculatorDiscrete teCalc =
new TransferEntropyCalculatorDiscrete(2, 2, 2, 2, 2, 2);
teCalc.initialise();
teCalc.addObservations(source, dest);
double originalResult = teCalc.computeAverageLocalOfObservations();
// Just make sure the significance check returns the right number of calculations, etc
int numSurrogates = 151;
EmpiricalMeasurementDistribution emd = teCalc.computeSignificance(numSurrogates);
assertEquals(numSurrogates, emd.distribution.length);
// And now make an analytic calculation:
@SuppressWarnings("unused")
AnalyticMeasurementDistribution amd = teCalc.computeSignificance();
// Not really anything I can check here, just want to make sure it runs ok
double resultAfterSigTest = teCalc.computeAverageLocalOfObservations();
assertEquals(originalResult, resultAfterSigTest, 0.000001);
}
public void testLocals() throws Exception {
int length = 500;
int[][] data2D = new int[length][2];
int[][][] data3D = new int[length][2][2];
for (int type = 0; type < NUM_ORDINARY_TESTS; type++) {
System.out.println("Testing operation " + type + " for TE discrete embeddings with locals...");
for (int delay = 1; delay < 4; delay++) {
for (int selfLag = 1; selfLag < 4; selfLag++) {
for (int k = 1; k < 4; k++) {
for (int l = 1; l < 4; l++) {
for (int k_tau = 1; k_tau < 4; k_tau++) {
for (int l_tau = 1; l_tau < 4; l_tau++) {
for (int cDelay = 1; cDelay < 4; cDelay++) {
int[][] data = generateData(length, type, delay, selfLag);
int[] source = data[0];
int[] dest = data[1];
TransferEntropyCalculatorDiscrete teCalc =
new TransferEntropyCalculatorDiscrete(2, k, k_tau, l, l_tau, cDelay);
teCalc.initialise();
teCalc.addObservations(source, dest);
double result = teCalc.computeAverageLocalOfObservations();
double[] locals = teCalc.computeLocalFromPreviousObservations(source, dest);
// Now check that the average of the locals gives us the correct value
int startTimeBasedOnDestPast = (k-1)*k_tau + 1;
int startTimeBasedOnSourcePast = (l-1)*l_tau + cDelay;
int startObservationTime = Math.max(startTimeBasedOnDestPast, startTimeBasedOnSourcePast);
double sumLocals = MatrixUtils.sum(locals);
double avLocals = sumLocals / (double) (length - startObservationTime);
// System.out.printf("Testing locals for k=%d, k_tau=%d, l=%d, l_tau=%d, delay=%d, expecting no locals for first %d points\n",
// k, k_tau, l, l_tau, delay, startObservationTime);
assertEquals(length - startObservationTime, teCalc.getNumObservations());
for (int t = 0; t < startObservationTime; t++) {
// First startObservationTime time steps should have
// local value of 0
assertEquals(0, locals[t], 0.000001);
}
assertEquals(result, avLocals, 0.000001);
// Now test for the 2D methods:
teCalc.initialise();
MatrixUtils.copyIntoColumn(data2D, 0, source);
MatrixUtils.copyIntoColumn(data2D, 1, dest);
// Make sure we only calculate source -> dest:
teCalc.setPeriodicBoundaryConditions(false);
teCalc.addObservations(data2D, 1);
double result2D = teCalc.computeAverageLocalOfObservations();
assertEquals(result, result2D, 0.00001);
int numObs = teCalc.getNumObservations();
// Now compute across both columns:
teCalc.initialise();
teCalc.setPeriodicBoundaryConditions(true);
teCalc.addObservations(data2D, 1);
assertEquals(numObs*2, teCalc.getNumObservations());
double result2DBoth = teCalc.computeAverageLocalOfObservations();
double[][] locals2D = teCalc.computeLocalFromPreviousObservations(data2D, 1);
sumLocals = MatrixUtils.sum(locals2D);
avLocals = sumLocals / (double) (teCalc.getNumObservations());
assertEquals(result2DBoth, avLocals, 0.000001);
// Now compute for one source and dest only:
teCalc.initialise();
teCalc.addObservations(data2D, 0, 1);
double resultAcrossCols = teCalc.computeAverageLocalOfObservations();
assertEquals(result, resultAcrossCols, 0.0000001);
// Now test the 3D methods
teCalc.initialise();
MatrixUtils.copyIntoColumn3D(data3D, 0, 0, source);
MatrixUtils.copyIntoColumn3D(data3D, 1, 1, dest);
// Make sure we only calculate source -> dest:
teCalc.setPeriodicBoundaryConditions(false);
teCalc.addObservations(data3D, 1, 1);
double result3D = teCalc.computeAverageLocalOfObservations();
assertEquals(result, result3D, 0.00001);
numObs = teCalc.getNumObservations();
// Now compute across all columns:
teCalc.initialise();
teCalc.setPeriodicBoundaryConditions(true);
teCalc.addObservations(data3D, 1, 1);
assertEquals(numObs*4, teCalc.getNumObservations());
double result3DAll = teCalc.computeAverageLocalOfObservations();
double[][][] locals3D = teCalc.computeLocalFromPreviousObservations(data3D, 1, 1);
sumLocals = MatrixUtils.sum(locals3D);
avLocals = sumLocals / (double) (teCalc.getNumObservations());
assertEquals(result3DAll, avLocals, 0.000001);
// Now compute for one source and dest only:
teCalc.initialise();
teCalc.addObservations(data3D, 0, 0, 1, 1);
resultAcrossCols = teCalc.computeAverageLocalOfObservations();
assertEquals(result, resultAcrossCols, 0.0000001);
}
}
}
}
}
}
}
}
}
public int[][] generateParityData(int length, int delay, int k, int l) {
int[][] data = new int[length][2];
int k_tau = 1;
int l_tau = 1;
int startTimeBasedOnDestPast = (k-1)*k_tau + 1;
int startTimeBasedOnSourcePast = (l-1)*l_tau + delay;
int startObservationTime = Math.max(startTimeBasedOnDestPast, startTimeBasedOnSourcePast);
Random random = new Random();
for (int t = 0; t < startObservationTime; t++) {
data[t][0] = random.nextInt(2);
data[t][1] = random.nextInt(2);
}
for (int t = startObservationTime; t < length; t++) {
data[t][1] = 0;
for (int ki = 0; ki < k; ki++) {
data[t][1] ^= data[t-1-ki][1];
}
for (int li = 0; li < l; li++) {
data[t][1] ^= data[t-delay-li][0];
}
data[t][0] = random.nextInt(2);
}
return data;
}
public void testComplexParity() throws Exception {
int length = 10000;
for (int delay = 1; delay < 4; delay++) {
for (int k_dest = 1; k_dest < 4; k_dest++) {
for (int l_source = 1; l_source < 4; l_source++) {
for (int k = 1; k < 4; k++) {
for (int l = 1; l < 4; l++) {
for (int k_tau = 1; k_tau < 4; k_tau++) {
for (int l_tau = 1; l_tau < 4; l_tau++) {
for (int cDelay = 1; cDelay < 4; cDelay++) {
int[][] data = generateParityData(length, delay, k_dest, l_source);
int[] source = MatrixUtils.selectColumn(data, 0);
int[] dest = MatrixUtils.selectColumn(data, 1);
TransferEntropyCalculatorDiscrete teCalc =
new TransferEntropyCalculatorDiscrete(2, k, k_tau, l, l_tau, cDelay);
teCalc.initialise();
teCalc.addObservations(source, dest);
double result = teCalc.computeAverageLocalOfObservations();
if ((k >= k_dest) && (cDelay <= delay) &&
(l - delay + cDelay >= l_source) &&
((k_dest==1) || (k_tau == 1)) &&
(((l_source==1) && (cDelay==delay)) || (l_tau == 1))) {
// Our embedding parameters will be satisfactory
// to detect the parity operation
if (result <= 0.9) {
System.out.printf("(k=%d >= k_dest=%d) && (cDelay=%d <= delay=%d) && " +
"(l=%d - delay=%d + cDelay=%d >= l_source=%d) && ((k_dest=%d==1) || (k_tau=%d == 1)) && " +
"(((l_source=%d==1)&&(cDelay=%d==delay=%d)) || (l_tau=%d == 1)) was true but result=%.3f too low\n",
k, k_dest, cDelay, delay, l, delay, cDelay, l_source, k_dest, k_tau, l_source, cDelay, delay, l_tau, result);
}
assertTrue(result > 0.9);
} else {
// It's still quite possible that the parity operation is detected, because
// we can have some strange behaviour emerge here, e.g. initial conditions lock in
// dest behaviour as either a copy or inversion of source (in which case you always detect
// the one bit dependence with k=1).
// So instead, let's look for conditions where we know it *cannot* be detected!
if (cDelay > delay) {
if (result > 0.1) {
System.out.printf("(cDelay=%d > delay=%d) was false but result=%.3f too high\n",
cDelay, delay, result);
}
assertTrue(result < 0.1);
}
}
}
}
}
}
}
}
}
}
}
}

288
java/unittests/infodynamics/utils/KdTreeTest.java
View File

@ -502,6 +502,48 @@ public class KdTreeTest extends TestCase {
long endTimeValidate = Calendar.getInstance().getTimeInMillis();
System.out.printf("All %d nearest neighbours found in: %.3f sec\n",
K, ((double) (endTimeValidate - startTime)/1000.0));
// And also test the K nearest neighbours for a few points that
// weren't in the original data set:
for (int n = 0; n < 100; n++) {
// Generate a sample vector
double[][] sampleVectors = new double[variables][];
for (int v = 0; v < variables; v++) {
int vLength = data[v][0].length;
sampleVectors[v] = rg.generateNormalData(vLength, 0, 1);
}
// Now pull the kNNs for it:
PriorityQueue<NeighbourNodeData> nnPQ =
kdTree.findKNearestNeighbours(K, sampleVectors);
assertTrue(nnPQ.size() == K);
// Now find the K nearest neighbours with a naive all-pairs comparison
double[][] distancesAndIndices = new double[samples][2];
for (int t2 = 0; t2 < samples; t2++) {
double maxNorm = 0;
for (int v = 0; v < variables; v++) {
double normForThisVariable = normCalculator.norm(
sampleVectors[v], data[v][t2]);
if (maxNorm < normForThisVariable) {
maxNorm = normForThisVariable;
}
}
distancesAndIndices[t2][0] = maxNorm;
distancesAndIndices[t2][1] = t2;
}
int[] timeStepsOfKthMins =
MatrixUtils.kMinIndices(distancesAndIndices, 0, K);
for (int i = 0; i < K; i++) {
// Check that the ith nearest neighbour matches
NeighbourNodeData nnData = nnPQ.poll();
if (timeStepsOfKthMins[K - 1 - i] != nnData.sampleIndex) {
// We have an error:
System.out.printf("Erroneous match between indices %d (expected) " +
" and %d\n", timeStepsOfKthMins[K - 1 - i], nnData.sampleIndex);
}
assertEquals(timeStepsOfKthMins[K - 1 - i], nnData.sampleIndex);
}
}
}
public void testCountNeighboursWithinRSeparateArrays() {
@ -534,6 +576,16 @@ public class KdTreeTest extends TestCase {
data, samples, kdTree, rs[i], true);
verifyCountNeighboursWithinRForSeparateArraysArrayArgs(EuclideanUtils.NORM_MAX_NORM,
data, samples, kdTree, rs[i], false);
// And test the array return variant for radii returned:
verifyCountNeighboursWithinRForSeparateArraysArrayArgsPlusRadiusOut(EuclideanUtils.NORM_MAX_NORM,
data, samples, kdTree, rs[i], true);
verifyCountNeighboursWithinRForSeparateArraysArrayArgsPlusRadiusOut(EuclideanUtils.NORM_MAX_NORM,
data, samples, kdTree, rs[i], false);
// And test the array return variant for new data points too:
verifyCountNeighboursWithinRForSeparateArraysArrayArgsNewPoints(EuclideanUtils.NORM_MAX_NORM,
data, samples, kdTree, rs[i], true);
verifyCountNeighboursWithinRForSeparateArraysArrayArgsNewPoints(EuclideanUtils.NORM_MAX_NORM,
data, samples, kdTree, rs[i], false);
// Must test with EuclideanUtils.NORM_EUCLIDEAN_SQUARED instead
// of EuclideanUtils.NORM_EUCLIDEAN if we want the min distances
// to match when tested inside verifyNearestNeighbourForSeparateArrays()
@ -545,6 +597,14 @@ public class KdTreeTest extends TestCase {
data, samples, kdTree, rs[i], true);
verifyCountNeighboursWithinRForSeparateArraysArrayArgs(EuclideanUtils.NORM_EUCLIDEAN_SQUARED,
data, samples, kdTree, rs[i], false);
verifyCountNeighboursWithinRForSeparateArraysArrayArgsPlusRadiusOut(EuclideanUtils.NORM_EUCLIDEAN_SQUARED,
data, samples, kdTree, rs[i], true);
verifyCountNeighboursWithinRForSeparateArraysArrayArgsPlusRadiusOut(EuclideanUtils.NORM_EUCLIDEAN_SQUARED,
data, samples, kdTree, rs[i], false);
verifyCountNeighboursWithinRForSeparateArraysArrayArgsNewPoints(EuclideanUtils.NORM_EUCLIDEAN_SQUARED,
data, samples, kdTree, rs[i], true);
verifyCountNeighboursWithinRForSeparateArraysArrayArgsNewPoints(EuclideanUtils.NORM_EUCLIDEAN_SQUARED,
data, samples, kdTree, rs[i], false);
}
}
@ -694,6 +754,198 @@ public class KdTreeTest extends TestCase {
((double) (endTimeValidate - endTimeCount)/1000.0));
}
private void verifyCountNeighboursWithinRForSeparateArraysArrayArgsPlusRadiusOut(int normType,
double[][][] data, int samples, KdTree kdTree, double r,
boolean allowEqualToR) {
int variables = data.length;
// Set up the given norm type:
EuclideanUtils normCalculator = new EuclideanUtils(normType);
kdTree.setNormType(normType);
long startTime = Calendar.getInstance().getTimeInMillis();
int totalCount = 0;
int[] counts = new int[samples];
boolean[] withinR = new boolean[samples];
double[] distancesWithinRInOrder = new double[samples];
double[][] distancesAndIndicesWithinR = new double[samples][2];
for (int t = 0; t < samples; t++) {
// int count = kdTree.countPointsWithinR(t, r, allowEqualToR);
kdTree.findPointsWithinR(t, r, 0, allowEqualToR, withinR,
distancesWithinRInOrder, distancesAndIndicesWithinR);
int count = 0;
// Run through the list of points returned as within r and check them
for (int t2 = 0; distancesAndIndicesWithinR[t2][1] >= 0; t2++) {
count++;
assertTrue(withinR[(int) distancesAndIndicesWithinR[t2][1]]);
// Reset this marker
withinR[(int) distancesAndIndicesWithinR[t2][1]] = false;
// Check that it's really within r
double maxNorm = 0;
for (int v = 0; v < variables; v++) {
double normForThisVariable = normCalculator.norm(
data[v][t], data[v][(int) distancesAndIndicesWithinR[t2][1]]);
if (maxNorm < normForThisVariable) {
maxNorm = normForThisVariable;
}
}
assertTrue((!allowEqualToR && (maxNorm < r)) ||
( allowEqualToR && (maxNorm <= r)));
// Check that its distance was properly returned in the other arrays:
assertEquals(maxNorm, distancesAndIndicesWithinR[t2][0], 0.000001);
assertEquals(distancesAndIndicesWithinR[t2][0],
distancesWithinRInOrder[(int) distancesAndIndicesWithinR[t2][1]],
0.00000001);
}
// Check that there were no other points marked as within r:
int count2 = 0;
for (int t2 = 0; t2 < samples; t2++) {
if (withinR[t2]) {
count2++;
}
}
assertEquals(0, count2); // We set all the ones that should have been true to false already
// Now reset the boolean array
counts[t] = count;
totalCount += count;
}
long endTimeCount = Calendar.getInstance().getTimeInMillis();
System.out.printf("All neighbours within %.3f (average of %.3f) found in: %.3f sec\n",
r, (double) totalCount / (double) samples,
((double) (endTimeCount - startTime)/1000.0));
// Now find the neighbour count with a naive all-pairs comparison
for (int t = 0; t < samples; t++) {
int naiveCount = 0;
for (int t2 = 0; t2 < samples; t2++) {
double norm = Double.POSITIVE_INFINITY;
if (t2 != t) {
double maxNorm = 0;
for (int v = 0; v < variables; v++) {
double normForThisVariable = normCalculator.norm(
data[v][t], data[v][t2]);
if (maxNorm < normForThisVariable) {
maxNorm = normForThisVariable;
}
}
norm = maxNorm;
}
if ((!allowEqualToR && (norm < r)) ||
( allowEqualToR && (norm <= r))) {
naiveCount++;
}
}
if (naiveCount != counts[t]) {
System.out.printf("All pairs : count %d\n", naiveCount);
System.out.printf("kdTree search: count %d\n", counts[t]);
}
assertEquals(naiveCount, counts[t]);
}
long endTimeValidate = Calendar.getInstance().getTimeInMillis();
System.out.printf("All neighbours within %.3f (average of %.3f) validated in: %.3f sec\n",
r, (double) totalCount / (double) samples,
((double) (endTimeValidate - endTimeCount)/1000.0));
}
private void verifyCountNeighboursWithinRForSeparateArraysArrayArgsNewPoints(int normType,
double[][][] data, int samples, KdTree kdTree, double r,
boolean allowEqualToR) {
int variables = data.length;
int numNewSamplesToCheck = 100;
double[][][] allSampleVectors = new double[numNewSamplesToCheck][][];
// Set up the given norm type:
EuclideanUtils normCalculator = new EuclideanUtils(normType);
kdTree.setNormType(normType);
long startTime = Calendar.getInstance().getTimeInMillis();
int totalCount = 0;
int[] counts = new int[samples];
boolean[] withinR = new boolean[samples];
int[] indicesWithinR = new int[samples];
// Test the range search for a few points that
// weren't in the original data set:
for (int n = 0; n < numNewSamplesToCheck; n++) {
// Generate a sample vector
double[][] sampleVectors = new double[variables][];
for (int v = 0; v < variables; v++) {
int vLength = data[v][0].length;
sampleVectors[v] = rg.generateNormalData(vLength, 0, 1);
}
allSampleVectors[n] = sampleVectors;
kdTree.findPointsWithinR(r, sampleVectors, allowEqualToR, withinR, indicesWithinR);
int count = 0;
// Run through the list of points returned as within r and check them
for (int t2 = 0; indicesWithinR[t2] != -1; t2++) {
count++;
assertTrue(withinR[indicesWithinR[t2]]);
// Reset this marker
withinR[indicesWithinR[t2]] = false;
// Check that it's really within r
double maxNorm = 0;
for (int v = 0; v < variables; v++) {
double normForThisVariable = normCalculator.norm(
sampleVectors[v], data[v][indicesWithinR[t2]]);
if (maxNorm < normForThisVariable) {
maxNorm = normForThisVariable;
}
}
assertTrue((!allowEqualToR && (maxNorm < r)) ||
( allowEqualToR && (maxNorm <= r)));
}
// Check that there were no other points marked as within r:
int count2 = 0;
for (int t2 = 0; t2 < samples; t2++) {
if (withinR[t2]) {
count2++;
}
}
assertEquals(0, count2); // We set all the ones that should have been true to false already
// Now reset the boolean array
counts[n] = count;
totalCount += count;
}
long endTimeCount = Calendar.getInstance().getTimeInMillis();
System.out.printf("All neighbours within %.3f (average of %.3f) found in: %.3f sec\n",
r, (double) totalCount / (double) samples,
((double) (endTimeCount - startTime)/1000.0));
// Now find the neighbour count with a naive all-pairs comparison
for (int n = 0; n < numNewSamplesToCheck; n++) {
int naiveCount = 0;
for (int t2 = 0; t2 < samples; t2++) {
double norm = Double.POSITIVE_INFINITY;
double maxNorm = 0;
for (int v = 0; v < variables; v++) {
double normForThisVariable = normCalculator.norm(
allSampleVectors[n][v], data[v][t2]);
if (maxNorm < normForThisVariable) {
maxNorm = normForThisVariable;
}
}
norm = maxNorm;
if ((!allowEqualToR && (norm < r)) ||
( allowEqualToR && (norm <= r))) {
naiveCount++;
}
}
if (naiveCount != counts[n]) {
System.out.printf("All pairs : count %d\n", naiveCount);
System.out.printf("kdTree search: count %d\n", counts[n]);
}
assertEquals(naiveCount, counts[n]);
}
long endTimeValidate = Calendar.getInstance().getTimeInMillis();
System.out.printf("All neighbours within %.3f (average of %.3f) validated in: %.3f sec\n",
r, (double) totalCount / (double) samples,
((double) (endTimeValidate - endTimeCount)/1000.0));
}
public void testCountNeighboursWithinRExclusionWindow() {
int dimensionsPerVariable = 3;
int samples = 1000;
@ -1106,6 +1358,42 @@ public class KdTreeTest extends TestCase {
}
}
}
// And now test some fresh samples:
int numNewSamples = 1000;
for (int i = 0; i < r1.length; i++) {
for (int n = 0; n < numNewSamples; n++) {
double[][] newSample = rg.generateNormalData(variables, dimensionsPerVariable, 0, 1);
double[] variable1 = newSample[0];
double[][] newSampleSansVariable1 = new double[variables - 1][];
for (int v = 1; v < variables; v++) {
newSampleSansVariable1[v-1] = newSample[v];
}
// Establish our ground truth by searching all variables:
int count = kdTree.countPointsWithinR(newSample, r1[i], allowEqualsR);
// Now search in variable 1 first, then use that
// to help the others:
nnSearcherVar1.findPointsWithinR(r1[i], new double[][] {variable1}, allowEqualsR,
isWithinR, indicesWithinR);
int countUsingVar1 = kdTree.countPointsWithinR(r1[i], newSample,
allowEqualsR, 0, isWithinR);
assertEquals(count, countUsingVar1);
// And then search the kdTree which does not have variable 1:
int countAdditionalCriteria =
nnSearcherExceptVar1.countPointsWithinR(newSampleSansVariable1, r1[i],
allowEqualsR, isWithinR);
assertEquals(count, countAdditionalCriteria);
// Finally, reset our boolean array:
for (int t2 = 0; indicesWithinR[t2] != -1; t2++) {
isWithinR[indicesWithinR[t2]] = false;
}
}
}
}
public void testPreviouslyTestedVariableMethodMultipleRs() throws Exception {

247
java/unittests/infodynamics/utils/UnivariateNearestNeighbourTest.java
View File

@ -117,6 +117,21 @@ public class UnivariateNearestNeighbourTest extends TestCase {
checkRangeFinderArrayCalls(false, 100);
}
public void testRangeFinderArrayCallsRaidusOut() throws Exception {
checkRangeFinderArrayCallsPlusRadiusOut(true, 100);
checkRangeFinderArrayCallsPlusRadiusOut(false, 100);
}
public void testRangeFinderArrayCallsMaxRadiusOut() throws Exception {
checkRangeFinderArrayCallsPlusMaxRadiusOut(true, 100);
checkRangeFinderArrayCallsPlusMaxRadiusOut(false, 100);
}
public void testRangeFinderArrayCallsNewData() throws Exception {
checkRangeFinderArrayCallsNewData(true);
checkRangeFinderArrayCallsNewData(false);
}
public void checkRangeFinder(boolean strict, int exclWindow) throws Exception {
int numTimeStepsInitial = 10000;
int duplicateSteps = 100;
@ -242,6 +257,238 @@ public class UnivariateNearestNeighbourTest extends TestCase {
}
}
public void checkRangeFinderArrayCallsPlusRadiusOut(boolean strict, int exclWindow) throws Exception {
int numTimeStepsInitial = 10000;
int duplicateSteps = 100;
int numTimeSteps = numTimeStepsInitial+duplicateSteps;
double[] dataRaw = rg.generateNormalData(numTimeStepsInitial, 0, 1);
double[] data = new double[numTimeSteps];
// Now duplicate some of the time steps as a test:
System.arraycopy(dataRaw, 0, data, 0, numTimeStepsInitial);
System.arraycopy(dataRaw, 0, data, numTimeStepsInitial, duplicateSteps);
UnivariateNearestNeighbourSearcher searcher =
new UnivariateNearestNeighbourSearcher(data);
int[] counts = new int[data.length];
boolean[] withinR = new boolean[data.length];
double[] distancesWithinRInOrder = new double[data.length];
double[][] distancesAndIndicesWithinR = new double[data.length][2];
double r = 0.2;
long startTime = Calendar.getInstance().getTimeInMillis();
for (int t = 0; t < data.length; t++) {
searcher.findPointsWithinR(t, r, exclWindow, !strict,
withinR, distancesWithinRInOrder, distancesAndIndicesWithinR);
int count = 0;
// Run through the list of points returned as within r and check them
for (int t2 = 0; distancesAndIndicesWithinR[t2][1] >= 0; t2++) {
count++;
assertTrue(withinR[(int) distancesAndIndicesWithinR[t2][1]]);
// Reset this marker
withinR[(int) distancesAndIndicesWithinR[t2][1]] = false;
// Check that it's really within r
double maxNorm = Math.abs(data[t] - data[(int) distancesAndIndicesWithinR[t2][1]]);
assertTrue((strict && (maxNorm < r)) ||
( !strict && (maxNorm <= r)));
// Check that its distance was properly returned in the other arrays:
assertEquals(maxNorm, distancesAndIndicesWithinR[t2][0], 0.000001);
assertEquals(distancesAndIndicesWithinR[t2][0],
distancesWithinRInOrder[(int) distancesAndIndicesWithinR[t2][1]],
0.00000001);
}
// Check that there were no other points marked as within r in withinR:
int count2 = 0;
for (int t2 = 0; t2 < data.length; t2++) {
if (withinR[t2]) {
count2++;
}
}
assertEquals(0, count2); // We set all the ones that should have been true to false already
// Now reset the boolean array
counts[t] = count;
}
long endTimeNNs = Calendar.getInstance().getTimeInMillis();
System.out.printf("Found all neighbours within %.3f (mean = %.3f) in: %.3f sec\n",
r, MatrixUtils.mean(counts),
((double) (endTimeNNs - startTime)/1000.0));
// Now do brute force:
int[] bruteForceCounts = new int[data.length];
for (int t = 0; t < data.length; t++) {
int count = 0;
for (int t2 = 0; t2 < data.length; t2++) {
if (Math.abs(t2 - t) <= exclWindow) {
continue;
}
double norm = Math.abs(data[t] - data[t2]);
if ((strict && (norm < r) ) ||
(!strict && (norm <= r))) {
count++;
}
}
bruteForceCounts[t] = count;
}
long endTimeBruteForce = Calendar.getInstance().getTimeInMillis();
System.out.printf("Found all neighbours within %.3f (mean = %.3f) by brute force in: %.3f sec\n",
r, MatrixUtils.mean(bruteForceCounts),
((double) (endTimeBruteForce - endTimeNNs)/1000.0));
// Now validate the nearest neighbour counts:
for (int t = 0; t < data.length; t++) {
assertEquals(bruteForceCounts[t], counts[t]);
}
}
public void checkRangeFinderArrayCallsPlusMaxRadiusOut(boolean strict, int exclWindow) throws Exception {
int numTimeStepsInitial = 10000;
int duplicateSteps = 100;
int numTimeSteps = numTimeStepsInitial+duplicateSteps;
double[] dataRaw = rg.generateNormalData(numTimeStepsInitial, 0, 1);
double[] data2 = new double[numTimeSteps];
// Now duplicate some of the time steps as a test:
System.arraycopy(dataRaw, 0, data2, 0, numTimeStepsInitial);
System.arraycopy(dataRaw, 0, data2, numTimeStepsInitial, duplicateSteps);
// And generate a first set of data:
dataRaw = rg.generateNormalData(numTimeStepsInitial, 0, 1);
double[] data1 = new double[numTimeSteps];
// Now duplicate some of the time steps as a test:
System.arraycopy(dataRaw, 0, data1, 0, numTimeStepsInitial);
System.arraycopy(dataRaw, 0, data1, numTimeStepsInitial, duplicateSteps);
UnivariateNearestNeighbourSearcher searcher =
new UnivariateNearestNeighbourSearcher(data2);
boolean[] withinR = new boolean[data2.length];
double[] distancesForFirstVariable = new double[data1.length];
double[][] distancesAndIndicesWithinR = new double[data1.length][2];
double r = 0.2;
for (int t = 0; t < data2.length; t++) {
// Compute the distances for the first variable:
for (int t2 = 0; t2 < data1.length; t2++) {
if (Math.abs(t2 - t) <= exclWindow) {
// Within the exclusion window (always includes the point itself for exclWindow==0)
withinR[t2] = false;
distancesForFirstVariable[t2] = Double.POSITIVE_INFINITY;
continue;
}
distancesForFirstVariable[t2] = Math.abs(data1[t] - data1[t2]);
if ((strict && (distancesForFirstVariable[t2] < r)) ||
(!strict && (distancesForFirstVariable[t2] <= r))) {
withinR[t2] = true;
} else {
withinR[t2] = false;
}
}
// Now compute the max distances across both variables:
searcher.findPointsWithinRAndTakeRadiusMax(t, r, exclWindow, !strict,
withinR, distancesForFirstVariable, distancesAndIndicesWithinR);
// Now check all of the results -- first those said to be within r
for (int t2 = 0; distancesAndIndicesWithinR[t2][1] >= 0; t2++) {
// Check that the first variable was within r
assertTrue(withinR[(int) distancesAndIndicesWithinR[t2][1]]);
// Reset this marker
withinR[(int) distancesAndIndicesWithinR[t2][1]] = false;
// And check that the max norm was reported
double maxNorm = (Math.abs(data2[t] - data2[(int) distancesAndIndicesWithinR[t2][1]]));
if (maxNorm < distancesForFirstVariable[(int) distancesAndIndicesWithinR[t2][1]]) {
maxNorm = distancesForFirstVariable[(int) distancesAndIndicesWithinR[t2][1]];
}
assertTrue((strict && (maxNorm < r)) ||
( !strict && (maxNorm <= r)));
// Check that its distance was properly returned in the array:
assertEquals(maxNorm, distancesAndIndicesWithinR[t2][0], 0.000001);
}
// Check that there were no other points that should have been returned:
int count2 = 0;
for (int t2 = 0; t2 < data2.length; t2++) {
if (withinR[t2]) {
// This won't count points that came through above
// since we already disabled withinR for them,
// nor the points within the exclusion window
double maxNorm = Math.abs(data2[t] - data2[t2]);
if ((strict && (maxNorm < r)) ||
(!strict && (maxNorm <= r))) {
count2++;
}
}
}
assertEquals(0, count2); // We set all the ones that should have been true to false already
}
}
public void checkRangeFinderArrayCallsNewData(boolean strict) throws Exception {
int numTimeSteps = 10000;
double[] data = rg.generateNormalData(numTimeSteps, 0, 1);
int newSamplesToCheck = 1000;
double[] samplePoints = rg.generateNormalData(newSamplesToCheck, 0, 1);
UnivariateNearestNeighbourSearcher searcher =
new UnivariateNearestNeighbourSearcher(data);
int[] counts = new int[newSamplesToCheck];
boolean[] withinR = new boolean[data.length];
int[] indicesWithinR = new int[data.length];
double r = 0.2;
long startTime = Calendar.getInstance().getTimeInMillis();
for (int n = 0; n < newSamplesToCheck; n++) {
int simpleCount = searcher.countPointsWithinR(samplePoints[n], r, !strict);
searcher.findPointsWithinR(r, samplePoints[n], !strict,
withinR, indicesWithinR);
int count = 0;
// Run through the list of points returned as within r and check them
for (int t2 = 0; indicesWithinR[t2] != -1; t2++) {
count++;
assertTrue(withinR[indicesWithinR[t2]]);
// Reset this marker
withinR[indicesWithinR[t2]] = false;
// Check that it's really within r
double maxNorm = Math.abs(samplePoints[n] - data[indicesWithinR[t2]]);
assertTrue((strict && (maxNorm < r)) ||
( !strict && (maxNorm <= r)));
}
// Check that there were no other points marked as within r:
int count2 = 0;
for (int t2 = 0; t2 < data.length; t2++) {
if (withinR[t2]) {
count2++;
}
}
assertEquals(0, count2); // We set all the ones that should have been true to false already
assertEquals(simpleCount, count);
// Now reset the boolean array
counts[n] = count;
}
long endTimeNNs = Calendar.getInstance().getTimeInMillis();
System.out.printf("Found all neighbours within %.3f (mean = %.3f) in: %.3f sec\n",
r, MatrixUtils.mean(counts),
((double) (endTimeNNs - startTime)/1000.0));
// Now do brute force:
int[] bruteForceCounts = new int[newSamplesToCheck];
for (int n = 0; n < newSamplesToCheck; n++) {
int count = 0;
for (int t2 = 0; t2 < data.length; t2++) {
double norm = Math.abs(samplePoints[n] - data[t2]);
if ((strict && (norm < r) ) ||
(!strict && (norm <= r))) {
count++;
}
}
bruteForceCounts[n] = count;
}
long endTimeBruteForce = Calendar.getInstance().getTimeInMillis();
System.out.printf("Found all neighbours within %.3f (mean = %.3f) by brute force in: %.3f sec\n",
r, MatrixUtils.mean(bruteForceCounts),
((double) (endTimeBruteForce - endTimeNNs)/1000.0));
// Now validate the nearest neighbour counts:
for (int n = 0; n < newSamplesToCheck; n++) {
assertEquals(bruteForceCounts[n], counts[n]);
}
}
public void testFindKNearestNeighbours() throws Exception {
for (int K = 1; K < 5; K++) {
double[] data = rg.generateNormalData(1000, 0, 1);

63
readme-template.txt
View File

@ -1,6 +1,6 @@
Java Information Dynamics Toolkit (JIDT)
Copyright (C) 2012-2014 Joseph T. Lizier
Copyright (C) 2014-2015 Joseph T. Lizier and Ipek Özdemir
Copyright (C) 2014-2016 Joseph T. Lizier and Ipek Özdemir
Version @VERSION@ (see release notes below)
@ -31,22 +31,23 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
Website
=============
Full information on the JIDT (usage, etc) is provided at the project page and wiki on google code:
Full information on the JIDT (usage, etc) is provided at the project page and wiki on github:
http://code.google.com/p/information-dynamics-toolkit/
https://github.com/jlizier/jidt/
https://github.com/jlizier/jidt/wiki
=============
Installation
=============
"Full" description of any required installation is at: http://code.google.com/p/information-dynamics-toolkit/wiki/Installation
"Full" description of any required installation is at: https://github.com/jlizier/jidt/wiki/Installation
However, if you are reading this file, you've downloaded a distribution and you're halfway there!
There are no dependencies to download; unless:
a. You don't have java installed - download it from http://www.java.com/
b. You wish to build the project using the build.xml script - this requires ant: http://ant.apache.org/
c. You wish to run the JUnit test cases - this requires JUnit: http://www.junit.org/ - for how to run JUnit with our ant script see http://code.google.com/p/information-dynamics-toolkit/wiki/JUnitTestCases
c. You wish to run the JUnit test cases - this requires JUnit: http://www.junit.org/ - for how to run JUnit with our ant script see https://github.com/jlizier/jidt/wiki/JUnitTestCases
Then just put the jar in a relevant location in your file structure.
@ -61,10 +62,10 @@ A research paper describing the toolkit is included in the top level directory -
A tutorial, providing background to the information-theoretic measures, various estimators, and then to the JIDT toolkit itself is included in the tutorial folder (see "JIDT-TutorialSlides.pdf" for the tutorial slides, and "README-TutorialAndExercise.pdf" for further description of the tutorial exercises).
Javadocs for the toolkit are included in the full distribution at javadocs.
They can also be generated using "ant javadocs" (useful if you are on an SVN view).
Further, they will soon be posted on the web.
They can also be generated using "ant javadocs" (useful if you are on a git clone).
Further, they will are posted on the web via links at https://github.com/jlizier/jidt/wiki/Documentation
The project wiki also contains further information on various aspects; see http://code.google.com/p/information-dynamics-toolkit/ to start.
The project wiki also contains further information on various aspects; see https://github.com/jlizier/jidt/wiki to start.
Further documentation is provided by the Usage demo examples below.
@ -76,33 +77,33 @@ You can also join our email discussion group jidt-discuss at http://groups.googl
Several sets of demonstration code are distributed with the toolkit:
a. demos/AutoAnalyser -- a GUI tool to compute transfer entropy on a chosen data set with the toolkit, and also automatically generate code in Java, Python and Matlab to show how to do this calculation with the toolkit. See description at https://code.google.com/p/information-dynamics-toolkit/wiki/AutoAnalyser
a. demos/AutoAnalyser -- a GUI tool to compute transfer entropy on a chosen data set with the toolkit, and also automatically generate code in Java, Python and Matlab to show how to do this calculation with the toolkit. See description at https://github.com/jlizier/jidt/wiki/AutoAnalyser
b. demos/java -- basic examples on easily using the Java toolkit -- run these from the shell scripts in this directory -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/SimpleJavaExamples
b. demos/java -- basic examples on easily using the Java toolkit -- run these from the shell scripts in this directory -- see description at https://github.com/jlizier/jidt/wiki/SimpleJavaExamples
c. Several demo sets mirror the SimpleJavaExamples to demonstrate the use of the toolkit in non-Java environments:
i. demos/octave -- basic examples on easily using the Java toolkit from Octave or Matlab environments -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/OctaveMatlabExamples
i. demos/octave -- basic examples on easily using the Java toolkit from Octave or Matlab environments -- see description at https://github.com/jlizier/jidt/wiki/OctaveMatlabExamples
ii. demos/python -- basic examples on easily using the Java toolkit from Python -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/PythonExamples
ii. demos/python -- basic examples on easily using the Java toolkit from Python -- see description at https://github.com/jlizier/jidt/wiki/PythonExamples
iii. demos/r -- basic examples on easily using the Java toolkit from R -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/R_Examples
iii. demos/r -- basic examples on easily using the Java toolkit from R -- see description at https://github.com/jlizier/jidt/wiki/R_Examples
iv. demos/julia -- basic examples on easily using the Java toolkit from Julia -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/JuliaExamples
iv. demos/julia -- basic examples on easily using the Java toolkit from Julia -- see description at https://github.com/jlizier/jidt/wiki/JuliaExamples
v. demos/clojure -- basic examples on easily using the Java toolkit from Clojure -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/Clojure_Examples
v. demos/clojure -- basic examples on easily using the Java toolkit from Clojure -- see description at https://github.com/jlizier/jidt/wiki/Clojure_Examples
d. demos/octave/CellularAutomata -- using the Java toolkit to plot local information dynamics profiles in cellular automata; the toolkit is run under Octave or Matlab -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/CellularAutomataDemos
d. demos/octave/CellularAutomata -- using the Java toolkit to plot local information dynamics profiles in cellular automata; the toolkit is run under Octave or Matlab -- see description at https://github.com/jlizier/jidt/wiki/CellularAutomataDemos
e. demos/octave/SchreiberTransferEntropyExamples -- recreates the transfer entropy examples in Schreiber's original paper presenting this measure; shows the correct parameter settings to reproduce these results -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/SchreiberTeDemos
e. demos/octave/SchreiberTransferEntropyExamples -- recreates the transfer entropy examples in Schreiber's original paper presenting this measure; shows the correct parameter settings to reproduce these results -- see description at https://github.com/jlizier/jidt/wiki/SchreiberTeDemos
f. demos/octave/DetectingInteractionLags -- demonstration of using the transfer entropy with source-destination lags; the demo is run under Octave or Matlab -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/DetectingInteractionLags
f. demos/octave/DetectingInteractionLags -- demonstration of using the transfer entropy with source-destination lags; the demo is run under Octave or Matlab -- see description at https://github.com/jlizier/jidt/wiki/DetectingInteractionLags
g. demos/java/InterregionalTransfer -- higher level example using collective transfer entropy to infer effective connections between "regions" of data -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/InterregionalTransfer
g. demos/java/InterregionalTransfer -- higher level example using collective transfer entropy to infer effective connections between "regions" of data -- see description at https://github.com/jlizier/jidt/wiki/InterregionalTransfer
h. demos/octave/NullDistributions -- investigating the correspondence between analytic and bootstrapped distributions for TE and MI under null hypotheses of no relationship; the demo is run under Octave or Matlab -- see description at https://code.google.com/p/information-dynamics-toolkit/wiki/NullDistributions
h. demos/octave/NullDistributions -- investigating the correspondence between analytic and bootstrapped distributions for TE and MI under null hypotheses of no relationship; the demo is run under Octave or Matlab -- see description at https://github.com/jlizier/jidt/wiki/NullDistributions
i. java/unittests -- the JUnit test cases for the Java toolkit are included in the distribution -- these case also be browsed to see simple use cases for the various calculators in the toolkit -- see description at http://code.google.com/p/information-dynamics-toolkit/wiki/JUnitTestCases
i. java/unittests -- the JUnit test cases for the Java toolkit are included in the distribution -- these case also be browsed to see simple use cases for the various calculators in the toolkit -- see description at https://github.com/jlizier/jidt/wiki/JUnitTestCases
=============
Citation
@ -112,7 +113,7 @@ Please cite your use of this toolkit as:
Joseph T. Lizier, "JIDT: An information-theoretic toolkit for studying the dynamics of complex systems", Frontiers in Robotics and AI 1:11, 2014; doi:10.3389/frobt.2014.00011
A pre-print of this paper is distributed with this toolkit (InfoDynamicsToolkit.pdf) and is available at arXiv:1408.3270
A pre-print of this paper is distributed with this toolkit (InfoDynamicsToolkit.pdf) and is available at arXiv:1408.3270 (https://arxiv.org/abs/1408.3270)
=============
Notices
@ -132,6 +133,24 @@ Notices for this software are found in the notices/JAMA directory.
Release notes
===============
v1.3.1 21/10/2016
-----------------
(after 385 commits recorded by github, repository as at https://github.com/jlizier/jidt/tree/269e263a84998807c5c02f36397b585a19205938 save for this file update)
Major update to TransferEntropyCalculatorDiscrete so as to implement arbirtray source and dest embeddings and source-dest delay;
Conditional TE calculators (continuous) handle empty conditional variables;
Added auto-embedding method for AIS and TE which maximises bias corrected AIS;
Added getNumSeparateObservations() method to TE calculators to make reconstructing/separating local values easier after multiple addObservations() calls;
Fixed kernel estimator classes to return proper densities, not probabilities;
Bug fix in mixed discrete-continuous MI (Kraskov) implementation;
Added simple interface for adding joint observations for MultiInfoCalculatorDiscrete
Including compiled class files for the AutoAnalyser demo in distribution;
Updated Python demo 1 to show use of numpy arrays with ints;
Added Python demo 7 and 9 for TE Kraskov with ensemble method and auto-embedding respectively;
Added Matlab/Octave example 10 for conditional TE via Kraskov (KSG) algorithm;
Added utilities to prepare for enhancing surrogate calculations with fast nearest neighbour search;
Minor bug patch to Python readFloatsFile utility;
v1.3 10/7/2015 at r691
----------------------
Added AutoAnalyser (Code Generator) GUI demo for MI and TE;

2
version-template.txt
View File

@ -1,6 +1,6 @@
Java Information Dynamics Toolkit
Copyright (C) 2012-2014 Joseph T. Lizier
Copyright (C) 2014 Joseph T. Lizier and Ipek Özdemir
Copyright (C) 2014-2016 Joseph T. Lizier and Ipek Özdemir
Version @VERSION@