{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "47696870-1fda-48e0-86e7-ce8f7db2d46f",
   "metadata": {},
   "source": [
    "# Directed functional/effective network inference with TE on Cellular Automata\n",
    "\n",
    "In this activity we will perform pairwise functional and then effective network inference using MI and TE on our previous Cellular Automata data set, where we know the structure of the underlying network and have a good understanding of the dynamics.\n",
    "\n",
    "## A. Functional network inference with MI\n",
    "\n",
    "First we will infer the functional network to represent the pairwise statistical relationships between the cells as the nodes in our network.\n",
    "\n",
    "1. Make sure you have a sample data set for CA rule 54 from our tutorial activities in the previous modules -- or to download again see the link on the tutorial website.\n",
    "2. Now open the MI AutoAnalyser. Select a Discrete estimator, select your CA data file, tick the checkbox for `All pairs?` and -- most importantly -- untick the checkbox for `Compute result?` (we don't want to run this computation, it will take a lot of time!), and click `Generate Code`.\n",
    "3. Paste the code into the code cells below.\n",
    "4. Examine how the generated code is computing the MI for each cell pair.\n",
    "5. Make a few changes to the code here for our purposes:\n",
    "    * Once we get to directed network inference, it will make more sense for our outer loop to be over targets, so that we consider the set of sources afterwards for that target. So, swap the order of the loops over `s` and `d` (so that the outer loop is over `d`)\n",
    "    * We also don't want to compute the functional network over the whole 10000 cells - this will take too long and is too large to visualise. Let's just work with the first 100. So, before the loops make a new variable `networkSize=100;`\n",
    "    * Then have the loops over `s` and `d` only run up to `networkSize` instead of all the way up to `10000`.\n",
    "    * Create a 2D array to store the MI values before the for loops, `results = numpy.zeros((networkSize,networkSize));`\n",
    "    * Remove the `print` statement, and instead assign the result to the results array, as `results[s,d] = result;`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9e39673b-6d6e-42a5-8520-e54ed6ce4279",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paste import and JVM startup lines here:\n",
    "\n",
    "from jpype import *\n",
    "import numpy\n",
    "import sys\n",
    "# Our python data file readers are a bit of a hack, python users will do better on this:\n",
    "sys.path.append(\"/home/joseph/JIDT/infodynamics-dist-1.6.1/demos/python\")\n",
    "import readIntsFile\n",
    "\n",
    "if (not isJVMStarted()):\n",
    "    # Add JIDT jar library to the path\n",
    "    jarLocation = \"/home/joseph/JIDT/infodynamics-dist-1.6.1/infodynamics.jar\"\n",
    "    # Start the JVM (add the \"-Xmx\" option with say 1024M if you get crashes due to not enough memory space)\n",
    "    startJVM(getDefaultJVMPath(), \"-ea\", \"-Djava.class.path=\" + jarLocation, convertStrings=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8418503d-4979-43a8-b6c4-da47d216437d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paste analysis code here:\n",
    "\n",
    "# 0. Load/prepare the data:\n",
    "dataRaw = readIntsFile.readIntsFile(\"/home/joseph/TeachingPlayground/CSYS5030/Week10/ca54.txt\")\n",
    "# As numpy array:\n",
    "data = numpy.array(dataRaw)\n",
    "# 1. Construct the calculator:\n",
    "calcClass = JPackage(\"infodynamics.measures.discrete\").MutualInformationCalculatorDiscrete\n",
    "calc = calcClass(2, 2, 0)\n",
    "# 2. No other properties to set for discrete calculators.\n",
    "\n",
    "networkSize=100;\n",
    "results = numpy.zeros((networkSize,networkSize));\n",
    "# Compute for all pairs:\n",
    "for d in range(networkSize):\n",
    "    for s in range(networkSize):\n",
    "        # For each source-dest pair:\n",
    "        if (s == d):\n",
    "            continue\n",
    "        source = JArray(JInt, 1)(data[:, s].tolist())\n",
    "        destination = JArray(JInt, 1)(data[:, d].tolist())\n",
    "\n",
    "        # 3. Initialise the calculator for (re-)use:\n",
    "        calc.initialise()\n",
    "        # 4. Supply the sample data:\n",
    "        calc.addObservations(source, destination)\n",
    "        # 5. Compute the estimate:\n",
    "        result = calc.computeAverageLocalOfObservations()\n",
    "\n",
    "        results[s,d] = result;"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a913bede-d3ed-4fe7-8190-4ef57c112764",
   "metadata": {},
   "source": [
    "6. Run the above code cells, and then run the next code cell to plot the adjacency matrix of the functional network, as well as say the first 100 time steps of the CA dynamics so we can interpret what's happening:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "50626790-8bc7-49cd-a16c-7893f8c90394",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 95156 characters]",
      "text/plain": [
       "<Figure size 1400x600 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the functional connectivity\n",
    "import matplotlib.pyplot as plt\n",
    "plt.figure(figsize=(14,6))\n",
    "plt.subplot(1,2,1) # left subplot\n",
    "plt.imshow(results)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('MI between all cell pairs');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('MI (bits)');\n",
    "# Plot the first 100 time steps for the cells we're analysing:\n",
    "plt.subplot(1,2,2) # right subplot\n",
    "plt.imshow(data[:99,:networkSize]) # Plotting the first 100 time steps for first networkSize cells\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('time');\n",
    "plt.title('Raw CA values');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('CA values');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab1b4970-442e-4f11-8524-d3128fe5863d",
   "metadata": {},
   "source": [
    "7. Look at the results for the adjacency matrix for the MI functional network in figure 1 (left):\n",
    "    * What do you observe in terms of the functional network structure? Which sources have the highest MI to each target?\n",
    "    * How does it compare to what you know the underlying causal structure to be? Is that what you expected to see?\n",
    "    * Can you interpret the features of the network via the dynamics that you can see in figure 2 (right)? What generates the modular block structure?\n",
    "\n",
    "**A note** for when you are constructing functional connectivity for other data sets which don't have an obvious ordinal layout like the CA data does: you might consider running a _clustering_ method over the FC adjacency matrix in order to group variables with similar relationships to others. This will give you visual insights into the modularity in the FC. To do this, you would use code like the following:\n",
    "```python\n",
    "fc = results\n",
    "# Import relevant libraries\n",
    "from scipy.cluster.hierarchy import linkage, optimal_leaf_ordering, leaves_list\n",
    "from scipy.spatial.distance import pdist\n",
    "# Perform hierarchical clustering using average linkage\n",
    "distances = numpy.max(fc) - fc\n",
    "Z = linkage(distances, method='average')\n",
    "# Compute the optimal leaf ordering\n",
    "optimal_Z = optimal_leaf_ordering(Z, distances)\n",
    "# Get the order of the leaves\n",
    "optimal_order = leaves_list(optimal_Z)\n",
    "# ...\n",
    "# And utilise this clustering to reorder the variables in your imshow plot call with:\n",
    "plt.imshow(fc[optimal_order,:][:,optimal_order],cmap=cm.coolwarm, norm=mplcolors.CenteredNorm())\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "389f8eb6-8b07-4ad0-9038-05c53c49c272",
   "metadata": {},
   "source": [
    "## B. Directed functional network inference with TE\n",
    "\n",
    "Now, we move on to infer a directed functional network with TE, to show the directed pairwise statistical relationships in the network.\n",
    "\n",
    "1. Start by copying the above two code cells for the functional network with MI (and plots), and pasting it into the code cell below that we will edit for TE.\n",
    "2. Open the TE AutoAnalyser - we're just going to generate some dummy code to grab a few lines from. Select a Discrete estimator, select your CA data file, tick the checkbox for `All pairs?`, set the `k_HISTORY` at 4 (we won't have enough data here for it to be larger, since we're assuming _heterogeneous_ variables here), and -- most importantly -- untick the checkbox for `Compute result?` (we don't want to run this computation, it will take a lot of time!), and click `Generate Code`.\n",
    "3. Copy the lines constructing the TE calculator, and paste it into your code cell below, replacing where the MI calculator was previously constructed.\n",
    "4. Down the bottom of the code where the adjacency matrix is plotted, in the `title()` and `set_label()` functions replace \"MI\" with \"TE\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f3f214ba-23ab-4c25-b822-b666e31e46f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 103376 characters]",
      "text/plain": [
       "<Figure size 1400x600 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Copy the MI analysis and plotting code cells into here; we will edit these to use TE\n",
    "\n",
    "# Paste analysis code here:\n",
    "\n",
    "# 0. Load/prepare the data:\n",
    "dataRaw = readIntsFile.readIntsFile(\"/home/joseph/TeachingPlayground/CSYS5030/Week10/ca54.txt\")\n",
    "# As numpy array:\n",
    "data = numpy.array(dataRaw)\n",
    "# 1. Construct the calculator:\n",
    "calcClass = JPackage(\"infodynamics.measures.discrete\").TransferEntropyCalculatorDiscrete\n",
    "calc = calcClass(2, 4, 1, 1, 1, 1)\n",
    "# 2. No other properties to set for discrete calculators.\n",
    "\n",
    "networkSize=100;\n",
    "results = numpy.zeros((networkSize,networkSize));\n",
    "pValues = numpy.zeros((networkSize,networkSize));\n",
    "# Compute for all pairs:\n",
    "for d in range(networkSize):\n",
    "    for s in range(networkSize):\n",
    "        # For each source-dest pair:\n",
    "        if (s == d):\n",
    "            pValues[s,d] = 1;\n",
    "            continue\n",
    "        source = JArray(JInt, 1)(data[:, s].tolist())\n",
    "        destination = JArray(JInt, 1)(data[:, d].tolist())\n",
    "\n",
    "        # 3. Initialise the calculator for (re-)use:\n",
    "        calc.initialise()\n",
    "        # 4. Supply the sample data:\n",
    "        calc.addObservations(source, destination)\n",
    "        # 5. Compute the estimate:\n",
    "        result = calc.computeAverageLocalOfObservations()\n",
    "        # 6. Compute the (statistical significance via) null distribution analytically:\n",
    "        measDist = calc.computeSignificance()\n",
    "\n",
    "        results[s,d] = result;\n",
    "        pValues[s,d] = measDist.pValue;\n",
    "\n",
    "# Plot the functional connectivity\n",
    "import matplotlib.pyplot as plt\n",
    "plt.figure(figsize=(14,6))\n",
    "plt.subplot(1,2,1) # left subplot\n",
    "plt.imshow(results)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('TE between all cell pairs');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('TE (bits)');\n",
    "# Plot the first 100 time steps for the cells we're analysing:\n",
    "plt.subplot(1,2,2) # right subplot\n",
    "plt.imshow(data[:99,:networkSize]) # Plotting the first 100 time steps for first networkSize cells\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('time');\n",
    "plt.title('Raw CA values');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('CA values');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a636f3fa-e549-4dea-97ef-3271ba2bf503",
   "metadata": {},
   "source": [
    "5. Run the script, and observe the results for the adjacency matrix for the TE directed functional network in the figure:\n",
    "    * Compare the results to those for the MI functional network. Has the location of the strongest sources for each target changed? Is this what you expected? Is this beginning to inform us of a more sensible model of the dynamics for each target, given who we know the sources for each target are?\n",
    "    * Do you still observe a modular block structure? How might we explain this with reference to the dynamics of the CA and the TE measure?\n",
    "    * Could you select a reasonable threshold to select parent cells for each target cell here? Try replotting the TE results but thresholding for parent selection via, e.g. `plt.imshow(results > 0.1);` for a threshold at 0.1 bits, to see if you can identify a threshold that works well everywhere.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a8130ca4-055b-4aa8-abae-802a42270691",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27888 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Try replotting the TE results with a threshold for parent selection\n",
    "\n",
    "threshold = 0.09\n",
    "plt.figure()\n",
    "plt.imshow(results > threshold)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('TE between all cell pairs -- compared to threshold');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('TE > %.2f' % threshold);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9154c103-527c-4dda-8981-82fc10b9d695",
   "metadata": {},
   "source": [
    "7. Let's look into making a more principled threshold selection, following methods described in the lectures. We will add some code to the cell above to compute the p-values for the TE estimates from each source, analytically:\n",
    "    * Go back to the AutoAnalyser, ticking the checkboxes for `Add stat. signif.?` and `analytically?` to generate some template code of how to do this.\n",
    "    * Copy and paste the calculation of the `measDist` object after the TE calculation, and then store its member value `measDist.pValue` into a 2D array `pValues` as we have already done for `results`. Recall - what is the meaning of the p-value result for a given source?\n",
    "    * Where the if statement checks `if (s == d):` set `pValues[s,d] = 1;` otherwise seeing a 0 value there may confuse us.\n",
    "8. Re-run the above TE calculations, and also run the following code cell to plot the p-values.<br/>\n",
    "Take a look at the new figure below plotting the p-values. We will need to zoom in to see which sources have significant p-values here. We'll zoom in to only see sources which pass a Bonferroni corrected threshold `0.05 / (networkSize * (networkSize-1))` (to correct for the multiple comparisons here): uncomment the last line in the next code cell which now zooms in for us, and re-run this code cell.<br/>\n",
    "Any p-value that is below the extreme maximal colour would pass a statistical test here and be inferred as a parent in the directed functional network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "27d4d33b-0593-4463-b5e1-3b807719b36e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27276 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the p-values for each pairwise calculation:\n",
    "plt.figure()\n",
    "plt.imshow(pValues)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('p-value of TE between all cell pairs');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('p-value');\n",
    "plt.clim(0,0.05/(networkSize*(networkSize-1))); # Can use this to highlight significant p-values"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ad83b7f-e951-466d-b25d-11b1b60e22fa",
   "metadata": {},
   "source": [
    "8. Compared to the MI functional network and thresholding the TE value, these directed functional networks capture the direct parents on either side of the target, and we've addressed the issue of a more principled threshold selection. Still, are there many sources which appear to be parents here that you would not expect, given that we know only one source on either side of the target cell are its direct parents? Think about how we could extend the standard algorithm to exclude those other sources from being selected? (This is the subject of the next parts of the lecture)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3510c19-2295-4faf-ba1a-7bfc0aa8c5d7",
   "metadata": {},
   "source": [
    "<hr/>\n",
    "\n",
    "## C. Effective network inference with multivariate TE\n",
    "\n",
    "I've put together demonstration code to show a rudimentary implementation of the effective network inference on the CA data in the code cell below.\n",
    "\n",
    "1. Inspect the code, comparing it to the pairwise TE directed functional network code that we produced in the previous activity. Try to understand the main features:\n",
    "    * Inside the loop over targets, where we infer the set of parents for each, we have an additional while loop over rounds of adding parents to the set.\n",
    "    * Within each round, we have a loop over the candidate sources.\n",
    "    * Each candidate source is evaluated using a conditional TE, conditioning on previously selected parents.\n",
    "    * After all candidates are evaluated, we identify the candidate with the strongest conditional TE, and check the statistical significance of this value.\n",
    "    * If the source is statistically significant, we continue to another round to try to select another parent. Else we stop for this target.\n",
    "2. Try to run the code, and compare the results to your expectations for a good multivariate model explaining the dynamics, as well as to the functional networks from MI and pairwise TE above.\n",
    "    * Think through why the inference method might still not completely match the underlying structure, including: statistical fluctuations from the short data set effecting who the strongest sources are, and statistical power, hard coded history length used which may leave more information in spurious sources, no use of pruning step here, conservative p-value threshold and use of analytic surrogates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "658117b4-9400-435d-aba5-c5796805f6e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Beginning greedy selection of parents for 0\n",
      "Selected source 1, with TE(1->0 | )=0.07791, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 26 was not significant (TE(26->0 | 1)=0.05884, p-value=0.000114), quitting\n",
      "Beginning greedy selection of parents for 1\n",
      "Selected source 0, with TE(0->1 | )=0.13104, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 2, with TE(2->1 | 0)=0.22481, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->1 | 0,2)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 2\n",
      "Selected source 0, with TE(0->2 | )=0.14630, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 3, with TE(3->2 | 0)=0.09115, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 1 was not significant (TE(1->2 | 0,3)=0.10124, p-value=0.000024), quitting\n",
      "Beginning greedy selection of parents for 3\n",
      "Selected source 1, with TE(1->3 | )=0.14275, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 4, with TE(4->3 | 1)=0.08704, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 2 was not significant (TE(2->3 | 1,4)=0.10587, p-value=0.000006), quitting\n",
      "Beginning greedy selection of parents for 4\n",
      "Selected source 2, with TE(2->4 | )=0.14010, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 5, with TE(5->4 | 2)=0.08473, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 3 was not significant (TE(3->4 | 2,5)=0.10505, p-value=0.000008), quitting\n",
      "Beginning greedy selection of parents for 5\n",
      "Selected source 3, with TE(3->5 | )=0.13906, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 6, with TE(6->5 | 3)=0.09188, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 4 was not significant (TE(4->5 | 3,6)=0.09862, p-value=0.000052), quitting\n",
      "Beginning greedy selection of parents for 6\n",
      "Selected source 4, with TE(4->6 | )=0.13429, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 7, with TE(7->6 | 4)=0.09303, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 5, with TE(5->6 | 4,7)=0.12510, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->6 | 4,7,5)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 7\n",
      "Selected source 5, with TE(5->7 | )=0.13553, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 8, with TE(8->7 | 5)=0.09639, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 6, with TE(6->7 | 5,8)=0.11499, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->7 | 5,8,6)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 8\n",
      "Selected source 6, with TE(6->8 | )=0.13900, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 9, with TE(9->8 | 6)=0.09033, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 7, with TE(7->8 | 6,9)=0.10844, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->8 | 6,9,7)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 9\n",
      "Selected source 7, with TE(7->9 | )=0.14066, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 10, with TE(10->9 | 7)=0.09559, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 8 was not significant (TE(8->9 | 7,10)=0.09648, p-value=0.000096), quitting\n",
      "Beginning greedy selection of parents for 10\n",
      "Selected source 8, with TE(8->10 | )=0.13843, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 11, with TE(11->10 | 8)=0.08855, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 9, with TE(9->10 | 8,11)=0.10910, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->10 | 8,11,9)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 11\n",
      "Selected source 9, with TE(9->11 | )=0.14139, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 12, with TE(12->11 | 9)=0.10071, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 10, with TE(10->11 | 9,12)=0.11161, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->11 | 9,12,10)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 12\n",
      "Selected source 10, with TE(10->12 | )=0.14382, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 13, with TE(13->12 | 10)=0.09522, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 11, with TE(11->12 | 10,13)=0.12305, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->12 | 10,13,11)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 13\n",
      "Selected source 11, with TE(11->13 | )=0.14849, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 14, with TE(14->13 | 11)=0.10258, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 12 was not significant (TE(12->13 | 11,14)=0.10509, p-value=0.000007), quitting\n",
      "Beginning greedy selection of parents for 14\n",
      "Selected source 12, with TE(12->14 | )=0.14077, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 15, with TE(15->14 | 12)=0.10336, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 13, with TE(13->14 | 12,15)=0.10749, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->14 | 12,15,13)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 15\n",
      "Selected source 13, with TE(13->15 | )=0.14288, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 16, with TE(16->15 | 13)=0.10124, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 14 was not significant (TE(14->15 | 13,16)=0.10392, p-value=0.000011), quitting\n",
      "Beginning greedy selection of parents for 16\n",
      "Selected source 14, with TE(14->16 | )=0.13457, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 17, with TE(17->16 | 14)=0.11320, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 15, with TE(15->16 | 14,17)=0.10791, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->16 | 14,17,15)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 17\n",
      "Selected source 15, with TE(15->17 | )=0.11197, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 18, with TE(18->17 | 15)=0.10284, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 16, with TE(16->17 | 15,18)=0.12803, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->17 | 15,18,16)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 18\n",
      "Selected source 16, with TE(16->18 | )=0.13004, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 19, with TE(19->18 | 16)=0.10265, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 17, with TE(17->18 | 16,19)=0.10946, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->18 | 16,19,17)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 19\n",
      "Selected source 17, with TE(17->19 | )=0.13081, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 20, with TE(20->19 | 17)=0.10275, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 18 was not significant (TE(18->19 | 17,20)=0.10006, p-value=0.000034), quitting\n",
      "Beginning greedy selection of parents for 20\n",
      "Selected source 18, with TE(18->20 | )=0.13303, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 21, with TE(21->20 | 18)=0.08988, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 19 was not significant (TE(19->20 | 18,21)=0.10372, p-value=0.000011), quitting\n",
      "Beginning greedy selection of parents for 21\n",
      "Selected source 19, with TE(19->21 | )=0.13992, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 22, with TE(22->21 | 19)=0.08786, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 20 was not significant (TE(20->21 | 19,22)=0.09277, p-value=0.000271), quitting\n",
      "Beginning greedy selection of parents for 22\n",
      "Selected source 20, with TE(20->22 | )=0.15380, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 23, with TE(23->22 | 20)=0.07865, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 21 was not significant (TE(21->22 | 20,23)=0.07921, p-value=0.008005), quitting\n",
      "Beginning greedy selection of parents for 23\n",
      "Selected source 21, with TE(21->23 | )=0.15310, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 24, with TE(24->23 | 21)=0.06898, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 22 was not significant (TE(22->23 | 21,24)=0.06368, p-value=0.146416), quitting\n",
      "Beginning greedy selection of parents for 24\n",
      "Selected source 22, with TE(22->24 | )=0.15717, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 26, with TE(26->24 | 22)=0.07524, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 8 was not significant (TE(8->24 | 22,26)=0.02772, p-value=0.999531), quitting\n",
      "Beginning greedy selection of parents for 25\n",
      "Selected source 23, with TE(23->25 | )=0.19065, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 27, with TE(27->25 | 23)=0.07402, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 7 was not significant (TE(7->25 | 23,27)=0.01014, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 26\n",
      "Selected source 24, with TE(24->26 | )=0.17819, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 28, with TE(28->26 | 24)=0.09103, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 49 was not significant (TE(49->26 | 24,28)=0.00782, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 27\n",
      "Selected source 25, with TE(25->27 | )=0.16700, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 29, with TE(29->27 | 25)=0.09229, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 26 was not significant (TE(26->27 | 25,29)=0.01630, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 28\n",
      "Selected source 26, with TE(26->28 | )=0.16551, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 30, with TE(30->28 | 26)=0.08512, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 29 was not significant (TE(29->28 | 26,30)=0.01896, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 29\n",
      "Selected source 27, with TE(27->29 | )=0.15484, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 31, with TE(31->29 | 27)=0.10524, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 22 was not significant (TE(22->29 | 27,31)=0.00921, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 30\n",
      "Selected source 28, with TE(28->30 | )=0.15783, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 32, with TE(32->30 | 28)=0.09955, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 39 was not significant (TE(39->30 | 28,32)=0.01262, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 31\n",
      "Selected source 29, with TE(29->31 | )=0.15404, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 33, with TE(33->31 | 29)=0.09283, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 30 was not significant (TE(30->31 | 29,33)=0.01680, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 32\n",
      "Selected source 30, with TE(30->32 | )=0.13371, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 34, with TE(34->32 | 30)=0.08275, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 23 was not significant (TE(23->32 | 30,34)=0.03166, p-value=0.996373), quitting\n",
      "Beginning greedy selection of parents for 33\n",
      "Selected source 31, with TE(31->33 | )=0.13549, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 35, with TE(35->33 | 31)=0.09871, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 34 was not significant (TE(34->33 | 31,35)=0.02273, p-value=0.999986), quitting\n",
      "Beginning greedy selection of parents for 34\n",
      "Selected source 32, with TE(32->34 | )=0.13525, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 36, with TE(36->34 | 32)=0.10019, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 35 was not significant (TE(35->34 | 32,36)=0.02611, p-value=0.999828), quitting\n",
      "Beginning greedy selection of parents for 35\n",
      "Selected source 33, with TE(33->35 | )=0.13421, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 37, with TE(37->35 | 33)=0.11027, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 39 was not significant (TE(39->35 | 33,37)=0.02189, p-value=0.999993), quitting\n",
      "Beginning greedy selection of parents for 36\n",
      "Selected source 34, with TE(34->36 | )=0.12593, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 38, with TE(38->36 | 34)=0.10314, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 37 was not significant (TE(37->36 | 34,38)=0.01725, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 37\n",
      "Selected source 35, with TE(35->37 | )=0.14024, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 39, with TE(39->37 | 35)=0.10695, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 38 was not significant (TE(38->37 | 35,39)=0.01560, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 38\n",
      "Selected source 36, with TE(36->38 | )=0.13679, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 40, with TE(40->38 | 36)=0.09422, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 37 was not significant (TE(37->38 | 36,40)=0.02958, p-value=0.998676), quitting\n",
      "Beginning greedy selection of parents for 39\n",
      "Selected source 37, with TE(37->39 | )=0.12580, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 41, with TE(41->39 | 37)=0.13320, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 38 was not significant (TE(38->39 | 37,41)=0.02304, p-value=0.999982), quitting\n",
      "Beginning greedy selection of parents for 40\n",
      "Selected source 38, with TE(38->40 | )=0.12272, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 41, with TE(41->40 | 38)=0.09695, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 39, with TE(39->40 | 38,41)=0.10920, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->40 | 38,41,39)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 41\n",
      "Selected source 39, with TE(39->41 | )=0.13592, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 42, with TE(42->41 | 39)=0.10905, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 40, with TE(40->41 | 39,42)=0.12550, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->41 | 39,42,40)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 42\n",
      "Selected source 41, with TE(41->42 | )=0.11922, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 43, with TE(43->42 | 41)=0.25783, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->42 | 41,43)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 43\n",
      "Selected source 42, with TE(42->43 | )=0.11900, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 44, with TE(44->43 | 42)=0.28720, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->43 | 42,44)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 44\n",
      "Selected source 42, with TE(42->44 | )=0.14920, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 45, with TE(45->44 | 42)=0.11132, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 43, with TE(43->44 | 42,45)=0.14363, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->44 | 42,45,43)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 45\n",
      "Selected source 44, with TE(44->45 | )=0.12428, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 46, with TE(46->45 | 44)=0.24916, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->45 | 44,46)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 46\n",
      "Selected source 45, with TE(45->46 | )=0.12399, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 47, with TE(47->46 | 45)=0.26091, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->46 | 45,47)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 47\n",
      "Selected source 45, with TE(45->47 | )=0.12162, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 48, with TE(48->47 | 45)=0.11167, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 46, with TE(46->47 | 45,48)=0.14071, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->47 | 45,48,46)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 48\n",
      "Selected source 46, with TE(46->48 | )=0.12709, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 49, with TE(49->48 | 46)=0.11844, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 47 was not significant (TE(47->48 | 46,49)=0.08805, p-value=0.000949), quitting\n",
      "Beginning greedy selection of parents for 49\n",
      "Selected source 48, with TE(48->49 | )=0.11259, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 50, with TE(50->49 | 48)=0.23814, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->49 | 48,50)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 50\n",
      "Selected source 48, with TE(48->50 | )=0.12994, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 51, with TE(51->50 | 48)=0.10482, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 49, with TE(49->50 | 48,51)=0.12437, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->50 | 48,51,49)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 51\n",
      "Selected source 49, with TE(49->51 | )=0.11616, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 53, with TE(53->51 | 49)=0.11242, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 50 was not significant (TE(50->51 | 49,53)=0.05023, p-value=0.623212), quitting\n",
      "Beginning greedy selection of parents for 52\n",
      "Selected source 51, with TE(51->52 | )=0.12733, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 53, with TE(53->52 | 51)=0.27399, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->52 | 51,53)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 53\n",
      "Selected source 52, with TE(52->53 | )=0.11901, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 54, with TE(54->53 | 52)=0.26656, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->53 | 52,54)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 54\n",
      "Selected source 53, with TE(53->54 | )=0.11665, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 55, with TE(55->54 | 53)=0.26716, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->54 | 53,55)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 55\n",
      "Selected source 54, with TE(54->55 | )=0.12393, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 56, with TE(56->55 | 54)=0.26661, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->55 | 54,56)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 56\n",
      "Selected source 55, with TE(55->56 | )=0.12523, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 57, with TE(57->56 | 55)=0.27512, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->56 | 55,57)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 57\n",
      "Selected source 56, with TE(56->57 | )=0.13185, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 58, with TE(58->57 | 56)=0.28306, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->57 | 56,58)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 58\n",
      "Selected source 57, with TE(57->58 | )=0.12285, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 59, with TE(59->58 | 57)=0.28861, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->58 | 57,59)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 59\n",
      "Selected source 58, with TE(58->59 | )=0.12628, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 60, with TE(60->59 | 58)=0.26169, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->59 | 58,60)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 60\n",
      "Selected source 59, with TE(59->60 | )=0.13068, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 61, with TE(61->60 | 59)=0.25961, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->60 | 59,61)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 61\n",
      "Selected source 60, with TE(60->61 | )=0.12023, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 62, with TE(62->61 | 60)=0.24485, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->61 | 60,62)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 62\n",
      "Selected source 61, with TE(61->62 | )=0.10845, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 63, with TE(63->62 | 61)=0.25705, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->62 | 61,63)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 63\n",
      "Selected source 62, with TE(62->63 | )=0.12700, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 64, with TE(64->63 | 62)=0.24803, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->63 | 62,64)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 64\n",
      "Selected source 63, with TE(63->64 | )=0.12259, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 65, with TE(65->64 | 63)=0.26119, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->64 | 63,65)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 65\n",
      "Selected source 64, with TE(64->65 | )=0.12419, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 66, with TE(66->65 | 64)=0.25090, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->65 | 64,66)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 66\n",
      "Selected source 65, with TE(65->66 | )=0.10626, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 67, with TE(67->66 | 65)=0.23712, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->66 | 65,67)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 67\n",
      "Selected source 66, with TE(66->67 | )=0.10846, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 68, with TE(68->67 | 66)=0.20476, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->67 | 66,68)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 68\n",
      "Selected source 66, with TE(66->68 | )=0.12373, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 69, with TE(69->68 | 66)=0.09107, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 67 was not significant (TE(67->68 | 66,69)=0.09391, p-value=0.000198), quitting\n",
      "Beginning greedy selection of parents for 69\n",
      "Selected source 67, with TE(67->69 | )=0.14446, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 70, with TE(70->69 | 67)=0.07713, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 68 was not significant (TE(68->69 | 67,70)=0.09303, p-value=0.000252), quitting\n",
      "Beginning greedy selection of parents for 70\n",
      "Selected source 68, with TE(68->70 | )=0.14417, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 71, with TE(71->70 | 68)=0.08040, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 69 was not significant (TE(69->70 | 68,71)=0.09713, p-value=0.000080), quitting\n",
      "Beginning greedy selection of parents for 71\n",
      "Selected source 69, with TE(69->71 | )=0.14195, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 72, with TE(72->71 | 69)=0.08513, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 70 was not significant (TE(70->71 | 69,72)=0.10068, p-value=0.000028), quitting\n",
      "Beginning greedy selection of parents for 72\n",
      "Selected source 70, with TE(70->72 | )=0.13848, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 73, with TE(73->72 | 70)=0.09579, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 71, with TE(71->72 | 70,73)=0.10978, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->72 | 70,73,71)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 73\n",
      "Selected source 72, with TE(72->73 | )=0.10856, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 74, with TE(74->73 | 72)=0.19560, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->73 | 72,74)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 74\n",
      "Selected source 72, with TE(72->74 | )=0.11529, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 75, with TE(75->74 | 72)=0.09592, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 73 was not significant (TE(73->74 | 72,75)=0.09325, p-value=0.000237), quitting\n",
      "Beginning greedy selection of parents for 75\n",
      "Selected source 73, with TE(73->75 | )=0.11639, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 76, with TE(76->75 | 73)=0.07048, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 74 was not significant (TE(74->75 | 73,76)=0.10575, p-value=0.000006), quitting\n",
      "Beginning greedy selection of parents for 76\n",
      "Selected source 74, with TE(74->76 | )=0.13751, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 77, with TE(77->76 | 74)=0.08101, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 75 was not significant (TE(75->76 | 74,77)=0.07533, p-value=0.018416), quitting\n",
      "Beginning greedy selection of parents for 77\n",
      "Selected source 75, with TE(75->77 | )=0.13769, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 78 was not significant (TE(78->77 | 75)=0.06155, p-value=0.000043), quitting\n",
      "Beginning greedy selection of parents for 78\n",
      "Selected source 76, with TE(76->78 | )=0.14316, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 79, with TE(79->78 | 76)=0.07438, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 77 was not significant (TE(77->78 | 76,79)=0.08875, p-value=0.000792), quitting\n",
      "Beginning greedy selection of parents for 79\n",
      "Selected source 77, with TE(77->79 | )=0.15220, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 80, with TE(80->79 | 77)=0.10448, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 78 was not significant (TE(78->79 | 77,80)=0.08096, p-value=0.005370), quitting\n",
      "Beginning greedy selection of parents for 80\n",
      "Selected source 78, with TE(78->80 | )=0.15729, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 81, with TE(81->80 | 78)=0.08118, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 79 was not significant (TE(79->80 | 78,81)=0.04394, p-value=0.850247), quitting\n",
      "Beginning greedy selection of parents for 81\n",
      "Selected source 79, with TE(79->81 | )=0.16130, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 82 was not significant (TE(82->81 | 79)=0.06457, p-value=0.000014), quitting\n",
      "Beginning greedy selection of parents for 82\n",
      "Selected source 80, with TE(80->82 | )=0.15371, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 83 was not significant (TE(83->82 | 80)=0.06269, p-value=0.000028), quitting\n",
      "Beginning greedy selection of parents for 83\n",
      "Selected source 81, with TE(81->83 | )=0.18840, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 84 was not significant (TE(84->83 | 81)=0.03140, p-value=0.233831), quitting\n",
      "Beginning greedy selection of parents for 84\n",
      "Selected source 82, with TE(82->84 | )=0.19803, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 86 was not significant (TE(86->84 | 82)=0.02681, p-value=0.469007), quitting\n",
      "Beginning greedy selection of parents for 85\n",
      "Selected source 83, with TE(83->85 | )=0.20001, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 86 was not significant (TE(86->85 | 83)=0.03313, p-value=0.170082), quitting\n",
      "Beginning greedy selection of parents for 86\n",
      "Selected source 84, with TE(84->86 | )=0.18842, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 87 was not significant (TE(87->86 | 84)=0.04916, p-value=0.002812), quitting\n",
      "Beginning greedy selection of parents for 87\n",
      "Selected source 85, with TE(85->87 | )=0.14847, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 89 was not significant (TE(89->87 | 85)=0.03851, p-value=0.052956), quitting\n",
      "Beginning greedy selection of parents for 88\n",
      "Selected source 86, with TE(86->88 | )=0.16400, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 89 was not significant (TE(89->88 | 86)=0.06256, p-value=0.000030), quitting\n",
      "Beginning greedy selection of parents for 89\n",
      "Selected source 87, with TE(87->89 | )=0.18001, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 91 was not significant (TE(91->89 | 87)=0.04701, p-value=0.005378), quitting\n",
      "Beginning greedy selection of parents for 90\n",
      "Selected source 88, with TE(88->90 | )=0.17941, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 92 was not significant (TE(92->90 | 88)=0.04332, p-value=0.015417), quitting\n",
      "Beginning greedy selection of parents for 91\n",
      "Selected source 89, with TE(89->91 | )=0.17080, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 93 was not significant (TE(93->91 | 89)=0.05793, p-value=0.000156), quitting\n",
      "Beginning greedy selection of parents for 92\n",
      "Selected source 90, with TE(90->92 | )=0.16960, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 94 was not significant (TE(94->92 | 90)=0.06661, p-value=0.000007), quitting\n",
      "Beginning greedy selection of parents for 93\n",
      "Selected source 91, with TE(91->93 | )=0.16975, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 95 was not significant (TE(95->93 | 91)=0.06463, p-value=0.000014), quitting\n",
      "Beginning greedy selection of parents for 94\n",
      "Selected source 92, with TE(92->94 | )=0.16190, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 96, with TE(96->94 | 92)=0.06737, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 76 was not significant (TE(76->94 | 92,96)=0.02476, p-value=0.999933), quitting\n",
      "Beginning greedy selection of parents for 95\n",
      "Selected source 93, with TE(93->95 | )=0.12975, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 97, with TE(97->95 | 93)=0.08925, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 94 was not significant (TE(94->95 | 93,97)=0.03155, p-value=0.996555), quitting\n",
      "Beginning greedy selection of parents for 96\n",
      "Selected source 94, with TE(94->96 | )=0.11606, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 97, with TE(97->96 | 94)=0.08777, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 95 was not significant (TE(95->96 | 94,97)=0.07105, p-value=0.042711), quitting\n",
      "Beginning greedy selection of parents for 97\n",
      "Selected source 95, with TE(95->97 | )=0.12327, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 99, with TE(99->97 | 95)=0.09932, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 96 was not significant (TE(96->97 | 95,99)=0.02781, p-value=0.999503), quitting\n",
      "Beginning greedy selection of parents for 98\n",
      "Selected source 96, with TE(96->98 | )=0.12891, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 99, with TE(99->98 | 96)=0.07989, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 97 was not significant (TE(97->98 | 96,99)=0.10021, p-value=0.000032), quitting\n",
      "Beginning greedy selection of parents for 99\n",
      "Selected source 97, with TE(97->99 | )=0.10751, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 98 was not significant (TE(98->99 | 97)=0.04242, p-value=0.019635), quitting\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 27656 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "network = numpy.zeros((networkSize,networkSize), dtype=int);\n",
    "# Compute for all targets:\n",
    "for d in range(networkSize):\n",
    "\n",
    "    print('Beginning greedy selection of parents for %d' % d);\n",
    "\n",
    "    destination = JArray(JInt, 1)(data[:, d].tolist())\n",
    "    conditionalSet = [];\n",
    "\n",
    "    while (True):\n",
    "        # Precondition: we have already selected the parents in\n",
    "        #  conditionalSet, now we check if we can add to this:\n",
    "        \n",
    "        results = numpy.zeros((networkSize));\n",
    "        pValues = numpy.zeros((networkSize));\n",
    "\n",
    "        # 1. Construct the calculator:\n",
    "        if (len(conditionalSet) > 0):\n",
    "            calcClass = JPackage(\"infodynamics.measures.discrete\").ConditionalTransferEntropyCalculatorDiscrete\n",
    "            calc = calcClass(2, 4, len(conditionalSet))\n",
    "        else:\n",
    "            calcClass = JPackage(\"infodynamics.measures.discrete\").TransferEntropyCalculatorDiscrete\n",
    "            calc = calcClass(2, 4, 1, 1, 1, 1)\n",
    "        # 2. No other properties to set for discrete calculators.\n",
    "        for s in range(networkSize):\n",
    "            # For each source-dest pair:\n",
    "            \n",
    "            if ( (s == d) or (s in conditionalSet) ):\n",
    "                # If s is the target, or already in the conditioning set, skip evaluating TE from it\n",
    "                pValues[s] = 1;\n",
    "                continue\n",
    "            source = JArray(JInt, 1)(data[:, s].tolist())\n",
    "            conditional = JArray(JInt, 2)(data[:, conditionalSet].tolist())\n",
    "\n",
    "            # 3. Initialise the calculator for (re-)use:\n",
    "            calc.initialise()\n",
    "            # 4. Supply the sample data:\n",
    "            if (len(conditionalSet) > 0):\n",
    "                calc.addObservations(source, destination, conditional)\n",
    "            else:\n",
    "                calc.addObservations(source, destination)\n",
    "\n",
    "            # 5. Compute the estimate:\n",
    "            result = calc.computeAverageLocalOfObservations()\n",
    "            # 6. Compute the (statistical significance via) null distribution analytically:\n",
    "            measDist = calc.computeSignificance()\n",
    "\n",
    "            results[s] = result;\n",
    "            pValues[s] = measDist.pValue;\n",
    "\n",
    "        # Check which was the strongest source:\n",
    "        maxSourceIndex = numpy.argmax(results);\n",
    "        maxTE = results[maxSourceIndex];\n",
    "        if (pValues[maxSourceIndex] < 0.05/(networkSize*(networkSize-1))):\n",
    "            print('Selected source %d, with TE(%d->%d | %s)=%.5f, p-value=%.5f (conditioning on %d parents)' %\\\n",
    "                (maxSourceIndex, maxSourceIndex, d, (','.join(str(x) for x in conditionalSet)), maxTE, \\\n",
    "                pValues[maxSourceIndex], len(conditionalSet) ));\n",
    "            # Add this new source:\n",
    "            conditionalSet.append(maxSourceIndex);\n",
    "        else:\n",
    "            print('-- Max TE source %d was not significant (TE(%d->%d | %s)=%.5f, p-value=%.6f), quitting' %\\\n",
    "                (maxSourceIndex, maxSourceIndex, d, (','.join(str(x) for x in conditionalSet)), maxTE, pValues[maxSourceIndex]));\n",
    "            break\n",
    "    \n",
    "    # Postcondition: conditionalSet holds the parents for target d\n",
    "    network[conditionalSet, d] = 1;\n",
    "\n",
    "# Plot the effective connectivity\n",
    "plt.figure()\n",
    "plt.imshow(network)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('Multivariate effective network via TE(k=4)');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('Connections');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b3c6596-b81e-4e4c-97c8-2b9f9c79b640",
   "metadata": {},
   "source": [
    "A sample results plot is shown on the tutorial website (yours will not match precisely due to stochastic generation of the dynamics but should have similar features).\n",
    "\n",
    "3. _Optional extensions -- advanced_:\n",
    "    1. Can you incorporate code to address any of those issues to improve the network inference here?\n",
    "    2. E.g. try changing the history length to only 1 and try to understand the changes this makes in terms of what is being modelled, and why this might be a better choice for situations where you want to more closely replicate a causal structure between the observed variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "12d17406-7041-4192-913e-92ad570c9081",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Beginning greedy selection of parents for 0\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6612 bits\n",
      "-- Max TE source 1 was not significant (TE(1->0 | )=0.06158, p-value=0.000044), quitting\n",
      "Beginning greedy selection of parents for 1\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6709 bits\n",
      "Selected source 0, with TE(0->1 | )=0.12902, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 2, with TE(2->1 | 0)=0.17369, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->1 | 0,2)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 2\n",
      "Optimal k=6, giving bias corrected AIS(k=6)=0.6537 bits\n",
      "Selected source 0, with TE(0->2 | )=0.14858, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 3 was not significant (TE(3->2 | 0)=0.06173, p-value=0.999973), quitting\n",
      "Beginning greedy selection of parents for 3\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6667 bits\n",
      "Selected source 1, with TE(1->3 | )=0.14497, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 4 was not significant (TE(4->3 | 1)=0.07023, p-value=0.050791), quitting\n",
      "Beginning greedy selection of parents for 4\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6570 bits\n",
      "Selected source 2, with TE(2->4 | )=0.14010, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 5, with TE(5->4 | 2)=0.08473, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 3 was not significant (TE(3->4 | 2,5)=0.10505, p-value=0.000008), quitting\n",
      "Beginning greedy selection of parents for 5\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6661 bits\n",
      "Selected source 3, with TE(3->5 | )=0.14556, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 6 was not significant (TE(6->5 | 3)=0.07433, p-value=0.023169), quitting\n",
      "Beginning greedy selection of parents for 6\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6417 bits\n",
      "Selected source 4, with TE(4->6 | )=0.14373, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 7 was not significant (TE(7->6 | 4)=0.07348, p-value=0.027448), quitting\n",
      "Beginning greedy selection of parents for 7\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6545 bits\n",
      "Selected source 5, with TE(5->7 | )=0.13413, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 8 was not significant (TE(8->7 | 5)=0.07891, p-value=0.008800), quitting\n",
      "Beginning greedy selection of parents for 8\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6515 bits\n",
      "Selected source 6, with TE(6->8 | )=0.14252, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 9 was not significant (TE(9->8 | 6)=0.07710, p-value=0.013056), quitting\n",
      "Beginning greedy selection of parents for 9\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6544 bits\n",
      "Selected source 7, with TE(7->9 | )=0.14066, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 10, with TE(10->9 | 7)=0.09559, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 8 was not significant (TE(8->9 | 7,10)=0.09648, p-value=0.000096), quitting\n",
      "Beginning greedy selection of parents for 10\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6556 bits\n",
      "Selected source 8, with TE(8->10 | )=0.14305, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 11 was not significant (TE(11->10 | 8)=0.07291, p-value=0.030701), quitting\n",
      "Beginning greedy selection of parents for 11\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6526 bits\n",
      "Selected source 9, with TE(9->11 | )=0.14237, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 12 was not significant (TE(12->11 | 9)=0.08396, p-value=0.002732), quitting\n",
      "Beginning greedy selection of parents for 12\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6397 bits\n",
      "Selected source 10, with TE(10->12 | )=0.14705, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 13 was not significant (TE(13->12 | 10)=0.07982, p-value=0.007191), quitting\n",
      "Beginning greedy selection of parents for 13\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6435 bits\n",
      "Selected source 11, with TE(11->13 | )=0.14636, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 14 was not significant (TE(14->13 | 11)=0.08450, p-value=0.002397), quitting\n",
      "Beginning greedy selection of parents for 14\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6351 bits\n",
      "Selected source 12, with TE(12->14 | )=0.14077, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 15, with TE(15->14 | 12)=0.10336, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 13, with TE(13->14 | 12,15)=0.10749, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->14 | 12,15,13)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 15\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6604 bits\n",
      "Selected source 13, with TE(13->15 | )=0.14265, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 16 was not significant (TE(16->15 | 13)=0.07769, p-value=0.011505), quitting\n",
      "Beginning greedy selection of parents for 16\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6435 bits\n",
      "Selected source 14, with TE(14->16 | )=0.14204, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 17 was not significant (TE(17->16 | 14)=0.08598, p-value=0.001666), quitting\n",
      "Beginning greedy selection of parents for 17\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6551 bits\n",
      "Selected source 15, with TE(15->17 | )=0.11296, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 19 was not significant (TE(19->17 | 15)=0.09216, p-value=0.000334), quitting\n",
      "Beginning greedy selection of parents for 18\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6622 bits\n",
      "Selected source 16, with TE(16->18 | )=0.12867, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 20 was not significant (TE(20->18 | 16)=0.09407, p-value=0.000198), quitting\n",
      "Beginning greedy selection of parents for 19\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6806 bits\n",
      "Selected source 17, with TE(17->19 | )=0.12995, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 21 was not significant (TE(21->19 | 17)=0.09094, p-value=0.000462), quitting\n",
      "Beginning greedy selection of parents for 20\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6801 bits\n",
      "Selected source 18, with TE(18->20 | )=0.13473, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 22 was not significant (TE(22->20 | 18)=0.09311, p-value=0.000257), quitting\n",
      "Beginning greedy selection of parents for 21\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6948 bits\n",
      "Selected source 19, with TE(19->21 | )=0.13675, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 23 was not significant (TE(23->21 | 19)=0.07392, p-value=0.025166), quitting\n",
      "Beginning greedy selection of parents for 22\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6966 bits\n",
      "Selected source 20, with TE(20->22 | )=0.15059, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 24 was not significant (TE(24->22 | 20)=0.06366, p-value=0.149247), quitting\n",
      "Beginning greedy selection of parents for 23\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7016 bits\n",
      "Selected source 21, with TE(21->23 | )=0.15310, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 24, with TE(24->23 | 21)=0.06898, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 22 was not significant (TE(22->23 | 21,24)=0.06368, p-value=0.146416), quitting\n",
      "Beginning greedy selection of parents for 24\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7191 bits\n",
      "Selected source 22, with TE(22->24 | )=0.15717, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 26, with TE(26->24 | 22)=0.07524, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 8 was not significant (TE(8->24 | 22,26)=0.02772, p-value=0.999531), quitting\n",
      "Beginning greedy selection of parents for 25\n",
      "Optimal k=3, giving bias corrected AIS(k=3)=0.7198 bits\n",
      "Selected source 23, with TE(23->25 | )=0.19033, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 27, with TE(27->25 | 23)=0.07390, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 7 was not significant (TE(7->25 | 23,27)=0.01012, p-value=0.999452), quitting\n",
      "Beginning greedy selection of parents for 26\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7011 bits\n",
      "Selected source 24, with TE(24->26 | )=0.17819, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 28, with TE(28->26 | 24)=0.09103, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 49 was not significant (TE(49->26 | 24,28)=0.00782, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 27\n",
      "Optimal k=3, giving bias corrected AIS(k=3)=0.7190 bits\n",
      "Selected source 25, with TE(25->27 | )=0.16672, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 29, with TE(29->27 | 25)=0.09213, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 26 was not significant (TE(26->27 | 25,29)=0.01627, p-value=0.960508), quitting\n",
      "Beginning greedy selection of parents for 28\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7109 bits\n",
      "Selected source 26, with TE(26->28 | )=0.16551, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 30, with TE(30->28 | 26)=0.08512, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 29 was not significant (TE(29->28 | 26,30)=0.01896, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 29\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7181 bits\n",
      "Selected source 27, with TE(27->29 | )=0.15484, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 31, with TE(31->29 | 27)=0.10524, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 22 was not significant (TE(22->29 | 27,31)=0.00921, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 30\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7109 bits\n",
      "Selected source 28, with TE(28->30 | )=0.15783, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 32, with TE(32->30 | 28)=0.09955, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 39 was not significant (TE(39->30 | 28,32)=0.01262, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 31\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7147 bits\n",
      "Selected source 29, with TE(29->31 | )=0.15404, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 33, with TE(33->31 | 29)=0.09283, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 30 was not significant (TE(30->31 | 29,33)=0.01680, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 32\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7256 bits\n",
      "Selected source 30, with TE(30->32 | )=0.13371, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 34, with TE(34->32 | 30)=0.08275, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 23 was not significant (TE(23->32 | 30,34)=0.03166, p-value=0.996373), quitting\n",
      "Beginning greedy selection of parents for 33\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7148 bits\n",
      "Selected source 31, with TE(31->33 | )=0.13549, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 35, with TE(35->33 | 31)=0.09871, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 34 was not significant (TE(34->33 | 31,35)=0.02273, p-value=0.999986), quitting\n",
      "Beginning greedy selection of parents for 34\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7076 bits\n",
      "Selected source 32, with TE(32->34 | )=0.13525, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 36, with TE(36->34 | 32)=0.10019, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 35 was not significant (TE(35->34 | 32,36)=0.02611, p-value=0.999828), quitting\n",
      "Beginning greedy selection of parents for 35\n",
      "Optimal k=3, giving bias corrected AIS(k=3)=0.7201 bits\n",
      "Selected source 33, with TE(33->35 | )=0.13426, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 37, with TE(37->35 | 33)=0.10375, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 39 was not significant (TE(39->35 | 33,37)=0.02879, p-value=0.354675), quitting\n",
      "Beginning greedy selection of parents for 36\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7190 bits\n",
      "Selected source 34, with TE(34->36 | )=0.12593, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 38, with TE(38->36 | 34)=0.10314, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 37 was not significant (TE(37->36 | 34,38)=0.01725, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 37\n",
      "Optimal k=3, giving bias corrected AIS(k=3)=0.7242 bits\n",
      "Selected source 35, with TE(35->37 | )=0.14001, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 39, with TE(39->37 | 35)=0.10677, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 38 was not significant (TE(38->37 | 35,39)=0.01558, p-value=0.971557), quitting\n",
      "Beginning greedy selection of parents for 38\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6991 bits\n",
      "Selected source 36, with TE(36->38 | )=0.13679, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 40, with TE(40->38 | 36)=0.09422, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 37 was not significant (TE(37->38 | 36,40)=0.02958, p-value=0.998676), quitting\n",
      "Beginning greedy selection of parents for 39\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6966 bits\n",
      "Selected source 37, with TE(37->39 | )=0.12580, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 41, with TE(41->39 | 37)=0.13320, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 38 was not significant (TE(38->39 | 37,41)=0.02304, p-value=0.999982), quitting\n",
      "Beginning greedy selection of parents for 40\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6712 bits\n",
      "Selected source 38, with TE(38->40 | )=0.12011, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 42 was not significant (TE(42->40 | 38)=0.09898, p-value=0.000049), quitting\n",
      "Beginning greedy selection of parents for 41\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6697 bits\n",
      "Selected source 39, with TE(39->41 | )=0.13323, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 43 was not significant (TE(43->41 | 39)=0.09484, p-value=0.000160), quitting\n",
      "Beginning greedy selection of parents for 42\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6332 bits\n",
      "Selected source 40, with TE(40->42 | )=0.12817, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 44, with TE(44->42 | 40)=0.11252, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 46 was not significant (TE(46->42 | 40,44)=0.03880, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 43\n",
      "Optimal k=6, giving bias corrected AIS(k=6)=0.6187 bits\n",
      "Selected source 42, with TE(42->43 | )=0.11160, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 44, with TE(44->43 | 42)=0.21369, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->43 | 42,44)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 44\n",
      "Optimal k=6, giving bias corrected AIS(k=6)=0.6381 bits\n",
      "Selected source 42, with TE(42->44 | )=0.15287, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 45 was not significant (TE(45->44 | 42)=0.07258, p-value=0.998266), quitting\n",
      "Beginning greedy selection of parents for 45\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6240 bits\n",
      "Selected source 43, with TE(43->45 | )=0.12943, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 46 was not significant (TE(46->45 | 43)=0.10099, p-value=0.000027), quitting\n",
      "Beginning greedy selection of parents for 46\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6111 bits\n",
      "Selected source 45, with TE(45->46 | )=0.12629, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 47, with TE(47->46 | 45)=0.23314, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->46 | 45,47)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 47\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6174 bits\n",
      "Selected source 45, with TE(45->47 | )=0.12637, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 48, with TE(48->47 | 45)=0.10740, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 46 was not significant (TE(46->47 | 45,48)=0.12167, p-value=0.147287), quitting\n",
      "Beginning greedy selection of parents for 48\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6537 bits\n",
      "Selected source 46, with TE(46->48 | )=0.12709, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 49, with TE(49->48 | 46)=0.11844, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 47 was not significant (TE(47->48 | 46,49)=0.08805, p-value=0.000949), quitting\n",
      "Beginning greedy selection of parents for 49\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6397 bits\n",
      "Selected source 47, with TE(47->49 | )=0.11005, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 50, with TE(50->49 | 47)=0.12019, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 48 was not significant (TE(48->49 | 47,50)=0.10393, p-value=0.591487), quitting\n",
      "Beginning greedy selection of parents for 50\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6280 bits\n",
      "Selected source 48, with TE(48->50 | )=0.12994, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 51, with TE(51->50 | 48)=0.10482, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 49, with TE(49->50 | 48,51)=0.12437, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->50 | 48,51,49)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 51\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6286 bits\n",
      "Selected source 49, with TE(49->51 | )=0.11889, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 53, with TE(53->51 | 49)=0.11589, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 50 was not significant (TE(50->51 | 49,53)=0.04865, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 52\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6143 bits\n",
      "Selected source 51, with TE(51->52 | )=0.12556, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 53, with TE(53->52 | 51)=0.23132, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->52 | 51,53)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 53\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6208 bits\n",
      "Selected source 52, with TE(52->53 | )=0.11609, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 54, with TE(54->53 | 52)=0.23556, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->53 | 52,54)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 54\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6187 bits\n",
      "Selected source 53, with TE(53->54 | )=0.11340, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 55, with TE(55->54 | 53)=0.23985, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->54 | 53,55)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 55\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6038 bits\n",
      "Selected source 54, with TE(54->55 | )=0.12172, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 56, with TE(56->55 | 54)=0.24592, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->55 | 54,56)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 56\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6078 bits\n",
      "Selected source 55, with TE(55->56 | )=0.11873, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 57, with TE(57->56 | 55)=0.24465, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->56 | 55,57)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 57\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6142 bits\n",
      "Selected source 56, with TE(56->57 | )=0.12496, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 58, with TE(58->57 | 56)=0.23027, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->57 | 56,58)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 58\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6248 bits\n",
      "Selected source 57, with TE(57->58 | )=0.11080, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 59, with TE(59->58 | 57)=0.23613, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->58 | 57,59)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 59\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6126 bits\n",
      "Selected source 58, with TE(58->59 | )=0.12738, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 60, with TE(60->59 | 58)=0.23203, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->59 | 58,60)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 60\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6158 bits\n",
      "Selected source 59, with TE(59->60 | )=0.13188, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 61, with TE(61->60 | 59)=0.22056, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->60 | 59,61)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 61\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6353 bits\n",
      "Selected source 60, with TE(60->61 | )=0.12045, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 62, with TE(62->61 | 60)=0.21675, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->61 | 60,62)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 62\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6316 bits\n",
      "Selected source 61, with TE(61->62 | )=0.10600, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 63, with TE(63->62 | 61)=0.23463, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->62 | 61,63)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 63\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6430 bits\n",
      "Selected source 62, with TE(62->63 | )=0.12431, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 64, with TE(64->63 | 62)=0.20409, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->63 | 62,64)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 64\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6368 bits\n",
      "Selected source 63, with TE(63->64 | )=0.11987, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 65, with TE(65->64 | 63)=0.21017, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->64 | 63,65)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 65\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6436 bits\n",
      "Selected source 64, with TE(64->65 | )=0.12212, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 66, with TE(66->65 | 64)=0.20336, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->65 | 64,66)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 66\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6755 bits\n",
      "Selected source 65, with TE(65->66 | )=0.10722, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 67, with TE(67->66 | 65)=0.19044, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->66 | 65,67)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 67\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6890 bits\n",
      "Selected source 66, with TE(66->67 | )=0.11194, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 68, with TE(68->67 | 66)=0.17258, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->67 | 66,68)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 68\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6844 bits\n",
      "Selected source 66, with TE(66->68 | )=0.12962, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 69 was not significant (TE(69->68 | 66)=0.07536, p-value=0.018816), quitting\n",
      "Beginning greedy selection of parents for 69\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6773 bits\n",
      "Selected source 67, with TE(67->69 | )=0.14957, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 70 was not significant (TE(70->69 | 67)=0.06070, p-value=0.224626), quitting\n",
      "Beginning greedy selection of parents for 70\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6652 bits\n",
      "Selected source 68, with TE(68->70 | )=0.14417, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 71, with TE(71->70 | 68)=0.08040, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 69 was not significant (TE(69->70 | 68,71)=0.09713, p-value=0.000080), quitting\n",
      "Beginning greedy selection of parents for 71\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6655 bits\n",
      "Selected source 69, with TE(69->71 | )=0.14858, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 72 was not significant (TE(72->71 | 69)=0.07301, p-value=0.030104), quitting\n",
      "Beginning greedy selection of parents for 72\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6746 bits\n",
      "Selected source 70, with TE(70->72 | )=0.13770, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 73 was not significant (TE(73->72 | 70)=0.06895, p-value=0.063788), quitting\n",
      "Beginning greedy selection of parents for 73\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6931 bits\n",
      "Selected source 72, with TE(72->73 | )=0.11749, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 74, with TE(74->73 | 72)=0.16310, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->73 | 72,74)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 74\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6895 bits\n",
      "Selected source 72, with TE(72->74 | )=0.11845, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 75 was not significant (TE(75->74 | 72)=0.07940, p-value=0.007908), quitting\n",
      "Beginning greedy selection of parents for 75\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.7029 bits\n",
      "Selected source 73, with TE(73->75 | )=0.12306, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 77 was not significant (TE(77->75 | 73)=0.06046, p-value=0.231637), quitting\n",
      "Beginning greedy selection of parents for 76\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6987 bits\n",
      "Selected source 74, with TE(74->76 | )=0.14415, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 77 was not significant (TE(77->76 | 74)=0.06255, p-value=0.175025), quitting\n",
      "Beginning greedy selection of parents for 77\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.7160 bits\n",
      "Selected source 75, with TE(75->77 | )=0.14867, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 78 was not significant (TE(78->77 | 75)=0.03722, p-value=0.971371), quitting\n",
      "Beginning greedy selection of parents for 78\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.7089 bits\n",
      "Selected source 76, with TE(76->78 | )=0.14661, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 79 was not significant (TE(79->78 | 76)=0.03583, p-value=0.981741), quitting\n",
      "Beginning greedy selection of parents for 79\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.7151 bits\n",
      "Selected source 77, with TE(77->79 | )=0.15474, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 80 was not significant (TE(80->79 | 77)=0.04640, p-value=0.775017), quitting\n",
      "Beginning greedy selection of parents for 80\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.7244 bits\n",
      "Selected source 78, with TE(78->80 | )=0.15919, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 81 was not significant (TE(81->80 | 78)=0.04549, p-value=0.805648), quitting\n",
      "Beginning greedy selection of parents for 81\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7582 bits\n",
      "Selected source 79, with TE(79->81 | )=0.16130, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 82 was not significant (TE(82->81 | 79)=0.06457, p-value=0.000014), quitting\n",
      "Beginning greedy selection of parents for 82\n",
      "Optimal k=3, giving bias corrected AIS(k=3)=0.7570 bits\n",
      "Selected source 80, with TE(80->82 | )=0.15369, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 83, with TE(83->82 | 80)=0.05854, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 81 was not significant (TE(81->82 | 80,83)=0.02490, p-value=0.582148), quitting\n",
      "Beginning greedy selection of parents for 83\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7402 bits\n",
      "Selected source 81, with TE(81->83 | )=0.18840, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 84 was not significant (TE(84->83 | 81)=0.03140, p-value=0.233831), quitting\n",
      "Beginning greedy selection of parents for 84\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7311 bits\n",
      "Selected source 82, with TE(82->84 | )=0.19803, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 86 was not significant (TE(86->84 | 82)=0.02681, p-value=0.469007), quitting\n",
      "Beginning greedy selection of parents for 85\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7330 bits\n",
      "Selected source 83, with TE(83->85 | )=0.20001, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 86 was not significant (TE(86->85 | 83)=0.03313, p-value=0.170082), quitting\n",
      "Beginning greedy selection of parents for 86\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.7321 bits\n",
      "Selected source 84, with TE(84->86 | )=0.19506, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 88 was not significant (TE(88->86 | 84)=0.02863, p-value=0.999228), quitting\n",
      "Beginning greedy selection of parents for 87\n",
      "Optimal k=3, giving bias corrected AIS(k=3)=0.7553 bits\n",
      "Selected source 85, with TE(85->87 | )=0.14877, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 89 was not significant (TE(89->87 | 85)=0.03667, p-value=0.000212), quitting\n",
      "Beginning greedy selection of parents for 88\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7416 bits\n",
      "Selected source 86, with TE(86->88 | )=0.16400, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 89 was not significant (TE(89->88 | 86)=0.06256, p-value=0.000030), quitting\n",
      "Beginning greedy selection of parents for 89\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7332 bits\n",
      "Selected source 87, with TE(87->89 | )=0.18001, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 91 was not significant (TE(91->89 | 87)=0.04701, p-value=0.005378), quitting\n",
      "Beginning greedy selection of parents for 90\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7454 bits\n",
      "Selected source 88, with TE(88->90 | )=0.17941, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 92 was not significant (TE(92->90 | 88)=0.04332, p-value=0.015417), quitting\n",
      "Beginning greedy selection of parents for 91\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7346 bits\n",
      "Selected source 89, with TE(89->91 | )=0.17080, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 93 was not significant (TE(93->91 | 89)=0.05793, p-value=0.000156), quitting\n",
      "Beginning greedy selection of parents for 92\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7299 bits\n",
      "Selected source 90, with TE(90->92 | )=0.16960, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 94 was not significant (TE(94->92 | 90)=0.06661, p-value=0.000007), quitting\n",
      "Beginning greedy selection of parents for 93\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7217 bits\n",
      "Selected source 91, with TE(91->93 | )=0.16975, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 95 was not significant (TE(95->93 | 91)=0.06463, p-value=0.000014), quitting\n",
      "Beginning greedy selection of parents for 94\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7151 bits\n",
      "Selected source 92, with TE(92->94 | )=0.16190, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 96, with TE(96->94 | 92)=0.06737, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 76 was not significant (TE(76->94 | 92,96)=0.02476, p-value=0.999933), quitting\n",
      "Beginning greedy selection of parents for 95\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7154 bits\n",
      "Selected source 93, with TE(93->95 | )=0.12975, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 97, with TE(97->95 | 93)=0.08925, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 94 was not significant (TE(94->95 | 93,97)=0.03155, p-value=0.996555), quitting\n",
      "Beginning greedy selection of parents for 96\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7125 bits\n",
      "Selected source 94, with TE(94->96 | )=0.11606, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 97, with TE(97->96 | 94)=0.08777, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 95 was not significant (TE(95->96 | 94,97)=0.07105, p-value=0.042711), quitting\n",
      "Beginning greedy selection of parents for 97\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.7026 bits\n",
      "Selected source 95, with TE(95->97 | )=0.12327, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 99, with TE(99->97 | 95)=0.09932, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 96 was not significant (TE(96->97 | 95,99)=0.02781, p-value=0.999503), quitting\n",
      "Beginning greedy selection of parents for 98\n",
      "Optimal k=4, giving bias corrected AIS(k=4)=0.6782 bits\n",
      "Selected source 96, with TE(96->98 | )=0.12891, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 99, with TE(99->98 | 96)=0.07989, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 97 was not significant (TE(97->98 | 96,99)=0.10021, p-value=0.000032), quitting\n",
      "Beginning greedy selection of parents for 99\n",
      "Optimal k=5, giving bias corrected AIS(k=5)=0.6631 bits\n",
      "Selected source 97, with TE(97->99 | )=0.10758, p-value=0.00000 (conditioning on 0 parents)\n",
      "-- Max TE source 98 was not significant (TE(98->99 | 97)=0.03670, p-value=0.975665), quitting\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 28320 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "##########\n",
    "# Option 1 -- attempting optimising embedding length selection\n",
    "##########\n",
    "\n",
    "network = numpy.zeros((networkSize,networkSize), dtype=int);\n",
    "# Compute for all targets:\n",
    "for d in range(networkSize):\n",
    "\n",
    "    print('Beginning greedy selection of parents for %d' % d);\n",
    "\n",
    "    destination = JArray(JInt, 1)(data[:, d].tolist())\n",
    "    conditionalSet = [];\n",
    "\n",
    "    #######################\n",
    "    # Let's optimise the selection of the embedding length for the target, like we did for AIS in module 10:\n",
    "    kMax = 20;\n",
    "    biasCorrectedAisResults = numpy.zeros((kMax))\n",
    "    for k in range(1,kMax+1):\n",
    "        # 1. Construct the calculator:\n",
    "        aisCalcClass = JPackage(\"infodynamics.measures.discrete\").ActiveInformationCalculatorDiscrete\n",
    "        aisCalc = aisCalcClass(2, k)\n",
    "        # 2. No other properties to set for discrete calculators.\n",
    "        # 3. Initialise the calculator for (re-)use:\n",
    "        aisCalc.initialise()\n",
    "        # 4. Supply the sample data:\n",
    "        aisCalc.addObservations(destination)\n",
    "        # 5. Compute the estimate:\n",
    "        ais = aisCalc.computeAverageLocalOfObservations()\n",
    "        # 6. Compute the (statistical significance via) null distribution analytically:\n",
    "        measDistAis = aisCalc.computeSignificance()\n",
    "        biasAis = measDistAis.getMeanOfDistribution()\n",
    "        biasCorrectedAisResults[k-1] = ais - biasAis\n",
    "    optimalK = numpy.argmax(biasCorrectedAisResults)+1 # Need to add 1 for the array offset\n",
    "    print('Optimal k=%d, giving bias corrected AIS(k=%d)=%.4f bits' % (optimalK,optimalK,biasCorrectedAisResults[optimalK-1])) # subtract 1 for array indexing\n",
    "    \n",
    "    while (True):\n",
    "        # Precondition: we have already selected the parents in\n",
    "        #  conditionalSet, now we check if we can add to this:\n",
    "        \n",
    "        results = numpy.zeros((networkSize));\n",
    "        pValues = numpy.zeros((networkSize));\n",
    "\n",
    "        # 1. Construct the calculator:\n",
    "        if (len(conditionalSet) > 0):\n",
    "            calcClass = JPackage(\"infodynamics.measures.discrete\").ConditionalTransferEntropyCalculatorDiscrete\n",
    "            calc = calcClass(2, optimalK, len(conditionalSet))\n",
    "        else:\n",
    "            calcClass = JPackage(\"infodynamics.measures.discrete\").TransferEntropyCalculatorDiscrete\n",
    "            calc = calcClass(2, optimalK, 1, 1, 1, 1)\n",
    "        # 2. No other properties to set for discrete calculators.\n",
    "        for s in range(networkSize):\n",
    "            # For each source-dest pair:\n",
    "            \n",
    "            if ( (s == d) or (s in conditionalSet) ):\n",
    "                # If s is the target, or already in the conditioning set, skip evaluating TE from it\n",
    "                pValues[s] = 1;\n",
    "                continue\n",
    "            source = JArray(JInt, 1)(data[:, s].tolist())\n",
    "            conditional = JArray(JInt, 2)(data[:, conditionalSet].tolist())\n",
    "\n",
    "            # 3. Initialise the calculator for (re-)use:\n",
    "            calc.initialise()\n",
    "            # 4. Supply the sample data:\n",
    "            if (len(conditionalSet) > 0):\n",
    "                calc.addObservations(source, destination, conditional)\n",
    "            else:\n",
    "                calc.addObservations(source, destination)\n",
    "\n",
    "            # 5. Compute the estimate:\n",
    "            result = calc.computeAverageLocalOfObservations()\n",
    "            # 6. Compute the (statistical significance via) null distribution analytically:\n",
    "            measDist = calc.computeSignificance()\n",
    "\n",
    "            results[s] = result;\n",
    "            pValues[s] = measDist.pValue;\n",
    "\n",
    "        # Check which was the strongest source:\n",
    "        maxSourceIndex = numpy.argmax(results);\n",
    "        maxTE = results[maxSourceIndex];\n",
    "        if (pValues[maxSourceIndex] < 0.05/(networkSize*(networkSize-1))):\n",
    "            print('Selected source %d, with TE(%d->%d | %s)=%.5f, p-value=%.5f (conditioning on %d parents)' %\\\n",
    "                (maxSourceIndex, maxSourceIndex, d, (','.join(str(x) for x in conditionalSet)), maxTE, \\\n",
    "                pValues[maxSourceIndex], len(conditionalSet) ));\n",
    "            # Add this new source:\n",
    "            conditionalSet.append(maxSourceIndex);\n",
    "        else:\n",
    "            print('-- Max TE source %d was not significant (TE(%d->%d | %s)=%.5f, p-value=%.6f), quitting' %\\\n",
    "                (maxSourceIndex, maxSourceIndex, d, (','.join(str(x) for x in conditionalSet)), maxTE, pValues[maxSourceIndex]));\n",
    "            break\n",
    "    \n",
    "    # Postcondition: conditionalSet holds the parents for target d\n",
    "    network[conditionalSet, d] = 1;\n",
    "\n",
    "# Plot the effective connectivity\n",
    "plt.figure()\n",
    "plt.imshow(network)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('Multivariate effective network via TE(k optimal)');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('Connections');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0af11be4-7a4e-4caa-8c2b-0d3ead573113",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Beginning greedy selection of parents for 0\n",
      "Selected source 1, with TE(1->0 | )=0.27248, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 2, with TE(2->0 | 1)=0.17113, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 26, with TE(26->0 | 1,2)=0.06543, p-value=0.00000 (conditioning on 2 parents)\n",
      "Selected source 46, with TE(46->0 | 1,2,26)=0.05114, p-value=0.00000 (conditioning on 3 parents)\n",
      "-- Max TE source 7 was not significant (TE(7->0 | 1,2,26,46)=0.04778, p-value=0.003977), quitting\n",
      "Beginning greedy selection of parents for 1\n",
      "Selected source 0, with TE(0->1 | )=0.37738, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 2, with TE(2->1 | 0)=0.50355, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->1 | 0,2)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 2\n",
      "Selected source 1, with TE(1->2 | )=0.27738, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 3, with TE(3->2 | 1)=0.50802, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->2 | 1,3)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 3\n",
      "Selected source 2, with TE(2->3 | )=0.38842, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 4, with TE(4->3 | 2)=0.49992, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->3 | 2,4)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 4\n",
      "Selected source 5, with TE(5->4 | )=0.27112, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 3, with TE(3->4 | 5)=0.50853, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->4 | 5,3)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 5\n",
      "Selected source 4, with TE(4->5 | )=0.38022, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 6, with TE(6->5 | 4)=0.50266, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->5 | 4,6)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 6\n",
      "Selected source 7, with TE(7->6 | )=0.27094, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 5, with TE(5->6 | 7)=0.50870, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->6 | 7,5)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 7\n",
      "Selected source 8, with TE(8->7 | )=0.38147, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 6, with TE(6->7 | 8)=0.49916, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->7 | 8,6)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 8\n",
      "Selected source 9, with TE(9->8 | )=0.27445, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 7, with TE(7->8 | 9)=0.50600, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->8 | 9,7)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 9\n",
      "Selected source 8, with TE(8->9 | )=0.37041, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 10, with TE(10->9 | 8)=0.50245, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->9 | 8,10)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 10\n",
      "Selected source 9, with TE(9->10 | )=0.27927, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 11, with TE(11->10 | 9)=0.50692, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->10 | 9,11)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 11\n",
      "Selected source 10, with TE(10->11 | )=0.37547, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 12, with TE(12->11 | 10)=0.50070, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->11 | 10,12)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 12\n",
      "Selected source 13, with TE(13->12 | )=0.27907, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 11, with TE(11->12 | 13)=0.51154, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->12 | 13,11)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 13\n",
      "Selected source 14, with TE(14->13 | )=0.37413, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 12, with TE(12->13 | 14)=0.50436, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->13 | 14,12)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 14\n",
      "Selected source 15, with TE(15->14 | )=0.28789, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 13, with TE(13->14 | 15)=0.51122, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->14 | 15,13)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 15\n",
      "Selected source 14, with TE(14->15 | )=0.36312, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 16, with TE(16->15 | 14)=0.50727, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->15 | 14,16)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 16\n",
      "Selected source 17, with TE(17->16 | )=0.29516, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 15, with TE(15->16 | 17)=0.51144, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->16 | 17,15)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 17\n",
      "Selected source 18, with TE(18->17 | )=0.36780, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 16, with TE(16->17 | 18)=0.50450, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->17 | 18,16)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 18\n",
      "Selected source 19, with TE(19->18 | )=0.28256, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 17, with TE(17->18 | 19)=0.50892, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->18 | 19,17)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 19\n",
      "Selected source 20, with TE(20->19 | )=0.36186, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 18, with TE(18->19 | 20)=0.50449, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->19 | 20,18)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 20\n",
      "Selected source 19, with TE(19->20 | )=0.28277, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 21, with TE(21->20 | 19)=0.50906, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->20 | 19,21)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 21\n",
      "Selected source 22, with TE(22->21 | )=0.36805, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 20, with TE(20->21 | 22)=0.49829, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->21 | 22,20)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 22\n",
      "Selected source 23, with TE(23->22 | )=0.27836, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 21, with TE(21->22 | 23)=0.49986, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->22 | 23,21)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 23\n",
      "Selected source 24, with TE(24->23 | )=0.36614, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 22, with TE(22->23 | 24)=0.49434, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->23 | 24,22)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 24\n",
      "Selected source 25, with TE(25->24 | )=0.27014, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 23, with TE(23->24 | 25)=0.49670, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->24 | 25,23)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 25\n",
      "Selected source 24, with TE(24->25 | )=0.35762, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 26, with TE(26->25 | 24)=0.49618, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->25 | 24,26)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 26\n",
      "Selected source 27, with TE(27->26 | )=0.27766, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 25, with TE(25->26 | 27)=0.49805, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->26 | 27,25)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 27\n",
      "Selected source 28, with TE(28->27 | )=0.35481, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 26, with TE(26->27 | 28)=0.49670, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->27 | 28,26)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 28\n",
      "Selected source 27, with TE(27->28 | )=0.27512, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 29, with TE(29->28 | 27)=0.49766, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->28 | 27,29)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 29\n",
      "Selected source 28, with TE(28->29 | )=0.35762, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 30, with TE(30->29 | 28)=0.49618, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->29 | 28,30)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 30\n",
      "Selected source 29, with TE(29->30 | )=0.27512, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 31, with TE(31->30 | 29)=0.49766, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->30 | 29,31)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 31\n",
      "Selected source 30, with TE(30->31 | )=0.36045, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 32, with TE(32->31 | 30)=0.49562, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->31 | 30,32)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 32\n",
      "Selected source 31, with TE(31->32 | )=0.27119, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 33, with TE(33->32 | 31)=0.49861, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->32 | 31,33)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 33\n",
      "Selected source 32, with TE(32->33 | )=0.36129, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 34, with TE(34->33 | 32)=0.49701, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->33 | 32,34)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 34\n",
      "Selected source 35, with TE(35->34 | )=0.28268, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 33, with TE(33->34 | 35)=0.49873, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->34 | 35,33)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 35\n",
      "Selected source 36, with TE(36->35 | )=0.35199, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 34, with TE(34->35 | 36)=0.49719, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->35 | 36,34)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 36\n",
      "Selected source 37, with TE(37->36 | )=0.27666, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 35, with TE(35->36 | 37)=0.49613, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->36 | 37,35)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 37\n",
      "Selected source 36, with TE(36->37 | )=0.34637, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 38, with TE(38->37 | 36)=0.49812, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->37 | 36,38)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 38\n",
      "Selected source 37, with TE(37->38 | )=0.29689, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 39, with TE(39->38 | 37)=0.49857, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->38 | 37,39)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 39\n",
      "Selected source 38, with TE(38->39 | )=0.33985, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 40, with TE(40->39 | 38)=0.50224, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->39 | 38,40)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 40\n",
      "Selected source 39, with TE(39->40 | )=0.33250, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 41, with TE(41->40 | 39)=0.50034, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->40 | 39,41)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 41\n",
      "Selected source 40, with TE(40->41 | )=0.31742, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 42, with TE(42->41 | 40)=0.51496, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->41 | 40,42)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 42\n",
      "Selected source 41, with TE(41->42 | )=0.36585, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 43, with TE(43->42 | 41)=0.50910, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->42 | 41,43)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 43\n",
      "Selected source 42, with TE(42->43 | )=0.31459, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 44, with TE(44->43 | 42)=0.51579, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->43 | 42,44)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 44\n",
      "Selected source 45, with TE(45->44 | )=0.35632, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 43, with TE(43->44 | 45)=0.51836, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->44 | 45,43)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 45\n",
      "Selected source 44, with TE(44->45 | )=0.32582, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 46, with TE(46->45 | 44)=0.52571, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->45 | 44,46)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 46\n",
      "Selected source 47, with TE(47->46 | )=0.36487, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 45, with TE(45->46 | 47)=0.50696, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->46 | 47,45)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 47\n",
      "Selected source 48, with TE(48->47 | )=0.33650, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 46, with TE(46->47 | 48)=0.50289, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->47 | 48,46)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 48\n",
      "Selected source 49, with TE(49->48 | )=0.33771, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 47, with TE(47->48 | 49)=0.50402, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->48 | 49,47)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 49\n",
      "Selected source 48, with TE(48->49 | )=0.30329, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 50, with TE(50->49 | 48)=0.50570, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->49 | 48,50)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 50\n",
      "Selected source 49, with TE(49->50 | )=0.33491, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 51, with TE(51->50 | 49)=0.50414, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->50 | 49,51)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 51\n",
      "Selected source 50, with TE(50->51 | )=0.32800, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 52, with TE(52->51 | 50)=0.50815, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->51 | 50,52)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 52\n",
      "Selected source 51, with TE(51->52 | )=0.36089, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 53, with TE(53->52 | 51)=0.50113, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->52 | 51,53)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 53\n",
      "Selected source 54, with TE(54->53 | )=0.32834, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 52, with TE(52->53 | 54)=0.52379, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->53 | 54,52)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 54\n",
      "Selected source 53, with TE(53->54 | )=0.34284, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 55, with TE(55->54 | 53)=0.51955, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->54 | 53,55)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 55\n",
      "Selected source 54, with TE(54->55 | )=0.32938, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 56, with TE(56->55 | 54)=0.52335, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->55 | 54,56)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 56\n",
      "Selected source 55, with TE(55->56 | )=0.36715, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 57, with TE(57->56 | 55)=0.51667, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->56 | 55,57)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 57\n",
      "Selected source 58, with TE(58->57 | )=0.33639, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 56, with TE(56->57 | 58)=0.52064, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->57 | 58,56)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 58\n",
      "Selected source 59, with TE(59->58 | )=0.35103, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 57, with TE(57->58 | 59)=0.52108, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->58 | 59,57)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 59\n",
      "Selected source 58, with TE(58->59 | )=0.34866, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 60, with TE(60->59 | 58)=0.52179, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->59 | 58,60)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 60\n",
      "Selected source 61, with TE(61->60 | )=0.35550, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 59, with TE(59->60 | 61)=0.51328, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->60 | 61,59)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 61\n",
      "Selected source 62, with TE(62->61 | )=0.37021, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 60, with TE(60->61 | 62)=0.51221, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->61 | 62,60)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 62\n",
      "Selected source 61, with TE(61->62 | )=0.29466, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 63, with TE(63->62 | 61)=0.52297, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->62 | 61,63)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 63\n",
      "Selected source 62, with TE(62->63 | )=0.39648, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 64, with TE(64->63 | 62)=0.50277, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->63 | 62,64)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 64\n",
      "Selected source 63, with TE(63->64 | )=0.33048, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 65, with TE(65->64 | 63)=0.52044, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->64 | 63,65)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 65\n",
      "Selected source 66, with TE(66->65 | )=0.39981, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 64, with TE(64->65 | 66)=0.49118, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->65 | 66,64)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 66\n",
      "Selected source 67, with TE(67->66 | )=0.31855, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 65, with TE(65->66 | 67)=0.51194, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->66 | 67,65)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 67\n",
      "Selected source 68, with TE(68->67 | )=0.32525, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 66, with TE(66->67 | 68)=0.51323, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->67 | 68,66)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 68\n",
      "Selected source 67, with TE(67->68 | )=0.31908, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 69, with TE(69->68 | 67)=0.51193, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->68 | 67,69)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 69\n",
      "Selected source 70, with TE(70->69 | )=0.33742, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 68, with TE(68->69 | 70)=0.51024, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->69 | 70,68)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 70\n",
      "Selected source 69, with TE(69->70 | )=0.31716, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 71, with TE(71->70 | 69)=0.51373, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->70 | 69,71)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 71\n",
      "Selected source 70, with TE(70->71 | )=0.34273, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 72, with TE(72->71 | 70)=0.50957, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->71 | 70,72)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 72\n",
      "Selected source 73, with TE(73->72 | )=0.32779, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 71, with TE(71->72 | 73)=0.50901, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->72 | 73,71)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 73\n",
      "Selected source 74, with TE(74->73 | )=0.33979, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 72, with TE(72->73 | 74)=0.50850, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->73 | 74,72)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 74\n",
      "Selected source 75, with TE(75->74 | )=0.31392, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 73, with TE(73->74 | 75)=0.50721, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->74 | 75,73)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 75\n",
      "Selected source 74, with TE(74->75 | )=0.33478, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 76, with TE(76->75 | 74)=0.50927, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->75 | 74,76)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 76\n",
      "Selected source 75, with TE(75->76 | )=0.30378, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 77, with TE(77->76 | 75)=0.50703, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->76 | 75,77)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 77\n",
      "Selected source 76, with TE(76->77 | )=0.34331, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 78, with TE(78->77 | 76)=0.50962, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->77 | 76,78)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 78\n",
      "Selected source 77, with TE(77->78 | )=0.34533, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 79, with TE(79->78 | 77)=0.50530, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->78 | 77,79)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 79\n",
      "Selected source 80, with TE(80->79 | )=0.36884, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 78, with TE(78->79 | 80)=0.50374, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->79 | 80,78)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 80\n",
      "Selected source 81, with TE(81->80 | )=0.34737, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 79, with TE(79->80 | 81)=0.49474, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->80 | 81,79)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 81\n",
      "Selected source 82, with TE(82->81 | )=0.33260, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 80, with TE(80->81 | 82)=0.49979, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->81 | 82,80)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 82\n",
      "Selected source 81, with TE(81->82 | )=0.29183, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 83, with TE(83->82 | 81)=0.49825, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->82 | 81,83)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 83\n",
      "Selected source 82, with TE(82->83 | )=0.34073, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 84, with TE(84->83 | 82)=0.49884, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->83 | 82,84)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 84\n",
      "Selected source 85, with TE(85->84 | )=0.29026, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 83, with TE(83->84 | 85)=0.49974, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->84 | 85,83)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 85\n",
      "Selected source 84, with TE(84->85 | )=0.33897, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 86, with TE(86->85 | 84)=0.50073, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->85 | 84,86)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 86\n",
      "Selected source 85, with TE(85->86 | )=0.30560, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 87, with TE(87->86 | 85)=0.50036, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->86 | 85,87)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 87\n",
      "Selected source 88, with TE(88->87 | )=0.33350, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 86, with TE(86->87 | 88)=0.50134, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->87 | 88,86)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 88\n",
      "Selected source 87, with TE(87->88 | )=0.28777, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 89, with TE(89->88 | 87)=0.49946, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->88 | 87,89)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 89\n",
      "Selected source 90, with TE(90->89 | )=0.34915, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 88, with TE(88->89 | 90)=0.49767, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->89 | 90,88)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 90\n",
      "Selected source 89, with TE(89->90 | )=0.27764, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 91, with TE(91->90 | 89)=0.49812, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->90 | 89,91)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 91\n",
      "Selected source 92, with TE(92->91 | )=0.34917, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 90, with TE(90->91 | 92)=0.49767, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->91 | 92,90)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 92\n",
      "Selected source 93, with TE(93->92 | )=0.28012, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 91, with TE(91->92 | 93)=0.49852, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->92 | 93,91)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 93\n",
      "Selected source 92, with TE(92->93 | )=0.35476, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 94, with TE(94->93 | 92)=0.49672, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->93 | 92,94)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 94\n",
      "Selected source 93, with TE(93->94 | )=0.29015, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 95, with TE(95->94 | 93)=0.49952, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->94 | 93,95)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 95\n",
      "Selected source 96, with TE(96->95 | )=0.36414, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 94, with TE(94->95 | 96)=0.49638, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->95 | 96,94)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 96\n",
      "Selected source 97, with TE(97->96 | )=0.27358, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 95, with TE(95->96 | 97)=0.49912, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->96 | 97,95)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 97\n",
      "Selected source 96, with TE(96->97 | )=0.37279, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 98, with TE(98->97 | 96)=0.49435, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->97 | 96,98)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 98\n",
      "Selected source 97, with TE(97->98 | )=0.28095, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 99, with TE(99->98 | 97)=0.49982, p-value=0.00000 (conditioning on 1 parents)\n",
      "-- Max TE source 0 was not significant (TE(0->98 | 97,99)=0.00000, p-value=1.000000), quitting\n",
      "Beginning greedy selection of parents for 99\n",
      "Selected source 98, with TE(98->99 | )=0.39472, p-value=0.00000 (conditioning on 0 parents)\n",
      "Selected source 97, with TE(97->99 | 98)=0.20780, p-value=0.00000 (conditioning on 1 parents)\n",
      "Selected source 35, with TE(35->99 | 98,97)=0.05974, p-value=0.00000 (conditioning on 2 parents)\n",
      "-- Max TE source 31 was not significant (TE(31->99 | 98,97,35)=0.03076, p-value=0.002200), quitting\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 27360 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "##########\n",
    "# Option 2 -- hard-coding embedding length at 1 to give a model closer to causal structure\n",
    "##########\n",
    "\n",
    "network = numpy.zeros((networkSize,networkSize), dtype=int);\n",
    "# Compute for all targets:\n",
    "for d in range(networkSize):\n",
    "\n",
    "    print('Beginning greedy selection of parents for %d' % d);\n",
    "\n",
    "    destination = JArray(JInt, 1)(data[:, d].tolist())\n",
    "    conditionalSet = [];\n",
    "\n",
    "    #######################\n",
    "    optimalK = 1\n",
    "    \n",
    "    while (True):\n",
    "        # Precondition: we have already selected the parents in\n",
    "        #  conditionalSet, now we check if we can add to this:\n",
    "        \n",
    "        results = numpy.zeros((networkSize));\n",
    "        pValues = numpy.zeros((networkSize));\n",
    "\n",
    "        # 1. Construct the calculator:\n",
    "        if (len(conditionalSet) > 0):\n",
    "            calcClass = JPackage(\"infodynamics.measures.discrete\").ConditionalTransferEntropyCalculatorDiscrete\n",
    "            calc = calcClass(2, optimalK, len(conditionalSet))\n",
    "        else:\n",
    "            calcClass = JPackage(\"infodynamics.measures.discrete\").TransferEntropyCalculatorDiscrete\n",
    "            calc = calcClass(2, optimalK, 1, 1, 1, 1)\n",
    "        # 2. No other properties to set for discrete calculators.\n",
    "        for s in range(networkSize):\n",
    "            # For each source-dest pair:\n",
    "            \n",
    "            if ( (s == d) or (s in conditionalSet) ):\n",
    "                # If s is the target, or already in the conditioning set, skip evaluating TE from it\n",
    "                pValues[s] = 1;\n",
    "                continue\n",
    "            source = JArray(JInt, 1)(data[:, s].tolist())\n",
    "            conditional = JArray(JInt, 2)(data[:, conditionalSet].tolist())\n",
    "\n",
    "            # 3. Initialise the calculator for (re-)use:\n",
    "            calc.initialise()\n",
    "            # 4. Supply the sample data:\n",
    "            if (len(conditionalSet) > 0):\n",
    "                calc.addObservations(source, destination, conditional)\n",
    "            else:\n",
    "                calc.addObservations(source, destination)\n",
    "\n",
    "            # 5. Compute the estimate:\n",
    "            result = calc.computeAverageLocalOfObservations()\n",
    "            # 6. Compute the (statistical significance via) null distribution analytically:\n",
    "            measDist = calc.computeSignificance()\n",
    "\n",
    "            results[s] = result;\n",
    "            pValues[s] = measDist.pValue;\n",
    "\n",
    "        # Check which was the strongest source:\n",
    "        maxSourceIndex = numpy.argmax(results);\n",
    "        maxTE = results[maxSourceIndex];\n",
    "        if (pValues[maxSourceIndex] < 0.05/(networkSize*(networkSize-1))):\n",
    "            print('Selected source %d, with TE(%d->%d | %s)=%.5f, p-value=%.5f (conditioning on %d parents)' %\\\n",
    "                (maxSourceIndex, maxSourceIndex, d, (','.join(str(x) for x in conditionalSet)), maxTE, \\\n",
    "                pValues[maxSourceIndex], len(conditionalSet) ));\n",
    "            # Add this new source:\n",
    "            conditionalSet.append(maxSourceIndex);\n",
    "        else:\n",
    "            print('-- Max TE source %d was not significant (TE(%d->%d | %s)=%.5f, p-value=%.6f), quitting' %\\\n",
    "                (maxSourceIndex, maxSourceIndex, d, (','.join(str(x) for x in conditionalSet)), maxTE, pValues[maxSourceIndex]));\n",
    "            break\n",
    "    \n",
    "    # Postcondition: conditionalSet holds the parents for target d\n",
    "    network[conditionalSet, d] = 1;\n",
    "\n",
    "# Plot the effective connectivity\n",
    "plt.figure()\n",
    "plt.imshow(network)\n",
    "plt.xlabel('target');\n",
    "plt.ylabel('source');\n",
    "plt.title('Multivariate effective network via TE(k=1)');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('Connections');"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
