{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c0061bfc-c926-499a-95c9-807631eacb62",
   "metadata": {},
   "source": [
    "# Module 11 -- Transfer Entropy in Cellular Automata\n",
    "\n",
    "In this activity we will examine Transfer Entropy (TE) in Elementary Cellular Automata rule 54 in a similar fashion to what we did for AIS in the previous module. We will compute the local TE values and how they relate to emergent structure.\n",
    "\n",
    "1. Make sure that you had *downloaded* the sample time-series as directed in the AIS activity on Cellular Automata, else go back and do that now.\n",
    "2. Now open the AutoAnalyser and select Transfer Entropy. Select a Discrete estimator, select your CA data file, tick the checkbox for `All pairs?`, enter the `k` history value that you determined for AIS in the Information Storage module (my analysis suggested `k=15`), and -- most importantly -- untick the checkbox for `Compute result?` (we don't want to run this computation, it will take a lot of time!), and click `Generate Code`.\n",
    "3. Copy and paste the generated code into the code cells below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "107f38bd-4d2f-4e02-8aee-aa1b2f4524a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paste the import and JVM startup lines here:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38596e45-ab36-40b1-a228-fa8ed2450f11",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paste the code loading and analysing the data here:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73e466e3-f1e0-4aed-a71a-89d8d3386d32",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This code block plots a sample of 100 time steps for 100 cells of the CA:\n",
    "import matplotlib.pyplot as plt\n",
    "plt.imshow(data[:99,:99]) # Plotting the first 100 time steps for first 100 cells\n",
    "plt.xlabel('cells');\n",
    "plt.ylabel('timestep');\n",
    "cbar = plt.colorbar()\n",
    "cbar.set_label('CA values');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3fac42a-2551-42c5-af57-837cc1631baa",
   "metadata": {},
   "source": [
    "4. Now, examine how the generated code is computing the TE for each cell pair. We want to change a few things here:\n",
    "    * We don't want to compute TE for all pairs, only for sources in the neighbourhood of the target since we know those are directly contributing information to the state update of the target (see comparison of TE to causality in the lecture). We'll focus on sources one cell to the left of the target (i.e. with cell index being one less).\n",
    "    * As we did for AIS in the previous session, we would get a much more accurate result if we could make a TE estimation by pooling all of the samples for the pairs (with source one cell to left of target) together. Reflect on why pooling the samples for technically different paired processes is valid here too, and think of a data set where it would not be a valid approach.\n",
    "\n",
    "    As such, change the code so that, instead of making a separate calculation for each source-target pair, we make one calculation where the samples from all pairs (with source cell one to left of the target) are used together:\n",
    "    * First remove the for-loop over sources (but leave all of the code inside that loop).\n",
    "    * Then assign the value for `s` inside the loop over destination cells to be one less than `d`. You can't just use `s = d-1;` though, because we use periodic boundary conditions so that `d==0` has `s==9999` as its neighbour. Instead, use: `s = (d - 1) % 10000;`\n",
    "    * Next, move step 3 (calculator initialisation) before the loop, and step 5 (the calculation of the final value, and the `print` statement) after the loop.\n",
    "    * You'll also need to rewrite the `print` statement without `s` and `d`, since they're not used outside the loop (make sure you rewrite the part of the print statement that references them with `%d` symbols as well!). Also, when you print the result, check the number of samples that were used via `calc.getNumObservations()`, which should report a number of observations consistent with having used all of the samples from all relevant pairs. If it still reports 599, obviously you've only used samples from one pair.\n",
    "5. Run your code and check the TE value computed is arouns 0.08 bits.\n",
    "6. Now we move on to compute the **local** TE values.\n",
    "    * We'll insert new code in the cell below to compute the local values. Add another loop over all cells (copying the whole for loop that is already earlier in the code, but don't copy anything from outside the for loop). Delete the copied step 4 line where the sample data are supplied, replacing it with a calculation of local values of TE into this destination cell using: `localTEForThisCell = calc.computeLocalFromPreviousObservations(source, destination);`. Note that this call evaluates the local TE values for the given samples, based on the PDFs computed from across the whole of the CA in the previous for loop.\n",
    "    * Create an empty matrix `locals = numpy.zeros(data.shape);` before this new for loop. Store all of the columns of local TE values together in this one 2D matrix for all of the cells; e.g. after computing `localTEForThisCell` for column `d` store it as `locals[:,d] = localTEForThisCell;`\n",
    "    * After the loop, let's plot a small segment of these local values. A simple way to do this would be:\n",
    "      ```python\n",
    "      plt.imshow(locals[:99,:99]) # Plotting the first 100 time steps for first 100 cells\n",
    "      plt.xlabel('cells');\n",
    "      plt.ylabel('timestep');\n",
    "      cbar = plt.colorbar()\n",
    "      cbar.set_label('TE (bits)');\n",
    "      ```\n",
    "      However the colours could be arranged a little better (so that we use blue for positive AIS and red for negative). So instead first import some extra libraries from matplotlib: `import matplotlib.cm as cm; import matplotlib.colors as mplcolors` and then add the following argument to the `plt.imshow()` call: `cmap=cm.coolwarm_r, norm=mplcolors.CenteredNorm()`\n",
    "    * Run the code!\n",
    "    * (You can ignore the first `k` time steps here being 0 -- it's simply not possible to compute a local value until we've accumulated `k` time steps).\n",
    "    * Observe which emergent structures execute the strongest information transfer to the right (i.e. from a source one cell to the left). You may find it useful to replot the raw values here again for comparison, copying step 1 from above. Does this align with our intuition from the lecture? What do the areas with negative local TE values mean? (Recall the meaning of negative values of local conditional mutual information, and think about what the source at the left is telling us, in the context of the target past, when the local TE goes negative).\n",
    "    * For further details, you can compare to Fig. 4b and section IV.E (and references therein) of the JIDT paper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8fbed096-0a97-4cfc-8220-a143b1166f49",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Insert the code to compute the local TE values here:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "003ef103-af9a-4d8c-9189-f17b357a935c",
   "metadata": {},
   "source": [
    "7. _Optional extensions_:\n",
    "    * Was the calculated TE statistically significant?\n",
    "    * Try examining information flow one step to the left at each point in space and time instead (think about what you would need to change in the code). What do you expect from the results?\n",
    "    * Try running your code on other CA rules, with and without emergent structures.\n",
    "    * Try examining conditional transfer entropies, as per item 10 in the tutorial."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
