{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "87e17b68-7539-4539-aeab-9df6fc819f22",
   "metadata": {},
   "source": [
    "# AIS in written English\n",
    "\n",
    "Author: J. Lizier, 2023-; based on the original Matlab tutorials.\n",
    "\n",
    "In this activity we will revisit the written English data set we first use in Module 2, considering it as a time series (where later characters have later time indices), and examine Active Information Storage (AIS) that past characters provide about the next.\n",
    "\n",
    "1. We will start by taking copies of relevant parts of your saved code from the CA activity in the AIS module. We are going to use that code to recreate a similar bias corrected AIS versus k plot for this data set.\n",
    "    1. For the first cell below, copy and paste the import/startup lines from your CA notebook.\n",
    "    2. For the second cell below, leave the code as is, aside from changing the path to where your data set is. Notice that there is a little more data plumbing this time. The vector `processedStr` holds each character in character format, but we will need to supply our symbols to JIDT as integers in the range `[0,alphabetSize-1]`, where `alphabetSize` is 27 (26 letters plus the space). So we convert the `processedStr` to a time series of integers.\n",
    "    3. For the third cell, copy and paste the code from the CA notebook which computed and plotted AIS vs k (the last code cell before we went on to examine the local AIS values).\n",
    "2. Now, make sure to delete the lines in the third cell which load in the CA data (we're no longer using this!)\n",
    "3. Now, let's check what we need to change about the calculation of AIS for each value of k in the main loop:\n",
    "    1. When we worked with the CAs, the alphabet size we were dealing with there was 2. For this data set it will be 27. Can you see where we tell the estimator this? If unsure, go back to the AutoAnalyser, change the `base` parameter to 27, and see where this is placed in the generated code.\n",
    "    2. Remove the loop over variables `v` in the code, since we've only got one time series here (unlike the CAs example where we have a time series for each cell or variable `v`). Pass your whole time series `data` to the estimator in a single call to `calc.addObservations()`\n",
    "4. Finally, change your loop over the value of `k` to stop at 4. At the moment, the multivariate state space here for `k=5` with an alphabet size of 27 is too large for JIDT to handle (I have some new algorithms coming!)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7307f279-64fd-4596-b279-4b26df51f698",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paste the import and JVM startup lines here (copy from the CA activity):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55ca6967-2ca7-45e8-9858-6bb0356c2bee",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specifics required for the text processing here:\n",
    "import string\n",
    "import re\n",
    "\n",
    "# Load and pre-parse the scripts\n",
    "filename = '/home/joseph/TeachingPlayground/CSYS5030/WrittenText/Seinfeld-scripts-textOnly.txt'\n",
    "with open(filename, 'rt') as f:\n",
    "    str = f.read()\n",
    "p = re.compile('[!\"#\\$%&\\'\\(\\)\\*\\+\\,-\\.\\/:;<=>\\?@\\[\\]\\\\\\^_`{\\|}~0-9]*');\n",
    "processedStr = p.sub('', str); # Remove punctuation characters and digits\n",
    "processedStr = ' '.join(processedStr.split('\\n')); # Replace newline characters with spaces\n",
    "processedStr = processedStr.lower(); # Convert all upper case into lower case\n",
    "processedStr = numpy.array(list(processedStr)); # Finally convert this into a numpy array so we can work with it\n",
    "numpy.unique(processedStr)\n",
    "\n",
    "# Additional data plumbing for JIDT:\n",
    "data = processedStr.view(numpy.int32) - ord('a'); # Converts the letters to an integer representing their offset from 'a'.\n",
    "# Use of int32 rather than int64 is important to ensure compatibility with Java int size.\n",
    "# But spaces have been turned into the value ord(' ') - ord('a') (i.e. the difference in character encodings of space and 'a').\n",
    "# We need to replace this with the integer value 26, such that all of our symbols are consecutive for JIDT:\n",
    "data[data == ord(' ') - ord('a')] = 26;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab68f364-d9ed-45d3-8408-ce724d0233c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paste the code computing AIS vs k here (copy from the CA activity and then we'll adapt):\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc5119d3-c9d3-4139-ad43-3346098ea85d",
   "metadata": {},
   "source": [
    "5. Now run the code, and observe how AIS -- both the raw and bias corrected values -- change with history length k.\n",
    "    * What value of k should we use when computing AIS for this data set?\n",
    "    * From this data set, how many previous characters are we able to confidently say have an impact on predicting the next character?\n",
    "    * What sources of error or limitations might there be in our experiment?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50cfe176-4bf8-44a1-86f6-0ef354486190",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
