class TrivialVacuumEnvironment(Environment):\n",
+ " \"\"\"This environment has two locations, A and B. Each can be Dirty\n",
+ " or Clean. The agent perceives its location and the location's\n",
+ " status. This serves as an example of how to implement a simple\n",
+ " Environment.\"\"\"\n",
+ "\n",
+ " def __init__(self):\n",
+ " super().__init__()\n",
+ " self.status = {loc_A: random.choice(['Clean', 'Dirty']),\n",
+ " loc_B: random.choice(['Clean', 'Dirty'])}\n",
+ "\n",
+ " def thing_classes(self):\n",
+ " \"\"\"Return the Thing/Agent classes that may populate this vacuum world.\"\"\"\n",
+ " return [Wall, Dirt, ReflexVacuumAgent, RandomVacuumAgent, TableDrivenVacuumAgent, ModelBasedVacuumAgent]\n",
+ "\n",
+ " def percept(self, agent):\n",
+ " \"\"\"Returns the agent's location, and the location status (Dirty/Clean).\"\"\"\n",
+ " return agent.location, self.status[agent.location]\n",
+ "\n",
+ " def execute_action(self, agent, action):\n",
+ " \"\"\"Change agent's location and/or location's status; track performance.\n",
+ " Score 10 for each dirt cleaned; -1 for each move.\"\"\"\n",
+ " if action == 'Right':\n",
+ " agent.location = loc_B\n",
+ " agent.performance -= 1\n",
+ " elif action == 'Left':\n",
+ " agent.location = loc_A\n",
+ " agent.performance -= 1\n",
+ " elif action == 'Suck':\n",
+ " if self.status[agent.location] == 'Dirty':\n",
+ " agent.performance += 10\n",
+ " self.status[agent.location] = 'Clean'\n",
+ "\n",
+ " def default_location(self, thing):\n",
+ " \"\"\"Agents start in either location at random.\"\"\"\n",
+ " return random.choice([loc_A, loc_B])\n",
+ "![](\"images/general_learning_agent.jpg\")
"
]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 2x2 Grid Work\n",
+ "Below is all of my added work, to create a 2x2 grid. I found it was easier to start from scratch than modify what was present."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 108,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "class DirectionType(type):\n",
+ " \"\"\"Fancy python trickery that allows us to use either\n",
+ " >> Direction.Right\n",
+ " or\n",
+ " >> Direction[\"Right\"]\n",
+ " and get the same outcome (1,0)\n",
+ " \"\"\"\n",
+ " Right = (1,0)\n",
+ " Left = (-1,0)\n",
+ " Down = (0,-1)\n",
+ " Up = (0,1)\n",
+ " \n",
+ " def __getitem__(self, key):\n",
+ " match key.lower():\n",
+ " case \"right\": return self.Right\n",
+ " case \"left\": return self.Left\n",
+ " case \"down\": return self.Down\n",
+ " case \"up\": return self.Up\n",
+ " case _: return (0,0) # default case\n",
+ "\n",
+ "class Directions(metaclass=DirectionType):\n",
+ " \"\"\"A simple helper class for moving around a 2d grid.\"\"\"\n",
+ " pass\n",
+ "\n",
+ "def add_direction(position, direction):\n",
+ " return (position[0] + direction[0], position[1] + direction[1])\n",
+ "\n",
+ "all_literal_directions = [\"Right\",\"Left\",\"Up\",\"Down\"]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 109,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grid2x2 = [\n",
+ " (x,y)\n",
+ " for x in range(2)\n",
+ " for y in range(2)\n",
+ "]\n",
+ "\n",
+ "class VacuumEnvironment2d(TrivialVacuumEnvironment):\n",
+ "\n",
+ " def __init__(self):\n",
+ " super().__init__()\n",
+ " # We're overriding the status they set here\n",
+ " self.status = {\n",
+ " location: random.choice(['Clean', 'Dirty'])\n",
+ " for location in grid2x2\n",
+ " }\n",
+ "\n",
+ " def execute_action(self, agent, action):\n",
+ " \"\"\"Change agent's location and/or location's status; track performance.\n",
+ " Score 10 for each dirt cleaned; -1 for each move.\"\"\"\n",
+ " location_x, location_y = agent.location\n",
+ " \n",
+ " if action in all_literal_directions:\n",
+ " agent.location = add_direction(agent.location, Directions[action])\n",
+ " agent.performance -= 1\n",
+ " elif action == \"Suck\":\n",
+ " if self.status[agent.location] == 'Dirty':\n",
+ " agent.performance += 10\n",
+ " self.status[agent.location] = 'Clean'\n",
+ " \n",
+ "\n",
+ " def default_location(self, thing):\n",
+ " \"\"\"Agents start in either location at random.\"\"\"\n",
+ " all_locations = list(self.status.keys())\n",
+ " return random.choice(all_locations)\n",
+ "\n",
+ "\n",
+ "new_2d_env = VacuumEnvironment2d()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Simple Reflex Agent (2d)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 163,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def SimpleReflexAgentProgram2d():\n",
+ " \"\"\"This agent takes action based solely on the percept. [Figure 2.10]\"\"\"\n",
+ " \n",
+ " def program(percept):\n",
+ " loc, status = percept\n",
+ " if status == \"Dirty\":\n",
+ " return \"Suck\"\n",
+ " \n",
+ " match loc:\n",
+ " case (0,0): return \"Right\"\n",
+ " case (1,0): return \"Up\"\n",
+ " case (1,1): return \"Left\"\n",
+ " case (0,1): return \"Down\"\n",
+ "\n",
+ " return program"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 164,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "State of the Environment Before: {(0, 0): 'Clean', (0, 1): 'Clean', (1, 0): 'Dirty', (1, 1): 'Clean'}.\n",
+ "State of the Environment After: {(0, 0): 'Clean', (0, 1): 'Clean', (1, 0): 'Clean', (1, 1): 'Clean'}.\n"
+ ]
+ },
+ {
+ "ename": "AttributeError",
+ "evalue": "'function' object has no attribute 'performance'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",
+ "Cell \u001b[0;32mIn[164], line 13\u001b[0m\n\u001b[1;32m 9\u001b[0m new_2d_env\u001b[38;5;241m.\u001b[39mrun()\n\u001b[1;32m 11\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mState of the Environment After: \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m.\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat(new_2d_env\u001b[38;5;241m.\u001b[39mstatus))\n\u001b[0;32m---> 13\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mModelBasedVacuumAgent scored \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m.\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat(\u001b[43msimple_vacuum_agent_2d\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mperformance\u001b[49m))\n",
+ "\u001b[0;31mAttributeError\u001b[0m: 'function' object has no attribute 'performance'"
+ ]
+ }
+ ],
+ "source": [
+ "new_2d_env = VacuumEnvironment2d()\n",
+ "\n",
+ "program = SimpleReflexAgentProgram2d()\n",
+ "simple_reflex_agent = Agent(program)\n",
+ "\n",
+ "new_2d_env.add_thing(simple_reflex_agent)\n",
+ "\n",
+ "print(\"State of the Environment Before: {}.\".format(new_2d_env.status))\n",
+ "\n",
+ "new_2d_env.run()\n",
+ "\n",
+ "print(\"State of the Environment After: {}.\".format(new_2d_env.status))\n",
+ "\n",
+ "print(\"ModelBasedVacuumAgent scored {}.\".format(simple_reflex_agent.performance))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Model-Based Reflex Agent (2d)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 112,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def ModelBasedVacuumAgent2d():\n",
+ " model = {\n",
+ " location: None\n",
+ " for location in grid2x2\n",
+ " }\n",
+ "\n",
+ " def program(percept):\n",
+ " \"\"\"Same as ReflexVacuumAgent, except if everything is clean, do NoOp.\"\"\"\n",
+ " location, status = percept\n",
+ " model[location] = status\n",
+ " \n",
+ " if status == \"Dirty\":\n",
+ " return \"Suck\"\n",
+ " \n",
+ " adjacent_locations = []\n",
+ " for direction in all_literal_directions:\n",
+ " adjacent_spot = add_direction(location, Directions[direction])\n",
+ " if adjacent_spot in model and model[adjacent_spot] != \"Clean\":\n",
+ " adjacent_locations.append(direction)\n",
+ " \n",
+ " if len(adjacent_locations) == 0:\n",
+ " return \"NoOp\"\n",
+ " else:\n",
+ " return adjacent_locations[0]\n",
+ " \n",
+ "\n",
+ " return Agent(program)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 160,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "State of the Environment Before: {(0, 0): 'Dirty', (0, 1): 'Dirty', (1, 0): 'Clean', (1, 1): 'Dirty'}.\n",
+ "State of the Environment After: {(0, 0): 'Clean', (0, 1): 'Clean', (1, 0): 'Clean', (1, 1): 'Clean'}.\n",
+ "ModelBasedVacuumAgent scored 27.\n"
+ ]
+ }
+ ],
+ "source": [
+ "new_2d_env = VacuumEnvironment2d()\n",
+ "\n",
+ "model_based_vacuum_agent_2d = ModelBasedVacuumAgent2d()\n",
+ "\n",
+ "new_2d_env.add_thing(model_based_vacuum_agent_2d)\n",
+ "\n",
+ "print(\"State of the Environment Before: {}.\".format(new_2d_env.status))\n",
+ "\n",
+ "new_2d_env.run()\n",
+ "\n",
+ "print(\"State of the Environment After: {}.\".format(new_2d_env.status))\n",
+ "\n",
+ "print(\"ModelBasedVacuumAgent scored {}.\".format(model_based_vacuum_agent_2d.performance))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
"kernelspec": {
- "display_name": "Python 3",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
@@ -702,9 +1098,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.10.12"
}
},
"nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
}
diff --git a/requirements-new.txt b/requirements-new.txt
new file mode 100644
index 000000000..ca1d36dc2
--- /dev/null
+++ b/requirements-new.txt
@@ -0,0 +1,16 @@
+graphviz
+ipython
+ipythonblocks
+ipywidgets
+jupyter
+keras
+matplotlib
+networkx
+numpy
+opencv-contrib-python
+pandas
+pillow
+pytest
+pytest-cov
+qpsolvers
+scipy
diff --git a/vacuum_world_TableAgent.ipynb b/vacuum_world_TableAgent.ipynb
new file mode 100644
index 000000000..c9dea2760
--- /dev/null
+++ b/vacuum_world_TableAgent.ipynb
@@ -0,0 +1,1188 @@
+{
+ "cells": [
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "dr6k9g_eBgZq"
+ },
+ "outputs": [],
+ "source": [
+ "%run bootstrap.ipynb"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "V6Q8plqMBgZr"
+ },
+ "source": [
+ "# THE VACUUM WORLD \n",
+ "\n",
+ "In this notebook, we will be discussing **the structure of agents** through an example of the **vacuum agent**. The job of AI is to design an **agent program** that implements the agent function: the mapping from percepts to actions. We assume this program will run on some sort of computing device with physical sensors and actuators: we call this the **architecture**:\n",
+ "\n",
+ "![](\"images/model_based_reflex_agent.jpg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "zyksQ_h3BgZ0"
+ },
+ "source": [
+ "We will now create a model-based reflex agent for the environment:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "3A2Duk96BgZ0"
+ },
+ "outputs": [],
+ "source": [
+ "# Delete the previously added simple reflex agent\n",
+ "trivial_vacuum_env.delete_thing(simple_reflex_agent)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XtwrWWOTBgZ0"
+ },
+ "source": [
+ "We need another function UPDATE-STATE which will be responsible for creating a new state description."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "clzgegrqBgZ1",
+ "outputId": "96fe5330-c93b-442e-d229-84e34f330a51"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "ModelBasedVacuumAgent is located at (1, 0).\n"
+ ]
+ }
+ ],
+ "source": [
+ "# TODO: Implement this function for the two-dimensional environment\n",
+ "def update_state(state, action, percept, model):\n",
+ " pass\n",
+ "\n",
+ "# Create a model-based reflex agent\n",
+ "model_based_reflex_agent = ModelBasedVacuumAgent()\n",
+ "\n",
+ "# Add the agent to the environment\n",
+ "trivial_vacuum_env.add_thing(model_based_reflex_agent)\n",
+ "\n",
+ "print(\"ModelBasedVacuumAgent is located at {}.\".format(model_based_reflex_agent.location))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "1_gBNveWBgZ1",
+ "outputId": "f5fb1a28-d4b6-4553-cbf0-654a307a2411"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "State of the Environment: {(0, 0): 'Clean', (1, 0): 'Clean'}.\n",
+ "ModelBasedVacuumAgent is located at (0, 0).\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Run the environment\n",
+ "trivial_vacuum_env.step()\n",
+ "\n",
+ "# Check the current state of the environment\n",
+ "print(\"State of the Environment: {}.\".format(trivial_vacuum_env.status))\n",
+ "\n",
+ "print(\"ModelBasedVacuumAgent is located at {}.\".format(model_based_reflex_agent.location))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "u0_PtNesBgZ3"
+ },
+ "source": [
+ "## GOAL-BASED AGENT PROGRAM\n",
+ "\n",
+ "A goal-based agent needs some sort of **goal** information that describes situations that are desirable, apart from the current state description.\n",
+ "\n",
+ "**Figure 2.13** of the book shows a model-based, goal-based agent:\n",
+ "![](\"images/model_goal_based_agent.jpg\")
\n",
+ "\n",
+ "**Search** (Chapters 3 to 5) and **Planning** (Chapters 10 to 11) are the subfields of AI devoted to finding action sequences that achieve the agent's goals.\n",
+ "\n",
+ "## UTILITY-BASED AGENT PROGRAM\n",
+ "\n",
+ "A utility-based agent maximizes its **utility** using the agent's **utility function**, which is essentially an internalization of the agent's performance measure.\n",
+ "\n",
+ "**Figure 2.14** of the book shows a model-based, utility-based agent:\n",
+ "![](\"images/model_utility_based_agent.jpg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "P0nWyfdCBgZ3"
+ },
+ "source": [
+ "## LEARNING AGENT\n",
+ "\n",
+ "Learning allows the agent to operate in initially unknown environments and to become more competent than its initial knowledge alone might allow. Here, we will breifly introduce the main ideas of learning agents. \n",
+ "\n",
+ "A learning agent can be divided into four conceptual components. The **learning element** is responsible for making improvements. It uses the feedback from the **critic** on how the agent is doing and determines how the performance element should be modified to do better in the future. The **performance element** is responsible for selecting external actions for the agent: it takes in percepts and decides on actions. The critic tells the learning element how well the agent is doing with respect to a fixed performance standard. It is necesaary because the percepts themselves provide no indication of the agent's success. The last component of the learning agent is the **problem generator**. It is responsible for suggesting actions that will lead to new and informative experiences. \n",
+ "\n",
+ "**Figure 2.15** of the book sums up the components and their working: \n",
+ "