Skip to content

Agents

custom_grid_env.agents.base_agent.Agent(action_space, **kwargs)

Bases: Protocol

Protocol for all agents in the custom grid environment.

Any class that implements a 'get_action' method with the correct signature is considered an Agent.

Initializes the agent with the given action space.

Parameters:

Name Type Description Default
action_space Space

The action space of the environment.

 required
**kwargs Any

Additional keyword arguments for agent configuration.

 {}
Source code in src/custom_grid_env/agents/base_agent.py 
15
16
17
18
19
20
21
22
def __init__(self, action_space: gym.spaces.Space, **kwargs: Any):
    """Initializes the agent with the given action space.

    Args:
        action_space (gym.spaces.Space): The action space of the environment.
        **kwargs (Any): Additional keyword arguments for agent configuration.
    """
    ...

get_action(observation)

Returns an action based on the given observation.

Parameters:

Name Type Description Default
observation dict

The current observation from the environment.

 required

Returns:

Name Type Description
int int

The action to take.
Source code in src/custom_grid_env/agents/base_agent.py 
24
25
26
27
28
29
30
31
32
33
def get_action(self, observation: Dict[str, Any]) -> int:
    """Returns an action based on the given observation.

    Args:
        observation (dict): The current observation from the environment.

    Returns:
        int: The action to take.
    """
    ...

custom_grid_env.agents.base_agent.BaseAgent(action_space, **kwargs)

Base class for all agents to provide a default constructor.

Agents can inherit from this class to satisfy the Agent protocol without re-implementing the constructor.

Attributes:

Name Type Description
action_space Space

The action space of the environment.
env Optional[Any]

Reference to the environment instance.
perceived_agent_pos Optional[List[int]]

Agent's belief of its position.
perceived_ghost_pos Optional[List[int]]

Agent's belief of the ghost's position.

Initializes the agent with the given action space.

Parameters:

Name Type Description Default
action_space Space

The action space of the environment.

 required
**kwargs Any

Additional keyword arguments, such as 'env'.

 {}
Source code in src/custom_grid_env/agents/base_agent.py 
49
50
51
52
53
54
55
56
57
58
59
def __init__(self, action_space: gym.spaces.Space, **kwargs: Any):
    """Initializes the agent with the given action space.

    Args:
        action_space (gym.spaces.Space): The action space of the environment.
        **kwargs (Any): Additional keyword arguments, such as 'env'.
    """
    self.action_space = action_space
    self.env = kwargs.get("env")
    self.perceived_agent_pos = None
    self.perceived_ghost_pos = None

custom_grid_env.agents.adversarial_agents.MinimaxAgent(action_space, depth_limit=4, **kwargs)

Bases: AdversarialAgent

Agent that uses Minimax with Alpha-Beta pruning.

Source code in src/custom_grid_env/agents/adversarial_agents.py 
14
15
16
17
18
19
20
21
22
23
24
25
def __init__(
    self, action_space: gym.spaces.Space, depth_limit: int = 4, **kwargs: Any
):
    """Initializes the adversarial agent.

    Args:
        action_space (gym.spaces.Space): The action space.
        depth_limit (int): The depth limit for the search.
        **kwargs (Any): Additional arguments, should include 'env'.
    """
    super().__init__(action_space, **kwargs)
    self.depth_limit = depth_limit

get_action(observation)

Returns the best action using minimax.

Parameters:

Name Type Description Default
observation dict

The current observation.

 required

Returns:

Name Type Description
int int

The best action.
Source code in src/custom_grid_env/agents/adversarial_agents.py 
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
def get_action(self, observation: Dict[str, Any]) -> int:
    """Returns the best action using minimax.

    Args:
        observation (dict): The current observation.

    Returns:
        int: The best action.
    """
    agent_pos = self._get_agent_pos()
    ghost_pos = self._get_ghost_pos()

    # Check if we are acting as the ghost
    is_ghost = "agent_relative_pos" in observation

    alpha = -float("inf")
    beta = float("inf")

    actions = [0, 1, 2, 3]

    if not is_ghost:
        best_score = -float("inf")
        best_action = 0
        for action in actions:
            new_ap, new_gp, term, info = self._get_next_state(
                agent_pos, ghost_pos, 0, action
            )
            score = self._min_value(new_ap, new_gp, 1, term, info, alpha, beta)
            if score > best_score:
                best_score = score
                best_action = action
            alpha = max(alpha, best_score)
    else:
        best_score = float("inf")
        best_action = 0
        for action in actions:
            new_ap, new_gp, term, info = self._get_next_state(
                agent_pos, ghost_pos, 1, action
            )
            score = self._max_value(new_ap, new_gp, 1, term, info, alpha, beta)
            if score < best_score:
                best_score = score
                best_action = action
            beta = min(beta, best_score)

    return best_action

custom_grid_env.agents.adversarial_agents.ExpectimaxAgent(action_space, depth_limit=4, **kwargs)

Bases: AdversarialAgent

Agent that uses Expectimax algorithm.

Source code in src/custom_grid_env/agents/adversarial_agents.py 
14
15
16
17
18
19
20
21
22
23
24
25
def __init__(
    self, action_space: gym.spaces.Space, depth_limit: int = 4, **kwargs: Any
):
    """Initializes the adversarial agent.

    Args:
        action_space (gym.spaces.Space): The action space.
        depth_limit (int): The depth limit for the search.
        **kwargs (Any): Additional arguments, should include 'env'.
    """
    super().__init__(action_space, **kwargs)
    self.depth_limit = depth_limit

get_action(observation)

Returns the best action using expectimax.

Parameters:

Name Type Description Default
observation dict

The current observation.

 required

Returns:

Name Type Description
int int

The best action.
Source code in src/custom_grid_env/agents/adversarial_agents.py 
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
def get_action(self, observation: Dict[str, Any]) -> int:
    """Returns the best action using expectimax.

    Args:
        observation (dict): The current observation.

    Returns:
        int: The best action.
    """
    agent_pos = self._get_agent_pos()
    ghost_pos = self._get_ghost_pos()

    is_ghost = "agent_relative_pos" in observation

    actions = [0, 1, 2, 3]

    if not is_ghost:
        best_score = -float("inf")
        best_action = 0
        for action in actions:
            outcomes = self._get_agent_outcomes(agent_pos, action)
            score = 0
            for next_ap, prob in outcomes:
                term = False
                info = {}
                if next_ap == ghost_pos:
                    term = True
                    info["caught_by_ghost"] = True
                elif self.env.grid[next_ap[0], next_ap[1]]["is_goal"]:
                    term = True
                    info["reached_goal"] = True

                score += prob * self._expect_value(
                    next_ap, ghost_pos, 1, term, info
                )

            if score > best_score:
                best_score = score
                best_action = action
    else:
        best_score = float("inf")
        best_action = 0
        for action in actions:
            new_ap, new_gp, term, info = self._get_next_state(
                agent_pos, ghost_pos, 1, action
            )
            score = self._max_value(new_ap, new_gp, 1, term, info)
            if score < best_score:
                best_score = score
                best_action = action
    return best_action

custom_grid_env.agents.chase_ghost_agent.ChaseGhostAgent(action_space, **kwargs)

Bases: BaseAgent

Ghost agent that chases the player using simple pathfinding.

This replicates the original built-in ghost behavior.

Source code in src/custom_grid_env/agents/base_agent.py 
49
50
51
52
53
54
55
56
57
58
59
def __init__(self, action_space: gym.spaces.Space, **kwargs: Any):
    """Initializes the agent with the given action space.

    Args:
        action_space (gym.spaces.Space): The action space of the environment.
        **kwargs (Any): Additional keyword arguments, such as 'env'.
    """
    self.action_space = action_space
    self.env = kwargs.get("env")
    self.perceived_agent_pos = None
    self.perceived_ghost_pos = None

get_action(observation)

Choose action to move toward the agent.

Parameters:

Name Type Description Default
observation dict

Ghost's observation with 'agent_relative_pos'.

 required

Returns:

Name Type Description
int int

Action (0=left, 1=down, 2=right, 3=up).
Source code in src/custom_grid_env/agents/chase_ghost_agent.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
def get_action(self, observation: dict) -> int:
    """Choose action to move toward the agent.

    Args:
        observation (dict): Ghost's observation with 'agent_relative_pos'.

    Returns:
        int: Action (0=left, 1=down, 2=right, 3=up).
    """
    agent_relative = observation["agent_relative_pos"]
    row_diff = agent_relative[0]  # Positive = agent is below ghost
    col_diff = agent_relative[1]  # Positive = agent is to the right of ghost

    neighbors = observation["neighbors"]

    # Prioritize direction with larger distance
    if abs(row_diff) >= abs(col_diff):
        # Try vertical first
        if row_diff > 0 and neighbors["down"]["accessible"]:
            return 1  # down
        elif row_diff < 0 and neighbors["up"]["accessible"]:
            return 3  # up
        # Fall back to horizontal
        if col_diff > 0 and neighbors["right"]["accessible"]:
            return 2  # right
        elif col_diff < 0 and neighbors["left"]["accessible"]:
            return 0  # left
    else:
        # Try horizontal first
        if col_diff > 0 and neighbors["right"]["accessible"]:
            return 2  # right
        elif col_diff < 0 and neighbors["left"]["accessible"]:
            return 0  # left
        # Fall back to vertical
        if row_diff > 0 and neighbors["down"]["accessible"]:
            return 1  # down
        elif row_diff < 0 and neighbors["up"]["accessible"]:
            return 3  # up

    # If no good move, try any accessible direction
    for action, direction in [(3, "up"), (2, "right"), (1, "down"), (0, "left")]:
        if neighbors[direction]["accessible"]:
            return action

    # No move possible, stay in place
    return 0

custom_grid_env.agents.random_ghost_agent.RandomGhostAgent(action_space, **kwargs)

Bases: BaseAgent

Ghost agent that moves randomly.

Source code in src/custom_grid_env/agents/base_agent.py 
49
50
51
52
53
54
55
56
57
58
59
def __init__(self, action_space: gym.spaces.Space, **kwargs: Any):
    """Initializes the agent with the given action space.

    Args:
        action_space (gym.spaces.Space): The action space of the environment.
        **kwargs (Any): Additional keyword arguments, such as 'env'.
    """
    self.action_space = action_space
    self.env = kwargs.get("env")
    self.perceived_agent_pos = None
    self.perceived_ghost_pos = None

get_action(observation)

Returns a random action.

Parameters:

Name Type Description Default
observation dict

The current observation.

 required

Returns:

Name Type Description
int int

A random action from the action space.
Source code in src/custom_grid_env/agents/random_ghost_agent.py 
 9
10
11
12
13
14
15
16
17
18
def get_action(self, observation: dict) -> int:
    """Returns a random action.

    Args:
        observation (dict): The current observation.

    Returns:
        int: A random action from the action space.
    """
    return int(self.action_space.sample())

custom_grid_env.agents.random_player_agent.RandomPlayerAgent(action_space, **kwargs)

Bases: BaseAgent

Example random agent for demonstration.

Source code in src/custom_grid_env/agents/base_agent.py 
49
50
51
52
53
54
55
56
57
58
59
def __init__(self, action_space: gym.spaces.Space, **kwargs: Any):
    """Initializes the agent with the given action space.

    Args:
        action_space (gym.spaces.Space): The action space of the environment.
        **kwargs (Any): Additional keyword arguments, such as 'env'.
    """
    self.action_space = action_space
    self.env = kwargs.get("env")
    self.perceived_agent_pos = None
    self.perceived_ghost_pos = None

get_action(observation)

Returns a random action.

Parameters:

Name Type Description Default
observation dict

The current observation.

 required

Returns:

Name Type Description
int int

A random action from the action space.
Source code in src/custom_grid_env/agents/random_player_agent.py 
 9
10
11
12
13
14
15
16
17
18
def get_action(self, observation: dict) -> int:
    """Returns a random action.

    Args:
        observation (dict): The current observation.

    Returns:
        int: A random action from the action space.
    """
    return int(self.action_space.sample())