Introduction

We distinguish 2 types of learning processes:

Classical conditioning:

• Involuntary behavior.
• Pairing of stimuli with biologically significant events.
• Reflexes: automatically elicited by the appropriate stimuli.
• Example: sight of sweets may cause a child to salivate - not voluntarily “chosen”.

Operator conditioning:

• Voluntary behavior.
• Reinforced by reward/punishment.
• Example: a child may learn to open a box to get the sweets inside.

Operator conditioning, also known as instrumental conditioning, is a learning process where behaviors are modified through the association of stimuli with reinforcement or punishment. In it, operants—behaviors that affect one's environment—are conditioned to occur or not occur depending on the environmental consequences of the behavior.

In Operator Conditioning, we distinguish:

In these lectures we will focus on Reinforcement learning.

Reinforcement learning is learning what to do—how to map situations to actions—to maximize a numerical reward signal. The learner is not told which actions to take, but instead must discover which actions yield the most reward by trying them. In the most interesting and challenging cases, actions may affect not only the immediate reward but also the next situation and, through that, all subsequent rewards. These two characteristics—trial-and-error search and delayed reward—are the two most important distinguishing features of reinforcement learning.

We formalize the problem of reinforcement learning using ideas from dynamical systems’ theory, specifically, as the optimal control of incompletely-known Markov decision processes. The details of this formalization will be discussed in the next section.

The basic idea is simply to capture the most significant aspects of the real problem facing a learning agent interacting over time with its environment to achieve a goal. A learning agent must be able to sense the state of its environment to some extent and must be able to take actions that influence the state. The agent also must have a goal or goals relating to the state of the environment. Markov decision processes are intended to include just these three aspects—sensation, action, and goal—in their simplest possible forms without trivializing any of them. Any method that is well suited to solving such problems, we consider to be a reinforcement learning method.

Reinforcement learning differs from both supervised and unsupervised learning, so we consider it to be a third machine learning paradigm.