0 votes
in Reinforcement Learning by
What is the role of the Discount Factor in Reinforcement Learning?

1 Answer

0 votes
by
The discount factor, , is a real value ∈ [0, 1], cares for the rewards agent achieved in the past, present, and future. In different words, it relates the rewards to the time-domain. Let’s explore the two following cases:

When we set the discount factor  = 1, it implies that we consider all the future rewards. Then the agent will learn forever, looking for all the future rewards, which may lead to infinity.

When we set the discount factor  = 0, it implies that we consider only the immediate reward and not the reward obtained from the future time steps. Then the agent will never learn as it will consider only the immediate reward.

Therefore the optimal value of the discount factor lies between 0.2 and 0.8 it set the importance to immediate and future rewards depending on the tasks. In some tasks, future rewards are more desirable than immediate rewards, and vice versa.

For example, in a chess game, the goal is to defeat the opponent's king. If we give more importance to the immediate reward, which is acquired by actions such as our pawn defeating any opposing chessman, then the agent will learn to perform this sub-goal instead of learning the actual goal. So, in this case, we give greater importance to future rewards than the immediate reward, whereas in some cases, we prefer immediate rewards over future rewards. For example, you may prefer to eat a fresh chocolate chip today and not in 13 days later.

Related questions

0 votes
asked May 6, 2023 in Reinforcement Learning by Robin
+1 vote
asked Apr 4, 2023 in Feature Engineering by john ganales
...