Contents
What is the role of Q in reinforcement learning?
Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. “Q” refers to the function that the algorithm computes – the expected rewards for an action taken in a given state.
What is Q value in MDP?
There are two important characteristic utilities of a MDP — values of a state, and q-values of a chance node. The * in any MDP or RL value denotes an optimal quantity. Q-value of a state, action pair: The q-value is the optimal sum of discounted rewards associated with a state-action pair.
What does the Q function and the V function in?
The $V$ function states what the expected overall value (not reward!) of a state $s$ under the policy $pi$ is. The $Q$ function states what the value of a state $s$ and an action $a$ under the policy $pi$ is. This means, $$Q^pi(s, pi(s)) = V^pi(s)$$.
Which is an alternative form of the Q-function?
The Q -function can be expressed in terms of the error function, or the complementary error function, as An alternative form of the Q -function known as Craig’s formula, after its discoverer, is expressed as:
What is the Q and V function in reinforcement learning?
A policy π, according to Reinforcement Learning: An Introduction by Sutton and Barto is a function π: S → A (this could be probabilistic). The V function states what the expected overall value (not reward!) of a state s under the policy π is. The Q function states what the value of a state s and an action a under the policy π is.
What is the V function in machine learning?
You have it right, the V function gives you the value of a state, and Q gives you the value of an action in a state (following a given policy π ). I found the clearest explanation of Q-learning and how it works in Tom Mitchell’s book “Machine Learning” (1997), ch. 13, which is downloadable.