Policy iteration

$V^{\pi }(s)={\sum }_{s^{\prime }}P(s^{\prime }|s,\pi (s))\left[R(s,\pi (s),s^{\prime })+\gamma V^{\pi }(s^{\prime })\right]$

Policy Improvement

${\pi }^{\prime }(s)=\arg {\max }_a{\sum }_{s^{\prime }}P(s^{\prime }|s,a)\left[R(s,a,s^{\prime })+\gamma V^{\pi }(s^{\prime })\right]$