Escape velocity is the speed you need to get to infinity with zero kinetic energy. Any faster and you get to infinity with a little extra kinetic energy. So since potential is U=0 at a distance r=infinity, setting K=0 the total energy would be E=K+U=0. Since energy is conserved and since you're escaping from radius R at speed v, your energy there, E=K+U = 1/2 m v^2 - GMm/R (your mass is m) must also be zero. Thus you get v^2 = 2GM/R (your mass m cancels) for the escape velocity.
The v = (GM/R)^(1/2) is the speed you need so that you go just far enough forward to balance out how much you fall towards the Sun, keeping in a circular orbit just above the surface. Its when the centripetal acceleration v^2/r is given by, and thus equal to, the gravitational acceleration GM/r^2.