While constantly circling around a center, the rock will always have two vectors acting on it (forces in two direction).
1. towards the center
2. at tangent or perpendicular to the string
The direction of these two forces keep changing as the rock travels along the circular path. Taking these two forces into account, the final or resultant force would be along the circumference of the circle. In your case, when the string is cut, the force acting towards the center is removed and the only force acting on the rock would be the force that is perpendicular to the string, ( which is upward if seen from above ).
Another example would be- if the sun's gravitational pull suddenly disappears, then earth would continuously keep moving towards the tangent direction of its orbit.