As his hand moves back with the ball, the time taken to stop the ball is greater. This means that the rate of change of momentum is less (because the rate of him changing the speed of the ball is much slower). Newton's second law of motion states that a body's rate of change in momentum is equal to the net force acting on it. Therefore, the lower the rate of change of momentum, the lower the force. Alternatively, work done on the ball is the same but there's a greater distance, therefore less force is applied.