When integrating by substitution the first thing to do is change the limits of the integral by subbing them into the equation for u. This gives
u = 5 as the lower limit and
u = 2 for the upper limit.
The next step is to differentiate u wrt to x in order to find dx in terms of du.
du/dx = -2x which rearranges to
dx = -du/2x. Substituting this into the integral gives,
-(x^2)/(2sqrt(u)), x^2 in terms of u is x^2 = 6 - u giving the final integral in terms of u as
-(6 - u)/(2sqrt(u)) between u = 5 and u = 2. This is now a simple integral like those in core 2.
When worked through the final answer will be (13/5)sqrt(5) - (16/3)sqrt(2), leaving the answer in surd form.