Find a and b (both real) when (a+b*i)^2=i.

Every complex number has a real and imaginary part. For the complex number z=a+bi the notation for real and imaginary parts respectively are Re(z)=a and Im(z)=b. If you know this, many complex algebra equations will become much simpler to solve.

In this specific case, firstly consider LHS, giving z=a²+ 2abi+(ib)²=(a²-b²)+(2ab)*i. (since i²=-1). Consequently, Re(z)=a²-b² and Im(z)=2ab. Next consider the RHS, write its real and imaginary parts: Re(i)=0 and Im(i)=1. Equate LHS and RHS, getting a system of equations:  a²-b²=0 and 2ab=1.
The solutions are a=-1/sq(2), b=1/sq(2) and a=1/sq(2), b=-1/sq(2).