First, need to get all the terms in the equation to be the same. Using the following identity, it is possible to achieve this:
sin2(x) + cos2(x) = 1
1 - cos2(x) = sin2(x)
Substituting this into the equation in the question:
5cos2(x) - cos(x) = 1 - cos2(x)
6cos2(x) - cos(x) - 1 = 0
Replace the term cos(x) with y:
6y2 - y - 1 = 0
Product = -6
Sum = -1
There numbers that satisfy this are -3 and 2. Therefore, the factorised form of the eqation is:
(2y - 1)(3y + 1) = 0
The roots of this equation are: y = cos(x) = -1/3 or 1/2. Therefore these are the possible values of cos(x).