How to solve system of equations with complex numbers