Please check the calculation parameters in RFEM 5 to see if the iterative solver is set. If that is the case, then set the direct solver.
Open the calculation parameters (menu Calculation → Calculation Parameters) and check to see if the direct solver is set (see the image).