Yes, you can remove an evaluator (and their comments) even if the project has started. This can be done by removing the desired Reviewer from the People section under the Manage button.
Under Manage you will be able to remove reviewers from the project using the Remove button.
You can delete the Reviewer even when the project is in the Evaluation stage. In the scenario where the Reviewer had already begun to score, you can still delete them (their associated scores to date would be removed as well and no longer counting towards the Evaluation).