El Faraghy v Faraghy and others [2007] EWCA Civ 1149, [2007] All ER (D) 248 (Nov)
Lord Justice Ward (at para 32) gave guidance on recusal applications.
If circumstances permit it, an informal approach should first be made to the judge, making the complaint and inviting recusal. While judges must not yield to tenuous or frivolous objections, a judge can deny the complaint but still pass the case to a colleague.
If a judge does not feel able to do so, then it may be preferable, if it is possible to arrange it, to have another judge take the decision. Where the appearance of justice is at stake, it is better that justice be done independently by another, rather than require the judge to sit in judgment of his own behaviour.