Here's the set of questions that together can help us all figure out if an evaluation might make a difference:
What are the decisions that the findings from the evaluation could inform?
Are those decisions going to be based on evidence about program effectiveness?
When are those decisions going to be made?
Can the evaluation change anyone's mind?
If these questions were applied systematically and early in program design and implementation, we'd have more good and useful evaluations -- ones that are well-timed and use appropriate methods. We'd have better clarity about the purpose of the evaluations we conduct. The timing and methods would match the needs of the decision makers, and greater transparency could mitigate against political influences. At the same time, we'd end up with fewer evaluations that are purely symbolic.