Designing progressive data analysis systems affords several degrees of freedom. Algorithms, systems, and techniques require evidence to demonstrate their feasibility, effectiveness, and usability. While this characteristic holds true for all visual analytics systems, the dynamic nature of progressive systems poses new challenges for the creation of this evidence due to the tighter connections between the two classes of systems involved: visualization and progressive data processing. The approaches to evaluating visualization and progressive data processing systems are fundamentally different and often incompatible. Evaluating visualization systems usually borrows methodologies from Human-Computer Interaction (HCI) and psychology, while progressive data processing utilizes standardized performance benchmarks. At the highest level, progressive systems offer a natural trade-off: wait time versus quality and confidence in the results. At one extreme is simply waiting for the results, and the task of helping humans with the waiting process is related to wait-time engineering. At the other extreme is the temptation to make decisions too quickly, regardless of the quality of the results. This time/quality trade-off can be critical in time-constrained exploration and also important in other contexts, since time is a precious resource that should be used wisely. We want to build interfaces where users can make accurate decisions as soon as they have meaningful results that they can have confidence in—but no sooner. At a lower level, progressive systems should efficiently manage both human and machine capabilities. Humans have limitations in their attention span: they have great difficulty remaining engaged with long latency, and too many blinking visualizations that have to be monitored can overwhelm their perceptual and cognitive capacities. Therefore, progressive systems should strive to ensure that human attention remains effective. On the other hand, some algorithms are effective but return results that do not comply with human capabilities, such as unstable intermediary results that are hard to track visually. Evaluating solutions that trade speed/quality and are compatible with human capabilities adds further levels of complexity.