Scientific Visualizations are important to scientists and engineers in many fields, but also to managers and to the general public. In order to achieve good results there have to be means to evaluate the quality of visualizations and to compare visualizations to each other. In this paper, after a short introduction and an overview of some related work, the notion of a 'visualization background' is introduced. It includes the prior knowledge of the user; the aims of the user; the application domain; amount, structure, and distribution of the data; and the available hardware and software. Next, the problem of quantifying visualization quality is discussed. Then, six subqualities are presented, namely data resolution quality, semantic quality, mapping quality, image quality, presentation and interaction quality, and user quality. The reference model defines visualization quality as six pairs of two values each: for each of the six subqualities, a weight value C (representing the importance of the subquality for the visualization background) and a subquality value Q (a measure of how well the visualization meets the requirements of the visualization background in this subquality) are given. Finally, the Q-VIS graph is introduced which offers a compact, easy to perceive representation of this visualization quality. Thus, a tool for evaluating and comparing visualizations and visualization systems is presented which can help to achieve better visualizations in the future.