Evidence presented supports the linguistic relativity of color categories in three different paradigms. Firstly, a series of cross-cultural investigations, which had set out to replicate the seminal work of Rosch Heider with the Dani of New Guinea, failed to find evidence of a set of universal color categories. Instead, we found evidence of linguistic relativity in both populations tested. Neither participants from a Melanesian hunter-gatherer culture, nor those from an African pastoral tribe, whose languages both contain five color terms, showed a cognitive organization of color resembling that of English speakers. Further, Melanesian participants showed evidence of Categorical Perception, but only at their linguistic category boundaries. Secondly, in native English speakers verbal interference was found to selectively remove the defining features of Categorical Perception. Under verbal interference, the greater accuracy normally observed for cross-category judgements compared to within-category judgements disappeared. While both visual and verbal codes may be employed in the recognition memory of colors, participants only make use of verbal coding when demonstrating Categorical Perception. Thirdly, in a brain- damaged patient suffering from a naming disorder, the loss of labels radically impaired his ability to categorize colors. We conclude that language affects both the perception of and memory for colors.