Visual processing skill is used to gather visual information from environment however, there are cases that Visual Processing Disorder (VPD) occurs. The so called visual figure-ground discrimination is a type of VPD where color is one of the factors that contributes on this type. In line with this, color plays a vital role in everyday living, but individuals that have limited and inaccurate color perception suffers from Color Vision Deficiency (CVD) and still not aware on their case. To resolve this case, this study focuses on the design of KULAY, a Head-Mounted Display (HMD) device that can assess whether a user has a CVD or not thru the standard Hardy-Rand-Rittler (HRR) test. This test uses pattern recognition in order to evaluate the user. In addition, color vision deficiency simulation and color correction thru color transformation is also a concern of this research. This will enable people with normal color vision to know how color vision deficient perceives and vice-versa. For the accuracy of the simulated HRR assessment, its results were validated thru an actual assessment done by a doctor. Moreover, for the preciseness of color transformation, Structural Similarity Index Method (SSIM) was used to compare the simulated CVD images and the color corrected images to other reference sources. The output of the simulated HRR assessment and color transformation shows very promising results indicating effectiveness and efficiency of the study. Thus, due to its form factor and portability, this device is beneficial in the field of medicine and technology.