This paper presents another outlook on image description, classification and retrieval. Some popular image description methods are Histogram of Oriented Gradients (HoG), Speed Up Robust Features (SURF) and Scale Invariant Feature Transform (SIFT). While SURF and SIFT both use ”interest points” to describe an image, HoG uses all of the points in the image. One of the goals of this paper is to improve HoG by creating a feature vector containing more information about the image. The proposed description method is called the Histogram of Second order Oriented Gradients (HSoG) and it was shown to perform better than HoG using a dataset comprising of airplanes, cars and motorbikes by supervised learning. The second goal is to tackle image clustering for aid in unsupervised learning and this paper explores a method called Localized Clustering with a comparison to K-Means. The localized clustering approach does not require the number of clusters as an input but it does return what it determines the number of clusters should be. Finally, The retrieval process presented involves training a linear SVM with known labels (supervised) to evaluate the effectiveness of HoG vs HSoG and HSoG out performs HoG.