Although sparse representation based classification (SRC) has gained great success, doubts on the necessity of sparse constraint come in recent years. And collaborative representation based classification (CRC) has attracted much attention from researchers in fields of signal processing, image processing and pattern recognition. In this paper, an algorithm called collaborative dictionary learning with structured incoherence (CDLSI) is proposed for collaborative representation based detection (CRD), which can be viewed as a binary classification problem, in hyperspectral imagery (HSI). An inter-class incoherence term is added to make sub-dictionaries to be as independent as possible. During the optimizing procedure, sub-dictionaries are updated atoms-by-atoms with metaface method. Specifically, considering the non-sparse representation of CRC, the coefficients are iteratively optimized with l2 -norm regularization during the coding procedure in CDLSI. Once the sub-dictionaries are obtained, the collaborative representation based technique is then used for detection. The proposed algorithm is applied to several real hyperspectral images for detection. Experimental results confirm the effectiveness of the proposed approach, and prove the superiority to the traditional algorithms.