Spectral Computed Tomography (CT) has an advantage of providing energy spectrum information, which is valued on multi-energy material decomposition for material discrimination and accurate image reconstruction. However, due to the non-ideal physical effects of photon counting detectors (PCDs), such as charge sharing, pulse pileup and K-escape, serious spectral distortion is unavoidable in practical systems. The degraded spectrum will induce error into the decomposition model and affect the accuracy of material decomposition. Recently, artificial neural network has demonstrated great potential in the tasks of image segmentation, object detection, natural language processing, and etc. By adjusting the interconnection relationship among a large number of internal nodes, a neural network provides us a way to mine information from huge data depending on the complexity of the network system. Considering the difficulty of modeling the spectral CT system spectrum including the response function of a PCD and the superiority of data-driven characteristics of a neural network, we proposed a novel multi-energy material decomposition method using a neural network without the knowledge of spectral information. On one hand, specific linear attenuation coefficients can be obtained directly through our method. It would help further material recognition and spectral CT reconstruction. On the other hand, the network outputs show outstanding performance on image denoising and artifacts suppression. Our method can fit for different selections of training materials and different settings of imaging systems such as different number of energy bins and energy bin thresholds. According to our test results, the trained neural network has a good generalization ability.