In X-ray object detection, scale awareness and spatial awareness (SA) are two independent issues. Previous research has attempted to address these two issues separately using different methods but lacked a unified approach to solve both problems simultaneously. This paper presents a single-stage object detector, referred to as scale and SA-based prohibited item detection, which consists of the multi-level fusion module (MLF) and the SA module. MLF is designed to learn multi-level features to enhance scale awareness, and SA is designed to learn and focus on object spatial information to enhance SA. Compared with previous research, our method shows outstanding performance in detection accuracy. Moreover, our single-stage object detector outperforms the accuracy of two-stage object detectors by 1.5 AP.