We present a fully automatic method for segmenting orbital structures (globes, optic nerves, and extraocular muscles) in CT images. Prior anatomical knowledge, such as shape, intensity, and spatial relationships of organs and landmarks, were utilized to define a volume of interest (VOI) that contains the desired structures. Then, VOI was used for fast localization and successful segmentation of each structure using predefined rules. Testing our method with 30 publicly available datasets, the average Dice similarity coefficient for right and left sides of [0.81, 0.79] eye globes, [0.72, 0.79] optic nerves, and [0.73, 0.76] extraocular muscles were achieved. The proposed method is accurate, efficient, does not require training data, and its intuitive pipeline allows the user to modify or extend to other structures.