Multimedia annotation is domain specific and is assigned with the help of a domain expert to semantically enrich the data. These annotations are used for not only retrieval tasks but also to answer domain specific complex queries. To accomplish this, we propose to use MPEG-7 to annotate medical images and capture semantic information. In particular, we discuss the MPEG-7 based annotations for images of a human brain. Using MPEG-7, human brain images can be represented in an XML format. This MPEG-7 based XML file can be used to store the semantic medical information along with the low level features of the image. We also present the database design to store and query the patient images for image-guided neurosurgery.