As most countries are facing the growing population of seniors, automatic detection for abnormal behaviors has been a promising goal for a vision system operating in supportive home environment. In this paper, we investigate a novel approach for fall detection which is frequently observed in elderly people motions using a panorama camera mounting on the ceiling, we employ and modify a combination of two different features representing fall events: optical flow and human shape variation, which allows fall detection conducted from coarse to fine. In the pre-processing step, we analysis the raw video data to extract the meaningful motion region,then we designed an energy function as representing phase and magnitude of optical flow vector for the coarse detection in temporal domain, where the information entropy is adopted as the abnormal coefficient to estimate the consistency of motion directions. Once the optical flow changes abnormal, shape context descriptor is introduced to do the template matching for the fine detection, here we propose a novel shape matching descriptor which improves the rotation invariance based on the traditional shape context, while remaining its tolerance to most shape distortion. Our method is evaluated on a panorama-view fall detection database including fall events and confounding events, we demonstrate more effective performance and less computational costs on the fall detection regardless of challenging conditions and encourage the potential use of a vision-based system to provide safety and security in the homes of the elderly.