Nowadays, there are many families that live separately. Especially, communication with elderly person living alone is very important. We propose a new tele-surveillance system to support of their communications and to give an alarm for the accident on the elderly person living alone. The systems are set on two sites; the room for the elderly person living alone and the living room of his/her family's house. This system tracks the persons in real-time and analyzes the person's condition. Then, the system transmits the information of their head position and their condition to another site. The computer of recipient site generates the computer graphics (CG) animation of the tracked person (avatar) and display on a monitor. This method reduces traffic on network and keeps the privacy for the tracked person. The tracking part of this system uses the omni-directional image sensor that can capture a surrounding image in whole direction at a time by using a hyperbolic mirror and a video camera. The system detects the person's head in images captured by omni-directional image sensors. Then, the position of the person’s head in a room is computed. We made a prototype system of this proposed system with graphical user interface using touch panels. Experiments and evaluations showed good feasibility of the proposed system.