Since the concept of structural control was proposed by J.P.T. Yao, tremendous progress has been made over the last three decades toward making active structural control a viable technology for enhancing structural functionality and safety against natural hazards such as strong earthquakes and high winds. However, because of the high dimensionality and multiple-input-multi-output nature of civil structure model, it is difficult to design a control strategy to achieve desired stability, robustness with centralized control method. Hierarchical decentralized control strategy is employed in this paper in response to this difficulty. In this approach, the structure to be controlled is divided into a set of substructures. A hierarchical decentralized control consists of two levels: the low-level subsystem by which each substructure is controlled independently by local controllers and local information, and the high-level system that we call it global control system, which takes the outputs of every substructure as its input, and eliminates interconnection among substructures. With LQR active control algorithm in Matlab environment, the centralized control and hierarchical decentralized control strategy are implemented on a 20-storey shear building under dynamic excitation. The simulated results show that both the centralized control and hierarchical decentralized control are able to control the vibration of building. Compared with centralized control, hierarchical decentralized control method has a good control effect with preferably suppressed vibration response and less control force. The study in this paper presents a promising technology for structural vibration control.