In this work, we present a novel network named CRRCNN (Cascade Rotational Region-based CNN) to detect dense objects with oriented bounding boxes. The CRRCNN consists of a Faster RCNN and a Cascade RCNN with Rotational RoIAlign. The Faster RCNN consists of RPN (Region Proposal Network) and RCNN (Region-based CNN). RPN generates horizontal bounding boxes. Rotational region proposals are generated through quadrilateral vertices regression of RCNN, and therefore Faster RCNN is regarded as a Rotational Region Proposal Network (RRPN). To generate accurate rotational bounding boxes, a Cascade RCNN with Rotational RoIAlign is proposed following the Faster RCNN, which will be demonstrated to be crucial for accurate arbitrary-oriented object detection, especially for dense objects. Feature Pyramid Network is also employed to obtain rich context information. The two networks mentioned above are unified and learned end-to-end by jointly optimizing. Experiments on the challenging DOTA dataset demonstrate the effectiveness of our approach.