There has been a rapidly growing interest in graphene-based optoelectronics. This exceptional material exhibits broadband optical response, ultrahigh carrier mobility and more importantly, potential compatibility with silicon complementary metal-oxide semiconductor (CMOS) technology. Here we present our recent works that integrate graphene with silicon channel waveguides and photonic crystal cavities. By coupling graphene to an optical cavity, we demonstrated an efficient electro-optic modulator that features a modulation depth of 10 dB and a switching energy of 300 fJ. Several high-speed modulators are also tested, showing a speed up to 0.57 GHz. In addition, we implemented a graphene photodetector on a silicon waveguide. The 53-μm-long graphene channel couples to the evanescent field of the waveguide mode, resulting in more than 60% absorption of the input light. We demonstrated a responsivity of 0.108 A/W in our photodetector. A data transmission of 12 Gbps and response time in excess of 20 GHz are also achieved. These results show the feasibility of graphene as a building block for silicon photonic integrated circuits. In particular, on-chip graphene active devices such as modulators and photodetectors are promising for their broadband response, high-speed operation, low power consumption and ease-to-fabrication.