We demonstrate a hybrid material platform for high-speed integrated optical modulation through integration of graphene with silicon-on-insulator (SOI) substrates after adding a thin layer of an oxide material. The modulation is performed by charge accumulation in the graphene and Si layers of the resulting capacitor to change the index of refraction of both layers (through free-carrier plasma dispersion effect). The advantages of graphene layer include stronger free-carrier plasma dispersion effect, and larger carrier mobility (to achieve smaller device resistance and thus, higher operation speed). We also report solving some of the major challenges in achieving high-quality hybrid platform, especially avoiding the tearing of the graphene layer during the mechanical transfer through adding a layer of hexagonal boron nitride (h-BN) on the two sides of the graphene layer. The h-BN layer also works as an isolation layer to maintain the intrinsic carrier mobility of graphene. We demonstrate reduced graphene resistance by a factor of 3 through h-BN encapsulation. The potential performance measures of the resulting structure along with its extension to double-layer graphene modulators will be discussed. The hybrid graphene modulator has the potential for applications including optical interconnection, optical signal processing, and optical computing.