Graphene, which is carbon arranged in atomically thin sheets, has drawn significant attention in many fields due to its unique electronic and optical properties. Photodetectors are particularly strong candidates for graphene applications due to the need for a broadband photoresponse from the ultraviolet to terahertz regions, high-speed operation, and low fabrication costs, which have not been achieved with the present technology. Here, graphene-based transistors were investigated as simple photodetectors for a broad range of wavelength. The photoresponse mechanism was determined to be dependent on factors such as the operation wavelength, the components near the graphene channel of the photodetector, and temperature. Here, we report the detailed mechanism that defines the photoresponse of graphene-based transistors. Graphene transistors were prepared using doped silicon (Si) substrates with a SiO2 layer, and source and drain electrodes. Single-layer graphene was fabricated by chemical vapor deposition, transferred onto the substrates, and the graphene channel region was then formed. The photoresponse was measured in the visible, near-infrared (NIR), and mid- and long-wavelength IR (MWIR and LWIR) regions. The results indicated that the photoresponse was enhanced by the Si substrate gating at visible wavelengths. Cooling was required at wavelengths longer than NIR due to thermal noise. Enhancement by the thermal effect of the insulator layer becomes dominant in the LWIR region, which indicates that the photoresponse of graphene-based transistors can be controlled by the surrounding materials, depending on the operation wavelength. These results are expected to contribute to the development of high-performance graphenebased photodetectors.