As an important component of the Earth’s ecosystem, soil moisture plays a vital role in the global water cycle and serves as an important parameter in the study of hydrology, meteorology, and agroecology. Based on the energy balance theory of underlying surface, the atmospheric temperature data recorded by an automatic weather station as well as unmanned aerial vehicle (UAV)-borne thermal infrared and multispectral remote-sensing data were used to establish inversion models of relative soil moisture at different depths based on remote-sensing UAV data and date from near-ground quadrats, respectively. Spatial differences and data accuracy verification were then performed using the 2017 spring wheat moisture data as a control. The results showed that: (1) the relative moisture of farmland soil can be effectively estimated using the proposed soil moisture inversion model. In terrestrial ecosystems, the ratio of actual to potential evapotranspiration, which is often used to characterize potential drought levels, is linearly correlated to soil moisture at different depths; (2) during the inversion of farmland soil moisture, the UAV-based observation method is superior to the near-ground quadrat observation method in both efficiency and accuracy. In addition, the relative soil moisture estimation model based on UAV data has a high accuracy, with R2 reaching 0.629, and a root mean square error (RMSE) of <0.100; and (3) the number and size of quadrats are important factors affecting the inversion accuracy. The data collected by the UAVs covered a wide range and had high spatial matching degree at the field scale. Especially, during estimation of the relative moisture of surface soil (0 to 10 and 0 to 20 cm), the linear fitting between the inversion model based on UAV data and the measured value was optimal. The error was minimal (RMSE < 0.07) and R2 was <0.714, so this method is more suitable for estimating and dynamically monitoring relative soil moisture of farmland at the field scale.