Modern agricultural practices require accurate prediction of crop yields, particularly in the face of a changing climate. Despite the improvement in estimating crop yield over the years through machine learning (ML) algorithms, the increased volatility and complexity of weather patterns continue to make use of conventional ML models unreliable for understanding intricate relationships. We therefore explore the potential of advanced ML, specifically a transformer-based architecture coupled with a temporal convolutional network (TCN) for crop yield prediction. We argue that transformers’ ability to model long-range dependencies within data sequences makes them well suited to handle complex relationships in comprehensive crop datasets. In addition, the integration of a TCN would complement the transformer’s strengths by focusing on temporal feature extraction. Alongside climatic variables, the study integrates soil properties, moderate resolution imaging spectroradiometer (MODIS), and average yield to analyze the complex factors influencing crop growth and yield. The proposed TCN-transformer (TCNT) model is trained and evaluated using corn and soybean yield values selected from 264 counties within Illinois, Iowa, and Wisconsin (the United States corn belt region). Furthermore, experimental results show the superior performance of our TCNT framework over other state-of-the-art models for both in-season and end-of-season yield predictions.