Vehicle-based monitoring has the potential to become an accurate and cost-efficient way to monitor infrastructure assets, but a number of challenges must be addressed for such a technique to be implemented widely. The majority of vehicle-based infrastructure sensing has assumed that the vehicle’s speed profile is identical every time it passes over the asset of interest. Ultimately, however this technology will be most practical if damage detection schemes can be applied regardless of the speed of the vehicle. Thus methods must be designed to handle speed variability to make this method more practical. In this paper we investigate the effects of variable speed when monitoring infrastructure from the dynamic response of a passing vehicle, which we measure by placing accelerometers on the vehicle of interest. We have conducted a series of laboratory tests to study this phenomenon, in which a vehicle crosses over a scaled model bridge structure with a varying speed profile. We quantify the ability of several features to detect changes in the infrastructure, independent of the variable speed. We show that aligning signals to normalize for speed variability improves the classification results. This work brings us closer to the ultimate goal of using vehicle-based monitoring to ensure more efficient and more reliable infrastructure in the future.