Coherent control of nonlinear and ultrafast plasmon-polariton mediated interactions has attracted wide attention for its potential for enhancing functionality in nano-scale photonic devices and applications. Contemporary research in ultrafast and nonlinear plasmonics primarily utilizes noble metals, such as gold and silver, as material platforms because of their high performance both in linear and nonlinear optical properties. Unfortunately, noble metals possess numerous drawbacks including low melting points, chemical instabilities, and an incompatibility with standard CMOS processing techniques, all of which hamper their incorporation into functional plasmonic devices. Here we investigate the mid-infrared ultrafast and nonlinear properties of the alternative plasmonic material, aluminum-doped zinc oxide (AZO). By performing time-resolved pump-probe spectroscopy, we observe an unprecedentedly large and ultrafast (sub-picosecond) response in AZO thin films for both intra- and inter-band pumping frequencies. These two nonlinearities arise from distinct electron excitation dynamics and, as such, can be controlled simultaneously and independently to provide a novel method of dynamic tunability. We demonstrate this phenomenon with two-color excitation and find our AZO films exhibit a THz modulation bandwidth. We also probed the nonlinear response of AZO films at the epsilon-near-zero (ENZ) frequency and observed a dramatic increase in the Kerr nonlinearity with an induced refractive index change on the order of unity. In summary, our ultrafast and nonlinear studies strongly support AZO as an alternative plasmonic material with qualities pertinent to the development and realization of practical plasmonic technologies.