We observe ultrafast and large nonlinear optical responses of a semiconductor thin film where excitons are weakly confined and their center-of-mass motion is quantized. The observed degenerate four-wave mixing (DFWM) signal of a GaAs double-hetero-structure layer exhibits an anomalous thickness-dependence, and is much enhanced at the thickness of 110nm. The decay time of the response is the order of ps, which depends on the thickness of the layer and on the number of the layer. These phenomena are elucidated by the theory of the nonlocal interaction between excitonic states and the light beyond the long wavelength approximation regime. The enhancement at the thickness is due to the size-resonant enhancement of the internal field with a nano-scale spatial structure, where the second quantized level of the exciton center-of-mass motion mainly contributes to the large response at the thickness. We, furthermore, demonstrate ultrafast response in a reflective type nonlinear polarization rotation switch with the 110nm thick three double-hetero. The signal intensity of 20% for the reflective pulse was obtained at a pump intensity of 10nJ/cm2, and the decay time is 1.5ps at low temperatures. These results provide a novel design of a semiconductor structure in order to obtain both large and ultrafast nonlinear response that is suitable for device application.