Recognizing the online Arabic handwritten script has been gaining more interest because of the impressive advances in mobile device requiring more and more intelligent handwritten recognizers. Since it was demonstrated within many previous research that Deep Neural Networks (DNN) exhibit a great performance, we propose in this work a new system based on a DNN in which we try to optimize the training process by a smooth construct of the deep architecture. The Output’s error of each unit in the previous layer will be computed and only the smallest error will be maintained in the next iteration. This paper uses LMCA database for training and testing data. The experimental study reveals that our proposed DBNN using generated Bottleneck features can outperform state of the art online recognizers.