With the development of technology, artificial intelligence comes into our lives more and also comes up as a solution to many problems. Recently, deep learning approaches have been bringing fast and highly accurate solutions to problems. In this work, within the scope of Advanced Driver Assistance Systems (ADAS), deep learning based driver drowsiness detection system is proposed. First, face regions of drivers are detected using SSD MobileNet object detection method. The aim is to detect the eye, mouth and head positions of the drivers from this face region and to make a situation estimation with the combinations of these detected objects which are “normal”, “drowsy” and “danger”. The proposed approach examines the driver's behaviour over a certain period of time for making a decision, rather than a one-time eye closure or yawning decision. The detected eye, mouth and head positions are monitored and recorded over a period of time. Finally, these merged patterns are classified with Convolutional Neural Networks (CNN). Experimental results show that the performance of proposed novel CNN approach outperforms existing approaches in literature.