A framework for using contactless technologies in operating rooms

Gulmez G., Cebbar K., Kul S., SAYAR A.

2021 International Conference on INnovations in Intelligent SysTems and Applications, INISTA 2021, Kocaeli, Turkey, 25 - 27 August 2021 identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/inista52262.2021.9548346
  • City: Kocaeli
  • Country: Turkey
  • Keywords: Contactless Interaction, Human Computer Interaction, Image Processing, Motion Detection, Speech-to-Text, Virtual Mouse
  • Kocaeli University Affiliated: Yes


© 2021 IEEE.Contactless technologies, which have become widespread with technology advancement, have evolved in many fields such as industry, health, and construction. The applications of these technologies in the medical field are of great importance for the future of modern medicine. Operating rooms are one of the best examples of using those technologies in medicine. The systems to be used in operating room environments are expected to meet some medical requirements, be useful, and be prepared for critical scenarios. Sterilization is one of these requirements needs to be met during the operation. This paper aims to provide a solution to sterilization problems in operating rooms by using voice and gesture commands. In this way, doctors can complete their jobs by checking the working environment and the patient's condition during a surgery. The proposed system is developed on DICOM viewer called Weasis. OpenCV is used to detect hand movements via the camera, and Kaldi is used to detect voice commands. The detector created with Kaldi runs on Docker to integrate with other components. Commands received through hand gestures can directly change the position of the mouse and run Weasis interface commands. When sufficient environmental conditions are not provided, the error rate is reduced by using hand and voice commands together.