Inter national J our nal of Electrical and Computer Engineering (IJECE) V ol. 6, No. 4, August 2016, pp. 1602 1609 ISSN: 2088-8708, DOI: 10.11591/ijece.v6i4.9798 1602       I ns t it u t e  o f  A d v a nce d  Eng ine e r i ng  a nd  S cie nce   w     w     w       i                       l       c       m     F ast Obstacle Distance Estimation using Laser Line Imaging T echnique f or Smart Wheelchair Fitri Utaminingrum, Hurriyatul Fitriyah, Randy Cah ya W ihandika, M Ali F auzi, Dahnial Syauqy , Rizal Maulana F aculty of Computer Science, Bra wijaya Uni v ersity , Malang, Indonesia Article Inf o Article history: Recei v ed Dec 25, 2015 Re vised May 26, 2016 Accepted Jun 9, 2016 K eyw ord: Obstacle a v oidance Distance approximation Laser line image Blob method Smart wheelchair ABSTRA CT This paper presents an approach of obstacle distance estimation for smart wheelchair . A smart wheelchair w as equipped with a camera and a laser line. The camera w as used to capture an image from the e n vi ronment in order to sense the pathw ay condition. The laser line w as used in combination with camera to recognize an obstacle in the pathw ay based on the shape of laser line image in certain angle. A blob method detection w as then applied on the laser line image to separate and recognize the pattern of the detected obstacles. The laser line projector and camera which w as mounted in fix ed-certain position ensured a fix ed relation between blobs-g ap and obstacle-to-wheelchair distance. A simple linear re gression from 16 obtained data w as used to respresent this relation as the estimated obstacle distance. As a result, the a v erage error between the estimation and the act ual distance w as 1.25 cm from 7 data testing e xperiments. Therefore, the e xperiment results sho w that the proposed method w as able to estimate the distance between wheelchair and the obstacle. Copyright c 2016 Institute of Advanced Engineering and Science . All rights r eserved. Corresponding A uthor: Fitri Utaminingrum F aculty of Computer Science, Bra wijaya Uni v ersity Malang, Indonesia f 3 ning r um @ ub:ac:id 1. INTR ODUCTION An automatic mo vi ng object such as smart wheelchair requires a se n s ing ability to w ards its en vironment con- dition. Basicall y , Smart Wheelchair is a con v entional wheelchair which is actuated by electrical motor and controlled by a central processing unit so that it can perform set of actions based on the instruction from the user . These user instructions are generally gi v en via jo ysticks or human v oice such as done by Al-Rousan and Assaleh in 2011 [1]. Ho we v er , the user still plays an important role to guide and monitor the mo v ement of the wheelchair especially when there are obstacles in the front or beside the wheelchair . One of the most important information to concei v e is obstruc- tions in pathw ay which are b umpiness, hole and presence of obstacle. The smart wheelchair w ould ha v e to decide an action once the poor pathw ay condition is detected. An a v oidance system for these conditions plays important role to secure the mo ving object or the rider . Therefore, in order to pro vide more con v enient use of the smart wheelchair , it is important to de v elop a better method to sense and detect obstacles in the en vironment around it. Generally , a processing of en vironmental image captured by a camera is performed to sense the pathw ay condition. In this paper , a detection of poor pathw ay condition by se n s ing the vie w of en vironment is presented. The sight sensing utilizes camera, then the images captured were analysed using image processing method. Ho we v er , by only utilizing a camera, the process of recognizing a poor pathw ay will need a longer time and comple x computation. Therefore, in order to simplify image analysis and reduce algorithm comple xity , a laser will be used in combination with camera based image processing system such as implemented by Zhang [2] on weld line detection. Based on the inno v ation and laser scanning method performed by T ian [3], we propose a ne w method to detect and recognize a poor pathw ay based on the shape of the l aser line image shot in certain angle and then captured by a camera. After obtaining the image, a blob method detection w as performed to separate and recognize the pattern o f the obstacles. Before performing the obstacle detection by using laser line image, it is preferred to perform filtering process to obtain good quality image source such as performed by Utaminingrum [4]. This study focuses on de v eloping a lo w-computationally poor pathw ays detection method that implement the J ournal Homepage: http://iaesjournal.com/online/inde x.php/IJECE       I ns t it u t e  o f  A d v a nce d  Eng ine e r i ng  a nd  S cie nce   w     w     w       i                       l       c       m     Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE ISSN: 2088-8708 1603 use of microcomputer which is embedded in a smart wheelchair system. An acti v e imaging method that utilize a laser to illuminate the re gion of interest is selected to simplify image analysis and reduce algorithm comple xity . In detail, this paper is di vided into v e sections. Section 2 pro vides the o v ervie w of related w orks. In section 3, the proposed method will be discussed and then section 4 will pro vide results and discussion. Finally , section 5 pro vides conclusion and future w ork. 2. RELA TED W ORK Man y approaches ha v e been proposed on obstacle detection and collision a v oidance. Obstacle detection based on ultrasonic sensor has been widely used in autonomous mobile robot application. Ultrasonic sensor w as used to map the obstacle by measuring the distance between the robot and obstacles. A simple approach for detecting obstacle and a v oiding collision has been proposed such by Gageik in a quadrocopter [5] and o v erlapped ultrasonic sensor method by Kim [6]. Ev en though ultrasonic are useful in smok y en vironment, the e xperiment sho wed not all surf aces can be detected which mak es other sensor were required. Another general problem, distance estimation using ultrasonic requires additional time since t he w a v e needs to tra v el into the surf ace of the obstacle and return back to the recei v er . F arther distance e v en requires longer tra v el time of the ultrasonic w a v e. In 2011, Dreszer et al. implemented the use of Microsoft Kinect sensor to monitor and learn the en vironment for obstacle detection and collision a v oidance [7]. The Kinect sensor w as equipped with infrared depth camera and run in small PC which w as then mounted on an in v erted pendulum robot so it can mo v e and a v oid obstacles. Similarly , Nissimo v et al. implemented Kinect sensor in an agricultural robotic v ehicles to e xplore greenhouse en vironment [8]. The research pro vided an approach for obstacle detection and a v oidance by using color and depth information obtained from Kine ct 3D. Ho we v er , by using Kinect sensor , shin y and smooth surf ace of an object such as glass will pre v ent infrared w a v e to be reflected back which cause some error . Barreto et al. proposed a method to measure a distance based on laser distance triangulation for image processing [9]. He de v eloped a measurement technique on an embedded system which required smaller processing po wer compared to personal computer as x86 architecture. He use d a CMOS camera and a laser to implement the laser triangul ation distance measurement . Ho we v er , the system w as only aimed to det ect the distance. F or a mo ving wheelchair , instead of only considering the distance, it may also g ain benefit if the system can recognize the height of an obstacle to decide the ne xt action: whether to completely a v oid or to dri v e abo v e it. 3. PR OPOSED METHOD This study aims to detect an obstacle based on the response gi v en by the laser line projection on the pathw ay . First, a line of laser w as projected onto the pathw ay and a colored image of its vie w w as acquired using a fix ed CCD (Char ge-Coupled De vice) camera. A laser light w as chosen for its focus and high intensity characteristic. The laser line and the camera were mounted in specific angle triangular configuration which will be discuss ed more in the ne xt subsection. The captured image w as then analyzed through image processing methods which consist of color -space con v ersion, se gmentation, closing morphol og i cal filtering and blob detection (Figure 1). The number of detected blobs and their centroids w ould be used as features in the obstacle distance calculation. The main processing unit used to process the image w as Raspberry Pi 2 with 900 MHz quadcore ARM Corte x-A7 CPU and 1 GB of RAM equipped with standard Raspbian Wheezy OS and OpenCV v ersion 3.0. 3.1. Image acquisition of P athways view The laser line used in this study w as LN60-650 (class 2) that has 650 nm w a v elength and 60 f an angle. It projected a focused red line onto the pathw ay . The laser line projector w as mounted in v ertical pole in fix ed height of 1.4 metres abo v e the floor and at fix ed angle of 60 resulting a projection at 2.44 meters a w ay horizontally from the laser mounted on the wheelchair . A CCD camera w as used to acquire the images that result RGB color images each of 320x240 pix el size. This camera w as also mounted in a horizontal pole with fix ed 0.75 meters height abo v e the floor . Since the laser and camera were placed in a fix ed position, the projected laser w as ensured to be in a fix ed location in the captured image as well. The triangular configurations of laser line image and CCD camera are sho wn in Figure 2 while the acquired image illustration i s sho wn in Figure 3. These configurations were arranged to assure the wheel-chair w ould response to obstacle in 1 meter beforehand in order to secure the rider from stumbling o v er or crashing into the obstacle. F ast Obstacle Distance Estimation using Laser Line Ima ging T ec hnique for Smart Wheelc hair (F Utaminingrum) Evaluation Warning : The document was created with Spire.PDF for Python.
1604 ISSN: 2088-8708 Figure 1. Feature e xtraction of obstacle detection using image processing. (a) Mounting (b) Angle configuration Figure 2. Laser line and camera mounted on the wheelchair Figure 3. Captured image illustration. IJECE V ol. 6, No. 4, August 2016: 1602 1609 Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE ISSN: 2088-8708 1605 3.2. Color -Space Con v ersion fr om RGB to HSV The acquired image in RGB color -space w as then con v erted into HSV color -space. HSV color -space which comprises Hue, Saturation and V alue is mainly used in se gmentation where the object of interest is a color -specific. The study done by Mesk o in 2013 and by Chmelar in 2015 used this color -space to detect red laser projection [11, 10]. 3.3. Laser -Line Segmentation using Thr esholding Method Mesk o and Chmelar used laser line that has characteristics of red color and high intensity . In this study , the red color has Hue of (0, 70, 70) to (255, 255, 255) in 0 - 255 intensity le v el. This range w as then used as threshold to se gment the red laserline from the background. Pix els in the cropped images whose intensity f all within both thresholds w as then mask ed as 1, otherwise it w as a non-laser object and mask ed as 0. 3.4. Mor phological Filtering Closing is a type of morphological filter for binary image that uses a structuring element to produce refined binary image within bounding box on the se gmented image. The closing used dilation process follo wed by erosion. In this study , a 2x2 square structuring elements is used for the closing process. If there is a v alue of 1 in the se gmented image, then dilation process will gi v e v alue of 1 into its surrounding 2x2 pix els. In the other hand, erosion process change the v alue of 1 in the se gmented image into 0 i f the 2x2 surrounding pix els did not follo w the structuring element. Both process w ould produce an enclosed se gmented laser line. The closing process is then follo wed by another dilation process using a horizontal-shape structuring element of 3x15 in order to connect the line discontinuities. 3.5. Blob Analysis Figure 4. Captured laser line image and the labeled-blob Image. A blob in binary image is a re gion of adjacent connected-component. This method w as performed after se g- mentation to label each contiguous fore ground pix els. Labelling is a process of scanning fore ground pix els (intensity of 1) from top-right to the top-left and mo v e to subsequent lo wer pix els [12] and label each fore ground pix els found in ascending order . This algorithm processes e v ery ro w at a time. The first fore ground pix el found is labeled as 1 and the second as 2 and so forth. This study used an 8-connected fore ground which means consecuti v e fore ground pix els w as assigned to its north, east, south, west, north-east, south-east, north-west and south-west neighbor’ s label. Each label w as then assigned as a blob . Figure 4 sho ws the result of blob detection on the laser line image. There are three detected blobs which were illustrated as circles on the laser line image since the line w as split into three parts. More obstacles generated more blobs such as sho wn in Figure 5. Therefore, it is still possible to detect small and thin objects such as the foot of a chair and table. The centroids were then calculated using the moment of the image or the center of mass such as illustrated in Equation 1 and 2. center x is centroid coordinate in horizontal axis, while center y is centroid coordinate in v ertica l axis. n represents number of pix els belonging to the se gmented blob . x k and y k are the coordinates of blobs in horizontal and v ertical axis respecti v ely . The number of label and all centroids w ould be used in obstacle detection and obstacle-to-wheelchair distance estimation. F ast Obstacle Distance Estimation using Laser Line Ima ging T ec hnique for Smart Wheelc hair (F Utaminingrum) Evaluation Warning : The document was created with Spire.PDF for Python.
1606 ISSN: 2088-8708 Figure 5. Captured laser line image and the labeled-blob Image. center x = 1 n n X k =1 x k (1) center y = 1 m m X k =1 y k (2) 3.6. Obstacle-Distance Estimation using Linear Regr ession A no obstacle condition w as defined where only 1 blob w as detected in the labeled-blob image, meaning that there w as no object destructs the laser projection. Once an obstacle presents in the pathw ay , the number of connected-components label becomes tw o or three where one higher blob appears on the obstacle. T w o labels mean the obstacle presents on the side of pathw ay while three labels mean the obstacle presents in the middle of pathw ay . The centroid coordinate of higher blob ( center H x ; center H y ) w as then subtracted with the centroid of lo wer blob ( center L x ; center L y ) resulting a blobs-g ap. The laser projector and camera that w as mounted in fix ed position ensured a fix ed relation between blobs- g ap and obstacle-to-wheelchair distance. A simple linear re gression w as used to respresent this relation. Gi v en there are n number of collected blobs-g aps ( x i ), y i correspond to obstacle-to-wheelchair distances, x and y are respecti v e a v erages, the coef ficient a and constant b of linear re gression y = ax + b were calculated using Equation 3 and 4. a = P n i =1 ( x i ( x ))( y i ( y )) P n i =1 ( x i ( x )) 2 (3) b = y a x (4) In addition of measuring the distance, the triangular configuration of the laser line and the camera also pro vides indirect information about the height of the obstacle based on assumption. When there are more than 1 detected blob, it means that there is an obstacle obstructing the laser line projection. If the wheelchair mo v es forw ard, the pix el distance of detected bl ob will increase and it will be used to calculate the distance. Ho we v er , if after specific time the blob number get lo wered, then it can be concluded that the height of the obstacle is lo w , and the wheelchair can continue to mo v e forw ard and pass o v er the obstacle. Therefore, the hei ght assumption of the obstacle can be used to decide what action should be tak en by the wheelchair; whether it should stop or pass o v er the obstacle. 4. RESUL T AND DISCUSSION Re gression analysis of the relation between blobs -g a p and the actual obstacle-to-wheel chair distance is sho wn in Figure 6. In order to obtain the re gression formula, 16 data with dif ferent actual obstacle distance were captured using laser line imaging technique. There are tw o types of dat a, the triangle dots sho w the data for obstacle with 20 cm width while the circle dots sho w the data for obstacle with 33 cm width. IJECE V ol. 6, No. 4, August 2016: 1602 1609 Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE ISSN: 2088-8708 1607 From the data sho wn in the Figure, the re gression formula can be obtained from the number of pix el and actual distance by using Equation 3 and 4: y = 1 : 06 x + 155 : 73 (5) Figure 6. Re gression Analysis of the Blob Distance in Pix els and the Actual Distance. Where x is the number of pix el, and y is the distance between the wheelchair and the obstacles. Finally , Equation 5 w as then used to estimate actual distance of 7 testing data. The results of these calculation are sho wn in T able 1. From the T able, it can be concluded that the a v erage error of measurements is 1.25 cm. The w orst distance estimation happened when the actual distance w as 80 cm, b ut w as estimated as 74.19 cm. it w as probably caused by bad data on the training phase whi ch lead to less accurate re gression formula. The best di stance estimation happened when the actual distance w as 90 cm where it only g a v e error of 0.06 cm. The captured line laser image had 320x240 pix els which w as completely computed in 83 ms. T able 1. Experiment Results Using Re gression F ormula. Actual (cm) Blob Pixels (pixels) Estimated (cm) Err or (cm) 80 76.92 74.19 5.81 90 62.07 89.94 0.06 100 52.94 99.61 0.39 110 43.31 109.82 0.18 120 33.01 120.74 0.74 130 24.68 129.57 0.43 140 15.93 138.84 1.16 A v erage 1.25 5. CONCLUSION In this paper , a method to estimate the distance between wheelchair and obstacle has been presented. By using laser line along with camera which w as mounted in fix ed-certain position, a linear re gression can be performed to relate the number of pix els in the captured image with the actual distance between the wheelchair and obstacles. A F ast Obstacle Distance Estimation using Laser Line Ima ging T ec hnique for Smart Wheelc hair (F Utaminingrum) Evaluation Warning : The document was created with Spire.PDF for Python.
1608 ISSN: 2088-8708 simple linear re gression from 16 obtained data w as used to represent this relation as the estimated obstacle distance. As a result, the a v erage error between the estimation and the actual distance w as 1.25 cm from 7 data testing e xperiments. From this result, it can be seen that this method is able to estimate the distance between wheelchair and the obstacles. Ho we v er , there were still dra wbacks in this research since it hea vily depends on the laser line image. One of the problem happened when the light captured by the camera w as too bright which mak e it hard to obtain clear laser line image. REFERENCES [1] M. Rousan and K. As saleh, ”A W a v elet and Neural Netw ork-based V oice System for a Smart Wheelchair Control”, J ournal of The F r anklin Institute , v ol. 348:90-100, 2011. [2] L. Zhang, et al., ”A No v el Laser V ision Sensor for W eld Line Detection on W all-climbing Robot”, Optics and Laser T ec hnolo gy , v ol. 60:69-79, 2014. [3] Q. T ian, et al., ”An Experimental Ev aluation Method for the Performance of a Laser Line Scanning System with Multiple Sensors” Optics and Laser T ec hnolo gy , v ol.52:241-249, 2014. [4] F . Utaminingrum, et al., ”Speedy Filters for Remo ving Impulse Noise based on an Adapti v e W indo w Observ a- tion”, International J ournal of Electr onics and Communications V ol 69(1):95-100, 2015. [5] N. Gageik, et al., ”Obstacle Detection And Collision A v oidance Using Ultrasonic Di stance Sensors F or An Autonomous Quadrocopter”. Univer sity Of W urzb ur g , Aer ospace Informati o n T ec hnolo gy (Germany) W urzb ur g , 2012. [6] S. Kim S. and H. Kim, ”Simple and Comple x Obstacle Detection Using an Ov erlapped Ultrasonic Sensor Ring”, 12th International Confer ence on Contr ol, A utomation and Systems Oct. 17-21, K orea, 1992. [7] D. Dreszer , et al., A utonomization of T wo-Wheeled In verted P endulum Robot Kinect-Based Obstacle Detection and A voidance System , Springer -V erlag Berlin Heidelber g, 2011. [8] S. Nissimo v , et al., ”Obstacle Detection in a Greenhouse En vironment us ing the Kinect Sensor”, Computer s and Electr onics in Agricultur e 113:104115, 2015. [9] S. V . F . Barreto, et al., ”A Method for Image Processing and Distance Measuring based on Laser Distance T rian- gulation”, Electr onics, Cir cuits, and Systems (ICECS) 2013 IEEE 20th International Confer ence 695698, 2013. [10] P . Chmelar , et al., ”The Laser Color detection for 3D Range Scanning Using Gaussian Mixture Model”, 25th International Confer ence of Radioelektr onika , P ardubice, 248 - 253, 2015. [11] M. Mesk o and S. T ot, ”Laser Spot Detection”, J ournal of Infr omation, Contr ol and Mana g ement System 11(1): 35 - 41, 2013. [12] R. M. Haralick and L. G. Saphiro, Computer and Robot V ision , Addison-W esle y , 1992. BIOGRAPHIES OF A UTHORS Fitri Utaminingrum w as born in Surabaya, East Ja v a, Indonesia. She recei v ed her Bachelor de- gree in Electrical Engineering (BEng.) from National Institute of T echnology (2000-2004), and master de gree in the same major (MEng.) from Bra wijaya Uni v ersity Malang, Indonesia in 2007. In addition, she obtained the de gree of Doctor of Engineering in the field of Computer Science and Electrical Engineering from K umamoto Uni v ersity , Japan (2011-2014). She al so has successfully completed International Joint Education Program from Science and technology at Graduate School of Science and T echnology , K umamoto Uni v ersity , Japan. She has been w orking as part time lec- turer in se v eral institution, such as Sek olah T inggi T eknik Angkatan darat (STT AD) from 2006 until 2007. Sek olah T inggi T eknik Atlas Nusantara (STT AR) start from 2006 until 2007. In addition, she also has been teaching the students of V ocational Education De v elopment Center (VEDC) Malang- Indonesia at 2007 and Malang Joint Campus (MJC) at 2007. She has been full time l ecturer in Bra wijaya Uni v ersity start from 2008. IJECE V ol. 6, No. 4, August 2016: 1602 1609 Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE ISSN: 2088-8708 1609 Hurriyatul Fitriyah is a lecturer in F aculty of Computer Science, Uni v ersity of Bra wijaya. She recei v ed the bachelor de gree in Ph ysics Engineering from Institut T eknologi Sepuluh Nopember (ITS), Surabaya, Indonesia in 2007 and the Master De gree in Electrical and Electronic Engineering from Uni v ersiti T eknologi Petronas (UTP), Perak, Malaysia in 2012. Her research interest includes computer vision and pattern recognition. She is one of the author in ”Surf ace Imaging for Biomedi- cal Application” published by CRC Press in 2014 and the co-in v entor in patent of ”A methodology and Apparatus for Objecti v e Assessment and Rating of Psoriasis Lesion Thickness using Digital Imaging” in Malaysia, German and US. Randy Cah ya W ihandika , male, recei v ed the bac helor de gree from Electronic Engineering Poly- technic Institute of Surabaya, Indonesia, in 2011 and master de gree at Department of Informatics, Institut T eknologi Sepuluh Nopember , Surabaya, Indonesia in 2013. His research interests include computer vision, digital image processing, and pattern recognition. M. A li F auzi is curre ntly w orking at Intelligent System Laboratory , Bra wijaya Uni v ersity . H e ob- tained his Bachelor De gree in Inf ormatics from Institut T eknologi Sepuluh Nopember (Indonesia) in 2011. His Ma ster De gree in Informatics obtained from Institut T eknologi Sepuluh Nopember (Indonesia) in 2013. His researches are in fields of Intelligent Syste m, and Natural Language Pro- cessing. Dahnial Syauqy recei v ed Bachelor De gree in Electrical Engineering from Bra wijaya Uni v ersity (Indonesia) in 2009. He recei v ed his Master De gree in Electrical Engineering from National Cen- tral Uni v ersity (T aiw an) in 2014. He is currently w orking at Laboratory of Computer System and Robotics in Bra wijaya Uni v ersity . His current research interests focus in the areas of electronics, embedded system and signal processing. Rizal Maulana is currently w orking at Laboratory of Computer System and Robotics in Bra wijaya Uni v ersity . He obtained Bachelor De gree in Elec trical Engineering from Bra wijaya Uni v ersity (Indonesia) in 2011. His Master De gree in Electrical Engineering obtained from National Central Uni v ersity (T aiw an) in 2014. His researches are in fields of electronic s, robotics and biomedical signal processing. F ast Obstacle Distance Estimation using Laser Line Ima ging T ec hnique for Smart Wheelc hair (F Utaminingrum) Evaluation Warning : The document was created with Spire.PDF for Python.