Worker-in-the-Loop Cyber-Physical System for Safe Human-Robot Collaboration in Construction
Publication: Computing in Civil Engineering 2021
ABSTRACT
Efficient implementation of robots in dynamic, human-populated construction sites will require the assistance and supervision of workers. Several hazards exist when workers are within the robots’ working envelope, including high mental stress induced by the robot as well as the risk of physical collision. To mitigate these risks, this study proposes a cyber-physical system (CPS) for safe, worker-centered human-robot collaboration (HRC). The physical components of the CPS include a wearable biosensor for assessing workers’ mental conditions and a video camera for evaluating the risk of collision accidents. The cyber component incorporates a machine learning layer that employs a kernelized logistic regression to translate worker bio-signals into distinct physiological states. Simultaneously, the system monitors the relative distance between workers and robots by applying computer vision techniques (i.e., video frame differencing, dilation, and contour detection). To test the feasibility of the proposed CPS, six subjects were asked to perform a series of bricklaying tasks jointly with a vehicle robot with different risks of collision (various distances from the robot) and at different levels of cognitive load (various tasks’ complexity). The findings revealed that the proposed CPS allows the co-bot to adjust its speed relative to the subject’s physiological states with 93.70% accuracy. In addition, the proposed system shows promising performance in collision avoidance with the accurate monitoring of position between robots and humans. The results unfold the feasibility of the proposed CPS in enhancing safe and intelligent HRC in construction under the trend of Industry 4.0.
Get full access to this chapter
View all available purchase options and get full access to this chapter.
REFERENCES
Allen, R. J., Baddeley, A. D., and Hitch, G. J. (2006). “Is the binding of visual features in working memory resource-demanding?” Journal of Experimental Psychology: General.
Bock, T., and Linner, T. (2016). “Single-Task Construction Robots by Category.” Construction Robots, Cambridge University Press, Cambridge, 14–290.
Choi, H. S., Han, C. S., Lee, K. Y., and Lee, S. H. (2005). “Development of hybrid robot for construction works with pneumatic actuator.” Automation in Construction, 14(4), 452–459.
Habibnezhad, M., Shayesteh, S., Liu, Y., and Fardhosseini, M. S. (2020). “The Architecture of an Intelligent Digital Twin for a Cyber- Physical Route-Finding System in Smart Cities.”
Halme, R. J., Lanz, M., Kämäräinen, J., Pieters, R., Latokartano, J., and Hietanen, A. (2018). “Review of vision-based safety systems for human-robot collaboration.” Procedia CIRP (The International Academy for Production Engineering), 72, 111–116.
Jebelli, H., Hwang, S., and Lee, S. (2018a). “EEG Signal-Processing Framework to Obtain High-Quality Brain Waves from an Off-the-Shelf Wearable EEG Device.” Journal of Computing in Civil Engineering, 32(1), 1–12.
Jebelli, H., Hwang, S., and Lee, S. H. (2018b). “EEG-based workers’ stress recognition at construction sites.” Automation in Construction, Elsevier B.V., 93, 315–324.
Karpenko, A., Jacobs, D., Baek, J., and Levoy, M. (2011). “Digital Video Stabilization and Rolling Shutter Correction using Gyroscopes.” Stanford Tech Report CTSR, 7(3).
Kartika, I., and Mohamed, S. S. (2011). “Frame differencing with post-processing techniques for moving object detection in outdoor environment.” Proceedings - 2011 IEEE 7th International Colloquium on Signal Processing and Its Applications, IEEE, 172–176.
Kulić, D., and Croft, E. (2007). “Pre-collision safety strategies for human-robot interaction.” Autonomous Robots, 22(2), 149–164.
Liu, Y., Habibnezhad, M., and Jebelli, H. (2021a). “Brain-computer interface for hands-free teleoperation of construction robots.” Automation in Construction, Elsevier B.V., 123, 103523.
Liu, Y., Habibnezhad, M., and Jebelli, H. (2021b). “Brainwave-driven human-robot collaboration in construction.” Automation in Construction, Elsevier B.V., 124, 103556.
Liu, Y., Habibnezhad, M., Shayesteh, S., Jebelli, H., and Lee, S. (2021c). “Paving the Way for Future EEG Studies in Construction: Dependent Component Analysis for Automatic Ocular Artifact Removal from Brainwave Signals.” Journal of Construction Engineering and Management, 147(8), 04021087.
Lotte, F., Congedo, M., Lécuyer, A., and Arnaldi, B. (2007). “A review of classification algorithms for EEG-based brain-computer interfaces.” Journal of Neural Engineering, 4(2).
Stamou, G. N., and Pitas, I. (2005). “Object tracking based on morphological elastic graph matching.” IEEE International Conference on Image Processing 2005, IEEE, I–709.
Suzuki, S., and Be, K. A. (1985). “Topological structural analysis of digitized binary images by border following.” Computer Vision, Graphics and Image Processing, 30(1), 32–46.
Szeliski, R. (2011). Computer Vision. Texts in Computer Science, Springer London, London.
Zhu, J., and Hastie, T. (2005). “Kernel Logistic Regression and the Import Vector Machine.” Journal of Computational and Graphical Statistics, 14(1), 185–205.
Information & Authors
Information
Published In
Computing in Civil Engineering 2021
Pages: 1075 - 1083
History
Published online: May 24, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Tuuli Turja, Who sets the pace in robotisation? ‘Working at a robotic pace’ according to the justifying, criticising and problem-solving discourse frames, Human Technology, 10.14254/1795-6889.2024.20-2.6, 20, 2, (361-383), (2024).
- Yogesh Gautam, Rosina Adhikari, Amit Ojha, Houtan Jebelli, William E. Sitzabee, Predictive Maintenance of Building Facility: A Digital Twin Framework Using Long Short-Term Memory Encode–Decode Model, Digital Twins in Construction and the Built Environment, 10.1061/9780784485606.ch8, (173-188), (2024).
- Yizhi Liu, Amit Ojha, Shayan Shayesteh, Houtan Jebelli, SangHyun Lee, Human-centric robotic manipulation in construction: generative adversarial networks based physiological computing mechanism to enable robots to perceive workers’ cognitive load, Canadian Journal of Civil Engineering, 10.1139/cjce-2021-0646, 50, 3, (224-238), (2023).
- Yizhi Liu, Houtan Jebelli, Intention‐aware robot motion planning for safe worker–robot collaboration, Computer-Aided Civil and Infrastructure Engineering, 10.1111/mice.13129, 39, 15, (2242-2269), (2023).
- Xiaoshan Zhou, Pin‐Chao Liao, Brain‐regulated learning for classifying on‐site hazards with small datasets, Computer-Aided Civil and Infrastructure Engineering, 10.1111/mice.13078, 39, 3, (458-472), (2023).
- Xiaoshan Zhou, Pin-Chao Liao, Weighing Votes in Human–Machine Collaboration for Hazard Recognition: Inferring a Hazard-Based Perceptual Threshold and Decision Confidence from Electroencephalogram Wavelets, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13351, 149, 9, (2023).