张俊豪, 罗国富, 杨幸博, 等.基于深度学习和机器视觉的多源数据感知技术研究[J].河南理工大学学报（自然科学版）,2021,40(4):107-113.

ZHANG J H, LUO G F, YANG X B, et al.Study on multi-source data perception technology based ondeep learning and machine vision[J].Journal of Henan Polytechnic University(Natural Science) ,2021,40(4):107-113.

基于深度学习和机器视觉的多源数据感知技术研究

张俊豪, 罗国富, 杨幸博, 李亚为, 张宁波

郑州轻工业大学 机电工程学院，河南 郑州450002

摘要:机械臂位姿误差容易导致运行轨迹偏离，影响机械臂定位效果。为此，将机器视觉技术与深度学习算法有机结合，形成一种高性能的多源数据感知方法，以采集高精度的机械臂位姿数据。首先，基于视觉成像原理，设计多摄像机机械臂位姿数据采集方案布局，以获取视觉图像坐标与机械臂位姿的关系；其次，以卷积神经网络为核心，构建具有5层卷积层、4层最大池化层以及3层全连接层的深度学习模型，用以融合多摄像机采集的机械臂多源图像数据；再次，运用批量梯度下降法，优化模型的卷积核犷和偏置参数b ，以深度刻画图像特征；最后，结合机械臂位姿模型，得到精准的运行位姿数据。经测试检验，用本文方法感知机械臂的仰俯角、偏航角、翻滚角的最大误差值均小于0.6°，数据感知度较高，可以为机械臂工作路线的规划、机械臂行为的精准控制提供准确的数据基础。

关键词:深度学习;机器视觉;机械臂;多源数据;位姿;感知

doi:10.16186/j.cnki.1673-9787.2020040048

基金项目:国家自然科学基金资助项目（51775517）；河南省重点科技攻关项目（202102210396，202102210262）

收稿日期:2020/04/09

修回日期:2020/06/09

出版日期:2021/07/15

Study on multi-source data perception technology based ondeep learning and machine vision

ZHANG Junhao, LUO Guofu, YANG Xingbo, LI Yawei, ZHANG Ningbo

College of Mechanical and Electrical Engineering， Zhangzhou University of Light Industry， Zhangzhou 450002 ， Henan， China

Abstract:The position errors of the manipulator may easily cause the deviation of the running trajectory and affect the positioning effect of the manipulator. Therefore， a high-performance multi-source data sensing method was formed to collect high-precision manipulator position data by combining machine vision technology with deep learning algorithm. Firstly， based on the principle of visual imaging， the layout of the position data acquisition scheme of the manipulator with multi-camera was designed to obtain the relationship between the visual image coordinates and the posture of the manipulator. Next， by using the convolutional neural network as the core， a deep learning model with 5 convolutional layers ，4 maximum pooling layers and 3 fully connected layers was constructed to fuse the multi-source image data of the manipulator collected by multi-camera. Then， by using the method of batch gradient descent， the convolution kernel and offset parameter bof the model were optimized to characterize the image features in depth. Finally， the accurate operation position data were obtained by combining with the position model of the manipulator. After testing and verification， the maximum error values of the studied method for sensing the pitch angle，yaw angle and roll angle of the manipulator were all less than 0.6°. The data perception degree was high， and it could provide an accurate data basis for planning the working route of the manipulator and for accurately controlling the behavior of the manipulator.

Key words:deep learning;machine vision;manipulator;multi-source data;position;perception

 基于深度学习和机器视觉的多源数据感知技术研究_张俊豪.pdf