doi:10.16186/j.cnki.1673-9787.2020040048

Received:2020/04/09

Revised:2020/06/09

Published:2021/07/15

Study on multi-source data perception technology based ondeep learning and machine vision

ZHANG Junhao, LUO Guofu, YANG Xingbo, LI Yawei, ZHANG Ningbo

College of Mechanical and Electrical Engineering， Zhangzhou University of Light Industry， Zhangzhou 450002 ， Henan， China

Abstract:The position errors of the manipulator may easily cause the deviation of the running trajectory and affect the positioning effect of the manipulator. Therefore， a high-performance multi-source data sensing method was formed to collect high-precision manipulator position data by combining machine vision technology with deep learning algorithm. Firstly， based on the principle of visual imaging， the layout of the position data acquisition scheme of the manipulator with multi-camera was designed to obtain the relationship between the visual image coordinates and the posture of the manipulator. Next， by using the convolutional neural network as the core， a deep learning model with 5 convolutional layers ，4 maximum pooling layers and 3 fully connected layers was constructed to fuse the multi-source image data of the manipulator collected by multi-camera. Then， by using the method of batch gradient descent， the convolution kernel and offset parameter bof the model were optimized to characterize the image features in depth. Finally， the accurate operation position data were obtained by combining with the position model of the manipulator. After testing and verification， the maximum error values of the studied method for sensing the pitch angle，yaw angle and roll angle of the manipulator were all less than 0.6°. The data perception degree was high， and it could provide an accurate data basis for planning the working route of the manipulator and for accurately controlling the behavior of the manipulator.

Key words:deep learning;machine vision;manipulator;multi-source data;position;perception

基于深度学习和机器视觉的多源数据感知技术研究_张俊豪.pdf