[1]唐晓龙,黄惠.面向未知环境的理鞋机器人系统设计与实现[J].深圳大学学报理工版,2022,39(4):472-479.[doi:10.3724/SP.J.1249.2022.04472]
 TANG Xiaolong and HUANG Hui.Design and implementation of shoes arrangement robot system for unknown environment[J].Journal of Shenzhen University Science and Engineering,2022,39(4):472-479.[doi:10.3724/SP.J.1249.2022.04472]
点击复制

面向未知环境的理鞋机器人系统设计与实现()
分享到:

《深圳大学学报理工版》[ISSN:1000-2618/CN:44-1401/N]

卷:
第39卷
期数:
2022年第4期
页码:
472-479
栏目:
电子与信息科学
出版日期:
2022-07-12

文章信息/Info

Title:
Design and implementation of shoes arrangement robot system for unknown environment
文章编号:
202204015
作者:
唐晓龙黄惠
深圳大学计算机与软件学院,广东深圳 518060
Author(s):
TANG Xiaolong and HUANG Hui
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, Guangdong Province, P. R. China
关键词:
人工智能智能感知目标检测目标匹配位姿估计机器人
Keywords:
artificial intelligence intelligent sensing object detection object matching pose estimation robot
分类号:
TP391;TP242
DOI:
10.3724/SP.J.1249.2022.04472
文献标志码:
A
摘要:
近年来机器人和人工智能技术发展迅速,但实现机器人在家居环境中的接触式操作仍然是一个具有挑战性的问题.为实现机器人自主整理鞋子的功能,提出利用实例分割网络和最小外接矩形识别鞋子和其朝向的方法,借助深度相机的点云信息,估计机器人的抓取位姿和放置位姿,并利用卷积神经网络和余弦相似度对同一双鞋子进行配对,设计了一套基于三维视觉的机器人自主理鞋系统.对系统中鞋子朝向识别的准确率和鞋子匹配的准确率进行评估,并进行真实机器人的自主整理测试.结果显示,所提方法对鞋子朝向识别的准确率达到96.2%,加入VGG16网络后,鞋子匹配算法的匹配准确率从62.6%提升到87.4%.该方法能准确实现机器人对鞋子及其朝向的识别进而对鞋子进行匹配,提升了机器人鞋子整理的稳定性.
Abstract:
In recent years, the robot and artificial intelligence technology have developed rapidly, but there still remains a challenging problem to realize the contact operation of robot in home environment. In order to make the robot arrange shoes automatically, we design an autonomous shoes arrangement robot system based on the 3D vision. In this system, the instance segmentation network and minimum enclosing rectangle are used to recognize the shoes and their orientation, and the grasping pose and placing pose of robot are estimated accurately by means of the point cloud information of depth camera. In addition, the convolution neural network and cosine similarity are adopted to match with a pair of shoes. Afterwards, we evaluate the accuracy of shoe orientation recognition and shoe matching in the system, and then carry out a real robot arrangement experiment. The result show that this method could assure 96.2% accuracy of shoe orientation recognition, and the matching accuracy of shoes increases from 62.6% to 87.4% when the VGG16 network is added to the shoes matching algorithm. In conclusion, this method can accurately recognize the shoes and their orientation, and then match with a pair of shoes, meanwhile, improves the stability of the robot shoes arrangement.

参考文献/References:

[1] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA: IEEE, 2005: 886-893.
[2] LOWE D G. Object recognition from local scale-invariant features [C]// Proceedings of the 7th IEEE International Conference on Computer Vision. Kerkyra, Greece: IEEE, 1999: 1150-1157.
[3] LIENHART R, MAYDT J. An extended set of haar-like features for rapid object detection [C]// Proceedings of International Conference on Image Processing. New York, USA: IEEE, 2002: I-900-I-903.
[4] FREUND Y, SCHAPIRE R E. Experiments with a new boosting algorithm [C]// Proceedings of the 13th International Conference on Machine Learning. Bari, Italy: ACM, 1996: 148-156.
[5] LIAW A, WIENER M. Classification and regression by random forest [J]. R News, 2002, 2/3:18-22.
[6] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF) [J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359.
[7] GALVEZ-L?PEZ D, TARDOS J D. Bags of binary words for fast place recognition in image sequences [J]. IEEE Transactions on Robotics, 2012, 28(5): 1188-1197.
[8] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [C]// Conference and Workshop on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2012: 1097-1105.
[9] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 779-788.
[10] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// European Conference on Computer Vision. Cham, Switzerland: Springer, 2016: 21-37.
[11] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 580-587.
[12] GIRSHICK R. Fast R-CNN [C]// Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chilepp: IEEE, 2015: 1440-1448.
[13] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6): 1137-1149.
[14] HE Kaiming, GKIOXARI G, DOLL?R P, et al. Mask R-CNN [C]// Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2961-2969.
[15] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015: 3431-3440.
[16] MAHLER J, MATL M, LIU X, et al. Dex-net 3.0: computing robust vacuum suction grasp targets in point clouds using a new analytic model and deep learning [C]// IEEE International Conference on Robotics and Automation. Brisbane, Australia: IEEE, 2018: 5620-5627.
[17] JIANG Y, MOSESON S, SAXENA A. Efficient grasping from RGBD images: Learning using a new rectangle representation [C]// IEEE International Conference on Robotics and Automation. Shanghai, China: IEEE, 2011: 3304-3331.
[18] LENZ I, LEE H, SAXENA A, et al. Deep learning for detecting robotic grasps [J]. International Journal of Robotics Research, 2015: 34(3/4): 705-724.
[19] REDMON J, ANGELOVA A. Real-time grasp detection using convolutional neural networks [C]// IEEE International Conference on Robotics and Automation. Seattle, USA: IEEE, 2015: 1316-1322.
[20] LIN T Y, DOLL?R P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, IEEE, 2017: 2117-2125.

相似文献/References:

[1]潘长城,徐晨,李国.解全局优化问题的差分进化策略[J].深圳大学学报理工版,2008,25(2):211.
 PAN Chang-cheng,XU Chen,and LI Guo.Differential evolutionary strategies for global optimization[J].Journal of Shenzhen University Science and Engineering,2008,25(4):211.
[2]骆剑平,李霞.求解TSP的改进混合蛙跳算法[J].深圳大学学报理工版,2010,27(2):173.
 LUO Jian-ping and LI Xia.Improved shuffled frog leaping algorithm for solving TSP[J].Journal of Shenzhen University Science and Engineering,2010,27(4):173.
[3]蔡良伟,李霞.基于混合蛙跳算法的作业车间调度优化[J].深圳大学学报理工版,2010,27(4):391.
 CAI Liang-wei and LI Xia.Optimization of job shop scheduling based on shuffled frog leaping algorithm[J].Journal of Shenzhen University Science and Engineering,2010,27(4):391.
[4]张重毅,刘彦斌,于繁华,等.CDA市场环境模型进化研究[J].深圳大学学报理工版,2010,27(4):413.
 ZHANG Zhong-yi,LIU Yan-bin,YU Fan-hua,et al.Research on the evolution model of CDA market environment[J].Journal of Shenzhen University Science and Engineering,2010,27(4):413.
[5]姜建国,周佳薇,郑迎春,等.一种双菌群细菌觅食优化算法[J].深圳大学学报理工版,2014,31(1):43.[doi:10.3724/SP.J.1249.2014.01043]
 Jiang Jianguo,Zhou Jiawei,Zheng Yingchun,et al.A double flora bacteria foraging optimization algorithm[J].Journal of Shenzhen University Science and Engineering,2014,31(4):43.[doi:10.3724/SP.J.1249.2014.01043]
[6]蔡良伟,刘思麒,李霞,等.基于蚁群优化的正则表达式分组算法[J].深圳大学学报理工版,2014,31(3):279.[doi:10.3724/SP.J.1249.2014.03279]
 Cai Liangwei,Liu Siqi,Li Xia,et al.Regular expression grouping algorithm based on ant colony optimization[J].Journal of Shenzhen University Science and Engineering,2014,31(4):279.[doi:10.3724/SP.J.1249.2014.03279]
[7]宁剑平,王冰,李洪儒,等.递减步长果蝇优化算法及应用[J].深圳大学学报理工版,2014,31(4):367.[doi:10.3724/SP.J.1249.2014.04367]
 Ning Jianping,Wang Bing,Li Hongru,et al.Research on and application of diminishing step fruit fly optimization algorithm[J].Journal of Shenzhen University Science and Engineering,2014,31(4):367.[doi:10.3724/SP.J.1249.2014.04367]
[8]刘万峰,李霞.车辆路径问题的快速多邻域迭代局部搜索算法[J].深圳大学学报理工版,2015,32(2):196.[doi:10.3724/SP.J.1249.2015.02000]
 Liu Wanfeng,and Li Xia,A fast multi-neighborhood iterated local search algorithm for vehicle routing problem[J].Journal of Shenzhen University Science and Engineering,2015,32(4):196.[doi:10.3724/SP.J.1249.2015.02000]
[9]蔡良伟,程璐,李军,等.基于遗传算法的正则表达式规则分组优化[J].深圳大学学报理工版,2015,32(3):281.[doi:10.3724/SP.J.1249.2015.03281]
 Cai Liangwei,Cheng Lu,Li Jun,et al.Regular expression grouping optimization based on genetic algorithm[J].Journal of Shenzhen University Science and Engineering,2015,32(4):281.[doi:10.3724/SP.J.1249.2015.03281]
[10]王守觉,鲁华祥,陈向东,等.人工神经网络硬件化途径与神经计算机研究[J].深圳大学学报理工版,1997,14(1):8.
 Wang Shoujue,Lu Huaxiang,Chen Xiangdong and Zeng Yujuan.On the Hardware for Artificial Neural Networks and Neurocomputer[J].Journal of Shenzhen University Science and Engineering,1997,14(4):8.

备注/Memo

备注/Memo:
Received: 2021-03-10; Accepted: 2021-08-06; Online (CNKI): 2022-06-22
Foundation: Project of Department of Education of Guangdong Province (2018KZDXM058, 2020SFKC059)
Corresponding author: Professor HUANG Hui. E-mail: huihuang@szu.edu.cn
Citation: TANG Xiaolong, HUANG Hui. Design and implementation of shoes arrangement robot system for unknown environment [J]. Journal of Shenzhen University Science and Engineering, 2022, 39(4): 472-479.(in Chinese)
基金项目:广东省教育厅资助项目 (2018KZDXM058,2020SFKC059)
作者简介:唐晓龙(1994—),深圳大学硕士研究生.研究方向:三维视觉.E-mail: tangtxl80@gmail.com
引文:唐晓龙,黄惠.面向未知环境的理鞋机器人系统设计与实现[J].深圳大学学报理工版,2022,39(4):472-479.
更新日期/Last Update: 2022-07-30