彭维平, 杨玉莹, 王戈,等.VEC中基于DRL的“端-多边-云”协作计算卸载算法[J].河南理工大学学报（自然科学版）,2024,43(6):156-163.

PENG W P, YANG Y Y, WANG G,et al.An “end-multi-edge-cloud” cooperative computation offloading algorithm based on DRL in VEC[J].Journal of Henan Polytechnic University(Natural Science) ,2024,43(6):156-163.

VEC中基于DRL的“端-多边-云”协作计算卸载算法

彭维平, 杨玉莹, 王戈, 宋成, 阎俊豪

河南理工大学 计算机科学与技术学院，河南 焦作 454000

摘要: 目的  为了解决车载边缘计算中用户服务质量低以及边缘节点资源不足的问题，  方法 结合车载边缘计算和停车边缘计算技术，提出“端-多边-云”协作计算卸载模型，并设计基于DRL的协作计算卸载与资源分配算法（cooperative computation offloading and resource allocation algorithm based on DRL，DRL-CCORA）。首先，将路边停放车辆的算力构建成停车边缘服务器（parking edge server，PES），联合边缘节点为车辆任务提供计算服务，减轻边缘节点的负载；其次，将计算卸载与资源分配问题转化为马尔可夫决策过程模型，综合时延、能耗和服务质量构建奖励函数，并根据任务需要的计算资源、任务的最大容忍时延以及车辆到PES的距离对计算任务进行预分类处理，缩减问题的规模；最后，利用双深度Q网络（double deep q network，DDQN）算法获得计算卸载和资源分配的最优策略。  结果  结果表明，相较于对比算法，所提算法的用户总服务质量提高了6.25%，任务的完成率提高了10.26%，任务计算的时延和能耗分别降低了18.8%、5.26%。  结论  所提算法优化了边缘节点的负载，降低了任务完成的时延和能耗，提高了用户的服务质量。

关键词:车载边缘计算;停车边缘计算;计算卸载;资源分配;双深度Q网络

doi:10.16186/j.cnki.1673-9787.2022110060

基金项目:国家重点研发计划项目（2018YFC0604502）；国家自然科学基金资助项目（61872126）；河南省高校青年骨干教师计划项目（2019GGJS061）

收稿日期:2022/11/28

修回日期:2023/05/14

出版日期:2024-09-24

An “end-multi-edge-cloud” cooperative computation offloading algorithm based on DRL in VEC

PENG Weiping, YANG Yuying, WANG Ge, SONG Cheng, YAN Junhao

School of Computer Science and Technology，Henan Polytechnic University，Jiaozuo 454000，Henan，China

Abstract: Objectives To address the problems of low service quality of users and insufficient resources of edge node in vehicular edge computing（VEC），  Methods  combined vehicular edge computing with parking edge computing technology，an "end-multi-edge-cloud" cooperative computation offloading model was proposed，and a cooperative computation offloading and resource allocation algorithm based on DRL（DRL-CCORA） was designed.Firstly，the computing power of roadside parking vehicles were constructed into parking edge server（PES），which jointly provided computing services for vehicle tasks with edge node，and reduced the load of edge node.Secondly，the problem of computation offloading and resource allocation was transformed into a Markov Decision Process model，and a reward function was constructed based on time delay，energy consumption and service quality.And the computing tasks were pre-classified according to the computing resources required by the task and the maximum allowable delay of the task and the distance from the vehicle to PES，the scale of the problem was reduced.Finally，the double deep Q network（DDQN） algorithm was used to obtain the optimal strategy of computation offloading and resource allocation.  Results The results showed that，compared to the contrasting algorithm，the proposed algorithm improved the overall user service quality by more than 6.25%，improved the task completion rate by more than 10.26%，and reduced the time delay and energy consumption of task computing by more than 18.8% and 5.26%，respectively.  Conclusions The proposed algorithm optimized the load of the edge node，reduced the time delay and energy consumption of task completion，and improved the service quality of users.

Key words:vehicular edge computing;parking edge computing;computation offloading;resource allocation;double deep Q network