Adaptive CU Split Decision Based on Deep Learning and Multifeature Fusion for H.266VVC

Abstract EN

With the development of technology, the hardware requirement and expectations of user for visual enjoyment are getting higher and higher.

The multitype tree (MTT) architecture is proposed by the Joint Video Experts Team (JVET).

Therefore, it is necessary to determine not only coding unit (CU) depth but also its split mode in the H.266/Versatile Video Coding (H.266/VVC).

Although H.266/VVC achieves significant coding performance on the basis of H.265/High Efficiency Video Coding (H.265/HEVC), it causes significantly coding complexity and increases coding time, where the most time-consuming part is traversal calculation rate-distortion (RD) of CU.

To solve these problems, this paper proposes an adaptive CU split decision method based on deep learning and multifeature fusion.

Firstly, we develop a texture classification model based on threshold to recognize complex and homogeneous CU.

Secondly, if the complex CUs belong to edge CU, a Convolutional Neural Network (CNN) structure based on multifeature fusion is utilized to classify CU.

Otherwise, an adaptive CNN structure is used to classify CUs.

Finally, the division of CU is determined by the trained network and the parameters of CU.

When the complex CUs are split, the above two CNN schemes can successfully process the training samples and terminate the rate-distortion optimization (RDO) calculation for some CUs.

The experimental results indicate that the proposed method reduces the computational complexity and saves 39.39% encoding time, thereby achieving fast encoding in H.266/VVC.