7.6  Experimental results


To evaluate the performance of the search algorithm for the quantizer settings, experiments are carried out, using the “Ballet” and “Breakdancers” sequences. The impact of using a variable number of reference views is measured by using two different, so called, rendering structures. As portrayed by Figure 7.4 (a) and (b), the investigated rendering structures include one and two reference views, respectively.
PIC
(a)
PIC
(b)

Figure 7.4 Rendering structures based on (a) one reference view and (b) two reference views.


To generate the R-D surfaces for both sequences, we set qtmin = qdmin = 27 and qtmax = qdmax = 51, so that 25 × 25 R-D points are obtained. For coding experiments, we have employed the open-source H.264/MPEG-4 AVC encoder x264 [4] and we have encoded the first depth and texture frames of the reference view(s) in intra-mode. The presented experiments attempt to quantify the rendering quality obtained in three ways by using:

The pre-defined depth bit rate of 10% is based on experimental results published in the framework of the ATTEST project jointly carried out by Op De Beeck et al. and Smolic et al. [1893]. First, considering the experimental setup that employs one reference view for rendering, it can be observed in Figure 7.5 that the proposed joint bit-allocation framework consistently outperforms the pre-defined depth bit-rate coding scheme. For example, when inspecting Figure 7.5(a) and Figure 7.5(b), it can be seen that the joint bit-allocation framework yields a quality improvement of 0.8 dB and 1.0 dB at a bit rate 75 kbit per frame, respectively. Additionally, employing the sub-optimal search does not sacrifice the rendering performance compared to full search. Next, let us consider the second experimental setup that uses two reference views for rendering (see Figure 7.6). It can be first noted that the proposed bit-allocation framework also consistently outperforms the pre-defined depth bit-rate coding scheme. Specifically, a rendering-quality improvement of up to 0.9 dB and 0.4 dB is obtained for the sequences “Breakdancers” and “Ballet”, respectively. Note that the depth of the “Ballet” sequence is encoded at 20% of the texture bit rate because a ratio of 10% is not sufficient and even provides annoying results in terms of noise. Again, as previously observed, a fast hierarchical search can be employed without loss of rendering quality. Thus, the sub-optimal hierarchical search provides a fast and accurate estimation of the optimal R-D point of operation.

Additionally, as expected, the rendering quality does not only depend on the sequence characteristics, but also on the rendering structure. For example, a rendering quality of 31.2 dB and 32.5 dB can be obtained at a total bit rate of 150 kbit per frame for the depth and texture images when using one and two reference views, respectively. This result simply highlights that the rendering quality, and thus, the bit-rate distribution is not only influenced by the texture and depth characteristics of the sequence, but also by the number of reference views employed for rendering. Finally, it can be noted that the R-D curve, which is obtained by optimizing the quantization parameters, presents a smooth logarithmically increasing rendering quality. In contrast, not optimizing the quantization parameters yields an R-D curve with non-smooth and even erratic rendering quality.


PIC
(a)
PIC
(b)

Figure 7.5 (a) and (b) Obtained rendering quality for the sequences “Breakdancers” and “Ballet”, respectively, when performing joint bit allocation using one reference view.



PIC
(a)
PIC
(b)

Figure 7.6 (a) and (b) Obtained rendering quality for the sequences “Breakdancers” and “Ballet”, respectively, when performing joint bit allocation using two references views.


Finally, Table 7.1 summarizes the measured rendering distortions for the “Breakdancers” and “Ballet” multi-view sequences and the two rendering structures depicted in Figure 7.4. As expected, it can be noted that the ratio between the depth and texture bit rate depends not only on the multi-view sequence, but also on the number of reference views employed for rendering. Specifically, let us select four R-D points corresponding to the maximal rendering quality in the Tables denoted (a) (b) (c) and (d) (see Table 7.1). These four points (underlined in the table) are selected such that the sum of the texture and depth bit rates is less or equal to 100 kbit/frame, i.e., Rmax ≤ 100. For the “Breakdancers” sequence, it can be seen that the depth bit rate corresponds to 40% and 29% of the total bit rate when using one and two reference views, respectively. Next, for the “Ballet” sequence, a bit-rate ratio of 50% and 38% is measured, when using one and two reference views, respectively. Therefore, significant variations in bit-rate distribution are observed for the two multi-view sequences. The obtained bit-rate ratios can be explained by examining the properties of the multi-view sequences. Specifically, the “Ballet” sequence shows richer 3D content when compared to the “Breakdancers” sequence. As a result, rendering high-quality virtual images of the “Ballet” sequence requires a higher depth bit rate when compared to the “Ballet” sequence.


(a) “Breakdancers” sequence.







Depth bit rate (kbit/frame)
62.937.722.613.3 7.8














207.831.931.731.531.130.4
Texture 89.4 31.831.731.531.130.4
bit rate 50.8 31.731.531.431.030.3
(kbit/frame) 30.8 31.331.231.130.830.2
18.0 30.630.530.530.329.8







(b) “Ballet” sequence.







Depth bit rate (kbit/frame)
69.147.030.219.212.1














140.326.626.626.526.425.9
Texture 75.6 26.626.526.526.425.9
bit rate 46.5 26.526.526.426.325.8
(kbit/frame) 27.6 26.426.326.326.225.8
15.8 25.925.925.925.825.5







(c) “Breakdancers” sequence.







Depth bit rate (kbit/frame)
125.376.146.325.915.9














450.4 33.0 32.932.732.331.7
Texture 198.5 33.0 32.932.732.431.7
bit rate 110.8 32.9 32.732.632.231.6
(kbit/frame) 66.6 32.5 32.432.332.031.4
39.2 31.7 31.631.531.330.9







(d) “Ballet” sequence.







Depth bit rate (kbit/frame)
142.794.462.938.824.1














299.0 27.4 27.427.327.227.1
Texture 160.1 27.4 27.427.327.327.1
bit rate 96.5 27.4 27.427.427.327.1
(kbit/frame) 57.6 27.3 27.327.327.227.1
33.5 27.0 27.027.027.027.0








Table 7.1 (a) and (b) show the rendering quality (PSNR in dB) using one reference view. (c) and (d) show the rendering quality (PSNR in dB) using two reference views. The underlined points corresponds to the maximal rendering quality for Rmax ≤ 100 kbit∕frame.