Overview of our omnidirectional stereo (ODS) to multi‐sphere image (MSI) translation network: The input consists of a left and right eye video frame of an ODS recording I˚$\mathring{I}$, which are first encoded using a ResNet50 [HZRS16] encoder ΨE, see Section 3.7.1. The sum of these deep image features of both views and the disparity encoding of a single spherical layer's radius r as di=1/ri$d_i = 1/r_i$ is fed into our decoder ΨD to generate the corresponding multi‐sphere image (MSI) layer Lr$\mathbf {L}_r$, as described in Section 3.7.2. Utilizing several MSI layers, we can render input ODS frames I˚Ψ$\mathring{I}_\Psi$ or novel viewpoints utilizing ray‐tracing for volume rendering (see Equation 1).