PresentationPDF Available

Efficient Representation of Layered Depth Images for Real-time Volumetric Tests

Authors:

Abstract

Presentation of Research Paper "Efficient Representation of Layered Depth Images for Real-time Volumetric Tests"
1
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Matthias Trapp, Jürgen Döllner
Hasso-Plattner-Institute,
University of Potsdam, Germany
9th June 2008
University of Manchester
Efficient Representation
of Layered Depth Images
for Real-time Volumetric Tests
2
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Motivation: Volumetric Tests
…determine if a 3D point is
inside or outside a given volume
Areas of application:
Generalized clipping,
Rendering with hybrid styles,
3D magic lenses,…
Characteristics:
Volume representation = LDI
Test performed in shader program
Applicable in real-time
3
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
LDI Representation of Solid Meshes
Depth Layers LDI = (LDI0,,LDI7)
3D LDI Texture Space [0,1]3
Non-Convex Polygonal Mesh S with d = 7
3D World Space IR3
s
t
x
y
Depth-
Peeling
z
O O
4
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Depth-Peeling to 3D Textures
Algorithm for LDI creation:
1. Setup orthographic projection, adjust near/far planes
2. Determine depth complexity of shape
3. Create and initialize 3D texture (LDI)
4. Depth-peel shape:
Render-to-texture (into slice of 3D texture)
Use linear depth buffer values
uniform sampler3D LDI;
uniform int pass;
varying float linearDepth;
void main(void)
{
if((pass > 0) && (linearDepth <= texelFetch3D(LDI, ivec3(gl_FragCoord.xy, pass-1),0).x))
{
discard;
}
gl_FragDepth = linearDepth;
}
GLSL fragment shader for 2nd depth test (SM4)
5
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Volumetric Parity Test (VPT) Algorithm
“to determine if a point is inside or outside
a complex 3D volume represented by an LDI”
Given: arbitrary 3D point:
Requested: Boolean parity:
Solution:
1. Transformation into LDI texture space:
2. Perform ray-marching through depth maps
6
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Ray-Marching through LDI Texture Space
Algorithm:
Construct ray R:
Sample form each slice:
Compare depth values:
r
t
Tr
(s,1,0)
(s,0,1)
Ray R
Inside
Outside
d0d1d2d3
(s,0,0)
(Ts ,Tt ,0) (Ts ,Tt ,1)
pT =0 pT =1 pT =0 pT =1
S
(s,1,1)
7
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Shader Implementation (GLSL)
bool volumetricParityTestSM4(
in vec3 T, // Point in LDI texture-space
in sampler3D LDI, // layered depth image
in int depth, // LDI slices
in bool initParity) // initial parity
{
// initial parity; true = outside
bool parity = initParity;
// for each texture layer do
for(int i = 0; i < depth; i++)
{ // perform depth test
if(T.r <= texelFetch3D(LDI, ivec3(T.st, i), 0).x)
{
parity = !parity; // swap parity
}//endif
}//endfor
return parity;
}
8
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Limitations of Current Representation
Memory Consumptions:
3D texture size:
resolution2* d
Usually sparsely utilized
Depth-Peeling Performance:
Depends on depth complexity of shape
Ray-Marching Performance:
Depends on number of texture samples
So far:
d
samples per ray necessary
Goals:
Minimal texture size: Cropping, Compression
Minimal peeling passes: Viewpoint Selection
Minimal texture samples per test: Compression
R
d0d1
d2d3
9
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Outline Next Topics
1. Preprocessing Pipeline: Overview
2. Viewpoint Selection
3. Bounding Box Calculation & Cropping
4. LDI Compression Algorithm
5. Performance Results
6. Limitations & Future Work
7. Conclusions
10
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Preprocessing Pipeline: Overview
CPU
GPU
Depth-Peeling
To 3D Texture 3D Texture (LDI)
Shape S
2D Texture (LDIComp)
Cropping
3D Texture (LDICrop)
Compression
AABB
Calculation
Viewpoint
Selection
Orientation ODP
Data Flow
Control Flow
Preprocessing
AABBLDI
5 Step process involves GPU and CPU:
1. Viewpoint selection (ODP )
2. Depth-peeling to 3D texture (LDI)
3. Calculate axis-aligned bounding box (AABBLDI )
4. Perform cropping (LDICrop )
5. Compress cropped texture (LDIComp )
11
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Viewpoint Selection
Find viewpoint for LDI creation based on:
Minimal depth complexity: d
Maximal texture coverage: o
Determine camera orientation
Altering viewpoint on bounding sphere
Determine and store in list :
Depth complexity
Altered camera orientation
Texture coverage
Sort list to obtain result
procedure orientation(S,OC,sH,sV)
{
// for each segment on sphere
for(h
¬
0; h < sH; h
¬
h+1)
for(v
¬
0; v < sV; v
¬
v+1)
OC
¬
adjustOrientation(OC, sH, sV)
// calculate metric
d
¬
depthComplexity(S,OC)
o
¬
coverage(S,OC)
// store result
append(list,(OC,d,o))
endfor
endfor
// sort results
sortDepthComplexityAscending(list)
sortCoverageDecending(list)
ODP
¬
getOrientation(list, 0)
}
12
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Bounding Box Calculation & Cropping
δ
γ
β
α
s
t
(0,0,0) (w,0,0)
(w,h,0)
(0,h,0)
hShE
vE
vS
AABB
α
δ
γ
β
procedure calculateAABB(VDS)
hS ¬ 0; vS ¬ 0; hE ¬ w; vE ¬ h;
// determine horizontal start hS
for(s ¬ 0; s < w; s
¬
s+1)
if intersectV(0,h,s,LDI0)
hS ¬ s; break;
endif endfor
// determine vertical start vS
for(t ¬ 0; t < h; t
¬
t+1)
if intersectH(hS,w,t,LDI0)
vS ¬ t; break;
endif endfor
// determine horizontal end hE
for(s
¬
w; s > hS; s
¬
s-1)
if intersectV(vS,h,s,LDI0)
hE
¬
s; break;
endif endfor
// determine vertical end vE
for(t
¬
h; t > vS; t
¬
t-1)
if intersectH(hS,hE,t,LDI0)
vE
¬
t; break;
endif endfor
// deliver result
AABB
¬
(hS, vS, hE - hS, vE - vS)
end
13
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Compressing Depth Ranges
Depth Ranges:
Group depth values: di= 0,…,d-1
DRj= (Sj,Ej ), j = 0,…,d/2
Interval [Sj, Ej ], Sj= d2j , Ej= d 2j+1 denotes inside of volume
Lossless Compression Idea:
Sequential align all depth ranges of a Ray R
Fetch start (Sj ) and end (Sj )with single instruction
3D Texture (LDICrop)
S0E0S1E1S2E2
S5E5S6E6
S3E3S4E4
NIL NIL
NIL NIL
NIL NIL NIL NIL NIL NIL
NIL NIL NIL NIL NIL NIL
r
LDI0LDI1LDI2LDI3LDI4LDI5
tUnused texel
Depth value
DR0
DR1
14
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Compression Algorithm Overview
Two-phase process:
1. Extract depth ranges from 3D texture into intermediate representation
2. Pack depth ranges from intermediate representation into 2D texture
O0S0S1S2S3S4S5S6
NIL O1O2NIL ISoffset RSstart
R0E0E1E2E3E4E5E6
NIL R1R2NIL ISranges
R0 = 3 R1 = 2 R2 = 2
RSend
Index Segment IS Range Segment RS
Offset = wb ∙ hb
32bit
Intermediate Representation
2D Texture (LDIComp)
RSend
ISranges
wC
RSstart
ISoffset
3D Texture (LDICrop)
S0E0S1E1S2E2
S5E5S6E6
S3E3S4E4
NIL NIL
NIL NIL
NIL NIL NIL NIL NIL NIL
NIL NIL NIL NIL NIL NIL
r
LDI0LDI1LDI2LDI3LDI4LDI5
tUnused texel
Depth value
Pack Alpha
Pack Luminance
Extract
15
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Compression - Extract Phase
Create intermediate representation:
Prepare data structure for pack phase
Read-back LDI from GPU
Create 4 arrays from LDI:
Index Segments (ISoffset, ISranges )
Range Segments (RSstart , RSend)
procedure extractLDI(LDI)
vector ISoffset,ISranges,RSstart,RSend;
for(s
¬
0; s < w; s
¬
s + 1)
for(t
¬
0; t < h; t
¬
t + 1)
// check if texel is set
if(LDI[s,t,0] NIL)
layers
¬
0
// iterate over depth
for(r
¬
0; r < d; r
¬
r + 1)
depthValue
¬
LDI[s,t,r]
if(depthValue NIL)
if(r & 2 = 1)
append(RSstart , depthValue)
else
append(RSend , depthValue)
endif
else break
endfor
ISoffset[t ∙ w + s] ¬ || RSstart ||
ISranges[t ∙ w + s] ¬ layers / 2
endif endfor endfor
end
O0S0S1S2S3S4S5S6
NIL O1O2NIL ISoffset RSstart
R0E0E1E2E3E4E5E6
NIL R1R2NIL ISranges
R0 = 3 R1 = 2 R2 = 2
RSend
Index Segment IS Range Segment RS
Offset = wb ∙ hb
32bit
Intermediate Representation
16
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Compression - Pack Phase
Pack arrays into 2D texture
Texture format: Luminance-Alpha
Float point precision: unclamped values
Texture resolution:
Store successively:
ISoffset , RSstart in luminance channel
ISranges , RSend in alpha channel
Issues:
2D texture format introduces unwrapping step in decompression
Better: 1D texture, but resolution limited to 8192 pixel
Possible Solution: Uniform Buffer (NVIDIA G80) + Geometry Shader
 
RSIShw cc +==
2D Texture (LDIComp)
RSend
ISranges
wC
RSstart
ISoffset
17
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
VPT for Compressed LDI
bool testAABB2D(const in vec2 v, const in vec2 b1, const in vec2 b2){
return (v.x>=b1.x)&&(v.y>=b1.y)&&(v.x<=b2.x)&&(v.y<=b2.y);}
bool volumetricParityTestCompressedSM4(
const in vec4 T, // point in LDI texture space
const in sampler2D compressedLDI, // compressed LDI
const in vec4 bounds, // crop bounds
const in uvec2 size, // resolution of uncompressed LDI
const in bool initParity) // initial parity
{ // initial parity; true = outside
bool parity = initParity;
// compensate cropping
if(testAABB2D(T.xy, bounds.xy, bounds.xy + bounds.zw)){
//map cropped cordinates to [0;1]
T.xy = (T.xy - bounds.xy) / bounds.zw;
// retrieve compressed texture size (CTS)
uvec2 CTS = uvec2(textureSize2D(compressedLDI, 0));
// uncompressed vds coordinates (UVC)
uvec2 UVC = uvec2(T.st * vec2(size));
// convert to linearized UVC (LUVC)
unsigned int LUVC = UVC.y * size.x + UVC.x;
// coordinate of the index segment (ISC)
ivec2 ISC = ivec2(LUVC % CTS.x, LUVC / CTS.x);
// sample from index segment IS
uvec2 IS = uvec2(texelFetch2D(compressedLDI, ISC, 0).ra);
if(IS.x != 0u) { // depth ranges available ?
for(unsigned int i = 0u; i < IS.y; i++) {
// calculate range sample coordinate (RSC)
ivec2 RSC = ivec2((IS.x + i) % CTS.x, (IS.x + i) / CTS.x);
// sample range depth range DR
vec2 DR = texelFetch2D(compressedLDI, RSC, 0).ra;
if(T.z <= DR.x && T.z >= DR.y){ // perform parity test
parity = !parity;
break;
} } } }
return parity;}
3D world space
V
RS
IS
2D Texture LDIComp
OV
RV
Offset into RS
#Depth Ranges
Depth Range Start
Depth Range End
RSC
S1SR
ER
...
...
S1
E1E2
ISC
18
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Performance Results
Sphere Complex Potato Cow Hebe Knot
#Vertex 994 768 6,146 2,903 34,344 23,232
dmin 2 2 6 8 10 6
dmax 2 6 12 16 18 12
tcrop 0.077 0.078 0.203 0.234 0.313 0.2
tcomp 0.437 0.297 0.219 0.09 0.078 0.125
tpeel 0.187 0.187 25.766 2.156 73.078 25.875
tview 2.532 3.085 7.172 8.469 12.187 9.265
Mcrop 2,097,152 2,097,152 5,683,200 3,026,688 3,358,720 3,143,880
Mcomp 3,742,848 3,011,058 3,317,888 1,204,352 1,089,288 1,835,528
Cration 1.78 1.44 0.58 0.39 0.32 0.58
19
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Limitations & Future Work
Limitations:
Specific approach for depth ranges
Too much instruction for decompression
Small compression ratio 1:2-3
GPU & CPU (hybrid) approach
Future Work:
Toward a complete GPU implementation:
(Bounding box calculation, cropping, compression)
Main Goal: enable preprocessing in real-time
New filtering concepts necessary
to compensate undersampling / alaising artefacts
Undersampling
Alaising
20
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Conclusions
Efficient LDI Representation trough:
Automatic viewpoint selection
Fast bounding box calculation & cropping
Lossless LDI compression
Decompression algorithm applicable in shader
Compression Characteristics:
Ratio 1:2-3 (common for lossless compression)
Effective for non-convex meshes with high depth complexity
Compression should be avoided for symmetric convex meshes
Preprocessing, no real-time performance
Future Work:
Implementation fully GPU-based
21
TP.CG 2008 :: 06/09/2008 Matthias Trapp :: Efficient Representation of LDIs for Volumetric Tests
Questions ?
Contact:
Matthias Trapp
matthias.trapp@hpi.uni-potsdam.de
Computer Graphics Systems Group
Prof. Dr. Jürgen Döllner
www.hpi.uni-potsdam.de/3d
Researchgroup 3D-Geoinformation
www.3dgi.de
... However, their proposed approaches are tailored to texture-based volume rendering, which produces lower quality images compared to raycasting. Layered depth images (LDI) have been applied for volumetric tests for polygonal objects [33], but creation of the LDI requires a preprocessing step and may be prone to aliasing artifacts if the LDI does not have high enough resolution. Moreover, this approach requires a separate volume texture to store the LDI, which may grow rapidly when resolution is increased to enable high-quality selections. ...
Article
Volumetric datasets are increasingly used in medical applications. In many of these applications, visualization and interaction is generally performed on cross-sectional two-dimensional (2D) views of three-dimensional (3D) imaging modalities. Displaying 3D volumetric medical datasets on traditional 2D screens can present problems such as occlusion and information overload, especially when multiple data sources are present. Displaying desired information while showing the relationship to the rest of the dataset(s) can be challenging. In this paper, we present an interactive focus + context visualization approach that uses the volumetric Magic Lens interaction paradigm. We propose to use the Magic Lens as a volumetric brush to perform volume editing tasks, therefore combining data exploration with volumetric editing. Polygon-assisted ray casting methods are used for real-time rendering and editing frame rates, while providing compact storage of editing states for undo/redo operations. We discuss the application of our methods to radiation therapy, which is an important cancer treatment modality. We envision that this approach will improve the treatment planning process by improving the therapists' understanding of information from various sources and will help identify if the alignment of the patient in the treatment room coincides with the prepared treatment plan. Copyright © 2013 John Wiley & Sons, Ltd.
ResearchGate has not been able to resolve any references for this publication.