PresentationPDF Available

Programmable Rendering Pipelines for Interactive Visualization

Presentation

Programmable Rendering Pipelines for Interactive Visualization

Abstract

Invited Talk at the Department of Computer Graphics, Institute for Computer Science, Rostock/Germany, 2014.
Programmable Rendering Pipelines
for Interactive Visualization
Dr. Matthias Trapp
Today...
Interactive rendering w. hardware-accelerated rasterization (GPUs)
In the context of visualization pipeline
Fixed-Function Rendering is Dead!
PROGRAMMBLE RENDERING PIPELINE
Shading Languages at a Glance
Shading Languages (SL)
Production Rendering
(Off-line Shading Languages) Real-time Rendering
(On-line Shading Languages)
Low Level Languages High Level Languages
GLSL
HLSL
Cg
Brooks
RenderMan
MetaSL
Houdini (VEX)
Gelato
API Depended SL API Independed SL
A Hardware-Accelerated Programmable Pipeline...
...can be quite complex
GPU :: Brief Overview
What is a Shader Program?
Shader Object:
Typed (Vertex, Control, Evaluation, Geometry, Fragment)
Modular, encapsulate source code
Shader Program = Executable Unit
Group shader objects of different types (see OpenGL 4.0 subroutines)
At least one shader object of each type must contain a main function
Shader Program
Vertex
Shader Object Vertex
Shader Object Geometry
Shader Object Fragment
Shader Object Fragment
Shader Object
What is a (GLSL) Shader Object?
// Application constants
uniform mat4 m4MVP;
// Stream Mapping
in vec4 v4VertexIn;
in vec3 v3NormalIn;
// Interpolant (Output)
out vec3 v3Normal;
void main(void) {
v3Normal = v3NormalIn;
gl_Position = m4MVP * v4Vertex;
}
// Application constants
uniform vec4 v4ColorIn;
// Interpolant
in vec3 v3Normal;
// Per-fragment output
out vec4 v4Color;
// Forward declaration
vec4 lighting(const in vec3 normal, const in vec4 color);
void main(void) {
v4Color = lighting(v3Normal, v4ColorIn);
}
uniform vec3 v3LightDirection;
vec4 lighting(const in vec3 normal, const in vec4 color)
{
return vec4(dot(normalize(normal), v3LightDirection) * color.rgb, color.a);
}
Programming Model
Shader
Input Register
Temporal
Register
Output Register
Textures
Constants
„GPUs are the only type of parallel processor that has ever seen
widespread success“
(Matt Pharr)
Programming Model
Interpolants
(varying)(in/out)
Per-Scene or Per-Primitive Constants
(uniform)
Vertex
Shader Geometry
Shader Fragment
Shader
Application Raster
Buffer
Vertex Attributes
(attribute) (in)
Connector
(varying)(in/out)Outputs
(out)
VERTEX STAGE
What can you do using vertex shader?
Operates on vertex data: Position, Normal, Color,…
Replaces parts of fixed-function pipeline:
Vertex/Normal transformation
Handling per-vertex lighting
Generation/transformation of texture coordinates
Things you can do:
Customized computation of/on vertex attributes
Applications :: Level-of-Abstraction for City Models
http://www.youtube.com/watch?v=tU5d6WuSglk
http://www.researchgate.net/publication/236892943_Interactive_Visualization_of_Generalized_Virtual_3D_City_Models_using_Level-of-Abstraction_Transitions
Applications :: Level-of-Abstraction for City Models
Applications :: Landmark Scaling
http://www.researchgate.net/publication/225091340_A_Concept_of_Effective_Landmark_Depiction_in_Geovirtual_3D_Environments_by_View-Dependent_Deformation
Applications :: Global Deformations
Global Deformations [Baar, 1984]
Affine transformation matrix is
computed per-vertex
Enables complex transformation
that cannot be described globally
Applications :: Multi-Perspective Views
Applications :: Multi-Perspective Views
http://www.youtube.com/watch?v=2bYDKbzocSg
Applications :: Multi-Perspective Views
Integration Nokia Gate5 GmbH CAVE
Cartography-oriented Visualization
http://www.researchgate.net/publication/236892940_An_Immersive_Visualization_System_for_Virtual_3D_City_Models
GEOMETRY STAGE
What can you do using geometry shader?
Computing at primitive level (including adjacent primitives)
Primitive transformation
Primitive conversion
Primitive amplification
Primitive Culling
Note: Geometry shader are not for tesselation!
Applications :: Primitive Conversion
Example: Rendering of Street Networks
Applications :: Primitive Conversion
Line Primitives Texturing Distance Field Polygons
Applications :: Primitive Conversion
Applications :: Primitive Conversion
http://de.slideshare.net/autopolit/rendering-of-complex-3d-treemaps-grapp-2013
http://www.researchgate.net/publication/236895537_Interactive_Rendering_of_Complex_3D-Treemaps
Applications :: 3D Iso-Contours of DEMs
Applications :: 3D Iso-Contours of DEMs
http://www.researchgate.net/publication/225091331_3D_Isocontours_--_Real-time_Generation_and_Visualization_of_3D_Stepped_Terrain_Models
Applications :: 3D Iso-Contours of DEMs
http://www.youtube.com/watch?v=7w3yRp3Xqp8
Application :: View-dependent Texture Atlas
Application :: View-dependent Texture Atlas
Preprocessing Parameterization &
Packaging
Texture Atlas
Processing
Global Record
Compositing
Render
Texture-Atlas
Texture-Atlas
Per-Frame
Final Image
Polygonal Scene Preprocessing
Application :: View-dependent Texture Atlas
http://www.youtube.com/watch?v=llLKU-Oa2iU
http://de.slideshare.net/autopolit/viewdependent-texture-atlases
http://www.researchgate.net/publication/225091329_Interactive_Rendering_to_Perspective_Texture-Atlases
Applications :: Visibility Culling
Visibility Culling: omit rasterization of primitives that are not visible
View-Frustum Culling
Backface Culling
Size and Drop-Culling
Occlusion Culling
FRAGMENT STAGE
Per-Fragment Operations
What can you do using fragment shader?
Per-fragment lighting using local models
(Multi-) Texturing
Alpha Mapping
(Procedural) Texture Synthesis
Clipping
Fixed Functionality
Per-Fragment
Toon Shading
Texturing and Color Mapping
Mask 1 Data 1 Mask 2 Data 2 Data 1 Data 2 Mask
http://de.slideshare.net/autopolit/dynamic-mapping-of-raster-data
http://www.researchgate.net/publication/221360249_Dynamic_Mapping_of_Raster-Data_for_3D_Geovirtual_Environments
Texturing and Color Mapping
Texture coordinates generation and blending
Texturing and Color Mapping
Application :: Clipping
http://de.slideshare.net/autopolit/25-d-clipsurfaces-trappdoellnerclean
http://www.researchgate.net/publication/250927714_2.5D_Clip-Surfaces_for_Technical_Visualization
Application :: Clipping
h
Application :: Clipping
http://www.youtube.com/watch?v=mBasfz37VoY
Render-to-Texture
Important concept: G-Buffers [Saito, SIGGRAPH 1991]
Image-based representation of geometric information
Rasterization target is a | are texture(s) and not the framebuffer
OpenGL: Framebuffer Objects
Render-to-Texture Variants
Applications :: Non-Planar Projections
Non-Planar Projection Surface
Applications :: Non-Planar Projects
t
s
(-1, -1) (1, 0)
(1, 1)
(0, -1)
O = (0, 0)
Fst = (s, t)
y
x
z
(1,1,-1)
(-1,-1,1)
O
S=(x,y,z)
http://de.slideshare.net/autopolit/nonplanar-projections-grapp-2008
http://www.researchgate.net/publication/227087614_Generalization_of_Single-Center_Projections_Using_Projection_Tile_Screens
http://de.slideshare.net/autopolit/stereoscopy-for-nonplanar-projections
http://www.researchgate.net/publication/225091334_Interactive_Stereo_Rendering_For_Non-Planar_Projections_of_3D_Virtual_Environments
Applications :: Non-Planar Projects
http://www.youtube.com/watch?v=Y6SBylq5SFA
http://www.youtube.com/watch?v=RevbRJD3pPE
Applicationss :: Non-Planar Projections
Heutiges Köln
Rekonstruktion
http://de.slideshare.net/autopolit/colonia3d
Applications :: Non-Planar Projects
http://www.youtube.com/watch?v=HoC_mmy51CE
http://www.youtube.com/watch?v=xAqMM6G3de0
Applications :: Non-Planar Projections
T i1
T i4
T i2
T i3
Applications :: Non-Planar Projections
Advanced Image-based Scene-Representations
Distance Fields, A-Buffer and K-Buffer, Layered Depth-Images
Schichten mit Tiefenwerten LDI = (LDI0,,LDI7)
3D LDI Texturkoordinatensystem [0,1]3
Polygonales 3D-Objekt
3D Weltkoordinatensystem IR3
s
t
x
y
Depth-
Peeling
z
O O
0
1
Tiefe bezüglich virtueller Kamera
http://www.researchgate.net/publication/225091328_Real-Time_Volumetric_Tests_Using_Layered_Depth_Images
http://www.researchgate.net/publication/220862274_Efficient_Representation_of_Layered_Depth_Images_for_Real-time_Volumetric_Tests
Applications :: 3D Lenses
Context (LOA1)
Single Focus (LOA0)
3D Lens Shape
http://de.slideshare.net/autopolit/3-d-generalization-lenses
http://www.researchgate.net/publication/4358748_3D_Generalization_Lenses_for_Interactive_Focus__Context_Visualization_of_Virtual_City_Models
Applications :: 3D Lenses
LOA0
LOA1
LOA1
LOA2
WRAP UP
State-of-the-Art
State-of-the-Art
http://www.youtube.com/watch?v=utnu1mFynB0
http://www.researchgate.net/publication/259616187_Hardware-Accelerated_Attribute_Mapping_for_Interactive_Visualization_of_Complex_3D_Trajectories
Take aways...
Fixed-function rendering pipeline is dead
Today, GPU programming is about:
Manage State
Manage Buffer
Manage Shader Programs
Knowledge of shading languages is essential for GPU programming
Resources
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.