| Slide1 : Computer Vision in Graphics Production
Visual Media Research Group
Centre for Vision, Speech and Signal Processing
University of Surrey
|
| Slide2 : Overview Where is computer vision useful in content production?
Where is vision used?
What can and can’t current computer vision do? |
| Slide3 : Computer Vision in Content Production Computer vision: analysis & interpretation of real images/video
Match moving
Matting
Model building
Human motion capture
Studio Production
Facial animation
Image-based illumination
|
| Slide4 : Match Moving Reconstruction of camera movement to composite CG
Products: 2d3, RealViz … (semi-automatic camera tracking)
… standard tool in film production Oxford University/2d3 |
| Slide5 : Matting Separation of foreground and background objects
- actor/background separation
- wire/set removal
Studio: chroma-key (solved)
Post-production: Imagineer, RealViz, …. (open-problem) |
| Slide6 : Model Building Capture of real shape for CG models
objects
environments
characters
Shape capture
active sensors (laser/structured-light)
- accurate 3D surface measurements
- static objects or environments
structure from images
- low-accuracy
- static objects
Problem: unstructured surface measurements |
| Slide7 : Model Building (2) Walking with Dinosaurs - FrameStore |
| Slide8 : Structured Models |
| Slide9 : Courtesy Stanford Computer Graphics Lab. Animation of David |
| Slide10 : Human Motion Capture Capture of real motion
Marker based systems
widely used in performance animation whole-body/face
character animation (Golum)
‘realistic’ motion characteristics
accurate
real-time?
widely used in film production (with post-production) |
| Slide11 : Human Motion Capture(2) Markerless human motion capture
advantages: unintrusive; simultaneous appearance capture
model-based visual tracking
low-accuracy
visual ambiguity (uniform apperance, non-rigid shape)
|
| Slide12 : Human Motion Capture(3) |
| Slide13 : Studio Production Modelling actors
3D animated models
animation
re-illumination
loss of visual realism to captured images
Free-viewpoint video
post-production of arbitrary camera views/paths
‘matrix’ flowmo shots
aim: quality equivalent to captured video |
| Slide14 : Studio Production (2) |
| Slide16 : Limitations:
- shape detail face/hands/hair
- not video quality Studio Production (3) |
| Slide17 : Studio Production: Free-view video |
| Slide18 : Studio Production: Free-view video |
| Slide19 : Studio Production: Free-view video |
| Slide20 : Studio Production: Free-view video |
| Slide21 : Facial Animation Production of photo-realistic faces
2D video-based approaches
video rewrite [Bregler’97]
concatenative synthesis [Cossato’98,Ezzat’02]
photo-realistic
limited viewpoint, illumination
3D markers
Performance animation [Phigin’98]
shape only
3D video (shape+appearance)
concatenative synthesis
photo-realistic
control of viewpoint, illumination
|
| Slide22 : 3D Video |
| Slide23 : 3D Video Database 51 people: Expressions + Speech (short sentence) |
| Slide24 : Initial Face Synthesis from Speech Input: Output: |
| Slide25 : Image-based illumination Illumination of CG scenes with real illumination
high-dynamic range capture of illumination
environment mapping illumination to CG scene
widely used in production Debevec SIGGRAPH’00 FiatLux |
| Slide26 : Computer Vision in Content Production Match moving - yes, automatic camera tracking available
Matting - partial, many unsolved problems
Model capture – yes, tools for semi-automatic restructuring
Human motion capture – no, inaccurate
Studio Production – partial, free viewpoint video
Facial animation – yes, 3D video
Image-based illumination – yes, widely used
Other applications of computer vision….
|
| Slide27 : Future Directions Video-based rendering
photo-realistic, free-viewpoint rendering of dynamic scenes
relighting required
Video-based animation
animation from captured video (face,whole-body…)
control of motion, viewpoint, illumination
|
| Slide28 : People
Jon Starck, Joel Mitchelson, Gordon Collins,
Eng-Jon Ong, Ioannis Ypsilos, Rob Dilks,
Michael Kalkavouras Collaborators
BBC, BT, Sony, Canon, Philips,
Hensons, Framestore, Snell&Wilcox,
3D Scanners, AvatarMe
EPSRC – DTI |