图像滤波

1 .Image Filtering in Spatial domain Computer Vision Jia-Bin Huang, Virginia Tech

2 .Administrative stuffs Any questions? Office hours - Jia-Bin (440 Whittemore Hall ) Friday at 11 AM - 12 PM (this week only) Friday at 3:00 – 4:00 PM (lectures, HW discussions) Friday at 4:00 – 5:00 PM (final project discussions) Office hours - Akrit (264 Whittemore Hall) Wed at 10:30 AM – 11:30 AM MATLAB t utorial session by Akrit Friday 3-4 PM, Whittemore Hall 340A Bring your laptop with MATLAB installed

3 .Previous class: Light and Color Reflection models diffuse/specular reflectance, albedo Surface orientation and light intensity Color vision physics of light, trichromacy , color consistency, color spaces (RGB, HSV, Lab ) Object cast light and shadows to each other Interpret images from local differences

4 .Reflection models Albedo: fraction of light that is reflected Determines color (amount reflected at each wavelength) Very low albedo (hard to see shape) Higher albedo Slide credit: Derek Hoiem

5 .Reflection models Specular reflection: mirror-like Light reflects at incident angle Reflection color = incoming light color Slide credit: Derek Hoiem

6 .Reflection models Diffuse reflection Light scatters in all directions (proportional to cosine with surface normal) Observed intensity is independent of viewing direction Reflection color depends on light color and albedo Slide credit: Derek Hoiem

7 .Surface orientation and light intensity Amount of light that hits surface from distant point source depends on angle between surface normal and source 1 2   prop to cosine of relative angle Slide credit: Derek Hoiem

8 .Application: Photometric S tereo Assume: A Lambertian object A local shading model (each point on a surface receives light only from sources visible at that point) A set of known light source directions A set of pictures of an object, obtained in exactly the same camera/object configuration but using different sources Orthographic projection Goal: reconstruct object shape and albedo S n ??? S 1 S 2 F&P 2 nd ed., sec. 2.2.4 Slide credit: S. Lazebnik

9 .Example … Input Recovered normal field x y z Recovered surface model Recovered Albedo Slide credit: S. Lazebnik

10 .Photometric Stereo N L 1 L 2 V L 3 Can write this as a matrix equation: Slide credit: N. Snavely

11 .Solving the equations Intensity (Known) Light direction (Known ) Albedo x Surface normal (Unknown) How do we get the albedo and surface normal from G? Slide credit: N. Snavely

12 .More than three lights Get better results by using more lights What’s the size of L T L ? Least squares solution: Solve for N, k d as before In MATLAB G = L\I; Slide credit: N. Snavely

13 .Next three classes: three views of filtering Image filters in spatial domain Filter is a mathematical operation on values of each patch Smoothing, sharpening, measuring texture Image filters in the frequency domain Filtering is a way to modify the frequencies of images Denoising , sampling, image compression Templates and Image Pyramids Filtering is a way to match a template to the image Detection, coarse-to-fine registration Slide credit: Derek Hoiem

14 .Today’s class Pixels and image filtering Application: representing textures Denoising and non-linear image filtering

15 .Smoothing Sharpening Input https://en.wikipedia.org/wiki/Albert_Einstein_in_popular_culture#/media/File:Einstein_tongue.jpg Why should we care?

16 .Why should we care? Image Pyramid Image interpolation/resampling Source: D Forsyth Source : N Snavely

17 .Why should we care? Representing textures with filter banks LM filter bank. Code here

18 .The raster image (pixel matrix) 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99 0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33 0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93

19 .Image filtering Image filtering: for each pixel, compute function of local neighborhood and output a new value Same function applied at each position Output and input image are typically the same size 5 1 4 1 7 1 5 3 10 Local image data 7 Modified image data Some function Slide Credit: L. Zhang

20 .Image filtering Linear filtering function is a weighted sum/difference of pixel values Really important! Enhance images Denoise , smooth, increase contrast, etc. Extract information from images Texture, edges, distinctive points, etc. Detect patterns Template matching Slide credit: Derek Hoiem 0.5 0.5 0 0 1 0 0 0 0 kernel 8 Modified image data Local image data 6 1 4 1 8 1 5 3 10

21 .Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! What’s the next best thing? Source: S. Seitz Question: Noise reduction

22 .First attempt at a solution Let’s replace each pixel with an average of all the values in its neighborhood Assumptions: Expect pixels to be like their neighbors Expect noise processes to be independent from pixel to pixel 22 Slide credit: Kristen Grauman

23 .1 1 1 1 1 1 1 1 1 Example: box f ilter Slide credit: David Lowe

24 .0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 Image filtering Slide credit: S. Seitz

25 .0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 Image filtering Slide credit: S. Seitz

26 .0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 Image filtering Slide credit: S. Seitz

27 .0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 20 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 Image filtering Slide credit: S. Seitz

28 .0 10 20 30 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 Image filtering Slide credit: S. Seitz

29 .0 10 20 30 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 0 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 ? Image filtering Slide credit: S. Seitz