本章首先复习了上一讲相关相机的知识,然后介绍相机模型、单应性、解线性方程组的齐次方程组、Keypoint-based对齐、RANSAC、混合、iphone缝合器是如何工作的等相关内容。单应性(单应性)在数学中相当于射影线性变换在视觉(最常用)中:单应性是指两个图像平面之间的线性变换正如投影三维表面到正面视图或是将两个仅因旋转而不同的视图关联起来

注脚

展开查看详情

1.Image Stitching Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Add example

2.Administrative stuffs HW 3 is out due 11:59 PM Oct 17 Please start early. D eadlines are firm. N o emails requesting extensions Getting help? *Five* free late days without penalty Piazza Office hours No free late dates for final projects

3.Review: Camera Projection Matrix O w i w k w j w t R 3

4.Review: Camera Calibration Method 1: Use an object (calibration grid) with known geometry Correspond image points to 3d points Get least squares solution (or non-linear solution) 4 Known 3d locations Known 2d image coordinates Unknown Camera Parameters

5.Review: Camera Calibration Method 1: Use an object (calibration grid) with known geometry Correspond image points to 3d points Get least squares solution (or non-linear solution) 4 Known 3d locations Known 2d image coordinates Unknown Camera Parameters

6.Constraints for ,   Review: Calibration by vanishing points     Orthogonality constraints VP (2D) VP (3D)       Unknown c amera parameters     = 0 = 0 = 0   … Eqn (1) … Eqn (2) … Eqn (3)   Eqn (1 ) – Eqn (2) Eqn (2) – Eqn ( 3 ) Solve for  

7.Rotation matrix Set directions of vanishing points   Review: Calibration by vanishing points Unknown c amera parameters     Special properties of R inv ( R )= R T Each row and column of R has unit length

8.Measuring height R H v z r b t H b 0 t 0 v v x v y vanishing line (horizon) image cross ratio Slide by Steve Seitz 8

9.This class: Image Stitching Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

10.Concepts introduced/reviewed in today’s lecture Camera model Homographies Solving homogeneous systems of linear equations Keypoint -based alignment RANSAC Blending How the iphone stitcher works

11.Illustration Camera Center

12.Problem set-up x = K [R t] X x = K [R t] X t=t=0 x= Hx where H = K R R -1 K -1 Typically only R and f will change (4 parameters), but, in general, H has 8 parameters f f . x x X

13.Homography Definition General mathematics: homography = projective linear transformation Vision (most common usage): homography = linear transformation between two image planes Examples Project 3D surface into frontal view Relate two views that differ only by rotation

14.Homography example: Image rectification To unwarp (rectify) an image solve for homography H given p and p ’ : w p ’=Hp p p’

15.Homography example: Planar mapping Freedom HP Commercial

16.Image Stitching Algorithm Overview Detect keypoints (e.g., SIFT) Match keypoints (e.g., 1 st /2 nd NN < thresh) Estimate homography with four matched keypoints (using RANSAC) Combine images

17.Computing homography Assume we have four matched points: How do we compute homography H ? Direct Linear Transformation (DLT)

18.Computing homography Direct Linear Transform Apply SVD: UDV T = A h = V smallest (column of V corr. to smallest singular value) Matlab [U, S, V] = svd(A); h = V(:, end); Explanations of SVD and solving homogeneous linear systems

19.Computing homography Assume we have four matched points: How do we compute homography H ? Normalized DLT Normalize coordinates for each image Translate for zero mean Scale so that average distance to origin is ~ sqrt (2) This makes problem better behaved numerically (see HZ p. 107-108) Compute using DLT in normalized coordinates Unnormalize :

20.Computing homography Assume we have matched points with outliers: How do we compute homography H ? Automatic Homography Estimation with RANSAC Choose number of samples N HZ Tutorial ‘99

21.Computing homography Assume we have matched points with outliers: How do we compute homography H ? Automatic Homography Estimation with RANSAC Choose number of samples N Choose 4 random potential matches Compute H using normalized DLT Project points from x to x ’ for each potentially matching pair: Count points with projected distance < t E.g., t = 3 pixels Repeat steps 2-5 N times Choose H with most inliers HZ Tutorial ‘99

22.Automatic Image Stitching Compute interest points on each image Find candidate matches Estimate homography H using matched points and RANSAC with normalized DLT Project each image onto the same surface and blend Matlab : maketform , imtransform

23.RANSAC for Homography Initial Matched Points

24.RANSAC for Homography Final Matched Points

25.Verification

26.RANSAC for Homography

27.Choosing a Projection Surface Many to choose: planar, cylindrical, spherical, cubic, etc.

28.Planar Mapping f f x x For red image: pixels are already on the planar surface For green image: map to first image plane

29.Planar Projection Planar Photos by Russ Hewett