特征追踪

下载 0

乐乐

发布于

2989

人观看

#信息技术

什么是特征追踪？本章节介绍了什么是计算机视觉的特征追踪，从几何的角度分析了特征追踪的特性，特征追踪的所需要使用的方法，如何进行特征追踪等，另外就关于计算机视觉的特征追踪如何运用到我们生活中做了相关的介绍。

展开查看详情

1 . Lecture 13: Tracking mo3on features – op3cal ﬂow Professor Fei-‐Fei Li Stanford Vision Lab Fei-Fei Li! Lecture 14 - 1 ! 3-‐Nov-‐14

2 . What we will learn today? •  Introduc3on •  Op3cal ﬂow •  Feature tracking •  Applica3ons •  (Supplementary) Technical note Reading: [Szeliski] Chapters: 8.4, 8.5 [Fleet & Weiss, 2005] hYp://www.cs.toronto.edu/pub/jepson/teaching/vision/2503/op3calFlow.pdf Fei-Fei Li! Lecture 14 - 2 ! 3-‐Nov-‐14

3 . From images to videos •  A video is a sequence of frames captured over 3me •  Now our image data is a func3on of space (x, y) and 3me (t) Fei-Fei Li! Lecture 14 - 3 ! 3-‐Nov-‐14

4 . Mo3on es3ma3on techniques •  Op3cal ﬂow –  Recover image mo3on at each pixel from spa3o-‐temporal image brightness varia3ons (op3cal ﬂow) •  Feature-‐tracking –  Extract visual features (corners, textured areas) and “track” them over mul3ple frames Fei-Fei Li! Lecture 14 - 4 ! 3-‐Nov-‐14

5 . Op3cal ﬂow Vector ﬁeld func3on of the spa3o-‐temporal image brightness varia3ons Picture courtesy of Selim Temizer -‐ Learning and Intelligent Systems (LIS) Group, MIT Fei-Fei Li! Lecture 14 - 5 ! 3-‐Nov-‐14

6 . Feature-‐tracking Courtesy of Jean-‐Yves Bouguet – Vision Lab, California Ins3tute of Technology Fei-Fei Li! Lecture 14 - 6 ! 3-‐Nov-‐14

7 . Feature-‐tracking Courtesy of Jean-‐Yves Bouguet – Vision Lab, California Ins3tute of Technology Fei-Fei Li! Lecture 14 - 7 ! 3-‐Nov-‐14

8 . Op3cal ﬂow •  Deﬁni3on: op3cal ﬂow is the apparent mo3on of brightness paYerns in the image •  Note: apparent mo3on can be caused by ligh3ng changes without any actual mo3on –  Think of a uniform rota3ng sphere under ﬁxed ligh3ng vs. a sta3onary sphere under moving illumina3on Source: Silvio Savarese GOAL: Recover image mo3on at each pixel from op3cal ﬂow Fei-Fei Li! Lecture 14 - 8 ! 3-‐Nov-‐14

9 . Es3ma3ng op3cal ﬂow I(x,y,t–1) I(x,y,t) •  Given two subsequent frames, es3mate the apparent mo3on ﬁeld u(x,y), v(x,y) between them •  Key assump3ons Source: Silvio Savarese •  Brightness constancy: projec3on of the same point looks the same in every frame •  Small mo9on: points do not move very far •  Spa9al coherence: points move like their neighbors Fei-Fei Li! Lecture 14 - 9 ! 3-‐Nov-‐14

10 . h n i cal The brightness constancy constraint tec note I(x,y,t–1) I(x,y,t) •  Brightness Constancy Equa3on: I ( x, y, t − 1) = I ( x + u ( x, y ), y + v( x, y ), t ) Linearizing the right side using Taylor expansion: Source: Silvio Savarese Image deriva3ve along x I ( x + u, y + u, t ) ≈ I ( x, y, t − 1) + I x ⋅ u ( x, y ) + I y ⋅ v( x, y ) + I t I ( x + u, y + u, t ) − I ( x, y, t − 1) = I x ⋅ u ( x, y ) + I y ⋅ v( x, y ) + I t T Hence, I x ⋅ u + I y ⋅ v + I t ≈ 0 → ∇I ⋅ [u v] + I t = 0 Fei-Fei Li! Lecture 14 - 10 ! 3-‐Nov-‐14

11 . h n i cal The brightness constancy constraint tec note Can we use this equa3on to recover image mo3on (u,v) at each pixel? T ∇I ⋅ [u v] + I t = 0 •  How many equa3ons and unknowns per pixel? • One equa3on (this is a scalar equa3on!), two unknowns (u,v) The component of the ﬂow perpendicular to the gradient (i.e., parallel to the edge) cannot be measured gradient Source: Silvio Savarese (u,v) If (u, v ) sa3sﬁes the equa3on, so does (u+u’, v+v’ ) if T (u+u’,v+v’) ∇I ⋅ [u ' v'] = 0 (u’,v’) edge Fei-Fei Li! Lecture 14 - 11 ! 3-‐Nov-‐14

12 . The aperture problem Source: Silvio Savarese Actual mo9on Fei-Fei Li! Lecture 14 - 12 ! 3-‐Nov-‐14

13 . The aperture problem Source: Silvio Savarese Perceived mo9on Fei-Fei Li! Lecture 14 - 13 ! 3-‐Nov-‐14

14 . The barber pole illusion Source: Silvio Savarese hYp://en.wikipedia.org/wiki/Barberpole_illusion Fei-Fei Li! Lecture 14 - 14 ! 3-‐Nov-‐14

15 . The barber pole illusion Source: Silvio Savarese hYp://en.wikipedia.org/wiki/Barberpole_illusion Fei-Fei Li! Lecture 14 - 15 ! 3-‐Nov-‐14

16 . Aperture problem cont’d * From Marc Pollefeys COMP 256 2003 Fei-Fei Li! Lecture 14 - 16 ! 3-‐Nov-‐14

17 .tec h n i cal Solving the ambiguity… note B. Lucas and T. Kanade. An itera3ve image registra3on technique with an applica3on to stereo vision. In Proceedings of the Interna6onal Joint Conference on Ar6ﬁcial Intelligence, pp. 674– 679, 1981. •  How to get more equa3ons for a pixel? •  Spa9al coherence constraint: •  Assume the pixel’s neighbors have the same (u,v) –  If we use a 5x5 window, that gives us 25 equa3ons per pixel Source: Silvio Savarese Fei-Fei Li! Lecture 14 - 17 ! 3-‐Nov-‐14

18 .t e c h nica note l Lucas-‐Kanade ﬂow •  Overconstrained linear system: Source: Silvio Savarese Fei-Fei Li! Lecture 14 - 18 ! 3-‐Nov-‐14

19 . Condi3ons for solvability •  When is this system solvable? •  What if the window contains just a single straight edge? Source: Silvio Savarese Fei-Fei Li! Lecture 14 - 19 ! 3-‐Nov-‐14

20 .t e c h nica note l Lucas-‐Kanade ﬂow •  Overconstrained linear system Least squares solu3on for d given by Source: Silvio Savarese The summa3ons are over all pixels in the K x K window Fei-Fei Li! Lecture 14 - 20 ! 3-‐Nov-‐14

21 . Condi3ons for solvability l e c h nica t note –  Op3mal (u, v) sa3sﬁes Lucas-‐Kanade equa3on When is This Solvable? •  ATA should be inver3ble •  ATA should not be too small due to noise Source: Silvio Savarese –  eigenvalues λ1 and λ 2 of ATA should not be too small •  ATA should be well-‐condi3oned –  λ 1/ λ 2 should not be too large (λ 1 = larger eigenvalue) Does this remind anything to you? Fei-Fei Li! Lecture 14 - 21 ! 3-‐Nov-‐14

22 . l e c h nica t note M = ATA is the second moment matrix ! (Harris corner detector…) •  Eigenvectors and eigenvalues of ATA relate to edge direc3on and magnitude •  The eigenvector associated with the larger eigenvalue points in Source: Silvio Savarese the direc3on of fastest intensity change •  The other eigenvector is orthogonal to it Fei-Fei Li! Lecture 14 - 22 ! 3-‐Nov-‐14

23 . Interpre3ng the eigenvalues l e c h nica t note Classiﬁca3on of image points using eigenvalues of the second moment matrix: λ2 “Edge” λ2 >> λ1 “Corner” λ1 and λ2 are large, λ1 ~ λ2 Source: Silvio Savarese λ1 and λ2 are small “Flat” “Edge” region λ1 >> λ2 λ1 Fei-Fei Li! Lecture 14 - 23 ! 3-‐Nov-‐14

24 . Edge l e c h nica t note Source: Silvio Savarese –  gradients very large or very small –  large λ1, small λ2 Fei-Fei Li! Lecture 14 - 24 ! 3-‐Nov-‐14

25 . Low-‐texture region l e c h nica t note Source: Silvio Savarese –  gradients have small magnitude –  small λ1, small λ2 Fei-Fei Li! Lecture 14 - 25 ! 3-‐Nov-‐14

26 . High-‐texture region l e c h nica t note Source: Silvio Savarese –  gradients are diﬀerent, large magnitudes –  large λ1, large λ2 Fei-Fei Li! Lecture 14 - 26 ! 3-‐Nov-‐14

27 . What we will learn today? •  Introduc3on •  Op3cal ﬂow •  Feature tracking •  Applica3ons •  (Supplementary) Technical note Fei-Fei Li! Lecture 14 - 27 ! 3-‐Nov-‐14

28 . What are good features to track? •  Can measure “quality” of features from just a single image •  Hence: tracking Harris corners (or equivalent) guarantees small error sensi3vity! Source: Silvio Savarese à Implemented in Open CV Fei-Fei Li! Lecture 14 - 28 ! 3-‐Nov-‐14

29 . Recap •  Key assump3ons (Errors in Lucas-‐Kanade) •  Small mo9on: points do not move very far •  Brightness constancy: projec3on of the same point looks the same in every frame •  Spa9al coherence: points move like their neighbors Source: Silvio Savarese Fei-Fei Li! Lecture 14 - 29 ! 3-‐Nov-‐14

0点赞

0收藏

0下载