什么是特征追踪?本章节介绍了什么是计算机视觉的特征追踪,从几何的角度分析了特征追踪的特性,特征追踪的所需要使用的方法,如何进行特征追踪等,另外就关于计算机视觉的特征追踪如何运用到我们生活中做了相关的介绍。

注脚

展开查看详情

1. Lecture  13:     Tracking  mo3on  features     –  op3cal  flow   Professor  Fei-­‐Fei  Li   Stanford  Vision  Lab   Fei-Fei Li! Lecture 14 - 1  ! 3-­‐Nov-­‐14  

2. What  we  will  learn  today?   •  Introduc3on   •  Op3cal  flow   •  Feature  tracking   •  Applica3ons   •  (Supplementary)  Technical  note   Reading:  [Szeliski]  Chapters:  8.4,  8.5     [Fleet  &  Weiss,  2005]   hYp://www.cs.toronto.edu/pub/jepson/teaching/vision/2503/op3calFlow.pdf   Fei-Fei Li! Lecture 14 - 2  ! 3-­‐Nov-­‐14  

3. From  images  to  videos   •  A  video  is  a  sequence  of  frames  captured  over  3me   •  Now  our  image  data  is  a  func3on  of  space  (x,  y)  and  3me  (t)   Fei-Fei Li! Lecture 14 - 3  ! 3-­‐Nov-­‐14  

4. Mo3on  es3ma3on  techniques   •  Op3cal  flow   –  Recover  image  mo3on  at  each  pixel  from  spa3o-­‐temporal   image  brightness  varia3ons  (op3cal  flow)   •  Feature-­‐tracking   –  Extract  visual  features  (corners,  textured  areas)  and  “track”   them  over  mul3ple  frames   Fei-Fei Li! Lecture 14 - 4  ! 3-­‐Nov-­‐14  

5. Op3cal  flow   Vector  field  func3on  of  the   spa3o-­‐temporal  image   brightness  varia3ons     Picture  courtesy  of  Selim  Temizer  -­‐  Learning  and  Intelligent  Systems  (LIS)  Group,  MIT     Fei-Fei Li! Lecture 14 - 5  ! 3-­‐Nov-­‐14  

6. Feature-­‐tracking   Courtesy  of  Jean-­‐Yves  Bouguet  –  Vision  Lab,  California  Ins3tute  of  Technology   Fei-Fei Li! Lecture 14 - 6  ! 3-­‐Nov-­‐14  

7. Feature-­‐tracking   Courtesy  of  Jean-­‐Yves  Bouguet  –  Vision  Lab,  California  Ins3tute  of  Technology   Fei-Fei Li! Lecture 14 - 7  ! 3-­‐Nov-­‐14  

8. Op3cal  flow   •  Defini3on:  op3cal  flow  is  the  apparent  mo3on  of   brightness  paYerns  in  the  image   •  Note:  apparent  mo3on  can  be  caused  by  ligh3ng   changes  without  any  actual  mo3on   –  Think  of  a  uniform  rota3ng  sphere  under  fixed  ligh3ng   vs.  a  sta3onary  sphere  under  moving  illumina3on   Source:  Silvio  Savarese   GOAL:  Recover  image  mo3on  at  each  pixel  from   op3cal  flow   Fei-Fei Li! Lecture 14 - 8  ! 3-­‐Nov-­‐14  

9. Es3ma3ng  op3cal  flow   I(x,y,t–1) I(x,y,t) •  Given  two  subsequent  frames,  es3mate  the  apparent  mo3on  field   u(x,y),  v(x,y)  between  them   •  Key  assump3ons   Source:  Silvio  Savarese   •  Brightness  constancy:    projec3on  of  the  same  point  looks  the  same  in   every  frame   •  Small  mo9on:    points  do  not  move  very  far   •  Spa9al  coherence:  points  move  like  their  neighbors   Fei-Fei Li! Lecture 14 - 9  ! 3-­‐Nov-­‐14  

10. h n i cal The  brightness  constancy  constraint   tec note I(x,y,t–1) I(x,y,t) •  Brightness  Constancy  Equa3on:   I ( x, y, t − 1) = I ( x + u ( x, y ), y + v( x, y ), t ) Linearizing  the  right  side  using  Taylor  expansion:   Source:  Silvio  Savarese   Image  deriva3ve  along  x   I ( x + u, y + u, t ) ≈ I ( x, y, t − 1) + I x ⋅ u ( x, y ) + I y ⋅ v( x, y ) + I t I ( x + u, y + u, t ) − I ( x, y, t − 1) = I x ⋅ u ( x, y ) + I y ⋅ v( x, y ) + I t T Hence,   I x ⋅ u + I y ⋅ v + I t ≈ 0 → ∇I ⋅ [u v] + I t = 0 Fei-Fei Li! Lecture 14 - 10   ! 3-­‐Nov-­‐14  

11. h n i cal The  brightness  constancy  constraint   tec note Can  we  use  this  equa3on  to  recover  image  mo3on  (u,v)  at  each   pixel?   T ∇I ⋅ [u v] + I t = 0 •  How  many  equa3ons  and  unknowns  per  pixel?   • One  equa3on  (this  is  a  scalar  equa3on!),  two  unknowns  (u,v)   The  component  of  the  flow  perpendicular  to  the  gradient  (i.e.,   parallel  to  the  edge)  cannot  be  measured   gradient   Source:  Silvio  Savarese   (u,v)   If  (u,  v  )  sa3sfies  the  equa3on,     so  does  (u+u’,  v+v’  )  if     T (u+u’,v+v’)   ∇I ⋅ [u ' v'] = 0 (u’,v’)   edge   Fei-Fei Li! Lecture 14 - 11   ! 3-­‐Nov-­‐14  

12. The  aperture  problem   Source:  Silvio  Savarese   Actual  mo9on   Fei-Fei Li! Lecture 14 - 12   ! 3-­‐Nov-­‐14  

13. The  aperture  problem   Source:  Silvio  Savarese   Perceived  mo9on   Fei-Fei Li! Lecture 14 - 13   ! 3-­‐Nov-­‐14  

14. The  barber  pole  illusion   Source:  Silvio  Savarese   hYp://en.wikipedia.org/wiki/Barberpole_illusion   Fei-Fei Li! Lecture 14 - 14   ! 3-­‐Nov-­‐14  

15. The  barber  pole  illusion   Source:  Silvio  Savarese   hYp://en.wikipedia.org/wiki/Barberpole_illusion   Fei-Fei Li! Lecture 14 - 15   ! 3-­‐Nov-­‐14  

16. Aperture  problem  cont’d   *  From  Marc  Pollefeys  COMP  256  2003   Fei-Fei Li! Lecture 14 - 16   ! 3-­‐Nov-­‐14  

17.tec h n i cal Solving  the    ambiguity…   note B.  Lucas  and  T.  Kanade.  An  itera3ve  image  registra3on  technique  with  an  applica3on  to  stereo   vision.  In  Proceedings  of  the  Interna6onal  Joint  Conference  on  Ar6ficial  Intelligence,  pp.  674– 679,  1981.   •  How  to  get  more  equa3ons  for  a  pixel?   •  Spa9al  coherence  constraint:       •         Assume  the  pixel’s  neighbors  have  the  same  (u,v)   –  If  we  use  a  5x5  window,  that  gives  us  25  equa3ons  per  pixel   Source:  Silvio  Savarese   Fei-Fei Li! Lecture 14 - 17   ! 3-­‐Nov-­‐14  

18.t e c h nica note l Lucas-­‐Kanade  flow   •  Overconstrained  linear  system:   Source:  Silvio  Savarese   Fei-Fei Li! Lecture 14 - 18   ! 3-­‐Nov-­‐14  

19. Condi3ons  for  solvability   •  When  is  this  system  solvable?   •  What  if  the  window  contains  just  a  single  straight  edge?   Source:  Silvio  Savarese   Fei-Fei Li! Lecture 14 - 19   ! 3-­‐Nov-­‐14  

20.t e c h nica note l Lucas-­‐Kanade  flow   •  Overconstrained  linear  system   Least  squares  solu3on  for  d  given  by   Source:  Silvio  Savarese   The  summa3ons  are  over  all  pixels  in  the  K  x  K  window   Fei-Fei Li! Lecture 14 - 20   ! 3-­‐Nov-­‐14  

21. Condi3ons  for  solvability   l e c h nica t note –  Op3mal  (u,  v)  sa3sfies  Lucas-­‐Kanade  equa3on   When  is  This  Solvable?   •  ATA  should  be  inver3ble     •  ATA  should  not  be  too  small  due  to  noise   Source:  Silvio  Savarese   –  eigenvalues  λ1  and  λ  2  of  ATA  should  not  be  too  small   •  ATA  should  be  well-­‐condi3oned   –   λ  1/  λ  2  should  not  be  too  large  (λ  1  =  larger  eigenvalue)   Does  this  remind  anything  to  you?   Fei-Fei Li! Lecture 14 - 21   ! 3-­‐Nov-­‐14  

22. l e c h nica t note M  =  ATA  is  the  second  moment  matrix  !   (Harris  corner  detector…)   •  Eigenvectors  and  eigenvalues  of  ATA  relate  to   edge  direc3on  and  magnitude     •  The  eigenvector  associated  with  the  larger  eigenvalue  points  in   Source:  Silvio  Savarese   the  direc3on  of  fastest  intensity  change   •  The  other  eigenvector  is  orthogonal  to  it   Fei-Fei Li! Lecture 14 - 22   ! 3-­‐Nov-­‐14  

23. Interpre3ng  the  eigenvalues   l e c h nica t note Classifica3on  of  image  points  using  eigenvalues  of  the   second  moment  matrix:   λ2 “Edge” λ2 >> λ1 “Corner” λ1 and λ2 are large, λ1 ~ λ2 Source:  Silvio  Savarese   λ1 and λ2 are small “Flat” “Edge” region λ1 >> λ2 λ1 Fei-Fei Li! Lecture 14 - 23   ! 3-­‐Nov-­‐14  

24. Edge   l e c h nica t note Source:  Silvio  Savarese   –   gradients  very  large  or  very  small   –   large λ1,  small  λ2   Fei-Fei Li! Lecture 14 - 24   ! 3-­‐Nov-­‐14  

25. Low-­‐texture  region   l e c h nica t note Source:  Silvio  Savarese   –   gradients  have  small  magnitude   –   small λ1,  small  λ2   Fei-Fei Li! Lecture 14 - 25   ! 3-­‐Nov-­‐14  

26. High-­‐texture  region   l e c h nica t note Source:  Silvio  Savarese   –   gradients  are  different,  large  magnitudes   –   large λ1,  large  λ2   Fei-Fei Li! Lecture 14 - 26   ! 3-­‐Nov-­‐14  

27. What  we  will  learn  today?   •  Introduc3on   •  Op3cal  flow   •  Feature  tracking   •  Applica3ons   •  (Supplementary)  Technical  note   Fei-Fei Li! Lecture 14 - 27   ! 3-­‐Nov-­‐14  

28. What  are  good  features  to  track?   •  Can  measure  “quality”  of  features  from  just  a   single  image   •  Hence:  tracking  Harris  corners  (or  equivalent)   guarantees  small  error  sensi3vity!   Source:  Silvio  Savarese   à    Implemented  in  Open  CV   Fei-Fei Li! Lecture 14 - 28   ! 3-­‐Nov-­‐14  

29. Recap   •  Key  assump3ons  (Errors  in  Lucas-­‐Kanade)   •  Small  mo9on:    points  do  not  move  very  far   •  Brightness  constancy:    projec3on  of  the  same  point  looks   the  same  in  every  frame   •  Spa9al  coherence:  points  move  like  their  neighbors   Source:  Silvio  Savarese   Fei-Fei Li! Lecture 14 - 29   ! 3-­‐Nov-­‐14