展开查看详情
2 .Learning Local Neighboring Structure for
Robust 3D Shape Representation
Zhongpai Gao1, Junchi Yan1, Guangtao Zhai1, Juyong Zhang2,
Yiyan Yang1, and Xiaokang Yang1
1Artificial
Intelligence Institute, Shanghai Jiao Tong University
2University of Science and Technology of China
April 8th, 2021
3 . 3
The success story of deep learning
Medical image processing
The data (e.g., audio and texts, images, CT scans) are regular and grid-structured
Convolutional neural networks (CNN)
- anisotropic filters
- hierarchical sampling
4 .3D geometric data — meshes
A lot of real-world data do not ‘live’ on grids and are graph-structured
• social networks, knowledge graphs, molecules, etc.
This paper focuses on 3D meshes that share a fixed-topology template
4
5 . 5
Enabling technology for many applications
3D shape correspondence [Donati 2020] 3D body reconstruction [Pavlakos 2019]
Animated Emojis Filters
[Curious Case of
Benjamin Button]
[Beowulf] [Samsung] [Instagram]
[Siren GDC 2018]
CG Actors in VFX, VR, Games Consumer Applications
6 .Recap: CNN on Euclidean data
Kipf & Welling (ICLR6 2017)
7 .Recap: CNN on Euclidean data
W is an anisotropic filter
Kipf & Welling (ICLR7 2017)
8 .Definition of graph
G = (𝒱𝒱, ℰ) xi ,3
𝒱𝒱 : Set of nodes {xi }, |𝒱𝒱|=𝑁𝑁
xi ,2
ℰ : Set of edges {(xi , xj )}
xi ,0
𝒩𝒩𝒾𝒾 : Set of one-ring neighbors of xi
xi ,1
{xi,0 , xi,1 , …, xi, 𝒩𝒩𝒾𝒾 −1 } xi ,4
xi ,5
8
9 . Previous methods
GCN using isotropic filters
xi ,3
� XW ,
Y=f A xi ,2
� is normalized adjacency matrix and
where A xi ,0
W is an isotropic filter. xi ,1
xi ,4
xi ,5
9
T. N. Kipf, M. Welling, Semi-Supervised Classification with Graph Convolutional Networks (ICLR 2017)
10 . Previous methods
SpiralNet
Bouritsas, Giorgos, et al. "Neural 3d morphable models: Spiral convolutional networks for 3d shape representation
10
learning and generation." Proceedings of the IEEE International Conference on Computer Vision. 2019.
11 .Permutation matrix
Inspired by permutation matrix, we learn a weighting matrix, i.e., a soft-
permutation matrix, to adapt the local structure of a vertex.
11
12 .LSA-Conv for 3D meshes
Local structure-aware anisotropic convolution (LSA-Conv)
xi ,0 xi ,1 xi ,2 xi ,3 xi ,4 xi ,5 x0 x1 x2 x3 x4 x5 x6 x7 x8 yi
xi ,3 x0 x1 x2
xi ,2
X i Pi → X i vec( X i )Τ W + b → yi
xi ,0 x3 x4 x5 yi
xi ,1
xi ,4 x6 x7 x8
xi ,5
Conventional convolution operation
12
13 .Architecture of autoencoder as the testbed
13
14 .Experiments
Datasets
DFAUST: A human body dataset that collects over 40,000 real meshes, capturing 129
dynamic performances from 10 subjects
COMA: A human facial dataset that consists of 12 classes of extreme expressions from 12
different subjects
Training
Adam optimizer, learning rate 0.001 with decay rate 0.99 in every epoch
Batch size is 32
total epoch number is 300.
14
20 .Parameter reduction for LSA-Conv (LSA-small)
xi ,0 xi ,1 xi ,2 xi ,3 xi ,4 xi ,5 Pi x0 x1 x2 x3 x4 x5 x6 x7 x8
xi ,3
x0 x1 x2
xi ,2
X i Pi → X i
xi ,0 x3 x4 x5
xi ,1
xi ,4
x6 x7 x8
xi ,5 vi PB → Pi
vec( X i )Τ W + b → yi
yi
vi
PB 20
24 . 24
Applications: 3D shape correspondences
For the FAUST-inter dataset, we achieve 2.501 cm: an improvement of 15%
over Groueix et al. (2018): 2.878cm.
Groueix et al. (2018) simply uses PointNet as the decoder.
25 . 25
Applications: monocular 3D face reconstruction
26 . 26
Learning neural dictionary
29 .3D geometric data — point clouds
LiDAR point clouds
29
热门城市
