no code implementations • 16 Apr 2024 • Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, Baining Guo
We introduce VASA, a framework for generating lifelike talking faces with appealing visual affective skills (VAS) given a single static image and a speech audio clip.
no code implementations • 28 Mar 2024 • BoWen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, Baining Guo
We introduce a radiance representation that is both structured and fully explicit and thus greatly facilitates 3D generative modeling.
no code implementations • 18 Mar 2024 • Ruicheng Wang, Jianfeng Xiang, Jiaolong Yang, Xin Tong
We propose a novel image editing technique that enables 3D manipulations on single images, such as object rotation and translation.
1 code implementation • 9 Jan 2024 • Ronglai Zuo, Fangyun Wei, Zenggui Chen, Brian Mak, Jiaolong Yang, Xin Tong
The objective of this paper is to develop a functional system for translating spoken languages into sign languages, referred to as Spoken2Sign translation.
1 code implementation • 12 Dec 2023 • Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jiaolong Yang, Seungryong Kim, Chong Luo
This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision.
no code implementations • 5 Sep 2023 • Yue Wu, Sicheng Xu, Jianfeng Xiang, Fangyun Wei, Qifeng Chen, Jiaolong Yang, Xin Tong
For the new task, we base our method on the generative radiance manifold representation and equip it with learnable facial and head-shoulder deformations.
no code implementations • ICCV 2023 • Jianfeng Xiang, Jiaolong Yang, Binbin Huang, Xin Tong
In this paper, we introduce a novel 3D-aware image generation method that leverages 2D diffusion models.
no code implementations • 28 Feb 2023 • Yizhong Zhang, Zhiqi Li, Sicheng Xu, Chong Li, Jiaolong Yang, Xin Tong, Baining Guo
A key challenge in emulating the remote hand touch is the realistic rendering of the participant's hand and arm as the hand touches the screen.
no code implementations • 19 Dec 2022 • Yangyu Huang, Xi Chen, Jongyoo Kim, Hao Yang, Chong Li, Jiaolong Yang, Dong Chen
To evaluate our method, we manually label the dense landmarks on 300W testset.
Ranked #1 on Face Alignment on 300W
no code implementations • CVPR 2023 • Yu Yin, Kamran Ghasedi, HsiangTao Wu, Jiaolong Yang, Xin Tong, Yun Fu
Furthermore, our method leverages explicit and implicit 3D regularizations using the in-domain neighborhood samples around the optimized latent code to remove geometrical and visual artifacts.
1 code implementation • 18 Nov 2022 • Daichi Horita, Jiaolong Yang, Dong Chen, Yuki Koyama, Kiyoharu Aizawa, Nicu Sebe
The structure generator generates an edge image representing plausible structures within the holes, which is then used for guiding the texture generation process.
1 code implementation • 12 Oct 2022 • Yue Wu, Yu Deng, Jiaolong Yang, Fangyun Wei, Qifeng Chen, Xin Tong
To achieve meaningful control over facial expressions via deformation, we propose a 3D-level imitative learning scheme between the generator and a parametric 3D face model during adversarial training of the 3D-aware GAN.
no code implementations • 9 Sep 2022 • Ziyu Wang, Yu Deng, Jiaolong Yang, Jingyi Yu, Xin Tong
Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e. g., chairs) with large topological variance.
1 code implementation • 29 Jul 2022 • Yucheol Jung, Wonjong Jang, Soongjin Kim, Jiaolong Yang, Xin Tong, Seungyong Lee
To achieve the goal, we propose an MLP-based framework for building a deformable surface model, which takes a latent code and produces a 3D surface.
no code implementations • ICCV 2023 • Jianfeng Xiang, Jiaolong Yang, Yu Deng, Xin Tong
This paper proposes a novel 3D-aware GAN that can generate high resolution images (up to 1024X1024) while keeping strict 3D consistency as in volume rendering.
1 code implementation • 24 May 2022 • Yuxuan Han, Ruicheng Wang, Jiaolong Yang
This paper deals with the challenging task of synthesizing novel views for in-the-wild photographs.
1 code implementation • 25 Apr 2022 • Hao Ouyang, Bo Zhang, Pan Zhang, Hao Yang, Jiaolong Yang, Dong Chen, Qifeng Chen, Fang Wen
We propose pose-guided multiplane image (MPI) synthesis which can render an animatable character in real scenes with photorealistic quality.
no code implementations • 31 Mar 2022 • Xiangjun Gao, Jiaolong Yang, Jongyoo Kim, Sida Peng, Zicheng Liu, Xin Tong
For this task, we propose a simple yet effective method to train a generalizable NeRF with multiview images as conditional input.
no code implementations • CVPR 2022 • Yu Deng, Jiaolong Yang, Jianfeng Xiang, Xin Tong
3D-aware image generative modeling aims to generate 3D-consistent images with explicitly controllable camera poses.
no code implementations • 13 Dec 2021 • Yizhong Zhang, Jiaolong Yang, Zhen Liu, Ruicheng Wang, Guojun Chen, Xin Tong, Baining Guo
The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies.
1 code implementation • 4 Aug 2021 • Kaixuan Wei, Ying Fu, Yinqiang Zheng, Jiaolong Yang
Enhancing the visibility in extreme low-light environments is a challenging task.
Ranked #4 on Image Denoising on SID SonyA7S2 x100
1 code implementation • 9 Jul 2021 • Wonjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang, Xin Tong, Seungyong Lee
Our framework, dubbed StyleCariGAN, automatically creates a realistic and detailed caricature from an input photo with optional controls on shape exaggeration degree and color stylization type.
1 code implementation • 26 May 2021 • Yuxuan Han, Jiaolong Yang, Ying Fu
We further propose a Disentanglement-Transformation (DT) metric to quantify the attribute transformation and disentanglement efficacy and find the optimal control factor between attribute-level and instance-specific directions based on it.
1 code implementation • ICCV 2021 • Haofei Xu, Jiaolong Yang, Jianfei Cai, Juyong Zhang, Xin Tong
Optical flow is inherently a 2D search problem, and thus the computational complexity grows quadratically with respect to the search window, making large displacements matching infeasible for high-resolution images.
no code implementations • ICCV 2021 • Jongyoo Kim, Jiaolong Yang, Xin Tong
For face texture completion, previous methods typically use some complete textures captured by multiview imaging systems or 3D scanners for supervised learning.
1 code implementation • CVPR 2021 • Yu Deng, Jiaolong Yang, Xin Tong
We propose a novel Deformed Implicit Field (DIF) representation for modeling 3D shapes of a category and generating dense correspondences among shapes.
4 code implementations • CVPR 2020 • Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, Xin Tong
Our method can also be used to embed real images into the disentangled latent space.
1 code implementation • CVPR 2020 • Sicheng Xu, Jiaolong Yang, Dong Chen, Fang Wen, Yu Deng, Yunde Jia, Xin Tong
We evaluate the accuracy of our method both in 3D and with pose manipulation tasks on 2D images.
1 code implementation • CVPR 2020 • Kaixuan Wei, Ying Fu, Jiaolong Yang, Hua Huang
Lacking rich and realistic data, learned single image denoising algorithms generalize poorly to real raw images that do not resemble the data used for training.
Ranked #5 on Image Denoising on ELD SonyA7S2 x200
1 code implementation • CVPR 2019 • Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, Hua Huang
Removing undesirable reflections from a single image captured through a glass window is of practical importance to visual computing systems.
Ranked #2 on Reflection Removal on SIR^2(Objects)
4 code implementations • 20 Mar 2019 • Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, Xin Tong
Recently, deep learning based 3D face reconstruction methods have shown promising results in both quality and efficiency. However, training deep neural networks typically requires a large volume of data, whereas face images with ground-truth 3D face shapes are scarce.
Ranked #3 on 3D Face Reconstruction on Florence (RMSE Cooperative metric)
1 code implementation • 7 Nov 2018 • Qingnan Fan, Jiaolong Yang, David Wipf, Baoquan Chen, Xin Tong
Image smoothing represents a fundamental component of many disparate computer vision and graphics applications.
1 code implementation • 10 Sep 2018 • Hanqing Wang, Jiaolong Yang, Wei Liang, Xin Tong
The key idea of our method is to leverage object mask and pose estimation from CNNs to assist the 3D shape learning by constructing a probabilistic single-view visual hull inside of the network.
1 code implementation • ECCV 2018 • Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua
Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model.
no code implementations • 14 Aug 2017 • Chen Zhou, Jiaolong Yang, Chunshui Zhao, Gang Hua
This work is devoted to a task that is indispensable for safety yet was largely overlooked in the past -- detecting obstacles that are of very thin structures, such as wires, cables and tree branches.
1 code implementation • ICCV 2017 • Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf
This paper proposes a deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering.
no code implementations • CVPR 2018 • Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf
While invaluable for many computer vision applications, decomposing a natural image into intrinsic reflectance and shading layers represents a challenging, underdetermined inverse problem.
no code implementations • 11 May 2016 • Jiaolong Yang, Hongdong Li, Dylan Campbell, Yunde Jia
The evaluation demonstrates that the proposed method is able to produce reliable registration results regardless of the initialization.
Ranked #6 on Point Cloud Registration on FP-O-H
no code implementations • CVPR 2016 • Jiaolong Yang, Hongdong Li, Yuchao Dai, Robby T. Tan
This paper deals with a challenging, frequently encountered, yet not properly investigated problem in two-frame optical flow estimation.
no code implementations • CVPR 2017 • Jiaolong Yang, Peiran Ren, Dong-Qing Zhang, Dong Chen, Fang Wen, Hongdong Li, Gang Hua
The network takes a face video or face image set of a person with a variable number of face images as its input, and produces a compact, fixed-dimension feature representation for recognition.
Ranked #2 on Face Identification on DroneSURF
no code implementations • CVPR 2015 • Jiaolong Yang, Hongdong Li
This paper proposes a simple method for estimating dense and accurate optical flow field.