Tae-Hyun Oh @ MIT

Information about computer vision and machine learning academic field

- Top-tier conferences

: CVPR, ICCV, ECCV, NIPS, ICML, and ICLR are considered high prestigious top-tier conferences, which have greater impact than most SCI journals. According to Google scholar metrics, all these conferences are listed in the top 100 publications across all academic fields. Out of them, CVPR is the 10th rank among all academic fields, e.g., Cell journal is just the 9th rank. In terms of acceptance rate, oral presentations are about 4% and poster presentations about 20%, i.e., highly competitive.

- Top-tier journals

: IEEE TPAMI and IJCV have among the highest impact factors across all computer science categories. As of 2019, the impact factor of TPAMI is 17.730.

Selected publications (International Journal)

~~Dense Relational Image Captioning via Multi-task Triple-Stream Networks~~

Dong-Jin Kim, Tae-Hyun Oh, Jinsoo Choi, In So Kweon

~~Under review~~

~~Robust and Efficient Relative Pose Estimation for Camera on a Selfie Stick~~

Kyungdon Joo, Hongdong Li, Tae-Hyun Oh, In So Kweon

~~IEEE TPAMI, under major revision review~~

~~Learning to Localize Sound Sources in Visual Scenes: Analysis and Applications~~

Arda Senocak, Tae-Hyun Oh, Junsik Kim, Ming-Hsuan Yang, In So Kweon

~~IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), to appear.~~

[Dataset] [Code(PyTorch)] [Featured by Seamless]

Qualcomm Innovation Paper Award 2018 by Qualcomm Korea R&D center

~~Globally Optimal Inlier Set Maximization for Atlanta World Understanding~~

Kyungdon Joo, Tae-Hyun Oh, In So Kweon, Jean-Charles Bazin

~~IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020~~

[Project page] [IEEE Xplore]

~~Gradient-based Camera Exposure Control for Outdoor Mobile Platforms~~

Inwook Shim, Tae-Hyun Oh, Joon-Young Lee, Dong-Geol Choi, Jinwook Choi, In So Kweon

~~IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2019~~

[Project Page] [IEEE Xplore] [arXiv]

~~Robust and Globally Optimal Manhattan Frame Estimation in Near Real Time~~

Kyungdon Joo, Tae-Hyun Oh, Junsik Kim, In So Kweon

~~IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019~~

[Code and Project page] [arXiv] [IEEE Xplore]

~~High-fidelity Depth Upsampling using Self-learning Framework~~

Inwook Shim, Tae-Hyun Oh, In So Kweon

~~Sensors, 2019~~

[PDF] [Video1] [Video2] [Video3]

~~A Closed-Form Solution to Rotation Estimation for Structure from Small Motion~~

Hyowon Ha, Tae-Hyun Oh, In So Kweon

~~IEEE Signal Processing Letters, 2018.~~

[IEEE Xplore]

~~Fast Randomized Singular Value Thresholding for Low-rank Optimization~~

Tae-Hyun Oh, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon

~~IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.~~

This conference version received Gold Prize (Acceptance rate 0.8%), 21th HumanTech Paper Award by Samsung.

[Project page] [arXiv]

~~Partial Sum Minimization of Singular Values in Robust PCA: Algorithm and Applications~~

Tae-Hyun Oh, Yu-Wing Tai, Jean-Chales Bazin, Hyeongwoo Kim, In So Kweon

~~IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.~~

[Project page] [arXiv]

~~Robust High Dynamic Range Imaging by Rank Minimization~~

Tae-Hyun Oh, Joon-Young Lee, Yu-Wing Tai, In So Kweon

~~IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2015.~~

[Project page] [Code] [IEEE Xplore]

~~An Autonomous Driving System for Unknown Environments using a Unified Map~~

Inwook Shim, Jongwon Choi, Seunghak Shin, Tae-Hyun Oh, Unghui Lee, Byungtae Ahn, Dong-Geol Choi, David Hyunchul Shim, In So Kweon

~~IEEE Transactions on Intelligence Transportation Systems (TITS), 2015.~~

Qualcomm Innovation Award 2013.

With this system, we won Youl-Jeong award (5th rank) from the autonomous vehicle challenge 2012 by Hyundai Motors, Korea.

[Video]

~~New Design Criteria for Robust PCA and a Compliant Bayesian-Inspired Algorithm~~

Tae-Hyun Oh, David Wipf, Yasuyuki Matsushita, In So Kweon

~~Preprint (arxiv)~~

[arXiv]

~~Human Attention Estimation for Natural Images: An Automatic Gaze Refinement Approach~~

Jinsoo Choi, Tae-Hyun Oh, In So Kweon

~~Preprint (arxiv)~~

~~Robust Low-rank Optimization with Priors~~

Tae-Hyun Oh

~~Doctoral dissertation, KAIST, Aug., 2017~~

~~A Novel Low-Rank Constraint Method with the Sparsity Model for Moving Object Analysis~~

Tae-Hyun Oh

~~Master Thesis, KAIST, Aug., 2012~~

[Project page]

Please reload

Selected publications (International Conference)

~~Supervoxel Attention Graphs for Long-Range Video Modeling~~

Yang Wang, Gedas Bertasius, Tae-Hyun Oh, Abhinav Gupta, Minh Hoai Nguyen, Lorenzo Torresani

~~Winter Conference on Applications of Computer Vision 2021~~

~~MDARTS: Multi-objective Differentiable Neural Architecture Search~~

Sunghoon Kim, Hyunjeong Kwon, Eunji Kwon, Youngchang Choi, Tae-Hyun Oh, Seokhyeong Kang

~~Design, Automation, and Test in Europe (DATE), 2021~~

~~Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels~~

Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A. Plummer, Stan Sclaroff, Kate Saenko

~~arXiv, 2020~~

[PDF]

~~Monocular Reconstruction of Neural Face Reflectance Fields~~

Mallikarjun B R, Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt

~~arXiv, 2020~~

[PDF] [Project page]

~~Listen to Look: Action Recognition by Previewing Audio~~

Ruohan Gao, Tae-Hyun Oh, Kristen Grauman, Lorenzo Torresani

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2020.~~

[PDF] [Project page]

~~Globally Optimal Relative Pose Estimation for Camera on a Selfie Stick~~

Kyungdon Joo, Hongdong Li, Tae-Hyun Oh, Yunsu Bok, In So Kweon

~~International Conference on Robotics and Automation (ICRA), 2020.~~

~~Linear RGB-D SLAM for Atlanta World~~

Kyungdon Joo, Tae-Hyun Oh, Francois Rameau, Jean-Charles Bazin, In So Kweon

~~International Conference on Robotics and Automation (ICRA), 2020.~~

~~Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach~~

Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh, In So Kweon

~~Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019 (Long paper)~~

[PDF]

Also, presented at

"Language and Vision Workshop",

"Visual Question Answering and Dialog Workshop" in conjunction with CVPR, 2019, and

"CLVL: 3rd Workshop on Closing the Loop Between Vision and Language" in conjunction with ICCV, 2019.

~~Visuomotor Understanding for Representation Learning of Driving Scenes~~

Seokju Lee, Junsik Kim, Tae-Hyun Oh, Yongseop Jeong, Donggeun Yoo, Stephen Lin, In So Kweon

~~British Machine Vision Conference (BMVC), 2019~~

[PDF] [Project page] [Dataset]

~~Neural Inverse Knitting: From Images to Manufacturing Instructions~~

Alexandre Kaspar*, Tae-Hyun Oh*, Liane Makatura, Petr Kellnhofer, Jacqueline Aslarus, Wojciech Matusik

(* Equally contributed)

~~International Conference on Machine Learning (ICML), Jun., 2019~~

This has been covered by more than 20 media including BBC News, Fortune, Engadget, ZDNet, TechCrunch, Geek, and MIT News. Also, this work was posted on the front page of the MIT CSAIL web page as a representative illustration of AI group.

[PDF] [Project page]

~~Speech2Face: Learning the Face Behind a Voice~~

Tae-Hyun Oh*, Tali Dekel*, Changil Kim*, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Wojciech Matusik

(* Equally contributed)

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2019.~~

This has been covered by lots of media. You can google it.

[PDF] [Project page]

Speech2Face synthesizes someone’s face image from hearing their speech. We train it with 2 millions of video clips with near 100,000 different people's faces.

The work is an effort to better understand the capabilities of machine perception, i.e., the speech-face association.
When we hear a voice on the radio or the phone call, we, human, often build a mental model to imagine how the person looks. Our work can be considered as a replication of a human mental model by machine. For the Speech2Face task, we rarely understood how strongly we human can parse and whether it is indeed correct or just noisy bias. The reconstructed face by Speech2Face could be used as a proxy to study these.

We can imagine a range of applications, including for privacy-minded people who want to share real photos of themselves off the internet or video calls.

~~Variational Prototyping-Encoder: One-Shot Learning with Prototypical Images~~

Junsik Kim, Tae-Hyun Oh, Seokju Lee, Fei Pan, In So Kweon

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2019.~~

[PDF] [Project page (code)]

~~Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioning~~

Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh, In So Kweon

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2019.~~

[PDF] [Project page] [Dataset] [Evaluation code (coming soon)]

Qualcomm Innovation Paper Award 2019 by Qualcomm Korea R&D center

Also presented at

Language and Vision Workshop in conjunction with CVPR, 2019, and

Visual Question Answering and Dialog Workshop in conjunction with CVPR, 2019

~~Noise-Tolerant Audio-Visual Online Person Verification using an Attention-based Neural Network Fusion~~

Suwon Shon, Tae-Hyun Oh, James Glass

~~International Conference on Acoustics, Speech, and Signal Processing (ICASSP), May, 2019~~

[PDF]

In this work, an attention mechanism based fusion method is applied for the multi-modal person verification task. The fusion method adaptively combines person speech and visual face information on a feature-level. We demonstrate that the attention implicitly weighs either modality according to their quality and distinctiveness; thus, it is tolerant against missing modality and outlier. Interestingly, even when a single modal input is given, our multi-modal network performs favorably against the networks trained only with either of modalities. This behavior evidently is similar to the multisensory representation used in human's person recognition capability [Bülthoff and Newell, Distinctive voices enhance the visual recognition of unfamiliar faces, Cognition'15].

~~On Learning Associations of Faces and Voices~~

Changil Kim, Hijung Valentina Shin, Tae-Hyun Oh, Alexandre Kaspar, Mohamed Elgharib, Wojciech Matusik

~~Asian Conference on Computer Vision (ACCV), Dec., 2018.~~

[PDF] [Project page]

If you see a picture of someone, can you anticipate their voice? If you hear someone's voice, can you guess what they look like?

This work conducted both human and machine experiments to see their capability of association between voice and face. Our experiments show that the human indeed has such capability and machines can have a similar capability like the human, opening many questions about how physical appearance and voice are correlated.

~~Learning-based Video Motion Magnification~~

Tae-Hyun Oh*, Ronnachai Jaroensri*, Changil Kim, Mohamed Elgharib,

Frédo Durand, William T. Freeman, Wojciech Matusik

(* Equally contributed)

~~European Conference on Computer Vision (ECCV), Sep., 2018.~~

Accepted as a full oral paper (2.3% acceptance rate)

[Project page] [Video results] [PDF] [Oral presentation]

Video motion magnification is a technique that magnifies subtle motion almost invisible by human eyes in video so that we can clearly observe it. This work presents the first learning-based method for video motion magnification. We show that physically plausible motion representation can be learned by deep neural networks only with synthetic data.

~~Semantic Soft Segmentation~~

Yağız Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys, Wojciech Matusik

~~ACM Transactions on Graphics (ACM SIGGRAPH), 2018~~

Selected as Video Trailer

This has been covered by more than 17 media including BBC news, MIT CSAIL news, Nvidia news, and Digital Trend. Refer to the project page for the detail media coverage.

[PDF] [Project page] [Video]

This work proposed a new concept, an automatic semantic soft segmentation, which provides semantically meaningful and accurate soft transitions between different object regions. It can enhance image editing and computational imaging applications.

~~Part-based Player Identification using Deep Convolutional Representation and Multi-scale Pooling~~

Arda Senocak, Tae-Hyun Oh, Junsik Kim, In So Kweon

~~In CVSports workshop in conjunction with CVPR, Jun., 2018~~

Selected as Oral paper.

~~Globally Optimal Inlier Set Maximization for Atlanta Frame Estimation~~

Kyungdon Joo, Tae-Hyun Oh, In So Kweon, Jean-Charles Bazin

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2018.~~

[Project page]

~~Learning to Localize Sound Source in Visual Scenes~~

Arda Senocak, Tae-Hyun Oh, Junsik Kim, Ming-Hsuan Yang, In So Kweon

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2018.~~

[Dataset] [Code(PyTorch)] [Featured by Seamless]

Qualcomm Innovation Paper Award 2018 by Qualcomm Korea R&D center

Presented in Sight & Sound Workshop (Oral) and VisionMeetsCognition Workshop (Oral, invited) in conjunction with CVPR 2018.

~~Contextually Customized Video Summaries via Natural Language~~

Jinsoo Choi, Tae-Hyun Oh, In So Kweon

~~IEEE Winter Conference on Applications of Computer Vision (WAVC), 2018.~~

The conference name of 'IEEE Workshop on Applications of Computer Vision (WACV)' is changed to 'Winter'.

[PDF]

~~Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks~~

Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh, Youngjin Yoon, In So Kweon

~~IEEE Winter Conference on Applications of Computer Vision (WAVC), 2018.~~

The conference name of 'IEEE Workshop on Applications of Computer Vision (WACV)' is changed to 'Winter'.

[PDF]

~~Co-domain Embedding using Deep Quadruplet Network for Unseen Traffic Sign Recognition~~

Junsik Kim , Seokju Lee , Tae-Hyun Oh , In So Kweon

~~AAAI Conference on Artificial Intelligence (AAAI), 2018.~~

Best Poster Award from Image Processing and Image Understanding Workshop (IPIU), Korea.

[PDF]

~~Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning~~

{Tae-Hyun Oh, Kyungdon Joo}*, Neel Joshi, Baoyuan Wang, In So Kweon, Sing Bing Kang

(*Equally contributed)

~~IEEE International Conference on Computer Vision, (ICCV), 2017.~~

Best Poster Presentation Award in IWRCV 2018.

[PDF] [Project Page]

~~Weakly- and Self-Supervised Learning for Content-aware Deep Image Retargeting~~

Donghyeon Cho, Jinsun Park, Tae-Hyun Oh, Yu-Wing Tai, In So Kweon

~~IEEE International Conference on Computer Vision, (ICCV), 2017.~~

Selected as Spotlight (Acceptance rate 2.61%)

[PDF]

~~A Pseudo-Bayesian Algorithm for Robust PCA~~

Tae-Hyun Oh, David Wipf, Yasuyuki Matsushita, In So Kweon

~~Neural Information Processing Systems (NIPS), Dec., 2016~~

[PDF]

~~Video-Story Composition via Plot Analysis~~

Jinsoo Choi, Tae-Hyun Oh, In So Kweon

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2016.~~

Selected as Spotlight (Accept rate 9.7%)

[PDF] [Presentation in Las Vegas] [Dataset]

~~Globally Optimal Manhattan Frame Estimation in Real-time~~

{Tae-Hyun Oh, Kyungdon Joo}*, Junsik Kim, In So Kweon (*Equally contributed)

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2016.~~

[Code and Project page]

~~A Multi-View Structured-Light System for Highly Accurate 3D Modeling~~

Hyowon Ha, Tae-Hyun Oh, In So Kweon

~~International Conference on 3D Vision (3DV), Oct., 2015.~~

~~Line Meets As-Projective-As-Possible Image Stitching With Moving DLT~~

Kyungdon Joo, Namil Kim, Tae-Hyun Oh, In So Kweon

~~IEEE International Conference on Image Processing (ICIP), Sep., 2015.~~

Selected as the Top 10% paper in ICIP 2015.

~~Fast Randomized Singular Value Thresholding for Nuclear Norm Minimization~~

Tae-Hyun Oh, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon

~~IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun., 2015~~

Received Gold Prize (Acceptance rate 0.8%), 21th HumanTech Paper Award by Samsung.

[Project page]

~~A Two Phase Approach for Pedestrian Detection~~

SoonMin Hwang, Tae-Hyun Oh, In So Kweon

~~Workshop in conjunction with the 12th Asian Conference on Computer Vision (ACCVW), 2014~~

~~Cost-Aware Depth Map Estimation for Lytro Camera~~

Min Jung Kim, Tae-Hyun Oh, In So Kweon

~~IEEE International Conference on Image Processing (ICIP), Oct., 2014~~

~~Balanced Optical Flow Refinement by Bidirectional Constraint~~

Jongwon Choi, Hyeongwoo Kim, Tae-Hyun Oh, In So Kweon

~~IEEE International Conference on Image Processing (ICIP), Oct., 2014~~

~~Partial Sum Minimization of Singular Values in RPCA for Low-Level Vision~~

Tae-Hyun Oh, Hyeongwoo Kim, Yu-Wing Tai, Jean-Chales Bazin, In So Kweon

~~IEEE International Conference on Computer Vision, (ICCV), Dec., 2013.~~

[Project page]

~~High Dynamic Range Imaging by a Rank-1 Constraint~~

Tae-Hyun Oh, Joon-Young Lee, In So Kweon

~~IEEE International Conference on Image Processing (ICIP), Sep., 2013~~

[Project page] [Code]

~~Hierarchical 3D Line Restoration based on Angular Proximity in Structured Environments~~

Kyungdon Joo, Tae-Hyun Oh, Hyeongwoo Kim, In So Kweon

~~IEEE International Conference on Image Processing (ICIP), Sep., 2013~~

~~Autonomous Homing based on Laser-Camera Fusion System~~

Dong-Geol Choi, Inwook Shim, Yunsu Bok, Tae-Hyun Oh, In So Kweon

~~IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Oct., 2012~~

~~A Tensor Voting Approach for Multi-View 3D Scene Flow Estimation and Refinement~~

Jaesik Park, Tae-Hyun Oh, Jiyoung Jung, Yu-Wing Tai, In So Kweon

~~The 12th European Conference on Computer Vision (ECCV) , Oct., 2012.~~

~~Real-Time Motion Detection based on Discrete Cosine Transform~~

Tae-Hyun Oh, Joon-Young Lee, In So Kweon

~~IEEE International Conference on Image Processing (ICIP), Sep., 2012.~~

Please reload

Other International Conference

Geometry

~~3D Vehicle Localization in Atlanta World~~

Kyungdon Joo, Tae-Hyun Oh, In So Kweon

~~The International Workshop on Frontiers of Computer Vision (IWFCV), 2019~~

Human Pose

~~Human Body Part Classification from Optical Flow~~

Junsik Kim, Kyungdon Joo, Tae-Hyun Oh, In So Kweon

~~The 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2016~~

Geometry

~~Line Assisted Vision Applications in Structured Environments~~

Kyungdon Joo, Tae-Hyun Oh, In So Kweon

~~The 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2015~~

Visual Tracking

~~Robust Pedestrian Tracking by Multi-Person Tracking~~

Kyungdon Joo, Junsik Kim, Tae-Hyun Oh, Jaesik Park, In So Kweon

~~The 9th International Workshop on Robust Computer Vision (IWRCV), Dec., 2014~~

Detection

~~A Cascade Framework for Pedestrian Detection~~

SoonMin Hwang, Tae-Hyun Oh, In So Kweon

~~The 9th International Workshop on Robust Computer Vision (IWRCV), Dec., 2014~~

Visual Tracking

~~A Fusion Approach for Robust Visual Object Tracking in Crowd Scenes~~

Tae-Hyun Oh, Kyungdon Joo, Junsik Kim, Jaesik Park, In So Kweon

~~The 11th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2014~~

Detection

~~A Simple and Real-Time Moving Object Detection Invariant to Cast Shadow~~

Tae-Hyun Oh, In So Kweon

~~The 11th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2014~~

Geometry

~~Single-camera based Vehicle Pose Estimation using Multiple Features on the Road Surface~~

Jongwon Choi, Tae-Hyun Oh, In So Kweon