Aniruddha Mahapatra

I am a sophomore year Masters in Computer Vision (MSCV) student at Carnegie Mellon University. I am advised by Prof. Jun-Yan Zhu. My research interest include computer vision and deep learning, specifically, image and video synthesis and editing using generative models. My goal is to create automated AI algorithms for generating (or editing) photorealistic images and videos which are very time-consuming, or otherwise impossible to do manually by existing tools.

I am fortunate to have worked with Aliaksandr Siarohin, Hsin-Ying Lee and Sergey Tulyakov at Snap Research, Kuldeep Kulkarni, Anandhavelu Natarajan and Subrata Mitra at Adobe Research, Jitendra Singh at IBM Research, and, Professor Biplab Banerjee and Ranita Biswas at Indian Institute of Technology, Roorkee.

Email  |  Resume  |  Google Scholar  |  LinkedIn

CMU
MS in Computer Vision
Aug. 22 - Dec. 23

Snap
Research Intern
May. 23 - Aug. 23

Adobe
Research Associate
Aug. 20 - Present

Adobe
Research Intern
May. 19 - Jul. 19

IBM
Remote Research Intern
Jun. 18 - Dec. 18

IIT Roorkee
B.Tech. Computer Science
Jul. 16 - Jul. 20

Adobe's Photoshop Elements 2023 new feature "Moving Elements" lets users generate aesthetic cinemagraphs from their photos.
This feature was based on our Controllable Animation of Fluid Elements in Still Images (CVPR 2022) work.

We introduce a fully automated method, Text2Cinemagraph, for creating cinemagraphs from text descriptions - an especially challenging task when prompts feature imaginary elements and artistic styles, given the complexity of interpreting the semantics and motions of these images. Our method also gives the user a coarse control over the direction of motion in the generated cinemagraphs using text-based direction.

Given a single input image, mask the region user wants to animate and any number of arrow directions and their associated speeds provided by the user to specify the direction of desired movement, we propose a method to interactively control the animation of fluid elements (like water, fire, clouds, etc.) to generate cinemagraphs from the single image.

We design an auto-regressive model, GEMS, for a novel task of conditional expansion of scene graphs from a given seed scene graph that adds nodes and edges hierarchically to the seed scene graph. We also propose novel metrics to evaluate the quality of expanded scene graphs to capture the coherence of predicted edges and nodes better than traditional MMD based metrics.

We introduce effective ways of dataset selection for pretraining large language models in an unsupervised way to facilitate domian adaptation to very limited data domains.

We propose a semantically-aware novel paradigm to perform image extrapolation that enables the addition of new object instances.

We prose an end-to-end trainable neural network-based unsupervised DA module for RS image classification that learns a shared embedding space for both the domains which are also deemed to be discriminative by jointly optimizing the contrastive loss and minimizing the difference of the two domain higher-order statistics.

We evaluate the potential of Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 optical imagery in crop classification for an Indian region using multi-class classification algorithm based on the support vector machine (SVM) by applying it to the temporal features extracted from the two imagery data.

• [Jul/2023] Paper "Text-Guided Synthesis of Eulerian Cinemagraphs” accepted at SIGGRAPH Asia 2023 (TOG)!
• [May/2023] Started Research Internship at Snap Research
• [Oct/2022] Paper “GEMS: Scene Expansion using Generative Models of Graphs” accepted at WACV 2023!
• [Oct/2022] Paper “Entity Extraction in Low Resource Domains with Selective Pre-training of Large Language Models” accepted at EMNLP 2022 (Oral)!
• [Sept/2022] Joined Generative Intelligence Lab (Jun-Yan Zhu's) group at CMU as Research Assistant.
• [Aug/2022] Started my Masters in Computer Vision (MSCV) program at Carnegie Mellon University (CMU).
• [Mar/2022] Paper “Controllable Animation of Fluid Elements in Still Images” accepted at CVPR 2022!
• [Feb/2022] Promoted to Research Associate 2 at Adobe Research, India!
• [Jun/2021] Paper “SemIE: Semantically-aware Image Extrapolation” accepted at ICCV 2021!
• [Aug/2020] Joined Adobe Research, India as Research Associate!
• [Jul/2020] Completed Bachelors in Computer Scinece from IIT Roorkee.
• [Oct/2019] Paper “Unsupervised Domain Adaptation for Remote Sensing Images Using Metric Learning and Correlation Alignment” accepted (Oral) at NCVPRIPG 2019!
• [May/2019] Started Research Internship at Adobe Research, India.
• [Apr/2019] Paper “Assessment of Sentinel-1 and Sentinel-2 Satellite Imagery for Crop Classification in Indian Region During Kharif and Rabi Crop Cycles” accepted at IGARSS 2019!

• Class Rank 2 | Ranked 2nd out of 76 students in IIT Roorkee B.Tech. Computer Science batch of 2020.
• KVPY Fellowship | National Fellowship awarded by Indian Institute of Science and Govt. of INDIA.

Source code from Jon Barron