Jen-Yuan (Joy) Huang

Hello! I am a graduate student at the School of Intelligence Science and Technology at Peking University, advised by Professor Tong Lin. My research interest lies in computer vision and generative models. I am a member of InstantX Team, where I have the opportunity to develop fancy AI-generated content (AIGC) models. Currently, I am visiting student scholar at Westlake University, hosted by Professor Huan Wang. I also work closely with Professor Yilun Du at Harvard University.

I received my Bachelor's degree in Industrial Engineering from Nanjing University, with a specialization in optimization and operation research. During my undergraduate studies, I had the opportunity to work closely with Professor Xiaopeng Luo.

Publications

InstantIR: Blind Image Restoration with Instant Generative Reference

InstantIR is a novel blind image restoration model based on Stable Diffusion. At each denoising step, we generate a restoration reference from current diffusion latent, which is used to align the low-quality image encoding and condition the generation process.

Remedy Text-to-image Diffusion Model Compression via Concept Knowledge Distillation

We identify the performance degradation in textual-inversion of T2I diffusion models after compression. By reformuolating the concept discovery process like textual-inversion as an implicit text generation model’s inference, we propose to a knowledge-distillation based retraining method via aligning gradients with respect to text condition.

CSGO: Content-Style Composition in Text-to-Image Generation

In this work, we devlop an image stylization model named CSGO, which transfers the style presented by an input reference image to a source image. To enable end-to-end training, we introduce an automatic construction pipeline and IMAGStyle, first large-scale style transfer dataset with 210K {content;style;target}-triplet.

InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generation

In this paper, we explore natural style transfer while maintaining content integrity. Through analyzing different components of the Stable Diffusion UNet, we identify layers that specialize in processing style and content. Furthermore, we introduce a style discriminator to enhance the stylization of the output.

Contact

E-mail: jenyuan@stu.pku.edu.cn
Address: Peking University, Beijing, China


Design and source code from: Jon Barron's website