rgmv
Reliability of Generative Models in Vision @ WACV 2024
Website for Reliability of Generative Models in Vision Tutorial at WACV 2024, January 8, 1pm to 5pm
Hosted by Changhoon Kim (ASU), Gowthami Somepalli (UMD), Tejas Gokhale (UMBC) and Yezhou Yang (ASU)
Agenda
Over the past few years, generative models have evolved from simple research concepts to production-ready tools, dramatically reshaping the tech landscape. Their outstanding generative capabilities have gained traction in various sectors, such as entertainment, art, journalism, and education. However, a closer look reveals that these models face several reliability issues that can impact their widespread adoption. A primary concern is the models’ ability to memorize training data, which might result in copyright breaches. Reliability concerns also encompass the model’s occasional failure to accurately follow prompts, inherent biases, misrepresentations, and hallucinations. Moreover, with increasing awareness, issues related to privacy and potential misuse underscore the urgent need to safeguard these models. To move forward responsibly with these models, we must adopt solutions to address memorization challenges, robust evaluation systems, and active fingerprinting solutions. These measures will help monitor the progress and ensure responsible and effective use of image-generative models.
In this tutorial, we will emphasize on the issues discussed above and the attendees will get an opportunity to learn about:
- The state-of-the-art image generative models and their associated reliability concerns,
- Training data memorization in diffusion models and some mitigation strategies,
- Techniques for incorporating fingerprint into model weights to trace malicious content origins,
- Methods to quantify and critically assess the inherent biases and potential failure modes within image generative models.
Tentative Schedule
| Time (UTC-10) | Topic | Presenter |
|---|---|---|
| 1300--1310 | Welcome and Introduction |
Yezhou Yang (Associate Professor, ASU |
| 1310--1325 | Recent Advances and Reliability Concerns in Image Generation |
Changhoon Kim (Ph.D. Candidate, ASU) |
| 1325--1355 | Understanding training data memorization in diffusion models and ways to mitigate it |
Gowthami Somepalli (Ph.D. Candidate, UMD) |
| 1355--1425 | Attribution and Fingerprinting of Image Generative Models |
Changhoon Kim (Ph.D. Candidate, ASU) |
| 1425--1455 | Challenges with Evaluation of Text-to-Image Models |
Tejas Gokhale (Assistant Professor, UMBC) |
| 1455--1520 | Invited Talk 1: Characterizing and Mitigating the Misalignment Between the Evaluation of Generative Models and their Intended Use Cases |
Joshua Feinglass (Ph.D. Candidate, ASU) |
| 1520--1545 | Invited Talk 2: Towards Robust Text-to-Image Generative Models in Resource-Efficient Manner |
Maitreya Patel (Ph.D. Student, ASU) |
| 1545--1600 | Concluding Remarks |
Changhoon Kim (Ph.D. Candidate, ASU) |
This website will be updated closer to the event date.
We acknowledge support from NSF Robust Intelligence grant #2132724