Image-guided topic modeling for interpretable privacy classification
| Type of publication: | Conference paper |
| Citation: | Baia_ECCVW_2024 |
| Publication status: | Accepted |
| Booktitle: | Proceedings of the European Conference on Computer Vision (ECCV) Workshops |
| Year: | 2024 |
| Abstract: | Predicting and explaining the private information contained in an image in human-understandable terms is a complex and contextual task. This task is challenging even for large language models. To facilitate the understanding of privacy decisions, we propose to predict image privacy based on a set of natural language content descriptors. These content descriptors are associated with privacy scores that reflect how people perceive image content. We generate descriptors with our novel Image-guided Topic Modeling (ITM) approach. ITM leverages, via multimodality alignment, both vision information and image textual descriptions from a vision language model. We use the ITM-generated descriptors to learn a privacy predictor, Priv×ITM, whose decisions are interpretable by design. Our Priv×ITM, classifier outperforms the reference interpretable method by 5 percentage points in accuracy and performs comparably to the current non-interpretable state-of-the-art model. |
| Keywords: | Interpretability, topic modeling, Vision language models |
| Projects |
Idiap |
| Authors | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
|
|
Notes
|
|
|
|
|
