GAN Dissection: Visualizing and Understanding Generative Adversarial Networks GAN剖析：可视化与理解生成对抗网络
Published:
Paper reading note: GAN Dissection: Visualizing and Understanding Generative Adversarial Networks, Bau et al., arXiv’18
Code available on GitHub, demo at project page.
We’re interested in image-generating GANs. The key question:
“Trained GANs appear to learn about objects in images: e.g., a door appears on a building rather than a tree. Do GANs contain internal variables that are related to human-understandable object concepts like doors or trees? If so, do these variables cause the generation of those objects, or are they merely correlated?”
The study examines three Progressive GANs trained on LSUN scene datasets. The methodology combines dissection and intervention:
- A trained segmentation model (trained on ADE20k, 336 object classes, 29 categories, 25 materials) is used to dissect intermediate GAN layers, identifying which units correlate with which object categories
- Ablation: Remove units to see if associated objects disappear
- Activation: Force units on to see if objects appear where they didn’t before
The same insights enable human-guided model improvement — by identifying units responsible for unwanted objects, we can remove those objects from generated images by disabling those units.
论文解读：GAN Dissection: Visualizing and Understanding Generative Adversarial Networks, Bau et al., arXiv’18
我们感兴趣的是生成图片的GANs。核心问题是：
“训练好的GAN看起来能够学习图片上的物品：例如，一个门出现在建筑上而不是树上。GAN是否包含了与人类理解的物体概念（如门或树）相关的内在变量？如果有，这些变量是否导致了这些物体的生成，还是仅仅相关？”
研究基于在LSUN场景数据集上训练的三个Progressive GANs。方法结合了分解（dissection）和干预（intervention）：
- 使用训练好的分割模型（ADE20k上训练，336物品类别、29大类、25材料）分解GAN中间层，识别单元与物品类别的对应关系
- 消除：去掉相关单元，观察物品是否消失
- 激活：强制打开单元，观察物品是否在新位置出现
相同的洞察可用于人工指导的模型提升——通过识别导致不需要物品生成的单元，关闭这些单元即可从生成图片中移除不需要的物品。