This talk addresses the problem of understanding the visual content of images and videos using weak forms of supervision, such as the fact that multiple images contain instances of the same objects, or the textual information available in television or film scripts. I will discuss several instances of this problem, including image cosegmentation, the joint localization and identification of movie characters and their actions, and the assignment of action labels to video frames using temporal ordering constraints. I will present the underlying discriminative clustering model, appropriate relaxations of the combinatorial optimization problems associated with learning its parameters, and efficient algorithms for solving the corresponding convex optimization problems. I will also present experimental results on standard image benchmarks and feature-length films. I will conclude with a brief discussion of our recent work on fully unsupervised object discovery in photographs and videos.