Pixels to Graphs by Associative Embedding: https://arxiv.org/abs/1706.07365v1
We show how to supervise a network such that all of the reasoning about a graph can be abstracted away into a single network. The use of associative embeddings and unordered output slots offer the network the flexibility necessary to making training of this task possible. Our results on Visual Genome clearly demonstrate the effectiveness of our approach.