Relational thought is the ability to understand that defining object properties might not lie in the object itself, but in its relations to other objects. This ability is highly relevant for many important human cognitive domains such as reasoning, categorization and understanding analogy and metaphor. It has also been reported that linguistic labels can boost children's understanding of spatial relations between objects and hence language might be a crucial component of relational thought. I will present experiments investigating relational thought across five species of great apes, including human children of two different linguistic and cultural backgrounds. Based on these data I will outline a methodological approach to studying cognitive structure in extinct species by mapping the observed patterns of behaviour to a phylogenetic tree of relatedness. In summary, I will argue that despite a clear inherited general capacity for relational thought and a bias for certain spatial relational strategies over others in the great apes, the structure of both child and adult spatial cognition systematically varies with language and culture.