A small toolkit of morphogens is used repeatedly to direct development, raising the question of how context dictates interpretation of the same cue. One example is the transforming growth factor β (TGF-β) pathway that in human embryonic stem cells fulfills two opposite functions: pluripotency maintenance and mesendoderm (ME) specification. Using proteomics coupled to analysis of genome occupancy, we uncover a regulatory complex composed of transcriptional effectors of the Hippo pathway (TAZ/YAP/TEAD), the TGF-β pathway (SMAD2/3), and the pluripotency regulator OCT4 (TSO). TSO collaborates with NuRD repressor complexes to buffer pluripotency gene expression while suppressing ME genes. Importantly, the SMAD DNA binding partner FOXH1, a major specifier of ME, is found near TSO elements, and upon fate specification we show that TSO is disrupted with subsequent SMAD-FOXH1 induction of ME. These studies define switch-enhancer elements and provide a framework to understand how cellular context dictates interpretation of the same morphogen signal in development.