Some concepts for candidate future “flagship” space observatories approach the payload limits of the largest launch vehicles planned for the next few decades, specifically in the available volume in the vehicle fairing. This indicates that an alternative to autonomous self-deployment similar to that of the James Webb Space Telescope will eventually be required. Moreover, even before this size limit is reached, there will be significant motivation to service, repair, and upgrade in-space missions of all sizes, whether to extend the life of expensive facilities or to replace outworn or obsolete onboard systems as was demonstrated so effectively by the Hubble Space Telescope program. In parallel with these challenges to future major space astronomy missions, the capabilities of in-space robotic systems and the goals for human space flight in the 2020s and 2030s offer opportunities for achieving the most exciting science goals of the early 21st Century. In this paper, we summarize the history of concepts for human operations beyond the immediate vicinity of the Earth, the importance of very large apertures for scientific discovery, and current capabilities and future developments in robot- and astronaut-enabled servicing and assembly.