Although multimedia compression formats and protocols to stream such content have been around for a long time, there has been limited success in the adoption of open standards for streaming over IP (Internet Protocol) networks. The elements of such an end-to-end system will be introduced outlining the responsibilities of each element. The technical and financial challenges in building a viable multimedia streaming end-to-end system will be analyzed in detail in this paper outlining some solutions and areas for further research. Also, recent migration to IP in the backend video delivery network infrastructures have made it possible to use IP based media streaming solutions in non-IP last mile access networks like cable and wireless networks in addition to the DSL networks. The advantages of using IP streaming solutions in such networks will be outlined. However, there is a different set of challenges posed by such applications. The real time constraints are acute in each element of the media delivery end-to-end system. Meeting these real time constraints in general purpose non real time server systems is quite demanding. Quality of service, resource management, session management, fail-over, reliability, and cost are some important but challenging requirements in such systems. These will also be analyzed with suggested solutions. Content protection and rights management requirements are also very challenging for open standards based multimedia delivery systems. Interoperability unfortunately interferes with security in most of the current day systems. Some approaches to solve the interoperability problems will also be presented. The requirements, challenges, and possible solutions for delivering broadcast, on demand, and interactive video delivery applications for IP based media streaming systems will be analyzed in detail.