The idea of ensemble learning can be used to solve problems about privacy preserving distributed data mining conveniently. Owners of distributed datasets can get an integrated model securely just by sharing and combining their sub models which are built on their respective sample sets, and generally the integrated model is more powerful than any sub model. However, sharing the sub models may cause serious privacy problems in some cases. So in this paper, we present a new method, based on which the data holders can integrate their sub polynomial regression models securely and efficiently without sharing them, and get the optimal combination regression model. In addition to theoretical analysis, we also verify the availability of the new method through experiments.