The article presents a method to improve regression estimates by combining multiple estimators into a hybrid estimator. This method efficiently uses all available data without overfitting, avoids local minima, and can be easily parallelized. The approach is based on the idea of convexity and can be applied to various optimization algorithms. Experimental results show that this ensemble method significantly enhances regression performance in real-world tasks like classification and time series prediction.