AN ADAPTIVE METHOD FOR BUILDING A MULTIVARIATE REGRESSION
DOI:
https://doi.org/10.20998/2079-0023.2024.01.01Keywords:
multivariate regression, integral measure, adaptive algorithm, regression analysis, expert coefficients, linear programmingAbstract
We propose an adaptive method for building a multivariate regression given by a weighted linear convolution of known scalar functions of deterministic input variables with unknown coefficients. As, for example, when multivariate regression is given by a multivariate polynomial. In contrast to the general procedure of the least squares method that minimizes only a single scalar quantitative measure, the adaptive method uses six different quantitative measures and represents a systemically connected set of different algorithms which allow each applied problem to be solved on their basis by an individual adaptive algorithm that, in the case of an active experiment, even for a relatively small volume of experimental data, implements a strategy of a statistically justified solving. The small amount of data of the active experiment we use in the sense that, for such an amount, the variances of estimates of unknown coefficients obtained by the general procedure of the least squares method do not allow to guarantee the accuracy acceptable for practice. We also proposed to significantly increase the efficiency of the proposed by O. A. Pavlov. and M. M. Holovchenko modified group method of data handling for building a multivariate regression which is linear with respect to unknown coefficients and given by a redundant representation. We improve it by including some criteria and algorithms of the adaptive method for building a multivariate regression. For the multivariate polynomial regression problem, the inclusion of a partial case of the new version of the modified group method of data handling in the synthetic method proposed by O. A. Pavlov, M. M. Golovchenko, and V. V. Drozd, for building a multivariate polynomial regression given by a redundant representation, also significantly increases its efficiency.
References
Pavlov A. A., Holovchenko M. N., Drozd V. V. Efficiency substantiation for a synthetical method of constructing a multivariate polynomial regression given by a redundant representation. Visnyk Nats. tekhn. un-tu "KhPI": zb. nauk. pr. Temat. vyp.: Systemnyy analiz, upravlinnya ta informatsiyni tekhnologiyi [Bulletin of the National Technical University "KhPI": a collection of scientific papers. Thematic issue: System analysis, management and information technology]. Kharkov, NTU "KhPI" Publ., 2023, no. 1 (9), P. 3–9. DOI: 10.20998/2079-0023.2023.01.01.
Pavlov A. A., Holovchenko M. N. Modified method of constructing a multivariate linear regression given by a redundant description. Visnyk Nats. tekhn. un-tu "KhPI": zb. nauk. pr. Temat. vyp.: Systemnyy analiz, upravlinnya ta informatsiyni tekhnologiyi [Bulletin of the National Technical University "KhPI": a collection of scientific papers. Thematic issue: System analysis, management and information technology]. Kharkov, NTU "KhPI" Publ., 2022, no. 2 (8), P. 3–8. DOI: 10.20998/2079-0023.2022.02.01.
Pavlov A., Holovchenko M., Mukha I. et al. A modified method and an architecture of a software for a multivariate polynomial regression building based on the results of a conditional active experiment. Lecture Notes on Data Engineering and Communications Technologies. 2023. Vol. 181. P. 207–222. DOI: 10.1007/978-3-031-36118-0_19.
Abdulrahman A. T., Alshammari N. S. Factor analysis and regression analysis to find out the influencing factors that led to the countries’ debt crisis. Advances and Applications in Statistics. 2022, vol. 78, pp. 1–16. DOI: 10.17654/0972361722047.
Flitman A. M. Towards analysing student failures: neural networks compared with regression analysis and multiple discriminant analysis. Computers and Operations Research. 1997, vol. 24, no. 4, pp. 367–377. DOI: 10.1016/s0305-0548(96)00060-3.
Johnson R. A., Wichern D. W. Applied multivariate statistical analysis, 5th edn. Upper Saddle River, Prentice-Hall, 2002. 767 p.
Knowles D., Parts L., Glass D., Winn J. M. Modeling skin and ageing phenotypes using latent variable models in Infer.NET. Predictive models in personalized medicine workshop, NIPS 2010. Available at: https://www.researchgate.net/publication/241194775 (accessed 18.05.2024).
Lio W., Liu B. Uncertain maximum likelihood estimation with application to uncertain regression analysis. Soft Computing. 2020, vol. 24, no. 13, pp. 9351–9360. DOI: 10.1007/s00500-020-04951-3.
Liu S.S., Zhu Y. Simultaneous maximum likelihood estimation for piecewise linear instrumental variable models. Entropy. 2022, vol. 24, no. 9, pp. 1235. DOI: 10.3390/e24091235.
Ruff L., Vandermeulen R., Goernitz N., Deecke D., Siddiqui S. A., Binder A., Müller E., Kloft M. Deep one-class classification. Proceedings of the 35th international conference on machine learning, PMLR 80. 2018, pp. 4393–4402. Available at: http://proceedings.mlr.press/v80/ruff18a/ruff18a.pdf (accessed 18.05.2024).
Scott J. T. Factor analysis and regression. Econometrica. 1966, vol. 34, no. 3, pp. 552–562. DOI: 10.2307/1909769.
Buckley J. J., Feuring T. Linear and non-linear fuzzy regression: Evolutionary algorithm solutions. Fuzzy Sets and Systems. 2000, vol. 112, no. 3, pp. 381–394. DOI: 10.1016/s0165-0114(98)00154-7.
Draper N. R., Smith H. Applied regression analysis, 3rd edn. New York, Wiley & Sons, 1998. 736 p. DOI: 10.1002/9781118625590.
Ivakhnenko, A.G. Modelirovanie slozhnykh sistem [Complex systems modelling]. Kyiv, Vyshcha shkola Publ., 1987. 63 p.
Kapanoglu M., Koc I. O., Erdogmus S. Genetic algorithms in parameter estimation for nonlinear regression models: an experimental approach. Journal of Statistical Computation and Simulation. 2007, vol. 77, no. 10, pp. 851–867. DOI: 10.1080/ 10629360600688244.
Mohan S. Parameter estimation of nonlinear Muskingum models using genetic algorithm. Journal of hydraulic engineering. 1997, vol. 123, no. 2, pp. 137–142. DOI: 10.1061/(asce)0733-9429(1997)123:2(137).
Nastenko E., Pavlov V., Boyko G., Nosovets O. Mnogokriterial'nyy algoritm shagovoy regressii [Multicriteria stepwise regression algorithm]. Biomedychna inzheneriya i tekhnolohiya. 2020, no. 3, pp. 48–53. DOI: 10.20535/2617-8974.2020.3.195661
Öztürk O. B., Başar E. Multiple linear regression analysis and artificial neural networks based decision support system for energy efficiency in shipping. Ocean Engineering. 2022, vol. 243, p. 110209. DOI: 10.1016/j.oceaneng.2021.110209.
Rajković D., Jeromela A. M., Pezo L., Lončar B., Grahovac N., Špika A. K. Artificial neural network and random forest regression models for modelling fatty acid and tocopherol content in oil of winter rapeseed. Journal of Food Composition and Analysis. 2023, vol. 115, p. 105020. DOI: 10.1016/j.jfca.2022.105020.
Tam V. W. Y., Butera A., Le K. N., Da Silva L. C. F., Evangelista A. C. J. A prediction model for compressive strength of CO2 concrete using regression analysis and artificial neural networks. Construction and Building Materials. 2022, vol. 324, p. 126689. DOI: 10.1016/j.conbuildmat.2022.126689
Hudson D. J. Statistics lectures, volume 2: Maximum likelihood and least squares theory. CERN Reports 64(18). Geneva, CERN, 1964. (Russ. ed.: Hudson D. Statistika dlja fizikov: Lekcii po teorii verojatnostej i jelementarnoj statistike. Moscow, Mir Publ., 1970. 296 p.). DOI: 10.5170/CERN-1964-018.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
