Comprehensive Analysis of Data Preprocessing and Models in Data Mining
DOI:
https://doi.org/10.61173/3ca44e45Keywords:
DataPreprocessing, Optimization Models, Decision-Making Models, Principal Component Analysis, Hierarchical Analysis, Data MingAbstract
Nowadays, the data of the Internet shows explosive growth, in which the massive data contains great economic value. How to deal with and mine data has become an important topic nowadays. In statistics, data mining is often used to build corresponding mathematical models to analyze and process data, so as to get the results. Model to analyze and process the data, so as to get the results people want. This paper summarizes the data processing in the field of data analysis and processing technology. methods in the field of data analysis and processing technology, and first introduces the commonly used data preprocessing methods. On this basis, this paper describes in detail the commonly used data optimization models, predictive In details, this paper describes the commonly used data optimization models, prediction, decision-making models, such as: Linear programming model ,Dynamic programming model. In details, its process in data analysis and processing is detailed from the basic principles, processing steps and practical applications. Finally, we summarize the data preprocessing techniques and model building mentioned throughout the paper. This paper alsolooks ahead to the current challenges that society faces in the data domain, and speculates on future research directions with high relevance.References
[1] Jansen B J, Aldous K K, Salminen J, et al. Understanding Audiences, Customers, and Users Via Analytics: An Introduction to the Employment of Web, Social, and Other Types of Digital People Data[M]. Springer Nature, 2023.
[2] Albahra S, Gorbett T, Robertson S, et al. Artificial intelligence and machine learning overview in pathology & laboratory medicine: A general review of data preprocessing and basic supervised concepts[C]//Seminars in Diagnostic Pathology. WB Saunders, 2023, 40(2): 71-87.
[3] Côté P O, Nikanjam A, Ahmed N, et al. Data cleaning and machine learning: a systematic literature review[J]. Automated Software Engineering, 2024, 31(2): 54.
[4] Gomathi S. Data Migration and Cleaning[J].
[5] Chicco D, Oneto L, Tavazzi E. Eleven quick tips for data cleaning and feature engineering[J]. PLOS Computational Biology, 2022, 18(12): e1010718.
[6] Gracias A, Shad R, Broklyn P. DATA INTEGRATION AND ANALYSIS IN PRECISION MEDICINE[J].
[7] Nargesian F, Asudeh A, Jagadish H V. Responsible data integration: Next-generation challenges[C]//Proceedings of the 2022 International Conference on Management of Data. 2022: 2458-2464.
[8] Nascimento J, Silva J, Cupertino Bernardes R, et al. Statistical data transformation in agrarian sciences for variance analysis: a systematic review[J]. F1000Research, 2024, 13: 459.
[9] Drosu A, Cofaru C, Popescu M V, et al. The influence of data aggregation levels on accident severity researches[C]//IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2024, 1311(1): 012041. Dean&Francis ISSN 2959-6157
[10] Pang K, Zou L, Kang N, et al. The construction of fuzzy linguistic attribute partial ordered structure diagram[J]. Computational and Applied Mathematics, 2023, 42(6): 240.
[11] Dudic B, Nguyen N T, Ašonja A. Data Normalization for Root Assessment Methodology[J]. International Journal of Industrial Engineering and Management, 2024: in press-in press.
[12] Celik A. Evaluating the impact of data normalization on rice classification using machine learning algorithms[J]. Acadlore Trans. Mach. Learn, 2024, 3(3): 162-171.
[13] Setiawan R, Imelda I. Application of min-max normalization in backpropagation to detect early malnutrition[J]. Indonesian Journal of Multidisciplinary Science, 2024, 3(10).
[14] Rahmad Ramadhan L, Anne Mudya Y. A Comparative Study of Z-Score and Min-Max Normalization for Rainfall Classification in Pekanbaru[J]. Journal of Data Science, 2024, 2024(04): [15]Pambudi E A, Fahrezi A G, Badharudin A Y. Forecasting Brown Sugar Production Using k-NN Minkowski Distance and Z-Score Normalization[J]. Journal of Information Systems and Informatics, 2023, 5(2): 580-589.
[16] Rahmad Ramadhan L, Anne Mudya Y. A Comparative Study of Z-Score and Min-Max Normalization for Rainfall Classification in Pekanbaru[J]. Journal of Data Science, 2024, 2024(04): [17]BALASUBRAMANIAM S. Practical Application of Linear programming technique in textile spinning industry[J].
[18] Alzalg B. Combinatorial and algorithmic mathematics: From foundation to optimization[M]. John Wiley & Sons, Inc., 2024.
[19] Chakraborty A, Chandru V, Rao M R. A linear programming primer: from Fourier to Karmarkar[J]. Annals of Operations Research, 2020, 287: 593-616.
[20] Khadapi M, Ramadani S. Optimization Of Sandal Production Using Linear Programming[J]. Journal of Artificial Intelligence and Engineering Applications (JAIEA), 2024, 3(3): 631-638.
[21] Nakayama K. Eigenvalue Problem and Nonlinear Programming Problem: For Economic Studies[M]. Springer Nature, 2023.
[22] Gürsoy N C, Alptekin N. Modelling Strategy of Airline Tankering with Nonlinear Programming[J]. Verimlilik Dergisi, 2023, 57(2): 365-392.
[23] Ponce‐Ortega J M, Hortua A C, El‐Halwagi M, et al. A property‐based optimization of direct recycle networks and wastewater treatment processes[J]. AIChE Journal, 2009, 55(9): 2329-2344.
[24] Eisenbrand F, Hunkenschröder C, Klein K M, et al. Sparse Integer Programming Is Fixed-Parameter Tractable[J]. Mathematics of Operations Research, 2024.
[25] Ribeiro C C, Urrutia S, de Werra D. Combinatorial Models for Scheduling Sports Tournaments[M]. Berlin: Springer, 2023.
[26] Laaksonen A. Guide to competitive programming[M]. Cham: Springer, 2020.
[27] Hinderer K, Rieder U, Stieglitz M. Dynamic optimization[M]. Springer, 2016.
[28] Bellman R. Dynamic programming[J]. science, 1966, 153(3731): 34-37.
[29] ФЕДОРОВ Є, НЕЧИПОРЕНКО О, НЕСКОРОДЄВА Т, et al. DYNAMIC PROGRAMMING FOR SOLVE THE INVENTORY MANAGEMENT PROBLEM IN LOGISTICS[J]. Computer systems and information technologies, 2024 (1): 118- 126.
[30] Buckley J J. Fuzzy hierarchical analysis[J]. Fuzzy sets and systems, 1985, 17(3): 233-247.
[31] Pauer F, Schmidt K, Babac A, et al. Comparison of different approaches applied in analytic hierarchy process–An example of information needs of patients with rare diseases[J]. BMC medical informatics and decision making, 2016, 16: 1-11.
[32] Stanton N A. Hierarchical task analysis: Developments, applications, and extensions[J]. Applied ergonomics, 2006, 37(1): 55-79.
[33] Tramarico C L, Salomon V A P, Marins F A S. Analytic hierarchy process and supply chain management: A bibliometric study[J]. Procedia Computer Science, 2015, 55: 441-450.
[34] Bridges Jr C C. Hierarchical cluster analysis[J]. Psychological reports, 1966, 18(3): 851-854.
[35] Raudenbush S W. Hierarchical linear models: Applications and data analysis methods[J]. Advanced Quantitative Techniques in the Social Sciences Series/SAGE, 2002.
[36] Hillerman T, Souza JCF, Reis ACB, Carvalho RN. Applying clustering and AHP433-459.
[37] methods for evaluating suspect healthcare claims. J Comput Sci. 2017;19.97-111. https://doi.org/10.1016/jjocs.2017.02.007.
[38] Marza B, Bratu R, Serbu R, Stan S, Oprean-Stan C. Applying ahp and fuzzy Ahp Management methods to assess the level of financial and digital inclusion. Economic Comput Economic Cybernetics Stud Res. 2021;55(4):165-82. https://doi. org/10.24818/18423264/55.4.21.11.
[39] Johnson R A, Wichern D W. Applied multivariate statistical analysis[J]. 2002.
[40] Abdi H, Williams L J. Principal component analysis[J]. Wiley interdisciplinary reviews: computational statistics, 2010, 2(4): 433-459.
[41] Makkulau M, Ampa A T, Yahya I, et al. Variance The Estimation Eigen Value of Principal Component Analysis and Nonlinear Principal Component Analysis[C]//ITM Web of Conferences. EDP Sciences, 2024, 58: 04001.
[42] Nargundkar A, Gulia V, Sur A. Multi-Variate Analysis of Shell & Tube Heat Exchanger using Principal Component Analysis[C]//E3S Web of Conferences. EDP Sciences, 2024, 552: 01025.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 by the authors.

This work is licensed under a Creative Commons Attribution 4.0 International License.
