Penalized Maximum Likelihood Estimation dengan Algoritma Gradient Descent pada Model Regresi Logistik Multinomial

Abstract

Maximum likelihood estimation (MLE) is commonly used for parameter estimation of statistical models, including multinomial logistic regression. However, multicollinearity in logistic regression limits its use. Penalized Maximum Likelihood Estimation (PMLE) overcomes this problem with a penalty on the regression coefficients, resulting in a more stable model and better generalisation. The gradient descent algorithm is used to find the MLE and PMLE solutions without dependence on the starting point. The analysis results show that PMLE has higher accuracy than MLE on the Iris datasets. On the generated dataset with 800 observations and 100 predictor variables, as well as 80 observations and 100 predictor variables, PMLE showed a significant improvement in accuracy. This shows that PMLE is effective in controlling model complexity and improving prediction accuracy, especially on datasets with many predictor variables.