Published

2023-07-12

An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces

Un método adaptativo para optimizar la función de verosimilitud en modelos lineales mixtos bajo espacios de búsqueda restringidos

DOI:

https://doi.org/10.15446/rce.v46n2.104019

Keywords:

Hybrid genetic algorithm, Linear mixed model, Optimization, Positive denite matrices (en)
Algoritmo genético híbrido, Modelo lineal mixto, Optimización, Matrices denidas positivas (es)

Downloads

Authors

  • Juan Carlos Salazar Uribe Universidad Nacional de Colombia
  • Mauricio Alejandro Mazo Lopera Universidad Nacional de Colombia
  • Juan Carlos Correa Morales Universidad Nacional de Colombia

Linear mixed effects models are highly flexible in handling correlated data by considering covariance matrices that explain variation patterns between and within clusters. For these covariance matrices, there exist a wide list of possible structures proposed by researchers in multiple scientific areas. Maximum likelihood is the most common estimation method in linear mixed models and it depends on the structured covariance matrices for random effects and errors. Classical methods used to optimize the likelihood function, such as Newton-Raphson or Fisher's scoring, require analytical procedures to obtain parametrical restrictions to guarantee positive definiteness for the structured matrices and it is not, in general, an easy task. To avoid dealing with complex restrictions, we propose an adaptive method that incorporates the so-called Hybrid Genetic Algorithms with a penalization technique based on minimum eigenvalues to guarantee positive definiteness in an evolutionary process which discards non-viable cases. The proposed method is evaluated through simulations and its performance is compared with that of Newton-Raphson algorithm implemented in SAS® PROC MIXED V9.4.

Los modelos lineales mixtos son muy flexibles cuando se trabaja con datos correlacionados ya que estos consideran matrices de covarianza que explican los patrones de variación entre individuos y dentro de sus observaciones. Para estas matrices de covarianza existe una amplia lista de posibles estructuras propuestas por investigadores en múltiples áreas científicas. El método de máxima verosimilitud es el más común para la estimación de los parámetros en modelos lineales mixtos y depende de las matrices de covarianza estructuradas para efectos aleatorios y errores. Los métodos clásicos utilizados para optimizar la función de verosimilitud, como Newton-Raphson o Fisher's scoring, requieren desarrollos analíticos para obtener restricciones sobre los parámetros que garanticen matrices estructuradas y definidas positivas, y en general, esto no es una tarea fácil. Para evitar lidiar con restricciones complejas, proponemos un método adaptativo que incorpora los llamados Algoritmos Genéticos Híbridos con una técnica de penalización basada en valores propios mínimos con el _n de garantizar matrices positivas definidas en un proceso evolutivo que descarta casos no viables. El método propuesto se evalúa a través de simulaciones y se compara su desempeño con el algoritmo de Newton-Raphson implementado en SAS® PROC MIXED V9.4.

References

Box, G., Jenkins, G. & Reinsel, G. (1970), Time Series Analysis: Forecasting and Control, fourth edn, John Wiley and Sons, New Jersey.

Revista Colombiana de Estadística - Applied Statistics 46 (2023) 1-23 An Adaptive Method for Likelihood Optimizar in LMM Under Restricted Search Spaces19

Chehouri, A., R. Younes, J. P. & Ilinca, A. (2016), 'A constraint-handling technique for genetic algorithms using a violation factor', Journal of Computer Science 12(7), 350-362. DOI: https://doi.org/10.3844/jcssp.2016.350.362

Coello, C. (2002), 'Theoretical and Numerical Constraint-Handling Techniques used with Evolutionary Algorithms: A Survey of the State of the Art', Computer Methods in Applied Mechanics and Engineering 191, 1245-1287. DOI: https://doi.org/10.1016/S0045-7825(01)00323-1

Coley, D. A. (1998), Introduction to genetic algorithms for scientists and engineers, first edn, World scientific, River Edge, NJ, USA. DOI: https://doi.org/10.1142/3904

Demidenko, E. (2004), Mixed models: Theory and applications with R, second edn, Wiley, New Jersey. DOI: https://doi.org/10.1002/0471728438

Dempster, A., Laird, N. & Rubin, E. (1977), 'Maximum likelihood from incomplete data via the em algorithm', Journal of the Royal Statistical Society 39(1), 1-38. DOI: https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

El-Mihoub, T. A., Hopgood, A. A., Nolle, L. & Battersby, A. (2006), 'Hybrid Genetic Algorithms: A review', Engineering Letters 13, 124-137.

Henderson, C. R. (1984), Applications of linear models in animal breeding, Technical Report Press, University of Guelph, Guelph, Canada.

Holland, J. H. (1975), Adaptation in Natural and Artificial Systems, first edn, University of Michigan Press, Ann Arbor.

Kuri-Morales, A. F. & Gutiérrez-García, J. (2002), 'Penalty Function Methods for Constrained Optimization with Genetic Algorithms: A Statistical Analysis', MICAI 2002: Advances in Artificial Intelligence 2313, 108-117. DOI: https://doi.org/10.1007/3-540-46016-0_12

Laird, N. M. & Ware, J. H. (1982), 'Random-effects models for longitudinal data', Biometrics 38, 963-974. DOI: https://doi.org/10.2307/2529876

Lin, C. (2013), 'A rough penalty genetic algorithm for constrained optimization', Information Sciences 241, 119-137. DOI: https://doi.org/10.1016/j.ins.2013.04.001

Lindstrom, M. J. & Bates, D. M. (1988), 'Newton-Raphson and em algorithms for linear mixed-effects models for repeated-measures data', Journal of the American Statistical Association 83, 1014-1022. DOI: https://doi.org/10.1080/01621459.1988.10478693

Madar, V. (2015), 'Direct formulation to Cholesky decomposition of a general nonsingular correlation matrix', Statistics & Probability Letters 103, 142-147. DOI: https://doi.org/10.1016/j.spl.2015.03.014

Mebane, W. R. J. & Sekhon, J. S. (2011), 'Genetic optimization using derivatives: The rgenoud package for r', Journal of Statistical Software 42(11), 1-26. https://www.jstatsoft.org/v42/i11/ DOI: https://doi.org/10.18637/jss.v042.i11

Michalewicz, Z. (1998), Genetic algorithms + data structures= evolution programs, second edn, Springer-Verlag, Berlin Heidelberg.

Munkres, J. R. (2000), Topology, second edn, Prentice Hall, Upper Saddle River.

Mauricio A. Mazo-Lopera, Juan C. Salazar-Uribe & Juan C. Correa-Morales Patterson, H. D. & Thompson, R. (1971), 'Recovery of inter-block information when block sizes are unequal', Biometrika 58, 545-554. DOI: https://doi.org/10.1093/biomet/58.3.545

Pinheiro, J. C. (1994), Topics in Mixed Effects Models, PhD thesis, University of Wisconsin-Madison, USA.

Pinheiro, J. C. & Bates, D. M. (1996), 'Unconstrained parametrizations for variance-covariance matrices', Statistics and Computing 6, 289-296. DOI: https://doi.org/10.1007/BF00140873

Pothoff, R. & Roy, S. (1964), 'A generalized multivariate analysis of variance model useful especially for growth curve problems', Biometrika 51, 313-326. DOI: https://doi.org/10.1093/biomet/51.3-4.313

Rao, C. (1972), 'Estimation of variance and covariance components in linear models', Journal of the American Statistical Association 67(337), 112-115. DOI: https://doi.org/10.1080/01621459.1972.10481212

Reeves, C. & Rowe, J. E. (2002), Genetic algorithms: Principles and perspectives, first edn, Springer, New York. DOI: https://doi.org/10.1007/b101880

SAS Institute Inc. (2008), SAS/STAT® 9.2 User's Guide. The MIXED Procedure (Chapter). Cary, NC: SAS Institute Inc.

Schott, J. R. (1997), Matrix analysis for statistics, third edn, Wiley, New Jersey.

Scrucca, L. (2017), 'On some extensions to ga package: hybrid optimization, parallelization and islands evolution', The R Journal. 9(1), 187-206. DOI: https://doi.org/10.32614/RJ-2017-008

Verbeke, G. & Molenberghs, G. (1997), Linear Mixed Models in Practice- A SAS-Oriented Approach, first edn, Springer, New York. DOI: https://doi.org/10.1007/978-1-4612-2294-1

West, B., Welch, K. & Galecki, A. (2006), Linear Mixed Models-A practical guide using statistical software, second edn, Chapman and Hall/CRC, London. DOI: https://doi.org/10.1201/9781420010435

Wolfinger, R. (1993), 'Covariance structure selection in general mixed models', Communication’s in Statistics - Simulation and Computation 22, 1079-1106. DOI: https://doi.org/10.1080/03610919308813143

Wolfinger, R., Tobias, R. & Sall, J. (1994), 'Computing gaussian likelihoods and their derivates for general linear mixed models', SIAM Journal on Scientific Computing 15(6), 1294-1310. DOI: https://doi.org/10.1137/0915079

How to Cite

APA

Salazar Uribe, J. C., Mazo Lopera, M. A. and Correa Morales, J. C. (2023). An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces. Revista Colombiana de Estadística, 46(2), 121–143. https://doi.org/10.15446/rce.v46n2.104019

ACM

[1]
Salazar Uribe, J.C., Mazo Lopera, M.A. and Correa Morales, J.C. 2023. An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces. Revista Colombiana de Estadística. 46, 2 (Jul. 2023), 121–143. DOI:https://doi.org/10.15446/rce.v46n2.104019.

ACS

(1)
Salazar Uribe, J. C.; Mazo Lopera, M. A.; Correa Morales, J. C. An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces. Rev. colomb. estad. 2023, 46, 121-143.

ABNT

SALAZAR URIBE, J. C.; MAZO LOPERA, M. A.; CORREA MORALES, J. C. An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces. Revista Colombiana de Estadística, [S. l.], v. 46, n. 2, p. 121–143, 2023. DOI: 10.15446/rce.v46n2.104019. Disponível em: https://revistas.unal.edu.co/index.php/estad/article/view/104019. Acesso em: 2 apr. 2025.

Chicago

Salazar Uribe, Juan Carlos, Mauricio Alejandro Mazo Lopera, and Juan Carlos Correa Morales. 2023. “An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces”. Revista Colombiana De Estadística 46 (2):121-43. https://doi.org/10.15446/rce.v46n2.104019.

Harvard

Salazar Uribe, J. C., Mazo Lopera, M. A. and Correa Morales, J. C. (2023) “An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces”, Revista Colombiana de Estadística, 46(2), pp. 121–143. doi: 10.15446/rce.v46n2.104019.

IEEE

[1]
J. C. Salazar Uribe, M. A. Mazo Lopera, and J. C. Correa Morales, “An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces”, Rev. colomb. estad., vol. 46, no. 2, pp. 121–143, Jul. 2023.

MLA

Salazar Uribe, J. C., M. A. Mazo Lopera, and J. C. Correa Morales. “An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces”. Revista Colombiana de Estadística, vol. 46, no. 2, July 2023, pp. 121-43, doi:10.15446/rce.v46n2.104019.

Turabian

Salazar Uribe, Juan Carlos, Mauricio Alejandro Mazo Lopera, and Juan Carlos Correa Morales. “An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces”. Revista Colombiana de Estadística 46, no. 2 (July 12, 2023): 121–143. Accessed April 2, 2025. https://revistas.unal.edu.co/index.php/estad/article/view/104019.

Vancouver

1.
Salazar Uribe JC, Mazo Lopera MA, Correa Morales JC. An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces. Rev. colomb. estad. [Internet]. 2023 Jul. 12 [cited 2025 Apr. 2];46(2):121-43. Available from: https://revistas.unal.edu.co/index.php/estad/article/view/104019

Download Citation