Published

2020-01-01

Relationship Between Kendall's tau Correlation and Mutual Information

Relación entre la correlación tau de Kendall y la información mutua

DOI:

https://doi.org/10.15446/rce.v43n1.78054

Keywords:

Copula function, Kendall’s tau correlation, Mutual information (en)
Función de cópula, Correlación tau de Kendall, Información mutua (es)

Downloads

Authors

  • Mohammad Bolbolian Ghalibaf Hakim Sabzevari‎ University - Faculty of Mathematical Sciences and Computer‎ - Department of Statistics‎
Mutual information (MI) can be viewed as a measure of multivariate association in a random vector. However, the estimation of MI is difficult since the estimation of the joint probability density function (PDF) of non Gaussian distributed data is a hard problem. Copula function is an appropriate tool for estimating MI since the joint probability density function of
random variables can be expressed as the product of the associated copula density function and marginal PDF’s. With a little search, we find that the proposed copulas-based mutual information is much more accurate than conventional methods such as the joint histogram and Parzen window-based MI. In this paper, by using the copulas-based method, we compute MI for
some family of bivariate distribution functions and study the relationship between Kendall’s tau correlation and MI of bivariate distributions. Finally, using a real dataset, we illustrate the efficiency of this approach.

La información mutua (MI) puede ser vista como una medida de asociación multivariante en un vector aleatorio. Sin embargo, la estimación de MI es difícil ya que la estimación de la función de densidad de probabilidad conjunta (PDF) de datos distribuidos no gaussianos es un problema difícil. La función copula es una herramienta apropiada para estimar el MI ya que la función de densidad de probabilidad de las variables aleatorias se puede
expresar como el producto de la función de densidad de cópula asociada y de los PDF marginales. Con una pequeña búsqueda, encontramos que la información mutua propuesta basada en cópulas es mucho más precisa que los métodos convencionales, como el histograma de la articulación y el MI basado en ventana de Parzen. En este artículo, al utilizar el método basado en cópulas, calculamos el MI para algunas familias de funciones de distribución bivariadas y estudiamos la relación entre la correlación tau de Kendall y el MI de las distribuciones bivariadas. Finalmente, usando un conjunto de datos real, ilustramos la eficiencia de este enfoque.

References

Arellano-Valle, R. B., Contreras-Reyes, J. E. & Genton, M. G. (2013), ‘Shannon Entropy and Mutual Information for Multivariate Skew-Elliptical Distributions’, Scandinavian Journal of Statistics 40(1), 42–62.

Bell, C. B. (1962), ‘Mutual information and maximal correlation as measures of dependence’, The Annals of Mathematical Statistics 33(2), 587–595.

Blumentritt, T. & Schmid, F. (2012), ‘Mutual information as a measure of multivariate association: analytical properties and statistical estimation’, Journal of Statistical Computation and Simulation 82(9), 1257–1274.

Calsaverini, R. S. & Vicente, R. (2009), ‘An information-theoretic approach to statistical dependence: Copula information’, EPL (Europhysics Letters) 88(6), 68003.

Clayton, D. G. (1978), ‘A model for association in bivariate life tables and its application in epidemiological studies of familial tendency in chronic disease incidence’, Biometrika 65(1), 141–151.

Cook, R. D. & Johnson, M. E. (1981), ‘A family of distributions for modeling non-elliptically symmetric multivariate data’, Journal of the Royal Statistical Society, Series B 43(2), 210–218.

Cuadras, C. M. & Auge, J. (1981), ‘A continuous general multivariate distribution and its properties’, Communications in Statistics-Theory and Methods 10(4), 339–353.

Dobrowolski, E. & Kumar, P. (2014), ‘Some properties of the Marshall-Olkin and generalized Cuadras-Auge families of copulas’, Australian Journal of Mathematical Analysis and Applications 11(1), 1–13.

Fang, H. B., Fang, K. T. & Kotz, S. (2002), ‘The meta-elliptical distributions with given marginals’, Journal of Multivariate Analysis 82(1), 1–16.

Frank, M. J. (1979), ‘On the simultaneous associativity of F(x; y) and x + y F(x; y)’, Aequationes Mathematicae 99(1), 194–226.

Genest, C. (1987), ‘Frank’s family of bivariate distributions’, Biometrika 74(3), 145–159.

Genest, C. & MacKay, R. J. (1986a), ‘Copules archimediennes et families de lois bidimensionnelles dont les marges sont données’, Canadian Journal of Statistics 14(2), 145–159.

Genest, C. & MacKay, R. J. (1986b), ‘The joy of copulas: bivariate distributions with uniform marginals’, The American Statistician 40(4), 280–283.

Genest, C., Remillard, B. & Beaudoin, D. (2009), ‘Goodness-of-fit tests for copulas: A review and a power study’, Insurance: Mathematics and Economics 44(2), 199–213.

Guerrero-Cusumano, J. L. (1996a), ‘A measure of total variability for the multivariate t distribution with applications to finance’, Information Sciences 92(1), 47–63.

Guerrero-Cusumano, J. L. (1996b), ‘An asymptotic test of independence for multivariate t and Cauchy random variables with applications’, Information Sciences 93(1), 33–45.

Gumbel, E. J. (1960), ‘Distributions des valeurs extrêmes en plusieurs dimensions’, Publications de l’Institut de statistique de l’Université de Paris 9, 171–173.

Hougaard, P. (1986), ‘A class of multivanate failure time distributions’, Biometrika 73(3), 671–678.

Hutchinson, T. P. & Lai, C. D. (1990), Continuous bivariate distributions emphasising applications, Rumsby Scientific Publishing, Adelaide.

Jenison, R. L. & Reale, R. A. (2004), ‘The shape of neural dependence’, Neural computation 16(4), 665–672.

Joe, H. (1989), ‘Relative entropy measures of multivariate dependence’, Journal of the American Statistical Association 84(405), 157–164.

Kendall, M. G. (1938), ‘A new measure of rank correlation’, Biometrika 30(1/2), 81–93.

Kinney, J. B. & Atwal, G. S. (2014), ‘Equitability, mutual information, and the maximal information coefficient’, of the National Academy of Sciences 111(9), 3354–3359.

Kullback, S. (1952), ‘An application of information theory to multivariate analysis’, The Annals of Mathematical Statistics 23(1), 88–102.

Kullback, S. (1959), Information Theory and Statistics, Wiley, New York.

Kumar, P. (2012), ‘Statistical Dependence: Copula functions and mutual information based measures’, Journal of Statistics Applications and Probability: An International Journal 1(1), 1–14.

Kwak, N. & Choi, C. H. (2002), ‘Input feature selection by mutual information based on Parzen window’, IEEE Transactions on Pattern Analysis and Machine Intelligence 24(12), 1667–1671.

Maes, F., Collignon, A., Vandermeulen, D., Marchal, G. & Suetens, P. (1997), ‘Multimodality image registration by maximization of mutual information’, IEEE transactions on Medical Imaging 16(2), 187–198.

Mercier, G. (2005), Mesures de dépendance entre images rso, Technical report, GET/ENST Bretagne, Tech. Rep. RR-2005003-ITI.

Meyer, C. (2013), ‘The bivariate normal copula’, Communications in Statistics Theory and Methods 42(13), 2402–2422.

Nelsen, R. B. (1986), ‘Properties of a one-parameter family of bivariate distributions with specified marginals’, Communications in Statistics Theory and Methods 15(11), 3277–3285.

Nelsen, R. B. (2006), An Introduction to Copulas, Springer, New York.

Oakes, D. (1982), ‘A model for association in bivariate survival data’, Journal of the Royal Statistical Society, Series B 44(3), 414–422.

R Development Core Team (2012), R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. *http://www.R-project.org

Raftery, A. E. (1984), ‘A continuous multivariate exponential distribution’, Communications in Statistics-Theory and Methods 13(8), 947–965.

Raftery, A. E. (1985), ‘Some properties of a new continuous bivariate exponential distribution’, Statistics and Decisions, Supplement Issue 2, 53–58.

Shannon, C. & Weaver, W. (1949), The Mathematical Theory of Communication, University of Illinois Press, Urbana.

Sklar, A. (1959), ‘Fonctions de répartition à n dimensions et leurs marges’, Publications de l’Institut de statistique de l’Université de Paris 8, 229–231.

Zeng, X. & Durrani, T. S. (2011), ‘Estimation of mutual information using copula density function’, Electronics Letters 47(8), 493–494.

How to Cite

APA

Bolbolian Ghalibaf, M. (2020). Relationship Between Kendall’s tau Correlation and Mutual Information. Revista Colombiana de Estadística, 43(1), 3–20. https://doi.org/10.15446/rce.v43n1.78054

ACM

[1]
Bolbolian Ghalibaf, M. 2020. Relationship Between Kendall’s tau Correlation and Mutual Information. Revista Colombiana de Estadística. 43, 1 (Jan. 2020), 3–20. DOI:https://doi.org/10.15446/rce.v43n1.78054.

ACS

(1)
Bolbolian Ghalibaf, M. Relationship Between Kendall’s tau Correlation and Mutual Information. Rev. colomb. estad. 2020, 43, 3-20.

ABNT

BOLBOLIAN GHALIBAF, M. Relationship Between Kendall’s tau Correlation and Mutual Information. Revista Colombiana de Estadística, [S. l.], v. 43, n. 1, p. 3–20, 2020. DOI: 10.15446/rce.v43n1.78054. Disponível em: https://revistas.unal.edu.co/index.php/estad/article/view/78054. Acesso em: 28 mar. 2025.

Chicago

Bolbolian Ghalibaf, Mohammad. 2020. “Relationship Between Kendall’s tau Correlation and Mutual Information”. Revista Colombiana De Estadística 43 (1):3-20. https://doi.org/10.15446/rce.v43n1.78054.

Harvard

Bolbolian Ghalibaf, M. (2020) “Relationship Between Kendall’s tau Correlation and Mutual Information”, Revista Colombiana de Estadística, 43(1), pp. 3–20. doi: 10.15446/rce.v43n1.78054.

IEEE

[1]
M. Bolbolian Ghalibaf, “Relationship Between Kendall’s tau Correlation and Mutual Information”, Rev. colomb. estad., vol. 43, no. 1, pp. 3–20, Jan. 2020.

MLA

Bolbolian Ghalibaf, M. “Relationship Between Kendall’s tau Correlation and Mutual Information”. Revista Colombiana de Estadística, vol. 43, no. 1, Jan. 2020, pp. 3-20, doi:10.15446/rce.v43n1.78054.

Turabian

Bolbolian Ghalibaf, Mohammad. “Relationship Between Kendall’s tau Correlation and Mutual Information”. Revista Colombiana de Estadística 43, no. 1 (January 1, 2020): 3–20. Accessed March 28, 2025. https://revistas.unal.edu.co/index.php/estad/article/view/78054.

Vancouver

1.
Bolbolian Ghalibaf M. Relationship Between Kendall’s tau Correlation and Mutual Information. Rev. colomb. estad. [Internet]. 2020 Jan. 1 [cited 2025 Mar. 28];43(1):3-20. Available from: https://revistas.unal.edu.co/index.php/estad/article/view/78054

Download Citation

CrossRef Cited-by

CrossRef citations13

1. Shouji Fujimoto, Atushi Ishikawa, Takayuki Mizuno. (2022). Copula-Based Synthetic Data Generation in Firm-Size Variables. The Review of Socionetwork Strategies, 16(2), p.479. https://doi.org/10.1007/s12626-022-00128-6.

2. Naqibah Aminuddin Jafry, Jamaludin Suhaila, Fadhilah Yusof, Siti Rohani Mohd Nor, Nor Eliza Alias. (2024). Enhancing flood risk assessment in the Johor River Basin through trivariate copula. Journal of Water and Climate Change, 15(4), p.1820. https://doi.org/10.2166/wcc.2024.624.

3. Damaris Brigithe Neto Eras, Cecibel Alejandra Granja Daza, Luis Daniel Nauta Padilla. (2025). Análisis de los factores económicos y su incidencia en la sostenibilidad de las microempresas comerciales del cantón La Maná. Bastcorp International Journal, 4(1), p.263. https://doi.org/10.62943/bij.v4n1.2025.172.

4. Jonathan Rathjens, Arthur Kolbe, Jürgen Hölzer, Katja Ickstadt, Nadja Klein. (2024). Bivariate Analysis of Birth Weight and Gestational Age by Bayesian Distributional Regression with Copulas. Statistics in Biosciences, 16(1), p.290. https://doi.org/10.1007/s12561-023-09396-4.

5. Mateusz Oszczypała, Jakub Konwerski, Jarosław Ziółkowski, Jerzy Małachowski. (2024). Copula-Based Reliability Analysis of Vehicles Based on Censored Failures Data Using Reliability Importance Measures. IEEE Access, 12, p.154119. https://doi.org/10.1109/ACCESS.2024.3450076.

6. Mohammad Karami, Saeid Shabanlou, Hosein Mazaheri, Shahroo Mokhtari, Mohsen Najarchi. (2024). Integration of the Non-linear Time Series GARCH Model with Fuzzy Model Optimized with Water Cycle Algorithm for River Streamflow Forecasting. International Journal of Computational Intelligence Systems, 17(1) https://doi.org/10.1007/s44196-024-00570-0.

7. Fadhah Amer Alanazi, Mohammad Amin Hariri-Ardebili. (2021). Truncating Regular Vine Copula Based on Mutual Information: An Efficient Parsimonious Model for High-Dimensional Data. Mathematical Problems in Engineering, 2021, p.1. https://doi.org/10.1155/2021/4347957.

8. Emanuela Furfaro, Fushing Hsieh. (2023). Ordinal Conditional Entropy Displays Reveal Intrinsic Characteristics of the Rosenberg Self-Esteem Scale. Entropy, 25(9), p.1311. https://doi.org/10.3390/e25091311.

9. N. A. Jafry, J. Suhaila, F. Yusof, S. R. M. Nor, N. E. Alias. (2024). Modeling the dependency structure of four-dimensional flood variables using the copula approach. THE 3RD INTERNATIONAL CONFERENCE ON NATURAL SCIENCES, MATHEMATICS, APPLICATIONS, RESEARCH, AND TECHNOLOGY (ICON-SMART2022): Mathematical Physics and Biotechnology for Education, Energy Efficiency, and Marine Industries. THE 3RD INTERNATIONAL CONFERENCE ON NATURAL SCIENCES, MATHEMATICS, APPLICATIONS, RESEARCH, AND TECHNOLOGY (ICON-SMART2022): Mathematical Physics and Biotechnology for Education, Energy Efficiency, and Marine Industries. 3132, p.080002. https://doi.org/10.1063/5.0213956.

10. Babak Sohrabian, Abdullah Erhan Tercan. (2024). Copula-Based Data-Driven Multiple-Point Simulation Method. Spatial Statistics, 59, p.100802. https://doi.org/10.1016/j.spasta.2023.100802.

11. Edoardo Saccenti. (2023). Metabolomics. , p.59. https://doi.org/10.1007/978-3-031-39094-4_3.

12. Johny Pambabay-Calero, Sergio Bauz-Olvera, Ana Nieto-Librero, Ana Sánchez-García, Puri Galindo-Villardón. (2021). Hierarchical Modeling for Diagnostic Test Accuracy Using Multivariate Probability Distribution Functions. Mathematics, 9(11), p.1310. https://doi.org/10.3390/math9111310.

13. Soumik Purkayastha, Peter X.-K. Song. (2024). fastMI: A fast and consistent copula-based nonparametric estimator of mutual information. Journal of Multivariate Analysis, 201, p.105270. https://doi.org/10.1016/j.jmva.2023.105270.

Dimensions

PlumX

Plum Print visual indicator of research metrics
  • Citations
  • CrossRef - Citation Indexes: 12
  • Scopus - Citation Indexes: 10
  • Usage
  • SciELO - Full Text Views: 1285
  • SciELO - Abstract Views: 197
  • Captures
  • Mendeley - Readers: 20
-
see details

Article abstract page views

872

Downloads

License

Copyright (c) 2020 Revista Colombiana de Estadística

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).