Distance Metrics and Clustering Methods for Mixed-type Data
Abstract
In spite of the abundance of clustering techniques and algorithms, clustering mixed interval (continuous) and categorical (nominal and/or ordinal) scale data remain a challenging problem. In order to identify the most effective approaches for clustering mixed–type data, we use both theoretical and empirical analyses to present a critical review of the strengths and weaknesses of the methods identified in the literature. Here, the guidelines on approaches to use under different scenarios are provided, along with potential directions for future research.
- Authors:
-
[1];
- Univ. at Buffalo, Buffalo, NY (United States)
- Arenadotio, New York, NY (United States)
- Publication Date:
- Research Org.:
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1459931
- Report Number(s):
- SAND-2018-7091J
Journal ID: ISSN 0306-7734; 665360
- Grant/Contract Number:
- AC04-94AL85000
- Resource Type:
- Accepted Manuscript
- Journal Name:
- International Statistical Review
- Additional Journal Information:
- Journal Volume: 87; Journal Issue: 1; Journal ID: ISSN 0306-7734
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; discretisation; dummy coding; Gower's distance; k-mean clustering; machine learning; Mahalanobis distance; mixture model; multivariate data analysis; unsupervised learning
Citation Formats
Foss, Alexander H., Markatou, Marianthi, and Ray, Bonnie. Distance Metrics and Clustering Methods for Mixed-type Data. United States: N. p., 2018.
Web. doi:10.1111/insr.12274.
Foss, Alexander H., Markatou, Marianthi, & Ray, Bonnie. Distance Metrics and Clustering Methods for Mixed-type Data. United States. doi:10.1111/insr.12274.
Foss, Alexander H., Markatou, Marianthi, and Ray, Bonnie. Thu .
"Distance Metrics and Clustering Methods for Mixed-type Data". United States. doi:10.1111/insr.12274. https://www.osti.gov/servlets/purl/1459931.
@article{osti_1459931,
title = {Distance Metrics and Clustering Methods for Mixed-type Data},
author = {Foss, Alexander H. and Markatou, Marianthi and Ray, Bonnie},
abstractNote = {In spite of the abundance of clustering techniques and algorithms, clustering mixed interval (continuous) and categorical (nominal and/or ordinal) scale data remain a challenging problem. In order to identify the most effective approaches for clustering mixed–type data, we use both theoretical and empirical analyses to present a critical review of the strengths and weaknesses of the methods identified in the literature. Here, the guidelines on approaches to use under different scenarios are provided, along with potential directions for future research.},
doi = {10.1111/insr.12274},
journal = {International Statistical Review},
number = 1,
volume = 87,
place = {United States},
year = {2018},
month = {6}
}
Other availability
Search WorldCat to find libraries that may hold this journalFigures / Tables:
Figure 1: Simulation results: performance of k-modes clustering and latent class analysis (LCA) for various quantile splits of the data. A mixed-type data set was generated with two underlying clusters, and the interval scale variable was discretised using a quantile split with the number of bins shown along the x-axis.more »
All figures and tables
(11 total)
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Model-based clustering using copulas with applications
journal, July 2015
- Kosmidis, Ioannis; Karlis, Dimitris
- Statistics and Computing, Vol. 26, Issue 5
Contributions to the Mathematical Theory of Evolution
journal, January 1894
- Pearson, K.
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 185, Issue 0
Distance functions for categorical and mixed variables
journal, May 2008
- McCane, Brendan; Albert, Michael
- Pattern Recognition Letters, Vol. 29, Issue 7
Comparing clusterings—an information based distance
journal, May 2007
- Meilă, Marina
- Journal of Multivariate Analysis, Vol. 98, Issue 5
On the null distribution of distance between two groups, using mixed continuous and categorical variables
journal, December 1984
- Krzanowski, W. J.
- Journal of Classification, Vol. 1, Issue 1
poLCA : An R Package for Polytomous Variable Latent Class Analysis
journal, January 2011
- Linzer, Drew A.; Lewis, Jeffrey B.
- Journal of Statistical Software, Vol. 42, Issue 10
Scalable algorithms for clustering large datasets with mixed type attributes
journal, January 2005
- He, Zengyou; Xu, Xiaofei; Deng, Shengchun
- International Journal of Intelligent Systems, Vol. 20, Issue 10
Model-Based Gaussian and Non-Gaussian Clustering
journal, September 1993
- Banfield, Jeffrey D.; Raftery, Adrian E.
- Biometrics, Vol. 49, Issue 3
A monte carlo study of thirty internal criterion measures for cluster analysis
journal, June 1981
- Milligan, Glenn W.
- Psychometrika, Vol. 46, Issue 2
Estimating the Cluster Tree of a Density by Analyzing the Minimal Spanning Tree of a Sample
journal, May 2003
- Stuetzle, Werner
- Journal of Classification, Vol. 20, Issue 1
Cluster Validation by Prediction Strength
journal, September 2005
- Tibshirani, Robert; Walther, Guenther
- Journal of Computational and Graphical Statistics, Vol. 14, Issue 3
Robust mixture modeling using multivariate skew t distributions
journal, May 2009
- Lin, Tsung-I
- Statistics and Computing, Vol. 20, Issue 3
Model-Based Clustering
journal, October 2016
- McNicholas, Paul D.
- Journal of Classification, Vol. 33, Issue 3
Maximum Likelihood from Incomplete Data Via the EM Algorithm
journal, September 1977
- Dempster, A. P.; Laird, N. M.; Rubin, D. B.
- Journal of the Royal Statistical Society: Series B (Methodological), Vol. 39, Issue 1
Clustering mixed data: Clustering mixed data
journal, May 2011
- Hunt, Lynette; Jorgensen, Murray
- Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 1, Issue 4
kamila : Clustering Mixed-Type Data in R and Hadoop
journal, January 2018
- Foss, Alexander H.; Markatou, Marianthi
- Journal of Statistical Software, Vol. 83, Issue 13
One-Sample Likelihood Ratio Tests for Mixed Data
journal, January 2007
- de Leon, A. R.
- Communications in Statistics - Theory and Methods, Vol. 36, Issue 1
On Using Principal Components Before Separating a Mixture of Two Multivariate Normal Distributions
journal, January 1983
- Chang, Wei-Chien
- Applied Statistics, Vol. 32, Issue 3
A latent variables approach for clustering mixed binary and continuous variables within a Gaussian mixture model
journal, November 2011
- Morlini, Isabella
- Advances in Data Analysis and Classification, Vol. 6, Issue 1
A finite mixture model for the clustering of mixed-mode data
journal, April 1988
- Everitt, B. S.
- Statistics & Probability Letters, Vol. 6, Issue 5
A Gamma mixture model better accounts for among site rate heterogeneity
journal, September 2005
- Mayrose, I.; Friedman, N.; Pupko, T.
- Bioinformatics, Vol. 21, Issue Suppl 2
Mixtures of Shifted AsymmetricLaplace Distributions
journal, June 2014
- Franczak, Brian C.; Browne, Ryan P.; McNicholas, Paul D.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, Issue 6
Model-based clustering with non-elliptically contoured distributions
journal, June 2008
- Karlis, Dimitris; Santourian, Anais
- Statistics and Computing, Vol. 19, Issue 1
Distance between populations using mixed continuous and categorical variables
journal, January 1983
- Krzanowski, W. J.
- Biometrika, Vol. 70, Issue 1
Quadratic distances on probabilities: A unified foundation
journal, April 2008
- Lindsay, Bruce G.; Markatou, Marianthi; Ray, Surajit
- The Annals of Statistics, Vol. 36, Issue 2
Model-based clustering, classification, and discriminant analysis of data with mixed type
journal, November 2012
- Browne, Ryan P.; McNicholas, Paul D.
- Journal of Statistical Planning and Inference, Vol. 142, Issue 11
Statistical Modelling of Data on Teaching Styles
journal, January 1981
- Aitkin, Murray; Anderson, Dorothy; Hinde, John
- Journal of the Royal Statistical Society. Series A (General), Vol. 144, Issue 4
Estimating the number of clusters in a data set via the gap statistic
journal, May 2001
- Tibshirani, Robert; Walther, Guenther; Hastie, Trevor
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 63, Issue 2, p. 411-423
A new look at the statistical model identification
journal, December 1974
- Akaike, H.
- IEEE Transactions on Automatic Control, Vol. 19, Issue 6
Top 10 algorithms in data mining
journal, December 2007
- Wu, Xindong; Kumar, Vipin; Ross Quinlan, J.
- Knowledge and Information Systems, Vol. 14, Issue 1
A Population Background for Nonparametric Density-Based Clustering
journal, November 2015
- Chacón, José E.
- Statistical Science, Vol. 30, Issue 4
Enhancing the selection of a model-based clustering with external categorical variables
journal, June 2014
- Baudry, Jean-Patrick; Cardoso, Margarida; Celeux, Gilles
- Advances in Data Analysis and Classification, Vol. 9, Issue 2
Order selection in finite mixture models: complete or observed likelihood information criteria?
journal, June 2015
- Hui, Francis K. C.; Warton, David I.; Foster, Scott D.
- Biometrika, Vol. 102, Issue 3
A k-means type clustering algorithm for subspace clustering of mixed numeric and categorical datasets
journal, May 2011
- Ahmad, Amir; Dey, Lipika
- Pattern Recognition Letters, Vol. 32, Issue 7
What are the true clusters?
journal, October 2015
- Hennig, Christian
- Pattern Recognition Letters, Vol. 64
Stability-Based Validation of Clustering Solutions
journal, June 2004
- Lange, Tilman; Roth, Volker; Braun, Mikio L.
- Neural Computation, Vol. 16, Issue 6
Parsimonious skew mixture models for model-based clustering and classification
journal, March 2014
- Vrbik, Irene; McNicholas, Paul D.
- Computational Statistics & Data Analysis, Vol. 71
Mean shift: a robust approach toward feature space analysis
journal, May 2002
- Comaniciu, D.; Meer, P.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 5
Algorithm AS 136: A K-Means Clustering Algorithm
journal, January 1979
- Hartigan, J. A.; Wong, M. A.
- Applied Statistics, Vol. 28, Issue 1
Clustering via Nonparametric Density Estimation: The R Package pdfCluster
journal, January 2014
- Azzalini, Adelchi; Menardi, Giovanna
- Journal of Statistical Software, Vol. 57, Issue 11
Mixtures of Continuous and Categorical Variables in Discriminant Analysis
journal, September 1980
- Krzanowski, W. J.
- Biometrics, Vol. 36, Issue 3
The clustering of mixed-mode data: a comparison of possible approaches
journal, January 1990
- Everitt, B. S.; Merette, C.
- Journal of Applied Statistics, Vol. 17, Issue 3
A robust and scalable clustering algorithm for mixed type attributes in large database environment
conference, January 2001
- Chiu, Tom; Fang, DongPing; Chen, John
- Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '01
Multivariate Correlation Models with Mixed Discrete and Continuous Variables
journal, June 1961
- Olkin, I.; Tate, R. F.
- The Annals of Mathematical Statistics, Vol. 32, Issue 2
Classification with discrete and continuous variables via general mixed-data models
journal, May 2011
- de Leon, A. R.; Soo, A.; Williamson, T.
- Journal of Applied Statistics, Vol. 38, Issue 5
Resampling Method for Unsupervised Estimation of Cluster Validity
journal, November 2001
- Levine, Erel; Domany, Eytan
- Neural Computation, Vol. 13, Issue 11
Estimating the Dimension of a Model
journal, March 1978
- Schwarz, Gideon
- The Annals of Statistics, Vol. 6, Issue 2
Gifi Methods for Optimal Scaling in R : The Package homals
journal, January 2009
- Leeuw, Jan de; Mair, Patrick
- Journal of Statistical Software, Vol. 31, Issue 4
Challenges of Big Data analysis
journal, February 2014
- Fan, Jianqing; Han, Fang; Liu, Han
- National Science Review, Vol. 1, Issue 2
Extending mixtures of factor models using the restricted multivariate skew-normal distribution
journal, January 2016
- Lin, Tsung-I; McLachlan, Geoffrey J.; Lee, Sharon X.
- Journal of Multivariate Analysis, Vol. 143
Resampling approach for cluster model selection
journal, March 2011
- Volkovich, Z.; Barzily, Z.; Weber, G. -W.
- Machine Learning, Vol. 85, Issue 1-2
Ellipsoidally symmetric extensions of the general location model for mixed categorical and continuous data
journal, September 1998
- Liu, C.
- Biometrika, Vol. 85, Issue 3
Data clustering: 50 years beyond K-means
journal, June 2010
- Jain, Anil K.
- Pattern Recognition Letters, Vol. 31, Issue 8
A General Coefficient of Similarity and Some of Its Properties
journal, December 1971
- Gower, J. C.
- Biometrics, Vol. 27, Issue 4
Mixtures of Continuous and Categorical Variables in Discriminant Analysis: A Hypothesis-Testing Approach
journal, December 1982
- Krzanowski, W. J.
- Biometrics, Vol. 38, Issue 4
Mixtures of skew-<mml:math altimg="si111.gif" display="inline" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd" xmlns:sa="http://www.elsevier.com/xml/common/struct-aff/dtd"><mml:mi>t</mml:mi></mml:math> factor analyzers
journal, September 2014
- Murray, Paula M.; Browne, Ryan P.; McNicholas, Paul D.
- Computational Statistics & Data Analysis, Vol. 77
Flexible parametric bootstrap for testing homogeneity against clustering and assessing the number of clusters
journal, June 2015
- Hennig, Christian; Lin, Chien-Ju
- Statistics and Computing, Vol. 25, Issue 4
Maximum likelihood estimation for multivariate skew normal mixture models
journal, February 2009
- Lin, Tsung I.
- Journal of Multivariate Analysis, Vol. 100, Issue 2
Assessing a mixture model for clustering with the integrated completed likelihood
journal, July 2000
- Biernacki, C.; Celeux, G.; Govaert, G.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, Issue 7
Categorical-and-numerical-attribute data clustering based on a unified similarity metric without knowing cluster number
journal, August 2013
- Cheung, Yiu-ming; Jia, Hong
- Pattern Recognition, Vol. 46, Issue 8
Model-Based Clustering, Discriminant Analysis, and Density Estimation
journal, June 2002
- Fraley, Chris; Raftery, Adrian E.
- Journal of the American Statistical Association, Vol. 97, Issue 458
Comparing partitions
journal, December 1985
- Hubert, Lawrence; Arabie, Phipps
- Journal of Classification, Vol. 2, Issue 1
Clustering via nonparametric density estimation
journal, February 2007
- Azzalini, Adelchi; Torelli, Nicola
- Statistics and Computing, Vol. 17, Issue 1
Comparing clusterings: an axiomatic view
conference, January 2005
- Meilǎ, Marina
- Proceedings of the 22nd international conference on Machine learning - ICML '05
A semiparametric method for clustering mixed data
journal, July 2016
- Foss, Alex; Markatou, Marianthi; Ray, Bonnie
- Machine Learning, Vol. 105, Issue 3
Variable selection for model-based clustering using the integrated complete-data likelihood
journal, May 2016
- Marbac, Matthieu; Sedki, Mohammed
- Statistics and Computing, Vol. 27, Issue 4
How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis
journal, August 1998
- Fraley, C.
- The Computer Journal, Vol. 41, Issue 8
Clustering Methods Based on Likelihood Ratio Criteria
journal, June 1971
- Scott, A. J.; Symons, M. J.
- Biometrics, Vol. 27, Issue 2
Finite mixtures of multivariate skew t-distributions: some recent and new results
journal, October 2012
- Lee, Sharon; McLachlan, Geoffrey J.
- Statistics and Computing, Vol. 24, Issue 2
Multivariate Tests for Clusters
journal, September 1979
- Lee, Kerry L.
- Journal of the American Statistical Association, Vol. 74, Issue 367
FlexMix Version 2: Finite Mixtures with Concomitant Variables and Varying and Constant Parameters
journal, January 2008
- Grün, Bettina; Leisch, Friedrich
- Journal of Statistical Software, Vol. 28, Issue 4
How to find an appropriate clustering for mixed-type variables with application to socio-economic stratification: How to Find an Appropriate Clustering
journal, April 2013
- Hennig, Christian; Liao, Tim F.
- Journal of the Royal Statistical Society: Series C (Applied Statistics), Vol. 62, Issue 3
Identifiability of parameters in latent structure models with many observed variables
journal, December 2009
- Allman, Elizabeth S.; Matias, Catherine; Rhodes, John A.
- The Annals of Statistics, Vol. 37, Issue 6A
An examination of procedures for determining the number of clusters in a data set
journal, June 1985
- Milligan, Glenn W.; Cooper, Martha C.
- Psychometrika, Vol. 50, Issue 2
A Study of the Comparability of External Criteria for Hierarchical Cluster Analysis
journal, October 1986
- Milligan, Glenn W.; Cooper, Martha C.
- Multivariate Behavioral Research, Vol. 21, Issue 4
On Information and Sufficiency
journal, March 1951
- Kullback, S.; Leibler, R. A.
- The Annals of Mathematical Statistics, Vol. 22, Issue 1
The location model for mixtures of categorical and continuous variables
journal, January 1993
- Krzanowski, W. J.
- Journal of Classification, Vol. 10, Issue 1
Model selection for mixture-based clustering for ordinal data
journal, December 2016
- Fernández, D.; Arnold, R.
- Australian & New Zealand Journal of Statistics, Vol. 58, Issue 4
An EM algorithm for a mixture model of count data
journal, May 1993
- Fong, Daniel Y. T.; Yip, Paul
- Statistics & Probability Letters, Vol. 17, Issue 1
Applications of beta-mixture models in bioinformatics
journal, February 2005
- Ji, Y.; Wu, C.; Liu, P.
- Bioinformatics, Vol. 21, Issue 9
Simultaneous model-based clustering and visualization in the Fisher discriminative subspace
journal, April 2011
- Bouveyron, Charles; Brunet, Camille
- Statistics and Computing, Vol. 22, Issue 1
Mixture separation for mixed-mode data
journal, March 1996
- Lawrence, C. J.; Krzanowski, W. J.
- Statistics and Computing, Vol. 6, Issue 1
A mixture of generalized hyperbolic distributions: A MIXTURE OF GENERALIZED HYPERBOLIC DISTRIBUTIONS
journal, February 2015
- Browne, Ryan P.; McNicholas, Paul D.
- Canadian Journal of Statistics, Vol. 43, Issue 2
Pattern Clustering by Multivariate Mixture Analysis
journal, April 1970
- Wolfe, John H.
- Multivariate Behavioral Research, Vol. 5, Issue 3
Determining the number of clusters using information entropy for mixed data
journal, June 2012
- Liang, Jiye; Zhao, Xingwang; Li, Deyu
- Pattern Recognition, Vol. 45, Issue 6
Advances in multidimensional integration
journal, December 2002
- Cools, Ronald
- Journal of Computational and Applied Mathematics, Vol. 149, Issue 1
Generalization of the Mahalanobis Distance in the Mixed Case
journal, May 1995
- Barhen, A.; Daudin, J. J.
- Journal of Multivariate Analysis, Vol. 53, Issue 2
Estimating the Mahalanobis Distance from Mixed Continuous and Discrete Data
journal, June 2000
- Bedrick, Edward J.; Lapidus, Jodi; Powell, Joseph F.
- Biometrics, Vol. 56, Issue 2
Linear Fuzzy Clustering of Mixed Databases Based on Cluster-wise Optimal Scaling of Categorical Variables
conference, June 2007
- Honda, Katsuhiro; Uesugi, Ryo; Ichihashi, Hidetomo
- 2007 IEEE International Fuzzy Systems Conference
Simulating Data to Study Performance of Finite Mixture Modeling and Clustering Algorithms
journal, January 2010
- Maitra, Ranjan; Melnykov, Volodymyr
- Journal of Computational and Graphical Statistics, Vol. 19, Issue 2
A mixture of generalized latent variable models for mixed mode and heterogeneous data
journal, November 2011
- Cai, Jing-Heng; Song, Xin-Yuan; Lam, Kwok-Hap
- Computational Statistics & Data Analysis, Vol. 55, Issue 11
Model based clustering for mixed data: clustMD
journal, February 2016
- McParland, Damien; Gormley, Isobel Claire
- Advances in Data Analysis and Classification, Vol. 10, Issue 2
Exploratory latent structure analysis using both identifiable and unidentifiable models
journal, January 1974
- Goodman, Leo A.
- Biometrika, Vol. 61, Issue 2
A generalized Mahalanobis distance for mixed data
journal, January 2005
- de Leon, A. R.; Carrière, K. C.
- Journal of Multivariate Analysis, Vol. 92, Issue 1
Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.