skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ColdRoute: effective routing of cold questions in stack exchange sites

Abstract

Routing questions in Community Question Answer services such as Stack Exchange sites is a well-studied problem. Yet, cold-start—a phenomena observed when a new question is posted is not well addressed by existing approaches. Additionally, cold questions posted by new askers present significant challenges to state-of-the-art approaches. We propose ColdRoute to address these challenges. ColdRoute is able to handle the task of routing cold questions posted by new or existing askers to matching experts. Specifically, we use Factorization Machines on the one-hot encoding of critical features such as question tags and compare our approach to well-studied techniques such as CQARank and semantic matching (LDA, BoW, and Doc2Vec). Furthermore by using data from eight stack exchange sites, we are able to improve upon the routing metrics (Precision@1, Accuracy, MRR) over the state-of-the-art models such as semantic matching by 159.5, 31.84, and 40.36% for cold questions posted by existing askers, and 123.1, 27.03, and 34.81% for cold questions posted by new askers respectively.

Authors:
ORCiD logo [1];  [2];  [3];  [2];  [1]
  1. The Ohio State Univ., Columbus, OH (United States)
  2. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  3. Microsoft, Albuquerque, NM (United States)
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1460547
Grant/Contract Number:  
CCF-1645599; IIS-1550302; CNS-1513120; PAS0166; AC05-76RL01830
Resource Type:
Accepted Manuscript
Journal Name:
Data Mining and Knowledge Discovery
Additional Journal Information:
Journal Volume: 32; Journal Issue: 5; Journal ID: ISSN 1384-5810
Country of Publication:
United States
Language:
English
Subject:
96 KNOWLEDGE MANAGEMENT AND PRESERVATION; Question routing; Expert finding; Cold-start problem; Question answering services

Citation Formats

Sun, Jiankai, Vishnu, Abhinav, Chakrabarti, Aniket, Siegel, Charles, and Parthasarathy, Srinivasan. ColdRoute: effective routing of cold questions in stack exchange sites. United States: N. p., 2018. Web. doi:10.1007/s10618-018-0577-7.
Sun, Jiankai, Vishnu, Abhinav, Chakrabarti, Aniket, Siegel, Charles, & Parthasarathy, Srinivasan. ColdRoute: effective routing of cold questions in stack exchange sites. United States. doi:https://doi.org/10.1007/s10618-018-0577-7
Sun, Jiankai, Vishnu, Abhinav, Chakrabarti, Aniket, Siegel, Charles, and Parthasarathy, Srinivasan. Fri . "ColdRoute: effective routing of cold questions in stack exchange sites". United States. doi:https://doi.org/10.1007/s10618-018-0577-7. https://www.osti.gov/servlets/purl/1460547.
@article{osti_1460547,
title = {ColdRoute: effective routing of cold questions in stack exchange sites},
author = {Sun, Jiankai and Vishnu, Abhinav and Chakrabarti, Aniket and Siegel, Charles and Parthasarathy, Srinivasan},
abstractNote = {Routing questions in Community Question Answer services such as Stack Exchange sites is a well-studied problem. Yet, cold-start—a phenomena observed when a new question is posted is not well addressed by existing approaches. Additionally, cold questions posted by new askers present significant challenges to state-of-the-art approaches. We propose ColdRoute to address these challenges. ColdRoute is able to handle the task of routing cold questions posted by new or existing askers to matching experts. Specifically, we use Factorization Machines on the one-hot encoding of critical features such as question tags and compare our approach to well-studied techniques such as CQARank and semantic matching (LDA, BoW, and Doc2Vec). Furthermore by using data from eight stack exchange sites, we are able to improve upon the routing metrics (Precision@1, Accuracy, MRR) over the state-of-the-art models such as semantic matching by 159.5, 31.84, and 40.36% for cold questions posted by existing askers, and 123.1, 27.03, and 34.81% for cold questions posted by new askers respectively.},
doi = {10.1007/s10618-018-0577-7},
journal = {Data Mining and Knowledge Discovery},
number = 5,
volume = 32,
place = {United States},
year = {2018},
month = {6}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Adapting vector space model to ranking-based collaborative filtering
conference, January 2012

  • Wang, Shuaiqiang; Sun, Jiankai; Gao, Byron J.
  • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
  • DOI: 10.1145/2396761.2398458

Expert Finding for Question Answering via Graph Regularized Matrix Completion
journal, April 2015

  • Zhao, Zhou; Zhang, Lijun; He, Xiaofei
  • IEEE Transactions on Knowledge and Data Engineering, Vol. 27, Issue 4
  • DOI: 10.1109/TKDE.2014.2356461

CQArank: jointly model topics and expertise in community question answering
conference, January 2013

  • Yang, Liu; Qiu, Minghui; Gottipati, Swapna
  • Proceedings of the 22nd ACM international conference on Conference on information & knowledge management - CIKM '13
  • DOI: 10.1145/2505515.2505720

The anatomy of a large-scale social search engine
conference, January 2010

  • Horowitz, Damon; Kamvar, Sepandar D.
  • Proceedings of the 19th international conference on World wide web - WWW '10
  • DOI: 10.1145/1772690.1772735

Competition-based networks for expert finding
conference, January 2013

  • Aslay, Çiğdem; O'Hare, Neil; Aiello, Luca Maria
  • Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '13
  • DOI: 10.1145/2484028.2484183

node2vec: Scalable Feature Learning for Networks
conference, January 2016

  • Grover, Aditya; Leskovec, Jure
  • Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '16
  • DOI: 10.1145/2939672.2939754

Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora
conference, January 2009

  • Ramage, Daniel; Hall, David; Nallapati, Ramesh
  • Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 1 - EMNLP '09
  • DOI: 10.3115/1699510.1699543

Predicting Best Answerers for New Questions: An Approach Leveraging Distributed Representations of Words in Community Question Answering
conference, August 2015

  • Dong, Hualei; Wang, Jian; Lin, Hongfei
  • 2015 Ninth International Conference on Frontier of Computer Science and Technology (FCST)
  • DOI: 10.1109/FCST.2015.56

When relevance is not enough: promoting diversity and freshness in personalized question recommendation
conference, January 2013

  • Szpektor, Idan; Maarek, Yoelle; Pelleg, Dan
  • Proceedings of the 22nd international conference on World Wide Web - WWW '13
  • DOI: 10.1145/2488388.2488497

Modeling problem difficulty and expertise in stackoverflow
conference, January 2012

  • Hanrahan, Benjamin V.; Convertino, Gregorio; Nelson, Les
  • Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion - CSCW '12
  • DOI: 10.1145/2141512.2141550

Dual role model for question recommendation in community question answering
conference, January 2012

  • Xu, Fei; Ji, Zongcheng; Wang, Bin
  • Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12
  • DOI: 10.1145/2348283.2348387

Exploring user expertise and descriptive ability in community question answering
conference, August 2014

  • Yang, Baoguo; Manandhar, Suresh
  • 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014)
  • DOI: 10.1109/ASONAM.2014.6921604

Learning to Recommend Accurate and Diverse Items
conference, January 2017

  • Cheng, Peizhe; Wang, Shuaiqiang; Ma, Jun
  • Proceedings of the 26th International Conference on World Wide Web - WWW '17
  • DOI: 10.1145/3038912.3052585

Topic-sensitive probabilistic model for expert finding in question answer communities
conference, January 2012

  • Zhou, Guangyou; Lai, Siwei; Liu, Kang
  • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
  • DOI: 10.1145/2396761.2398493

Breaking Cycles In Noisy Hierarchies
conference, January 2017

  • Sun, Jiankai; Ajwani, Deepak; Nicholson, Patrick K.
  • Proceedings of the 2017 ACM on Web Science Conference - WebSci '17
  • DOI: 10.1145/3091478.3091495

Factorization Machines
conference, December 2010

  • Rendle, Steffen
  • 2010 IEEE 10th International Conference on Data Mining (ICDM), 2010 IEEE International Conference on Data Mining
  • DOI: 10.1109/ICDM.2010.127

Learning to rank for hybrid recommendation
conference, January 2012

  • Sun, Jiankai; Wang, Shuaiqiang; Gao, Byron J.
  • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
  • DOI: 10.1145/2396761.2398610

Towards expert finding by leveraging relevant categories in authority ranking
conference, January 2011

  • Zhu, Hengshu; Cao, Huanhuan; Xiong, Hui
  • Proceedings of the 20th ACM international conference on Information and knowledge management - CIKM '11
  • DOI: 10.1145/2063576.2063931

Mining Shapes of Expertise in Online Social Q&A Communities
conference, January 2016

  • Kumar, Varun; Pedanekar, Niranjan E.
  • Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion - CSCW '16 Companion
  • DOI: 10.1145/2818052.2869096

Question-answer topic model for question retrieval in community question answering
conference, January 2012

  • Ji, Zongcheng; Xu, Fei; Wang, Bin
  • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
  • DOI: 10.1145/2396761.2398669

A Regularized Competition Model for Question Difficulty Estimation in Community Question Answering Services
conference, January 2014

  • Wang, Quan; Liu, Jing; Wang, Bin
  • Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
  • DOI: 10.3115/v1/D14-1118

SocialTransfer: Transferring Social Knowledge for Cold-Start Cowdsourcing
conference, January 2014

  • Zhao, Zhou; Cheng, James; Wei, Furu
  • Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management - CIKM '14
  • DOI: 10.1145/2661829.2661871

Scalable Coordinate Descent Approaches to Parallel Matrix Factorization for Recommender Systems
conference, December 2012

  • Yu, Hsiang-Fu; Hsieh, Cho-Jui; Si, Si
  • 2012 IEEE 12th International Conference on Data Mining (ICDM)
  • DOI: 10.1109/ICDM.2012.168

Ranking user authority with relevant knowledge categories for expert finding
journal, April 2013


Identifying authoritative actors in question-answering forums: the case of Yahoo! answers
conference, January 2008

  • Bouguessa, Mohamed; Dumoulin, Benoît; Wang, Shengrui
  • Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08
  • DOI: 10.1145/1401890.1401994

Competition-based user expertise score estimation
conference, January 2011

  • Liu, Jing; Song, Young-In; Lin, Chin-Yew
  • Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR '11
  • DOI: 10.1145/2009916.2009975

Discovering value from community activity on focused question answering sites: a case study of stack overflow
conference, January 2012

  • Anderson, Ashton; Huttenlocher, Daniel; Kleinberg, Jon
  • Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '12
  • DOI: 10.1145/2339530.2339665

Social Recommendation with Cross-Domain Transferable Knowledge
journal, November 2015

  • Jiang, Meng; Cui, Peng; Chen, Xumin
  • IEEE Transactions on Knowledge and Data Engineering, Vol. 27, Issue 11
  • DOI: 10.1109/TKDE.2015.2432811

Routing questions to appropriate answerers in community question answering services
conference, January 2010

  • Li, Baichuan; King, Irwin
  • Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10
  • DOI: 10.1145/1871437.1871678

Hierarchies in Directed Networks
conference, November 2015


Expertise networks in online communities: structure and algorithms
conference, January 2007

  • Zhang, Jun; Ackerman, Mark S.; Adamic, Lada
  • Proceedings of the 16th international conference on World Wide Web - WWW '07
  • DOI: 10.1145/1242572.1242603

A Comprehensive Survey and Classification of Approaches for Community Question Answering
journal, August 2016

  • Srba, Ivan; Bielikova, Maria
  • ACM Transactions on the Web, Vol. 10, Issue 3
  • DOI: 10.1145/2934687

Summarizing Answers in Non-Factoid Community Question-Answering
conference, January 2017

  • Song, Hongya; Ren, Zhaochun; Liang, Shangsong
  • Proceedings of the Tenth ACM International Conference on Web Search and Data Mining - WSDM '17
  • DOI: 10.1145/3018661.3018704

Question routing in community question answering: putting category in its place
conference, January 2011

  • Li, Baichuan; King, Irwin; Lyu, Michael R.
  • Proceedings of the 20th ACM international conference on Information and knowledge management - CIKM '11
  • DOI: 10.1145/2063576.2063885

Topic-Level Expert Modeling in Community Question Answering
conference, December 2013

  • Zhao, Tong; Bian, Naiwen; Li, Chunping
  • Proceedings of the 2013 SIAM International Conference on Data Mining
  • DOI: 10.1137/1.9781611972832.86

Ad click prediction: a view from the trenches
conference, January 2013

  • McMahan, H. Brendan; Golovin, Daniel; Chikkerur, Sharat
  • Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '13
  • DOI: 10.1145/2487575.2488200

Factorization Machines with libFM
journal, May 2012

  • Rendle, Steffen
  • ACM Transactions on Intelligent Systems and Technology, Vol. 3, Issue 3
  • DOI: 10.1145/2168752.2168771

Tapping on the potential of q&a community by recommending answer providers
conference, January 2008

  • Guo, Jinwen; Xu, Shengliang; Bao, Shenghua
  • Proceeding of the 17th ACM conference on Information and knowledge mining - CIKM '08
  • DOI: 10.1145/1458082.1458204

A classification-based approach to question routing in community question answering
conference, January 2012

  • Zhou, Tom Chao; Lyu, Michael R.; King, Irwin
  • Proceedings of the 21st international conference companion on World Wide Web - WWW '12 Companion
  • DOI: 10.1145/2187980.2188201

    Works referencing / citing this record:

    Ranking user authority with relevant knowledge categories for expert finding
    journal, April 2013


    Topic-Level Expert Modeling in Community Question Answering
    conference, December 2013

    • Zhao, Tong; Bian, Naiwen; Li, Chunping
    • Proceedings of the 2013 SIAM International Conference on Data Mining
    • DOI: 10.1137/1.9781611972832.86

    Expertise networks in online communities: structure and algorithms
    conference, January 2007

    • Zhang, Jun; Ackerman, Mark S.; Adamic, Lada
    • Proceedings of the 16th international conference on World Wide Web - WWW '07
    • DOI: 10.1145/1242572.1242603

    Identifying authoritative actors in question-answering forums: the case of Yahoo! answers
    conference, January 2008

    • Bouguessa, Mohamed; Dumoulin, Benoît; Wang, Shengrui
    • Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08
    • DOI: 10.1145/1401890.1401994

    Tapping on the potential of q&a community by recommending answer providers
    conference, January 2008

    • Guo, Jinwen; Xu, Shengliang; Bao, Shenghua
    • Proceeding of the 17th ACM conference on Information and knowledge mining - CIKM '08
    • DOI: 10.1145/1458082.1458204

    The anatomy of a large-scale social search engine
    conference, January 2010

    • Horowitz, Damon; Kamvar, Sepandar D.
    • Proceedings of the 19th international conference on World wide web - WWW '10
    • DOI: 10.1145/1772690.1772735

    Routing questions to appropriate answerers in community question answering services
    conference, January 2010

    • Li, Baichuan; King, Irwin
    • Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10
    • DOI: 10.1145/1871437.1871678

    Competition-based user expertise score estimation
    conference, January 2011

    • Liu, Jing; Song, Young-In; Lin, Chin-Yew
    • Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR '11
    • DOI: 10.1145/2009916.2009975

    Question routing in community question answering: putting category in its place
    conference, January 2011

    • Li, Baichuan; King, Irwin; Lyu, Michael R.
    • Proceedings of the 20th ACM international conference on Information and knowledge management - CIKM '11
    • DOI: 10.1145/2063576.2063885

    Towards expert finding by leveraging relevant categories in authority ranking
    conference, January 2011

    • Zhu, Hengshu; Cao, Huanhuan; Xiong, Hui
    • Proceedings of the 20th ACM international conference on Information and knowledge management - CIKM '11
    • DOI: 10.1145/2063576.2063931

    Modeling problem difficulty and expertise in stackoverflow
    conference, January 2012

    • Hanrahan, Benjamin V.; Convertino, Gregorio; Nelson, Les
    • Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion - CSCW '12
    • DOI: 10.1145/2141512.2141550

    Factorization Machines with libFM
    journal, May 2012

    • Rendle, Steffen
    • ACM Transactions on Intelligent Systems and Technology, Vol. 3, Issue 3
    • DOI: 10.1145/2168752.2168771

    A classification-based approach to question routing in community question answering
    conference, January 2012

    • Zhou, Tom Chao; Lyu, Michael R.; King, Irwin
    • Proceedings of the 21st international conference companion on World Wide Web - WWW '12 Companion
    • DOI: 10.1145/2187980.2188201

    Dual role model for question recommendation in community question answering
    conference, January 2012

    • Xu, Fei; Ji, Zongcheng; Wang, Bin
    • Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12
    • DOI: 10.1145/2348283.2348387

    Adapting vector space model to ranking-based collaborative filtering
    conference, January 2012

    • Wang, Shuaiqiang; Sun, Jiankai; Gao, Byron J.
    • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
    • DOI: 10.1145/2396761.2398458

    Topic-sensitive probabilistic model for expert finding in question answer communities
    conference, January 2012

    • Zhou, Guangyou; Lai, Siwei; Liu, Kang
    • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
    • DOI: 10.1145/2396761.2398493

    Learning to rank for hybrid recommendation
    conference, January 2012

    • Sun, Jiankai; Wang, Shuaiqiang; Gao, Byron J.
    • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
    • DOI: 10.1145/2396761.2398610

    Question-answer topic model for question retrieval in community question answering
    conference, January 2012

    • Ji, Zongcheng; Xu, Fei; Wang, Bin
    • Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12
    • DOI: 10.1145/2396761.2398669

    Competition-based networks for expert finding
    conference, January 2013

    • Aslay, Çiğdem; O'Hare, Neil; Aiello, Luca Maria
    • Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '13
    • DOI: 10.1145/2484028.2484183

    Ad click prediction: a view from the trenches
    conference, January 2013

    • McMahan, H. Brendan; Golovin, Daniel; Chikkerur, Sharat
    • Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '13
    • DOI: 10.1145/2487575.2488200

    When relevance is not enough: promoting diversity and freshness in personalized question recommendation
    conference, January 2013

    • Szpektor, Idan; Maarek, Yoelle; Pelleg, Dan
    • Proceedings of the 22nd international conference on World Wide Web - WWW '13
    • DOI: 10.1145/2488388.2488497

    CQArank: jointly model topics and expertise in community question answering
    conference, January 2013

    • Yang, Liu; Qiu, Minghui; Gottipati, Swapna
    • Proceedings of the 22nd ACM international conference on Conference on information & knowledge management - CIKM '13
    • DOI: 10.1145/2505515.2505720

    SocialTransfer: Transferring Social Knowledge for Cold-Start Cowdsourcing
    conference, January 2014

    • Zhao, Zhou; Cheng, James; Wei, Furu
    • Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management - CIKM '14
    • DOI: 10.1145/2661829.2661871

    Mining Shapes of Expertise in Online Social Q&A Communities
    conference, January 2016

    • Kumar, Varun; Pedanekar, Niranjan E.
    • Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion - CSCW '16 Companion
    • DOI: 10.1145/2818052.2869096

    A Comprehensive Survey and Classification of Approaches for Community Question Answering
    journal, August 2016

    • Srba, Ivan; Bielikova, Maria
    • ACM Transactions on the Web, Vol. 10, Issue 3
    • DOI: 10.1145/2934687

    Summarizing Answers in Non-Factoid Community Question-Answering
    conference, January 2017

    • Song, Hongya; Ren, Zhaochun; Liang, Shangsong
    • Proceedings of the Tenth ACM International Conference on Web Search and Data Mining - WSDM '17
    • DOI: 10.1145/3018661.3018704

    Learning to Recommend Accurate and Diverse Items
    conference, January 2017

    • Cheng, Peizhe; Wang, Shuaiqiang; Ma, Jun
    • Proceedings of the 26th International Conference on World Wide Web - WWW '17
    • DOI: 10.1145/3038912.3052585

    Breaking Cycles In Noisy Hierarchies
    conference, January 2017

    • Sun, Jiankai; Ajwani, Deepak; Nicholson, Patrick K.
    • Proceedings of the 2017 ACM on Web Science Conference - WebSci '17
    • DOI: 10.1145/3091478.3091495

    Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora
    conference, January 2009

    • Ramage, Daniel; Hall, David; Nallapati, Ramesh
    • Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 1 - EMNLP '09
    • DOI: 10.3115/1699510.1699543