Modeling household online shopping demand in the U.S.: a machine learning approach and comparative investigation between 2009 and 2017

Barua, Limon; Zou, Bo; Zhou, Yan; Liu, Yulin

doi:10.1007/s11116-021-10250-z

Title: Modeling household online shopping demand in the U.S.: a machine learning approach and comparative investigation between 2009 and 2017

Journal Article · Thu Dec 02 00:00:00 EST 2021 · Transportation

DOI:https://doi.org/10.1007/s11116-021-10250-z· OSTI ID:1969080

Barua, Limon ^[1];

^[2]; Zhou, Yan ^[3]; Liu, Yulin ^[4]

Univ. of Illinois, Chicago, IL (United States)
Univ. of Illinois, Chicago, IL (United States); Univ. of California, Berkeley, CA (United States)
Argonne National Lab. (ANL), Lemont, IL (United States)
Univ. of California, Berkeley, CA (United States)

Despite the rapid growth of online shopping and research interest in the relationship between online and in-store shopping, national-level modeling and investigation of the demand for online shopping with a prediction focus remain limited in the literature. Here, this paper differs from prior work and leverages two recent releases of the U.S. National Household Travel Survey (NHTS) data for 2009 and 2017 to develop machine learning (ML) models, specifically gradient boosting machine (GBM), for predicting household-level online shopping purchases. The NHTS data allow for not only conducting nationwide investigation but also at the level of households, which is more appropriate than at the individual level given the connected consumption and shopping needs of members in a household. We follow a systematic procedure for model development including employing Recursive Feature Elimination algorithm to select input variables (features) in order to reduce the risk of model overfitting and increase model explainability. Among several ML models, GBM is found to yield the best prediction accuracy. Extensive post-modeling investigation is conducted in a comparative manner between 2009 and 2017, including quantifying the importance of each input variable in predicting online shopping demand, and characterizing value-dependent relationships between demand and the input variables. In doing so, two latest advances in machine learning techniques, namely Shapley value-based feature importance and Accumulated Local Effects plots, are adopted to overcome inherent drawbacks of the popular techniques in current ML modeling. The modeling and investigation are performed at the national level, with a number of findings obtained. The models developed and insights gained can be used for online shopping-related freight demand generation and may also be considered for evaluating the potential impact of relevant policies on online shopping demand.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE Office of Energy Efficiency and Renewable Energy (EERE), Office of Sustainable Transportation. Vehicle Technologies Office (VTO); National Science Foundation (NSF)

Grant/Contract Number:: AC02-06CH11357; 1663411

OSTI ID:: 1969080

Journal Information:: Transportation, Vol. 50, Issue 2; ISSN 0049-4488

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (73)

Deep Reinforcement Learning for Crowdsourced Urban Delivery Ahamed, Tanvir; Zou, Bo; Farazi, Nahid Parvez Transportation Research Part B: Methodological, Vol. 152 https://doi.org/10.1016/j.trb.2021.08.015	journal	October 2021
Process Variable Importance Analysis by Use of Random Forests in a Shapley Regression Framework Aldrich, Chris Minerals, Vol. 10, Issue 5 https://doi.org/10.3390/min10050420	journal	May 2020
Visualizing the effects of predictor variables in black box supervised learning models Apley, Daniel W.; Zhu, Jingyu Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 82, Issue 4 https://doi.org/10.1111/rssb.12377	journal	June 2020
Machine learning for international freight transportation management: A comprehensive review Barua, Limon; Zou, Bo; Zhou, Yan Research in Transportation Business & Management, Vol. 34 https://doi.org/10.1016/j.rtbm.2020.100453	journal	March 2020
A gradient boosting approach to understanding airport runway and taxiway pavement deterioration Barua, Limon; Zou, Bo; Noruzoliaee, Mohamadhossein International Journal of Pavement Engineering https://doi.org/10.1080/10298436.2020.1714616	journal	January 2020
Planning maintenance and rehabilitation activities for airport pavements: A combined supervised machine learning and reinforcement learning approach Barua, Limon; Zou, Bo International Journal of Transportation Science and Technology, Vol. 11, Issue 2 https://doi.org/10.1016/j.ijtst.2021.05.006	journal	June 2022
Transforming Last-mile Logistics Bates, Oliver; Friday, Adrian; Allen, Julian Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems https://doi.org/10.1145/3173574.3174100	conference	April 2018
Carbon emissions comparison of last mile delivery versus customer pickup Brown, Jay R.; Guiffrida, Alfred L. International Journal of Logistics Research and Applications, Vol. 17, Issue 6 https://doi.org/10.1080/13675567.2014.907397	journal	April 2014
Delivering supermarket shopping: more or less traffic? Cairns, Sally Transport Reviews, Vol. 25, Issue 1 https://doi.org/10.1080/0144164042000218391	journal	January 2005
The interactions between e-shopping and traditional in-store shopping: an application of structural equations model Cao, Xinyu Jason; Xu, Zhiyi; Douma, Frank Transportation, Vol. 39, Issue 5 https://doi.org/10.1007/s11116-011-9376-3	journal	October 2011
E-Shopping, Spatial Attributes, and Personal Travel: A Review of Empirical Studies Cao, Xinyu (Jason) Transportation Research Record: Journal of the Transportation Research Board, Vol. 2135, Issue 1 https://doi.org/10.3141/2135-19	journal	January 2009
Impact of drone delivery on sustainability and cost: Realizing the UAV potential through vehicle routing optimization Chiang, Wen-Chyuan; Li, Yuyu; Shang, Jennifer Applied Energy, Vol. 242 https://doi.org/10.1016/j.apenergy.2019.03.117	journal	May 2019
Advanced freight transportation systems for congested urban areas Crainic, Teodor Gabriel; Ricciardi, Nicoletta; Storchi, Giovanni Transportation Research Part C: Emerging Technologies, Vol. 12, Issue 2 https://doi.org/10.1016/j.trc.2004.07.002	journal	April 2004
Community Size and Social Relationships: A Comparison of Urban and Rural Social Patterns in Tirol Crowe, Patricia Ward Anthropological Quarterly, Vol. 54, Issue 4 https://doi.org/10.2307/3317236	journal	October 1981
Age related differences in learning to use a text-editing system Czaja, Sara J.; Hammond, Katka; Blascovich, James J. Behaviour & Information Technology, Vol. 8, Issue 4 https://doi.org/10.1080/01449298908914562	journal	August 1989
A comparison of online and in-person activity engagement: The case of shopping and eating meals Dias, Felipe F.; Lavieri, Patricia S.; Sharda, Shivam Transportation Research Part C: Emerging Technologies, Vol. 114 https://doi.org/10.1016/j.trc.2020.02.023	journal	May 2020
Applying gradient boosting decision trees to examine non-linear effects of the built environment on driving distance in Oslo Ding, Chuan; Cao, Xinyu (Jason); Næss, Petter Transportation Research Part A: Policy and Practice, Vol. 110 https://doi.org/10.1016/j.tra.2018.02.009	journal	April 2018
The interactions between online shopping and personal activity travel behavior: an analysis with a GPS-based activity travel diary Ding, Yu; Lu, Huapu Transportation, Vol. 44, Issue 2 https://doi.org/10.1007/s11116-015-9639-5	journal	July 2015
A working guide to boosted regression trees Elith, J.; Leathwick, J. R.; Hastie, T. Journal of Animal Ecology, Vol. 77, Issue 4 https://doi.org/10.1111/j.1365-2656.2008.01390.x	journal	July 2008
Exploring the Use of E-Shopping and Its Impact on Personal Travel Behavior in the Netherlands Farag, Sendy; Dijst, Martin; Lanzendorf, Martin Transportation Research Record: Journal of the Transportation Research Board, Vol. 1858, Issue 1 https://doi.org/10.3141/1858-07	journal	January 2003
Shopping online and/or in-store? A structural equation model of the relationships between e-shopping and in-store shopping Farag, Sendy; Schwanen, Tim; Dijst, Martin Transportation Research Part A: Policy and Practice, Vol. 41, Issue 2 https://doi.org/10.1016/j.tra.2006.02.003	journal	February 2007
Empirical Investigation of Online Searching and Buying and Their Relationship to Shopping Trips Farag, Sendy; Schwanen, Tim; Dijst, Martin Transportation Research Record: Journal of the Transportation Research Board, Vol. 1926, Issue 1 https://doi.org/10.1177/0361198105192600128	journal	January 2005
Home-Based Teleshoppers and Shopping Travel: Do Teleshoppers Travel Less? Ferrell, Christopher E. Transportation Research Record: Journal of the Transportation Research Board, Vol. 1894, Issue 1 https://doi.org/10.3141/1894-25	journal	January 2004
Home-Based Teleshopping and Shopping Travel: Where Do People Find the Time? Ferrell, Christopher E. Transportation Research Record: Journal of the Transportation Research Board, Vol. 1926, Issue 1 https://doi.org/10.1177/0361198105192600125	journal	January 2005
machine. Friedman, Jerome H. The Annals of Statistics, Vol. 29, Issue 5 https://doi.org/10.1214/aos/1013203451	journal	October 2001
Urban Resurgence and the Consumer City Glaeser, Edward L.; Gottlieb, Joshua D. Urban Studies, Vol. 43, Issue 8 https://doi.org/10.1080/00420980600775683	journal	July 2006
Selecting the most important self-assessed features for predicting conversion to mild cognitive impairment with random forest and permutation-based methods Gómez-Ramírez, Jaime; Ávila-Villanueva, Marina; Fernández-Blázquez, Miguel Ángel Scientific Reports, Vol. 10, Issue 1 https://doi.org/10.1038/s41598-020-77296-4	journal	November 2020
Gene Selection for Cancer Classification using Support Vector Machines Guyon, Isabelle; Weston, Jason; Barnhill, Stephen Machine Learning, Vol. 46, Issue 1/3, p. 389-422 https://doi.org/10.1023/A:1012487302797	journal	January 2002
The Elements of Statistical Learning Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome Springer Series in Statistics https://doi.org/10.1007/978-0-387-84858-7	book	January 2009
Age, gender and income: do they really moderate online shopping behaviour? Hernández, Blanca; Jiménez, Julio; José Martín, M. Online Information Review, Vol. 35, Issue 1 https://doi.org/10.1108/14684521111113614	journal	February 2011
Crowdsourcing Incentives for Multi-Hop Urban Parcel Delivery Network Hong, Huiting; Li, Xin; He, Daqing IEEE Access, Vol. 7 https://doi.org/10.1109/ACCESS.2019.2896912	journal	January 2019
A Variable Impacts Measurement in Random Forest for Mobile Cloud Computing Hur, Jae-Hee; Ihm, Sun-Young; Park, Young-Ho Wireless Communications and Mobile Computing, Vol. 2017 https://doi.org/10.1155/2017/6817627	journal	January 2017
Evaluating the environmental impacts of online shopping: A behavioral and transportation approach Jaller, Miguel; Pahwa, Anmol Transportation Research Part D: Transport and Environment, Vol. 80 https://doi.org/10.1016/j.trd.2020.102223	journal	March 2020
Design and modeling of a crowdsource-enabled system for urban parcel relay and delivery Kafle, Nabin; Zou, Bo; Lin, Jane Transportation Research Part B: Methodological, Vol. 99 https://doi.org/10.1016/j.trb.2016.12.022	journal	May 2017
Drone-Based Parcel Delivery Using the Rooftops of City Buildings: Model and Solution Kim, Junsu; Moon, Hongbin; Jung, Hosang Applied Sciences, Vol. 10, Issue 12 https://doi.org/10.3390/app10124362	journal	June 2020
Is The Cost Of Living Less In Rural Areas? Kurre, James A. International Regional Science Review, Vol. 26, Issue 1 https://doi.org/10.1177/0160017602238987	journal	January 2003
Picture of Online Shoppers: Specific Focus on Davis, California Lee, Richard J.; Sener, Ipek N.; Handy, Susan L. Transportation Research Record: Journal of the Transportation Research Board, Vol. 2496, Issue 1 https://doi.org/10.3141/2496-07	journal	January 2015
Relationships between the online and in-store shopping frequency of Davis, California residents Lee, Richard J.; Sener, Ipek N.; Mokhtarian, Patricia L. Transportation Research Part A: Policy and Practice, Vol. 100 https://doi.org/10.1016/j.tra.2017.03.001	journal	June 2017
An Empirical Investigation of the Impact of Gasoline Prices on Grocery Shopping Behavior Ma, Yu; Ailawadi, Kusum L.; Gauri, Dinesh K. Journal of Marketing, Vol. 75, Issue 2 https://doi.org/10.1509/jm.75.2.18	journal	March 2011
Accessibility or Innovation? Store Shopping Trips versus Online Shopping Maat, Kees; Konings, Rob Transportation Research Record: Journal of the Transportation Research Board, Vol. 2672, Issue 50 https://doi.org/10.1177/0361198118794044	journal	October 2018
On-line Shopping Behavior: Cross-Country Empirical Research Mahmood, M. Adam; Bagchi, Kallol; Ford, Timothy C. International Journal of Electronic Commerce, Vol. 9, Issue 1 https://doi.org/10.1080/10864415.2004.11044321	journal	October 2004
Telecommunications and Travel: The Case for Complementarity Mokhtarian, Patricia L. Journal of Industrial Ecology, Vol. 6, Issue 2 https://doi.org/10.1162/108819802763471771	journal	February 2008
The Gender Gap in Internet Use: Why Men Use the Internet More Than Women—A Literature Review Morahan-Martin, Janet CyberPsychology & Behavior, Vol. 1, Issue 1 https://doi.org/10.1089/cpb.1998.1.3	journal	January 1998
Age Differences in Technology Adoption Decisions: Implications for a Changing work Force Morris, Michael G.; Venkatesh, Viswanath Personnel Psychology, Vol. 53, Issue 2 https://doi.org/10.1111/j.1744-6570.2000.tb00206.x	journal	June 2000
Gradient boosting machines, a tutorial Natekin, Alexey; Knoll, Alois Frontiers in Neurorobotics, Vol. 7 https://doi.org/10.3389/fnbot.2013.00021	journal	January 2013
A two-step method to evaluate the Well-To-Wheel carbon efficiency of Urban Consolidation Centres Nocera, Silvio; Cavallaro, Federico Research in Transportation Economics, Vol. 65 https://doi.org/10.1016/j.retrec.2017.04.001	journal	October 2017
A comparison of random forests, boosting and support vector machines for genomic selection Ogutu, Joseph O.; Piepho, Hans-Peter; Schulz-Streeck, Torben BMC Proceedings, Vol. 5, Issue S3 https://doi.org/10.1186/1753-6561-5-S3-S11	journal	May 2011
Electronic commerce sales' response to gasoline price Ramcharran, Harri International Journal of Electronic Finance, Vol. 7, Issue 3/4 https://doi.org/10.1504/IJEF.2013.058603	journal	January 2013
Proactive vehicle routing with inferred demand to solve the bikesharing rebalancing problem Regue, Robert; Recker, Will Transportation Research Part E: Logistics and Transportation Review, Vol. 72 https://doi.org/10.1016/j.tre.2014.10.005	journal	December 2014
The Impact of Geographic Context on E-Shopping Behavior Ren, Fang; Kwan, Mei-Po Environment and Planning B: Planning and Design, Vol. 36, Issue 2 https://doi.org/10.1068/b34014t	journal	January 2009
The distribution network of Amazon and the footprint of freight digitalization Rodrigue, Jean-Paul Journal of Transport Geography, Vol. 88 https://doi.org/10.1016/j.jtrangeo.2020.102825	journal	October 2020
Online shopping habits and the potential for reductions in carbon dioxide emissions from passenger transport Smidfelt Rosqvist, Lena; Winslott Hiselius, Lena Journal of Cleaner Production, Vol. 131 https://doi.org/10.1016/j.jclepro.2016.05.054	journal	September 2016
Does e-shopping replace shopping trips? Empirical evidence from Chengdu, China Shi, Kunbo; De Vos, Jonas; Yang, Yongchun Transportation Research Part A: Policy and Practice, Vol. 122 https://doi.org/10.1016/j.tra.2019.01.027	journal	April 2019
Neighbourhood food environment and area deprivation: spatial accessibility to grocery stores selling fresh fruit and vegetables in urban and rural settings Smith, D. M.; Cummins, S.; Taylor, M. International Journal of Epidemiology, Vol. 39, Issue 1 https://doi.org/10.1093/ije/dyp221	journal	June 2009
Unbiased split selection for classification trees based on the Gini Index Strobl, Carolin; Boulesteix, Anne-Laure; Augustin, Thomas Computational Statistics & Data Analysis, Vol. 52, Issue 1 https://doi.org/10.1016/j.csda.2006.12.030	journal	September 2007
A phenomenological investigation of Internet usage among older individuals Trocchia, Philip J.; Janda, Swinder Journal of Consumer Marketing, Vol. 17, Issue 7 https://doi.org/10.1108/07363760010357804	journal	December 2000
Social Interaction Location Choice: A Latent Class Modeling Approach van den Berg, Pauline; Kemperman, Astrid; Timmermans, Harry Annals of the Association of American Geographers, Vol. 104, Issue 5 https://doi.org/10.1080/00045608.2014.924726	journal	July 2014
Why Don't Men Ever Stop to Ask for Directions? Gender, Social Influence, and Their Role in Technology Acceptance and Usage Behavior Venkatesh, Viswanath; Morris, Michael G. MIS Quarterly, Vol. 24, Issue 1 https://doi.org/10.2307/3250981	journal	March 2000
Mobility and Accessibility Effects of b2c E-Commerce: a Literature Review Visser, Evert-Jan; Lanzendorf, Martin Tijdschrift voor economische en sociale geografie, Vol. 95, Issue 2 https://doi.org/10.1111/j.0040-747X.2004.00300.x	journal	March 2004
Deliveries to residential units: A rising form of freight transportation in the U.S. Wang, Xiaokun (Cara); Zhou, Yiwei Transportation Research Part C: Emerging Technologies, Vol. 58 https://doi.org/10.1016/j.trc.2015.07.004	journal	September 2015
E-Shopping Versus city Centre Shopping: the role of Perceived city Centre Attractiveness Weltevreden, Jesse W. J.; Rietbergen, Ton Van Tijdschrift voor Economische en Sociale Geografie, Vol. 98, Issue 1 https://doi.org/10.1111/j.1467-9663.2007.00377.x	journal	February 2007
Shopping Online for Freedom, Control, and Fun Wolfinbarger, Mary; Gilly, Mary C. California Management Review, Vol. 43, Issue 2 https://doi.org/10.2307/41166074	journal	January 2001
The interaction between e-shopping and store shopping: empirical evidence from Nanjing, China Xi, Guangliang; Zhen, Feng; Cao, Xinyu (Jason) Transportation Letters, Vol. 12, Issue 3 https://doi.org/10.1080/19427867.2018.1546797	journal	November 2018
Nonlinear feature selection using Gaussian kernel SVM-RFE for fault diagnosis Xue, Yangtao; Zhang, Li; Wang, Bangjun Applied Intelligence, Vol. 48, Issue 10 https://doi.org/10.1007/s10489-018-1140-3	journal	February 2018
The interactions between e-shopping and store shopping in the shopping process for search goods and experience goods Zhai, Qing; Cao, Xinyu; Mokhtarian, Patricia L. Transportation, Vol. 44, Issue 5 https://doi.org/10.1007/s11116-016-9683-9	journal	February 2016
Satellite-based ground PM2.5 estimation using a gradient boosting decision tree Zhang, Tianning; He, Weihuan; Zheng, Hui Chemosphere, Vol. 268 https://doi.org/10.1016/j.chemosphere.2020.128801	journal	April 2021
A gradient boosting method to improve travel time prediction Zhang, Yanru; Haghani, Ali Transportation Research Part C: Emerging Technologies, Vol. 58 https://doi.org/10.1016/j.trc.2015.02.019	journal	September 2015
Causal Interpretations of Black-Box Models Zhao, Qingyuan; Hastie, Trevor Journal of Business & Economic Statistics, Vol. 39, Issue 1 https://doi.org/10.1080/07350015.2019.1624293	journal	July 2019
The association between spatial attributes and e-shopping in the shopping process for search goods and experience goods: Evidence from Nanjing Zhen, Feng; Du, Xiaojuan; Cao, Jason Journal of Transport Geography, Vol. 66 https://doi.org/10.1016/j.jtrangeo.2017.11.007	journal	January 2018
Explore the relationship between online shopping and shopping trips: An analysis with the 2009 NHTS data Zhou, Yiwei; Wang, Xiaokun (Cara) Transportation Research Part A: Policy and Practice, Vol. 70 https://doi.org/10.1016/j.tra.2014.09.014	journal	December 2014
High-Order Structure Exploration on Massive Graphs Zhou, Dawei; Zhang, Si; Yildirim, Mehmet Yigit ACM Transactions on Knowledge Discovery from Data, Vol. 15, Issue 2 https://doi.org/10.1145/3425637	journal	January 2021
Flight delays, capacity investment and social welfare under air transport supply-demand equilibrium Zou, Bo; Hansen, Mark Transportation Research Part A: Policy and Practice, Vol. 46, Issue 6 https://doi.org/10.1016/j.tra.2012.02.015	journal	July 2012
A Comparison of Online and In-Person Activity Engagement: The Case of Shopping and Eating Meals Dias, Felipe F.; Lavieri, Patricia S.; Sharda, Shivam The University of Texas at Austin https://doi.org/10.26153/tsw/8404	other	January 2020

Similar Records

Citywide Impacts of E-Commerce: Does Parcel Delivery Travel Outweigh Household Shopping Travel Reductions?

Conference · Tue Sep 10 00:00:00 EDT 2019 · OSTI ID:1969080

Stinson, Monique; Enam, Annesha; Moore, Amy; +1 more

Analytics-at-scale of Sensor Data for Digital Monitoring in Nuclear Plants (3^rd Annual Report)

Technical Report · Tue Feb 01 00:00:00 EST 2022 · OSTI ID:1969080

Walker, Cody McBroom; Agarwal, Vivek; Lybeck, Nancy J; +2 more

Development of Short-Term Forecasting Models Using Plant Asset Data and Feature Selection

Journal Article · Wed Jun 08 00:00:00 EDT 2022 · International Journal of Prognostics and Health Management · OSTI ID:1969080

Walker, Cody McBroom; Ramuhalli, Pradeep; Agarwal, Vivek; +2 more

Related Subjects

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION
online shopping demand
machine learning
gradient boosting machine (GBM)
prediction
National Household Travel Survey
Shapley value-based feature importance
accumulated local effects

Title: Modeling household online shopping demand in the U.S.: a machine learning approach and comparative investigation between 2009 and 2017

Citation Formats

References (73)

Similar Records

Related Subjects