Automated detection and sorting of microencapsulation via machine learning

Chu, Albert; Nguyen, Du; Talathi, Sachin S.; Wilson, Aaron C.; Ye, Congwang; Smith, William L.; Kaplan, Alan D.; Duoss, Eric B.; Stolaroff, Joshua K.; Giera, Brian

doi:10.1039/C8LC01394B

Title: Automated detection and sorting of microencapsulation via machine learning

Journal Article · 15 April 2019 · Lab on a Chip

DOI: https://doi.org/10.1039/C8LC01394B · OSTI ID:1507164

Chu, Albert ^[1]; Nguyen, Du ^[1]; Talathi, Sachin S. ^[1]; Wilson, Aaron C. ^[2]; Ye, Congwang ^[1]; Smith, William L. ^[1]; Kaplan, Alan D. ^[1]; Duoss, Eric B. ^[1]; Stolaroff, Joshua K. ^[1];

^[1]

Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Google, Inc., Mountain View, CA (United States)

Microfluidic-based microencapsulation requires significant oversight to prevent material and quality loss due to sporadic disruptions in fluid flow that routinely arise. State-of-the-art microcapsule production is laborious and relies on experts to monitor the process, e.g. through a microscope. Unnoticed defects diminish the quality of collected material and/or may cause irreversible clogging. To address these issues, we developed an automated monitoring and sorting system that operates on consumer-grade hardware in real-time. Using human-labeled microscope images acquired during typical operation, we train a convolutional neural network that assesses microencapsulation. Based on output from the machine learning algorithm, an integrated valving system collects desirable microcapsules or diverts waste material accordingly. Although the system notifies operators to make necessary adjustments to restore microencapsulation, we can extend the system to automate corrections. Since microfluidic-based production platforms customarily collect image and sensor data, machine learning can help to scale up and improve microfluidic techniques beyond microencapsulation.

View Journal Article

Cite

Export

Save

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC52-07NA27344; LDRD Exploratory Research 17-ERD-037; LLNL-JRNL-748383

OSTI ID:: 1507164

Alternate ID(s):: OSTI ID: 1568025

Report Number(s):: LLNL-JRNL-733470; LCAHAM; 885205

Journal Information:: Lab on a Chip, Vol. 19, Issue 10; ISSN 1473-0197

Publisher:: Royal Society of ChemistryCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 26 works

Citation information provided by
Web of Science

References (40)

Active droplet sorting in microfluidics: a review Xi, Heng-Dong; Zheng, Hao; Guo, Wei Lab on a Chip, Vol. 17, Issue 5 https://doi.org/10.1039/C6LC01435F	journal	January 2017
Microencapsulation of Dye- and Drug-Loaded Particles for Imaging and Controlled Release of Multiple Drugs Khung, Yit-Lung; Li Lee, Wei; Chui, Kit Leong Advanced Healthcare Materials, Vol. 1, Issue 2 https://doi.org/10.1002/adhm.201100007	journal	January 2012
CNC-loaded hydrogel particles generated from single- and double-emulsion drops Ye, Congwang; Kennedy, Lauren; Shirk, Kathryn Green Materials, Vol. 3, Issue 1 https://doi.org/10.1680/gmat.14.00016	journal	March 2015
Large-scale droplet production in microfluidic devices—an industrial perspective Holtze, Christian Journal of Physics D: Applied Physics, Vol. 46, Issue 11 https://doi.org/10.1088/0022-3727/46/11/114008	journal	February 2013
Fabrication of solid lipid microcapsules containing ascorbic acid using a microfluidic technique Comunian, Talita A.; Abbaspourrad, Alireza; Favaro-Trindade, Carmen S. Food Chemistry, Vol. 152 https://doi.org/10.1016/j.foodchem.2013.11.149	journal	June 2014
Applied Predictive Modeling Kuhn, Max; Johnson, Kjell https://doi.org/10.1007/978-1-4614-6849-3	book	January 2013
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Effective Formation of Silicone-in-Fluorocarbon-in-Water Double Emulsions: Studies on Droplet Morphology and Stability Lee, Doo-Hyun; Goh, Yeong-Mee; Kim, Joong-Soo Journal of Dispersion Science and Technology, Vol. 23, Issue 4 https://doi.org/10.1081/DIS-120014017	journal	September 2002
CO2 Absorption Kinetics of Micro-encapsulated Ionic Liquids Stolaroff, Joshuah K.; Ye, Congwang; Nguyen, Du T. Energy Procedia, Vol. 114 https://doi.org/10.1016/j.egypro.2017.03.1228	journal	July 2017
Optofluid-Based Reflective Displays Jin, Mingliang; Shen, Shitao; Yi, Zichuan Micromachines, Vol. 9, Issue 4 https://doi.org/10.3390/mi9040159	journal	April 2018
Nonviral gene vector formation in monodispersed picolitre incubator for consistent gene delivery Hsieh, Albert Tsung-Hsi; Hori, Nicole; Massoudi, Rustin Lab on a Chip, Vol. 9, Issue 18 https://doi.org/10.1039/b823191e	journal	January 2009
Novel Parallel Integration of Microfluidic Device Network for Emulsion Formation Tetradis-Meris, Georgios; Rossetti, Damiano; Pulido de Torres, Concepción Industrial & Engineering Chemistry Research, Vol. 48, Issue 19 https://doi.org/10.1021/ie900165b	journal	October 2009
Layer-by-Layer Assembled Nanocontainers for Self-Healing Corrosion Protection Shchukin, D. G.; Zheludkevich, M.; Yasakau, K. Advanced Materials, Vol. 18, Issue 13 https://doi.org/10.1002/adma.200502053	journal	July 2006
Microfluidics: Fluid physics at the nanoliter scale Squires, Todd M.; Quake, Stephen R. Reviews of Modern Physics, Vol. 77, Issue 3 https://doi.org/10.1103/RevModPhys.77.977	journal	October 2005
Microencapsulation of advanced solvents for carbon capture Stolaroff, Joshuah K.; Ye, Congwang; Oakdale, James S. Faraday Discussions, Vol. 192 https://doi.org/10.1039/C6FD00049E	journal	January 2016
Microfluidic Fabrication of Monodisperse Biocompatible and Biodegradable Polymersomes with Controlled Permeability Shum, Ho Cheung; Kim, Jin-Woong; Weitz, David A. Journal of the American Chemical Society, Vol. 130, Issue 29 https://doi.org/10.1021/ja802157y	journal	July 2008
Art on the Nanoscale and Beyond Yetisen, Ali K.; Coskun, Ahmet F.; England, Grant Advanced Materials, Vol. 28, Issue 9 https://doi.org/10.1002/adma.201502382	journal	December 2015
Double emulsion production in glass capillary microfluidic device: Parametric investigation of droplet generation behaviour Nabavi, Seyed Ali; Vladisavljević, Goran T.; Gu, Sai Chemical Engineering Science, Vol. 130 https://doi.org/10.1016/j.ces.2015.03.004	journal	July 2015
Microfluidic Melt Emulsification for Encapsulation and Release of Actives Sun, Bing Jie; Shum, Ho Cheung; Holtze, Christian ACS Applied Materials & Interfaces, Vol. 2, Issue 12 https://doi.org/10.1021/am100860b	journal	November 2010
Microencapsulation of theophylline in whey proteins: effects of core-to-wall ratio Lee, S. J.; Rosenberg, M. International Journal of Pharmaceutics, Vol. 205, Issue 1-2 https://doi.org/10.1016/S0378-5173(00)00504-4	journal	September 2000
Monodisperse Double Emulsions Generated from a Microcapillary Device Utada, A. S. Science, Vol. 308, Issue 5721 https://doi.org/10.1126/science.1109164	journal	April 2005
The present and future role of microfluidics in biomedical research Sackmann, Eric K.; Fulton, Anna L.; Beebe, David J. Nature, Vol. 507, Issue 7491 https://doi.org/10.1038/nature13118	journal	March 2014
High-volume production of single and compound emulsions in a microfluidic parallelization arrangement coupled with coaxial annular world-to-chip interfaces Nisisako, Takasi; Ando, Takuya; Hatsuzawa, Takeshi Lab on a Chip, Vol. 12, Issue 18 https://doi.org/10.1039/c2lc40245a	journal	January 2012
A review of microvalves Oh, Kwang W.; Ahn, Chong H. Journal of Micromechanics and Microengineering, Vol. 16, Issue 5 https://doi.org/10.1088/0960-1317/16/5/R01	journal	March 2006
Microfluidic impedance-based flow cytometry Cheung, Karen C.; Di Berardino, Marco; Schade-Kampmann, Grit Cytometry Part A, Vol. 77A, Issue 7 https://doi.org/10.1002/cyto.a.20910	journal	May 2010
A Survey on Transfer Learning Pan, Sinno Jialin; Yang, Qiang IEEE Transactions on Knowledge and Data Engineering, Vol. 22, Issue 10 https://doi.org/10.1109/TKDE.2009.191	journal	October 2010
Silicon and glass very large scale microfluidic droplet integration for terascale generation of polymer microparticles Yadavali, Sagar; Jeong, Heon-Ho; Lee, Daeyeon Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-03515-2	journal	March 2018
Microfluidic DNA microarray analysis: A review Wang, Lin; Li, Paul C. H. Analytica Chimica Acta, Vol. 687, Issue 1 https://doi.org/10.1016/j.aca.2010.11.056	journal	February 2011
Microfluidic systems for chemical kinetics that rely on chaotic mixing in droplets Bringer, Michelle R.; Gerdts, Cory J.; Song, Helen Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, Vol. 362, Issue 1818 https://doi.org/10.1098/rsta.2003.1364	journal	May 2004
Microfluidic platforms for lab-on-a-chip applications Haeberle, Stefan; Zengerle, Roland Lab on a Chip, Vol. 7, Issue 9 https://doi.org/10.1039/b706364b	journal	January 2007
Biosensors and their applications – A review Mehrotra, Parikha Journal of Oral Biology and Craniofacial Research, Vol. 6, Issue 2 https://doi.org/10.1016/j.jobcr.2015.12.002	journal	May 2016
NIH Image to ImageJ: 25 years of image analysis Schneider, Caroline A.; Rasband, Wayne S.; Eliceiri, Kevin W. Nature Methods, Vol. 9, Issue 7 https://doi.org/10.1038/nmeth.2089	journal	June 2012
REVIEW: Pregnancy tests: a review Chard, T. Human Reproduction, Vol. 7, Issue 5 https://doi.org/10.1093/oxfordjournals.humrep.a137722	journal	May 1992
Nanoencapsulation of phase change materials for advanced thermal energy storage systems Shchukina, E. M.; Graham, M.; Zheng, Z. Chemical Society Reviews, Vol. 47, Issue 11 https://doi.org/10.1039/C8CS00099A	journal	January 2018
Self-Healing Polymer Coatings Cho, Soo Hyoun; White, Scott R.; Braun, Paul V. Advanced Materials, Vol. 21, Issue 6 https://doi.org/10.1002/adma.200802008	journal	February 2009
Acceleration of a Diels–Alder reaction by a self-assembled molecular capsule Kang, Jongmin; Rebek, Julius Nature, Vol. 385, Issue 6611, p. 50-52 https://doi.org/10.1038/385050a0	journal	January 1997
Surface-Engineered Nanocontainers for Entrapment of Corrosion Inhibitors Shchukin, D. G.; Möhwald, H. Advanced Functional Materials, Vol. 17, Issue 9 https://doi.org/10.1002/adfm.200601226	journal	April 2007
Encapsulated liquid sorbents for carbon dioxide capture Vericella, John J.; Baker, Sarah E.; Stolaroff, Joshuah K. Nature Communications, Vol. 6, Issue 1 https://doi.org/10.1038/ncomms7124	journal	February 2015
High throughput production of single core double emulsions in a parallelized microfluidic device Romanowsky, Mark B.; Abate, Adam R.; Rotem, Assaf Lab on a Chip, Vol. 12, Issue 4 https://doi.org/10.1039/c2lc21033a	journal	January 2012
Deep Learning Ganatra, Sanket Zenodo https://doi.org/10.5281/zenodo.1189359	text	January 2018

Cited By (3)

Droplet Microfluidics-Enabled High-Throughput Screening for Protein Engineering Weng, Lindong; Spoonamore, James E. Micromachines, Vol. 10, Issue 11 https://doi.org/10.3390/mi10110734	journal	October 2019
Deep learning guided image-based droplet sorting for on-demand selection and analysis of single cells and 3D cell cultures. Anagnostidis, Vasileios; Sherlock, Benjamin; Metz, Jeremy Apollo - University of Cambridge Repository https://doi.org/10.17863/cam.63685	text	January 2020
Deep learning guided image-based droplet sorting for on-demand selection and analysis of single cells and 3D cell cultures Anagnostidis, Vasileios; Sherlock, Benjamin; Metz, Jeremy arXiv https://doi.org/10.48550/arxiv.1912.05490	text	January 2019