skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis

Abstract

Motivation: Cell movement in the early phase of Caenorhabditis elegans development is regulated by a highly complex process in which a set of rules and connections are formulated at distinct scales. Previous efforts have demonstrated that agent-based, multi-scale modeling systems can integrate physical and biological rules and provide new avenues to study developmental systems. However, the application of these systems to model cell movement is still challenging and requires a comprehensive understanding of regulatory networks at the right scales. Recent developments in deep learning and reinforcement learning provide an unprecedented opportunity to explore cell movement using 3D time-lapse microscopy images. Results: We present a deep reinforcement learning approach within an agent-based modeling system to characterize cell movement in the embryonic development of C.elegans. Our modeling system captures the complexity of cell movement patterns in the embryo and overcomes the local optimization problem encountered by traditional rule-based, agent-based modeling that uses greedy algorithms. Here, we tested our model with two real developmental processes: the anterior movement of the Cpaaa cell via intercalation and the rearrangement of the superficial left-right asymmetry. In the first case, the model results suggested that Cpaaa’s intercalation is an active directional cell movement caused by the continuousmore » effects from a longer distance (farther than the length of two adjacent cells), as opposed to a passive movement caused by neighbor cell movements. In the second case, a leader-follower mechanism well explained the collective cell movement pattern in the asymmetry rearrangement. These results showed that our approach to introduce deep reinforcement learning into agent-based modeling can test regulatory mechanisms by exploring cell migration paths in a reverse engineering perspective. Lastly, this model opens new doors to explore the large datasets generated by live imaging. Availability and implementation: Source code is available at https://github.com/zwang84/drl4cellmovement.« less

Authors:
 [1]; ORCiD logo [2];  [1]; ORCiD logo [3];  [1];  [3]
  1. Univ. of Tennessee, Knoxville, TN (United States). Dept. of Electrical Engineering and Computer Science
  2. Univ. of Tennessee, Knoxville, TN (United States). Dept. of Electrical Engineering and Computer Science; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Environmental Science Division
  3. Sloan-Kettering Inst., New York, NY (United States). Developmental Biology Program
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE; National Institutes of Health (NIH)
OSTI Identifier:
1463974
Grant/Contract Number:  
AC05-00OR22725; R01GM097576; P30CA008748
Resource Type:
Accepted Manuscript
Journal Name:
Bioinformatics
Additional Journal Information:
Journal Volume: 34; Journal Issue: 18; Journal ID: ISSN 1367-4803
Publisher:
Oxford University Press
Country of Publication:
United States
Language:
English
Subject:
59 BASIC BIOLOGICAL SCIENCES

Citation Formats

Wang, Zi, Wang, Dali, Li, Chengcheng, Xu, Yichi, Li, Husheng, and Bao, Zhirong. Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis. United States: N. p., 2018. Web. https://doi.org/10.1093/bioinformatics/bty323.
Wang, Zi, Wang, Dali, Li, Chengcheng, Xu, Yichi, Li, Husheng, & Bao, Zhirong. Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis. United States. https://doi.org/10.1093/bioinformatics/bty323
Wang, Zi, Wang, Dali, Li, Chengcheng, Xu, Yichi, Li, Husheng, and Bao, Zhirong. Wed . "Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis". United States. https://doi.org/10.1093/bioinformatics/bty323. https://www.osti.gov/servlets/purl/1463974.
@article{osti_1463974,
title = {Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis},
author = {Wang, Zi and Wang, Dali and Li, Chengcheng and Xu, Yichi and Li, Husheng and Bao, Zhirong},
abstractNote = {Motivation: Cell movement in the early phase of Caenorhabditis elegans development is regulated by a highly complex process in which a set of rules and connections are formulated at distinct scales. Previous efforts have demonstrated that agent-based, multi-scale modeling systems can integrate physical and biological rules and provide new avenues to study developmental systems. However, the application of these systems to model cell movement is still challenging and requires a comprehensive understanding of regulatory networks at the right scales. Recent developments in deep learning and reinforcement learning provide an unprecedented opportunity to explore cell movement using 3D time-lapse microscopy images. Results: We present a deep reinforcement learning approach within an agent-based modeling system to characterize cell movement in the embryonic development of C.elegans. Our modeling system captures the complexity of cell movement patterns in the embryo and overcomes the local optimization problem encountered by traditional rule-based, agent-based modeling that uses greedy algorithms. Here, we tested our model with two real developmental processes: the anterior movement of the Cpaaa cell via intercalation and the rearrangement of the superficial left-right asymmetry. In the first case, the model results suggested that Cpaaa’s intercalation is an active directional cell movement caused by the continuous effects from a longer distance (farther than the length of two adjacent cells), as opposed to a passive movement caused by neighbor cell movements. In the second case, a leader-follower mechanism well explained the collective cell movement pattern in the asymmetry rearrangement. These results showed that our approach to introduce deep reinforcement learning into agent-based modeling can test regulatory mechanisms by exploring cell migration paths in a reverse engineering perspective. Lastly, this model opens new doors to explore the large datasets generated by live imaging. Availability and implementation: Source code is available at https://github.com/zwang84/drl4cellmovement.},
doi = {10.1093/bioinformatics/bty323},
journal = {Bioinformatics},
number = 18,
volume = 34,
place = {United States},
year = {2018},
month = {4}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Virtual exploration of early stage atherosclerosis
journal, August 2016


Multidimensional regulation of gene expression in the C. elegans embryo
journal, April 2012


Assessing Normal Embryogenesis inCaenorhabditis elegansUsing a 4D Microscope: Variability of Development and Regional Specification
journal, April 1997

  • Schnabel, Ralf; Hutter, Harald; Moerman, Don
  • Developmental Biology, Vol. 184, Issue 2
  • DOI: 10.1006/dbio.1997.8509

Human-level control through deep reinforcement learning
journal, February 2015

  • Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David
  • Nature, Vol. 518, Issue 7540
  • DOI: 10.1038/nature14236

A cell-based simulation software for multi-cellular systems
journal, August 2010


A Comprehensive Survey of Multiagent Reinforcement Learning
journal, March 2008

  • Busoniu, Lucian; Babuska, Robert; De Schutter, Bart
  • IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 38, Issue 2
  • DOI: 10.1109/tsmcc.2007.913919

Mechanisms of cell positioning during C. elegans gastrulation
journal, January 2003


Four-dimensional realistic modeling of pancreatic organogenesis
journal, December 2008

  • Setty, Y.; Cohen, I. R.; Dor, Y.
  • Proceedings of the National Academy of Sciences, Vol. 105, Issue 51
  • DOI: 10.1073/pnas.0808725105

Systematic quantification of developmental phenotypes at single-cell resolution during embryogenesis
journal, July 2013


Mechanisms, mechanics and function of epithelial–mesenchymal transitions in early development
journal, November 2003


An Observation-Driven Agent-Based Modeling and Analysis Framework for C. elegans Embryogenesis
journal, November 2016


WDDD: Worm Developmental Dynamics Database
journal, November 2012

  • Kyoda, Koji; Adachi, Eru; Masuda, Eriko
  • Nucleic Acids Research, Vol. 41, Issue D1
  • DOI: 10.1093/nar/gks1107

Visualization of 3-Dimensional Vectors in a Dynamic Embryonic System—WormGUIDES
journal, January 2017

  • Wang, Eric; Santella, Anthony; Wang, Zi
  • Journal of Computer and Communications, Vol. 05, Issue 12
  • DOI: 10.4236/jcc.2017.512008

Spatio-temporal reference model of Caenorhabditis elegans embryogenesis with cell contact maps
journal, September 2009


Chiral Forces Organize Left-Right Patterning in C. elegans by Uncoupling Midline and Anteroposterior Axis
journal, September 2010


Cell Movement Is Guided by the Rigidity of the Substrate
journal, July 2000


Forces in Tissue Morphogenesis and Patterning
journal, May 2013


Multi-scale computational modeling of developmental biology
journal, May 2012


Automated cell lineage tracing in Caenorhabditis elegans
journal, February 2006

  • Bao, Z.; Murray, J. I.; Boyle, T.
  • Proceedings of the National Academy of Sciences, Vol. 103, Issue 8
  • DOI: 10.1073/pnas.0511111103

A semi-local neighborhood-based framework for probabilistic cell lineage tracing
journal, January 2014


Mounting Caenorhabditis elegans Embryos for Live Imaging of Embryogenesis
journal, August 2011


Quantitative semi-automated analysis of morphogenesis with single-cell resolution in complex embryos
journal, October 2012

  • Giurumescu, C. A.; Kang, S.; Planchon, T. A.
  • Development, Vol. 139, Issue 22
  • DOI: 10.1242/dev.086256

The Regulatory Landscape of Lineage Differentiation in a Metazoan Embryo
journal, September 2015


The embryonic cell lineage of the nematode Caenorhabditis elegans
journal, November 1983


Multiagent cooperation and competition with deep reinforcement learning
journal, April 2017


De Novo Inference of Systems-Level Mechanistic Models of Development from Live-Imaging-Based Phenotype Analysis
journal, January 2014


A Comprehensive Survey of Multiagent Reinforcement Learning
journal, March 2008

  • Busoniu, Lucian; Babuska, Robert; De Schutter, Bart
  • IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 38, Issue 2
  • DOI: 10.1109/TSMCC.2007.913919

Cell Movement Is Guided by the Rigidity of the Substrate
journal, July 2000