Efficacy of using a dynamic length representation vs. a fixed-length for neuroarchitecture search

Conference · 01 July 2024

DOI:https://doi.org/10.1145/3638530.3664149· OSTI ID:2429829

Coletti, Mark  ^[1]; Gunaratne, Chathika  ^[1]; Young, Steven  ^[1]; Varadarajan, Swetha ^[2]; Patton, Robert  ^[1]; Potok, Thomas  ^[1]

---

1. ORNL
2. Association for Computing Machinery (ACM)

Deep learning neuroarchitecture and hyperparameter search are important in finding the best configuration that maximizes learned model accuracy. However, the number of types of layers, their associated hyperparameters, and the myriad of ways to connect layers poses a significant computational challenge in discovering ideal model configurations. Here, we assess two different approaches for neuroarchitecture search for a LeNet style neural network, one that uses a fixed-length approach where there is a preset number of possible layers that can be toggled on or off via mutation, and a variable-length approach where layers can be freely added or removed via special mutation operators. We found that the variable-length implementation trained better models while discovering unusual layer configurations worth further exploration.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 2429829
• Resource Relation: Conference: Genetic Algorithms and Evolutionary Computation Conference (GECCO) - Melbourne, , Australia - 7/14/2024 8:00:00 AM-7/18/2024 8:00:00 AM
• Country of Publication: United States
• Language: English