Algorithm and Architecture Independent Benchmarking with SEAK
Many applications of high performance embedded computing are limited by performance or power bottlenecks. We have designed the Suite for Embedded Applications & Kernels (SEAK), a new benchmark suite, (a) to capture these bottlenecks in a way that encourages creative solutions; and (b) to facilitate rigorous, objective, end-user evaluation for their solutions. To avoid biasing solutions toward existing algorithms, SEAK benchmarks use a mission-centric (abstracted from a particular algorithm) and goal-oriented (functional) specification. To encourage solutions that are any combination of software or hardware, we use an end-user black-box evaluation that can capture tradeoffs between performance, power, accuracy, size, and weight. The tradeoffs are especially informative for procurement decisions. We call our benchmarks future proof because each mission-centric interface and evaluation remains useful despite shifting algorithmic preferences. It is challenging to create both concise and precise goal-oriented specifications for mission-centric problems. This paper describes the SEAK benchmark suite and presents an evaluation of sample solutions that highlights power and performance tradeoffs.
- Publication Date:
- OSTI Identifier:
- Report Number(s):
- DOE Contract Number:
- Resource Type:
- Resource Relation:
- Conference: IEEE International Parallel and Distributed Processing Symposium, May 23-27, 2016, Chicago, Illinois, 63-72
- IEEE, PISCATAWAY, New Jersey, United States.
- Research Org:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Org:
- Country of Publication:
- United States
- Mission Critical Benchmarking; algorithms and kernels; white and black box evaluation solutions