Algorithm and Architecture Independent Benchmarking with SEAK
Many applications of high performance embedded computing are limited by performance or power bottlenecks. We have designed the Suite for Embedded Applications & Kernels (SEAK), a new benchmark suite, (a) to capture these bottlenecks in a way that encourages creative solutions; and (b) to facilitate rigorous, objective, end-user evaluation for their solutions. To avoid biasing solutions toward existing algorithms, SEAK benchmarks use a mission-centric (abstracted from a particular algorithm) and goal-oriented (functional) specification. To encourage solutions that are any combination of software or hardware, we use an end-user black-box evaluation that can capture tradeoffs between performance, power, accuracy, size, and weight. The tradeoffs are especially informative for procurement decisions. We call our benchmarks future proof because each mission-centric interface and evaluation remains useful despite shifting algorithmic preferences. It is challenging to create both concise and precise goal-oriented specifications for mission-centric problems. This paper describes the SEAK benchmark suite and presents an evaluation of sample solutions that highlights power and performance tradeoffs.
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1322528
- Report Number(s):
- PNNL-SA-115612; 400403809
- Resource Relation:
- Conference: IEEE International Parallel and Distributed Processing Symposium, May 23-27, 2016, Chicago, Illinois, 63-72
- Country of Publication:
- United States
- Language:
- English
Similar Records
WE-AB-201-00: Treatment Planning System Commissioning and QA
WE-AB-201-01: Treatment Planning System Commissioning and QA: Challenges and Opportunities