The concept of transprecision computing

For many years, computing systems rely on guaranteed numerical precision of each step in complex computations. Moore’s law sustains exponential improvements in the semiconductor industry over several decades for building computing infrastructure, from tiny Internet-of- Things nodes, over personal smartphones, laptops or workstations, up to large high performance computing (HPC) computing server centers. With the paradigm of the ”power wall”, achievable improvements start to saturate. To that end, the concept of transprecision computing emerged, where existing over-conservative ”precis” computing assumptions are relaxed and replaced with more flexible and efficient policies to gain performance.
Unfortunately, it is non-straight forward to adopt and integrate general transprecision concepts into the variety of today’s computing infrastructure. The main challenge consists of leveraging domainspecific knowledge and provide full solutions covering from physical foundations over circuit-level up through the full software stack to the application level.
This work focuses on how transprecision concepts improve general computing. We identify and elaborate the standard number representations, especially the one defined in the IEEE 754 floating-point standard, as the enabler of low precision computing. We developed lightweight libraries that allow integrating transprecision concepts into algorithms. Finally, we focus on building automatized workflows for specific problems, where the solution space is enlarged by multiple orders of magnitude due to the various configurations of low precision. We demonstrate how heuristic optimization strategies applied on top of transprecision computing find near to optimal configurations of approximated kernels in a short time.