ACM, the Affiliation for Computing Equipment, immediately named Torsten Hoefler, a professor at ETH Zurich, the recipient of the ACM Prize in Computing for basic contributions to high-performance computing and the continuing AI revolution.
Hoefler, who was a guest on the @HPCpodcast on this website final November, developed lots of the core capabilities of contemporary supercomputers and outlined key elements of the algorithms for distributing AI fashions on them, the ACM mentioned.
The ACM Prize in Computing acknowledges early-to-mid-career laptop scientists whose analysis contributions have basic affect and broad implications. The award carries a prize of $250,000, from an endowment supplied by Infosys Ltd., a digital companies and consulting firm.
ACM cited the work of Hoefler and his colleagues to scale community design and programming in supercomputers, which has revolutionized the capabilities of those
giant techniques. For instance, AI algorithms can now be processed on tons of of 1000’s of nodes. Hoefler’s advances in interconnection networks, programming, and parallel algorithms broke new floor in facilitating using large-scale massively parallel clusters.
“His quite a few improvements throughout the entire supercomputer stack—together with key contributions akin to MPI-3 nonblocking collective operations, foundational parallelism methods for AI fashions, and high-performance networking techniques—have pushed the boundaries of parallel techniques design and translated into dramatic enhancements in supercomputer efficiency and scalability,” the ACM mentioned. “Lots of these improvements are included into the biggest and strongest machines immediately.”
The ACM cited these examples of Hoefler’s HPC-AI R&D:
Message Passing Interface 3: Hoefler performed a serious position within the evolution of the Message Passing Interface (MPI), an off-the-cuff trade normal for exchanging messages between quite a few particular person nodes all through an HPC community. A messaging normal permits synchronization of the actions of every particular person laptop, sharing information between nodes, and path and management of your entire parallel community.
The MPI-3 normal was adopted in 2012 and made potential advances in HPC for simulations and AI purposes over the previous a number of years. Hoefler chaired each the “Course of Topologies” and “Collective Operations” working teams for MPI-3. His nonblocking collective operations, akin to Allreduce, Allgather, Bcast, and their respective blocking variations are included in numerous collective communication libraries—even past MPI-3. These operations energy the core of distributed deep studying immediately.
3D Parallelism: Hoefler was among the many first to develop and uncover the now well-known notion of “3D parallelism,” which drives infrastructure design for the AI trade. Subsequently, he and his collaborators continued to develop strategies for environment friendly pipelining, sparse communication, mannequin sparsity and quantization. This work has
enabled a cumulative 10-1000x acceleration of AI workloads in fashionable computer systems.
Routing Protocols and Community Topologies: The low-level community routing protocols and community topologies that Hoefler and his colleagues developed for such networks as Myrinet and InfiniBand energy 1000’s of AI and HPC supercomputers. These contributions are central to high-performance AI techniques that practice large-language fashions, akin to ChatGPT.
“The capability of high-performance computer systems has develop into mind-boggling,” mentioned ACM President Yannis Ioannidis. “In simply the final two years, we have now ushered within the period of
exascale computer systems, which might carry out a billion billion calculations per second. However high-performance computer systems may do little with out the underlying algorithms and requirements that enable them to course of huge influxes of knowledge.
“Whereas Hoefler launched many of those improvements whereas he was nonetheless a scholar, they continue to be the definitive method to program massively parallel techniques immediately,” Ioannidis continued. “The ensuing capabilities of those techniques have allowed important advances in AI, the pure sciences, and lots of different areas.”
Salil Parekh, CEO, Infosys, mentioned, “We’re in the course of an thrilling period of AI, with nice promise for the long run. Torsten Hoefler performed an vital position in advancing high-performance computing, which in flip, fuelled the present AI revolution. The ACM Prize in Computing is meant to acknowledge to ‘early to mid-career professionals,’ whose work has depth and affect. With a lot of his vital work being achieved in his 20s, Hoefler is an instance for younger those that age isn’t an impediment to achievement in computing. Infosys is proud to be the sponsor of this award because it was based in 2007.”
Hoefler is a Professor of Pc Science at ETH Zurich (the Swiss Federal Institute of Expertise), the place he serves as Director of the Scalable Parallel Computing
Laboratory. He’s additionally the Chief Architect for AI and Machine Studying on the Swiss Nationwide Supercomputing Centre (CSCS). Hoefler obtained a Diplom Informatik (Grasp of Pc Science) from Chemnitz College of Expertise and a PhD in Pc Science from Indiana College.
Hoefler’s honors embrace the Max Planck-Humboldt Medal, an award for excellent mid- profession scientists; the IEEE CS Sidney Fernbach Award, which acknowledges excellent contributions within the utility of high-performance computer systems; and the ACM Gordon Bell Prize, which acknowledges excellent achievement in high-performance computing. He’s a member of the European Academy of Sciences (Academia Europaea), a Fellow of IEEE, and a Fellow of ACM.
Hoefler can be formally offered with the ACM Prize in Computing at ACM’s annual awards banquet on Saturday, June 14 at The Palace Resort in San Francisco.
