Nikolas Lamb, Clarkson Computer Science PhD student and NSF Graduate Research Fellow, presented his research on the use of artificial intelligence (AI) techniques, in particular computer vision and deep learning, to repair damaged objects at the Eurographics Symposium on Geometry Processing (SGP), the highest ranking venue for geometry processing techniques on July 5. Nikolas is advised on his research by Drs. Natasha Banerjee and Sean Banerjee, both associate professors in the Department of Computer Science. The results of Nikolas’ research from the site’s proceedings will appear as published work in the 2022 Computer Graphics Forum, the leading journal of in-depth technical articles on computer graphics. Nikolas’ work is Clarkson’s first to be presented at the SGP and to appear in the Computer Graphics Forum magazine.
Nikolas’ work offers users a new approach, called MendNet—an object To fixdeep neural Reportwork—which automatically synthesizes additively manufactured repair parts into 3D models of damaged objects. Nikolas’ approach to automated 3D repair synthesis is the first of its kind. Prior to Nikolas’ research, if a user’s treasured heirloom broke, with broken parts damaged beyond repair, restoring the broken object was a significant challenge, as the user had to painstakingly 3D model the geometry broken part complex. This is something most users are unlikely to do, and it’s no surprise that a lot of damaged items end up being thrown away, increasing environmental waste and having a huge impact on durability.
Nikolas’ research plays a key role in advancing Clarkson’s commitment to sustainability, using AI to automate the repair process, empowering end users to choose “repair” over “replace.” Users can now repair broken items, for example, ceramic items such as valuable tableware with minimal effort. Nikolas’ automated repair algorithm allows users to scan their broken item and can automatically synthesize the repair part and send the part to a 3D printer. Nikolas’ work capitalizes on the ubiquity of 3D printers and the emergence of 3D printers for materials such as ceramics and wood in the mainstream market. By tying AI, computer vision, and deep learning to the manufacturing process, Nikolas’ work dramatically transforms the landscape of advanced manufacturing, putting rapid manufacturing in the hands of the average user.
Nikolas’ work has a wider impact on the advancement of knowledge in fields such as archaeology, anthropology and paleontology, providing a user-friendly approach to repairing cultural heritage artifacts, damaged fossil specimens and fragmented remains, thereby reducing the busy work of researchers and allowing them to focus attention on research questions of domain interest. The work also has an impact on the automation of repairs in dentistry and medicine.
Nikolas is a member of the Terascale All-sensing Research Studio (TARS) at Clarkson University. TARS supports the research of 15 graduate students and nearly 20 undergraduate students each semester. TARS has one of the largest high-performance computing facilities at Clarkson, with over 275,000 CUDA cores and over 4,800 Tensor cores spread across over 50 GPUs, and 1 petabyte of storage (almost full!). TARS is home to the Gazebo, a massively dense multi-modal multi-viewpoint motion capture facility for imaging multi-person interactions containing 192 high-speed 226FPS cameras, 16 Microsoft Azure Kinect RGB-D sensors, 12 Sierra Olympic Viento thermal cameras -G, and 16 surface electromyography (sEMG) sensors, and the Cube, a one- and two-person 3D imaging facility containing 4 high-speed cameras, 4 RGB-D sensors and 5 thermal cameras. TARS is researching the use of deep learning to better understand the natural interactions between multiple people from massive datasets, to enable next-generation technologies, for example, intelligent agents and robots , to integrate seamlessly into future human environments.
The team thanks the Office of Information Technology for providing access to the ACRES GPU node with 4 V100s containing 20,480 CUDA cores and 2,560 Tensor cores.