In a field like artificial intelligence (AI), pushing boundaries is part of the job description. Newmetrix’s AI engine, nicknamed Vinnie, spends his days collecting insights and using them to reduce project risks. He’s always learning new ways to find and report safety hazards. But how, exactly, does an AI engine like Vinnie learn?
Below is a behind-the-scenes look at how Vinnie masters new skills, what’s next on his lesson plan, and AI’s limitations.
Practice makes perfect
Vinnie learns the same way as you and I: by studying. Working with our customers, we determine new types of object, materials, and hazards that Vinnie should learn to identify on job sites. Newmetrix researchers and engineers create a pipeline of photos for Vinnie to review, using the millions of customer images uploaded to date. For example, he recently learned to recognize standing water at a job site. We showed him thousands of photos of water in various situations, as well as job sites that were bone dry. Both types of images help make Vinnie smarter in the long run. When put into practice, his ability to identify standing water can help prevent workers from slipping on wet ground or encountering electrical hazards.
Vinnie detects a ladder on a construction jobsite (Image courtesy of ENR)
Next steps: Context is key
Every one of Vinnie’s learning objectives is inspired by a real-world need. The “why” behind our customers’ safety policies helps us understand their nuance, and consulting with industry experts—including our own staff members—ensures accuracy while adding context.
As we lay out Vinnie’s next set of goals, we remember to stay grounded while dreaming big. Image recognition features that will help keep workers safe are critically important, and so is the context that surrounds them. In that spirit, Vinnie is now working to learn how to interpret context in order to more effectively help safety managers do their jobs.
For example, Vinnie can already tell you what kind of ladders and lifts are present on a job site. Imagine if he could tell you which ladder should be used, and whether it’s being used safely, based on the situation in each photo. When Vinnie can recommend what type of lift or ladder to use in a given situation he’ll be transforming context into real safety results.
Looking toward the future
One of our partners recently began a project that will assess the structural reliability of a major pipe system. Doing so will involve taking photos inside pipes, cataloguing quality issues and addressing them as needed. When reviewing those photos, a trained human eye will always have a better understanding of context and nuance. However, the possibility of human error is unavoidable and the chance of errors skyrockets when even the most experienced image analysts become fatigued. That’s where Vinnie comes in to help—and where AI offers its greatest value. Vinnie won’t replace human workers and experts on a job site, but he can help streamline certain tasks, meaning human attention can be clearly focused where it matters most.
