Accurate, consistent character recognition is one of the most difficult challenges in AI.
What we mean by “character recognition” is the ability to recognize characters in video whether their face is partially obscured, facing away from the camera, wearing a mask or in heavy makeup. This is fundamental to maintaining a high level of accuracy in regards to what’s on screen.
But while live action character recognition is difficult, it gets even tougher with animation.
One reason is because animated characters, obviously, do not adhere to the usual standards of what a human face looks like, and those faces are far more malleable than a human’s.
But we at Resonance AI cracked the code, and are now delivering the most reliable animated character recognition available.
First, a motion-based shot detection model, and character detection model, create bounding boxes around each character on screen.
A tracking algorithm is then used to follow the characters through the shot boundaries.
A vector representation of each character is then extracted, and a human-in-the-loop process labels a small sample of the tracked characters.
These labels, along with the corresponding vector, prime the recognition model with examples of which characters to detect.
This then creates a visual fingerprint, and our model can then recognize and label the tracked animated characters on its own.
This means the characters can be recognized no matter where they go or what they do.
This is just one way Resonance AI is pushing the boundaries of artificial intelligence and helping change the future of media.