Object and Facial Recognition for Film and Images

No, we don’t do that. But we know that someone will.

We’re living in a time when computer vision tools are exploding in power, popularity, and use. Sometimes we know it, sometimes it’s in the background and invisible to us. That computer in your pocket already does it.

We made a list of computer vision object recognition facial recognition softwares, just the highlights that we think are most relevant to film archives. Just ask and we’ll give you a copy. We made it for our own education and we’re happy to share what we learn.

Our film inspection products, FI-16 and Waypoint, take photographs of your films so our computer vision tools can analyze their condition and produce condition reports for you. Our products also make video workprints.  This means that if you choose to send to recognition software or online services, you have video and still images to choose from. Or send them both, or send them to different softwares and services. Fun, eh?

There isn’t one best software or service and they’re probably never will be. It’s a Pandora’s box and that’s a good thing. Try them and see how you like them. Keep using the ones you like best.

We can tell you this about some of the key differences that you’ll encounter.

Some of them are optimized for video. Knowing that the source is video let’s the recognition software infer important things about the images in sequence. If a face is recognized in one frame and repeated for many more frames and then begins to turn around showing the back of that person’s face, video optimized software can figure out that it’s still the same person. If you feed thousands of still images to a recognition software and it finds a face, it won’t know that the back of that person’s head is the same person.

Video optimized recognition software can also use the soundtrack. Take the interview format for example. The camera may be on the interviewer when they ask the question but when the shot cuts or pans to the interviewee, the interviewer is still there. Humans know that intuitively. Video optimized recognition software is figuring that out too.

There can also be benefits to recognition software analyzing each frame separately. For example they could take a quick first pass through a long sequence of images and take note of which frames it can analyze the best. This will depend on computer vision factors like edges, exposure, and remember that some edges are contours of colors, not black or white, not contours of contrast.

After identifying the faces or objects in those key frames, the image optimized recognition software can then use those as a base for getting better results from all the other individual photographs.

So if all of that makes sense to you, there’s a third approach. A hybrid approach. Start by analyzing the video, but first break it out into thousands of individual still images. Then use all of the first two approaches, and compare the results to learn even more.

If it sounds repetitive, complicated, or even mind numbingly boring, please remember that that’s what computers are good at. That’s why we invented them. To do that work for us. Preferably while we are doing something else more interesting to us, things that computers simply cannot do. For example, computers don’t have social skills or empathy. That’s what humans are good at.

And that’s why we employ video everywhere in our products. Humans are incredibly sensitive to body language, motion, anything that moves or relates to human behaviors. During a film inspection the machine presents the photographs on the user’s computer screen and it looks just like video. It’s really still images going by so quickly that your brain will interpret it as video. Then your FI-16 can make a video work print, compressed so you can watch it on your computer in your pocket or any computer as far away as you’d like to send it. Waypoint overlays computer vision measured analysis in graphic form on copies of the still photographs and in video workprints that it makes from them. Again with the computer in your pocket and the computers as far away as you’d like to send them.

Object and facial recognition are wonderful tools and humans will always discover things they don’t. Work together on this. The important thing is to inspect your films, know what condition they’re in, use your judgment to apply your resources to preserve them and make them accessible. Let the computers do the boring parts.

Leave a Reply

Your email address will not be published. Required fields are marked *