Apple is working on to improve the capabilities of computer vision to make it detect and identify subjects more accurately, such as in situation where both the camera and the subject are in motion with the one method suggesting the use of thermal imaging to help determine what elements in view need to be tracked.
The application specifically relates to computer vision, namely the use of cameras to analyze a scene, and to acquire useful data from what is seen.
The filing identifies an issue with an existing computer vision analysis techniques in dynamic environment, in that many algorithms are constructed with a static environment in mind, but that camera setup is the only moving object, one that is also the only thing where it pose can definitely be tracked.
If there are multiple moving objects in a scene, the accuracy of camera pose tracking is reduced or even disabled depending on the motions of item within a scene. The motions of object could cause issues with recognition due to the occlusion of view by the camera.
While a 2d camera view can be translated to point in 3d space, the motion of the objects in this space could also issues with 3D positioning of the camera.
The algorithm could also analyze the thermal image and combine that with the imaging data to determine what items in a scene are separated from others, adding heat along side color, contrast and other attributes typically used for analysis.
This system Could use thermal imaging to identify a warm hand covering part of a 2D image, with the data potentially used to alter the display.
Apple is exploring the concept with Project Titan, an operation that is trialing self driving vehicle systems and sensors, and one where the addition of thermal cameras would be useful and relatively easy.
Apple files a larger number of patent applications with USPTO on a regular basis and relatively few make their way into customer products.