CLIAMS:WE CLAIM:
1. A method for analyzing video data to determine manufacturing waste to optimize productivity, the method comprising:
identifying, by a productivity assessment computing device, a plurality of entities present in each of a plurality of obtained frames of a video and a type of each of the plurality of entities;
plotting, by the productivity assessment computing device, movement of each of the plurality of entities across at least a subset of the frames to obtain a trajectory of each of the plurality of entities;
identifying, by the productivity assessment computing device, a plurality of interactions of one or more of the plurality of entities in each of the plurality of frames;
generating, by the productivity assessment computing device, a unique sequence encoding for each of a plurality of subtasks performed by each of the plurality of entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;
classifying, by the productivity assessment computing device, at least one of the subset of subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of subtasks and the trajectory and the type of each of the plurality of entities associated with the one or more of the interactions;
correlating, by the productivity assessment computing device, the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video to determine at least one waste duration value for the task; and outputting, by the productivity assessment computing device, the waste duration value.

2. The method as set forth in claim 1, wherein the identifying further comprises identifying one or more features of each of the plurality of entities in each of the plurality of frames and the determining further comprises correlating a mapping of each of the plurality of entities to a position on a floor plan with the features of each of the plurality of entities in adjacent ones of the plurality of frames.

3. The method as set forth in claim 1, wherein the plurality of entities are identified based on a plurality of classifiers, the subset of the subtasks is associated with the task based on a correlation with the classifiers, and the classifiers comprise training images of the plurality of entities and training videos of the tasks.

4. The method as set forth in claim 1, wherein the waste duration value is for transportation waste and comprises a duration that a material, one of the plurality of entities is interacting with a machine, one of the plurality of entities or a human, one of the plurality of entities and is moving from one location to another location.

5. The method as set forth in claim 1, wherein the waste duration value is for a waiting waste and comprises a duration that a human, one of the entities is idle and not interacting with any other of the plurality of entities.

6. The method as set forth in claim 1, wherein the waste duration value is for a motion waste and comprises a duration that a human, one of the entities is not interacting with a material, one of the entities or a machine, one of the entities and is moving from one location to another location.

7. The method as set forth in claim 6, further comprising determining, by the productivity assessment computing device, one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

8. A productivity assessment computing device comprising at least one processor and a memory coupled to the processor which is configured to be capable of executing programmed instructions comprising and stored in the memory to:
identify a plurality of entities present in each of a plurality of obtained frames of a video and a type of each of the plurality of entities;
plot movement of each of the plurality of entities across at least a subset of the plurality of frames to obtain a trajectory of each of the plurality of entities;
identify a plurality of interactions of one or more of the plurality of entities in each of the plurality of frames;
generate a unique sequence encoding for each of a plurality of subtasks performed by each of the plurality of entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;
classify at least one of the subset of subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of subtasks and the trajectory and the type of each of the entities associated with the one or more of the interactions;
correlate the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video to determine at least one waste duration value for the task; and
output the waste duration value.

9. The productivity assessment computing device as set forth in claim 8, wherein the processor coupled to the memory is further configured to be capable of executing at least one additional programmed instruction to:
identify one or more features of each of the plurality of entities in each of the plurality of frames; and
correlate a mapping of each of the plurality of entities to a position on a floor plan with the features of each of the plurality of entities in adjacent ones of the plurality of frames.

10. The productivity assessment computing device as set forth in claim 8, wherein the plurality of entities are identified based on a plurality of classifiers, the subset of the subtasks is associated with the task based on a correlation with the classifiers, and the classifiers comprise training images of the plurality of entities and training videos of the tasks.

11. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a transportation waste and comprises a duration that a material, one of the plurality of entities is interacting with a machine one of the entities or a human, one of the entities and is moving from one location to another location.

12. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a waiting waste and comprises a duration that a human, one of the entities is idle and not interacting with any other of the plurality of entities.

13. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a motion waste and comprises a duration that a human, one of the entities is not interacting with a material, one of the entities or a machine, one of the entities and is moving from one location to another location.

14. The productivity assessment computing device as set forth in claim 13, wherein the processor coupled to the memory is further configured to be capable of executing at least one additional programmed instruction to determine one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

15. A non-transitory computer readable medium having stored thereon instructions for determining and analyzing manufacturing waste to optimize productivity comprising executable code which when executed, by a processor, causes the processor to perform steps comprising:
identifying a plurality of entities present in each of a plurality of obtained frames of a video and a type of each of the plurality of entities;
plotting movement of each of the plurality of entities across at least a subset of the plurality of frames to obtain a trajectory of each of the plurality of entities;
identifying a plurality of interactions of one or more of the plurality of entities in each of the plurality of frames;
generating a unique sequence encoding for each of a plurality of subtasks performed by each of the plurality of entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;
classifying at least one of the subset of subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of subtasks and the trajectory and the type of each of the plurality of entities associated with the one or more of the interactions;
correlating the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video to determine at least one waste duration value for the task; and
outputting the waste duration value.

16. The non-transitory computer readable medium as set forth in claim 15, wherein the identifying further comprises identifying one or more features of each of the entities in each of the frames and the determining further comprises correlating a mapping of each of the plurality of entities to a position on a floor plan with the features of each of the plurality of entities in adjacent ones of the plurality of frames.

17. The non-transitory computer readable medium as set forth in claim 15, wherein the plurality of entities are identified based on a plurality of classifiers, the subset of the subtasks is associated with the task based on a correlation with the classifiers, and the classifiers comprise training images of the plurality of entities and training videos of the tasks.

18. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a transportation waste and comprises a duration that a material, one of the entities is interacting with a machine one of the entities or a human one of the entities and is moving from one location to another location.

19. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a waiting waste and comprises a duration that a human, one of the entities is idle and not interacting with any other of the plurality of entities.

20. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a motion waste and comprises a duration that a human, one of the plurality of entities is not interacting with a material, one of the plurality of entities or a machine, one of the entities and is moving from one location to another location.

21. The non-transitory computer readable medium as set forth in claim 20, further having stored thereon instructions which when executed, by the processor further cause the processor to perform at least one additional step comprising determining one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

Dated this 12th day of February, 2015
Shwetha A Chimalgi
Of K&S Partners
Agent for the Applicant
,TagSPECI:FIELD OF THE INVENTION
This technology relates to assessing manufacturing productivity and more particularly to methods and devices for analyzing video data to identify waste in a manufacturing environment.