Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:
SYSTEMS AND METHODS FOR ERGONOMICS ASSESSMENT AND COMPUTATION OF RISK SCORE FOR USERS

Applicant:
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th Floor,
Nariman Point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
The disclosure herein generally relates to ergonomic risk assessment, and, more particularly, to systems and methods for ergonomics assessment and computation of risk score for users.

BACKGROUND
Over the years, work-related musculoskeletal disorders have become prevalent in the manufacturing industry due to the repetitive and demanding working conditions leading to stressful and awkward body postures. This has led to exploring and standardization of advanced techniques in industrial human ergonomics posture assessment methods. Currently, there are several ergonomics posture risk assessment tools such a Rapid Entire Body Assessment (REBA), Rapid Upper Limb Assessment (RULA), etc. However, assessment of these ergonomics risk factors requires a considerable amount of image or video snapshots of the workers which requires humans to be present, henceforth less observations are captured which leads to incomplete assessment of such ergonomics in the workplace. An ergonomically deficient workplace can cause physical and emotional stress, low productivity, and poor quality of work. Moreover, this low productivity and poor quality of work can lead to huge losses as direct and indirect costs over the years.

SUMMARY
Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.
For example, in one aspect, there is provided a processor implemented method for ergonomics assessment and computation of risk score for users. The method comprises obtaining, via one or more hardware processors, a first model and a second model pertaining to a user, wherein each of the first model and the second model comprises a video feed comprising one or more tasks performed by the user; extracting, via the one or more hardware processors, a plurality of percentage correction key points based on one or more body parts involved in the one or more tasks performed by the user; calculating, via the one or more hardware processors, a plurality of body joint angles for the one or more tasks performed by the user using the plurality of percentage correction key points; computing, via the one or more hardware processors, at least one of a first risk score and a second risk score for each of the plurality of body joint angles pertaining to the first model and the second model; performing, via the one or more hardware processors, a comparison of the first risk score and the second risk score with a reference risk score; dynamically identifying, via the one or more hardware processors, at least one of the first model and the second model as a best model for each of the plurality of body joint angles based on the comparison; computing for each frame pertaining to the video feed comprised in the best model, via the one or more hardware processors, at least one of a first refined risk score and a second refined risk score pertaining to each of the plurality of body joint angles; computing, via the one or more hardware processors, an overall risk score for the best model using (i) the at least of the first refined risk score and a second refined risk score, and (ii) at least one of a frequency and a duration of a movement of the one or more body parts in successive frames involved in the one or more tasks performed by the user; and generating, via the one or more hardware processors, one or more ergonomics risk alerts for the user based on the overall score and one or more associated health centric parameters associated with the user.
In an embodiment, the second model is derived from the first model.
In an embodiment, the first model is a two-dimensional (2D) model, and the second model is a three-dimensional (3D) model.
In an embodiment, the step of dynamically identifying at least one of the first model and the second model as a best model is based on a mean absolute difference computed between (i) the first risk score and the reference risk score, and (ii) the second risk score and the reference risk score.
In another aspect, there is provided a processor implemented system for ergonomics assessment and computation of risk score for users. The system comprises: a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to obtain a first model and a second model pertaining to a user, wherein each of the first model and the second model comprises a video feed comprising one or more tasks performed by the user; extract a plurality of percentage correction key points based on one or more body parts involved in the one or more tasks performed by the user; calculate a plurality of body joint angles for the one or more tasks performed by the user using the plurality of percentage correction key points; compute at least one of a first risk score and a second risk score for each of the plurality of body joint angles pertaining to the first model and the second model; perform a comparison of the first risk score and the second risk score with a reference risk score; dynamically identify at least one of the first model and the second model as a best model for each of the plurality of body joint angles based on the comparison; compute for each frame pertaining to the video feed comprised in the best model, at least one of a first refined risk score and a second refined risk score pertaining to each of the plurality of body joint angles; compute an overall risk score for the best model using (i) the at least of the first refined risk score and a second refined risk score, and (ii) at least one of a frequency and a duration of a movement of the one or more body parts in successive frames involved in the one or more tasks performed by the user; and generate one or more ergonomics risk alerts for the user based on the overall score and one or more associated health centric parameters associated with the user.
In an embodiment, the second model is derived from the first model.
In an embodiment, the first model is a two-dimensional (2D) model, and the second model is a three-dimensional (3D) model.
In an embodiment, the step of dynamically identifying at least one of the first model and the second model as a best model is based on a mean absolute difference computed between (i) the first risk score and the reference risk score, and (ii) the second risk score and the reference risk score.
In yet another aspect, there are provided one or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause ergonomics assessment and computation of risk score for users by obtaining a first model and a second model pertaining to a user, wherein each of the first model and the second model comprises a video feed comprising one or more tasks performed by the user; extracting a plurality of percentage correction key points based on one or more body parts involved in the one or more tasks performed by the user; calculating a plurality of body joint angles for the one or more tasks performed by the user using the plurality of percentage correction key points; computing at least one of a first risk score and a second risk score for each of the plurality of body joint angles pertaining to the first model and the second model; performing a comparison of the first risk score and the second risk score with a reference risk score; dynamically identifying at least one of the first model and the second model as a best model for each of the plurality of body joint angles based on the comparison; computing for each frame pertaining to the video feed comprised in the best model, at least one of a first refined risk score and a second refined risk score pertaining to each of the plurality of body joint angles; computing an overall risk score for the best model using (i) the at least of the first refined risk score and a second refined risk score, and (ii) at least one of a frequency and a duration of a movement of the one or more body parts in successive frames involved in the one or more tasks performed by the user; and generating one or more ergonomics risk alerts for the user based on the overall score and one or more associated health centric parameters associated with the user.
In an embodiment, the second model is derived from the first model.
In an embodiment, the first model is a two-dimensional (2D) model, and the second model is a three-dimensional (3D) model.
In an embodiment, the step of dynamically identifying at least one of the first model and the second model as a best model is based on a mean absolute difference computed between (i) the first risk score and the reference risk score, and (ii) the second risk score and the reference risk score.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
FIG. 1 depicts an exemplary system for ergonomics assessment and computation of risk score for users, in accordance with an embodiment of the present disclosure.
FIG. 2 depicts an exemplary high level block diagram of the system of FIG. 1 for ergonomics assessment and computation of risk score for users, in accordance with an embodiment of the present disclosure.
FIG. 3 depicts an exemplary flow chart illustrating a method for ergonomics assessment and computation of risk score for users, using the systems of FIG. 1-2, in accordance with an embodiment of the present disclosure.
FIG. 4 depicts an experimental set-up illustrating placement of one or more video capturing device(s) for capturing tasks being performed by one or more users, in accordance with an embodiment of the present disclosure.
FIG. 5 depicts an illustration of a reference body plane, in accordance with an embodiment of the present disclosure.
FIG. 6 depicts joints mapped to reference numbers for both the first model (e.g., OpenPose (left)) and the second model (e.g., HuMoR (right)) output, in accordance with an embodiment of the present disclosure.
FIG. 7A depicts a graphical representation illustrating distribution of the reference risk score (e.g., say a Rapid Entire Body Assessment (REBA) reference risk) per posture compared to risk scores derived from the first pose model and the second pose model, in accordance with an embodiment of the present disclosure.
FIGS. 7B depicts a graphical representation illustrating distribution of the reference risk score (e.g., say a Rapid Upper Limb Assessment (RULA) reference risk score) per posture compared to risk scores derived from the first pose model and the second pose model, in accordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS
Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.
In the area of occupational health, musculoskeletal disorders (MSDs) are considered to be one of the most critical problems and also the leading cause of absenteeism from work. This is due to the frequent incorrect postures and repetitive actions of some body parts which leads to an accumulated stress on muscles leading to MSDs. Thus, worker postures and motions are crucial indicators of the possibility of musculoskeletal injuries. In order to tackle such problems, there are industry standard observational postural evaluation techniques, which ergonomists evaluate based on the spot manual observations or by observing videos of people performing certain tasks at workplace. The ease of use and adaptability of observational postural evaluation techniques are advantages. The current research shows that there are several ergonomic posture evaluation techniques however the most common three are Rapid Upper Limb Assessment (RULA), Rapid Entire Body Assessment (REBA), and Working Posture Analysis System (OWAS). The latest research also demonstrates that most research support RULA framework over others for their experimentations (e.g., refer “D. Kee, “Comparison of OWAS, RULA and REBA for assessing potential work-related musculoskeletal disorders,” International Journal of Industrial Ergonomics, vol. 83, p. 103140, 2021. [Online]. Available: https://doi.org/10.1016/j.ergon.2021.103140”).
The major disadvantage, in such observational postural assessment techniques is that it relies significantly on the evaluator’s subjective input and demand an expert (ergonomist) to undertake a manual analysis which is a highly time-consuming and an expensive process. This might cause a significant amount of inconsistency in assessment outcome. For instance, the work presented by Robertson et al., in 2009 (e.g., refer “M. Robertson, B. C. Amick III, K. DeRango, T. Rooney, L. Bazzani, R. Harrist, and A. Moore, “The effects of an office ergonomics training and chair intervention on worker knowledge, behavior and musculoskeletal risk,” Applied ergonomics, vol. 40, no. 1, pp. 124–135, 2009. [Online]. Available: https://doi.org/10.1016/j.apergo.2007.12.009.”), demonstrated a low intra-class correlation coefficient on the RULA score from four evaluators and serial assessments by the same evaluator were recommended to enhance RULA assessment’s efficiency, therefore, demonstrating irregularity in the manual process of the assessment (e.g., refer “S. Dockrell, E. O’Grady, K. Bennett, C. Mullarkey, R. Mc Connell, R. Ruddy, S. Twomey, and C. Flannery, “An investigation of the reliability of rapid upper limb assessment (rula) as a method of assessment of children’s computing posture,” Applied ergonomics, vol. 43, no. 3, pp. 632–636, 2012. [Online]. Available: https: //doi.org/10.1016/j.apergo.2011.09.009.”). To solve these issues, many researchers have suggested semi-automated motion capture input techniques for ergonomic posture evaluation (e.g., refer “C. Huang, W. Kim, Y. Zhang, and S. Xiong, “Development and validation of a wearable inertial sensors-based automated system for assessing work-related musculoskeletal disorders in the workspace,” International Journal of Environmental Research and Public Health, vol. 17, no. 17, p. 6050, 2020. [Online]. Available: https://doi.org/10.3390/ijerph17176050.”). Although such motion cap input techniques might offer high precision for recording human movements. However, the equipment and the sensors used by such motion capture systems come with great equipment costs along with the requirement for a skilled technician to operate and high interference from body attached sensors system are some of the drawbacks of such systems (e.g., refer “E. Valero, A. Sivanathan, F. Bosché, and M. Abdel-Wahab, “Musculoskeletal disorders in construction: A review and a novel system for activity tracking with body area network,” Applied ergonomics, vol. 54, pp. 120–130, 2016. [Online]. Available: https://doi.org/10.1016/j.apergo.2015.11.020.”). As a result, these are viewed as being impracticable for ergonomic posture assessment in several real-world workplace scenarios.
In recent years, various techniques for monocular 2D and 3D human position estimation are being actively developed in the field of computer vision. With the advent of supervised machine learning algorithms in recent years, tremendous advancements have been made which now make it possible to identify the body keypoints from a single or many RGB images (e.g., refer “S. Biswas, S. Sinha, K. Gupta, and B. Bhowmick, “Lifting 2d human pose to 3d: A weakly supervised approach,” in 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–9. [Online]. Available: https://doi.org/10.1109/IJCNN.2019.8851692”). A monocular image or multi views of the same object can be used to recreate the 3D human pose. Although Multiview capture techniques need extrinsic camera calibration to be setup, they are known to be more accurate than monocular ones (e.g., refer “D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.-P. Seidel, H. Rhodin, G. Pons-Moll, and C. Theobalt, “Xnect: Real-time multi-person 3d motion capture with a single RGB camera,” Acm Transactions On Graphics (TOG), vol. 39, no. 4, pp. 82–1, 2020. [Online]. Available: https://doi.org/10.1145/3386569.3392410”). Since, assessment would eventually be possible from images and videos recorded from any common RGB cameras, increasing the accessibility and application of the system, vision-based techniques might revolutionize marker-less less postural assessment. OpenPose is a popular open-source tool for estimating 2D human poses. It can also estimate 3D human poses by 3D triangulating from multiple views utilizing at least two synchronized and calibrated cameras. In comparison to other 2D/3D human pose estimation models, OpenPose can offer more joints from the feet and face and demonstrates improved tracking performance in obstructed or non-frontal tracking circumstances. However, the accuracy of OpenPose for analysing human kinematics is still entirely unknown, despite the fact that it appears robust and promising in demonstration videos.
Initial investigation by the present disclosure demonstrates there have been very fewer research on RGB-based postural assessment systems works which attempted to validate and improve the accuracy of such assessment system. In contrast to the study presented by Yu et al., in 2019, which employed a REBA assessment using a 3D human pose estimation framework based on another study presented by Zhou et al. in 2017, reported REBA score accuracy of 70-96%. Whereas a previous study presented in 2017 by the same author, Yan et al., using OWAS where a 2D human pose estimation framework was used demonstrated OWAS score accuracy of close to 90%. On the other hand, Li et al. (2020) was able to obtain RULA action level accuracy of 93% when they attempted to extract RULA score from 2D human poses in which 3D human pose estimations is encoded. Thus, these investigations demonstrated the novelty of vision-based approach’s, researchers and practitioners may find it useful to validate their work using a more widely used and standardized vision-based framework, such as the OpenPose, for future advancement. In the past, there has been a study conducted by Z. Cao et al., in 2019, which validated OpenPose for keypoints on the validation set for precision and recall, but not for the use of skeleton-based joint angle computation and ergonomic postural evaluation when compared to the reference a complete AI enabled end-to-end system for management alert and rick recommendations.
Embodiments of the present disclosure provide systems and method that compute the joint angles by using the OpenPose framework along with the HuMoR model for comparison purposes for RULA/REBA postural assessments. Furthermore, the OpenPose (a stick model which represents Percentage Correct Keypoints (PCK)) is compared with HuMoR model (which not only provided a volumetric 3D model of the correct key points but also provides a more accurate pose reconstruction even in the presence of noise and occlusions) for accurate 3D body joint angles computation and hence providing optimal inputs for appropriately evaluating RULA/REBA postural assessment frameworks. Moreover, particularly for situations where the ability of a non-AI enabled system potentially is unable to provide inputs for alerting the management and recommend ergonomics improvements. Therefore, the main goal of present disclosure is to validate an AI enabled 3D pose estimation for ergonomics system comparing OpenPose along with HuMoR based framework for computing optimal body joint angles (particularly in circumstances with occlusions) and automatically performing RULA/REBA ergonomic postural assessment by further creating an ergonomics risk recommender alert system for ergonomics improvements for the management. For this purpose, first the joint angles computed from OpenPose, and HuMoR models are compared with the reference body planes used for the system. Then next is to automatically calculate the RULA/REBA scores along with required action levels against the reference planes. The secondary goal is to compare and contrast the present disclosure’s AI enabled system outcomes from OpenPose and HuMoR based approaches. Then finally the effectiveness of the body reference plane technique is then assessed in a similar manner and contrasted with the effectiveness of the AI enabled based solution/system of the present disclosure.
Referring now to the drawings, and more particularly to FIGS. 1 through 7B, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
FIG. 1 depicts an exemplary system 100 for ergonomics assessment and computation of risk score for users, in accordance with an embodiment of the present disclosure. In an embodiment, the system 100 includes one or more hardware processors 104, communication interface device(s) or input/output (I/O) interface(s) 106 (also referred as interface(s)), and one or more data storage devices or memory 102 operatively coupled to the one or more hardware processors 104. The one or more processors 104 may be one or more software processing components and/or hardware processors. In an embodiment, the hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) is/are configured to fetch and execute computer-readable instructions stored in the memory. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices (e.g., smartphones, tablet phones, mobile communication devices, and the like), workstations, mainframe computers, servers, a network cloud, and the like.
The I/O interface device(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface device(s) can include one or more ports for connecting a number of devices to one another or to another server.
The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic-random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, a database 108 is comprised in the memory 102, wherein the database 108 comprises information pose estimation models such as a first model (e.g., OpenPose), a second model (HuMoR), and the like. The database 108 further comprises (i) Rapid Upper Limb Assessment (RULA), Rapid Entire Body Assessment (REBA) frameworks for computation of risk scores, (ii) one or more reference risk scores, (iii) one or more risk alerts, and (iv) one or more ergonomic alerts for user based on an overall score, (v) one or more associated health centric parameters associated with the user (e.g., Body Mass Index (BMI), age, etc.), and so on. The memory 102 further comprises (or may further comprise) information pertaining to input(s)/output(s) of each step performed by the systems and methods of the present disclosure. In other words, input(s) fed at each step and output(s) generated at each step are comprised in the memory 102 and can be utilized in further processing and analysis.
FIG. 2, with reference to FIG. 1, depicts an exemplary high level block diagram of the system 100 of FIG. 1 for ergonomics assessment and computation of risk score for users, in accordance with an embodiment of the present disclosure.
FIG. 3, with reference to FIGS. 1-2, depicts an exemplary flow chart illustrating a method for ergonomics assessment and computation of risk score for users, using the systems 100 of FIG. 1-2, in accordance with an embodiment of the present disclosure. In an embodiment, the system(s) 100 comprises one or more data storage devices or the memory 102 operatively coupled to the one or more hardware processors 104 and is configured to store instructions for execution of steps of the method by the one or more processors 104. The steps of the method of the present disclosure will now be explained with reference to components of the system 100 of FIG. 1, the block diagram of the system 100 depicted in FIG. 2, and the flow diagram as depicted in FIG. 3. Although process steps, method steps, techniques or the like may be described in a sequential order, such processes, methods, and techniques may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order practical. Further, some steps may be performed simultaneously.
At step 202 of the method of the present disclosure, the one or more hardware processors 104 obtain a first model and a second model pertaining to a user. Each of the first model and the second model comprises a video feed comprising one or more tasks performed by the user. The first model may be also referred to as ‘first pose model’, or ‘first pose estimation model’ or ‘first human pose estimation (HPE) model, and interchangeably used herein. The second model may be also referred to as ‘second pose model’ or ‘second pose estimation model’ or ‘second HPE model’, and interchangeably used herein. The first model is a two-dimensional (2D) model, and the second model is a three-dimensional (3D) model, in one embodiment of the present disclosure. More specifically, the first model is an OpenPose model, and the second model is a Human Motion model (also referred to as ‘HuMoR’ model). The second model is derived from the first model. In other words, the HuMoR model is derived from the OpenPose model.
Referring to the step 202, two motion capture methods are compared in determining suitable RULA and REBA scores. First was OpenPose, an open source 2D human pose estimation (HPE) model using a single Red Green Blue (RGB) video input. OpenPose can capture 3D motion by utilizing two or more RGB video cameras with a 3D reconstruction module that triangulates the 3D positions of located joints from the multiple views. For this method, the front facing cameras of three mobile phones were used as the input. In the camera order presented in FIG. 4, the models of the mobile devices used were the iPhone® XR, Samsung® S21 and Blackview® A80 Pro. Each camera was mounted on a tripod where it was elevated 156.5 centimeters (cms), 158.5 cms, and 151.5 cms from the ground, respectively, and were all angled approximately 10° downwards. The tripods were pointed towards the subject and were placed 3 meters away from the subject, separated by angles of 40° with the central camera placed directly in front of the subject. All cameras captured at a rate of 30 fps and at resolutions of 1920 x 1080, 1080 x 720 and 1080 x 720, respectively as illustrated inError! Reference source not found. FIG. 4. More specifically, FIG. 4, with reference to FIGS. 1 through 3, depicts an experimental set-up illustrating placement of one or more video capturing device(s) for capturing tasks being performed by one or more users, in accordance with an embodiment of the present disclosure. For OpenPose, no formal guidelines have been defined to ensure maximum accuracy for 3D motion capture, so the camera configuration used was based on the study performed by existing literature to produce a comparable quality of output achieved in that study. Slight modifications were made to this configuration to adapt to resource and the experiment location’s spatial constraints which includes using different camera models, camera capture settings, cameras elevations, camera distance from the subject and camera downward angles. Intrinsic and extrinsic camera parameters were determined by using the calibration toolkit provided with OpenPose. Video feed from the three cameras were (manually) synced by clapping during the recording phase of the experiment then aligning and trimming the videos post-recording. After recording and syncing footage, the footage from the three cameras were stacked horizontally and fed to OpenPose and the 3D reconstruction module to produce the 3D captured motion. The output of this method is an animated 3D stick representation of the captured actor’s motion where lines, representing limbs, are drawn to connect the joints. The second motion capture method was HuMoR, a generative human motion model (e.g., the second model) that can be used as a prior for robust 3D HPE from ambiguous observations from a single RGB video input. The input for this was simply the same footage from the central camera used for the OpenPose method. The output for this method is an animated imitating the motion of the target actor (e.g., user). The two motion capture methods with their corresponding inputs were run on a Google® Cloud Platform Compute (GCP) Engine equipped with an Intel(R) Xeon(R) CPU @ 2.00GHz and an NVIDIA® Tesla T4. Furthermore, for each of the methods, motion capture of the hands was disabled due to the memory constraints of Graphic Processing Unit (GPU). The tasks performed by a user can be seen in FIG. 4. Such tasks may comprise, but are not limited to, upright standing, trunk flexion at 30 degrees, sitting on a chair/stool, trunk flexed and rotated, placing an object on top, kneeling above the head, arms crossed, holding an object (e.g., a box/container), legs crossed while sitting or standing, sitting at a desk, simple lifting, complex lifting, and the like. Such tasks include various poses/postures by the user. For sake of brevity, graphical representation of the above tasks performed by the user is not shown in FIGS. However, it is to be understood by a person having ordinary skill in the art or person skilled in the art that tasks performed by the user(s) can be depicted by way illustration/figures, and these shall not be construed as limiting the scope of the present disclosure.
Referring to steps of FIG. 3, at step 204 of the method of the present disclosure, the one or more hardware processors 104 extract a plurality of percentage correction key points based on one or more body parts involved in the one or more tasks performed by the user.
At step 206 of the method of the present disclosure, the one or more hardware processors 104 calculate a plurality of body joint angles for the one or more tasks performed by the user using the plurality of percentage correction key points. The steps 204 and 206 are better understood by way of following description:
The system 100 and the method of the present disclosure extract the percentage correction key points and calculate the body joint angles, for which 24 joint angles were obtained from the output body models of the motion capture methods. This consists of neck flexion, side-bending and twisting, trunk flexion, side-bending and twisting, left/right shoulder vertical flexion, horizontal flexion, abduction and raise, left/right elbow flexion, left/right wrist flexion, deviation and twisting, and left/right leg knee flexion. FIG. 5, with reference to FIGS. 1 through 4, depicts an illustration of a reference body plane, in accordance with an embodiment of the present disclosure.
Before calculating joint angles, reference body planes (coronal, sagittal and transverse shown in FIG. 5) which joint angles were measured on were determined with three points each, primarily by the joint’s positions produced by each method, except the transverse plane. For this section, joints will be referred to by their assigned numbers as depicted in FIG. 6. More specifically, FIG. 6, with reference to FIGS. 1 through 5, depicts joints mapped to reference numbers for both the first model (e.g., OpenPose (left)) and the second model (e.g., HuMoR (right)) output, in accordance with an embodiment of the present disclosure.
The coronal plane was defined by joints 6, 16 and 17 for the HuMoR output and for the OpenPose output, it was defined by joints 2, 5 and 8. For the HuMoR output, the sagittal plane was defined by joints 0, 9 and 12. For the OpenPose output, the sagittal plane was defined by the coordinates of the 1 and 8 joint coordinates, and an arbitrary point in front of the OpenPose output located by adding the normal vector of the coronal plane to the midpoint between joints 1 and 8. The transverse plane for both motion capture outputs were defined as the plane perpendicular to sagittal and coronal planes. The calculated reference planes, coronal, sagittal and transverse, are denoted by their normal vectors P_C, P_S and P_T, respectively. Finally, P_C, P_S and P_T were negated if they did not point towards forwards, leftwards and upwards, respectively, relative to the output body models.
The body joint angles for both body models (the first model and the second model) were calculated using an approach similar to that stated in existing literature (e.g., refer M. M. Robertson, et al., S. Dockrell et al., and C Huang et al.). Limb vectors were represented by the difference between two joints.
v_L= p_b-p_a (1)
where v_L denotes the limb vector, and p_a and p_b denote the positions of the joints at the beginning and end of the limb, respectively. Below Table 1 shows limb vectors of interest and joints used to define them by way of illustrative examples.
Table 1
Limb Index, L Corresponding Limb OpenPose (first model) HuMoR (second model)
a b a b
1 Neck 1 0 12 15
2 Upper Spine - 6 12
3 Lower Spine - 0 6
4 Whole Spine 8 1 -
5/6 Left/Right Upper Arm 5/2 6/3 16/17 18/19
7/8 Left/Right Lower Arm 6/3 7/4 18/19 20/21
9/10 Left/Right Upper Leg 12/9 13/10 1/2 4/5
11/12 Left/Right Lower Leg 13/10 14/11 4/5 7/8
13 Shoulder to Shoulder 5 2 16 17
14 Hip to Hip 12 9 1 2
The limb vector and a reference vector, v_ref, were then projected on a body plane as follows:
v_((.))^(P_i )=v_((.) )-v_((.) )·P_i/|(|P_i |)|^2 P_i (2)
where v_((.))^(P_i ) is the projection vector v_((.) ) onto plane P_i. The subscript in v_((.) ) is interchangeable to represent either v_L or v_ref.
The joint angle, ?_k, was calculated as follows:
?_k=s(?v_L?_i^(P_i ),v_ref^(P_i ),? P?_i )·acos?((v_L^(P_i )·v_ref^(P_i ) )/|v_L^(P_i ) ||v_ref^(P_i ) | ) (3)
s(v_L^(P_i ),v_ref^(P_i ),? P?_i )={¦(1,& if (v_L^(P_i )×v_ref^(P_i ))· ? P?_i=0@-1,otherwise)¦ (4)
Where s is a sign function that negates the joint angle if rotation has occurred clockwise relative to the reference plane’s normal. This preserves the notion of negative rotation in the calculation of joint angles. v_ref is either another limb vector or a direction vector (v_left, v_right, v_up, v_down, v_forward and v_backward) where the actual vector representation is dependent on the coordinate system elected by the motion capture output. Prior to any calculation, the output body models were rotated to keep its orientation constant so that direction vectors such as v_left or v_right remained consistent with the output body model without having to recalculate them in the event of a body rotation.
Depending on the joint angle of interest, v_L, v_ref and P_i differed. Error! Reference source not found. shows the mapping of the required vectors and planes for each joint angle and each motion capture method, by way of illustrative examples.
Table 2
Joint Angle Index, k Corresponding Joint Angle Reference Plane Normal, P_i OpenPose (first model) HuMoR (second model)
v_L v_ref v_L v_ref
1 Neck Flexion P_S v_1 v_4 v_1 v_2
2 Neck Twist P_T v_1 v_forwards Same as OpenPose
3 Neck Side-Bending P_C v_1 v_4 v_1 v_2
4 Trunk Flexion P_S v_4 v_up Same as OpenPose
5 Trunk Twist P_T v_13 v_14
6 Trunk Side-Bending P_C v_4 v_up v_2 v_3
7/8 Left/Right Shoulder Vertical Flexion P_S/?-P?_S v_5/v_6 v_down Same as OpenPose
9/10 Left/Right Shoulder Horizontal Flexion ?-P?_T/P_T v_5/v_6 v_left/v_right
11/12 Left/Right Shoulder Abduction P_C/?-P?_C v_5/v_6 v_down
Not all joint angles of interest were calculated using the above method. For elbow and knee flexion, the joint angle was calculated in a much simpler manner as follows:
?_k=acos?((v_a·v_b )/|v_a ||v_b | ) (5)
where v_a and v_b are limb vectors corresponding to the limb segments on either side of knee/elbow of interest. Error! Reference source not found. shows the mapping of joint angles to the limb vectors used to calculate them for both motion capture methods, by way of illustrative examples.
Table 3
Joint Angle Index, k Corresponding Joint Angle v_a v_b
13/14 Left/Right Elbow Flexion v_5/v_6 v_7/v_8
15/16 Left/Right Knee Flexion v_9/v_10 v_11/v_12
The output from the two motion capture methods provided insufficient data to extract left/right shoulder raise, left/right wrist flexion, deviation and twisting and hence were assumed to be in a neutral position when fed into the RULA and REBA frameworks. It is to be understood by a person having ordinary skill in the art or person skilled in the art that the system 100 and the method of the present disclosure implement/execute known in the art RULA and REBA frameworks, and such frameworks shall not be construed as limiting the scope of the present disclosure.
Referring to steps of FIG. 3, at step 208 of the method of the present disclosure, the one or more hardware processors 104 compute at least one of a first risk score (e.g., a RULA score or also referred as RULA risk score) and a second risk score (e.g., a REBA score or also referred as REBA risk score) for each of the plurality of body joint angles pertaining to the first model and the second model.
After the video was fed into both OpenPose model and HuMoR model two sets of output were, one for each method. No action was required to sync the outputs together due to them using same source of video feed. Due to the occlusions to the body, the OpenPose output contained a few frames where the position of some joints was missing. Prior to joint angle calculation, the missing positions of the joints were imputed by interpolation using the joint positions in the frames where the joint position is known to estimate the missing joint positions for the frames in between. If the joint is missing in the frames before the first frame or after the last frame of successful capture, the first or last known position of the joint is assumed constant for the frames before and after, respectively. HuMoR was designed to propose an entire body pose when predicting the body motion regardless of occlusions, so joints positions are never missing across the recorded frames, hence imputation of missing joint positions was not required.
Then, for each motion capture output and for each frame within each posture, joint angles were calculated with method described. With these joint angles, a RULA and REBA rating/score was generated following the respective frameworks per frame. Non-specified joint angle thresholds (e.g., neck bend) were set to 20° as suggested by previous studies and non-postural parameter such as load, and muscle use were set manually according to the posture. For each posture, a nominal RULA and REBA rating/score was chosen to represent the risk of the overall task. For the static postures, (1)-(10), the median RULA and REBA ratings/score across the frames were chosen as the nominal ratings/scores. For the remaining dynamic postures, (11) and (12), the maximum RULA and REBA ratings/score across the frames were chosen as the nominal ratings.
For the manual assessment, the options selected from the human assessors from the given questionnaire were translated as appropriate input to the RULA and REBA frameworks which resulted in the corresponding RULA and REBA scores per posture. Some fields were incorrectly inputted by the participants (left blank or non-viable input), so these were replaced by the mode viable answer given by the other participants for the given field.
Assessing the efficacy of each motion capture method in generating accurate RULA and REBA scores was achieved through calculating two similarity metrics: mean absolute difference per posture. The metrics were calculated between the RULA and REBA scores generated by humans and those derived from either HuMoR or OpenPose. A higher mean absolute difference and a lower proportion agreement index indicates that the motion capture derived RULA and REBA scores are in good agreement with the human generated scores and hence supports that the automated system is accurate enough to perform ergonomic analysis. For each posture, the distribution of human scores and the scores derived from each motion capture system were also visualized to quantitatively assess how the motion capture system derived RULA and REBA scores compare to human generated ratings.
Referring to steps of FIG. 3, at step 210 of the method of the present disclosure, the one or more hardware processors 104 perform a comparison of the first risk score and the second risk score with a reference risk score. The reference risk score may be referred to as a risk score (or reference score) being computed by one or more subject matter experts (SMEs). It is to be understood by a person having ordinary skill in the art or person skilled in the art that based on the previous historical data, the system 100 and method may be configured to compute the reference risk score, wherein the historical data may be fed to the system 100 for training and learning purpose for inference and computation of reference risk score (e.g., either by SME or the system 100 itself). FIG. 7A, with reference to FIGS. 1 through 6, depicts a graphical representation illustrating distribution of the reference risk score (e.g., say a Rapid Entire Body Assessment (REBA) reference risk) per posture compared to risk scores derived from the first pose model and the second pose model, in accordance with an embodiment of the present disclosure. FIGS. 7B, with reference to FIGS. 1 through 7A, depicts a graphical representation illustrating distribution of the reference risk score (e.g., say a Rapid Upper Limb Assessment (RULA) reference risk score) per posture compared to risk scores derived from the first pose model and the second pose model, in accordance with an embodiment of the present disclosure. It is to be understood by a person having ordinary skill in the art or person skilled in the art that the system 100 and method of the present disclosure implement either a single reference risk score for both RULA and REBA frameworks, or there could be a reference risk score for each of these frameworks (e.g., a RULA reference risk score, a REBA reference risk score, and the like). In scenarios, where the system 100 and method implement both the RULA reference risk score, and the REBA reference risk score, at step 210, perform the comparison of the first risk score (e.g., the RULA score or also referred as RULA risk score) with the RULA reference risk score. Similarly, the system 100 and method perform the comparison of the second risk score (e.g., the REBA score or also referred as REBA risk score) with the REBA reference risk score. In scenario, where the system 100 implemented single reference risk score, the system 100 and method perform the comparison of (i) the first risk score and the second risk score with (ii) the reference risk score. In such cases, the value of reference risk score may be common for both RULA and REBA frameworks implemented herein by the present disclosure. The graphical representations in FIGS. 7A and 7B show that, for RULA and REBA, there consists of a subplot for one posture (e.g., posture 1). For sake of brevity graphical representation for posture/pose 1 is depicted for RULA and REBA scores in FIGS. 7A and 7B and it is to be understood by a person having ordinary skill in the art or person skilled in the art that graphical representations for others posture/poses can also be depicted in a similar way and such depiction of the graphical representations for posture/pose 1 for RULA and REBA scores shown in in FIGS. 7A and 7B shall not be construed as limiting the scope of the present disclosure. Each graphical representations of FIGS. 7A and 7B present a histogram, with bars, showing the distribution of human generated RULA/REBA scores (e.g., the reference risk score) for the given posture. The histogram further depicts various line/bar properties showing the scores derived from the first pose model (OpenPose) and the second pose model (HuMoR), respectively. Below Table 4 shows the mean absolute difference respectively, between the OpenPose/HuMoR derived and the reference risk score (e.g., SME generated RULA and REBA scores) per posture/pose.
Table 4
Posture/Pose Index RULA REBA
First model (OpenPose) Second model (HuMoR) First model (OpenPose) Second model (HuMoR)
1 0.44 0.44 0.78 0.78
2 0.89 0.11 2.56 1.56
3 0.11 0.11 0.33 0.33
4 0.44 0.44 2 2
5 0.67 1.44 1.78 1.78
6 0.78 0.78 0 0
7 1.11 0.11 1.22 1.22
8 1.33 1.33 1.22 1.22
9 0.11 0.89 0.33 1.89
10 0.22 0.22 0.22 0.22
11 0.44 0.44 0 0
12 0.67 0.67 0.22 0.22
Average 0.6 0.58 0.89 0.94

It can be observed from graphical representations of FIGS. 7A and 7B (and graphical representations of other poses – not shown in FIGS.) that there is a moderate amount of variability in the RULA and REBA assessments score against the reference risk score (e.g., subject matter expert (SME)/reference computed scores). Thus, this is for a number of reasons:
Due to the slight variability in human perception of the posture.
REBA scores are relatively more varied than the RULA scores even though the input is the same.
The results suggests that REBA framework could be more sensitive to slight changes to perception in posture, this could also be owing to the extra more granular scoring from REBA compared to RULA (5 score levels instead of 4)
Some postures show higher variance of human scores in general such as postures (1), (8), (10) and (12).
With posture (1) as depicted in FIG. 7A and 7B there may have been difficulties accurately assessing the posture. Posture (1) is supposed to be a neutral standing pose but due to the angle and nature of the posture, the human assessors assessed this posture to be more severe than actual.
Referring to steps of FIG. 3, at step 212 of the method of the present disclosure, the one or more hardware processors 104 dynamically identify at least one of the first model and the second model as a best model for each of the plurality of body joint angles based on the comparison. In other words, the best model can be either of the OpenPose (e.g., the first model) or HuMoR (e.g., the second model). More specifically, the at least one of the first model and the second model as the best model is identified based on a mean absolute difference computed between (i) the first risk score (RULA) and the reference risk score, and (ii) the second risk score (REBA) and the reference risk score. The mean absolute difference computed between (i) the first risk score (RULA) and the reference risk score, and (ii) the second risk score (REBA) and the reference risk score is depicted in Table 4 above. Below Pseudo code 1 illustrates an exemplary method for model selection as the best model.
Pseudo code 1: An exemplary method for model selection as the best model.
Input: Ergonomics RULA scores of ‘m’ different tested tasks using 3D pose estimation models such as HuMoR, and OpenPose along with the reference risk scores (e.g., REBA reference risk score, and RULA reference risk score).
Output: Appropriate model selection for each posture/pose based on mean absolute difference ratios
//Initialization
for each posture/pose in i=(1,N) do
Step 1: Subtract RULA scores calculated by first model (OpenPose), and second model (HuMoR) from RULA reference risk scores
Step 2: Select the best model
If the difference between the RULA reference risk score compared against the second model and first model is same then
Continue with the model selected in the previous frame;
else
return Assign either model
if the difference between both the model <=x% then
Take average of both the models;
end
if the difference between both the models > x% and score of second model < first model then
Assign second model as best model for the current frame;
end
if the difference between both the model > x% and score of first model < second model then
Assign first model as best model for the current frame
end
end
end
The best model is selected based on the following conditions described by way of illustrative examples, and such conditions shall not be construed as limiting the scope of the present disclosure.
Condition 1: If the Absolute Mean Difference values between the reference risk score (e.g., reference risk REBA score) and the OpenPose and HuMoR REBA scores are equal, then take either OpenPose or HuMoR model (i.e., continue with the model selected in the previous frame).
Condition 2: If the Absolute Mean Difference values vary by less than equal to, <=x%, then take average of values of both OpenPose and HuMoR REBA scores (value of ‘x’ is say 10)
Condition 3: If the Absolute Mean Difference values of HuMoR is less (<) than Difference values of OpenPose and difference is greater (>) then x%, select the HuMoR value and HuMoR model (e.g., the second model) as the best model.
Condition 4: If the Absolute Mean Difference values of OpenPose is less (<) than the Difference values of HuMoR and difference is greater (>) than x%, then select the OpenPose value and OpenPose model (e.g., the first model) as the best model.
In the present disclosure, the above pseudo code/model selector algorithm considered REBA scores of 12 different tested task postures as input. Depending on this, the pseudo code/model selector algorithm is able to select the appropriate model based on the evaluation on the difference between the absolute mean difference along with satisfying some conditions. This absolute mean difference is computed between the REBA reference risk scores against the first risk score of the first model (OpenPose) and the second risk score of the second model (HuMoR) computed scores on those 12 postures, as expected output. Below Tables 5 depicts best model selection with REBA score being considered.
Table 5
Posture REBA Reference risk score REBA OpO Score REBA HuM Score Reference-OpO Diff (Diff_RefO) Reference-Humor Diff (Diff_RefH) Model to select
1 1.78 0.78 0.78 1 1 Either
2 4.56 2.56 1.56 2 3 HuMoR
3 2.67 0.33 0.33 2.34 2.34 Either
4 5 2 2 3 3 Either
5 4.78 1.78 1.78 3 3 Average
6 5 0 0 5 5 Either
7 2.22 1.22 1.22 1 1 HuMoR
8 3 1.22 1.22 1.78 1.78 Either
9 3.11 0.33 1.89 2.78 1.22 OpPo
10 3.22 0.22 0.22 3 3 Either
11 5 0 0 5 5 Either
12 4.78 0.22 0.22 4.56 4.56 Either

Similarly, below Tables 6 depicts best model selection with RULA score being considered, and with score computed at the last column.
Table 6
Posture RULA Reference risk score RULA OpO Score RULA HuM Score Reference-OpO Diff (Diff_RefO) Reference-Humor Diff (Diff_RefH) Model to select Score calculated
1 0.44 0.44 0.44 0 0 Either 0.440
2 0.89 0.89 0.11 0 0.78 HuMoR 0.890
3 0.11 0.11 0.11 0 0 Either 0.110
4 0.44 0.44 0.44 0 0 Either 0.440
5 0.67 0.67 0.72 0 -0.05 Average 0.695
6 0.78 0.78 0.78 0 0 Either 0.780
7 1.11 1.11 0.11 0 1 HuMoR 1.110
8 1.33 1.33 1.33 0 0 Either 1.330
9 0.11 0.11 0.89 0 -0.78 OpPo 0.890
10 0.22 0.22 0.22 0 0 Either 0.220
11 0.44 0.44 0.44 0 0 Either 0.440
12 0.67 0.67 0.67 0 0 Either 0.670

Referring to steps of FIG. 3, at step 214 of the method of the present disclosure, the one or more hardware processors 104 compute for each frame pertaining to the video feed comprised in the best model, at least one of a first refined risk score (RULA score) and a second refined risk score (REBA score) pertaining to each of the plurality of body joint angles.
Once appropriate model is selected, it is dynamically allocated by the system 100 for further processing and analysis. Below Pseudo code 2 illustrates an exemplary method for dynamic allocation of the selected best model.
Pseudo code 2: An exemplary method for dynamic allocation of the selected best model.
Input: Table 1: Ergonomics poses/postures and associated model information
Table 2: Ergonomics poses/postures frames
Output: First refined risk score (RULA score) and second refined risk score (REBA score)
for each pose/posture (in Table 2) i=(1,N) do
Step 1: Fetch the model (first model (OpenPose)/second model (HuMoR)) name from Table 1 on the basis of ‘m’ tested task poses/postures
Step 2: Score calculator recommender system refined
if model = “Either” then
Select “Second model/HuMoR”
else
Select as it is filled
end
Step 3: First refined risk score and second refined risk score
if model = “second model/HuMoR” then
Full second risk score
else if model = “first model/OpenPose” then
Full first second risk score
else
Fill average of scores of both the first model and the second model
End
Step 4: Calculate the frequency of high risk poses within a window of certain number of frames (e.g., 5/10/20)
Calculate the frequency of duration of high risk poses within a window of certain number of frames (e.g., 5/10/20)
Calculate high risk postures ranging between p% (e.g., low), q% (e.g., medium), and r% (e.g., high) and s% (e.g., very high) of total frames by mapping all the ergonomics posture whose value is greater than high (e.g., r%), wherein values of p, q, r, and s are 20, 40, 60, and 80 respectively.
end
The above Pseudo code 2 is better understood by way of following description:
The above Pseudo code 2 computes a refined model recommender RULA/REBA risk score. As an input this algorithm considers all the selected models computed from Pseudo code 1. In Pseudo code 2, the system 100 first fetches the select models computed from Pseudo code 1. In the second step of the refined risk score calculator module, the system 100 maps those recommended models to its associated re-identified postures based on its mapping of previously identified posture from the Pseudo code 1 against set of frames (5, 10, 20 frames for one posture out of 12 tested task postures) associated with their posture. In the third step, a further refined risk score is then computed based on logical condition as highlighted below:
Condition 1 in Step2: In the refined risk score calculator module, If the model is “Either”, then select “HuMoR/second model” otherwise leave as previously computed. This logic substitutes all the models which were computed as “Either” of the model and provide the refined risk score calculator module refined version.
Condition 1 in Step3: In the refined RULA/REBA risk score refined, If the model is “second model/HuMoR”, then get the computed scores of the HuMoR model (e.g., the second refined risk score).
Condition 2 in Step3: Else if the model is “OpenPose/first model”, then get the computed scores of the OpenPose model (e.g., the second refined risk score). Else compute the average of the scores of both (HuMoR and OpenPose).
Furthermore, once all the results are obtained from the above steps, the frequency of high risk poses within a window of a certain number of frames (i.e., 5, 10, 20) is calculated. In addition, the system 100 calculates the duration of the high risk poses within a window of a certain number of frames (i.e., 5, 10, 20). Finally, the system 100 calculates high risk postures ranging between 20%(low), 40%(medium), 60%(high), and 80% (very high) of total frames by mapping all the ergonomics postures whose value is greater than high (60%). Below Table 7 depicts the first refined risk score for various frames of the video feed, by way of illustrative examples:
Table 7
Frame# Identified Posture RULA HuMoR RULA OpenPose Score Calculator Recommender System v1 Score Calculator Recommender System refined Refined RULA Risk Score
XYZ001001 2 59 54 HuMoR HuMoR 59
XYZ001002 2 50 43 HuMoR HuMoR 50
XYZ001003 3 51 58 Either HuMoR 51
XYZ001004 2 53 30 HuMoR HuMoR 53
XYZ001005 1 59 22 Either HuMoR 59
XYZ001006 1 25 44 Either HuMoR 25
XYZ001007 2 56 22 HuMoR HuMoR 56
XYZ001008 3 14 59 Either HuMoR 14
XYZ001009 4 27 10 Either HuMoR 27
XYZ001010 5 52 20 Average Average 36
XYZ001011 6 15 13 Either Average 14
XYZ001012 5 39 20 Average Average 29.5
XYZ001013 6 53 36 Either Average 44.5
XYZ001014 7 19 31 HuMoR HuMoR 19
XYZ001015 8 59 34 Either HuMoR 59
XYZ001016 9 39 37 OpPo OpPo 37
XYZ001017 9 17 52 OpPo OpPo 52
XYZ001018 10 24 46 Either OpPo 46
XYZ001019 11 45 28 Either OpPo 28
XYZ001020 12 16 26 Either OpPo 26

Below Table 8 depicts identified pose(s)/postures and frequency of identified pose(s)/posture(s):
Table 8
Tasks Identified pose(s)/posture(s) Frequency of identified pose(s)/posture(s)
0 1 2
1 2 4
2 3 2
3 4 1
4 5 2
5 6 2
6 7 1
7 8 1
8 9 2
9 10 1
10 11 1
11 12 1

Below Table 9 depicts identified pose(s)/postures, total successive frames, and associated duration by way of illustrative examples:
Table 9
Task Identified pose(s)/posture(s) Total successive frames fpr Total duration
0 1 1 0.100000 0.100000
1 2 3 0.150000 0.450000
2 3 1 0.150000 0.150000
3 4 0 0.137500 0.000000
4 5 1 0.127500 0.127500
5 6 1 0.121250 0.121250
6 7 0 0.117321 0.000000
7 8 0 0.114665 0.000000
8 9 1 0.112741 0.112741
9 10 0 0.111274 0.000000
10 11 0 0.110116 0.000000
11 12 0 0.109176 0.000000

Below Table 10 depicts first refined risk score and second refined risk score by way of illustrative examples:
Table 10
Frame# Identified Posture RULA HuMoR RULA OpenPose Score Calculator Recommender System v1 Score Calculator Recommender System refined Refined RULA Risk Score Posture Model to select Score Calculator Recommender System refined_new Refined RULA/ REBA Risk Score_new
0 XYZ001001 2 59 54 HuMoR HuMoR 59.0 2 HuMoR HuMoR 59.0
1 XYZ001002 2 50 43 HuMoR HuMoR 50.0 2 HuMoR HuMoR 50.0
2 XYZ001004 2 53 30 HuMoR HuMoR 53.0 2 HuMoR HuMoR 53.0
3 XYZ001007 2 56 22 HuMoR HuMoR 56.0 2 HuMoR HuMoR 56.0
4 XYZ001003 3 51 58 Either HuMoR 51.0 3 Either HuMoR 51.0
5 XYZ001008 3 14 59 Either HuMoR 14.0 3 Either HuMoR 14.0
6 XYZ001005 1 59 22 Either HuMoR 59.0 1 Either HuMoR 59.0
7 XYZ001006 1 25 44 Either HuMoR 25.0 1 Either HuMoR 25.0
8 XYZ001009 4 27 10 Either HuMoR 27.0 4 Either HuMoR 27.0
9 XYZ001010 5 52 20 Average Average 36.0 5 Average Average 36.0
10 XYZ001012 5 39 20 Average Average 29.5 5 Average Average 29.5
11 XYZ001011 6 15 13 Either Average 14.0 6 Either HuMoR 15.0
12 XYZ001013 6 53 36 Either Average 44.5 6 Either HuMoR 53.0
13 XYZ001014 7 19 31 HuMoR HuMoR 19.0 7 HuMoR HuMoR 19.0
14 XYZ001015 8 59 34 Either HuMoR 59.0 8 Either HuMoR 59.0
15 XYZ001016 9 39 37 OpPo OpPo 37.0 9 OpPo OpPo 37.0
16 XYZ001017 9 17 52 OpPo OpPo 52.0 9 OpPo OpPo 52.0
17 XYZ001018 10 24 46 Either OpPo 46.0 10 Either HuMoR 24.0
18 XYZ001019 11 45 28 Either OpPo 28.0 11 Either HuMoR 45.0
19 XYZ001020 12 16 26 Either OpPo 26.0 12 Either HuMoR 16.0

Referring to steps of FIG. 3, at step 216 of the method of the present disclosure, the one or more hardware processors 104 compute an overall risk score for the best model using (i) the at least of the first refined risk score and a second refined risk score, and (ii) at least one of a frequency and a duration of a movement of the one or more body parts in successive frames involved in the one or more tasks performed by the user. The overall risk score is computed using the above details specified in the Tables 8, and 9 wherein the overall risk score is shown in Table 10.
Referring to steps of FIG. 3, at step 218 of the method of the present disclosure, the one or more hardware processors 104 generate one or more ergonomics risk alerts for the user based on the overall score and one or more associated health centric parameters (BMI, age, and the like) associated with the user. The steps 216 and 218 are better understood by way of following description. Below Pseudo code 3 illustrates an exemplary method for generating one or more risk alerts and one or more ergonomic alerts for the user.
Pseudo code 3: An exemplary method for generating one or more risk alerts and one or more ergonomic alerts for the user.
Input: Data from Table 2 with details of ergonomics pose/postures, BMI, age, frequency, duration of high-risk scores.
Output: Model to predict RULA/REBA score (also referred as Overall risk score)
Step 1: Prepare the dataset and filter ergonomics poses/postures greater than r% and frequency of risk poses/postures greater than r%
Step 2: Run multiple variable regression model(s)
Step 3: y_i=ß_0+ß_1 x_i1+ß_2 x_i2+ß_3 x_i3+?ß_p x_ip+?
where, for i=n observations:
y_i= dependent variable
x_i= explanatory variable
ß_0 = y-intercept (constant term)
ß_p = slope coefficients for each explanatory variable
? = the model’s error term (also known as the residual(s))
Model equation:
Overall risk score (RULA/REBA score) = ß_0+ß_1 ?Age?_1+ß_2 ?BMI?_2+ß_3 ?frequency?_3+?ß_4 ?duration?_p+?
Step 4: Use this model to predict RULA/RUBA Score and compare with refined RULA/REBA score.
The above Pseudo code 3 is better understood by way of following description:
The Pseudo code 3 computes a multiple variable regression model to predict RULA/REBA scores based on the results computed from Pseudo code 1 and Pseudo code 2. The Pseudo code 3 considers results from Pseudo code 1 and Pseudo code 2 as dataset. For this, the system 100 computes the age and Body Mass Index (BMI) of only the persons/users who is at higher risk from Pseudo code 2. The system 100 has already computed the frequency and duration of the high-risk posed from Pseudo code 2. In order for our multiple variable regression model to predict a REBA score accurately; variables such as age, BMI, frequency, and duration of high-risk poses become explanatory variables become pivotal. Once the machine learning multiple variable regression model is computed/obtained, then the system 100 compares and contrasts against the previously computed (from Pseudo code 2) refined RULA/REBA scores. Therefore, from the results it is evident that due to the low quality of the dataset there was much variability however, considering a rich dataset would yield better outcomes.
Below Table 11 depicts overall risk score (e.g., REBA risk score/refined REBA risk score), frequency of high-risk score, duration of high-risk score, and the one or more associated health centric parameters (BMI, age, and the like) associated with the user, by way of illustrative examples.
Table 11
Refined_REBA_Score_Risk_Score_(Above_6_or_60%) Frequency_of_the_High_Risk_Score Duration_of_the_High_Risk_Score Age BMI
59.0 8 26 31 23
50.0 9 27 58 27
51.0 5 23 53 29
53.0 10 30 55 23
59.0 9 24 42 24

Based on the above details mentioned in Table 11, one or more one or more ergonomics risk alerts are generated for the user. Below illustrates a pseudocode that describes a method for ergonomics risk alerts generated for the user, by way of examples:
Pseudocode for generation of ergonomics risk alerts:
Input: A dataset 'data' containing columns: RULA Score, Frequency of High Risk, Duration of High Risk, AGE, and BMI
Output: A list of individuals who meet the risk alert criteria

1. Initialize an empty list called 'risk_alert_individuals'
2. Define the following thresholds:
- RULA_THRESHOLD = 60%
- FREQUENCY_THRESHOLD = 7
- DURATION_THRESHOLD = 25
- AGE_THRESHOLD = 50
- BMI_THRESHOLD = 25
3. For each individual 'i' in 'data' do
3.1. If 'i'.RULA Score = RULA_THRESHOLD
3.1.1. Add 'i' to 'risk_alert_individuals'
3.2. Else if 'i'.Frequency of High Risk > FREQUENCY_THRESHOLD AND 'i'.Duration of High Risk > DURATION_THRESHOLD
3.2.1. Add 'i' to 'risk_alert_individuals'
3.3. Else if 'i'.AGE > AGE_THRESHOLD AND 'i'.BMI > BMI_THRESHOLD
3.3.1. Add 'i' to 'risk_alert_individuals'
3.4. End If
4. End For
5. Return 'risk_alert_individuals'
End Algorithm
Based on the above algorithm pseudocode the below table is generated as an output:
The above algorithm pseudocode generated the following outcome for high-risk alert depicted by way examples in Table 12:
Table 12
Refined RULA Score Risk Score (Above 6 or 60%) Frequency of the High-Risk Score Duration of the High-Risk Score AGE BMI High Risk Alert
59 8 26 31 23 TRUE
50 9 27 58 27 TRUE
51 5 23 53 29 TRUE
53 10 30 55 23 TRUE
59 9 24 42 24 FALSE
25 6 14 41 25 FALSE
56 9 26 54 22 TRUE
14 7 29 38 28 FALSE
27 5 30 37 22 FALSE
36 10 11 52 21 FALSE
14 6 22 30 24 FALSE
29.5 8 17 41 22 FALSE
44.5 7 28 52 21 FALSE
19 7 15 57 30 TRUE
59 6 25 39 27 FALSE
37 5 24 40 22 FALSE
52 8 19 42 23 FALSE
46 8 27 58 25 TRUE
28 8 15 43 30 FALSE
26 6 28 50 29 FALSE

In Table 12, the flag TRUE indicates that there is a high ergonomic risk alert for a set of users and the flag FALSE indicates that there is a low/medium/moderate ergonomic risk alert for another set of users. For sake of brevity the above description and Table 12 is provided for RULA, and it is to be understood by a person having ordinary skill in the art or person skilled in the art that similar algorithm pseudocode can also be depicted/generated by the system along with outcome for high-risk alert for REBA, and such depiction of ergonomics risk alerts for RULA and REBA shall not be construed as limiting the scope of the present disclosure.
Based on the results and techniques described above, the embodiments of the present disclosure provide systems and method for computation of risk score and generation of risk alerts and ergonomic alerts for users. More specifically, the systems and methods described herein receive the monocular RGB videos/images as inputs, wherein the input is processed and the OpenPose 3D pose estimation model (e.g., first model) is executed, which helps identify correct percentage key points. Further, the same processed video is then fed into the HuMor model (e.g., second model) which outputs a more volumetric results and this helps identifying correct percentage keypoints more accurately. The correctly identified percentage key points are then mapped onto the affected body parts identified and their subsequent movement to obtain the correct body joint angles. This activity is accomplished by using the body movement based on body angles computed from the projection planes (such as Sagittal, Coronal and Traverse planes) by computing their appropriate reference direction relative to the body. Next, these computed body joint angles are then automatically used to compute the first risk score and the second risk score (RULA/REBA scores). Finally, an AI analytical engine (not shown in FIGS.) has been implemented which provides an ergonomics risk recommender alert consisting of three Pseudo codes (Pseudo code 1, Pseudo code 2, and Pseudo code 3).
Moreover, the system and method of the present disclosure analyzed the first model and the second model (e.g., OpenPose and HuMoR 3D pose estimation models) and their effects of correctly identifying percentage correct keypoints (PCK), which are essential for calculating accurate body joint angles for risk posture assessment. The results suggested that in some scenarios OpenPose (e.g., the first model) performed slightly better than HuMoR model (e.g., the second model), at the same time in many cases HuMoR model outperformed OpenPose, however in most cases HuMoR outperformed OpenPose. In addition, the system and method of the present disclosure further analyzed the reference risk scores computed by SME(s) (e.g., manually computed RULA/REBA scores) against the 3D pose estimation models (OpenPose and HuMoR model) computed RULA/REBA scores (e.g., the first risk score and the second risk score). The observation of this comparison has proved that depending on the mean absolute difference of manually and 3D pose estimation models (OpenPose and HuMoR) computed RULA/REBA scores form the basis of the model selection for identifying the best model. The method further enhances the system which includes a refined model recommender and a RULA/REBA risk score calculator which enables the system to recommend the best model based on the frames and their mapped identified postures from the Pseudo code 2.
The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g., any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g., hardware means like e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g., an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g., using a plurality of CPUs.
The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated by the following claims.
, Claims:
1. A processor implemented method, comprising:
obtaining, via one or more hardware processors, a first model and a second model pertaining to a user (202), wherein each of the first model and the second model comprises a video feed comprising one or more tasks performed by the user;
extracting, via the one or more hardware processors, a plurality of percentage correction key points based on one or more body parts involved in the one or more tasks performed by the user (204);
calculating, via the one or more hardware processors, a plurality of body joint angles for the one or more tasks performed by the user using the plurality of percentage correction key points (206);
computing, via the one or more hardware processors, at least one of a first risk score and a second risk score for each of the plurality of body joint angles pertaining to the first model and the second model (208);
performing, via the one or more hardware processors, a comparison of the first risk score and the second risk score with a reference risk score (210);
dynamically identifying, via the one or more hardware processors, at least one of the first model and the second model as a best model for each of the plurality of body joint angles based on the comparison (212);
computing, via the one or more hardware processors, for each frame pertaining to the video feed comprised in the best model, at least one of a first refined risk score and a second refined risk score pertaining to each of the plurality of body joint angles (214);
computing, via the one or more hardware processors, an overall risk score for the best model using (i) the at least of the first refined risk score and a second refined risk score, and (ii) at least one of a frequency and a duration of a movement of the one or more body parts in successive frames involved in the one or more tasks performed by the user (216); and
generating, via the one or more hardware processors, one or more ergonomics risk alerts for the user based on the overall score and one or more associated health centric parameters associated with the user (218).

2. The processor implemented method as claimed in claim 1, wherein the second model is derived from the first model.

3. The processor implemented method as claimed in claim 1, wherein the first model is a two-dimensional (2D) model, and the second model is a three-dimensional (3D) model.

4. The processor implemented method as claimed in claim 1, wherein the step of dynamically identifying at least one of the first model and the second model as the best model is based on a mean absolute difference computed between (i) the first risk score and the reference risk score, and (ii) the second risk score and the reference risk score.

5. A system (100), comprising:
a memory (102) storing instructions;
one or more communication interfaces (106); and
one or more hardware processors (104) coupled to the memory (102) via the one or more communication interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to:
obtain a first model and a second model pertaining to a user, wherein each of the first model and the second model comprises a video feed comprising one or more tasks performed by the user;
extract a plurality of percentage correction key points based on one or more body parts involved in the one or more tasks performed by the user;
calculate a plurality of body joint angles for the one or more tasks performed by the user using the plurality of percentage correction key points;
compute at least one of a first risk score and a second risk score for each of the plurality of body joint angles pertaining to the first model and the second model;
perform a comparison of the first risk score and the second risk score with a reference risk score;
dynamically identify at least one of the first model and the second model as a best model for each of the plurality of body joint angles based on the comparison;
compute for each frame pertaining to the video feed comprised in the best model, at least one of a first refined risk score and a second refined risk score pertaining to each of the plurality of body joint angles;
compute an overall risk score for the best model using (i) the at least of the first refined risk score and a second refined risk score, and (ii) at least one of a frequency and a duration of a movement of the one or more body parts in successive frames involved in the one or more tasks performed by the user; and
generate one or more ergonomics risk alerts for the user based on the overall score and one or more associated health centric parameters associated with the user.

6. The system as claimed in claim 5, wherein the second model is derived from the first model.

7. The system as claimed in claim 5, wherein the first model is a two-dimensional (2D) model, and the second model is a three-dimensional (3D) model.

8. The system as claimed in claim 5, wherein the at least one of the first model and the second model is dynamically identified as the best model based on a mean absolute difference computed between (i) the first risk score and the reference risk score, and (ii) the second risk score and the reference risk score.