DESC:FIELD OF INVENTION
The present invention relates to a system and method for foot measurement. Preferably, the present invention relates to a method for measuring foot dimensions through an application running on a mobile device. More preferably, the present invention enables measuring foot dimensions by a mobile device installed with customized software that uses the device’s camera. It takes a single image of a human foot and instantly provides the user's foot measurements (i.e., Length and width) without any physical reference object.

BACKGROUND OF THE INVENTION

With the advent of high penetration of the internet all over the world, online shopping has exploded with the choice and variety of offered goods and the shoe industry is amongst the list of fast selling goods. This boom in online purchasing has given rise to relatively high rates of online returns. The main reason for most returns is poor fitting, likely due to customers being unable to measure the dimensions and parameters required for proper fitting or they may be unwilling to do so. Shoe fitting is a matter of concern in footwear industries also because the size and shape of the foot depend on the wearer and the standards adopted by the manufacturer.

The current approaches for measuring foot size are usually traditional and laborious. One traditional method is the use of physical devices like the Brannock device, which measures both the length and width of the foot. These tools have the benefit of giving the user feedback in real time so they may, if necessary, adjust the measurements. However, they require the user to be physically present in a store or other location where the device is available. Another traditional method is the process of trial and error, in which the user tries on multiple pairs of shoes of different sizes to determine the best fit. This method is dependent on the user's ability to provide accurate feedback about the fit of the shoes, which may be difficult for some individuals, such as children or those with physical disabilities. There is also a group of people who purchase new shoes completely based on their previously worn shoe sizes. However, problems may still arise since different brands have different nomenclatures to represent the same foot dimensions.

Hence, it is clear that there is a demand for more convenient and at the same time fast and accurate methods of measuring foot size. Each of these methods has its limitations and inconveniences. For example, methods that rely on the user tracing their foot may be tedious or require a certain level of skill to do accurately. Another method of doing so is to place an object of known dimension like A4 paper or a coin on the ground and keep their feet next to the physical object and use an app to measure the foot by taking the known object’s dimension as reference. Though this comes with its advantages, its entire algorithm depends on the availability of the physical object of the particular dimension as required by the algorithm. With the advent of technologies like VR/AR and AI technologies along with the services by cloud service providers, we can make use of these technologies to upgrade to faster and more accurate measurement systems in an extremely convenient method, without any physical hassles. Thus, a need arises to improve customer service, disrupting how current online shoe purchasing operates.

OBJECTIVES OF THE INVENTION

The primary objective of the present invention is to provide a system and method for measuring foot dimensions using a mobile device.

Another objective of the present invention is to provide a method for measuring foot dimensions using augmented reality-based software installed in a mobile device of the user.

Yet another objective of the invention is to provide a faster and more accurate foot measurement system.

Still, another objective of the invention is to provide an application for measuring the length and width of a human foot with just a click of an image from the user’s mobile device equipped with a camera.

Yet another objective of the invention is to provide a methodology for foot measurement that does not require a physical reference for the foot measurement.

Other objectives and advantages of the present invention will become apparent from the following description taken in connection with the accompanying drawings, wherein, by way of illustration and example, the aspects of the present invention are disclosed.

SUMMARY OF THE INVENTION

The present invention provides an application for measuring the length and width of a human foot with just a click of an image from the user’s mobile device equipped with a camera. The methodology of the present invention is based on specialized software that performs Augmented reality-based conversions on the images captured by the camera of the user’s device and does not require any other physical object as a reference. Initially, the user can open the application and perform stepwise procedures based on the instructions provided. Once performed correctly, an AR object projection will appear on the mobile screen. The user is then required to place their foot as per instructions provided in the application. Thereafter, the user can capture an image. The user is then provided with their foot dimensions within a few seconds. Once the application receives the data from the user, it undergoes various pre-processing techniques like rotation, transformation, cropping, etc. in order to meet the desired requirements for performing foot segmentation. The segmented mask of the foot then undergoes various mathematical computations like numerical differentiation, trigonometric operations, etc. for finding desired parameters and check foot orientation. These parameters are then used to calculate the length and width of the user’s foot. In case the foot image cannot pass through the checks, the user is asked to repeat the same procedure and obtain their foot dimensions.

BRIEF DESCRIPTION OF DRAWINGS

The present invention will be better understood after reading the following detailed description of the presently preferred aspects thereof concerning the appended drawings, in which the features, other aspects, and advantages of certain exemplary embodiments of the invention will be more apparent from the accompanying drawing in which:

Figure 1 illustrates the procedure to scan the floor to create a mesh of the user’s surroundings.

Figure 2 illustrates a virtual 3D object seamlessly integrated into the user's view of the real world using AR.

Figure 3 illustrates the procedure of placing the foot inside the virtual 3-D object i.e. the projected box.

Figure 4 illustrates the pre-computation part of the algorithm in which a mobile application is launched to collect user data and send it to the cloud server for computation.

Figure 5 illustrates about the transformation of bounding box coordinates, foot image segmentation using YOLACT++, removing geometric distortion using perspective transformation, and algorithm applied to check whether foot is parallel and inside the box.

Figure 6 illustrates about the foot length and width computation using the contour extreme points and at last, we apply an error check.

Figure 7 shows the foot in bounding box along with the box corner coordinates.

Figure 8 shows the segmented foot and extreme points for the calculation of length and width.

Figure 9 shows the angle check used for determining whether the foot is parallel or not.

DETAILED DESCRIPTION OF THE INVENTION

The following description describes various features and functions of the disclosed system concerning the accompanying figures. In the figures, similar symbols identify similar components, unless context dictates otherwise. The illustrative aspects described herein are not meant to be limiting. It may be readily understood that certain aspects of the disclosed system can be arranged and combined in a wide variety of different configurations, all of which have not been contemplated herein.

Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

Features that are described and/or illustrated concerning one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.

The terms and words used in the following description are not limited to the bibliographical meanings, but, are merely used to enable a clear and consistent understanding of the invention. Accordingly, it should be apparent to those skilled in the art that the following description of exemplary embodiments of the present invention are provided for illustrative purposes only and not to limit the invention.

It is to be understood that the singular forms “a”, “an” and “the” include plural referents unless the context dictates otherwise.

It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, steps, or components but does not preclude the presence or addition of one or more other features, steps, components, or groups thereof.

The term ‘mobile device’ hereinafter refers to any handheld smart device configured with a camera, and includes devices such as but not limited to smartphones, laptops, tablets, phablets, and the like. Particularly, the mobile device is equipped with a monocular camera to magnify images of distant foot.

Accordingly, the present invention provides an application for measuring the length and width of a human foot with just a click of an image from the user’s mobile device equipped with a monocular camera. Initially, the user can open the application and perform stepwise procedures based on the instructions provided. Once performed correctly, an AR object projection will appear on the mobile screen. The user is then required to place their foot as per instructions provided in the application. Thereafter, the user can capture an image through the camera. The user is then provided with their foot dimensions within a few seconds.

Once the application receives the data from the user, it undergoes various pre-processing techniques like rotation, transformation, cropping, etc. in order to meet the desired requirements for performing foot segmentation. The segmented mask of the foot then undergoes various mathematical computations like numerical differentiation, trigonometric operations, etc. for finding desired parameters and checks like foot orientation. These parameters are then used to calculate the length and width of the user’s foot. In case the foot image cannot pass through the checks, the user is asked to repeat the same procedure and obtain their foot dimensions.

In an embodiment of the present invention, a system for augmented reality-based foot size measurement, comprising: at least one handheld computing device equipped with an image capturing device; a software module installed in the handheld computing device, comprising a data storage module for storing a plurality of instructions and user data, a processing module comprising an augmented reality (AR) based application, a mapping module, a textured floor detection module configured to detect whether floor is textured or not, a key point detection module, an image distortion correction module, a heel detection module, a foot segmentation module, a foot dimension calculation module, and a display module having graphical user interface.

In an embodiment of the present invention, the mapping module is configured to create a mesh of surroundings that enables one to understand the layout of a real-world space and to accurately place virtual objects in the real-world space. The key point detector module is configured to distinguish a barefoot image of a user from images depicting individuals wearing socks or shoes. The heel detection module is configured to instruct the user to place the foot inside the projected box, to ensure the foot of the user is placed parallel to the longer edge of the box. The image distortion correction module is configured to remove geometric distortion and distortion caused by camera perspective using the AR applied for checking whether the foot is parallel and inside the projected box. The foot segmentation module is configured to extract the foot from the rest of the image. The foot dimension calculation module is configured to calculate the dimension of the foot from the segmented image. The display module is configured to display the measured length and width of the foot.

Figure 1 shows the stepwise procedure to scan the floor by horizontally moving the mobile device in such a way that the application can understand the environment and create a mesh of the surroundings. An Augmented Reality based application is used to project a 3D model (reference object) in the real world. Scanning the environment in AR is necessary to understand the layout of the real-world space and to accurately place virtual objects in it. This process is known as "environment mapping" or "SLAM (Simultaneous Localization and Mapping)” and is performed by the mapping module. By scanning the environment using the mapping module, the AR system can build a map of the space, including information about the location and size of objects, and the layout of the room or area. This information is used to align virtual objects correctly with the real-world environment. The AR system can use the camera on a device such as a smartphone or tablet, or a separate sensor, to capture images and data about the environment. This data is then processed using computer vision algorithms to identify features such as corners and edges and to create a 3D map of the space. Once the environment has been mapped, the AR system can use the map to understand the device's position and orientation relative to the real-world environment. This is known as "6 degrees of freedom" and it allows the system to place virtual objects correctly in the real-world space.
Figure 2 illustrates an exemplary embodiment of the present invention, wherein a virtual 3D object has been seamlessly integrated into the user's view of the real world using augmented reality technology. The mapping module is used to project the AR object by tapping at any desired location on the mobile screen to create a mesh surrounding and can be rotated as per the user’s choice of capture. All four corners of the AR object must be visible on the mobile screen. The object as shown in Figure 2 has a length of 350mm and a width of 210mm, and a negligible height. This object was created using a software program known as Blender. Additionally, this virtual object also features child nodes located at each corner. This object is placed in a 3D space, where the z-axis is perpendicular to the X-Y plane of the object. This placement allows for a more realistic integration of the virtual object into the user's field of view and enhances the overall immersion of the augmented reality experience.

Figure 3 depicts guidance on how to correctly position the left foot within a rectangular box that has been projected. The user’s foot has to be placed in such a way that the heel and toe must be visible, and the foot is aligned with the direction of the arrow on top of the box. Also, the foot must be parallel with the longer edge of the box. The heel detection module is used to instruct the user through an audio recording to align their foot parallel to the length of the box, ensuring that the entire foot including both heel and toes is fully within the box and visible. The foot should also be aligned with an arrow that is visible in the image, and, also make sure that the foot is placed in the center of the box and not extending beyond it.

It is also important that the user is barefoot, which means without wearing any socks or footwear. In an embodiment of the present invention, the key point detection module is used to distinguish a barefoot image of a user from images depicting individuals wearing socks or shoes. When analyzing images of bare feet, the key point detection module accurately identifies the positions of eight distinct key points on the foot. However, when presented with images of individuals wearing footwear or socks, the key point detection module struggles to precisely locate these key points, resulting in the clustering of points around the general vicinity of the feet. To determine whether the user is barefoot or wearing socks/shoes, the technique employed involves calculating the summed Euclidean distance between each key point. If the sum of these distances falls below a predefined threshold value, it is concluded that the user is not barefoot. This approach works effectively because, in barefoot images, the key points are more widely distributed, leading to a higher cumulative distance. Conversely, when the feet are obscured by footwear or socks, the key points tend to form a compact cluster, resulting in a smaller cumulative distance. This enables the key point detection module to reliably discern whether the individual is wearing shoes, or socks, or is indeed barefoot.
A mobile application is configured in the user’s device to collect user data and the data collected is sent to a cloud server for computation, which performs the pre-computation part of the algorithm.

Figure 4 illustrates the process of how the user submits an image through the application to obtain measurements, according to an embodiment of the present invention. The process comprises the following steps:
An AR-based mobile application/software is launched in the user’s device;
The application uses the camera of the mobile device, or a separate sensor, to capture images and data about the environment. The floor is scanned by horizontally moving the mobile device to create a mesh of the user’s surroundings.
An augmented reality-based object is projected by tapping on the screen of the mobile device, such that all corners of the AR object are visible on the mobile screen.
The Bare Foot of the user is placed in the projected box in such a manner that the heel and toe of the user are visible and the foot is aligned with the direction of the arrow on the top of the box.
A heel detection module in the software application instructs the user to place the foot inside the projected box which also ensures that the foot of the user is placed parallel to the longer edge of the box.
The flash is turned on to get a clear visibility of the heel and toes as shown in step 6 before capturing the image.
Step 7 illustrates capturing the image as per the conditions mentioned in steps 5 and 6.
The foot is shown within the bounding box along with the box corner coordinates.
After capturing the image, the user may optionally perform an error check as shown in step 9 of Figure 4 before proceeding to the next step. The error check may include checking the height of the user and calculating foot size from a predetermined height-to-foot ratio.
Using the AR object that has child nodes placed at its four corners, the world coordinates are converted into screen coordinates in step 10.
Thereafter, the collected data comprising screen coordinates of the child nodes and the captured image, is uploaded to the server for further processing.

Figure 5 illustrates the data processing that includes the transformation of bounding box coordinates, foot image segmentation using YOLACT++, removal of geometric distortion using perspective transformation, and algorithm applied for checking whether the foot is parallel and inside the box. In the 13th step, the position of the projected box in relation to the x-axis of the captured image is established. By referring to the points labeled A, B, C, and D in "Figure 2", the orientation of the box is found by comparing the y-coordinate of points "C" and "D" to see if the box is tilted to the left or right. If the y-coordinate of point "C" is greater than that of point "D," the box is tilted to the left of the image and vice versa. We then calculate the tilt angle using the equation.

Tilt angle(?)=tilt factor* tan^(-1)?(| Y_c-Y_D |/| X_c-X_D | )
Equation 1

where, tilt factor = 1, if the box is tilted to the right side of the image and tilt factor = -1 if the box is tilted to the left side of the image.
In the 14th step of Figure 5, the rotation matrix for the tilt angle is determined. The image has to be rotated along an axis passing through the center and perpendicular to the plane of the image. The matrix is calculated using the equation
Rotation Matrix=[¦(a&ß&(1-a).centerX-ß.centerY@-ß&a&ß.centerX+(1-a).centerY)]
Equation 2
Where, a=cos??
ß=sin??
?=Tilt angle (Eq.1)

Once the rotation matrix is calculated it is used to rotate the entire image and perform the warp affine transformation by keeping the same size of the output as the original image size, as mentioned in step 16 of Figure 5.

In augmented reality (AR) applications, it is crucial to scan the surrounding environment to accurately estimate depth and create realistic virtual overlays. To ensure precise depth estimation, it is essential for the user to move their phone slowly and in a wide circular motion. To guide the user in this process, a sensor-based approach is employed, which monitors the motion of the device and provides error prompts if the phone is moved too quickly. The sensor-based approach utilizes various sensors available on modern smartphones, such as accelerometers, gyroscopes, and magnetometers. These sensors provide data about the device's orientation, rotation, and acceleration in three-dimensional space. By continuously monitoring the sensor data, the system can detect the speed at which the phone is being moved during the scanning process. To guide the user, the system establishes a threshold for the acceptable movement speed. If the user moves the phone too rapidly, exceeding the predefined threshold, the system triggers an error prompt. This prompt serves as a real-time feedback mechanism, notifying the user to slow down their movements to ensure accurate depth estimation. By utilizing the sensor-based approach and providing error prompts, the AR application encourages users to maintain an appropriate scanning speed, which ultimately improves the quality and precision of the depth estimation.

The purpose of the heel detection (STEP 4) module is to guide the user regarding the placement of their foot when taking a photograph. This model utilizes advanced algorithms and computer vision techniques to accurately identify the position of the heel in the image. By analyzing the image, the model can determine whether the user's foot is correctly positioned, specifically focusing on the placement of the heel. The detection process involves analyzing the visual features and patterns present in the image to identify the distinctive characteristics associated with the heel. These characteristics may include shape, texture, color, or other relevant attributes. The model applies various image processing techniques and machine learning algorithms to extract and analyze these features. Once the model has successfully detected the heel in the image, it can provide feedback or instructions to the user to adjust their foot positioning if necessary. This guidance could be in the form of visual cues, prompts, or real-time feedback, depending on the implementation of the system. By using a heel detection model, users can ensure that their foot is appropriately positioned for capturing images in specific contexts, such as medical examinations, footwear fitting applications, or other scenarios where accurate foot positioning is essential.

As the AR does not work properly on non-textured floor, the textured floor detection module is used to detect whether a floor is textured or not, and the number of contours is determined. The textured floor detection module uses a contour detection algorithm, such as OpenCV's `find Contours` function, to find the contours in the binary image. Contours represent the boundaries of objects or regions in an image. Count the number of contours detected in the image. The textured floor detection module compares the number of contours with the threshold value of 25. If the number of contours is less than 25, the floor is considered a non-textured floor. Otherwise, if the number of contours is equal to or greater than 25, the floor is considered a textured floor. The idea behind this approach is that a textured floor would have multiple contours due to the presence of patterns, lines, or other features. On the other hand, a non-textured floor would have fewer or no significant contours.

The corners of the box are connected to child nodes, and their world coordinates are converted to screen coordinates, as discussed in paragraph 17. After the image is rotated with the help of the rotation matrix, we again need to adjust the coordinates of the box corners. This is done by multiplying the box coordinates with the rotation matrix in step 17 of Figure 5. Screen coordinates are shown in figure 7.

In order to obtain precise measurements, we require a bounding box in the shape of a rectangle. This may not always be achieved due to an incorrect frame of reference. To correct this, the image distortion correction module is used to remove geometric distortion and distortion caused by camera perspective using perspective transformation and for checking whether the foot is parallel and inside the box. We have used perspective transformation in step 18 of Figure 5. Perspective Transformation is a technique used in image processing and computer vision to change the apparent perspective of an image, making it appear as if it was captured from a different viewpoint. It is typically used to correct for distortion caused by the camera's perspective when the image was captured, or to change the perspective of an image to make it more aesthetically pleasing. In a perspective transformation, the image is transformed from the original coordinates (x, y) to new coordinates (x', y') by applying a perspective transformation matrix. This matrix contains the parameters for the transformation, such as the distance of the image plane from the camera, the focal length of the camera lens, and the camera's principal point (the point in the image that corresponds to the center of projection). By adjusting these parameters, it is possible to change the perspective of the image and correct for any distortion. The most commonly used method for perspective transformation is the "four-point perspective transformation" in which the user selects four points in the image (source point), corresponding to the corners of a rectangular object, and then specifies the desired position (destination point) of these points in the transformed image. Source points are coordinates obtained after rotating the box coordinates in paragraph [21] and destination points are the coordinates corresponding to real-world dimensions (in millimeters) of the box (i.e. (210,0), (0,0), (0,350), (350,210)) This information is then used to calculate the perspective transformation matrix. Then we multiply the matrix with the image coordinates and then crop it at the destination points to get the proper view of the foot. This is done for better segmentation which will be discussed in the next paragraph.

Step 19 of Figure 5 illustrates the foot segmentation. The foot segmentation module is configured to extract the image of the foot from the rest of the image. The foot segmentation module performs foot segmentation using YOLACT++ (You Only Look At The Coefficients). The segmented foot mask is shown in figure 8. The goal of the YOLACT++ is to add an instance segmentation capability to an existing one-stage object detection model without using an explicit feature localization step, similar to how Mask R-CNN adds instance segmentation to Faster R-CNN. The invention proposes to break down the instance segmentation task into two simpler tasks. The first task is using a fully convolutional network (FCN) to produce a set of image-sized "prototype masks" that do not depend on any specific instance. The second task is adding an extra head to the object detection branch that predicts a vector of "mask coefficients" for each anchor. These coefficients encode the representation of an instance in the prototype space. Finally, for each instance that passes the box-based Non-Maxima Suppression (NMS), the foot segmentation module generates the final mask for that instance by combining the work of the two branches.

Step 20 of Figure 5 explains about finding the contour of the segmented mask of the foot. To do so, the foot segmentation module first convert the segmented mask output from the YOLACT++ model into grayscale format which is then used to obtain all possible contours in the mask. Of all the contours, we find the contour which has the maximum area which is the required contour of the user’s foot.

Step 20 of Figure 5 explains the boundary conditions for determining whether the foot is kept inside the box or not. It ensures that the y-coord of the extreme top point is at least 8 pixels (1 pixel = 1 mm in our case) below the upper edge of the box and the x-coord of the extreme right point is at least 8 pixels on the left of the right edge of the box.

The foot dimension calculation module is configured to calculate the dimension of the foot from the segmented image. To get the accurate foot measurement, we need to check whether the foot is parallel to the box length or not. The foot dimension calculation module determine the extreme points of the contour (i.e., extreme top and extreme bottom, points having minimum and maximum Y-coordinates respectively). The difference of Y-coordinates between these two points is calculated, and the difference is split into four equal parts, the first and third elements are added to the Y-coordinate of the extreme top, naming them as target Y values. Now, the points in the contour having the same Y coordinates as our target Y values are considered and the foot dimension calculation module takes the last element of each of them giving us desired points P and R as shown in Figure 9. A line passing through these two points (P and R) is assumed and the angle (?) of this line with the x-axis of the image is calculated. The foot dimension calculation module consider the images for further computation only if the angle (?) lies between 75° and 90°. The above description is mentioned in step 23 of the figure 5.

To find the length of the foot, we need the coordinates of the toe and heel. The coordinate of the toe is represented by the extreme top coordinates as per paragraph [26]. Step 26 of Figure 5 explains the algorithm used to find the coordinates of the heel point. According to our hypothesis, the gradient will be zero near the heel point. In order to get the heel, point we iterate through all the points near the heel region of the contour and calculate the gradients using the following equation:

(dy/dx)_i=(y_(i+1)-y_(i-1))/(x_(i+1)-x_(i-1) )
Equation 3

The first point where the gradient is zero is our heel point which is termed as extreme bottom with gradient, in the figure.

In step 28 of Figure 6, the foot dimension calculation module calculates the length of the foot using the Euclidean distance equation:

Length of the foot= v((?Toe?_x- ?Heel?_x )^2+ (?Toe?_y- ?Heel?_y )^2 )

The length which we obtain using the above equation is in millimeters.

Apart from foot length, our invention also provides the width of the user’s foot. In step 29 of Figure 7, we have depicted an algorithm to find the user’s foot width. Firstly, we find 1/3th of the absolute difference (Y-diff) between the y-coordinate of the toe point and the heel point. Then we find two points with maximum absolute difference of x-coordinates in the contour from toe to one-third of the Y-diff. We consider these two points for width calculation.

In step 30 of Figure 6, the foot dimension calculation module calculates the width of the foot using the absolute difference of the X-coordinates of point S and point T.

In step 31 of Figure 6 we apply a check for accurate projection of the 3D object (box) and accurate segmentation, for that we consider in any case the length of the foot is more than the upper limit (i.e. equal to (height of user)/6.6+1.75 ) and less than the lower limit (i.e. equal to (height of user)/6.6-1.75) we consider it as false measurement.

In an embodiment of the present invention, the display module is configured to display the measured length and width of the foot on the handheld computing device. Wherein the length and width are calculated by the foot dimension calculation module.

In a preferred embodiment, a method of operating the system for augmented reality-based foot size measurement, comprising:
capturing an image of a user’s foot using a handheld computing device equipped with a camera;
projecting an augmented reality (AR) object onto the display unit of the handheld computing device for foot positioning;
guiding in positioning the user’s foot within a projected box;
employing a heel detection module to ensure proper foot placement;
providing audio instructions and visual cues to assist the user in aligning the foot parallel to the length of the projected box;
activating flash for clear visibility of the foot’s heel and toes during image capture;
performing error checks, including height verification before proceeding to further processing;
converting world coordinates of AR object corners to screen coordinated for data processing; and
uploading captured image data, including screen coordinates and foot images, to a cloud server for computation.
transformation of bounding box coordinates;
determining the angle of the bounding box concerning horizontal axis;
finding rotation matrix for tilt angle;
rotation of image using rotation matrix and performing warp affine;
adjusting the bounding box coordinates by multiplication of rotation matrix and bounding box coordinates;
performing perspective transformation;
segmentation of image using YOLACT++;
finding contour of the maximum area;
checking of angle whether between 75-90 degrees;
calculation of heal point and toe point;
calculation of the length of the foot through the heal point and the toe point;
finding two points with maximum absolute difference of x-coordinate in the contour;
calculation of width from the X-coordinate of first point and last point of the contour;

The process of capturing an image on a mobile device is a sophisticated endeavor that hinges on the intricate interplay of integrated camera hardware components. These components encompass an amalgamation of high-quality lenses, advanced sensors, and image processing units. Furthermore, the handheld computing device capitalizes on its robust computational capacity, leveraging onboard processors such as Graphics Processing Units (GPUs) and Central Processing Units (CPUs). These computational powerhouses execute a specialized algorithm designed for depth perception, enabling us to discern the depth of field in the surrounding environment accurately. This in-depth information is invaluable, as it underpins more precise measurements and comprehensive environmental analyses. Once the image has been successfully captured and depth information extracted, the data is seamlessly transmitted over the internet to our cloud servers. These servers are equipped with GPU-accelerated processing capabilities, which are vital for executing intensive deep-learning models. It's within this cloud-based infrastructure that the final computation of foot dimensions takes place.

While this invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.
,CLAIMS:WE CLAIM:

1. A system for augmented reality-based foot size measurement, comprising:
i. at least one handheld computing device equipped with a camera;
ii. a software module installed in the handheld computing device, comprising:
a. a data storage module for storing a plurality of instructions and user data;
b. a processing module comprising an augmented reality (AR) based application;
c. a mapping module;
d. a textured floor detection module;
e. a key point detection module;
f. an image distortion correction module;
g. a heel detection module;
h. a foot segmentation module;
i. a foot dimension calculation module; and
j. a display module having a graphical user interface;
wherein,
a) the software module through the camera captures images by moving the handheld device and horizontally scanning floor to create a mesh of the user’s surroundings, so as to create a AR based projected box by tapping on screen of the handheld device;
b) the image distortion correction module is configured to remove geometric distortion and distortion caused by the camera perspective for checking whether foot is parallel and inside the projected box;
c) the foot segmentation module is configured to extract foot from the rest of the image;
d) the foot dimension calculation module is configured to calculate dimension of foot from the segmented image;
e) the display module is configured to display the measured length and width of the foot;
f) the key point detector module is configured to distinguish a barefoot image of a user from user images wearing socks or shoes; and
g) the heel detection module instructs the user to place the foot inside the projected box, so as to ensure the foot of the user is placed parallel to the longer edge of the box.

2. The system as claimed in claim 1, wherein the camera is a monocular camera configured to magnify images of distant foot.

3. The system as claimed in claim 1, wherein the camera comprises at least one sensor selected from a group of charged-coupled device (CCD) sensors and a complementary metal oxide semiconductor (CMOS) sensor.

4. The system as claimed in claim 1, wherein the textured floor detection module is configured to detect whether the floor is textured or not.

5. The system as claimed in claim 1, the software module comprises a sensor-based guidance module for monitoring device motion and real-time feedback to enable depth estimation during image capture.

6. The system as claimed in claim 1, wherein the processing module is installed with onboard processors, including graphic processing units (GPUs) and central processing units (CPUs) for executing specialized algorithms designed for depth perception and environmental analysis.

7. A method for measuring foot size through augmented reality-based, comprising:
i. capturing an image of a user’s foot using a handheld computing device equipped with an image-capturing device;
ii. projecting an augmented reality (AR) object onto the display unit of the handheld computing device for foot positioning;
iii. guiding in positioning the user’s foot within a projected box;
iv. employing a heel detection module to ensure proper foot placement;
v. providing audio instructions and visual cues to assist the user in aligning foot parallel to length of the projected box;
vi. activating flash for clear visibility of foot’s heel and toes during image capture;
vii. performing error checks, including height verification before proceeding to further processing;
viii. converting world coordinates of AR object corners to screen coordinated for data processing; and
ix. uploading captured image data, including screen coordinates and foot images, to a cloud server for computation.
x. transformation of bounding box coordinates;
xi. determining the angle of the bounding box with respect to the horizontal axis;

xii. finding rotation matrix for tilt angle;
xiii. rotation of image using rotation matrix and performing warp affine;
xiv. adjusting the bounding box coordinates by multiplication of rotation matrix and bounding box coordinates;
xv. performing perspective transformation;
xvi. segmentation of images using YOLACT++;
xvii. finding contour of maximum area;
xviii. checking of angle whether between 75-90 degrees;
xix. calculation of heal point and toe point;
xx. calculation of length of foot through the heal point and the toe point;
xxi. finding two points with maximum absolute difference of x-coordinate in the contour;
xxii. calculation of width from the X-coordinate of first point and last point of the contour;

8. The method as claimed in claim 6, wherein the handheld computing device comprises a plurality of sensors to monitor device motion and real-time feedback to enable depth estimation during image capture.

9. The method as claimed in claim 6, wherein processing of captured image data on the handheld computing device includes transforming bounding box coordinates and removing geometric distortion using perspective transformation.

10. The method as claimed in claim 6, wherein the processing module determines foot orientation by identifying extreme points of the foot contour and accepting images for computation based on predefined angle criteria, ensuring foot alignment parallel to the projected box length.