Description:SYSTEM AND METHOD FOR NAVIGATING MOBILE ROBOT IN AISLE

FIELD OF THE PRESENT DISCLOSURE
[0001] The present disclosure generally relates to autonomous guided vehicles, such as a mobile robot, implemented to move in a work area; and particularly to a system and method to support navigation of the mobile robot in an aisle defined by racks arranged on both sides, in the work area.

BACKGROUND
[0002] Autonomous guided vehicles (AGVs), also known as mobile robots, are increasingly being employed for transporting goods and materials from one place to another in constrained environments, such as a factory or a warehouse. Mobile robots offer the ability to carry payloads too heavy for a person to carry and without the supervision of a person, while also offering the flexibility to be reconfigured to follow a different route or carry different types of payloads. For example, mobile robots are used in warehouse environments to assist with inventory management by transporting goods from one area of the warehouse to another. In the warehouse, the mobile robot may travel from a loading area to a dropping area based on a control system and without intervention from users. Similarly, in a manufacturing plant, the mobile robots may be used to transport items, such as heavy vehicle components like engines, chassis, etc., along a route on a floor of the manufacturing plant to deliver the payload from one location to another, or to allow various manufacturing operations to be performed thereon.
[0003] However, in general, it has been a challenge to navigate the mobile robot in confined areas. For example, in warehouses, the mobile robot is required to travel in confined aisles which are surrounded by racks on both sides, for picking and dropping of items at shelves of the racks and the like. Herein, the mobile robot is required to keep its pose (orientation) correct at generally all times, to be able to navigate within such confined aisles, to avoid hitting one or the other rack in the aisle, which is a difficult problem to solve. Traditionally, warehouses require customized rack and shelving structures to accommodate and allow for navigation of the mobile robot therein, which is cost prohibitive to many distribution centres and require a significant investment in capital, and also time which is typically on the order of months or over a year for full installation and system integration to be completed.
[0004] Therefore, in light of the foregoing discussion, there exists a need to overcome problems associated with conventional techniques and provide systems and/or methods for navigating a mobile robot in confined spaces, particularly an aisle.

SUMMARY
[0005] In an aspect, a method for navigating a mobile robot in an aisle using an external feature, associated with the aisle, as a reference therefor is provided. The method comprises providing the external feature with a visual marker, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known. The method further comprises implementing a visual recognizer, with a known fixed focal length, mounted on the mobile robot to capture an image of the visual marker. The method further comprises determining an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer. The method further comprises determining an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker. The method further comprises determining an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker. The method further comprises localizing the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle.
[0006] In one or more embodiments, the method further comprises determining a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker; and localizing the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.
[0007] In one or more embodiments, the angular orientation of the visual marker with respect to the visual recognizer is determined using one or more of: homography technique and orthogonal iteration technique.
[0008] In one or more embodiments, the cross-track error for the mobile robot is determined using triangle similarity technique.
[0009] In another aspect, a system for navigating a mobile robot in an aisle using an external feature, associated with the aisle, as a reference therefor is provided. The system comprises a visual marker provided with the external feature, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known. The system also comprises a visual recognizer, with a known fixed focal length, mounted on the mobile robot. The system also comprises a controller. The controller is configured to implement the visual recognizer to capture an image of the visual marker. The controller is further configured to determine an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer. The controller is further configured to determine an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker. The controller is further configured to determine an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker. The controller is further configured to localize the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle.
[0010] In one or more embodiments, the controller is further configured to determine a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker; and localize the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.
[0011] In one or more embodiments, the controller is configured to use one or more of: homography technique and orthogonal iteration technique for determining angular orientation of the visual marker with respect to the visual recognizer.
[0012] In one or more embodiments, the controller is configured to use a triangle similarity technique for determining the cross-track error for the mobile robot.
[0013] In yet another aspect, a mobile robot adapted to navigate in an aisle using an external feature, associated with the aisle, as a reference therefor is provided. Herein, the external feature is provided with a visual marker, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known. The mobile robot comprises a visual recognizer, with a known fixed focal length, mounted thereon. The mobile robot also comprises a controller. The controller is configured to implement the visual recognizer to capture an image of the visual marker. The controller is further configured to determine an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer. The controller is further configured to determine an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker. The controller is further configured to determine an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker. The controller is further configured to localize the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle.
[0014] In one or more embodiments, the controller is further configured to determine a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker; and localize the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.
[0015] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE FIGURES
[0016] For a more complete understanding of example embodiments of the present disclosure, reference is now made to the following descriptions taken in connection with the accompanying drawings in which:
[0017] FIG. 1 illustrates a schematic of a system that may reside on and may be executed by a computer, which may be connected to a network, in accordance with one or more embodiments of the present disclosure;
[0018] FIG. 2 illustrates a schematic of an exemplary computing system for managing a mobile robot operating in a work area, in accordance with one or more embodiments of the present disclosure;
[0019] FIG. 3A illustrates an exemplary diagrammatic side view of the mobile robot, in accordance with one or more embodiments of the present disclosure;
[0020] FIG. 3B illustrates an exemplary diagrammatic top view of the mobile robot, in accordance with one or more embodiments of the present disclosure;
[0021] FIG. 4 illustrates an exemplary depiction of a work area for operation of the mobile robot therein, in accordance with one or more embodiments of the present disclosure;
[0022] FIG. 5 illustrates an exemplary depiction of a rack for navigation of the mobile robot in a corresponding aisle, in accordance with one or more embodiments of the present disclosure;
[0023] FIG. 6 illustrates an exemplary implementation of the system for navigating the mobile robot in an aisle with respect to a rack therein, in accordance with one or more embodiments of the present disclosure; and
[0024] FIG. 7 illustrates a flowchart listing steps involved in a method for navigating a mobile robot in an aisle with respect to a rack therein, in accordance with one or more embodiments of the present disclosure.

DETAILED DESCRIPTION
[0025] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure is not limited to these specific details.
[0026] Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearance of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced items. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not for other embodiments.
[0027] Furthermore, in the following detailed description of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be understood that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present disclosure.
[0028] Embodiments described herein may be discussed in the general context of computer-executable instructions residing on some form of computer-readable storage medium, such as program modules, executed by one or more computers or other devices. By way of example, and not limitation, computer-readable storage media may comprise non-transitory computer-readable storage media and communication media; non-transitory computer-readable media include all computer-readable media except for a transitory, propagating signal. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or distributed as desired in various embodiments.
[0029] Some portions of the detailed description that follows are presented and discussed in terms of a process or method. Although steps and sequencing thereof are disclosed in figures herein describing the operations of this method, such steps and sequencing are exemplary. Embodiments are well suited to performing various other steps or variations of the steps recited in the flowchart of the figure herein, and in a sequence other than that depicted and described herein. Some portions of the detailed descriptions that follow are presented in terms of procedures, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. In the present disclosure, a procedure, logic block, process, or the like, is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those utilizing physical manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as transactions, bits, values, elements, symbols, characters, samples, pixels, or the like.
[0030] In some implementations, any suitable computer usable or computer readable medium (or media) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-usable, or computer-readable, storage medium (including a storage device associated with a computing device) may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fibre, a portable compact disc read-only memory (CD-ROM), an optical storage device, a digital versatile disk (DVD), a static random access memory (SRAM), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, a media such as those supporting the internet or an intranet, or a magnetic storage device. Note that the computer-usable or computer-readable medium could even be a suitable medium upon which the program is stored, scanned, compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In the context of the present disclosure, a computer-usable or computer-readable, storage medium may be any tangible medium that can contain or store a program for use by or in connection with the instruction execution system, apparatus, or device.
[0031] In some implementations, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. In some implementations, such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. In some implementations, the computer readable program code may be transmitted using any appropriate medium, including but not limited to the internet, wireline, optical fibre cable, RF, etc. In some implementations, a computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
[0032] In some implementations, computer program code for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Java®, Smalltalk, C++ or the like. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. However, the computer program code for carrying out operations of the present disclosure may also be written in conventional procedural programming languages, such as the "C" programming language, PASCAL, or similar programming languages, as well as in scripting languages such as JavaScript, PERL, or Python. In present implementations, the used language for training may be one of Python, TensorflowTM, Bazel, C, C++. Further, decoder in user device (as will be discussed) may use C, C++ or any processor specific ISA. Furthermore, assembly code inside C/C++ may be utilized for specific operation. Also, ASR (automatic speech recognition) and G2P decoder along with entire user system can be run in embedded Linux (any distribution), Android, iOS, Windows, or the like, without any limitations. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the internet using an Internet Service Provider). In some implementations, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs) or other hardware accelerators, micro-controller units (MCUs), or programmable logic arrays (PLAs) may execute the computer readable program instructions/code by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
[0033] In some implementations, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus (systems), methods and computer program products according to various implementations of the present disclosure. Each block in the flowchart and/or block diagrams, and combinations of blocks in the flowchart and/or block diagrams, may represent a module, segment, or portion of code, which comprises one or more executable computer program instructions for implementing the specified logical function(s)/act(s). These computer program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program instructions, which may execute via the processor of the computer or other programmable data processing apparatus, create the ability to implement one or more of the functions/acts specified in the flowchart and/or block diagram block or blocks or combinations thereof. It should be noted that, in some implementations, the functions noted in the block(s) may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
[0034] In some implementations, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks or combinations thereof.
[0035] In some implementations, the computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed (not necessarily in a particular order) on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts (not necessarily in a particular order) specified in the flowchart and/or block diagram block or blocks or combinations thereof.
[0036] Referring now to the example implementation of FIG. 1, a schematic of a system 100 is illustrated that may reside on and may be executed by a computer (e.g., computer 12), which may be connected to a network (e.g., network 14) (e.g., the internet or a local area network). Examples of computer 12 may include, but are not limited to, a personal computer(s), a laptop computer(s), mobile computing device(s), a server computer, a series of server computers, a mainframe computer(s), or a computing cloud(s). In some implementations, each of the aforementioned may be generally described as a computing device. In certain implementations, a computing device may be a physical or virtual device. In many implementations, a computing device may be any device capable of performing operations, such as a dedicated processor, a portion of a processor, a virtual processor, a portion of a virtual processor, a portion of a virtual device, or a virtual device. In some implementations, a processor may be a physical processor or a virtual processor. In some implementations, a virtual processor may correspond to one or more parts of one or more physical processors. In some implementations, the instructions/logic may be distributed and executed across one or more processors, virtual or physical, to execute the instructions/logic. Computer 12 may execute an operating system, for example, but not limited to, Microsoft® Windows®; Mac® OS X®; Red Hat® Linux®, or a custom operating system. (Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States, other countries or both; Mac and OS X are registered trademarks of Apple Inc. in the United States, other countries or both; Red Hat is a registered trademark of Red Hat Corporation in the United States, other countries or both; and Linux is a registered trademark of Linus Torvalds in the United States, other countries or both).
[0037] In some implementations, the instruction sets and subroutines of system 100, which may be stored on storage device, such as storage device 16, coupled to computer 12, may be executed by one or more processors (not shown) and one or more memory architectures included within computer 12. In some implementations, storage device 16 may include but is not limited to: a hard disk drive; a flash drive, a tape drive; an optical drive; a RAID array (or other array); a random-access memory (RAM); and a read-only memory (ROM). In some implementations, network 14 may be connected to one or more secondary networks (e.g., network 18), examples of which may include but are not limited to: a local area network; a wide area network; or an intranet, for example.
[0038] In some implementations, computer 12 may include a data store, such as a database (e.g., relational database, object-oriented database, triplestore database, etc.) and may be located within any suitable memory location, such as storage device 16 coupled to computer 12. In some implementations, data, metadata, information, etc. described throughout the present disclosure may be stored in the data store. In some implementations, computer 12 may utilize any known database management system such as, but not limited to, DB2, in order to provide multi-user access to one or more databases, such as the above noted relational database. In some implementations, the data store may also be a custom database, such as, for example, a flat file database or an XML database. In some implementations, any other form(s) of a data storage structure and/or organization may also be used. In some implementations, system 100 may be a component of the data store, a standalone application that interfaces with the above noted data store and/or an applet / application that is accessed via client applications 22, 24, 26, 28. In some implementations, the above noted data store may be, in whole or in part, distributed in a cloud computing topology. In this way, computer 12 and storage device 16 may refer to multiple devices, which may also be distributed throughout the network.
[0039] In some implementations, computer 12 may execute application 20 for navigating a mobile robot in an aisle (as discussed later in more detail). In some implementations, system 100 and/or application 20 may be accessed via one or more of client applications 22, 24, 26, 28. In some implementations, system 100 may be a standalone application, or may be an applet / application / script / extension that may interact with and/or be executed within application 20, a component of application 20, and/or one or more of client applications 22, 24, 26, 28. In some implementations, application 20 may be a standalone application, or may be an applet / application / script / extension that may interact with and/or be executed within system 100, a component of system 100, and/or one or more of client applications 22, 24, 26, 28. In some implementations, one or more of client applications 22, 24, 26, 28 may be a standalone application, or may be an applet / application / script / extension that may interact with and/or be executed within and/or be a component of system 100 and/or application 20. Examples of client applications 22, 24, 26, 28 may include, but are not limited to, a standard and/or mobile web browser, an email application (e.g., an email client application), a textual and/or a graphical user interface, a customized web browser, a plugin, an Application Programming Interface (API), or a custom application. The instruction sets and subroutines of client applications 22, 24, 26, 28, which may be stored on storage devices 30, 32, 34, 36, coupled to user devices 38, 40, 42, 44, may be executed by one or more processors and one or more memory architectures incorporated into user devices 38, 40, 42, 44.
[0040] In some implementations, one or more of storage devices 30, 32, 34, 36, may include but are not limited to: hard disk drives; flash drives, tape drives; optical drives; RAID arrays; random access memories (RAM); and read-only memories (ROM). Examples of user devices 38, 40, 42, 44 (and/or computer 12) may include, but are not limited to, a personal computer (e.g., user device 38), a laptop computer (e.g., user device 40), a smart/data-enabled, cellular phone (e.g., user device 42), a notebook computer (e.g., user device 44), a tablet (not shown), a server (not shown), a television (not shown), a smart television (not shown), a media (e.g., video, photo, etc.) capturing device (not shown), and a dedicated network device (not shown). User devices 38, 40, 42, 44 may each execute an operating system, examples of which may include but are not limited to, Android®, Apple® iOS®, Mac® OS X®; Red Hat® Linux®, or a custom operating system.
[0041] In some implementations, one or more of client applications 22, 24, 26, 28 may be configured to effectuate some or all of the functionality of system 100 (and vice versa). Accordingly, in some implementations, system 100 may be a purely server-side application, a purely client-side application, or a hybrid server-side / client-side application that is cooperatively executed by one or more of client applications 22, 24, 26, 28 and/or system 100. In some implementations, one or more of client applications 22, 24, 26, 28 may be configured to effectuate some or all of the functionality of application 20 (and vice versa). Accordingly, in some implementations, application 20 may be a purely server-side application, a purely client-side application, or a hybrid server-side / client-side application that is cooperatively executed by one or more of client applications 22, 24, 26, 28 and/or application 20. As one or more of client applications 22, 24, 26, 28, system 100, and application 20, taken singly or in any combination, may effectuate some or all of the same functionality, any description of effectuating such functionality via one or more of client applications 22, 24, 26, 28, system 100, application 20, or combination thereof, and any described interaction(s) between one or more of client applications 22, 24, 26, 28, system 100, application 20, or combination thereof to effectuate such functionality, should be taken as an example only and not to limit the scope of the disclosure.
[0042] In some implementations, one or more of users 46, 48, 50, 52 may access computer 12 and system 100 (e.g., using one or more of user devices 38, 40, 42, 44) directly through network 14 or through secondary network 18. Further, computer 12 may be connected to network 14 through secondary network 18, as illustrated with phantom link line 54. System 100 may include one or more user interfaces, such as browsers and textual or graphical user interfaces, through which users 46, 48, 50, 52 may access system 100. In some implementations, the various user devices may be directly or indirectly coupled to network 14 (or network 18). For example, user device 38 is shown directly coupled to network 14 via a hardwired network connection. Further, user device 44 is shown directly coupled to network 18 via a hardwired network connection. User device 40 is shown wirelessly coupled to network 14 via wireless communication channel 56 established between user device 40 and wireless access point (i.e., WAP) 58, which is shown directly coupled to network 14. WAP 58 may be, for example, an IEEE 802.11a, 802.11b, 802.11g, Wi-Fi®, RFID, and/or BluetoothTM (including BluetoothTM Low Energy) device that is capable of establishing wireless communication channel 56 between user device 40 and WAP 58. User device 42 is shown wirelessly coupled to network 14 via wireless communication channel 60 established between user device 42 and cellular network / bridge 62, which is shown directly coupled to network 14.
[0043] In some implementations, some or all of the IEEE 802.11x specifications may use Ethernet protocol and carrier sense multiple access with collision avoidance (i.e., CSMA/CA) for path sharing. The various 802.11x specifications may use phase-shift keying (i.e., PSK) modulation or complementary code keying (i.e., CCK) modulation, for example, BluetoothTM (including BluetoothTM Low Energy) is a telecommunications industry specification that allows, e.g., mobile phones, computers, smart phones, and other electronic devices to be interconnected using a short-range wireless connection. Other forms of interconnection (e.g., Near Field Communication (NFC)) may also be used.
[0044] FIG. 2 is a block diagram of an example of a computing system 200 capable of implementing embodiments according to the present disclosure. The computing system 200 is implemented for issuing commands for managing and controlling operations of a fleet of mobile robots (as will be described later in more detail), which, in turn, may be utilized in a warehouse environment, a manufacturing plant and the like. In one embodiment, the application 20 for managing navigation of a mobile robot in an aisle (as mentioned above) may be executed as a part of the computing system 200 as described herein. Thereby, for example in case of a warehouse, the system 100 may be a broader system such as the warehouse management system (WMS) as known in the art, in which the computing system 200 may be executed for managing and controlling operations of the mobile robots. Hereinafter, the terms “system 100” and “computing system 200” have been broadly interchangeably used to represent means for managing and controlling operations of a fleet of mobile robots in the warehouse environment, without any limitations.
[0045] In the example of FIG. 2, the computing system 200 includes a processing unit 205 for running software applications (such as, the application 20 of FIG. 1) and optionally an operating system. Memory 210 stores applications and data for use by the processing unit 205. Storage 215 provides non-volatile storage for applications and data and may include fixed disk drives, removable disk drives, flash memory devices, and CD-ROM, DVD-ROM or other optical storage devices. An optional user input device 220 includes devices that communicate user inputs from one or more users to the computing system 200 and may include keyboards, mice, joysticks, touch screens, etc. A communication or network interface 225 is provided which allows the computing system 200 to communicate with other computer systems via an electronic communications network, including wired and/or wireless communication and including an Intranet or the Internet. In one embodiment, the computing system 200 receives instructions and user inputs from a remote computer through communication interface 225. Communication interface 225 can comprise a transmitter and receiver for communicating with remote devices. An optional display device 250 may be provided which can be any device capable of displaying visual information in response to a signal from the computing system 200. The components of the computing system 200, including the processing unit 205, the memory 210, the data storage 215, the user input devices 220, the communication interface 225, and the display device 250, may be coupled via one or more data buses 260.
[0046] In the embodiment of FIG. 2, a graphics system 230 may be coupled with the data bus 260 and the components of the computing system 200. The graphics system 230 may include a physical graphics processing unit (GPU) 235 and graphics memory. The GPU 235 generates pixel data for output images from rendering commands. The physical GPU 235 can be configured as multiple virtual GPUs that may be used in parallel (concurrently) by a number of applications or processes executing in parallel. For example, mass scaling processes for rigid bodies or a variety of constraint solving processes may be run in parallel on the multiple virtual GPUs. Graphics memory may include a display memory 240 (e.g., a framebuffer) used for storing pixel data for each pixel of an output image. In another embodiment, the display memory 240 and/or additional memory 245 may be part of the memory 210 and may be shared with the processing unit 205. Alternatively, the display memory 240 and/or additional memory 245 can be one or more separate memories provided for the exclusive use of the graphics system 230. In another embodiment, graphics system 230 includes one or more additional physical GPUs 255, similar to the GPU 235. Each additional GPU 255 may be adapted to operate in parallel with the GPU 235. Each additional GPU 255 generates pixel data for output images from rendering commands. Each additional physical GPU 255 can be configured as multiple virtual GPUs that may be used in parallel (concurrently) by a number of applications or processes executing in parallel, e.g., processes that solve constraints. Each additional GPU 255 can operate in conjunction with the GPU 235, for example, to simultaneously generate pixel data for different portions of an output image, or to simultaneously generate pixel data for different output images. Each additional GPU 255 can be located on the same circuit board as the GPU 235, sharing a connection with the GPU 235 to the data bus 260, or each additional GPU 255 can be located on another circuit board separately coupled with the data bus 260. Each additional GPU 255 can also be integrated into the same module or chip package as the GPU 235. Each additional GPU 255 can have additional memory, similar to the display memory 240 and additional memory 245, or can share the memories 240 and 245 with the GPU 235. It is to be understood that the circuits and/or functionality of GPU as described herein could also be implemented in other types of processors, such as general-purpose or other special-purpose coprocessors, or within a CPU.
[0047] Referring to FIGS. 3A and 3B, illustrated are exemplary diagrammatic views of a mobile robot 300, in accordance with one or more embodiments of the present disclosure. Herein, as illustrated in combination of FIGS. 3A and 3B, the mobile robot 300 includes a platform 302, which may be provided with drive wheels 304 underneath to allow for the mobile robot 300 to travel in the work area. The mobile robot 300 also includes a vertical tower 306 mounted on the platform 302. The vertical tower 306 may be arranged in an up-right, generally orthogonal, configuration with respect to the platform 302. The vertical tower 306 may be provided with a plurality of shelves 308 which may be spaced apart from each other. In an example, a vertical gap between two such shelves 308 may be defined based on a height of a tote or the like to be placed at such shelves 308. Further, the mobile robot 300 includes a stacker 310 which may be configured to move up and down along a length of the vertical tower 306 (as represented by axis ‘A’). It may be understood that the mobile robot 300 may include a drive train mechanism (not shown) to enable the stacker 310 to move up and down along the length of the vertical tower 306. For this purpose, in some examples, the vertical tower 306 may include rails (not shown) running along the length thereof, to enable the stacker 310 to move therein. Further, in the present embodiments, the stacker 310 may be rotatable about a point of contact with the vertical tower 306 so as to be aligned therewith either along a short edge or a long edge thereof. This allows the stacker 310 to orient itself in order to be implemented for picking items from one of the shelves 308, and further re-orient itself in order to be implemented for dropping the picked item to a rack or the like (as described in some detail later in the description); and vice-versa. In some examples, the stacker 310 may use extending arms for picking/dropping the items thereby. Such construction of the mobile robot 300 may be understood by a person skilled in the art and may further be appreciated from a video available at www.youtube.com/watch?v=SRDyQaVUqPE.
[0048] Referring to FIG. 4, illustrated is an exemplary depiction of a work area 400 in which a mobile robot (such as, the mobile robot 300 as described above) is operated, in accordance with one or more embodiments of the present disclosure. In the illustration of FIG. 4, although only two mobile robots 300 have been shown, it may be appreciated that there may be multiple (more than two) mobile robots, as part of the fleet of mobile robots, operating in the work area 400 at any given time, without any limitations. It may be appreciated that the work area 400 may be part of a larger floor space, e.g., in a warehouse environment (not shown) or the like. Herein, the work area 400 has been broadly divided into two regions, namely an open region 402 and a plurality of aisles 404 as defined by arrangement of racks 406 therein. As may be seen, each aisle 404 is surrounded by the racks 406 on both sides, i.e., one rack 406 on each side thereof. Further, as may be seen, the open region 402 may be linked/connected to each of the aisles 404, for providing an entry and exit to the aisles 404 for the mobile robot(s) 300. In the work area 400, the mobile robot 300 may be utilized for various operations, like transferring of goods, such as cartons, in the work area 400, which is typical, for example, for the warehouse environment. In an example, the mobile robot 300 may be implemented for stacking items (like parcels, totes, etc.) into the racks 406, by navigating in and through the aisles 404.
[0049] It may be contemplated by a person skilled in the art that a typical warehouse includes a plurality of racks (such as, the racks 406, also sometimes referred to as picking bays) that are arranged in spaced apart rows which define therebetween the aisle 404 (also sometimes referred to as a picking aisle), and each such aisle 404 generally providing access to two opposing racks 406. Though reference hereinafter is made to two rows of racks and one aisle, it can be appreciated that a plurality of rows and aisles are contemplated in the work area 400. For the purposes of the present disclosure, the aisle 404 preferably provides sufficient open space for the mobile robot 300 to move between the racks 406 so that the mobile robot 300 may be able to carry out the said operations, including stacking items (like parcels, totes, etc.) into the racks 406 by navigating in and through the aisles 404.
[0050] In the present embodiments, as shown in FIG. 4, the work area 400 includes a matrix of ground markers 420. In particular, the open region 402 in the work area 400 is provided with ground markers 420 therein. In some examples, the aisles 404 may also be provided with ground markers 420 arranged on the floor therein. Herein, the term "ground marker" is meant to include any number and all types of markers that may serve the distinguishing function, either in isolation or combination. Such ground markers may include, but are not limited to, geometric shapes or characters that superficially and/or structurally alter the appearance of the work area 400, that may be easily recognized by compatible sensing means (as discussed later in the description) provided in the mobile robot 300. In the present illustration, the ground markers 420 are shown as regular sized squares; however, other shapes including, but not limited to, circular, hexagonal, etc. may be contemplated without any limitations. Further, in some examples, each of the ground markers 420 may be unique. This may be achieved by providing the ground markers 420 with unique identification codes, like QR codes, barcodes, etc.
[0051] In one or more embodiments, the system 100 may include an odometry control arrangement (not shown). The odometry control arrangement is provided in the mobile robot 300. Herein, “odometry” refers to the use of data from motion sensors to estimate change in position over time. The odometry control arrangement is configured to control movement of the mobile robot 300 in the work area 400 based on the ground markers 420 therein. In particular, the odometry control arrangement may be implemented to control movements of the mobile robot 300 in the work area 400, such that the mobile robot 300 may be able to follow a predefined path (as discussed later). In the present embodiments, the odometry control arrangement is configured to control movement of the mobile robot 300 in the work area 400 based on the ground markers 420 positioned (laid out) therein. Specifically, the odometry control arrangement in the mobile robot 300 may enable the mobile robot 300 to move (change its position) in the work area 400 from a current ground marker 420 to a next ground marker 420, and thereby follow the predefined path as provided by the system 100.
[0052] Herein, the system 100 may define a path (i.e., the said predefined path) to be followed by the mobile robot 300 in the work area 400. The predefined path may be defined by virtually linking multiple ground markers 420 (as a virtual track), in various possible combinations, for the mobile robot 300 to travel thereon. Typically, the predefined path as provided by the system 100 is a navigation path including a set of straight lines passing through centres of the ground markers 420, in the matrix of ground markers 420 in the work area 400. Such arrangement using the ground markers may be contemplated by a person skilled in the art and thus has not been described further for the brevity of the present disclosure.
[0053] In an example, the mobile robot 300 may be implemented for stacking items (like parcels, totes, etc.) into the racks 406, by navigating in and through the aisles 404. According to embodiments of the present disclosure, as generally represented in FIG. 4, the racks 406 are provided with visual markers 430 to enable navigation (or specifically, better navigation) of the mobile robot 300 in the corresponding aisle 404. FIG. 5 illustrates an exemplary depiction of the rack 406 for allowing navigation of the mobile robot 300 in the corresponding aisle (such as, the aisle 404), in accordance with one or more embodiments of the present disclosure. As shown, the rack 406 may be in the form of a conventional case flow, which includes a frame 502 and a plurality of vertically spaced shelves 504 that are supported by the frame 502. Further, as may be seen, the shelves 504 are virtually divided into several sections, with each such section acting as a storage area 506. Further, each of the storage areas 506 is partially or fully filled with a particular item, for example in loose form; however, the storage area 506 may also be filled with containers, without any limitations.
[0054] Further, as illustrated, each of the storage areas 506, in the rack 406, is associated with one visual marker 430. In the present embodiments, as shown in FIGS. 4 and 5, the visual markers 430 may include any number and all types of markers that may serve the distinguishing function, either in isolation or combination. Such visual markers 430 may include, but are not limited to, geometric shapes or characters that superficially and/or structurally be pasted on the racks 406, that may be easily recognized by compatible sensing means (as discussed later in the description) provided in the mobile robot 300. In the present illustration, the visual markers 430 are shown as regular sized squares; however, other shapes including, but not limited to, circular, hexagonal, etc. may be contemplated without any limitations. Further, in some examples, each of the visual markers 430 may be unique. This may be achieved by providing the visual markers 430 with unique identification codes, like QR codes, barcodes, etc. For the purposes of the present disclosure, an absolute dimension of each of the visual markers 430 is known. Further, herein, an absolute position of each of the visual markers 430 with respect to the corresponding rack 406 and thereby the corresponding aisle 404, or in general the work area 400 is also known.
[0055] Referring back to FIGS. 3A-3B, in the present embodiments, the mobile robot 300 may include a ground marker recognizer 320. In the present embodiments, the ground marker recognizer 320 may be provided in the mobile robot 300. In the present embodiments, the ground marker recognizer 320 is configured to capture an image of a portion of the work area 400 underneath the mobile robot 300 when the mobile robot 300 is operating in the work area 400. The ground marker recognizer 320 may be configured to recognize presence of the ground marker 420 underneath the corresponding mobile robot 300 based on the captured image. In an example embodiment, with each of the ground marker 420 being unique, by detecting the unique ground marker 420 (i.e., detecting the identification code like QR code, bar code, etc.), and with prior information about the absolute position of each of the unique ground marker 420 in the work area 400, the present system 100 may be able to estimate a position of (localize) the corresponding mobile robot 300 in the work area 400. In the present example, the ground marker recognizer 320 may be in the form of, but not limited to, a camera (or generally any optical arrangement) provided in the platform 302 of the mobile robot 300 and pointed to a floor of the work area 400, and/or a scanner configured to distinguish colours when the ground markers, including the ground markers 420, may be of a substantially different from the floor of the work area 400, or the like. Such ground marker recognizer 320 may be contemplated by a person skilled in the art and thus has not been described in any more detail herein for the brevity of the present disclosure.
[0056] According to embodiments of the present disclosure, as shown in FIGS. 3A-3B, the mobile robot 300 also includes a visual recognizer 330. In the present embodiments, the visual recognizer 330 is mounted on the mobile robot 300 to capture an image of the visual marker 430. In particular, the visual recognizer 330 is mounted on the stacker 310 as provided in the mobile robot 300. As discussed, since the stacker 310 is rotatable in the mobile robot 300, this allows the mobile robot 300 to orient the visual recognizer 330 to point towards any of the two racks 406, or particularly the visual marker 430 disposed on any of the two racks 406, when the mobile robot 300 is in the corresponding aisle 404. In such example, the visual recognizer 330 may be mounted on one of an edge of the stacker 310 which may be parallel to and facing the shelves 504 in the rack 406 when the mobile robot 300 is travelling in the aisle 404. Further, the stacker 310 may be rotatable with respect to the vertical tower 306, such that the visual recognizer 330 therein is able to capture the image of the visual markers 430 on the racks 406 on both sides of the aisle 404. In alternative example, the stacker 310 may be provided with two visual recognizers 330 on both, opposing edges thereof, which may be parallel to and facing the shelves 504 in the racks 406 when the mobile robot 300 is travelling in the aisle 404, to be able to capture the image of the visual markers 430 on the racks 406 on both sides of the aisle 404, without any limitations. In the present examples, the visual recognizer 330 may be in the form of a camera as known in the art. Also, for the purposes of the present disclosure, a focal length of the visual recognizer 330 is fixed and is known.
[0057] FIG. 6 illustrates an exemplary implementation of the system 100 for navigating the mobile robot 300 in the aisle 404 with respect to the rack 406 therein, in accordance with one or more embodiments of the present disclosure. In the present embodiments, the mobile robot 300 may implement the ground marker recognizer 320, which may act as the primary navigation sensor, to detect the ground markers 420, and thereby allow the mobile robot 300 to follow a somewhat straight path as defined by the system 100 for carrying out the currently assigned operation thereto. This enables the mobile robot 300 to navigate in the open region 402, and to get to an entry of the aisle 404 (as may be instructed by the system 100). Further, this may enable the mobile robot 300 to navigate in the aisle 404, if provided with the ground markers 420, at least as per the path defined by the system 100. Further, the mobile robot 300 implements the visual recognizer 330 to detect the visual markers 430, which helps the mobile robot 300 to navigate in the aisle 404 by keeping its distance from the corresponding racks 406 and its orientation correct at, generally, all times, as discussed in more detail in the proceeding paragraphs.
[0058] For the purposes of navigation in the aisle 404, the mobile robot 300, as part of the system 100, includes a controller (not depicted in the mobile robot 300). Herein, the controller may generally be part of or equivalent to the computing system 200 as described above; and thus, hereinafter, the controller 200 has been referred by the reference numeral 200. Such controller 200 may be implemented in hardware, firmware or software, or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller 200 may be centralized or distributed, whether locally or remotely. Such controller 200 may be a multi-core processor, a single core processor, or a combination of one or more multi-core processors and one or more single core processors. For example, the one or more processors may be embodied as one or more of various processing devices, such as a coprocessor, a microprocessor, a digital signal processor (DSP), a processing circuitry with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. Further, the memory may include one or more non-transitory computer-readable storage media that can be read or accessed by other components in the device. The memory may be any computer-readable storage media, including volatile and/or non-volatile storage components, such as optical, magnetic, organic or other memory or disc storage, which can be integrated in whole or in part with the device. In some examples, the memory may be implemented using a single physical device (e.g., optical, magnetic, organic or other memory or disc storage unit), while in other embodiments, the memory may be implemented using two or more physical devices without any limitations.
[0059] According to embodiments of the present disclosure, the controller 200 is configured to implement the visual recognizer 330 to capture an image of the visual marker 430. Specifically, the controller 200 may provide instructions to the stacker 310 to travel to a height and be oriented, such that the visual marker 430 placed on an adjacent rack 406 is in the field of view of the visual recognizer 330. Once the visual marker 430 is in the field of view of the visual recognizer 330, the visual recognizer 330 may capture the image of rack 406 with the visual marker 430 therein.
[0060] The controller 200 is further configured to determine an apparent dimension of the visual marker 430 in the captured image of the visual marker 430 based on the known fixed focal length of the visual recognizer 330. Herein, for example, the controller 200 may determine the number of pixels along one of the edges of the visual marker 430 in the captured image thereof. The controller 200 may implement any of the suitable image processing techniques as known in the art to determine (count) the said number of pixels, which may be understood to have direct correlation to the dimension of the visual marker 430 in the captured image of the visual marker 430. Then, the controller 200 may determine the apparent dimension of the visual marker 430 based on the determined number of pixels and the known fixed focal length of the visual recognizer 330 which captured the said image. Such technique may be contemplated by a person skilled in the art of image processing and thus has not been described further herein for the brevity of the present disclosure.
[0061] The controller 200 is further configured to determine an angular orientation of the visual marker 430 with respect to the visual recognizer 330 based on the known absolute dimension of the visual marker 430 and the determined apparent dimension of the visual marker 430. That is, once the stacker 310 of the mobile robot 300 is aligned with the visual marker 430 on the rack 406 (or any known external physical feature, as discussed later), while in motion or stopped, the absolute positions (in the map frame) of the visual marker 430 which may be represented in X, Y, Z, Tx, Ty and Tz within the field of view of the visual recognizer 330 are obtained. In particular, given the known absolute dimensions of the visual marker 430 and the fixed focal length of the visual recognizer 330, the 6 degree-of-freedom (DOF), including X, Y, Z, Tx, Ty and Tz, of the visual marker 430 is obtained. In an embodiment, the controller is 200 configured to use one or more of: homography technique and orthogonal iteration technique for determining angular orientation of the visual marker 430 with respect to the visual recognizer 330. In other words, the angular orientation of the visual marker 430 with respect to the visual recognizer 330 is determined using one or more of: homography technique and orthogonal iteration technique. That is, herein, the 6 DOF pose estimate of the visual marker 430 is obtained using methods such as but not limited to homography and orthogonal iteration. Implementation of such technique(s) may be contemplated by a person skilled in the art and thus is not described herein for the brevity of the present disclosure.
[0062] The controller 200 is further configured to determine an angular error for the mobile robot 300 by translating the determined angular orientation of the visual marker 430 based on a pose of the visual recognizer 330 with respect to the mobile robot 300 while capturing the image of the visual marker 430. That is, the obtained angular error of the visual marker 430 is then translated into the angular error of the mobile robot 300 (plane perpendicular to the reference plane of the visual marker 430) using the transformations of the axes of the visual recognizer 330 to the frame of the mobile robot 300. Again, such implementation may be contemplated by a person skilled in the art and thus is not described herein for the brevity of the present disclosure.
[0063] In a similar manner, in some embodiments, the controller 200 is further configured to determine a cross-track error for the mobile robot 300 based on the known absolute dimension of the visual marker 430, and one or more of the known fixed focal length of the visual recognizer 330 and the determined apparent dimension of the visual marker 430 in the captured image of the visual marker 430. Herein, the cross-track error may represent the shortest distance from the centre-line of the mobile robot 300 to the reference trajectory (i.e., the line to be followed by the mobile robot 300 in the aisle 404). In an embodiment, the controller 200 is configured to use triangle similarity technique for determining the cross-track error for the mobile robot 300. In other words, the cross-track error for the mobile robot 300 is determined using triangle similarity technique. That is, such cross-track error is obtained by processing the image of the visual marker 430 obtained from the visual recognizer 330 using methods like triangle similarity: For example, the cross-track error may be calculated by the equation as given below.
Cross-Track Error (e) =
(Absolute Dimension of the Visual Marker x Focal Length of the Visual Recognizer) / Apparent Dimension of the Visual Marker.
[0064] The controller 200 is further configured to localize the mobile robot 300 in the aisle 404 based on the determined angular error for the mobile robot 300 and the known absolute position of the visual marker 430, to be utilized for navigating the mobile robot 300 in the aisle 404. That is, herein, based on the determined angular error for the mobile robot 300, the orientation of the mobile robot 300 may be corrected such that the mobile robot 300 follows a straight line path without hitting either of the racks 406 when moving in the aisle 404. Also, the controller 200 is further configured to localize the mobile robot 300 in the aisle 404 based on the determined cross-track error for the mobile robot 300, to be utilized for navigating the mobile robot in the aisle. That is, herein, based on the determined cross-track error for the mobile robot 300, the lateral distance of the mobile robot 300 from the racks 406 in the aisle 404 may be adjusted (corrected), such that the mobile robot 300 may be able to move in the aisle 404 without hitting either of the racks 406 corresponding thereto.
[0065] In particular, with both the angular error and the cross-track error estimated, the pose of the mobile robot 200 in the map frame is now determined, and fused or substituted with other sources of odometry to improve the pose estimate thereof for proper navigation in the aisle 404. The present disclosure provides a technique to use the ground markers 420 and the visual markers 430 that exist in different planes as an aid and or redundant form of localization for the mobile robot 300. Therefore, in the present disclosure, the mobile robot 300 as required to pick and drop items into the racks 406 may implement a combination of the grid of ground markers 420 provided on the floor in the aisle 404, and the visual markers 430 provided on the racks 406, for navigation within the aisle 404.
[0066] As may be contemplated, the correct localization (orientation and lateral distance from the racks 406) of the mobile robot 300 in the aisle 404 may also allow for proper stacking and picking of items into the racks 406. In one or more examples of the present disclosure, the ground markers 420 (specifically in the aisle 404) may be arranged at a first distance apart from each other; and the visual markers 430 (as arranged on the racks 406) may be arranged at a second distance apart from each other. In an example embodiment, the second distance is smaller than the first distance for the purposes of the present disclosure. This resultant higher density of the visual markers 430 in the aisle 404 helps to reduce dependency on the ground markers 420 for the mobile robot 300 to be implemented for stacking and picking of items to specific storage area 506 therein into the racks 406. In particular, when the physical location of pick and drop points on the racks 406 may not necessarily match the position of the ground marker 420 in the aisle 404, the visual markers 430 may provide the secondary source of odometry that utilizes features on a secondary plane from which odometry of the mobile robot 300 is corrected and localized. This enables the system 100 of the present disclosure to utilize the mobile robot 300 to robustly pick and drop items with respect to the external rack 406 within the confined aisle 404.
[0067] It may be appreciated that although the embodiment of the present disclosure has been described in terms of the visual markers 430 being placed on the racks 406, it may be appreciated that the visual markers 430 may be placed on ay suitable external feature (surface) which may be associated with the aisle 404 and may be used as a reference for the purposes of the present disclosure; i.e., may be in the field of view of the visual recognizer 330 when the mobile robot 300 is located in the aisle 404.
[0068] The present disclosure further provides a method for navigating a mobile robot in an aisle using an external feature, associated with the aisle, as a reference therefor. Various embodiments and variants disclosed above, with respect to the aforementioned system 100, apply mutatis mutandis to the present method. FIG. 7 is a flowchart 700 of a method for navigating a mobile robot in an aisle using an external feature, associated with the aisle, as a reference therefor. The various steps involved in the present method have been depicted as blocks in the flowchart 700 of FIG. 7, and the details for the same have been provided hereinafter.
[0069] At step 702, the method includes providing the external feature with a visual marker, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known. At step 704, the method includes implementing a visual recognizer, with a known fixed focal length, mounted on the mobile robot to capture an image of the visual marker. At step 706, the method includes determining an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer. At step 708, the method includes determining an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker. Herein, the angular orientation of the visual marker with respect to the visual recognizer is determined using one or more of: homography technique and orthogonal iteration technique. At step 710, the method includes determining an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker. In an embodiment, the method further includes determining a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker. Herein, the cross-track error for the mobile robot is determined using triangle similarity technique. At step 712, the method includes localizing the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle. Further, the method includes localizing the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.
[0070] The system and the method of the present disclosure provide for the mobile robot to use the ground markers and the visual markers that exist in different planes as an aid and or redundant form of localization therefore, and thereby enables the system to robustly pick, drop items with respect to the external rack within a confined aisle. The present disclosure provides techniques for navigation in between aisles, with the ground markers being used to navigate in the work area to pick and place items into an external rack; ad when the physical location of pick and drop points on the external racks do not necessarily match the ground markers, the secondary source of odometry that utilizes visual markers on a secondary plane are implemented from which robot odometry is corrected and localized.
[0071] The foregoing descriptions of specific embodiments of the present disclosure have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiment was chosen and described in order to best explain the principles of the present disclosure and its practical application, to thereby enable others skilled in the art to best utilize the present disclosure and various embodiments with various modifications as are suited to the particular use contemplated.
, Claims:WE CLAIM:
1. A method for navigating a mobile robot in an aisle using an external feature, associated with the aisle, as a reference therefor, the method comprising:
providing the external feature with a visual marker, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known;
implementing a visual recognizer, with a known fixed focal length, mounted on the mobile robot to capture an image of the visual marker;
determining an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer;
determining an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker;
determining an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker; and
localizing the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle.

2. The method as claimed in claim 1 further comprising:
determining a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker; and
localizing the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.

3. The method as claimed in claim 1, wherein the angular orientation of the visual marker with respect to the visual recognizer is determined using one or more of: homography technique and orthogonal iteration technique.

4. The method as claimed in claim 2, wherein the cross-track error for the mobile robot is determined using triangle similarity technique.

5. A system for navigating a mobile robot in an aisle using an external feature, associated with the aisle, as a reference therefor, the system comprising:
a visual marker provided with the external feature, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known;
a visual recognizer, with a known fixed focal length, mounted on the mobile robot; and
a controller configured to:
implement the visual recognizer to capture an image of the visual marker;
determine an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer;
determine an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker;
determine an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker; and
localize the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle.

6. The system as claimed in claim 5, wherein the controller is further configured to:
determine a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker; and
localize the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.

7. The system as claimed in claim 5, wherein the controller is configured to use one or more of: homography technique and orthogonal iteration technique for determining angular orientation of the visual marker with respect to the visual recognizer.

8. The system as claimed in claim 6, wherein the controller is configured to use a triangle similarity technique for determining the cross-track error for the mobile robot.

9. A mobile robot adapted to navigate in an aisle using an external feature, associated with the aisle, as a reference therefor, the external feature being provided with a visual marker, with at least an absolute dimension of the visual marker and an absolute position of the visual marker being known, the mobile robot comprising:
a visual recognizer, with a known fixed focal length, mounted thereon; and
a controller configured to:
implement the visual recognizer to capture an image of the visual marker;
determine an apparent dimension of the visual marker in the captured image of the visual marker based on the known fixed focal length of the visual recognizer;
determine an angular orientation of the visual marker with respect to the visual recognizer based on the known absolute dimension of the visual marker and the determined apparent dimension of the visual marker;
determine an angular error for the mobile robot by translating the determined angular orientation of the visual marker based on a pose of the visual recognizer with respect to the mobile robot while capturing the image of the visual marker; and
localize the mobile robot in the aisle based on the determined angular error for the mobile robot and the known absolute position of the visual marker, to be utilized for navigating the mobile robot in the aisle.

10. The mobile robot as claimed in claim 9, wherein the controller is further configured to:
determine a cross-track error for the mobile robot based on the known absolute dimension of the visual marker, and one or more of the known fixed focal length of the visual recognizer and the determined apparent dimension of the visual marker in the captured image of the visual marker; and
localize the mobile robot in the aisle based on the determined cross-track error for the mobile robot, to be utilized for navigating the mobile robot in the aisle.