Description:TECHNICAL FIELD
[001] The present invention relates generally to control a system and method to operate digitally still camera with multiview point.
BACKGROUND
[002] With the continuous improvement of the photographing technology of mobile terminals such as smart phones and tablet computers, the definition of an image photographed by a camera of the mobile terminal is also continuously improved, even then the visual effect of the distant view image is often affected.
[003] Embedded devices are often equipped with miniature camera modules. These miniature camera modules typically include only a single fixed-focus lens. Some of these devices have software applications downloaded or otherwise stored on them that permit limited choices in pre-capture camera settings, such as exposure duration and flash setting, and some provide limited post-capture image editing capabilities designed to compensate for the inadequacy of the built-in optics. Image processing software is however incapable of providing real images of objects that are too close or too far from the device, or of scenes including multiple objects that require greater depths of field in order to capture them without intolerable amounts of defocus blur or of scenes with moving objects without excessive motion-related blur, among other imaging issues.
[004] Therefore, there is a need of a system which overcomes the aforementioned problems.
SUMMARY
[004] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems.
[005] Before the present subject matter relating to a system and method to operate digitally still camera with multiview point, it is to be understood that this application is not limited to the particular system described, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosure. It is also to be understood that the terminology used in the description is for the purpose of describing the implementations or versions or embodiments only and is not intended to limit the scope of the present subject matter.
[006] This summary is provided to introduce aspects related to a system and method to operate digitally still camera with multiview point. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the present subject matter.
[007] In one embodiment, a system to operate digitally still camera with multiview point includes a plurality of cameras, a body, a three-dimensional space recognition device, a liquid crystal display, a tracking apparatus and a camera corrector. The plurality of cameras configures to capture images from different positions in a predetermined space. The body connected to the plurality of cameras unit configures to be rotatable by a set angle and performing a function of a digital still camera. The three-dimensional space recognition device configures to generate a free viewpoint image using the three-dimensional model data. The liquid crystal display means configured on a part of the first body to display image information. The tracking apparatus having at least three receivers configures to position around an operational area and configured to receive the first and second locate signals. The camera corrector being configured to calculate parameters indicating angles of the respective cameras by taking a correspondence between an actual point in a photographing region of the respective cameras and a point on a video or between different videos of the respective camera.
[008] In one embodiment, a method to operate digitally still camera with multiview point includes a step of determining an object location. The method includes a step of generating, based upon the object location, the camera location, and the camera orientation. The method includes a step of transmitting the generated free viewpoint image to a user device including a display unit.
BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS
[009] The foregoing detailed description of embodiments is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the disclosure, there is shown in the present document example constructions of the disclosure; however, the disclosure is not limited to the specific system or method disclosed in the document and the drawings.
[0010] The present disclosure is described in detail with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to refer various features of the present subject matter.
[0011] Figure 1 illustrates an exemplary system for tracking and controlling a mobile camera to image an object of interest (OOI), in an embodiment.
[0012] Figure 2 and 3 illustrate an exemplary relationships that defines a desired perspective, a field of view and the corresponding relative positions and orientations of the mobile camera and the OOI, in an embodiment.
[0013] Figure 4 illustrates a diagram showing an outline of three-dimensional space recognition.
[0014] In the above accompanying drawings, a non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.
[0015] Further, the figures depict various embodiments of the present subject matter for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the present subject matter described herein.
DETAILED DESCRIPTION
[0016] Some embodiments of this disclosure, illustrating all its features, will now be discussed in detail. The words "comprising," "having," "containing," and "including," and other forms thereof, are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Although a system and method to operate digitally still camera with multiview point, similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the exemplary, a system and method to operate digitally still camera with multiview point is now described.
[0017] Various modifications to the embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. For example, although the present disclosure will be described in the context of a system and method to operate digitally still camera with multiview point, one of ordinary skill in the art will readily recognize a system and method to operate digitally still camera with multiview point can be utilized in any situation. Thus, the present disclosure is not intended to be limited to the embodiments illustrated but is to be accorded the widest scope consistent with the principles and features described herein.
[0018] In one embodiment, a system to operate digitally still camera with multiview point includes a plurality of cameras, a body, a three-dimensional space recognition device, a liquid crystal display, a tracking apparatus and a camera corrector. The plurality of cameras configures to capture images from different positions in a predetermined space. The body connected to the plurality of cameras unit configures to be rotatable by a set angle and performing a function of a digital still camera. The three-dimensional space recognition device configures to generate a free viewpoint image using the three-dimensional model data. The liquid crystal display means configured on a part of the first body to display image information. The tracking apparatus having at least three receivers configures to position around an operational area and configured to receive the first and second locate signals. The camera corrector being configured to calculate parameters indicating angles of the respective cameras by taking a correspondence between an actual point in a photographing region of the respective cameras and a point on a video or between different videos of the respective camera.
[0019] In another implementation, the camera control apparatus having a processor and a memory containing machine readable instructions configures to execute by the processor perform the step of generating a movement plan for the first mobile camera to maintain a desired perspective of the first object of interest (OOI) based upon at least the first and second location data, the movement plan defining a velocity and orientation of the first mobile camera.
[0020] In another implementation, the camera adjusting part configures for executing camera adjustment for controlling the shooting area of at least one part of the plurality of cameras.
[0021] In another implementation, the machine-readable instructions further configures to confine movement of the first and second mobile camera and to coordinate movement of the first and second mobile camera to maintain clear lines of sight from cameras and ensures that the first and second mobile camera do not collide.
[0022] In one embodiment, a method to operate digitally still camera with multiview point includes a step of determining an object location. The method includes a step of generating, based upon the object location, the camera location, and the camera orientation. The method includes a step of transmitting the generated free viewpoint image to a user device including a display unit.
[0023] In another implementation, the method includes step of generating a movement plan for the mobile camera to maintain a desired perspective of the OOI from the mobile camera, the desired perspective defining a range and a direction of the mobile camera relative to the object location.
[0024] In another implementation, the method includes the step of controlling the plurality of camera restricting movement of the plurality of camera to an operational area.
[0025] In another implementation, the desired perspective further defines a desired magnification of the mobile camera.
[0026] Figure 1 illustrates an exemplary system 100 for tracking and controlling a mobile camera 102 to image an object of interest (OOI) 104.
[0027] In an embodiment, mobile camera 102 is mounted to a mobile camera platform 106. Mobile camera 102 is configured with a field of view 128 (represented as a dashed lines) and positioned and oriented (e.g., by control of drone 108) to capture a desired perspective 129 (represented as dashed lines) of OOI 104 at current location 120. OOI 104 may remain stationary, may remain at a certain location but change orientation, or as shown in figure 1, OOI 104 may follow a predictable path 124, moving to a current location 120 from a previous location 122 and then continuing movement to a next location 126.
[0028] Desired perspective 129 is a front view of the face of OOI 104 where OOI 104 is a person. Desired perspective 129 is defined relative to location and orientation of OOI 104, and thus to capture desired perspective 129, the location and orientation of mobile camera 102 also needs to change as that of OOI 104 changes. As the orientation of OOI 104 changes (i.e., OOI 104 rotates), the desired perspective is maintained by changing the location and/or field of view 128 of mobile camera 102. Where OOI 104 is stationary (e.g., at location 120), based upon a determined orientation of OOI 104, mobile camera 102 is controlled and positioned to obtain and maintain desired perspective 129 of OOI 104. As OOI 104 moves to next location 126, rotating as shown in FIG. 1, mobile camera 102 is controlled to move to maintain desired perspective 129 relative to OOI 104, wherein mobile camera 102 reaches location 154 as OOI 104 reaches location 126.
[0029] Figure 2 and 3 illustrate an exemplary relationships that defines a desired perspective, a field of view and the corresponding relative positions and orientations of the mobile camera and the OOI.
[0030] In an embodiment, for a defined field of view 128 of mobile camera 102, as a camera position relative to the OOI and a camera orientation. For example, a range 204 and an angle a relative to a facing direction (indicated by line 202) of OOI 104 define a location for mobile camera 102 relative to OOI 104, and a reciprocal of angle a and an offset angle ß define an orientation of mobile camera 102 to capture a desired image 302 of OOI 104. In the example of figure 2 and 3, offset angle ß moves the position of OOI 104 within image 302. For example, where OOI 104 is a leader in a race, by offsetting the position of the leader or changing magnification, other racers may also be included.
[0031] In an embodiment, in addition to controlling position and orientation of multiple mobile cameras 102, camera controller 140 controls a zoom or magnification feature of mobile camera 102 such that the perspective, including size and position of images obtained, of a leader in the race can be maintained. In an embodiment, multiple athletes are tracked by configuring each player with a tracking tag, such as by placing a tracking tag in each athletes' helmet. In this embodiment, event detector 160 of camera controller 140 determines whether a second runner or vehicle in a race is near the leader in the race. If there is no racer close to the leader, the perspective selector 162 of the camera controller selects a first perspective to moves position, orientation, and magnification, of camera 102 to maintain a desired size and position of the leader, and sets this perspective as desired perspective 129. When there is another racer close to the leader, an expert system method of the camera controller perspective selector 162 modifies the desired perspective 129 to move position, orientation, and magnification of at least one camera 102 to show both the leader and the racer closest to the leader as an expert cameraman would. Since most racers in race events travel in the same direction, in an embodiment the orientation of racers may in some embodiments be assumed to be in their direction of movement and not measured separately by the tracking tag. In these embodiments, the tag-based magnetometer or ADF and accelerometers may be omitted.
[0032] Figure 4 illustrates a diagram showing an outline of three-dimensional space recognition.
[0033] In an embodiment, the multi-view imaging system includes a plurality of cameras for capturing the same scene in a predetermined space. For example, the predetermined space is a meeting place where a sporting event is performed, and the same scene is a scene of a game performed in the meeting place. As another example, the predetermined space is a space to be monitored by using a monitoring camera, and the same scene may be a person or object in the monitored space.
[0034] The plurality of cameras each capture images of at least a part of the overlapping region from different positions in the space. For example, as shown, a plurality of cameras 400 are provided at positions that surround a space of a game venue that is sports and are different from each other. Further, the cameras 400 take different postures from each other, and accommodate at least a part of the space into the respective photographing regions. At least a part of the imaging area of each camera 400 is repeated because image data generated by imaging the same subject from a plurality of viewpoints is used for virtual reconstruction of a three-dimensional space (three-dimensional space reconstruction).
[0035] The plurality of cameras 400 thus provided are communicably connected to a control device, which will be described later, provided in the multi-viewpoint imaging system. When camera correction is performed immediately before image capturing, the accuracy of camera parameters is sufficient immediately after image capturing is started, but the accuracy generally decreases with the lapse of time due to factors such as vibration of the installation place of the camera 400 and operation of the camera 400. The following multi-viewpoint imaging system will be described below: by timely performing the camera correction of the camera 400 in which the accuracy of the camera parameters is reduced even during imaging, adverse effects of the accuracy reduction of the camera parameters on the three-dimensional space reconstruction are suppressed, and further, the accuracy and usability of the three-dimensional space recognition are stabilized.
[0036] Although the description provides implementations of a system and method to operate digitally still camera with multiview point, it is to be understood that the above descriptions are not necessarily limited to the specific features or methods or systems. Rather, the specific features and methods are disclosed as examples of implementations for the system and method to operate digitally still camera with multiview point.
, Claims:We claim:
1. A system for operating a digitally still camera with multi-view points, comprising:
a digital camera sensor for capturing images;
a memory unit for storing captured images;
a plurality of viewpoints, each configured to capture an image at a different angle or position;
a processor configured to control the digital camera sensor to capture images from the plurality of viewpoints, and to store said captured images in the memory unit;
an image processing module configured to combine the images captured from the plurality of viewpoints to generate a multi-view image; and
a user interface for enabling a user to select and view the multi-view image.
2. The system of Claim 1, further comprising a display unit for displaying the multi-view image to the user.
3. The system of Claim 1, wherein the plurality of viewpoints includes at least two viewpoints positioned at different locations.
4. The system of Claim 1, wherein the processor is further configured to adjust camera settings for each view point based on predetermined criteria.
5. A method for operating a digitally still camera with multi-view points, comprising the steps of:
capturing images from a plurality of view points, wherein each view point captures an image at a different angle or position;
storing the captured images in a memory unit;
combining the captured images to generate a multi-view image; and
displaying the multi-view image on a display unit.
6. The method of Claim 5, further comprising the step of adjusting camera settings for each view point based on predetermined criteria.
7. The method of Claim 5, further comprising the step of enabling a user to select and view the multi-view image via a user interface.