The title of the invention: the imaging device, orientation information processing method and program
Technical field
[0001]
　The present invention relates to an imaging device, about the orientation information processing method, and a program.
Background technique
[0002]
　In recent years, in an imaging apparatus such as digital cameras, it has appeared models equipped with electronic compass. Electronic compass, based on the geomagnetism detected by the geomagnetic sensor has a function of device calculates the directly facing orientation electronically. By mounting the electronic compass in a digital camera, that the digital camera is directly opposite orientation (i.e., orientation of the image capturing direction) is displayed on the display unit a two-dimensional compass image showing, to recognize the direction in photographer can.
[0003]
　Here, the azimuth information used in the image pickup apparatus such as a digital camera, apart from the purpose of displaying a compass image described above, as additional information of the captured image, the imaging direction of the orientation (hereinafter, the imaging direction) in order to record the Used. For example, a digital camera described in Patent Document 1, an imaging azimuth information obtained by the electronic compass, while superimposed on the captured image displayed on the display unit, imaging the imaging direction information as additional information of the captured image in association with the image to be recorded. The digital camera of the Patent Document 1 is equipped with an rotatable mechanism an imaging unit with respect to the body part, in accordance with the rotation state of the imaging unit with respect to the body part, resulting in an electronic compass imaging it can be recorded by correcting the azimuth information.
CITATION
Patent literature
[0004]
Patent Document 1: Patent No. 3781016
Summary of the invention
Problems that the Invention is to Solve
[0005]
　By the way, the digital camera described in Patent Document 1, in accordance with the rotation state of the imaging unit with respect to the body part, but to correct the azimuth obtained by the electronic compass, a change in the posture of the digital camera (e.g., rotation in the roll direction ) depending on the orientation to be displayed on the display unit (hereinafter, did not correct the orientation) for display. However, if the orientation of the display in response to a change in posture of the camera is not corrected, the following problem occurs.
[0006]
　Typical digital cameras, since the lens is installed on the back surface of the display screen, the image pickup direction (i.e., the optical axis of the lens) and the back of the display screen (i.e., a positive-to-the direction of the camera) and match to. User, such digital cameras have so as to stand up to the vertical (90 °) the display screen relative to the ground, consider a case of imaging with the imaging direction in the horizontal direction. In this case, the digital camera electronic compass is calculated horizontal direction of the imaging direction, compass image in which the said position upwards is displayed on the display unit.
[0007]
　In this state, the user, while maintaining the angle of the display screen the ground vertically, if the digital camera is rotated 90 ° in the roll direction for imaging (ie, if the vertical take you) But, facing the user changing an azimuth (yaw direction) is no, the imaging direction is also unchanged. Therefore, regardless of the change in the posture of the camera (the rotation in the roll direction), so as to always face the constant direction compass image indicating the imaging orientation with respect to the ground, to view the compass image, from the point of view of the user on is a natural.
[0008]
　To display this manner compass image, the image capturing direction that is calculated by the electronic compass, obtains the azimuth of the display is corrected by a rotation angle portion of the roll direction, displays the compass image indicating the direction for the display it may be displayed in the part. Thus, even by rotating the camera in the roll direction, from the user's perspective, the compass image on the display screen always seem to be facing a certain direction relative to the ground. Accordingly, the display mode of the compass image showing the imaging orientation, becomes natural to the user.
[0009]
　However, the orientation of the display which is corrected as described above, different from the actual imaging direction. Therefore, when the orientation for the display will be recorded as additional information of the captured image, it is impossible to record the correct imaging direction information, when the reproduction of the captured image, can no longer provide the correct imaging direction of the imaging image put away.
[0010]
　As described above, in the conventional digital camera with an electronic compass, a change in posture of the camera (e.g., change from horizontal imaging to vertical shooting) not corrected orientation for display according to, for the display and orientation, did not distinguish the orientation of the recording as an additional information of the captured image (correct imaging orientation).
[0011]
　Accordingly, the present invention has been made in view of the above circumstances, even when the orientation of the image sensing device is changed, it is possible to display the compass image in a natural manner when viewed from the user's point of view, to record the correct imaging direction it is intended to allow.
Means for Solving the Problems
[0012]
　In order to solve the above problems, according to an aspect of the present invention, an imaging unit that outputs a captured image by imaging a subject, a geomagnetic information detected by the geomagnetic sensor, the acceleration information detected by the acceleration sensor based on the orientation calculation unit that calculates the azimuth of the image pickup direction by the imaging unit, based on the acceleration information, and calculates the rotation angle in the roll direction of the imaging device, based on the rotation angle of the roll direction, the imaging an azimuth converter for converting the orientation of the display orientation direction, the compass image representing an orientation for the display, and a display unit for displaying together with the captured image, the orientation information indicating the orientation of the image pickup direction, the image pickup ; and a recording unit for recording on a recording medium in association with the image, the imaging apparatus is provided.
[0013]
　The orientation calculation unit, based on the acceleration information, and calculates the rotation angle in the pitch direction of the imaging device, based on the rotation angle of the pitch direction, detects whether or not the imaging direction is substantially vertical direction and, if the imaging direction is the substantially vertical direction, the orientation calculation unit, the place of the imaging direction of the azimuth, calculates the orientation of the direction perpendicular to the imaging direction, the display unit, the display instead of the compass image representing the orientation of use, the compass image representing the orientation of a direction perpendicular to the imaging direction, and displayed together with the captured image, the recording unit, the direction information indicative of the orientation of the imaging direction Alternatively, the orientation information indicating the orientation of the direction perpendicular to the imaging direction, may be recorded on the recording medium in association with the captured image.
[0014]
　The orientation calculation unit, based on the acceleration information, and calculates the rotation angle in the pitch direction of the imaging device, based on the rotation angle of the pitch direction, and detects whether or not the imaging direction is substantially a vertically downward If the imaging direction is substantially vertically downward, the orientation calculation unit, the place of the imaging direction of the azimuth, calculates the direction of the top surface direction of the image pickup apparatus, wherein the display unit, the orientation for the display instead of the compass image representing the compass image representing the orientation of the top surface direction, and displayed together with the captured image, the recording unit, instead of the direction information representing a direction of the imaging direction, of the top surface direction orientation information indicating the orientation may be recorded on the recording medium in association with the captured image.
[0015]
　The orientation calculation unit, based on the acceleration information, and calculates the rotation angle in the pitch direction of the imaging device, based on the rotation angle of the pitch direction, and detects whether or not the imaging direction is substantially a vertically upward If the imaging direction is substantially vertically upward, the orientation calculation unit, the place of the imaging direction of the azimuth, calculates the direction of the bottom surface direction of the image pickup apparatus, wherein the display unit includes an orientation for the display instead of the compass image representing the compass image representing the orientation of the bottom direction, and displayed together with the captured image, the recording unit, instead of the direction information representing a direction of the imaging direction represents the orientation of the bottom face direction the direction information may be recorded on the recording medium in association with the captured image.
[0016]
　The imaging apparatus may further include a reproducing unit for reproducing the captured image and the direction information recorded in the recording medium, said display unit, a compass image representing an orientation of said orientation information reproduced by said reproducing unit , it may be displayed together with the captured image reproduced by the reproducing unit.
[0017]
　Further, in order to solve the above problems, according to another aspect of the present invention, while imaging the subject by the imaging unit, a geomagnetic information detected by the geomagnetic sensor, based on the acceleration information detected by the acceleration sensor a step of calculating the orientation of the image capturing direction by the imaging unit, based on the acceleration information, and calculates the rotation angle in the roll direction of the imaging device, based on the rotation angle of the roll direction, the orientation of the imaging direction and converting the azimuth for display, the compass image representing an orientation for the display, and displaying together with a captured image output from the imaging unit, a direction information representing the orientation of the image pickup direction, the captured image comprising the steps of recording on a recording medium in association with the azimuth information processing method is provided.
[0018]
　Further, in order to solve the above problems, according to another aspect of the present invention, while imaging the subject by the imaging unit, a geomagnetic information detected by the geomagnetic sensor, based on the acceleration information detected by the acceleration sensor a step of calculating the orientation of the image capturing direction by the imaging unit, based on the acceleration information, and calculates the rotation angle in the roll direction of the imaging device, based on the rotation angle of the roll direction, the orientation of the imaging direction and converting the azimuth for display, the compass image representing an orientation for the display, and displaying together with a captured image output from the imaging unit, a direction information representing the orientation of the image pickup direction, the captured image program for executing the steps of recording on a recording medium in association with the, to the computer is provided.
[0019]
　According to the above configuration, while imaging the subject by the imaging unit, a geomagnetic information detected by the geomagnetic sensor, based on the acceleration information detected by the acceleration sensor, the orientation of the imaging direction is calculated by the imaging unit, wherein based on the acceleration information, and calculates the rotation angle in the roll direction of the imaging device, based on the rotation angle of the roll direction, the orientation of the imaging direction is converted into the orientation for display, compass representing the orientation for the display images are displayed together with the captured image output from the imaging unit, bearing information representative of the orientation of the imaging direction is recorded on a recording medium in association with the captured image. Thus, as the orientation to be displayed on the display unit at the time of imaging, while using the orientation for display by converting the orientation of the imaging direction in accordance with the posture of the imaging device (rotation angle in the roll direction), it is recorded as additional information of the captured image as a direction, it is possible to use the orientation of the imaging direction.
Effect of the invention
[0020]
　According to the present invention described above, even when the orientation of the image sensing device is changed, it is possible to it is possible to display the compass image in a natural manner when viewed from the user's point of view, to record the correct imaging orientation.
Brief description of the drawings
[0021]
Is a block diagram showing a hardware configuration of an imaging apparatus according to a first embodiment of the present invention; FIG.
Is a block diagram showing the functional configuration of FIG. 2 imaging device according to the embodiment.
It is a perspective view showing the imaging direction and attitude of [3] an imaging apparatus according to the embodiment.
Is a perspective view showing an imaging apparatus which is inclined in the roll direction according to [4] the same embodiment.
FIG. 5 is a perspective view showing an imaging device that is rotated 90 ° in the roll direction according to the embodiment.
FIG. 6 is a rear view showing a display screen of the image pickup apparatus in the state shown in FIG. 3.
Is a rear view showing a display screen in FIG. 7 imaging device in a state in FIG.
Is a rear view showing a display screen in FIG. 8 the image pickup apparatus in the state of FIG.
9 is a diagram showing a playback screen of an imaging apparatus according to the embodiment.
It is a block diagram showing a functional configuration of an imaging apparatus according to the second embodiment of FIG. 10 the present invention.
11 is a perspective view showing the top surface direction and orientation of the imaging apparatus according to the embodiment.
It is a perspective view showing the bottom direction and orientation of FIG. 12 imaging device according to the embodiment.
13 is a perspective view showing a display screen of an image pickup apparatus according to the embodiment.
14 is a diagram showing a playback screen of an image imaged in a state of FIG. 13.
15 is a perspective view showing a display screen of an image pickup apparatus that is rotated 90 ° in the roll direction according to the embodiment.
Is a diagram showing a playback screen of an image imaged in a state of FIG. 16 FIG. 15.
17 is a flowchart showing an azimuth processing method performed by the imaging apparatus according to the embodiment.
DESCRIPTION OF THE INVENTION
[0022]
　Reference will now be described in detail preferred embodiments of the present invention. In the specification and the drawings, components having substantially the same functional configuration are repeated explanation is omitted by referring to the figures.
[0023]
　The description will be made ​​in the following order.
　1. Hardware configuration of the imaging device
　2. The functional configuration of the imaging device
　　2.1. Calculation processing of the imaging orientation
　　2.2. Display processing of the compass image
　　2.3. The recording process of the imaging orientation
　　2.4. Playback processing of the image orientation
　3. Second embodiment
　　3.1. The functional configuration of the imaging device
　　3.2. Calculation processing of the imaging orientation
　　3.3. Display processing of the compass image
　　3.4. Orientation information processing method
　4. Summary
[0024]
　[1. Hardware configuration of the imaging apparatus]
　First, referring to FIG. 1, it will be described in detail about the hardware configuration of the imaging apparatus 10 according to the first embodiment of the present invention. Figure 1 is a block diagram showing a hardware configuration of the imaging apparatus 10 according to the present embodiment. Imaging device of the present invention is, for example, be implemented by a digital camera as an imaging apparatus 10 shown in FIG. 1, is not limited to such an example, is applicable to any electronic device with an imaging function.
[0025]
　1, the imaging apparatus 10 according to the present embodiment, for example, still images or imaging possible digital camera video (for example, a digital still camera, digital video camera) composed. The imaging device 10 captures a subject, and records a captured image obtained by the imaging to (either still or moving good.) In a recording medium as image data of a digital format.
[0026]
　1, the imaging apparatus 10 according to this embodiment generally comprises an imaging unit 110, a signal processing unit 120, a display unit 130, a recording medium 140, a control unit 150, the operation unit It includes a 160, a geomagnetic sensor 170, an acceleration sensor 172, a.
[0027]
　Imaging unit 110 outputs an analog image signal by imaging an object. Imaging unit 110 includes an imaging optical system 111, an imaging element 112, a timing generator 113, an optical component driving unit 114.
[0028]
　The imaging optical system 111 is composed of a focusing lens, and various lenses such as a zoom lens, an optical filter that removes unnecessary wavelengths, an aperture such as an optical component. An optical image incident from a subject (subject image), via the optical components in the imaging optical system 111 is focused on the exposure surface of the image pickup device 112. The imaging device 112 (image sensor), for example, composed of a solid-state imaging device such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The image pickup element 112, an optical image guided by the imaging optical system 111 performs photoelectric conversion and outputs an electric signal (analog image signal) representing the captured image.
[0029]
　The imaging optical system 111, the optical component driving unit 114 for driving an optical component of the imaging optical system 111 is mechanically connected. The optical component driving unit 114, for example, a zoom motor, a focus motor, aperture adjustment mechanism, and the like, a zoom lens, or moving the focus lens, or adjusting the aperture. Optical component driving unit 114, in accordance with an instruction from the control unit 150, which will be described later, to drive the optical components of the imaging optical system 111. Moreover, TG (Timing Generator) 113 receives an instruction from the controller 150, generates an operation pulse necessary for the imaging element 112. For example, TG 113 is four-phase pulses for vertical transfer, field shift pulses, two-phase pulses for horizontal transfer, generates various pulses such as a shutter pulse, and supplies the image pickup device 112. The TG113 by by driving the imaging device 112, a subject image is imaged (electronic shutter function). Moreover, TG 113 is, by adjusting the shutter speed of the imaging element 112, the exposure of the captured image are controlled.
[0030]
　Image signal the image sensor 112 has output the above are input to the signal processing unit 120. The signal processing unit 120 executes the predetermined signal processing on the image signal output from the image sensor 112, and outputs the image signal after the signal process on the display unit 130 and the control unit 150. The signal processing unit 120 comprises analog signal processing unit 121, an analog / digital (A / D) converter 122, a digital signal processing unit 123.
[0031]
　The analog signal processing unit 121 is a so-called analog front end that preprocesses the image signal. The analog signal processing unit 121, for example, the image signal output from the imaging element 112, CDS (correlated double sampling: correlated double sampling) processing, and gain processing by the programmable-gain amplifier (PGA). A / D converter 122 converts the analog image signal inputted from the analog signal processing unit 121 into a digital image signal, and outputs the digital signal processing unit 123. Digital signal processor 123, the inputted digital image signal, for example, noise removal, white balance adjustment, color correction, edge enhancement, performs digital signal processing such as gamma correction, display unit 130 and the control unit 150 output to equal.
[0032]
　Display unit 130 is, for example, a liquid crystal display: comprised of (LCD Liquid Crystal Display), flat panel displays such as an organic EL display. Display unit 130 according to the control by the control unit 150 displays the image data of the inputted various kinds. For example, the display unit 130 displays a captured image (through image) input in real time from the signal processing unit 120 during imaging. Thus, the user, while viewing the through image being captured by the imaging device 10, it is possible to operate the imaging apparatus 10. Further, when playing a captured image recorded on the recording medium 140, the display unit 130 displays the reproduced image. Thus, the user can confirm the content of the captured image recorded on the recording medium 140.
[0033]
　Recording medium 140, data of the captured image, and stores various data such as meta data. Recording medium 140 is, for example, a semiconductor memory such as a memory card, or an optical disk, and a disk-shaped recording medium such as a hard disk can be used. Incidentally, the optical disc may include, for example, Blu-ray disc (Blu-ray Disc), DVD (Digital Versatile Disc) or CD (Compact Disc) or the like. The recording medium 140 may be installed internally in an imaging device 10, or may be a removable medium detachable to the imaging apparatus 10.
[0034]
　Control unit 150 is configured by a microcontroller, controls the overall operation of the imaging apparatus 10. Control unit 150 comprises, for example, CPU151, EEPROM152, ROM (Read Only Memory) 153, RAM (Random Access Memory) 154. Incidentally, EEPROM stands for "Electrically Erasable Programmable ROM".
[0035]
　The ROM153 in the control unit 150, a program for executing various control processes in the CPU151 is stored. CPU151 operates based on the program while using the RAM 154, executes necessary computation and control processing for the respective control. The program storage device incorporated in the image pickup device 10 (e.g., EEPROM 152, ROM 153, etc.) can be stored in advance in the. Further, the program, a disk-shaped recording medium, stored in a removable recording medium such as a memory card, may be provided to the imaging apparatus 10, LAN, are downloaded to the imaging apparatus 10 via a network such as the Internet it may be.
[0036]
　Here, a specific example of control by the control unit 150. Control unit 150 controls the TG113 and optical component driving unit 114 of the imaging unit 110, controls the imaging processing by the imaging unit 110. For example, the control unit 150, the adjustment of the aperture of the imaging optical system 111, the setting of the electronic shutter speed of the imaging device 112, such as by the gain setting of the AGC of the analog signal processing section 121 performs automatic exposure control (AE function). Further, the control unit 150 moves the focus lens of the imaging optical system 111, by changing the focus position, automatically performs match autofocus control the focus of the imaging optical system 111 for a particular subject (AF function). Further, the control unit 150 moves the zoom lens of the imaging optical system 111, by changing the zoom position, to adjust the angle of view of the captured image. Further, the control unit 150, the captured image to the recording medium 140, and records various data such as metadata, also reading and reproducing data recorded on the recording medium 140. Further, the control unit 150 generates various display images to be displayed on the display unit 130 to display the display image by controlling the display unit 130.
[0037]
　Operation unit 160, the display unit 130 functions as a user interface. Operation unit 160 is, for example, a button, is constituted by a lever various operation keys such as, or a touch panel or the like, and outputs the instruction information in response to the user operation to the control unit 150.
[0038]
　The geomagnetic sensor 170 and the acceleration sensor 172 constitute an electronic compass (azimuth sensor) for detecting an image pickup direction. The imaging direction is the horizontal direction of the imaging direction of when imaging an object by the imaging device 10. Imaging orientation, for example, can be represented by the reference direction (eg North) azimuth θ (θ = 0 ° ~ 360 °) relative to the. Further, the imaging direction is the optical axis direction of the imaging optical system 111. In a typical digital camera, the imaging direction is a positive versus direction of the image pickup apparatus 10 coincides with the back of the display screen of the display unit 130.
[0039]
　Geomagnetic sensor 170, for example, a two-axis geomagnetic sensor or three-axis geomagnetic sensor, detects the geomagnetism in a location where the image pickup apparatus 10 exists. Two-axis geomagnetic sensor detects the geomagnetism in the longitudinal and lateral direction of the image pickup device 10, the three-axis geomagnetic sensor detects the longitudinal direction, the lateral direction and the vertical direction terrestrial magnetism of the imaging apparatus 10. Geomagnetic sensor 170 outputs geomagnetic data representing the detected geomagnetism to the control unit 150.
[0040]
　The acceleration sensor 172 detects an acceleration acting on the imaging device 10. The acceleration sensor 172 is, for example, before and after the image pickup device 10, left and right is constituted by a triaxial acceleration sensor for detecting accelerations in the vertical direction, for detecting acceleration in three axial directions applied to the imaging apparatus 10. Acceleration sensor 172 outputs acceleration information indicating the acceleration of the detected three-axis control section 150. Control unit 150 calculates the detected value of the geomagnetic sensor 170 (the geomagnetic information), by using the detection value of the acceleration sensor 172 (acceleration information), and orientation of the imaging apparatus 10, the image pickup direction. The details of this calculation method will be described later.
[0041]
　[2. Functional configuration of the imaging apparatus]
　Next, with reference to FIG. 2, a functional configuration of a main part of an imaging apparatus 10 according to the present embodiment, its processing will be described. Figure 2 is a block diagram showing a functional configuration of the imaging apparatus 10 according to the present embodiment.
[0042]
　As shown in FIG. 2, the control unit 150 of the imaging device 10, an azimuth calculation section 200, an azimuth conversion unit 202, a compass image generation unit 204, an additional information management unit 206, a recording unit 208, the reproduction unit 210 equipped with a door. These function units, the CPU151 shown in FIG. 1, is realized by executing a program stored in the ROM153 etc. is not limited to such an example, it may be implemented by dedicated hardware.
[0043]
　[2.1. Calculation processing of the image pickup direction]
　First, the orientation calculation unit 200, describes the processing of calculating the image capturing direction of the imaging device 10. An orientation calculation unit 200, a geomagnetic sensor 170 and the acceleration sensor 172 as described above (direction sensor) constitutes an electronic compass for positioning the imaging direction. Orientation calculation unit 200, based on the detection values ​​of the geomagnetic sensor 170 and the acceleration sensor 172, and calculates the imaging direction.
[0044]
　As described above, the geomagnetic sensor 170 detects the geomagnetism in a location where the image pickup apparatus 10 exists, as a detection value, and outputs the geomagnetic information. Further, the acceleration sensor 172 detects acceleration in three axial directions applied to the imaging apparatus 10. By using the acceleration information detected by the acceleration sensor 172 can detect the posture of the imaging device 10 (e.g. static posture). That is, when the image pickup apparatus 10 is in a static posture, acceleration acting on the imaging device 10 is the gravitational acceleration of the Earth. Therefore, based on the three-axis direction acceleration information detected by the acceleration sensor 172, if calculate the direction of the gravitational acceleration applied to the imaging apparatus 10 in the three-dimensional space, kill detects the posture of the imaging device 10. Posture of the imaging device 10 is represented by the inclination of the imaging apparatus 10 relative to the ground (e.g., the roll direction, the pitch direction, the yaw direction rotation angles).
[0045]
　Referring now to FIG 3 will be described in detail posture of the imaging device 10. 3 is a perspective view showing the imaging direction and orientation of the imaging apparatus 10 according to the present embodiment.
[0046]
　The imaging device 10, for example, comprises a rectangular housing 100 having parallel top face 101 and bottom face 102. The front 103 of the housing 100, the imaging optical system 111 of the imaging unit 110 is provided on the back surface 104 of the housing 100, the display screen of the display unit 130 (not shown.) Is provided. Roll axis 105 is a rotary shaft extending longitudinally of the housing 100, the imaging apparatus 10 is rotated in the roll direction about the roll axis 105, tilted to the left and right with respect to the ground. Similarly, the pitch axis 106, a rotating shaft extending in the lateral direction of the housing 100, the imaging apparatus 10 is rotated in the pitch direction about the pitch axis 106, tilted back and forth relative to the ground. Also, yaw axis 107, a rotating shaft extending in the vertical direction of the housing 100, the imaging apparatus 10 is rotated in the yaw direction about the yaw axis 107, changing the imaging direction.
[0047]
　As described above, the posture of the imaging device 10, represented by the roll direction with respect to the imaging device 10 is ground, the rotation angle when rotated in the pitch direction, the yaw direction (the roll angle alpha, the pitch angle beta, the yaw angle gamma) be able to. Incidentally, the roll axis 105, an imaging direction and the same direction of the image pickup apparatus 10. Further, if the imaging apparatus 10 is rotated in the yaw direction, since the imaging device 10 is directly opposite the horizontal direction is changed, (horizontal direction of the image pickup direction) imaging direction is also changed.
[0048]
　Further, by the acceleration sensor 172, the above-mentioned roll direction of the imaging device 10, the pitch direction, knowing the rotation angle in the yaw direction (inclination angle with respect to the ground), by subtracting the rotation angle partial from the detection values ​​of the geomagnetic sensor 170, horizontal calculate the direction of the geomagnetism, it is possible to find the correct imaging orientation. Incidentally, one axis, also by using the acceleration sensor of the two-axis, it is possible to detect the rotation angle of one or two directions of the image pickup apparatus 10, it is possible to calculate the imaging direction, use the three-axis acceleration sensor If you had a, it is possible to more accurately calculate the imaging orientation.
[0049]
　Returning to Figure 2, Continuing with calculation processing of the image pickup direction by the orientation calculation unit 200. Orientation calculation unit 200, based on the acceleration information detected by the acceleration sensor 172, and calculates the posture of the imaging device 10 relative to the ground. Orientation of the imaging device 10, for example, represented by the rotation angle of the image pickup apparatus 10 described above (the roll angle alpha, the pitch angle beta, the yaw angle gamma). Furthermore, the orientation calculation unit 200 includes a geomagnetic sensor installation information 222 stored in advance, from the information of the posture of the imaging apparatus 10 calculated in the above, it calculates the orientation of the geomagnetic sensor 170. Here, the geomagnetic sensor installation information 222 is information representative of the (direction of the geomagnetic sensor 170 with respect to the imaging device 10) the installation arrangement of the geomagnetic sensor 170 installed in the imaging device 10. Installation arrangement of the geomagnetic sensor 170 is known at the time of manufacturing the image pickup apparatus 10. Orientation calculation unit 200, the installation arrangement of the geomagnetic sensor 170 (default rotation angle), by adding the posture of the imaging device 10 relative to the ground (the roll angle alpha, the pitch angle beta, the yaw angle gamma), geomagnetism with respect to the ground seek the attitude of the sensor 170.
[0050]
　Furthermore, the orientation calculation unit 200 calculates the geomagnetism information geomagnetic sensor 170 has positioning, from the information of the posture of the geomagnetic sensor 170 calculated at extracts a horizontal vector of the geomagnetism, the reference azimuth (for example north) . Also, the orientation calculation unit 200 includes an optical system installation information 224 stored in advance, from the information of the posture of the imaging device 10, which is calculated above, the horizontal direction of the optical axis of the imaging optical system 111 (i.e., the imaging direction) to calculate the vector. Here, the optical system installation information 224 is information indicating the installation position of the imaging optical system 111 is installed in the imaging device 10 (the direction of the optical axis of the imaging optical system 111 with respect to the imaging device 10). The optical system installation information 224 are also known at the time of manufacture of the imaging apparatus 10. Orientation calculation unit 200 determines a reference direction of the vector calculated in the above, from the difference between the horizontal vector of an image pickup direction, the imaging direction of the horizontal direction (i.e., the imaging direction). For example, the orientation calculation unit 200 as the image pickup direction, obtaining a reference azimuth (e.g. north) azimuth θ (θ = 0 ° ~ 360 °) relative to the.
[0051]
　The processing of the above orientation calculation unit 200, the imaging direction can be accurately detected. Incidentally, for the user to take vertical, the image pickup device 10 even when rotated 90 ° in the roll direction (see FIG. 4 below), the orientation calculation unit 200, since the calculated horizontal vector of an image pickup direction, it is possible to calculate the correct imaging orientation.
[0052]
　[2.2. Display processing compass image]
　Next, the 2, with reference to FIGS. 3 to 8, the orientation converting unit 202, the compass image generation unit 204 and the display unit 130, the orientation of the display which is converted from the image bearing the process of displaying the compass image representing explained.
[0053]
　First, the direction converting unit 202 shown in FIG. 2, if necessary, the image capturing orientation calculated by the orientation calculation unit 200, and converts the azimuth for display. Here, the orientation of the display, a direction for displaying information representing an imaging direction (e.g. compass image 302 in FIG. 6) on the display unit 130. Compass image 302 displayed on the display unit 130, pointing toward the direction for the display in a predetermined reference direction (the direction of the top surface 101 of the imaging device 10). If the imaging device 10 is rotating in the roll direction, the orientation for the display becomes different from the actual imaging direction.
[0054]
　About the need for orientation for this display it will be explained. The user facing the north, the imaging device 10 as shown in FIG. 3 laterally have the imaging an object (i.e., horizontal imaging) from to that state, about the roll axis 105, as shown in FIG. 4 the image pickup device 10 is rotated in the roll direction, as shown in FIG. 5, the dimensional worlds imaging apparatus 10 in the longitudinal direction (roll angle α = 90 °), consider a case of imaging a same object (i.e., portrait) . In this case, the imaging apparatus 10 is not rotating in the yaw direction, since the change is not in the orientation (north) facing the user, the imaging direction of the imaging device 10 (N) are also invariant. Therefore, as shown in FIGS. 6 to 8, on the captured image 300 (through image) displayed on the display unit 130 of the imaging apparatus 10, when superimposing the compass image 302 representing the image capturing orientation of the imaging apparatus 10 regardless of the change in the posture (rotation in the roll direction), that the compass image 302 on the display screen is always oriented a certain direction with respect to the ground surface, it is natural when viewed from the user's point of view. Thus in order to display the compass image 302, an image pickup direction calculated by the direction calculating unit 200 (N), the rotation angle in the roll direction (the roll angle α = 90 °) amount corresponding to the correction, display may be obtained of the orientation of the (east).
[0055]
　Therefore, the direction converting unit 202, based on the detected value of the acceleration sensor 172, and calculates the rotation angle in the roll direction of the imaging apparatus 10 (the roll angle alpha), on the basis of the roll angle alpha, by the direction conversion unit 202 the calculated imaging direction is corrected, obtaining the orientation of the display.
[0056]
　Specifically, first, the direction converting unit 202, calculates the acceleration information by the acceleration sensor 172 detects, as described above, the posture of the imaging device 10 relative to the ground (the roll angle alpha, the pitch angle beta, the yaw angle gamma) of to. Then, the direction converting unit 202, the orientation calculation unit 200, the rotation and display unit installation information 226 stored in advance, from the information of the posture of the imaging apparatus 10 calculated in the above, in the roll direction with the image pickup apparatus 10 to calculate the angle of rotation of the display unit 130 (roll angle α). Here, the display unit installation information 226 is information representing the (orientation of the display unit 130 with respect to the imaging device 10) the installation arrangement of the display unit 130 which is installed in the imaging device 10. Installation arrangement of the display unit 130 is known at the time of manufacturing the image pickup apparatus 10. Direction conversion unit 202, the installation arrangement of the display section 130 (default rotation angle), by adding the posture of the imaging device 10 relative to the ground (the roll angle alpha, the pitch angle beta, the yaw angle gamma), the display unit 130 determine the roll angle of α.
[0057]
　Furthermore, the orientation converting unit 202, an image pickup direction calculated by the direction converting unit 202 corrects an amount corresponding to the roll angle α of the display unit 130, obtains the azimuth of the display. For example, the imaging orientation, as represented by the azimuthal angle with respect to the reference azimuth θ (θ = 0 ° ~ 360 °), the orientation converter 202, the roll angle of the display unit 130 from the azimuth angle .theta imaging direction a by subtracting (or adding) α, to calculate the azimuth angle φ of the orientation of the display (φ = θ ± α). Incidentally, when the imaging apparatus 10 is not rotated in the roll direction, since it is α = 0 °, the azimuth angle .phi orientation for display, equal to the azimuth angle .theta imaging azimuth (φ = θ).
[0058]
　Then, the direction converting unit 202, information representing the orientation of the display which is calculated as described above (for example, the value of the azimuth angle .phi), passed to the compass image generation unit 204.
[0059]
　Compass image generation unit 204, based on the information representing the orientation of the display which is calculated by the direction converting unit 202, and generates a compass image 302 to be displayed on the display unit 130. For example, in the case of obtaining the azimuth angle .phi as φ = θ-α is the compass image generating unit 204 refers to the reference azimuth (e.g. north) needle compass towards the orientation (azimuth angle .phi) for the display to generate a compass image 302 as. On the other hand, in the case of obtaining the azimuth angle .phi as φ = θ + α is, the compass image generation unit 204 generates a compass image 302 as orientation for the display (azimuth angle .phi) is upward on the display screen. Then, the display unit 130 based on an instruction from the control unit 150, on the captured image 300 input from the imaging unit 110, and displayed superimposed the compass image 302.
[0060]
　By the above display processing, as shown in FIGS. 6 to 8, on the captured image 300 in imaging by the imaging unit 110, the compass image showing the orientation of the display which is corrected by the orientation converting unit 202 (the azimuth angle .phi) 302 is displayed. Thus, the user, while viewing the image capturing direction of the captured image 300, can be imaged. Also, 6, 7, 8, respectively, the roll angle α is 0 ° in the display unit 130 of the imaging device 10, 30 °, shows a display state when it is 90 °. In either case, the compass image 302 indicating the shooting direction, when viewed from the user's point of view, as the north, which is an imaging direction calculated by the direction calculating section 200 (azimuth angle .theta) becomes upward to the ground surface Is displayed.
[0061]
　That is, even if the user, as shown in FIGS. 3 to 5 has rotated 90 ° the image pickup device 10 in the roll direction, the compass image 302 as shown in FIGS. 6-8, for the display unit 130 relatively rotate, but does not rotate with respect to the ground, the needle of the compass image 302 is always refers to the constant direction. Thus, the compass image 302, since it does not rotate following the rotation of the roll direction of the display unit 130, as long as the image capturing direction does not change, when viewed from the perspective of the user, the orientation indicated by the compass image 302 on the display screen It does not change. Therefore, when the user is capturing an object of certain orientations, when changing the posture of the imaging device 10 (e.g., when changing the vertical shooting from intercepted) even when viewed from the user's point of view , you can display a compass image 302 in a natural display mode.
[0062]
　[2.3. Recording processing of the image pickup direction]
　Next, with reference to FIG. 2 again, the additional information management unit 206 and the recording unit 208, together with the captured image, and records the captured orientation was calculated as described above as additional information of the captured image processing It will be described. In the following primarily the user presses the shutter button 161 of the imaging apparatus 10 (release operation), will be described processing for recording still images captured image (picture) on the recording medium 140.
[0063]
　As shown in FIG. 2, when recording on the recording medium 140 a captured image in accordance with the release operation, the orientation calculation unit 200, an imaging direction information indicating the calculated imaging azimuth (azimuth angle .theta) as described above, the additional and outputs it to the information management unit 206.
[0064]
　Additional information management unit 206 has a function of managing the additional information of the captured image to be recorded on the recording medium 140 (e.g. Exif information). This additional information will generally comprise various types of information regarding the captured image (e.g., image size, file format, compression, etc. encoding format) and an imaging date and time information, and a thumbnail image of the recorded image. Additional information management unit 206, in addition to these general information, including an imaging direction information obtained by the orientation calculation unit 200, the orientation information of the image pickup apparatus 10 to the additional information. Orientation information of the latter imaging device 10, for example, the posture of the imaging device 10 when (time release) for recording the captured image (e.g., horizontal imaging, taking right rotation, such as taking a left rotation) is information representing a. The attitude information can be calculated from the detected value of the acceleration sensor 172 by the orientation calculation unit 200 as described above. Additional information management unit 206 outputs the additional information of these captured image to the recording unit 208.
[0065]
　Recording unit 208, in accordance with the release operation, the additional information of the captured image obtained from the additional information management unit 206, a captured image obtained from the imaging unit 110, compression and coding, the recording medium 140 in association with each other Record. Thus, the imaging orientation information (for example, the azimuth angle θ) and, and orientation information of the image sensing device 10 (for example, next to take, take the right rotation, such as taking the left turn), and as additional information of the captured image, associated with the captured image It can be recorded on. This information is useful for reproduction and display of the captured image.
[0066]
　As described above, the recording unit 208, rather than the above direction converting unit 202 orientation for display corrected by the (azimuth angle .phi), an imaging direction calculated by the direction calculating section 200 (azimuth angle .theta), the captured image to the record as additional information. Thus, regardless of the posture of the imaging device 10 at the time of imaging (roll angle alpha), correct imaging azimuth (azimuth angle .theta) when capturing the imaged image by the image pickup apparatus 10 can be recorded.
[0067]
　In the above has been described recording processing of a still image, even in the recording process of video, periodically or at any time during video recording, in association with the videos, the imaging direction information and attitude information as additional information for moving picture recording it may be recorded in the medium 140.
[0068]
　[2.4. Reproduction processing of the image pickup direction]
　Next, with reference to FIG. 9, the reproducing unit 210 and the display unit 130 shown in FIG. 2, to reproduce the additional information recorded captured image on the recording medium 140, the display unit 130 a description will be given of a process that you want to view. Figure 9 is an explanatory diagram showing a reproduced image 304 obtained by reproducing a captured image 300 (see FIG. 8.) When taken longitudinally according to the present embodiment.
[0069]
　In accordance with the reproduction operation by the user, the reproduction unit 210 reads the captured image 300 and the additional information recorded in the recording medium 140, the reproduction (expansion and decoding) is. Then, the display unit 130, as shown in FIG. 9, the reproduction image 304 which has been reproduced by the reproducing unit 210 displays the compass image 306 indicating the shooting direction of the reproduction image 304.
[0070]
　In the reproduction and display processing, the reproduction unit 210, based on the orientation information of the image pickup apparatus 10 included in the additional information, by rotating the reproduction image 304 as necessary, is displayed on the display unit 130. For example, the reproduction unit 210, when reproducing the captured image 300 recorded on the recording medium 140, based on the posture information added to the image pickup image 300, or the imaging image 300 is portrait image and whether or not, to determine the direction of rotation of the time vertical shooting. If a portrait image, the playback unit 210, vertical direction is 90 ° rotated clockwise or counterclockwise a reproduced image 304 to be correctly displayed on the display unit 130.
[0071]
　Further, the reproduction unit 210, based on the imaging direction information added to the image pickup image 300, determines the imaging direction of when imaged the imaging image 300, information representing the imaging direction of the imaging image 300 (e.g. the azimuth angle θ), passed to the compass image generation unit 204. Then, the compass image generation unit 204, based on the information representing the imaging direction, and generates a compass image 306 to be displayed on the display unit 130, and outputs to the display unit 130. As a result, as shown in FIG. 9, the display unit 130 together with the reproduced image 304 obtained from the reproduction unit 210, and displays the compass image 306 obtained from the compass image generation unit 204. The reproduction image 304 of FIG. 9 is an image obtained by reproducing a captured image 300 when the vertical shooting in the state shown in FIG.
[0072]
　At this time, the compass image 306 to be displayed together with the reproduced image 304, it points to the actual imaging direction (azimuth angle θ = 0 °) when capturing the captured image 300. This is because, when the recording of the imaging azimuth information, rather than the orientation (azimuth angle φ = -90 °) for display which has been corrected by the orientation converting unit 202, an imaging azimuth (azimuth angle calculated by the orientation calculation unit 200 is because a record of θ = 0 °). If, when the recorded orientation (azimuth angle φ = -90 °) for display, the playback screen of FIG. 9, the compass image 306 points to the west (corresponding to the azimuth angle φ = -90 °.) It will be. However, in the present embodiment, since the orientation calculation unit 200 is recording the calculated imaging azimuth (azimuth angle θ = 0 °) at the time of imaging, compass image 306 to be displayed together with the reproduced image 304, the regeneration image 304 it is possible to point to the correct imaging orientation (azimuth angle θ = 0 °).
[0073]
　Further, when displaying the reproduced image 304, depending on the posture of the imaging device 10 at the time of imaging (portrait or horizontal imaging), automatically rotate the reproduced image 304 as the vertical direction of the reproduced image 304 is vertically display on. In the example of FIG. 9, portrait reproduction image 304 is displayed rotated 90 ° to the horizontally long display screen of the display unit 130. Thus, for example, even when reproducing the portrait captured image 300, without rotating the display unit 130, the reproduction image 304 representing the imaging content, displaying a compass image 306 showing the image pickup direction at the right direction it is (see FIG. 9.).
[0074]
　As described above, the imaging apparatus 10 according to the present embodiment includes an imaging direction calculated by the direction calculating section 200 (azimuth angle .theta), for display that has been corrected by the orientation converting unit 202 azimuth (azimuth angle φ the) and are properly used depending on the application. For example, at the time of imaging, by converting the azimuth for display-image-capture direction (azimuth .theta) (azimuth angle .phi), even when the posture of the imaging device 10 is changed in the roll direction, natural viewed from the user's perspective in a manner, it is possible to display a compass image 302 showing an imaging orientation of the captured image 300. On the other hand, when recording a captured image 300, as additional information of the captured image 300, and records the correct imaging direction before the correction (azimuth .theta). Therefore, when reproduction of a captured image 300, together with the reproduced image 304, it is possible to display the compass image 306 indicating the shooting direction of the captured image 300 (the azimuth angle .theta) in the correct orientation. Further, copying the captured image 300 on the personal computer (PC), a in the case of confirming the orientation in the application on the PC also, since the imaging direction (azimuth angle .theta) is recorded as the additional information, the above-mentioned application, it is possible to recognize the image capturing direction of the captured image 300.
[0075]
　[3. Second Embodiment
　Next, an imaging apparatus 10 according to the second embodiment will be described azimuth processing method according to the imaging apparatus 10. The second embodiment differs from the first embodiment, the process differs when the imaging direction is the vertical direction, the other functional configuration, since the first embodiment substantially identical , and the detailed description thereof is omitted.
[0076]
　The photographer is, when imaging or subject in his or her right above, an object directly below, so directs the imaging device 10 directly above, beneath, the imaging direction (the direction of the optical axis of the imaging optical system 111), a substantially vertical the direction (approximately vertical upward or substantially vertically downward). In this case, as the imaging direction and thus calculating the orientation of the imaging direction as in the first embodiment, even when the imaging direction is changed slightly, as the imaging orientation would shake greatly reacts sensitively problems there is. For example, when imaging towards beneath the imaging device 10, the imaging direction is become vertically downward, only slightly tilt the imaging device 10 in the pitch direction or yaw direction in this state, the imaging orientation swing in north, south, east, and west and will, thus compass image 302 is rapidly changed on the display screen.
[0077]
　Therefore, in the second embodiment, when the imaging direction is substantially vertical direction, the orientation of the direction perpendicular to the imaging direction (e.g., direction of the top surface 101 of the image pickup apparatus 10, or the direction of the bottom surface 102) the request, to display and record the orientation of the vertical direction. Thus, even the imaging direction is changed in the vicinity of the vertical direction, the bearing displayed and recorded, since the stabilized without fluctuation increases, so the imaging direction can be appropriately presented to the photographer.
[0078]
　Here, the "substantially vertical direction" in this specification is a substantially vertical direction, not only (the direction perpendicular to the horizontal plane) only exact vertical direction, a predetermined angle δ with respect to the vertical direction (e.g. also it includes a direction having a 0 <δ ≦ 10 °). Imaging person, when the imaging direction is recognized as the directly above or directly below the imaging direction, without an exact vertical direction, said to be "substantially vertical direction". Similarly, "substantially vertically downward" it is not strictly vertically downward only, also includes a direction having a predetermined angle δ with respect to the vertical downward direction, "substantially vertically above" includes not strictly vertically upward only, vertically upwardly also it includes a direction having a predetermined angle δ against.
[0079]
　[3.1. Functional configuration of the imaging apparatus]
　Next, with reference to FIG. 10, a functional configuration of a main part of an imaging apparatus 10 according to the second embodiment, its processing will be described. Figure 10 is a block diagram showing a functional configuration of the imaging apparatus 10 according to the second embodiment.
[0080]
　As shown in FIG. 10, the image pickup apparatus 10 according to the second embodiment, in addition to the function according to the first embodiment has the following functions. Orientation calculation unit 200, based on the acceleration information from the acceleration sensor 172, and calculates the rotation angle in the pitch direction of the image pickup device 10 (the pitch angle beta), on the basis of the pitch angle beta, the imaging direction is substantially vertical direction ( detecting whether substantially vertically above or substantially vertically downward).
[0081]
　For example, the pitch angle beta that is calculated, if the angle of pitch angle (90 °) or in the vicinity thereof representing the vertically upward [(90 ° -δ) <β <(90 ° + δ)], the orientation calculation unit 200, It determines that the imaging direction is substantially vertically upward. Further, when the calculated pitch angle beta is an angle of pitch angle (-90 °) or in the vicinity thereof representing a vertically downward [(-90 ° -δ) <β <(- 90 ° + δ)], the orientation calculation unit 200, it is determined that the imaging direction is substantially vertically downward. Thus, we obtain the pitch angle beta of the image pickup device 10 from the detected value of the acceleration sensor 172, by using the pitch angle beta, to determine whether or not the imaging direction is substantially vertical upward or downward.
[0082]
　Result of the determination, when the imaging direction is not a substantially vertical direction, as in the first embodiment described above, the orientation calculation unit 200 calculates the imaging direction of the orientation (image pickup direction), orientation converter 202 , it is converted to the orientation of the display for the imaging orientation. Display processing and subsequent compass image 302, the recording and reproduction processing of the image bearing information is also the same as in the first embodiment.
[0083]
　On the other hand, when the imaging direction is substantially vertical direction, the orientation calculation unit 200, instead of the imaging direction, and calculates the orientation of the direction perpendicular to the imaging direction. Here, a direction perpendicular to the imaging direction, for example, a bottom surface direction of the top face direction, the imaging device 10 of the imaging apparatus 10. Here, the top surface direction of the image pickup apparatus 10, as shown in FIG. 11, a top surface 101 directly opposite direction of the image pickup apparatus 10, the bottom surface direction of the image pickup apparatus 10, as shown in FIG. 12, the imaging device the bottom 102 of 10 is positive against direction. Orientation calculation unit 200, when the imaging direction is substantially vertically downward, the top surface direction of orientation of the imaging device 10 (hereinafter, top surface orientation) is calculated, if the imaging direction is substantially vertical upward, bottom direction of orientation (below, bottom orientation) is calculated. Incidentally, as the direction perpendicular to the imaging direction, in addition to the above top surface direction or bottom direction, or the like may be used a direction either right or left side of the image pickup device 10 is directly opposite.
[0084]
　Thus, when the imaging direction is substantially vertical direction, the orientation calculation unit 200, instead, the top face orientation or bottom orientation (azimuth angle of the image pickup direction (azimuth angle .theta) according to the first embodiment to calculate the ψ). The top face orientation or bottom orientation is the orientation of a direction perpendicular to the imaging direction. Therefore, the imaging device 10 facing directly above or directly below is slightly rotated in the roll direction or yaw direction, when the imaging direction is somewhat changed, imaging orientation (azimuth angle θ) is swing big, heaven plane orientation or bottom orientation (azimuth angle ψ) there is no thing that swing big. Therefore, when imaging an object located in a substantially vertically upward or substantially vertically below the imaging device 10, when calculating the top face orientation or bottom orientation (azimuth angle .psi), it is possible to obtain a stable orientation.
[0085]
　Then, the orientation calculation unit 200, the calculated top face orientation or information representing the bottom surface orientation, without the intervention of the directional converter 202 directly outputs the compass image generation unit 204. Then, as shown in FIG. 13, the compass image generation unit 204 generates a compass image 312 representing the top face orientation or bottom orientation calculated by the orientation calculation unit 200 (azimuth angle .psi), the display unit 130, the compass image 312 is superimposed on the captured image 310. Thus, the display unit 130 according to whether the second embodiment, instead of the compass image 302 representing the orientation of the display as in the first embodiment (azimuth angle .phi), the top face orientation or bottom orientation a compass image 312 representing the (azimuth angle ψ), is displayed together with the captured image 310 in imaging.
[0086]
　Further, even when recording a captured image 310, the recording unit 208 as additional information of the captured image 310, instead of the information representing the imaging direction (azimuth angle .theta) according to the first embodiment, the top surface orientation or orientation information representing the bottom surface orientation (azimuth angle .psi), recorded on the recording medium 140 in association with the captured image 310. Then, when the reproducing and displaying a captured image 310 recorded on the recording medium 140, first, the reproduction unit 210 reproduces the captured image 310 recorded on the recording medium 140, the compass image generation unit 204, the recording based on the direction information recorded in the medium 140, and generates a compass image 316 representing the top face orientation or bottom orientation (azimuth .psi). As a result, as shown in FIG. 14, the display unit 130, the compass image 316 representing the top face orientation or bottom orientation (azimuth angle .psi), and displayed together with the reproduced image 314.
[0087]
　Incidentally, the imaging apparatus 10 according to the first embodiment, at the time of reproduction, based on the recorded position information as additional information of the captured image 300, taken roll direction orientation (next to the image pickup apparatus 10 at the time of imaging, right rotation taken, to determine the left rotation takes etc.), it was automatically display rotated 90 ° the reproduced image 304 in accordance with the roll direction of orientation (see FIG. 9.). Imaging device 10 a captured image 300 taken substantially toward the horizontal direction, the directionality to be a display reference (e.g., vertical direction is displayed. As a vertically) since they have, as in FIG. 9 the convenience of the user is enhanced by Do rotation display processing.
[0088]
　In contrast, the imaging apparatus 10 according to the second embodiment, as direction information at the time of imaging, when the orientation information of the imaging direction is recorded, at the time of reproduction, as in the first embodiment, attitude determination and It performs rotation display processing. However, since the imaging direction is directly below (or directly above), as orientation information at the time of imaging, when the information representing the top face orientation (or bottom orientation) has been recorded, the imaging device 10, a first embodiment not performed posture determination and a rotation display process such as in the form. The captured image 310 captured toward the substantially vertical direction an imaging apparatus 10 has no directionality as a display reference. Therefore, the rotation display process or the like are the unnecessary for the user, it is preferable to omit.
[0089]
　[3.2. Calculation processing of the image pickup direction]
　Next, with reference to FIG. 11, when the image pickup toward right below the image pickup apparatus 10 will be described in detail a process of calculating the top surface orientation.
[0090]
　As shown in FIG. 11, the user, in order to image the feet of a subject, when directed beneath the imaging device 10, the imaging direction (optical axis direction of the imaging optical system 111) is substantially vertically downward. Orientation calculation unit 200 can detect that the imaging direction (optical axis direction of the imaging optical system 111) is substantially vertically downward.
[0091]
　Specifically, first, the orientation calculation unit 200, based on the acceleration information detected by the acceleration sensor 172, and calculates the posture of the imaging device 10 (roll angle alpha, the pitch angle beta, the yaw angle gamma) with respect to the ground. Then, the orientation calculation unit 200 outputs the calculated pitch angle β is, the upper limit value (-90 ° -δ) larger than, if it is less than the lower limit (-90 ° + δ), the imaging direction is a substantially vertically downward judge.
[0092]
　If it is detected that the imaging direction is substantially vertically downward, orientation calculation unit 200, a geomagnetic information of the geomagnetic sensor 170, based on the acceleration information from the acceleration sensor 172, and calculates the top surface orientation of the imaging apparatus 10.
[0093]
　Specifically, the orientation calculation unit 200, the orientation calculation unit 200 includes a geomagnetic sensor installation information 222 described above, the information of the posture of the imaging apparatus 10 calculated in the above, it calculates the orientation of the geomagnetic sensor 170. Furthermore, the orientation calculation unit 200 calculates the geomagnetism information geomagnetic sensor 170 has positioning, from the information of the posture of the geomagnetic sensor 170 calculated at extracts a horizontal vector of the geomagnetism, the reference azimuth (for example north) . Then, the orientation calculation unit 200 includes an optical system installation information 224 described above, the information of the posture of the imaging device 10 previously calculated, it calculates the imaging direction of the vector. Furthermore, the orientation calculation unit 200 calculates the vector of the upward 90 ° in the pitch direction with respect to the imaging direction of the vector (i.e., the vector of the top surface direction). Then, the orientation calculation unit 200 includes a reference orientation of the vector calculated in the above, from the difference between the top surface direction of the vector, obtains the top surface direction of the horizontal direction (i.e., top surface orientation). For example, the orientation calculation unit 200, as the top surface orientation, determine a reference azimuth (e.g. north) azimuth ψ (ψ = 0 ° ~ 360 °) relative to the.
[0094]
　[3.3. Display processing compass image]
　Next, with reference to FIGS. 13 to 16, when the image pickup toward right below the image pickup apparatus 10 will be described in detail a process of displaying a compass image 312, 316 representing the top surface orientation .
[0095]
　As described above, the orientation calculation unit 200 calculates the top surface orientation (azimuth angle .psi), and outputs the information representing the top panel orientation, not via the orientation converting unit 202 to the compass image generation unit 204. Then, the compass image generation unit 204 generates a compass image 312 representing the top panel surface orientation (azimuth .psi), the display unit 130 superimposes displays the compass image 312 on the captured image 310.
[0096]
　Referring now to FIGS. 13 and 15, a description will be given of a display mode of the compass image 312 at the time of imaging. For example, in the example of FIG. 13, since the top surface 101 of the imaging device 10 that are directed beneath is pointing north, the top surface orientation is North (ψ = 0 °), the compass image 312 refers to the north. In the example of FIG. 15, since the top surface 101 of the imaging device 10 that are directed beneath is facing northeast, top surface orientation is east (ψ = 90 °), the compass image 312 refers to the east.
[0097]
　FIG 13 As can be seen from the relationship between FIG. 15, an imaging device 10 facing directly below (see FIG. 11.), By the amount of rotated horizontally (rotating about the roll axis 105) relative to the ground, changing the orientation indicated by compass image 312 on the display screen (Northern → E) Accordingly, as shown in FIGS. 13 and 15, when the user points to the right while holding the imaging apparatus 10 toward the right under the heading similar to when rotating clockwise at a magnet on the ground, so as not to change the direction of the needle of the compass, it is possible to display the compass image 312. In general, the user when directed beneath and Yokoji the imaging device 10, the top surface direction of the direction and the imaging device 10 facing the user matches. Therefore, by displaying the compass image 312 as described above Figure 13 and Figure 15, viewed from the perspective of the user, it displays the direction in a natural display mode.
[0098]
　Next, with reference to FIGS. 14 and 16, a description will be given of a display mode of the compass image 316 at the time of image reproduction. Figure 14 is a display screen when playing a captured image 310 recorded in the state of FIG. 13, FIG. 16 is a display screen when playing a captured image 310 recorded in the state of FIG. 15.
[0099]
　In the second embodiment, unlike the first embodiment, as a direction for recording and orientation of the display, and using the same orientation (top surface orientation), of the top surface orientation information of the captured image 310 It is recorded as additional information. Therefore, at the time of shooting at the time of reproduction, it can be displayed the same of the captured image and compass image.
[0100]
　For example, the reproduction image 314 and the compass image 316 shown in FIG. 14 is displayed similarly to the display unit 130 and the captured image 310 and the compass image 312 at the time of imaging as shown in FIG. 13, the compass image 316 at the time of reproduction shows a northern ing. Further, the reproduced image 314 and the compass image 316 shown in FIG. 16 also, is displayed in the same way the display unit 130 and the captured image 310 and the compass image 312 at the time of imaging as shown in FIG. 15, the compass image 316 at the time of reproduction indicates the east ing. However, as can be seen from the relationship of FIG. 15 and FIG. 16, when reproducing a captured image 310 captured toward the substantially vertical direction an imaging device 10, rotation processing rows of the reproduced image 304 as shown in FIG. 9 not crack.
[0101]
　Incidentally, when the image pickup toward the image pickup device 10 just above the bearing calculation unit 200 calculates a bottom direction of the image pickup apparatus 10, the display unit 130, a compass image representing the bottom face orientation during the imaging and reproduction indicate. Such calculation and display processing, since toward just above the imaging device 10 described above is similar to that captured, detailed description thereof will be omitted.
[0102]
　[3.4. Orientation information processing method]
　Next, with reference to FIG. 17, the azimuth processing method will be described by the imaging apparatus 10 according to the second embodiment. Figure 17 is a flowchart showing an azimuth processing method according to the imaging apparatus 10 according to the second embodiment.
[0103]
　As shown in FIG. 17, first, the imaging device 10, the geomagnetic sensor 170 detects the location of the example of the three-axis direction terrestrial magnetism which the imaging apparatus 10 exists (S100). Further, the imaging device 10, the acceleration sensor 172 detects the gravitational acceleration, for example three axes acting on the imaging device 10 (S102). Then, the imaging device 10 based on the acceleration information detected by the acceleration sensor 172, the posture of the imaging device 10 (roll angle alpha, the pitch angle beta, the yaw angle gamma) for detecting a (S104).
[0104]
　Then, the imaging device 10 based on the posture information of the imaging device 10 obtained at S104, detects the imaging direction of the imaging unit 110, the imaging direction is substantially vertically downward, substantially vertically upward, or the symbolic vertically downward and to determine which one of the directions other than substantially vertically upward (S106). In particular, the imaging apparatus 10 calculates the attitude information of the image pickup apparatus 10, from the optical system installation information 224. The optical axis of the imaging optical system 111 (i.e., the imaging direction). Then, the imaging device 10 based on the pitch angle β of the posture information, it is determined whether or not the imaging direction is the substantially vertical direction. Further, when the imaging direction is the substantially vertical direction, the imaging device 10 based on the pitch angle beta, the imaging direction is determined whether it is substantially vertically downward or substantially vertically above the. Pitch angle beta is within the first angle range set in advance (for example, 90 ° -δ <β <90 ° + δ), it is judged that the imaging direction is substantially vertically upward. The pitch angle beta is within the second angle range set in advance (for example, -90 ° -δ <β <-90 ° + δ), it is judged that the imaging direction is substantially vertically downward. Further, the pitch angle β is, if the outside of the first and second angle range, it is determined that there is not a substantially vertical direction the imaging direction.
[0105]
　If the imaging direction is substantially vertically downward, the imaging apparatus 10 includes a geomagnetic information obtained in S 100, based on the orientation information of the image pickup apparatus 10 obtained in S104, in the top surface direction of the image pickup apparatus 10 orientation ( In other words, to calculate the top surface orientation) (S108). Then, the imaging device 10, the top surface orientation (azimuth angle .psi), used for recording and display of orientation information (S112). For example, the imaging device 10, the compass image 312 representing the top surface orientation, the display unit 130 together with captured image 310 (through image) (see FIG. 13.). Further, the imaging device 10, when the recording of the captured image 310, the direction information indicative of the top surface orientation is recorded as the additional information of the captured image 310. Further, the imaging device 10, when the reproduction of the captured image 310, on the basis of the orientation information, the compass image 316 representing the top surface orientation, the display unit 130 together with the reproduced image 314 (see FIG. 14.).
[0106]
　On the other hand, when the imaging direction is substantially vertically upward, the imaging apparatus 10 includes a geomagnetic information obtained in S 100, based on the orientation information of the image pickup apparatus 10 obtained in S104, the azimuth of the bottom surface direction of the image pickup apparatus 10 (ie, bottom orientation) is calculated (S110). Then, the imaging device 10, the bottom surface orientation, used for recording and display of orientation information (S112). Specific examples of this use are the same as in the top face direction, a detailed description thereof is omitted.
[0107]
　Further, if the imaging direction is not a substantially vertical direction, the image pickup apparatus 10 includes a geomagnetic information obtained in S 100, based on the orientation information of the image pickup apparatus 10 obtained in S104, the imaging direction of orientation (i.e., the imaging direction ) is calculated (S114). The imaging orientation (azimuth angle θ) is used to record the orientation information (S122). For example, the imaging device 10, when the recording of the captured image 300, the direction information indicative of an imaging direction, is recorded as additional information of the captured image 300.
[0108]
　Then, the imaging apparatus 10 uses the orientation information of the image pickup apparatus 10 obtained above S104, obtains the rotation angle in the roll direction of the display unit 130 (roll angle α) (S116). Further, the imaging device 10 based on the roll angle α obtained in S 116, the image pickup direction calculated in S 114 (azimuth angle .theta), and converts the azimuth (azimuth angle .phi) for display (S 118). For example, by subtracting also added to the roll angle α on Taii the azimuth angle θ of the imaging orientation, the azimuth angle φ is the calculation of the orientation of the display (φ = θ ± α).
[0109]
　Orientation for the display (azimuth angle .phi) is used for displaying the azimuth information (S 120). For example, the imaging device 10, the compass image 302 representing the orientation (azimuth angle .phi) for display on the display unit 130 together with captured image 300 (through image) (see FIGS. 6-8.).
[0110]
　As described above, in the azimuth processing method according to the second embodiment, depending on the posture of the imaging device 10, selectively using the orientation to be displayed and recorded. For example, when the imaging direction is substantially vertical direction, obtains a top face orientation or bottom orientation (azimuth angle .psi), the top panel face orientation or bottom azimuth (azimuth angle .psi) to both the display and recording of the direction information use to. On the other hand, when the imaging direction is not a substantially vertical direction, along with determining the imaging direction (azimuth angle .theta), by correcting the captured orientation (azimuth .theta) by the roll angle α min, orientation (azimuth angle .phi) for display also seek. And, while the use of imaging orientation (azimuth angle θ) to record the orientation information, to use the orientation of the display (the azimuth angle φ) to the display of orientation information.
[0111]
　[4. CONCLUSION
　The foregoing has described the imaging apparatus 10 and the azimuth processing method according to first and second embodiments of the present invention. According to the embodiment, the imaging device 10 according to the imaging direction representing the actual imaging direction to calculate the (azimuth angle .theta), (a rotation angle in the roll direction of the display unit 130. alpha) change in the posture of the imaging device 10 by correcting the imaging orientation (azimuth angle θ) on, also to calculate the orientation of the display (azimuth angle φ). Then, the imaging device 10, as a direction for recording together with the captured image, while using the imaging azimuth (azimuth angle .theta), as the orientation of the compass image 302 is visually recognized by the user is displayed on the display unit 130, the orientation ( to use the azimuth angle φ).
[0112]
　Thus, during shooting, even when vertical shooting user using the imaging apparatus 10, as viewed from the user point of view, can be displayed compass image 302 with the correct orientation. Further, the compass image 302 does not rotate following the rotation of the roll direction of the display unit 130, as viewed from the user point of view, always refers to the fixed direction. Therefore, irrespective of the rotation of the roll direction, as viewed from the user's point of view, it can be displayed compass image 302 in a natural display mode. Moreover, at the time of recording of the captured image 300, the right imaging direction representing the actual imaging direction can be recorded as additional information of the captured image 300. Therefore, at the time of image reproduction, along with the reproduced image 304, it can be presented to the correct imaging orientation to the user.
[0113]
　Further, according to the second embodiment, according to the imaging direction of the imaging device 10, an imaging direction, selectively using top surface orientation and a bottom surface orientation, only when photographing direction is not substantially vertical direction, display the above image pickup direction to use the orientation of use. On the other hand, when the imaging direction is substantially vertically downward or substantially vertically information, using the top face orientation or bottom orientation, also the display unit 130 is rotated in the roll direction, the roll angle top panel orientation or bottom orientation α I do not want to convert in accordance with the. Thus, a natural display mode that the user is watching the compass, it is possible to display the compass image 312 representing the top face orientation or bottom orientation, it is easy to intuitively recognize the user orientation information can be presented.
[0114]
　Having described in detail preferred embodiments of the present invention with reference to the accompanying drawings, the present invention is not limited thereto. If a person having ordinary skill in the relevant field of the art of the present invention, within the scope of the technical idea described in the claims, it is intended to cover various modifications , also these should be understood that such modifications and changes belong to the technical scope of the present invention.
Description of the code
[0115]
　10 imaging device
　100 housing
　101 top surface
　102 bottom
　105 roll axis
　106 pitch axis
　107 yaw axis
　110 the imaging unit
　111 imaging optical system
　112 imaging element
　120 signal processing unit
　130 display unit
　140 recording medium
　150 the control unit
　151 CPU
　160 operating portion
　170 geomagnetic sensor
　172 acceleration sensor
　200 orientation calculation unit
　202 orientation conversion unit
　204 compass image generation unit
　206 additional information management section
　208 recording unit
　210 playback unit
　300 and 310 captured image
　302,306,312,316 compass image
　304, 314 reproduced image
　α roll angle
　β pitch angle
　γ yaw angle
　θ azimuth angle of the image orientation
　azimuth of the orientation of the display Fai
　Pusai top surface orientation or azimuth angle of the bottom orientation
The scope of the claims
[Claim 1]
　An imaging unit that outputs a captured image by imaging a subject,
　a geomagnetic information detected by the geomagnetic sensor, based on the acceleration information detected by the acceleration sensor, orientation calculation unit that calculates the azimuth of the image pickup direction by the image pickup unit If,
　on the basis of the acceleration information to calculate the rotation angle in the roll direction of the imaging device, on the basis of the rotation angle in the roll direction, a direction converting unit for converting the orientation for display the orientation of the imaging direction,
　the the compass image representing an orientation of the display, and a display unit for displaying together with the captured image,
　the orientation information indicating the orientation of the imaging direction, and a recording section for recording on a recording medium in association with the captured image
comprising a, an imaging apparatus.
[Claim 2]
　The orientation calculation unit, based on the acceleration information, and calculates the rotation angle in the pitch direction of the imaging device, based on the rotation angle of the pitch direction, detects whether or not the imaging direction is substantially vertical direction and,
　if the imaging direction is the substantially vertical direction,
　the orientation calculation unit, the place of the imaging direction of the azimuth, calculates the orientation of the direction perpendicular to the imaging direction,
　the display unit, the display instead of the compass image representing the orientation of use, the compass image representing the orientation of a direction perpendicular to the imaging direction, and displayed together with the captured image,
　the recording unit, the direction information indicative of the orientation of the imaging direction Alternatively, the orientation information indicating the orientation of the direction perpendicular to the imaging direction, is recorded on the recording medium in association with the captured image, the imaging apparatus according to claim 1.
[Claim 3]
　The orientation calculation unit, based on the acceleration information, and calculates the rotation angle in the pitch direction of the imaging device, based on the rotation angle of the pitch direction, and detects whether or not the imaging direction is substantially a vertically downward ,
　when the imaging direction is substantially vertically downward,
　the orientation calculation unit, the place of the imaging direction of the azimuth, calculates the direction of the top surface direction of the image pickup apparatus,
　wherein the display unit, the orientation for the display instead of the compass image representing the compass image representing the orientation of the top surface direction, and displayed together with the captured image,
　the recording unit, instead of the direction information representing a direction of the imaging direction, of the top surface direction the direction information representing a direction, and records on the recording medium in association with the captured image, the imaging apparatus according to claim 2.
[Claim 4]
　The orientation calculation unit, based on the acceleration information, and calculates the rotation angle in the pitch direction of the imaging device, based on the rotation angle of the pitch direction, and detects whether or not the imaging direction is substantially a vertically upward ,
　when the imaging direction is substantially vertically upward,
　the orientation calculation unit, the place of the imaging direction of the azimuth, calculates the direction of the bottom surface direction of the image pickup apparatus,
　wherein the display unit includes an orientation for the display instead of the compass image representing the compass image representing the orientation of the bottom direction, and displayed together with the captured image,
　the recording unit, instead of the direction information representing a direction of the imaging direction represents the orientation of the bottom face direction the orientation information is recorded in said recording medium in association with the captured image, the imaging apparatus according to claim 2.
[Claim 5]
　Further comprising a reproducing unit for reproducing the captured image and the direction information recorded in the recording medium,
　said display unit, a compass image representing an orientation of said orientation information reproduced by the reproducing unit, by the reproducing unit displayed with reproduced the captured image, the imaging apparatus according to claim 1.
[6.]
　While imaging an object by the image pickup unit, a geomagnetic information detected by the geomagnetic sensor, based on the acceleration information detected by the acceleration sensor, calculating a direction of the imaging direction by the imaging unit,
　based on the acceleration information on, to calculate the rotation angle in the roll direction of the imaging device, on the basis of the rotation angle in the roll direction, and converting the azimuth for display the orientation of the imaging direction,
　the compass image representing an orientation for the display , and displaying together with a captured image output from the imaging unit,
　a direction information representing the orientation of the imaging direction, a step of recording on a recording medium in association with the captured image
including the azimuth processing method.
[7.]
　While imaging an object by the image pickup unit, a geomagnetic information detected by the geomagnetic sensor, based on the acceleration information detected by the acceleration sensor, calculating a direction of the imaging direction by the imaging unit,
　based on the acceleration information on, to calculate the rotation angle in the roll direction of the imaging device, on the basis of the rotation angle in the roll direction, and converting the azimuth for display the orientation of the imaging direction,
　the compass image representing an orientation for the display , and displaying together with a captured image output from the imaging unit,
　a direction information representing the orientation of the image capturing direction, and recording on a recording medium, in association with the captured image
program for causing a computer to execute the.