Description
Title of Invention: PHOTOGRAPHING APPARATUS AND
METHOD, IMAGE REPRODUCTION APPARATUS AND
METHOD, PROGRAM, AND RECORDING MEDIUM
Technical Field
[0001] The present technology relates to a photographing apparatus and method, an image
reproduction apparatus and method, a program, and a recording medium, and in
particular, to a photographing apparatus and method, an image reproduction apparatus
and method, a program, and a recording medium which enable more freely the
decision whether to reproduce recorded images to be performed.
Background Art
[0002] Conventionally, the recording of video using a camcorder is controlled using a video
photographing button in users' hand.
[0003] The reproduction of the recorded video can be performed using the camcorder, a
personal computer, or the like. Here, when for example, there is a scene which is not
necessary to be reproduced, a user just continuously watches the video or the user
actively skips the scene using a fast-forward button or a fast-forward icon.
[0004] A scene, which was continuously recorded because the user forgot to turn off a REC
button even when the photographing of a desired subject has ended, may be considered
as an example of an unnecessary scene. In addition, a scene which includes an
unplanned large camera shake, a scene in which the direction of a camera is suddenly
changed, a scene which includes large vibration attributable to the walking of the user
or shocks which the camera has received, or the like may be considered as examples of
unnecessary scenes.
[0005] As a method of preventing unnecessary scenes from being recorded, a technology for
detecting the acceleration of the walking of a user and allowing the start of recording if
the user stops walking has been proposed (for example, refer to Patent Literature 1).
[0006] Further, a technology for, when shocks are received, preventing recording from being
performed for a predetermined time thereafter has been proposed (for example, refer to
Patent Literature 2).
Citation List
Patent Literature
[0007] PTL 1: Japanese Unexamined Patent Application Publication No. 2009-267792
PTL 2: Japanese Unexamined Patent Application Publication No. 06-237463
Summary of Invention
Technical Problem
[0008] However, even when the technologies disclosed in Patent Literature 1 and Patent
Literature 2 are used, there are problems as described below.
[0009] In other words, the technologies disclosed in Patent Literature 1 and Patent Literature
2 may be called configurations in which a scene which is not necessary to be r e
produced is determined when photographing is performed, so that recording itself is
not performed. When such a configuration is used, there is a problem in that there is a
possibility that a scene which the user wants to photograph may be rejected.
[0010] Recently, attempts have been made to save the electric power of an apparatus and to
increase the capacity of a recording medium, and the cost of recording data has been
reduced year by year. Under such circumstances, the burden of cost is almost the same
as before in both the case where an unnecessary scene is decided when photographing
is performed and then the scene is not recorded and the case where the unnecessary
scene is recorded first and then is cut when reproduction is performed.
[001 1] Further, reference, used when a scene which is not necessary to be reproduced (for
example, the amount of vibration used to determine whether to skip a scene) is
specified, varies according to conditions or the permitted value of the user when pho
tographing is performed. However, it is extremely difficult to determine the reference
before photographing is performed.
[0012] In addition, there is a problem in that the conventional methods cannot respond to the
automatic exclusion of a scene, in addition to the case of walking and where shocks are
received.
[0013] The embodiments provide an apparatus. The apparatus includes a hardware
processor; and a storage medium coupled to the processor. The storage medium stores
instructions that, when executed by the processor, cause the apparatus to generate a
motion signal indicating movement of an image recording device, over time, during
generation of image data; and store the image data and the motion signal in the storage
medium.
[0014] The embodiments provide a reproduction apparatus. The reproduction apparatus
includes a receiver configured to receive recorded image data and a motion signal
having motion signal values indicating movement of an image recording device, at cor
responding times, during recording of the image data. The reproduction apparatus
further includes a hardware processor and a storage medium coupled to the processor.
The storage medium stores instructions that, when executed by the processor, cause the
apparatus to analyze the motion signal values; designate, based on the motion signal
values, at least a portion of the image data for non-display; and generate modified
image data consisting of the recorded image data with the at least a portion of the
image data for non-display removed.
[0015] The embodiments provide a method performed by a processing device. The method
includes a step of generating, by the processing device, a motion signal indicating
movement of an image recording device, over time, during generation of image data.
The method further includes a step of storing, in a storage medium, the image data and
the motion signal.
[0016] The embodiments further provide a non-transitory computer-readable medium. The
non-transitory computer-readable medium stores instructions which, when executed by
a computer, cause a computer to generate a motion signal indicating movement of an
image recording device, over time, during generation of image data; and store, in a
storage medium, the image data and the motion signal.
[0017] The embodiments further provide a method of reproducing images by a reproducing
device. The method includes the steps of receiving recorded image data; and receiving
a motion signal indicating movement of an image recording device, during recording
of the image data. The method also includes the steps of analyzing, by the reproducing
device, the motion signal; designating, based on the motion signal, at least a portion of
the image data for non-display; and generating modified image data consisting of the
recorded image data with the at least a portion of the image data for non-display
removed.
[0018] The embodiments further provide a non-transitory computer-readable medium. The
non-transitory computer-readable medium stores instructions which, when executed by
a computer, cause the computer to receive recorded image data; receive a motion
signal indicating movement of an image recording device, during recording of the
image data; analyze, by the computer, the motion signal; designate, based on the
motion signal, at least a portion of the image data for non-display; and generate
modified image data consisting of the recorded image data with the at least a portion of
the image data for non-display removed.
Brief Description of Drawings
[0019] [fig. 1]Fig. 1 is a block diagram illustrating an example of the configuration of a pho
tographing apparatus to which the present technology is applied according to an em
bodiment.
[fig.2]Fig. 2 is a block diagram illustrating an example of the configuration of a r e
production apparatus to which the present technology is applied according to an em
bodiment.
[fig.3]Fig. 3 is a graph illustrating an example of the waveform of a sensing signal
which is output from a gyro sensor.
[fig.4]Fig. 4 is a view illustrating an example of a motion signal.
[fig.5]Fig. 5 is a view illustrating another example of the motion signal.
[fig.6]Fig. 6 is a view illustrating further another example of the motion signal.
[fig.7]Fig. 7 is a view illustrating still another example of the motion signal.
[fig.8]Fig. 8 is a view illustrating still another example of the motion signal.
[fig.9]Fig. 9 is a view illustrating still another example of the motion signal.
[fig. 10] Fig. 10 is a view illustrating still another example of the motion signal.
[fig. 1l]Fig. 11 is a view illustrating still another example of the motion signal.
[fig. 12] Fig. 12 is a view illustrating an example of a method of recording image data
and motion data.
[fig.l3]Fig. 13 is a view illustrating another example of the method of recording image
data and motion data.
[fig. 14] Fig. 14 is a view illustrating still another example of the method of recording
image data and motion data.
[fig.l5]Fig. 15 is a flowchart illustrating an example of a recording process.
[fig.l6]Fig. 16 is a flowchart illustrating an example of a reproducing process.
[fig.l7]Fig. 17 is a flowchart illustrating an example of a scene skip control process.
[fig.l8]Fig. 18 is a view illustrating an example of the appearance of the pho
tographing apparatus.
[fig. 19] Fig. 19 is a block diagram illustrating an example of the configuration of a
personal computer.
Description of Embodiments
[0020] Hereinafter, embodiments of a technology disclosed here will be described with
reference to the accompanying drawings.
[0021] Fig. 1 is a block diagram illustrating an example of the configuration of a pho
tographing apparatus to which the present technology is applied according to an em
bodiment. The photographing apparatus 10 is configured as, for example, a camcorder.
[0022] The photographing apparatus 10 shown in the drawing includes an input unit 21, a
lens actuator 22, a gyro sensor 23, an imaging unit 24, a camera shake correction
operation unit 25, a motion amount calculation unit 26, an encoder 27, a recording unit
28, and a display unit 29.
[0023] The input unit 2 1 receives, for example, input manipulated by a user. For example,
manipulation, such as the start of photographing, the termination of photographing,
zoom-in, zoom-out, or the like, is input by the user.
[0024] When vibration, such as a camera shake, is applied during the photographing of a
video, the lens actuator 22 thereby realizes a function (camera shake correction
function) of negating effect attributable to the vibration by moving a lens to the extent
of the amount of applied vibration. The lens actuator 22 detects vibration, such as the
camera shake, during the photographing, based on a sensing signal which is output
from the gyro sensor 23 which will be described later.
[0025] The gyro sensor 23 detects and outputs the angular velocity of two axes or three axes.
That is, the gyro sensor 23 senses rotations, for example, in the pitch direction, in the
yaw direction, and in the roll direction, and detects the vibration of the photographing
apparatus 10. The sensing signal which is output from the gyro sensor is output to the
lens actuator 22 and the motion amount calculation unit 26.
[0026] The imaging unit 24 includes a sensor or the like which includes a photoelectric
conversion element, for example, a Charge-Coupled Device (CCD) sensor, a Com
plementary Metal-Oxide Semiconductor (CMOS) sensor, or the like, and outputs a
signal, which corresponds to incident light via the lens of the photographing apparatus
10, as an image signal.
[0027] The camera shake correction operation unit 25 generates a signal used to indicate the
size and direction of a camera shake based on the sensing signal which is supplied
from the gyro sensor 23, and outputs the generated signal to the lens actuator 22.
[0028] The motion amount calculation unit 26 generates a motion signal which is a signal
indicative of the motion amount of the photographing apparatus 10, over time, during
generation of image data, based on the sensing signal which is supplied from the gyro
sensor 23. Meanwhile, the motion signal will be described in detail later. The motion
signal which is generated by the motion amount calculation unit 26 is output to the
recording unit 28, together with the output of the encoder 27 which will be described
later. Here, the motion signal is output in such a way as to be associated with the re
generation time that the image data which is output from the encoder 27 is reproduced.
[0029] The encoder 27 encodes the image signal which is supplied from the imaging unit 24
using a predetermined encoding method, such as the Moving Picture Experts Group
(MPEG) method or the like. The encoded image data which is output from the encoder
27 is output to the recording unit 28, together with the motion signal which is
generated by the motion amount calculation unit 26.
[0030] The recording unit 28 records the image data and the motion signal in a recording
medium, such as a Hard Disk Drive (HDD), a semiconductor memory card, or the like.
Here, as described above, the motion signal is recorded in such a way as to be a s
sociated with the regeneration time that the image data is reproduced. Meanwhile, the
motion signal is converted into digital data using any method in which, for example,
the motion signal is encoded when the motion signal is recorded, and is recorded as
motion data.
[0031] The display unit 29 includes a display, for example, a Liquid Crystal Display (LCD)
or the like, and displays an image based on the image signal which is output from the
imaging unit 24.
[0032] Fig. 2 is a block diagram illustrating an example of the configuration of a r e
production apparatus which corresponds to the photographing apparatus 10 in Fig. 1
and to which the present technology is applied according to an embodiment. The re
production apparatus 50 includes, for example, an HDD recorder, a personal computer,
or the like.
[0033] The reproduction apparatus 50 shown in the drawing includes an input unit 61, a
drive 62, an operation decoder 63, a display unit 64, and a cache memory 65.
[0034] The input unit 6 1 receives, for example, an input manipulated by a user. For
example, manipulation, such as an instruction related to the start of reproduction, the
end of reproduction, or scene skip, is input by the user.
[0035] The drive 62 is equipped with a recording medium in which the image data and the
motion data are recorded using the recording unit 28 of the photographing apparatus
10. The drive 62 reads the image data and the motion data from the recording medium
based on, for example, the manipulation of the user which is input via the input unit
61. The image data and the motion data which are read from the recording medium are
supplied to the operation decoder 63.
[0036] The operation decoder 63 decodes, after receiving the image data and the motion data
from drive 62, for example, the image data and the motion data which are read from
the recording medium, and outputs the image signal, which is obtained by performing
a predetermined operation process using the cache memory 65, to the display unit 64.
[0037] The operation decoder 63 obtains and analyzes information indicative of the motion
of a camcorder (camera shake, shock, large change in the direction of lens, or the like)
of the video based on, for example, the motion signal which is obtained by decoding
the motion data, and controls the scene skip based on the information.
[0038] For example, the operation decoder 63 stores the image data which is obtained by
decoding the encoded image data in the cache memory 65. Meanwhile, the operation
decoder 63 specifies a frame to be skipped with reference to the information indicative
of the motion of the camcorder of the video, and then reconfigures a series of image
data of the video from which the specified frame is excluded. Thereafter, the operation
decoder 63 generates the image signal corresponding to the reconfigured image data
and outputs the image signal to the display unit 64. Here, the operation decoder 63 de
termines the extent of motion or the motion period of a frame to be skipped based on,
for example, the manipulation of the user which is input via the input unit 61.
[0039] The display unit 64 includes a display, for example, an LCD or the like, and displays
an image corresponding to the image signal which is supplied from the operation
decoder 63.
[0040] Next, the above-described motion signal will be described. Fig. 3 is a graph in which
the horizontal axis thereof indicates time and the vertical axis thereof indicates a signal
level and which illustrates an example of the waveform of the sensing signal which is
output from the gyro sensor 23. Meanwhile, this example illustrates an example of the
waveform of the rotation sensing signal of the gyro sensor 23 in the yaw direction.
[0041] The sensing signal, which is output from the gyro sensor 23 as shown in Fig. 3, may
be used as the motion signal without change.
[0042] Fig. 4 is a view illustrating another example of a motion signal generated by motion
amount calculation unit 26. The drawing is a graph in which the horizontal axis thereof
indicates time and the vertical axis thereof indicates a signal level and which illustrates
an example of the waveform of a signal which is obtained by binarizing the signal
shown in Fig. 3 by performing threshold determination. In this example, the signal in
Fig. 3 is binarized such that, for example, it is determined that a threshold is exceeded
when the level exceeds
±100 f git
from a DC center value, 1 is output when a sensing signal which is larger than the
threshold is output, and, otherwise, 0 is output.
[0043] As described above, it is possible to simply recognize, for example, the portion
where the motion of the camcorder is large and the portion where the motion is small
of the video by binarizing the sensing signal which is output from the gyro sensor 23.
[0044] As shown in Fig. 4, a signal, which is obtained by binarizing the sensing signal
which is output from the gyro sensor 23 using the threshold determination, may be
used as the motion signal.
[0045] Fig. 5 is a view illustrating further another example of a motion signal generated by
motion amount calculation unit 26. The drawing is a graph in which the horizontal axis
thereof indicates time and the vertical axis thereof indicates a signal level and which i l
lustrates an example of the waveform of a signal which is obtained by performing a
delayed-effect process on the signal shown in Fig. 4. Here, the delayed-effect process
is, for example, a process of performing a process such that, after inversion of the
signal shown in Fig. 3 to 0 or 1, subsequent inversion is not permitted for a prede
termined time.
[0046] For example, when the scene skip control is performed based on the signal as shown
in Fig. 4, it is necessary to repeatedly determine whether to perform reproduction by a
fine cycle whenever the threshold is exceeded, so that load is large when the re
production is performed. When the delayed-effect process is performed as shown in
Fig. 5, it is not necessary to repeatedly determine whether to perform reproduction by
the above-described fine cycle.
[0047] As shown in Fig. 5, a signal, which is obtained by binarizing the sensing signal
which is output from the gyro sensor 23 using the threshold determination and, further,
performing the delayed-effect process, may be used as the motion signal.
[0048] Fig. 6 is a view illustrating still another example of a motion signal generated by
motion amount calculation unit 26. The drawing is a graph in which the horizontal axis
thereof indicates time and the vertical axis thereof indicates a signal level and which i l
lustrates an example of the waveform of a signal which is obtained by ternarizing the
signal shown in Fig. 3 using stepwise determination with two thresholds, and, further,
by performing the delayed-effect process.
[0049] For example, when the scene skip control is performed based on the signal as shown
in Fig. 6, the size of a motion, which is determined that it is not necessary to be r e
produced when reproduction is performed, can be selected from a plurality of sizes by
providing a plurality of thresholds.
[0050] As shown in Fig. 6, a signal, which is obtained by multi-valuing the sensing signal
which is output from the gyro sensor 23 using the stepwise determination with the
plurality of thresholds and, further, by performing the delayed-effect process, may be
used as the motion signal.
[0051] Fig. 7 is a view illustrating still another example of a motion signal generated by
motion amount calculation unit 26. The drawing is a graph in which the horizontal axis
thereof indicates time and the vertical axis thereof indicates a signal level and which i l
lustrates an example of the waveform of a signal which is obtained by performing a
low-pass filter process on a signal which is obtained by ternarizing the signal shown in
Fig. 3 using the stepwise determination with two thresholds. That is, the low-pass filter
process is performed on the signal instead of performing the delayed-effect process as
the case shown in Fig. 6.
[0052] As shown in Fig. 7, a signal, which is obtained by multi-valuing the sensing signal
which is output from the gyro sensor 23 using the stepwise determination with the
plurality of thresholds and, further, by performing the low-pass filter process, may be
used as the motion signal.
[0053] Fig. 8 is a view illustrating still another example of a motion signal generated by
motion amount calculation unit 26. In the case of this example, an example of a
waveform, which is obtained by calculating the area value of a power spectrum in the
predetermined frequency range for each predetermined time with respect to the sensing
signal which is output from the gyro sensor and projecting the area value of the power
spectrum on the time axis, is shown. That is, the horizontal axis of Fig. 8 indicates time
and the vertical axis thereof indicates the area of the power spectrum.
[0054] With respect to the motion signal as shown in Fig. 8, effects, for example, periodic
motion, sudden strong motions, or the like can be excluded. That is, since a signal level
is not high at a portion where it is not certain whether a camera shake occurred or not
in a video, only a clear camera shake or shock can be recognized. The signal may be
used as the motion signal.
[0055] Fig. 9 is a view illustrating still another example of a motion signal generated by
motion amount calculation unit 26. In the case of this example, a graph, in which the
horizontal axis thereof indicates time and the vertical axis thereof indicates the level of
a signal and which illustrates an example of the waveform of a signal which is
obtained by binarizing the signal shown in Fig. 8 using the threshold determination, is
shown. The signal may be used as the motion signal.
[0056] Fig. 10 is a view illustrating still another example of a motion signal generated by
motion amount calculation unit 26. In the case of this example, a graph, in which the
horizontal axis thereof indicates time and the vertical axis thereof indicates the level of
a signal and which illustrates an example of the waveform of a signal which is
obtained by performing the delayed-effect process on the signal shown in Fig. 9, is
shown. The signal may be used as the motion signal.
[0057] Fig. 11 is a view illustrating still another example of a motion signal generated by
motion amount calculation unit 26. In the case of this example, a graph, in which the
horizontal axis thereof indicates time and the vertical axis thereof indicates a signal
level and which illustrates an example of the waveform of a signal which is obtained
by performing the low-pass filter process on the signal which is obtained by ternarizing
the signal shown in Fig. 8 using stepwise determination with two thresholds, is shown.
The signal may be used as the motion signal.
[0058] Next, a method of recording the image data and the motion data will be described.
[0059] Fig. 12 is a view illustrating a method of synchronizing the waveform of the motion
signal with the time line of the image data and recording the waveform of the motion
signal. A synchronized waveform 111 displays, for example, the waveform of the
motion signal, like the cases which were described with reference to Figs. 3 and 8.
[0074]
[0060] The recording unit 28 synchronizes the waveform of the motion signal with the time
line of the image data by, for example, sampling and quantizing the waveform 111,
and records the waveform. Meanwhile, each value, obtained through the quantization,
is filed with data which indicates time on the time line and then recorded, so that the
motion data is recorded with the image data.
[0061] Therefore, for example, the motion data is reproduced by the operation decoder 63 of
the reproduction apparatus 50, so that the waveform of the motion signal can be r e
produced, like the cases which were described with reference to Figs. 3 and 8. In this
case, for example, before the image is reproduced, the user can view the waveform 111
and perform edit, such as designation of scenes to be skipped on the time line.
[0062] Fig. 13 is a view illustrating another example of the method of recording the image
data and the motion data. The drawing illustrates an example of a case where motion
size is extracted as status based on, for example, the motion signal which was multivalued
(including binarization), as shown in Figs. 4 to 7 and Figs. 9 to 11, and the
status is recorded for each frame of video data. In this case, for example, it is assumed
that the motion signal is ternarized, and motion size is indicated as the "intensity" of a
camera shake, shock or the like, and the status of first intensity to the status of third
intensity are set.
[0063] As shown in Fig. 13, the status of each frame is displayed using the values 0 to 3
written on the time line. The status can be recorded by, for example, storing the status
in the header of a packet included in the video data.
[0064] Further, as shown in the drawing, if color-coded (here, hatching) bars based on the
status of the frame are displayed on the time line, the large camera shake (large
motion) interval or the less camera shake interval of the video can be recognized at one
view.
[0065] Fig. 14 is a view illustrating another example of the method of recording the image
data and the motion data. Fig. 14 illustrates an example of a case where the change in
motion size is extracted based on, for example, the motion signal which was multi
valued (including binarization), as shown in Figs. 4 to 7 and Figs. 9 to 11, and the
change in motion size is recorded as a flag which is associated with the time line of the
video data. In this case, the motion data is stored with the image data by, for example,
recording data 121, as shown in the lower right of Fig. 14, after filing the data 121 in
which time stamp is associated with "status" which is information indicative of the
change in the motion size.
[0066] According to the data 121 in Fig. 14, the facts that the motion size is "0" at time
stamp "00:00" and the motion size changes from "0" to "1"
r >
at the time stamp "01:05" are shown. Likewise, the facts that the motion size changes
from "1" to "2" at the time stamp "02:40", the motion size changes from "2" to "3" at
the time stamp "03:00", and the motion size changes from "3" to "2" at the time stamp
"04:12",..., are shown.
[0067] For example, a scene to be skipped may be controlled by designating a scene skip
start point and a scene skip end point using a flag based on the change in motion size.
When the flagged motion data is recorded as described above, for example, the change
point of the motion when reproduction is performed may be easily recognized and a
scene skip start point and a scene skip end point can be simply identified. Further, the
load of recording can be restricted to be low, compared to, for example, the case where
status is recorded for each frame.
[0068] In this way, the image data and the motion data are recorded.
[0069] Next, an example of recording process performed by the photographing apparatus 10
in Fig. 1 will be described with reference to the flowchart of Fig. 15.
[0070] In step S21, photographing starts using the photographing apparatus 10.
[0071] In step S22, the motion amount calculation unit 26 obtains the sensing signal which
is supplied from the gyro sensor 23.
[0072] In step S23, the motion amount calculation unit 26 generates the motion signal. At
this time, the motion signal is generated, for example, as described above with
reference to Figs. 3 to 11.
[0073] In step S24, the encoder 27 encodes the image signal which is supplied from the
imaging unit 24 using a predetermined encoding method such as MPEG or the like.
[0074] Meanwhile, as described above, the motion signal which is generated in the process
of step S23 is associated with a regeneration time (that is, time line) in the case where
the image data which is encoded in the process of step S24 is reproduced.
[0075] In step S25, the recording unit 28 generates the motion data by digitizing the motion
signal which is generated in the process of step S23 in association with the time line.
[0076] In step S26, the recording unit 28 records the image data encoded in step S24 and the
motion data generated in step S25 in the recording medium. For example, a data file
including the image data and the motion data is generated.
[0077] In step S27, it is determined whether the termination of photographing is instructed
or not. For example, when a predetermined photographing termination manipulation is
performed using the input unit 21, it is determined that the termination of pho
tographing is instructed.
[0078] In step S27, when it is determined that the termination of photographing is not in
structed, the process returns to step S22.
[0079] Meanwhile, in step S27, when it is determined that the termination of photographing
is instructed, the recording process is terminated.
[0080] In this way, the recording process is performed.
[0081] Next, an example of the reproducing process performed by the reproduction
apparatus 50 in Fig. 2 will be described with reference to the flowchart in Fig. 16.
Meanwhile, the process is performed by the reproduction apparatus 50 when the re
production of video is instructed, and the recording medium, which includes data
recorded as the results of the recording process which was described above with
reference to Fig. 15, is mounted on the drive 62 of the reproduction apparatus 50
before the process is performed.
[0082] In step S41, the image data and the motion data are read from the recording medium
which is mounted on the drive 62.
[0083] In step S42, the operation decoder 63 decodes the image data and the motion data
which are read in the process of step S41.
[0084] In step S43, the cache memory 65 stores the image data which was decoded and then
obtained in the process of step S42.
[0085] In step S44, the operation decoder 63 performs the scene skip control process which
will be described later with reference to a flowchart in Fig. 17. Therefore, for example,
the frame of a scene to be skipped is designated and then the image data is re
configured.
[0086] In step S45, the operation decoder 63 generates the image signal corresponding to the
image data which was reconfigured in accordance with the process of step S44.
[0087] In step S46, the display unit 64 displays an image corresponding to the image signal
which was generated in the process of step S45.
[0088] In this way, the reproducing process is performed. [0103]
[0089] Next, an example of the scene skip control process in step S44 of Fig. 16 will be
described in detail with reference to the flowchart in Fig. 17.
[0090] In step S61, the operation decoder 63 analyzes the motion signal obtained based on
the motion data. Therefore, for example, information (camera shake, shock, large
change in the direction of lens, or the like) indicative of the motion of a camcorder of
the video is obtained.
[0091] In step S62, the operation decoder 63 specifies a frame to be skipped based on the
results of the analysis in step S61.
[0092] Here, a user designates a scene to be skipped by displaying, for example, the time
line and the motion signal as shown in Fig. 12 on the display unit 64, so that a frame
which configures the scene is specified as a frame to be skipped.
[0093] Further, for example, when the status of each frame is recorded as shown in Fig. 13,
the frame of preset status is specified as the frame to be skipped. For example, when
setting is made such that the determination reference of the frame to be skipped is
equal to or greater than first status, the frames of the first status to third status are de
termined to be the frames to be skipped. Further, for example, when setting is made
such that the determination reference of the frame to be skipped is equal to or greater
than the second status, the frames of the second status to the third status are determined
to be the frames to be skipped. In addition, for example, when setting is made such that
the determination reference of the frame to be skipped is equal to or greater than the
third status, the frame of the third status is determined to be the frame to be skipped.
[0094] Further, for example, when flagged motion data is recorded as shown in Fig. 14, a
frame within an interval designated by the user is specified as the frame to be skipped
based on the flag.
[0095] In step S63, the operation decoder 63 reconfigures a series of image data of the video
excluding the frames specified in the process of step S62. Therefore, the scenes
including, for example, a camera shake, shock, the large change in the direction of
lens, or the like are skipped and the video is reproduced.
[0096] In this way, the scene skip control process is performed.
[0097] As described above, according to the present technology, the motion data is recorded
in association with the time line when photographing (recording) is performed, and a
predetermined scene is skipped based on the motion data when reproduction is
performed.
[0098] Conventionally, for example a technology for detecting the acceleration of the
walking of a user and allowing the start of recording if the user stops walking and a
technology for, when shocks are received, preventing recording from being performed
for a predetermined time thereafter have been proposed.
[0099] However, in other words, the technologies may be called configurations in which a
scene which is not necessary to be reproduced is determined when photographing is
performed, so that recording itself is not performed. When such a configuration is
used, there is a problem in that there is a possibility that a scene which the user wants
to photograph may be rejected.
[0100] Further, recently, attempts have been made to save the electric power of an apparatus
and to increase the capacity of a recording medium, and the cost of recording data has
been reduced year by year. Under such a circumstance, the burden of cost is almost the
same as before in both a case where an unnecessary scene is determined when pho
tographing is performed and then the scene is not recorded and a case where the u n
necessary scene is recorded first and then is cut when reproduction is performed.
[0101] Further, reference, used when a scene which is not necessary to be reproduced (for
example, the amount of vibration used to determine whether to skip a scene) is
specified, varies according to conditions or the permitted value of the user when pho
tographing is performed. However, it is extremely difficult to determine the reference
before photographing is performed.
[0102] On the other hand, according to the present technology, all scenes are recorded when
photographing is performed and an unnecessary scene is specified when reproducing is
performed, so that, for example, a scene which the user wants to photograph is not
rejected.
[0103] Further, a scene to be skipped can be specified based on, for example, status which is
associated with a time line, so that it is able to flexibly respond to a reference which
differs according to a condition when photographing is performed and the permitted
value of the user.
[0104] Further, even in the case of a scene which is generally regarded that it is not
necessary to be reproduced because of a large camera shake, the scene may be a
precious memory for a user when, for example, the scene is photographed by a small
child, so that such a scene is not completely cut and can be preserved. Therefore,
according to the present technology, all the memories of the user can be recorded.
[0105] Further, if reference, which is used to determine a scene which is not necessary to be
reproduced, is set, the scene can be automatically skipped.
[0106] Therefore, according to the present technology, whether to reproduce a recorded
image or not can be more freely decided.
[0107] Hereinbefore, an example in which the motion signal is generated based on the
sensing signal of the gyro sensor 23 has been described. However, the motion signal
may be generated based on, for example, a sensing signal output from an acceleration
sensor.
[0108] In this way, a frame which is photographed when a camera faces down can be
specified using, for example, an acceleration sensor as an inclination sensor, which
refers to the vector direction of gravitation, in addition to motion, such as a camera
shake, shock, or the like. When, for example, the user forgets to press a termination
button after the photographing is terminated, there is a case where the images of
ground are continuously photographed while a camera faces downward. If a motion
signal is generated based on the sensing signal output from the acceleration sensor and
motion data is recorded in association with a time line, a scene which is photographed
while the camera faces downward can be skipped.
[0109] Further, an image sensor may be mounted on the photographing apparatus 10, and
sensor data obtained based on a sensing signal output from the image sensor may be
recorded in association with the time line.
[0110] Fig. 18 is a view illustrating an example of the appearance of the photographing
apparatus 10. The drawing shows an example of the appearance of the photographing
apparatus 10 viewed from the side of an observer, and, in this case, an object is present
on the slightly left side of a space in the depth direction. As shown in the drawing, the
image sensor 31 is provided, for example, in the lower part of the display of the display
unit 29 which displays an image toward a photographer.
[0111] The image sensor 3 1 includes, for example, a compact CCD camera, and recognizes
the facial image of a photographed image. That is, for example, a sensing signal used
to indicate whether the face of the user is recognized is output by the image sensor 31.
Therefore, for example, a frame obtained when the user performs photographing while
looking at the direction of the object can be specified.
[0112] Generally, when an image is photographed, the user performs photographing toward
the direction of the object. Meanwhile, for example, when the user forgets to press a
termination button after photographing is terminated, it is conceivable that the user
does not look at the direction of the object. That is, it is conceivable that it is highly
worthy of reproducing the image of a frame which is photographed when the face of
the user is recognized by the image sensor 31. However, it is conceivable that it is less
worthy of reproducing the image of a frame which is photographed when the face of
the user is not recognized by the image sensor 31.
[0113] As described above, if the sensor data, obtained based on the sensing signal output
from the image sensor 31, is recorded in association with the time line, it is possible to
simply determine a scene which is not necessary to be reproduced after all. Further, if a
scene which is not necessary to be reproduced is determined using the sensor data
while a scene to be skipped is determined based on the above-described motion data, it
is possible to more effectively and automatically skip the scene.
[01 14] That is, it is not necessary that information, which is recorded in association with the
time line in the present technology, is not necessarily information indicative of motion,
and may be various types of incidental information such as data sensed by a prede
termined sensor.
[0115] Meanwhile, the above-described series of processes can be performed using
hardware and can be performed using software. When the above-described series of
processes are performed using software, a program which configures the software is
installed in a computer provided with dedicated hardware, or, for example, a generalpurpose
personal computer 700, as shown in Fig. 19, which is capable of performing
various types of functions by installing various types of programs from a network or a
recording medium (i.e., a non-transitory recording medium) storing the software
program.
[01 16] In Fig. 19, a Central Processing Unit (CPU) 701 executes various types of processes
according to a program which is stored in a Read Only Memory (ROM) 702 or a
program which is loaded in a Random Access Memory (RAM) 703 from a storage unit
708. Further, the RAM 703 appropriately stores data or the like which is necessary
when the CPU 701 executes the various types of processes.
[01 17] The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus
704. Further, an input/output interface 705 is connected to the bus 704.
[0118] The input/output interface 705 is connected to the input unit 706 which includes a
keyboard, a mouse, or the like, an output unit 707 which includes a display such as a
Liquid Crystal Display (LCD), a speaker or the like, a storage unit 708 which includes
a hard disk or the like, and a communication unit 709 which includes a network
interface card such as a modem, a Local Area Network (LAN) card, or the like. The
communication unit 709 processes a communication process via a network which
includes the Internet.
[01 19] A drive 710 is connected to the input/output interface 705 as necessary, and a
removable media 7 11, such as a magnetic disk, an optical disc, a magneto-optical disc,
a semiconductor memory or the like, is appropriately mounted thereon. A computer
program which is read from the drive 710 and the removable media 7 11 is installed in
the storage unit 708.
[0120] When the above-described series of processes are performed by software, a program
which configures the software is installed from a network, such as the Internet or the
like, or a recording medium which includes the removable media 711 or the like.
[0121] Meanwhile, in addition to the main body of the apparatus shown in Fig. 19, the
recording medium includes not only the removable media 711 which includes a
magnetic disc (including a floppy disc (registered trademark)), an optical disc
(including a Compact Disk-Read Only Memory (CD-ROM) and a Digital Versatile
Disc (DVD)), a magneto-optical disc (including a Mini-Disk (MD, registered
trademark)), a semiconductor memory or the like, which are distributed to deliver a
program to the user and in which the program is recorded, but also a hard disc which is
included in the ROM 702 or the storage unit 708 which is delivered to the user while
being built in the main body of the apparatus in advance and in which a program is
recorded.
[0122] Meanwhile, the above-described series of processes in the present specification
include processes which are performed in temporal sequence along the written order
and processes which are performed in parallel or individually even though the
processes are not always processed in temporal sequence.
[0123] Further, the embodiment of the present technology is not limited to the abovedescribed
embodiment and various modifications are possible in the range which does
not depart from the gist of the present technology.
[0124] Additionally, the present technology may also be configured as below.
(1)
An apparatus comprising:
a hardware processor; and
a storage medium coupled to the processor and storing instructions that, when
executed by the processor, cause the apparatus to:
generate a motion signal indicating movement of an image recording device, over
time, during generation of image data; and
store the image data and the motion signal in the storage medium.
(2)
The apparatus according to (1), further comprising a gyro sensor.
(3)
The apparatus according to (1) or (2), further comprising an acceleration sensor .
(4)
The apparatus according to any one of (1) to (3), further comprising an image
detection unit configured to detect a facial image in the image data.
(5)
The apparatus according to any one of (1) to (4), wherein the processor is configured
to generate, from the motion signal, a first processed signal having a plurality of first
processed signal values at points in time, the first processed signal values corre
sponding to one of a predetermined number of threshold values.
(6)
The apparatus according to (5), wherein the processor is configured to generate a
second processed signal having second processed signal values, the second processed
signal values:
corresponding to the value of the first processed signal at a corresponding point in
time, under occurrence of a first condition; and
corresponding to the value of the first processed signal at an earlier point in time, upon
occurrence of a second condition.
(7)
The apparatus according to (5) or (6), wherein the processor is configured to generate
the first processed signal by applying a low-pass filter to the motion signal.
(8)
The apparatus according to any one of (5) to (7), wherein the processor is configured to
designate flags on the first processed signal based on the threshold values.
(9)
The apparatus according to any one of (1) to (7), further comprising:
a display unit configured to display the images.
(10)
A reproduction apparatus comprising:
a receiver configured to receive:
recorded image data; and
a motion signal having motion signal values indicating movement of an image
recording device, at corresponding times, during recording of the image data; and
a hardware processor; and
a storage medium coupled to the processor and storing instructions that, when executed
by the processor, cause the apparatus to:
analyze the motion signal values;
designate, based on the motion signal values, at least a portion of the image data for
non-display; and
generate modified image data consisting of the recorded image data with the at least a
portion of the image data for non-display removed.
(11)
The apparatus according to (10), wherein:
the motion signal values comprise a first value corresponding to motion of the image
recording device at first points in time, and a second value corresponding to motion of
the image recording device at second points in time, the first value being different from
the second value;
the processor is configured to designate the at least a portion of the image data for non
display, in accordance with the motion signal values.
(12)
The apparatus according to (10) or (11), wherein:
the motion signal is divided into segments by flags having flag values indicative of
transitions of the motion signal between motion signal values; and
the processor is configured to designate the at least a portion of the image data for non
display in accordance with the flag values.
(13)
The apparatus according to any one of (10) to (12), further comprising:
a display unit configured to display the modified image data.
Reference Signs List
Meanwhile, the present technology can include one or more of the following config
urations.
10 photographing apparatus
2 1 input unit
22 lens actuator
23 gyro sensor
24 imaging unit
25 camera shake correction operation unit
26 motion amount calculation unit
27 encoder
28 recording unit
29 display unit
50 reproduction apparatus
6 1 input unit
62 drive
63 operation decoder
64 display unit
65 cache memory
PCT/JP2012/003343
Claims
An apparatus comprising:
a hardware processor; and
a storage medium coupled to the processor and storing instructions that,
when executed by the processor, cause the apparatus to:
generate a motion signal indicating movement of an image recording
device, over time, during generation of image data; and
store the image data and the motion signal in the storage medium.
The apparatus according to claim 1, further comprising a gyro sensor.
The apparatus according to claim 1, further comprising an acceleration
sensor .
The apparatus according to claim 1, further comprising an image
detection unit configured to detect a facial image in the image data.
The apparatus according to claim 1, wherein the processor is
configured to generate, from the motion signal, a first processed signal
having a plurality of first processed signal values at points in time, the
first processed signal values corresponding to one of a predetermined
number of threshold values.
The apparatus according to claim 5, wherein the processor is
configured to generate a second processed signal having second
processed signal values, the second processed signal values:
corresponding to the value of the first processed signal at a corre
sponding point in time, under occurrence of a first condition; and
corresponding to the value of the first processed signal at an earlier
point in time, upon occurrence of a second condition.
The apparatus according to claim 5, wherein the processor is
configured to generate the first processed signal by applying a low-pass
filter to the motion signal.
The apparatus according to claim 5, wherein the processor is
configured to designate flags on the first processed signal based on the
threshold values.
The apparatus according to claim 1, further comprising:
a display unit configured to display the images.
A reproduction apparatus comprising:
a receiver configured to receive:
recorded image data; and
a motion signal having motion signal values indicating movement of an
PCT/JP2012/003343
image recording device, at corresponding times, during recording of the
image data; and
a hardware processor; and
a storage medium coupled to the processor and storing instructions that,
when executed by the processor, cause the apparatus to:
analyze the motion signal values;
designate, based on the motion signal values, at least a portion of the
image data for non-display; and
generate modified image data consisting of the recorded image data
with the at least a portion of the image data for non-display removed.
The apparatus according to claim 10, wherein:
the motion signal values comprise a first value corresponding to motion
of the image recording device at first points in time, and a second value
corresponding to motion of the image recording device at second points
in time, the first value being different from the second value;
the processor is configured to designate the at least a portion of the
image data for non-display, in accordance with the motion signal
values.
The apparatus according to claim 10, wherein:
the motion signal is divided into segments by flags having flag values
indicative of transitions of the motion signal between motion signal
values; and
the processor is configured to designate the at least a portion of the
image data for non-display in accordance with the flag values.
The apparatus according to claim 10, further comprising:
a display unit configured to display the modified image data.
A method performed by a processing device, comprising:
generating, by the processing device, a motion signal indicating
movement of an image recording device, over time, during generation
of image data; and
storing, in a storage medium:
the image data; and
the motion signal.
A non-transitory computer-readable medium storing instructions which,
when executed by a computer, cause the computer to:
generate, by the computer, a motion signal indicating movement of an
image recording device, over time, during generation of image data;
and
WO 2012/164873 PCT/JP2012/003343
store, in a storage medium:
the image data; and
the motion signal.
[Claim 16] A method of reproducing images by a reproducing device, comprising:
receiving recorded image data;
receiving a motion signal indicating movement of an image recording
device, during recording of the image data;
analyzing, by the reproducing device, the motion signal;
designating, based on the motion signal, at least a portion of the image
data for non-display; and
generating modified image data consisting of the recorded image data
with the at least a portion of the image data for non-display removed.
[Claim 17] A non-transitory computer-readable medium storing instructions which,
when executed by a computer, cause the computer to:
receive recorded image data;
receive a motion signal indicating movement of an image recording
device, during recording of the image data;
analyze, by the computer, the motion signal;
designate, based on the motion signal, at least a portion of the image
data for non-display; and
generate modified image data consisting of the recorded image data
with the at least a portion of the image data for non-display removed.

International application No.
PCT/JP2012/003343
A. CLASSIFICATION OF SUBJECT MATTER
Int.Cl. H04N5/91 (2006.01) i , H04N5/225 (2006 .01) i , H04N5/232 (2006 .01) i
According to International Patent Classification (IPC) or to both national classification and IPC
~ . FIELDS SEARCHED
Minimum documentation searched (classification system followed by classification symbols)
Int.Cl. H04N5/91, H04N5/225, H04N5/232
Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched
Electronic data base consulted during the international search (name of data base and, where practicable, search terms used)
C. DOCUMENTS CONSIDERED TO BE RELEVANT
Category Citation of document, with indication, where appropriate, of the relevant passages Relevant to claim No.
X JP 2005-348178 A(Canon Inc.) 2005.12.15, 1-2, 9-10,
[0022] - [0032] , [0043] - [0054] , fig.1-2,4 13-17
(No Family)
3-8, 11-12
JP 2001-154226 A(OLYMPUS CORPORATION)
2001.06.08, [0023] , fig.l
(No Family)
Further documents are listed in the continuation of Box C. See patent family annex.
Special categories of cited documents: "T" later document published after the international filing date or
A" document defining the general state of the art which is not priority date and not in conflict with the application but cited to
considered to be of particular relevance understand the principle or theory underlying the invention
E" earlier application or patent but published on or after the inter
"X" document of particular relevance; the claimed invention cannot
national filing date be considered novel or cannot be considered to involve an
L" document which may throw doubts on priority claim(s) or which inventive step when the document is taken alone
is cited to establish the publication date of another citation or other
special reason (as specified) "Y" document of particular relevance; the claimed invention cannot
O" document referring to an oral disclosure, use, exhibition or other be considered to involve an inventive step when the document is
means combined with one or more other such documents, such
P" document published prior to the international filing date but later combination being obvious to a person skilled in the art
than the priority date claimed "&" document member of the same patent family
Date of the actual completion of the international search Date of mailing of the international search report
17 .08 .2012 2 8 .08 .2012
Name and mailing address of the ISA/JP Authorized officer 5C 4 4 5 0
Japan Patent Office Ayako Tanaka
3-4-3, Kasumigaseki, Chiyoda-ku, Tokyo 100-8915, Japan Telephone No. +81-3-3581-1 101 Ext. 3541
Form PCT/ISA/210 (second sheet) (July 2009)
International application No.
PCT/JP2012/003343
C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT
Category* Citation of document, with indication, where appropriate, of the relevant passages Relevant to claim No
J P 2010-045765 A (Canon Inc.) 2010.02.25,
the whole document
& US 2010/0014835 Al & CN 101631220 A
J P 2007-286434 A (Canon Inc.) 2007.11.01, [0021] 5-8 , 11-12
& US 2008/0204566 Al & US 2011/0109756 Al
& US 2011/0109757 Al & US 2011/0109783 Al
& US 2011/0115933 Al &WO 2007/029740 Al
& CN 101258741 A & CN 101674413 A
& CN 101674414 A & CN 101674415 A
& CN 101697572 A & CN 102271222 A
J P 2005-115253 A (Nikon Corporation)
2005.04.28, [0030]
(No Family)
J P 2009-100314 A (Sony Corporation) 2009.05.07, 8, 12
[0134] ,fig.22
& US 2009/0103888 Al & CN 101414302 A
Form PCT/ISA/210 (continuation of second sheet) (July 2009)
INTERNATIONAL SEARCH REPORT International application No.
PCT/JP2 0 1 2 / 0 0 3 3 4 3
Box No. I I Observations where certain claims were found unsearchable (Continuation of item 2 of first sheet)
This international search report has not been established in respect of certain claims under Article 17(2)(a) for the following reasons:
1. Claims Nos.:
because they relate to subject matter not required to be searched by this Authority, namely:
2. if " Claims Nos.:
because they relate to parts of the international application that do not comply with the prescribed requirements to such an
extent that no meaningful international search can be carried out, specifically:
3• Claims Nos.:
because they are dependent claims and are not drafted in accordance with the second and third sentences of Rule 6.4(a).
Box No. Observations where unity of invention is lacking (Continuation of item 3 of first sheet)
This International Searching Authority found multiple inventions in this international application, as follows:
D l (JP 2005-348178 A (Canon Inc.) 2005 .12 .15) discloses [An apparatus
comprising: a hardware processor; and a storage medium coupled to the
processor and storing instructions that, when executed by the
processor, cause the apparatus to: generate amotion signal indicating
movement of an image recording device, over time, during generation
of image data; and store the image data and the motion signal in the
storage medium] . Therefore, claims 1-2, 9-10, 13-17 lack novelty over
Dl, and involve no special technical features.
Thus, there are 5 inventions in the claims of this application, (see
extra sheet)
Note that claims 1-2, 9-10, 13-17 which involve no special technical
features, are grouped into Invention 1 .
1. As all required additional search fees were timely paid by the applicant, this international search report covers all searchable
claims.
2. As all searchable claims could be searched without effort justifying additional fees, this Authority did not invite payment of
additional fees.
3 . ™ As only some of the required additional search fees were timely paid by the applicant, this international search report covers
only those claims for which fees were paid, specifically claims Nos.:
No required additional search fees were timely paid by the applicant. Consequently, this international search report
restricted to the invention first mentioned in the claims; it is covered by claims Nos.:
Remark on Protest e additional search fees were accompanied by the applicant's protest and, where applicable, the
payment of a protest fee.
; The additional search fees were accompanied by the applicant's protest but the applicable protest
fee was not paid within the time limit specified in the invitation
f ; No protest accompanied the payment of additional search fees.
Form PCT/ISA/210 (continuation of first sheet (2)) ( uly 2009)
International application No.
PCT/JP2012/003343
(Invention 1 ) claims 1-3, 9-10, 13-17
An apparatus comprising: a hardware processor; and a storage medium
coupled to the processor and storing instructions that, when executed
by the processor, cause the apparatus to: generate a motion signal
indicating movement of an image recording device, over time, during
generation of image data; and store the image data and the motion signal
in the storage medium.
(Invention 2 ) claims 4
An apparatus comprising: a hardware processor; and a storage medium
coupled to the processor and storing instructions that, when executed
by the processor, cause the apparatus to: generate a motion signal
indicating movement of an image recording device, over time, during
generation of image data; and store the image data and the motion signal
in the storage medium; and an image detection unit configured to detect
a facial image in the image data.
(Invention 3 ) claims 5-6, 11
An apparatus comprising: a hardware processor; and a storage medium
coupled to the processor and storing instructions that, when executed
by the processor, cause the apparatus to: generate a motion signal
indicating movement of an image recording device, over time, during
generation of image data; and store the image data and the motion signal
in the storage medium; wherein the processor is configured to generate,
from the motion signal, a first processed signal having a plurality
of first processed signal values at points in time, the first processed
signal values corresponding to one of a predetermined number of
threshold values; and to generate a second processed signal having
second processed signal values, the second processed signal values:
corresponding to the value of the first processed signal at a
corresponding point in time, under occurrence of a first condition;
and corresponding to the value of the first processed signal at an
earlier point in time, upon occurrence of a second condition
(Invention 4 ) claim 7
An apparatus comprising: a hardware processor; and a storage medium
coupled to the processor and storing instructions that, when executed
by the processor, cause the apparatus to: generate a motion signal
indicating movement of an image recording device, over time, during
generation of image data; and store the image data and the motion signal
in the storage medium; wherein the processor is configured to generate,
from the motion signal, a first processed signal, by applying a
low-pass filter to the motion signal, having a plurality of first
processed signal values at points in time, the first processed signal
values corresponding to one of a predetermined number of threshold
values .