DESCRIPTION
Technical Field
[0001]
The present invention relates to a technique for
detecting an operation which is performed by indicating a
position on a projection screen of an image.
Background Art
[0002]
There are projectors having a function of detecting a
position on a projection screen of an image iitdicated by an
indicator such as a pen, and displaying a handwritten image
in accordance with the detected position, or receiving an
operation for selecting an object (for example, menu or icon) .
PTL 1 discloses a technique capable of executing a process
across multiple screens during display in such a projector.
PTL 1 discloses the execution of a process across a plurality
of display regions in a case where images which are input from
a plurality of image sources are divided and displayed by one
projector.
Citation List
Patent Literature
2
[0003]
PTL 1: JP-A-2013-97177
Summary of Invention
Technical Problem
[0004]
There is a display method called a multi-monitor in which
a substantial screen region is expanded by connecting a
plurality of monitors to one personal computer (PC). The use
of this display method allows a display region having a
projected image displayed therein to be substantially expanded
using a plurality of projectors. However, in a case where an
operation which is performed across the inside and outslde of
a display region corresponding to one projector is performed,
there is the possibility of a defect of a process based on the
operation occurring. PTL 1 does not disclose a defect of a
process in a case where a plurality of projectors are used.
The invention is devised in view of such circumstances,
and one of the objects thereof is to eliminate a defect of a
process based on an operation which is performed across the
inside and outside of a display region in which a projected
image is displayed.
Solution to Problem
[0005]
3
In order to achieve the above object, an information
processing device according to the invention includes:
detection unit for repeatedly detecting a position indicated
by an indicator with respect to a projection screen onto which
an image is projected, and supplying position information of
the detected position; and generation unit for generating
position information of a third position located between a
first position located inside of a display region of the image
in the projection screen and a second position located outside
of the display region, in a case where the detection unit
detects the first position, and then detects the second
position.
According to the invention, it is possible to eliminate
a defect of a process based on an operation which is performed
across the inside and outside of the display region in which
the projected image is displayed.
[0006]
In order to achieve the above object, an information
processing device according to the invention includes:
detection unit for repeatedly detecting a position indicated
by an indicator with respect to a projection screen onto which
an image is projected, and supplying position information of
the detected position; and generation unit for generating
position information of a third position located between a
first position located inside of a display region of the image
4
and a second position located outside of the display region
in the projection screen, in a case where the detection unit
detects the second position, and then detects the first
position.
According to the invention, it is possible to eliminate
a defect of a process based on an operation which is performed
across the inside and outside of the display region in which
the projected image is displayed.
[0007]
In the invention, the detection unit may detect the
indicated position on the basis of image data obtained by
capturing an image of an imaging region including the display
region in a predetermined period, and the generation unit may
specify the first position and the second position on the basis
of two pieces of the image data· of which times of the imaging
are different from each other by one period, and generate the
position information of the third position.
According to the invention, it is possible to detect an
operation which is performed across the inside and outside of
the display region, on the basis of two pieces of image data
of which the times of imaging are consecutive.
[0008]
In the invention, the detection unit may detect the
indicated position on the basis of the image data obtained by
capturing an image of the imaging region wider than the display
5
region.
According to the invention, it is possible to detect an
operation which is performed across the inside and outside of
the display region by making the imaging region wider than the
display region.
[0009]
In the invention, the generation unit may generate
information indicating that the indicator moves to an outside
of the display region, in association with the third position.
According to the invention, it is possible to detect the
position of movement of the indicator to the outside of the
display region with a good degree of accuracy.
[0010]
In the invention, a first display region, as a display
surface, corresponding to a first image and a second display
region corresponding to a second image may be arranged in the
projection screen, and the generation unit may generate the
position information of the third position, in a case where
the first position is detected inside the first display region,
and the second position is detected inside the second display
region.
According to the invention, it is possible to eliminate
a defect of a process based on an operation which is performed
across the first display region and the second display region.
[0011]
6
In the invention, the generation unit may delete
information indicating that the indicator moves to an outside
of the display region, when the information corresponds to
movement from the first display region to the second display
region in a case where the information is supplied by the
detection unit.
According to the invention, it is possible to reduce the
possibility of the position of movement of the indicator from
the first display region to the second display region being
misidentified.
[0012]
In the invention, the generation unit may generate
information indicating Lhat the indicator moves to the outside
of the display region in association with the third position,
with respect to the first display region.
According to the invention, it is possible to detect the
position of movement of the indicator from the first display
region to the second display region with a good degree of
accuracy.
[0013]
In the invention, the generation unit may generate the
position information of the third position with respect to the
second display region.
According to the invention, it is possible to detect the
position, on the second display region, of movement of the
7
indicator from the first display region with a good degree of
accuracy.
Meanwhile, the invention can be conceived as a projector,
an information processing method and a program, in addition
to an information processing device.
Brief Description of Drawings
[0014]
[Fig. 1] Fig. 1 is a diagram illustrating an entire
configuration of a projection system according to a first
embodiment of the invention.
[Fig. 2] Fig. 2isablockdiagramillustratingahardware
configuration of the projection system according to the
embodiment.
[Fig. 3] Fig. 3 is a block diagram illustrating a
functional configuration of the projection system according
to the embodiment.
[Fig. 4] Figs. 4(a) and 4(b) are flow diagrams
illustrating an interpolation process according to the
embodiment.
[Fig. 5] Figs. 5(a) to 5(e) are diagrams illustrating
an interpolation process relating to a primary screen according
to the embodiment.
[Fig. 6] Figs. 6(a) to 6(d) are diagrams illustrating
an interpolation process relating to a secondary screen
8
I
I
according to the embodiment.
[Fig. 7] Fig. 7 is a diagram illustrating a drawing
process according to the embodiment.
[Fig. 8] Fig. 8 is a block diagram illustrating a
functional configuration of a projection system according to
a second embodiment of the invention.
[Fig. 9] Fig. 9 is a diagram illustrating an
interpolation process according to the embodiment.
[Fig. 10] Figs. 10 (a) to 10 (d) are diagrams illustrating
an interpolation process according to the embodiment.
[Fig. 11] Figs. 11 (a) to 11 (c) are diagrams illustrating
an example of a defect occurring in a drawing process during
a two-screen display.
Description of Embodiments
[ 0015 J
Hereinafter, embodiments of the invention will be
described with reference to the accompanying drawings.
[First Embodiment]
Fig. 1 is a diagram illustrating an entire configuration
of a projection system 1 according to a first embodiment of
the invention. The projection system 1 includes a projector
10-1, a projector 10-2, a personal computer (PC) 20, and an
indicator 30.
The projectors 10-1 and 10-2 are liquid crystal
9
projectors herein, and are disposed so as to be next to each
other in a horizontal direction. The projectors 10-1 and 10-2
are projection-type display devices that project an image onto
a screen 40 on the basis of an image signal which is input from
the PC 20. The projectors 10-1 and 10-2 project a color image
on the basis of an image signal indicating an input image
corresponding to each color component of three primary colors
of, for example, R(Red), G(Green), and B(Blue). The screen
40 is a reflection-type screen herein, and is a projection
screen onto which an image is projected by the projectors 10-1
and 10-2.
[0016]
The projector 10-1 projects a primary screen SCl (first
display region) onto the screen 40. The projector 10-2
projects a secondary screen SC2 (second display region) onto
the screen 40. The primary screen SC1 is a screen corresponding
to a primary monitor in a display method of a multi-monitor,
and the secondary screen SC2 is a screen corresponding to a
secondary monitor in a display method of a multi-monitor.
Therefore, the primary screen SC1 and the secondary screen SC2
are arranged, and thus one screen region SC is displayed as
a whole. In the present embodiment, the projectors 10-1 and
10-2 are assumed to be adjusted in advance so that the side
of the right edge of the primary screen SC1 and the side of
the left edge of the secondary screen SC2 are coincident with
10
each other. The projectors 10-1 and 10-2 function as an
information processing device that executes information
processing relating to a display of the screen region SC.
In the following description, a component having a branch
number of "1" attached to the end of a reference numeral is
a component of the projector 10-1, and a component having a
branch number of "2" attached to the end of a reference numeral
is a component of the projector 10-2.
[0017]
The PC 20 is an information processing device serving
as a signal source (picture source) of an image signal which
is input to the projectors 10-1 and 10-2. The PC 20 is
cormnunicably connected to each of the projector,; 10-1 and 10-2.
The PC 20 associates the projector 10-1 as a display device
that displays the primary screen, and associates the projector
10-2 as a display device that displays the secondary screen,
on the basis df, for example, a function of an operating system
(OS).
Meanwhile, in the example of Fig. 1, the PC 20 is connected
to each of the projectors 10-1 and 10-2 in a wired manner, but
may be wirelessly connected thereto, and a connection system
or a communication system particularly does not matter.
[0018]
The indicator 30 is a pen-type device (operation device)
herein, and is used by a user U in order to indicate a position
11
on the screen 4 0. The indicator 30 is used by the user U, for
example, in order to perform an operation for overlapping the
screen region SC to handwrite characters or figures. In
addition, the indicator 30 is used by the user U in order to
perform an operation for selecting an operable object included
in the screen region SC. The indicator 30 may be an operation
device having other shapes such as a wand shape, without being
limited to a pen type. In addition, the indicator 30 may not
an operation device, and may be substituted by the hand, finger
or the like of the user U.
[0019]
Fig. 2 is a block diagram illustrating a hardware
configuration of Lhe projection system 1. The projectors 10-1
and 10-2 executes functions depending on whether the device
displays the primary screen or displays the secondary screen,
but the hardware configuration thereof is common to the
projectors. Hereinafter, the hardware configuration of the
projector 10-1 will be representatively described.
The projector 10-1 includes a central processing unit
(CPU) 11, a read only memory (ROM) 12, a random access memory
(RAM) 13, an operating unit 14, an image processing unit 15,
a projection unit 16, a camera unit 17, and an interface 18.
The CPU 11 is a microcomputer that controls each unit
of the projector 10-1 by reading out and executing a program
stored in storage unit such as the ROM 12 through the RAM 13.
12
The operating unit 14 is operation unit including an operator
(for example, physical key) for performing the turn-on/off of
a power supply of the projector 10-1 or various types of
operations.
[0020]
The image processing unit 15 includes an image processing
circuit such as, for example, an application specific
integrated circuit (ASIC), and takes charge of image processing
which is performed by the projector 10-1. The image processing
unit 15 performs predetermined image processing such as
resizing or keystone correction on, for example, an image
signal which is input from the PC 20 to the interface 18, in
accordance with control of the CPU 11.
[0021]
The projection unit 16 projects an image (primary screen
SC1) onto the screen 40 on the basis of an image signal after
image processing performed by the image processing unit 15.
The projection unit 16 includes a light source 161, a liquid
crystal panel 162, an optical system 163, a light source control
unit 164, a panel drive unit 165, and an optical system drive
unit 166. The light source 161 is, for example, a solid-state
light source including a light emitting diode (LED) or a
semiconduqtor diode, and emits light to the liquid crystal
panel 162. The liquid crystal panel 162 is, for example, a
transmission-type liquid crystal panel, and is a light
13
modulator that modulates light incident from the light source
161. The liquid crystal panel 162 is provided corresponding
to each color of three primary colors of RGB. The optical
system 163 includes, for example, a lens or a drive circuit
for lens adjustment, and expands light (image light) modulated
by the liquid crystal panel 162 to project the expanded light
onto the screen 40. The light source control unit 164 drives
the light source 161 in accordance with control of the CPU 11.
The panel drive unit 165 drives the liquid crystal panel 162
on the basis of an image signal which is supplied from the CPU
11. The optical system drive unit 166 drives a drive circuit
of the optical system 163 in accordance with control of the
CPU 11.
[0022]
The camera unit 17 includes an image sensor (for example,
CMOS sensor or CCD sensor), and captures an image of the screen
40. In the present embodiment, the camera unit 17 captures
an image of the screen 40 at the angle of view wider than a
range onto which the projection unit 16 projects an image, in
accordance with control of the CPU 11. That is, in the camera
unit 17, a region wider than the primary screen SCl inclusive
of the entire primary screen SCl is set to an imaging region.
The interface 18 is an interface for connection to the PC 20.
The interface 18 realizes a function relating to the input and
output of data between the projector 10-1 and the PC 20.
14
[ 0023 J
The projector 10-2 projects the secondary screen SC2 onto
the screen 40 through the projection unit 16. In addition,
in the camera unit 17 of the projector 10-2, a region wider
than the secondary screen SC2 inclusive of the entire secondary
screen SC2 is set to an imaging region.
[0024]
The PC 20 includes a CPU 21, a ROM 22, a RAM 23, an
operating unit 24, an image processing unit 25, a display unit
26, a storage unit 27, an interface 28, and an interface 29.
The CPU 21 is a microcomputer that controls each unit
of the PC 20 by reading out and executing a program stored in
storage unit such as the ROM 22 through the RAM 2 3. The
operating unit 24 is operation unit including a plurality of
operators (for example, physical keys) such as a keyboard or
a numeric keypad. The image processing unit 25 includes an
image processing circuit such as, for example, an ASIC, and
performs various types of image processing in accordance with
control of the CPU 21. The display unit 26 is for example,
a liquid crystal display, and displays various types of image
(screens) in accordance with control of the CPU 21. The storage
unit 27 includes a recording medium such as, for example, a
hard disk, and stores various types of data. The storage unit
27 may be any of internal storage unit, external storage unit
or a combination thereof. The interface 28 is an interface
15
for connection to the projector 10-1. The interface 2 8
realizes a function relating to the input and output of data
between the projector 10-1 and the PC 20. The interface 29
is an interface for connection to the projector 10-2. The
interface 29 realizes a function relating to the input and
output of data between the projector 10-2 and the PC 20.
[0025]
As shown in Fig. 1, in a case where a two-screen display
is performed using two projectors, there is the possibility
of a defect of a process described below occurring. Figs. llA
to llC are diagrams illustrating an example of a defect
occurring in a drawing process for displaying a handwritten
image which is drawn by the user U.
As shown in Fig. ll(a), a case is considered in which
the user U uses the indicator 30 to perform an operation for
drawing a line extending across the primary screen SCl and the
secondary screen SC2. Originally, a line shown in Fig. ll(a)
is to be displayed by a drawing process, but the line may be
interrupted in the vicinity of a screen boundary between the
primary screen SCl and the secondary screen SC2. Subsequently,
a reason of a line being interrupted in a drawing process will
be described.
[0026]
Generally, the projector repeatedly detects the position
(typically, tip position) of the indicator 30, and supplies
16
coordinate information of the detected position to a PC or the
like that performs a drawing process. The coordinate
information which is supplied by the projector is position
information obtained by expressing the position of the
indicator 30 in a coordinate form, and specifically includes
''push information" and ''pop information''. The "push
information" is coordinate information indicating the
detected position of the indicator 30. In the drawing process,
a line is drawn by linking positions indicated by two pieces
of coordinate information detected temporally continuously
with a line. The "pop information" is coordinate information
which is supplied in a case where the position of the indicator
30 is detected in the inner side of a display region of a
projected image, and then the position of the indicator 30 is
not detected in the inner side of the display region. The pop
information indicates, for example, the· position of the
indicator 30 which is finally detected in the inner side of
the display region.
[0027]
Hereinafter, a defect of a drawing process occurring in
each of the primary screen SC1 and the secondary screen SC2
will be described.
As shown in Fig. 11 (b-1) , in a case where push information
PuA of a position within the primary screen SC1 is detected,
and then the indicator 30 moves to the outside of the primary
17
screen SC1, pop information Po of a position indicated by the
push information PuA is supplied, and the supply of coordinate
information relating to the primary screen SC1 is terminated.
For this reason, a line is not drawn in a region interposed
between the position indicated by the push information PuA and
the screen boundary of the right edge of the primary screen
SCl.
As shown in Fig. 11(b-2), in a case where the indicator
30 moves to the inside of the secondary screen SC2 and the pieces
of push information PuB and PuC are sequentially supplied, a
line that links positions indicated by the pieces of push
information PuB and PuC is drawn. However, since the push
information which is initially supplied within the secondary
screen SC2 is the push information PuB, a line is not drawn
in a region interposed between the screen boundary of the left
edge of the secondary screen SC2 and the position indicated
by the push information PuB.
[0028]
From the above reason, in a case where the operation of
the user U described in Fig. 11(a) is performed, the drawing
process shown in Fig. 11 (c) is performed, and a line is
interrupted in the vicinity of the boundary between the primary
screen SC1 and the secondary screen SC2.
In the projection system 1, the function for eliminating
the defect of the drawing process described above is realized
18
in the projectors 10-1 and 10-2.
[0029]
Fig. 3 is a block diagram illustrating a functional
configuration of the projection system 1.
The PC 20 includes image signal supply unit 201 and 202,
detection unit 203, conversion unit 204, execution unit 205,
and display unit 2 0 6. The projector 10-1 includes image signal
input unit 101-1, projection unit 102-1, imaging unit 103-1,
detection unit 104-1, generation unit 105-1, and output unit
106-1. The projector 10-2 includes image signal input unit
101-2, projection unit 102-2, imaging unit 103-2, detection
unit 104-2, generation unit 105-2, and output unit 106-2.
[0030]
In the PC 20, the image signal supply unit 201 is unit
for supplying an image signal indicating an image (picture)
to the projector 10-1. The image signal supply unit 202 is
unit for supplying an image signal indicating an image
(picture) to the projector 10-2. The image signal supply unit
201 and 202 supply, for example, an image signal corresponding
to each color component of three primary colo~s of R, G, and
B. The image signal supply unit 201 and 202 reproduce an image
signal on the basis of data which is read out from, for example,
an internal recording medium, such as a hard disk device, of
the PC 20, or an external recording medium exemplified by data
recording media such as a digital versatile disk (DVD), and
19
supply the reproduced image signal. The image signal is, for
example, an image signal indicating an image of a document file
exemplified by a material for presentation or an image
exemplified by a moving image or a still image, but an image
indicated by the image signal particularly does not matter.
The image signal supply unit 201 is realized by the CPU 21,
the storage unit 27 and the interface 28. The image signal
supply unit 202 is realized by the CPU 21, the storage unit
27 and the interface 29.
[0031]
In the projector 10-1, the image signal input unit 101-1
supplies an image signal indicating an input image, input from
Lhe PC 20, to the projection unit 102-1. In the projector 10-2,
the image signal input unit 101-2 supplies an image signal
indicating an input image, input from the PC 20, to the
projection unit 102-2. The image signal input unit 101-1 and
101-2 are realized by, for example, the CPU 11, the image
processing unit 15 and the interface 18.
[0032]
The projection unit 102-1 is unit for projecting the
primary screen SC1 onto the screen 40 on the basis of the image
signal supplied from the image signal input unit 101-1. The
projection unit 102-2 is unit for projecting the secondary
screen SC2 onto the screen 40 on the basis of. the image signal
supplied from the image signal input unit 101-2. The
20
I'
i
projection unit 102-1 and 102-2 are realized by, for example,
the CPU 11, the image processing unit 15 and the projection
unit 16.
[0033]
The imaging unit 103-1 captures an image of an imaging
region including the primary screen SCl of the screen 40 in
a predetermined period, generates image data indicating the
captured image (hereinafter, referred to as ''imaging data''),
and supplies the generated data to the detection unit 104-1.
The imaging unit 103-2 captures an image of an imaging region
including the secondary screen SC2 of the screen 40 in a
predetermined period, generates imaging data, and supplies the
generated data to Lhe delection unit 104-2. The imaging period
of the imaging unit 103-1 and the imaging period of the imaging
unit 103-2 are, for example, equal to or higher than 50 Hz and
equal to or lower than 60 Hz. In a case where the indicator
30 indicates a position on the screen 40, the imaging unit 103-1
and 103-2 generate imaging data indicating a captured image
including the indicator 30. The imaging unit 103-1 and 103-2
are realized by, for example, the CPU 11, the image processing
unit 15 and the camera unit 17.
[0034]
The detection unit 104-1 and 104-2 are unit for
repeatedly detecting the position of the indicator 30
indicating a position on the screen 40, and supplying
21
coordinate information of the detected position. The
detection unit 104-1 and 104-2 analyze the imaging data
supplied for each imaging period, and detect the position of
the indicator 30, more specifically, the position of the tip
of the indicator 30. In a case where the position of the
indicator 30 is detected from the imaging data, the detection
unit 104-1 and 104-2 supply push information as coordinate
information of the position. In addition, in a case where the
indicator 30 moves to the outside of a display region, the
detection unit 104-1 and 104-2 supply pop information as
coordinate information of a position at which the indicator
30 is not detected.
Meanwhile, each of the pieces of coordinate information
which are supplied by the detection unit 104-1 and 104-2
indicates coordinates of the coordinate system of the input
images of the projectors 10-1 and 10-2. Therefore, the
detection unit 104-1 and 104-2 perform a process of converting
the coordinate system of the coordinates obtained from the
imaging data into the coordinate system of the input image,
and then supply coordinate information. The detection unit
104-1 and 104-2 are realized by, for example, the CPU 11 and
the image processing unit 15.
[0035]
The generation unit 105-1 performs an interpolation
process of interpolating the coordinate information on the
22
basis of the coordinate information supplied by the detection
unit 104-1. The generation unit 105-2 performs an
interpolation process on the basis of the coordinate
information supplied by the detection unit 104-2. The
generation unit 105-1 and 105-2 are realized by, for example,
the CPU 11 and the image processing unit 15.
[0036]
Here, the details of the interpolation process executed
in the projector 10-1 and the interpolation process executed
in the projector 10-2 will be described. Hereinafter, as shown
in Fig. 11 (a), a description is given by taking an example of
an interpolation process in which a straight line in a
horizontal direcLlon across the primary screen SCl and the
secondary screen SC2 is drawn by the user U, using the indicator
30.
[0037]

Fig. 4 (a) is a flow diagram illustrating an interpolation
process relating to the primary screen SC1 which is executed
in the projector 10-1. Figs. 5 (a) to 5 (e) are diagrams
illustrating the interpolation process relating to the primary
screen SC1. Figs. 5(a) to 5(e) show a line drawn by linking
positions indicated by push information, for convenience of
description. In addition, in Figs. 5(a) to 5 (e), the imaging
region of the imaging unit 103-1 is denoted by T1.
23
[0038]
When the position of the indicator 30 is detected within
the primary screen SCl, the detection unit 104-1 supplies push
information of the detected position to the generation unit
105-1 (step Sl-1). The generation unit 105-1 determines
whether the indicator 30 (that is, position indicated by the
indicator 30) has moved from the inside of the primary screen
SCl to the outside of the primary screen SCl, on the basis of
the supplied push information (step S2-1). The movement of
the indicator 30 to the outside of the primary screen SCl,
herein, refers to the movement of the indicator 30 to the inside
of the secondary screen SC2.
[0039]
As shown in Fig. 5 (a) , in a case where the detection unit
104-1 continuously supplies pieces of push information Pul and
Pu2 within the primary screen SCl, the generation unit 105-1
determines ''NO'' in step S2-1, and passes the supplied push
information to supply the information to the output unit 106-1
(step S3-1).
I I
i
l
Subsequently, as shown in Fig. 5(b), it is assumed that
the detection unit 104-1 continuously supplies pieces of push
information Pu3 and Pu4. In this case, the detection unit 104-1
supplies the push information Pu3 to the output unit 106-1 in
step S3-l, and then determines that the indicator 30 has moved
from the inside of the primary screen SCl to the outside of
24
the primary screen SCl (step S2-1; YES). As shown in Fig. 5 (b),
the imaging region Tl is a region wider than the primary screen
SCl. Therefore, the detection unit 104-1 can supply push
information of a position located outside the primary screen
SCl.
The imaging region Tl is set so as to be capable of
supplying push information outside the primary screen SCl, for
example, regardless of the moving speed of the indicator 30.
The relative size of the imaging region Tl with respect to the
primary screen SCl may be adjusted on the basis of calculation,
experiment or the like. According to the knowledge of the
inventors, a region in which the primary screen SCl is expanded
approximately 10% in each direction suffices [or the imaging
region Tl.
[0040]
After "YES'' is determined in step S2-1, the generation
unit 105-1 generates push information on the screen boundary
of the primary screen SCl (step S4-1). Here, the generation
unit 105-1 generates push information Pu34 at a position shown
in Fig. 5(c). Specifically, the generation unit 105-1
generates the push information Pu34 at a position (third
position) where the screen boundary and a line segment that
links a position (first position) indicated by the push
information Pu3 to a position (second position) indicated by
the push information Pu4 intersect each other.
25
[0041]
Next, the generation unit 105-1 generates pop
information on the screen boundary of the primary screen SCl
(step S5-l). Here, the generation unit 105-1 generates pop
information Pol at a position shown in Fig. 5 (d). Specifically,
the generation unit 105-1 generates the pop information Pol
at the same position as the position at which the push
information Pu34 is generated. In step S5-l, the generation
unit 105-1 deletes the pop information supplied from the
detection unit 104-1, and replaces this information with newly
generated pop information. Through the process of step S5-l,
at a position on the screen boundary of the primary screen SCl,
the movement of the indicator 30 to the outside of the primary
screen SCl can be specified on the basis of the pop information
Pol.
[0042]
The generation unit 105-1 supplies the push information
generated in step S4-l and the pop information generated in
step S5-l to the output unit 106-1 (step S6-l). Since the push
information Pu34 is generated on the screen boundary, as shown
in Fig. 5(e), a line drawn by linking positions indicated by
the ·push information arrives up to the screen boundary of the
primary screen SCl.
[0043]

26
Fig. 4 (b) is a flow diagram illustrating an interpolation
process relating to the secondary screen SC2 executed in the
projector 10-2. Figs. 6(a) to 6(d) are diagrams illustrating
the interpolation process relating to the secondary screen SC2.
Figs. 6 (a) to 6 (d) show a line drawn by linking positions
indicated by push information, for convenience of description.
In addition, in Figs. 6(a) to 6(d), the imaging region of the
imaging unit 103-2 is denoted by T2. The imaging region T2
is set on the basis of the same viewpoint as that of the imaging
region Tl.
[0044]
When the position of the indicator 30 is detected, the
detection unit 104-2 supplies push information of the detected
position to the generation unit 105-2 (step S1-2). The
generation unit 105-2 determines whether the indicator 30 has
moved from the outside of the secondary screen SC2 to the inside
of the primary screen SC1, on the basis of the supplied push
information (step 82-2). The movement of the indicator 30 to
the inside of the secondary screen SC2, herein, refers to the
movement of the indicator 30 from the inside of the primary
screen SC1 to the inside of the secondary screen SC2.
[0045]
As shown in Fig. 6(a), it is assumed that the detection
unit 104-2 continuously supplies pieces of push information
PuS and Pu6. The push information PuS indicates, for example,
27
the same position as the push information Pu4, but may indicate
a different position. Here, the generation unit 105-2
determines ''YES'' in step S2-2.
Meanwhile, as shown in Fig. 6 (a), since the imaging
region T2 is a region wider than the secondary screen SC2, the
detection unit 104-2 can supply push information outside the
secondary screen SC2.
[0046]
In a case where ''YES" is determined in step S2-2, the
generation unit 105-2 generates push information on the screen
boundary of the secondary screen SC2 (step S3-2). Here, the
generation unit 105-2 generates push information Pu56 at a
position shown in Fig. 6(b). Specifically, the generation
unit 105-2 generates the push information Pu56 at a position
(third position) where the screen boundary and a line segment
that links a position (first position) indicated by the push
information PuS to a position (second position) indicated by
the push information Pu6 intersect each other.
[0047]
Next, the generation unit 105-2 supplies the push
information supplied in step S1-2 and the push information
generated in step S3-2 to the output unit 106-2 (step S4-2).
As shown in Fig. 6(c), since the push information Pu56 is
generated on the screen boundary, a line is drawn which extends
from a position indicated by the push information Pu56 on the
28
screen boundary of the secondary screen SC2 to a position
indicated by the push information Pu6 within the secondary
screen SC2.
[0048]
In step 82-2, in a case where the indicator 30 does not
move from the outside of the secondary screen SC2 to the inside
of the secondary screen SC2, that is, it is determined that
the indicator 30 moves to the inside of the secondary screen
SC2 (step S2-2; NO), the generation unit 105-2 passes the push
information supplied in step 81-2, and supplies the information
to the output unit 106-1 (step S5-2). As shown in Fig. 6(d),
in a case where the detection unit 104-2 supplies push
information Pu7 subsequently to the push information Pu6, the
generation unit 105-2 supplies the push information Pu7 to the
output unit 106-1.
The above is the description of the interpolation process
executed in the projector 10-1 and the projector 10-2.
[0049]
Referring back to Fig. 3, the output unit 106-1 outputs
the coordinate information generated by the generation unit
105-1 to the PC 20. The output unit 10 6-2 outputs the
coordinate information generated by the generation unit 105-2
to the PC 20. The output unit 106-1 and 106-2 are realized
by, for example, the CPU 11 and the interface 18.
Meanwhile, in the present embodiment, the generation
29
unit 105-1 executes a function of passing the push information
supplied from the detection unit 104-1 to the output unit 106-1,
but the coordinate information may be supplied from the
detection unit 104-1 to the output unit 106-1 without going
through the generation unit 105-1. Likewise, the coordinate
information may be supplied from the detection unit 104-2 to
the output unit 106-2 without going through the generation unit
105-2.
[0050]
In the PC 20, the detection unit 203 is unit for repeatedly
detecting a position indicated by the indicator 30, and
supplying coordinate information of the detected position.
Specifically, the detection unit 203 detects a position
indicated by the coordinate information by the coordinate
information output by the output unit 106-1 and 106-2 being
input. The detection unit 203 supplies the coordinate
information of the detected position to the conversion unit
204. The detection unit 203 is realized by the CPU 21 and the
interfaces 28 and 29.
[0051]
The conversion unit 204 is unit for performing conversion
into operation information on the basis of the coordinate
information supplied from the detection unit 203. The
conversion unit 204 performs conversion into operation
information which is allocated a human interface device (HID)
30
;i
"
III
device class specified by, for example, a universal serial bus
(USB) standard. This operation information is a command
described in a form capable of being construed by the HID. The
conversion unit 204 supplies the converted operation
information to the execution unit 205. The conversion unit
204 is realized by executing, for example, a dedicated device
driver.
[0052]
The execution unit 205 is unit for executing a
predetermined process on the basis of the operation information
supplied from the conversion unit 204. The execution unit 205
performs a drawing process of drawing characters or figures
handwritten by the user U at a position specified by opeiation
information on an image indicated by an image signal which is
supplied by the image signal supply unit 201 and 202. The
execution unit 205 supplies an image signal after the drawing
process to the image signal supply unit 201 and 202 and the
display unit 206. The execution unit 205 is realized by
executing, for example, a dedicated device driver.
[0053]
The display unit 206 is unit for displaying an image on
the basis of the image signal supplied from the execution unit
205. The image displayed by the display unit 206 is observed
by a user (user U or other users) who operates the PC 20. The
display unit 206 is realized by the cooperation of the CPU 11
31
and the display unit 26.
[0054]
Fig. 7 is a diagram illustrating the screen region SC
which is displayed in accordance with the interpolation process
described in Figs. 5(a) to 6(d). As shown in Fig. 7, since
the push information Pu34 is generated on the primary screen
SCl, and the push information Pu56 is further generated on the
secondary screen SC2, a line extending across the screen
boundary between the primary screen SCl and the secondary
screen SC2 is displayed without being interrupted.
[0055]
In the above description of the projection system 1, a
case has been described in which a line extending from the
primary screen SCl across the secondary screen SC2 is drawn.
However, even in a case where a line extending from the
secondary screen SC2 across the primary screen SCl is drawn,
the interpolation process is performed, and thus a line
extending across the screen boundary is displayed without being
interrupted. In addition, even in a case where the indicator
30 moves from the primary screen SCl or the secondary screen
SC2 to the outside of the screen region SC, the interpolation
process is performed, and thus the occurrence of a defect of
a line not being drawn in the vicinity of the screen boundary
of the screen region SC is eliminated.
[0056]
32
Further, according to the projection system 1, a defect
of processes other than the drawing process may also be
eliminated. For example, a case is considered in which the
user U performs a drag operation for moving an operable object
such as an icon across the primary screen SC1 and the secondary
screen SC2. This drag operation is an operation for moving
the indicator 30 in a state where the indicator is brought into
contact with an object on the screen 40.
In this case, after the object moves up to the screen
boundary of the primary screen SC1, the PC 20 recognizes that
the object is temporarily dropped on the basis of the pop
information, but recognizes that the object is selected again
on the basis of the push information generated on Lhe ~creen
boundary of the secondary screen SC2, and receives a drag
operation again. Therefore, the user U can perform the drag
operation for moving the object across the screen boundary
without a sense of discomfort. That is, the pop information
is generated on the screen boundary by the projector 10-1, and
thus a defect of a process based on the pop information is also
eliminated.
[0057]
[Second Embodiment]
Next, a second embodiment of the invention will be
described.
The projection system 1 of the present embodiment is
33
different from that in the aforementioned first embodiment,
in that the PC 20 rather than the projectors 10-1 and 10-2
performs an interpolation process. Therefore, the projectors
10-1 and 10-2 of the present embodiment may detect the position
of the indicator 30, using the same algorithm as that of a
projector having a configurat~on of the related art.
Hereinafter, the same components or functions as those in the
aforementioned first embodiment are denoted by the same
reference numerals and signs as those in the aforementioned
first embodiment. In addition, the entire configuration and
the hardware configuration of the projection system 1 of the
present embodiment may be the same as those in the
aforementioned first embodiment, and thus the description
thereof will not be given.
[0058]
Fig. 8 is a bloc.k diagram illustrating a functional
configuration of the projection system of the present
the present embodiment are the same as those in the I
I
embodiment. The functions of the projectors 10-1 and 10-2 of
aforementioned first embodiment, except that the generation
unit 105-1 and 105-2 are not included. That is, the output
unit 106-1 outputs the coordinate information supplied by the
detection unit 104-1 to the PC 20. The output unit 106-2
outputs the coordinate information supplied by the detection
unit 104-2 to the PC 20.
34
[0059]
The PC 20 includes generation unit 207 in addition to
the functions described in the aforementioned first
embodiment.
In PC 20, the detection unit 203 is unit for repeatedly
detecting a position indicated by the indicator 30 on the basis
of each of the pieces of coordinate information which are input
from the output unit 106-1 and the output unit 106-2, and
supplying coordinate information of the detected position to
the generation unit 207.
The generation unit 207 performs an interpolation
process on the basis of the coordinate information supplied
by the detection unit 203. The generation unit 207 is realized
by, for example, the CPU 21 and the image processing unit 25.
[0060]
Fig. 9 is a flow diagram illustrating an interpolation
process executed in the PC 20. Figs. 10 (a) to 10 (d) are
diagrams illustrating the interpolation process. Hereinafter,
as shown in Fig. ll(a), a description is given by taking an
example of a process in which a straight line in a horizontal
direction across the primary screen SCl and the secondary
screen SC2 is drawn by the user U, using the indicator 30. Figs.
10 (a) to 10 (d) show a line drawn by linking positions indicated
by push information, for convenience of description. In
addition, in the present embodiment, the imaging region of the
35
imaging unit 103-1 may be a region equal to the primary screen
SC1. In addition, the imaging region of the imaging unit 103-2
may be a region .equal to the secondary screen SC2.
[0061]
When the position of the indicator 30 is detected, the
detection unit 203 supplies push information of the detected
position to the generation unit 207 (step S11). The generation
unit 207 determines whether the indicator 30 has moved from
the primary screen SC1 to the secondary screen SC2, on the basis
of the supplied push information (step S12).
[0062]
As shown in Fig. 10(a), a case is considered in which
the detection unit 203 detects push information Pua, pop
information Po and pieces of push information Pub and Puc in
order. The push information Pua indicates a pcisi tion detected
within the primary screen SC1. The pop information Po is pop
information supplied by the indicator 30 moving to the outside
of the primary screen SC1. The pieces of push information Pub
and Puc indicate positions detected within the secondary screen
SC2. The generation unit 207 determines "YES'' in the process
of step S12, for example, in a case where the push information
of the secondary screen SC2 is supplied subsequently to the
push information of the primary screen SCl being supplied.
[0063]
When "YES" is determined in the process of step S12, the
36
I
generation unit 207 determines whether an interval between
positions indicated by two pieces of push information
specifying movement from the primary screen SCl to the
secondary screen SC2 is an interval based on the resolution
(for example, imaging period) of the imaging unit 103-1 and
103-2 (step Sl3). In a case where a continuous line is drawn
by the indicator 30, an interval between the position indicated
by the push information Pua and the position indicated by the
push information Pub is supposed to fall within a range of a
predetermined distance based on the resolution of the imaging
unit 103-1 and 103-2.
[0064]
In a case where it is determined that the interval between
the position indicated by the push information Pua and the
position indicated by the push information Pub is an interval
based on the resolution of the imaging unit 103-1 and 103-2
(step Sl3; YES), the generation unit 207 generates push
information on the screen boundary of the primary screen SCl
(step S14). Here, the generation unit 207 generates push
information Pusl shown in Fig. 10 (b). Specifically, the
generation unit 207 generates the push information Pusl at a
position (third position) where the screen boundary and a line
segment that links the position (first position) indicated by
the push information Pua to the position (second position)
indicated by the push information Pub intersect each other.
37
I
The intention of generating this push information is the same
as that in step S4-l of the aforementioned first embodiment.
[0065]
Next, the generation unit 207 generates pop information
on the screen boundary of the primary screen SCl (step Sl5).
Here, the generation unit 207 generates pop information Po2
at a position of the screen boundary shown in Fig. 10 (c). The
generation unit 207 generates the pop information Po2 at the
same position as the position indicated by the push information
Pusl. In step Sl5, the generation unit 207 deletes the pop
information supplied from the detection unit 203, and replaces
the information with newly generated pop information. The
intention of generating this pop informaLlor1 is the same as
that in step SS-1 of the aforementioned first embodiment.
[ 00 66]
Next, the generation unit 207 generates push information
on the screen boundary of the secondary screen SC2 (step Sl6).
Here, the generation unit 207 generates push information Pus2 i
~--
at a position on the screen boundary shown in Fig. lO(d). L
Specifically, the generation unit 207 generates the push
information Pus2 at a position (third position) where the
screen boundary and a line segment that links the position
(first position) indicated by the push information Pua and the
position (second position) indicated by the push information
Pub intersect each other. The intention of generating this
38
push information is the same as that in step S3-2 of the
aforementioned first embodiment.
[0067]
The generation unit 207 supplies the push information
supplied from the detection unit 203 and the push information
and the pop information generated by the generation unit 207
to the conversion unit 20 4 (step S1 7) . Functions executed by
the conversion unit 204 and the execution unit 205 are the same
as those in the aforementioned embodiment, and thus the
description thereof will not be given.
Meanwhile, in the present embodiment, the generation
unit 207 executes a function of passing the push information
supplied from the detecLlon ur1it 203, but coordinate
information may be supplied from the detection unit 203 to the
conversion unit 204 without going through the generation unit
207.
[0068]
According to the projection system 1 of the second
embodiment described above, even in a case where the projectors
10-1 and 10-2 do not have a function of performing an
interpolation process, it is possible to eliminate a defect
of a process similarly to the aforementioned first embodiment.
Thus, an application program for realizing the generation unit
207 is installed on the PC 20 in a state where the projectors
10-1 and 10-2 are configured as projectors having a
39
configuration of the related art, and thus it is also possible
to eliminate a defect of a process similarly to the
aforementioned first embodiment.
Further, according to the projection system l of the
present embodiment, the imaging regions of the projectors 10-l
and 10-2 may not be made larger than the display region of a
projected image.
[0069]
[Modification Examples]
The invention can be carried out in different forms from
those in the aforementioned embodiment. In addition,
respective modification examples
appropriately combir1ed.
shown below may be
In the projection system l of each of the aforementioned
embodiments, two display regions are displayed and one screen
region SC is displayed using two projectors, but three or more
display regions may be displayed and one screen region may be
displayed using three or more projectors. In the
interpolation process in this case, a process may be performed
in which one of two display regions next to each other is
estimated as the primary screen of the aforementioned first
embodiment, and the other is estimated as the secondary screen
of the aforementioned first embodiment.
In addition, in the invention, a plurality of display
regions may be arranged in a vertical direction or other
40
directions without being arranged in a horizontal direction.
[0070]
Further, even in a case where one display region is
displayed using one projector, a defect of a process is
eliminated by an action described in each of the aforementioned
embodiments. As can be seen from Figs. 5(e) or 6(d), since
a line is displayed by a drawing process up to a position closer
to the screen boundary by the interpolation process being
performed, a drawn line is not interrupted as compared with
a case where the interpolation process is not performed.
[0071]
In the projection system 1 of each of the aforementioned
embodiments, the push informaLion io; generated on the screen
boundary in the interpolation process, but the push information
may be generated at a position between the first position
indicated by one piece of push information and the second
position indicated by one piece of push information supplied
thereafter. In this case, even in a case where a line does
not arrive on the screen boundary, the interruption of a line
is not likely to be conspicuous as compared with a case where
the interpolation process is not performed.
[0072]
In the projection system 1 of each of the aforementioned
embodiments, the movement of the indicator 30 across the inside
and outside of the display region is detected on the basis of
41
two pieces of imaging data of which the imaging times are
consecutive. Without being limited to this example, in the
projection system 1, the movement of the indicator 30 across
the inside and outside of the display region may be detected
on the basis of a plurality of pieces of imaging data of which
the imaging times are not consecutive.
[0073]
In the projection system 1 of the aforementioned second
embodiment, as is the case with the aforementioned first
embodiment, the imaging regions of the projectors 10-1 and 10-2
may be made wider than the display region of an image to be
projected. In this case, the same interpolation process as
those in the projectors 10-1 and 10-2 is executed by the CPU
21 of the PC 20.
[0074]
Some of the components or operations of the projection
system 1 of each of the aforementioned embodiments may be
omitted. For example, the generation of the pop information
in the interpolation process may be omitted in a case where !
ja
defect of a process based on the pop information does not
occur, or the like. In addition, in the projection system 1
of each of the aforementioned embodiments, the push information
is generated on each of the primary screen SC1 and the secondary
screen SC2 in the interpolation process. In a case where a
defect does not occur in a process such as the drawing process,
42
the push information may be generated on either the primary
screen SCl or the secondary screen SC2 in the projection system
1. In addition, the drawing process based on the generated
push information or pop information may be performed by the
projectors 10-1 and 10-2.
[0075]
A method for causing the projectors 10-1 and 10-2 to r :l
detect a position indicated by the indicator 30 may not be a ' I
method of using the imaging unit. For example, insofar as a I function of outputting information based on a position detected
by the indicator 30 is included by the adoption or the like
of a technique such as a touch panel or a light curtain, the
projectors 10-1 and 10-2 may not include the imaging unit.
In addition, the projectors 10-1 and 10-2 may acquire
imaging data from external imaging unit, and detect the
position of the indicator 30.
[ 0076]
The PC 20 may be replaced by information processing
devices other than a PC, such as a smartphone or a tablet-type
computer, which have an information processing function.
[0077]
The projectors 10-1 and 10-2 are not limited to devices
including a plurality of liquid crystal panels (light valves)
corresponding to each color component of three primary colors.
The projectors 10-1 and 10-2 may include a single liquid crystal
43
panel. In this case, a color corresponding to each pixel is
set using an optical filter or the like. In addition, the
liquid crystal panel may be a reflection type without being
limited to a transmission type. In addition, the projectors
10-1 and 10-2 are not limited to a liquid crystal-type projector,
and may be a projector using, for example, a digital mirror
device ( DMD) , a liquid crystal on silicon (LCOS) , or the like.
[0078]
In the aforementioned embodiment, each function realized
by the projectors 10-1 and 10-2 or the PC 20 can be realized
by a combination of a plurality of programs, or realized by
connection of a plurality of hardware resources. In a case
where the functions (for example, detection unit 104-1 and
generation unit 105-1, detection unit 104-2 and generation unit
105-2) of the projectors 10-1 and 10-2, or the functions (for
example, detection unit 203 and generation unit 207) of the
PC 2 0 are realized using a program, this program may be provided
in a state where the program is stored in a computer readable
recording medium such as a magnetic recording medium (such as
a magnetic tape, a magnetic disc (hard disk drive (HOD), or
flexible disk (FD))), an optical recording medium (such as an
optical disc), a magneto-optical recording medium, or a
semiconductor memory, and may be delivered through a network.
In addition, the invention can also be ascertained as an
information processing method.
44
I
I
I- I
Reference Signs List
[0079]
1: projection system, 10-1, 10-2: projector, 101-1,
101-2: image signal input means, 102-1, 102-2: projection means,
103-1, 103-2: imaging means, 104-1, 104-2: detection means,
105-1, 105-2: generation means, 106-1, 106-2: output means,
11, 21: CPU, 12, 22: ROM, 13, 23: RAM, 14, 24: operating unit,
15, 25: image processing unit, 16: projection unit, 17: camera
unit, 18, 28, 29: interface, 201, 202: image signal supply means,
203: detection means, 204: conversion means, 205: execution
means, 206: display means, 207: generation means, 26: display
unit, 27: storage unit

We claim:
[Claim 1]
An information processing device comprising:
detection means . for repeatedly detecting a position
indicated by an indicator with respect to a projection screen
onto which an image is projected, and supplying position
information of the detected position; and
generation means for generating position information of
a third position located between a first position located
inside of a display region of the image in the projection screen
and a second position located outside of the display region,
in a case where the detection means detects the first position,
and then detects the second position.
[Claim 2]
The information processing device according to claim 1,
wherein the detection means detects the indicated position on
the basis of image data obtained by capturing an image of an
imaging region including the display region in a predetermined
period, and
the generation means specifies the first position and
the second position on the basis of fwo pieces of the image
data of which times of the imaging are different from each other
by one period, and generates the position information of the
46
third position.
[Claim 3] (Amended)
The information processing device according to claim 2,
wherein the detection means detects the indicated position on
the basis of the image data obtained by capturing an image of
the imaging region wider than the display region.
[Claim 4]
The information processing device according to claim 1,
wherein the generation means generates information indicating
that the indicator moves to an outside of the display region,
in association with the third position.
[Claim 5]
The information processing device according to claim 1,
wherein a first display region corresponding to a first image
and a second display region corresponding to a second image
are arranged in the projection screen, and
the generation means generates the position information
of the third position, in a case where the first position is
detected inside the first display region, and the second
position is detected inside the second display region.
[Claim 6] (Amended)
47
The information processing device according to claim 5,
wherein the generation means deletes information indicating
that the indicator moves to an outside of the first display
region, when the information corresponds to movement from the
first display region to the second display region in a case
where the information is supplied by the detection means.
[Claim 7] (Amended)
The information processing device according to claim 6,
wherein the generation means generates information indicating
that the indicator moves to the outside of the first display
region in association with the third position, with respect
to the first display region.
[Claim 8] (Amended)
The information processing device according to claim 7,
wherein the generation means generates the position
information of the third positio~with respect to the second
display region.
[Claim 9]
An information processing device comprising:
detection means for repeatedly detecting a position
indicated by an indicator with respect to a projection screen
onto which an image is projected, and supplying position
48
I
information of the detected position; and
generation means for generating position information of
a third position located between a first position located
inside of a display region of the image and a second position
located outside of the display region in the projection screen,
in a case where the detection means detects the second position,
and then detects the first position.
[Claim 10]
The information processing device according to claim 9,
wherein the detection means detects the indicated position on
the basis of image data obtained by capturing an image of an
imaging region including the display region in a predetermined
period, and
the generation means specifies the first position and
the second position on the basis of two pieces of the image
data of which times of the imaging are different from each other
by one period, and generates the position information of the
third position.
[Claim 11]
The information processing device according to claim 9,
wherein the detection means detects the indicated position on
the basis of the image data obtained by capturing an image of
the imaging region wider than the display region.
49
'i I
I
1-
[Claim 12]
A projector comprising:
projection means for projecting an image onto a
projection screen;
detection means for repeatedly detecting a position
indicated by an indicator with respect to the projection screen,
and supplying position information of the detected position;
generation means for generating position information of
a third position located between a first position located
inside of a display region of the image in the projection screen
and a second position located outside of the display region,
in a case where the detection means detects the first position,
and then detects the second position; and
output means for outputting the position information
supplied by the detection means and the position information
generated by the generation means.
[Claim 13]
A projector comprisi~g:
projection means for projecting an image onto a
projection screen;
detection means for repeatedly detecting a position
indicated by an indicator with respect to the projection screen,
and supplying position information of the detected position;
50
I
I_
I
generation means for generating position information of
a third position located between a first position located
inside of a display region of the image and a second position
located outside of the display region in the projection screen,
in a case where the detection means detects the second position,
and then detects the first position; and
output means for outputting the position information
supplied by the detection means and the position information
generated by the generation means.
[Claim 14]
An information processing method comprising:
a step of repeatedly detecting a position indicated by
an indicator with respect to a projection screen onto which
an image is projected, and supplying position information of
the detected position; and
a step of generating position information of a third
position located between a first position located inside of
a display region of the image in the projection screen and a
second position located outside of the display region, in a
case where the first position is detected, and then the second
position is detected.
[Claim 15]
An information processing method comprising:
51
a step of repeatedly detecting a position indicated by
an indicator with respect to a projection screen onto which
an image is projected, and supplying position information of
the detected position; and
a step of generating position information of a third
position located between a first position located inside of
a display region of the image and a second position located
outside of the display region in the projection screen, in a
case where the second position is detected, and then the first
position is detected.