Title of the invention: image processing apparatus, image processing method, and computer-readable medium
Technical field
[0001]
This disclosure relates to an image processing device, an image processing method, and a computer-readable medium.
Background technology
[0002]
Conventionally, abnormality detection of a monitoring target such as a fence is often performed by a monitor in a monitoring room monitoring camera images of multiple cameras. For example, when the observer determines that there is a suspicious point in the monitored object, the observer detects an abnormality in the monitored object by pointing the direction of the camera toward the monitored object and zooming in. However, when the abnormality of the monitored object is detected by human hands, it takes a lot of cost and time, and the detection and countermeasure of the abnormality may be delayed.
Therefore, recently, a system for monitoring an abnormality to be monitored by using an optical fiber has been proposed (for example, Patent Document 1).
[0003]
In the technique described in Patent Document 1, the optical fiber detection sensor identifies the deflection and the like generated in the fence, detects the intrusion of a moving object such as a person, and detects the place where the intrusion is detected. Then, the image captured by the current camera in which the moving object is currently captured and the image captured by the adjacent camera adjacent to the current camera are displayed separately on the same screen.
Prior art literature
Patent documents
[0004]
Patent Document 1: Japanese Unexamined Patent Publication No. 2009-017416
Outline of the invention
Problems to be solved by the invention
[0005]
The technique described in Patent Document 1 displays a camera image of a camera showing a moving object and a camera image of an adjacent camera adjacent to the camera. However, there is a problem that it is difficult for the observer to visually recognize that an abnormality has occurred only by displaying the camera image.
[0006]
Therefore, an object of the present disclosure is to provide an image processing device, an image processing method, and a computer-readable medium capable of solving the above-mentioned problems and displaying an image so that the occurrence of an abnormality can be visually recognized easily. ..
Means to solve problems
[0007]
The image processing device according to one aspect is
Display part and
A control unit that displays a sensing data image showing the sensing data of the optical fiber and a camera image of a camera that captures an area in which a predetermined event is detected by the sensing data on the display unit.
Equipped with.
[0008]
The image processing method according to one aspect is
It is an image processing method using an image processing device.
Acquire a sensing data image showing the sensing data of the optical fiber and a camera image of a camera that captures an area in which a predetermined event is detected by the sensing data.
The sensing data image and the camera image are displayed.
[0009]
The non-temporary computer-readable medium according to one aspect is
On the computer
A procedure for acquiring a sensing data image showing the sensing data of an optical fiber and a camera image of a camera that captures an area in which a predetermined event is detected by the sensing data.
The procedure for displaying the sensing data image and the camera image,
The program for executing is stored.
The invention's effect
[0010]
According to the above aspect, it is possible to obtain an effect that an image can be displayed so that the occurrence of an abnormality can be visually recognized easily.
A brief description of the drawing
[0011]
FIG. 1 is a diagram showing an example of a configuration of a monitoring system according to the first embodiment.
FIG. 2 is a bird's-eye view of the entire area where the fence according to the first embodiment is installed, as viewed from above.
FIG. 3 is a diagram showing an example of fence position information according to the first embodiment.
FIG. 4 is a diagram showing an example of sensing data generated by the optical fiber detection unit according to the first embodiment.
FIG. 5 is a diagram showing an example of machine learning by the control unit according to the first embodiment.
FIG. 6 is a diagram showing an example of fence event information according to the first embodiment.
FIG. 7 is a diagram showing an example of camera information according to the first embodiment.
FIG. 8 is a diagram showing a display example 1 of a display unit according to the first embodiment.
FIG. 9 is a diagram showing another example of sensing data generated by the optical fiber detection unit according to the first embodiment.
FIG. 10 is a diagram showing still another example of sensing data generated by the optical fiber detection unit according to the first embodiment.
FIG. 11 is a diagram showing a modified example of display example 1 of the display unit according to the first embodiment.
FIG. 12 is a diagram showing a modified example of display example 1 of the display unit according to the first embodiment.
FIG. 13 is a diagram showing a modified example of display example 1 of the display unit according to the first embodiment.
FIG. 14 is a diagram showing a display example 2 of a display unit according to the first embodiment.
FIG. 15 is a diagram showing a display example 3 of a display unit according to the first embodiment.
FIG. 16 is a block diagram showing an example of a hardware configuration of a computer that realizes the image processing apparatus according to the first embodiment.
FIG. 17 is a flow chart showing an example of an operation flow of the image processing apparatus according to the first embodiment.
FIG. 18 is a bird's-eye view of an entire room in which a fence according to a second embodiment is installed, as viewed from above.
FIG. 19 is a diagram showing an example of a method of laying an optical fiber cable on a fence according to a second embodiment.
FIG. 20 is a diagram showing a display example 1 of a display unit according to the second embodiment.
FIG. 21 is a diagram showing a display example 2 of the display unit according to the second embodiment.
FIG. 22 is a diagram showing a modified example of display example 2 of the display unit according to the second embodiment.
FIG. 23 is a diagram showing a modified example of display example 2 of the display unit according to the second embodiment.
FIG. 24 is a diagram showing a modified example of display example 2 of the display unit according to the second embodiment.
FIG. 25 is a diagram showing a modified example of display example 2 of the display unit according to the second embodiment.
FIG. 26 is a diagram showing a display example 3 of the display unit according to the second embodiment.
FIG. 27 is a diagram showing a display example 4 of the display unit according to the second embodiment.
FIG. 28 is a diagram showing a display example 5 of the display unit according to the second embodiment.
Embodiment for carrying out the invention
[0012]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the embodiment described below, as an example, the monitoring target to be monitored is assumed to be a fence, but the monitoring target is not limited to the fence.
[0013]

First, the configuration of the monitoring system according to the first embodiment will be described with reference to FIG.
As shown in FIG. 1, the monitoring system according to the first embodiment monitors the fence 10, the optical fiber cable 20, the optical fiber detection unit 30, the image processing device 40, and a plurality of cameras 50 (as shown in FIG. 1). In FIG. 1, three cameras (50A to 50C) are provided. Further, the image processing device 40 includes a control unit 41 and a display unit 42. The fence 10 may be composed of one fence 10, but in the first embodiment, it is assumed that a plurality of fences 10 are connected to each other.
[0014]
The optical fiber cable 20 is a cable configured by covering one or more optical fibers, and is laid on the fence 10 and buried in the ground along the fence 10. Specifically, the optical fiber cable 20 extends from the optical fiber detection unit 30 along the fence 10, is folded back at a folding point, and returns to the optical fiber detection unit 30. Of these, one between the optical fiber detection unit 30 and the turning point is laid in the fence 10, and the other is buried in the ground along the fence 10. However, the method of laying / burying the optical fiber cable 20 shown in FIG. 1 is an example and is not limited thereto.
[0015]
FIG. 2 is a diagram showing a more specific example of the entire area where the fence 10 according to the first embodiment is installed, and is a bird's-eye view of the entire area as viewed from above. As shown in FIGS. 1 and 2, the first embodiment is an example in which the fence 10 is installed outdoors.
[0016]
The camera 50 is a camera that captures an area in which the fence 10 is installed, and is realized by, for example, a fixed camera, a PTZ (Pan Tilt Zoom) camera, or the like. A plurality of cameras 50 may be installed so that the entire area where the fence 10 is installed can be photographed, and the number of installed cameras and the installation interval are not particularly limited. For example, when a high-performance camera 50 having a long maximum shooting distance is used, the number of installed cameras can be reduced and the installation interval can be lengthened.
[0017]
The monitoring system according to the first embodiment monitors the fence 10 and its surroundings by using an optical fiber sensing technique using an optical fiber as a sensor.
Specifically, the optical fiber detection unit 30 injects pulsed light into at least one optical fiber included in the optical fiber cable 20. Then, as the pulsed light is transmitted through the optical fiber in the direction of the fence 10, backscattered light is generated for each transmission distance. This backscattered light returns to the optical fiber detection unit 30 via the same optical fiber.
[0018]
At this time, the optical fiber detection unit 30 incidents the pulsed light in the clockwise direction, receives the backward scattered light with respect to the pulsed light from the clockwise direction, and incidents the pulsed light in the counterclockwise direction. , Receives backward scattered light for this pulsed light from the counterclockwise direction. Therefore, the optical fiber detection unit 30 receives backscattered light from two directions.
[0019]
Here, the fence 10 vibrates when an event such as a person grasping and shaking the fence 10 occurs, and the vibration of the fence 10 is transmitted to the optical fiber. Further, the vibration pattern of the vibration of the fence 10 transmitted to the optical fiber is a dynamically fluctuating fluctuation pattern, and differs depending on the type of the event occurring on the fence 10. In the first embodiment, for example, the following events are assumed as predetermined events that occur in the fence 10.
(1) A person grabs the fence 10 and shakes it.
(2) A person hits the fence 10.
(3) A person climbs the fence 10
(4) A person hangs a ladder on the fence 10 and climbs the ladder.
(5) People and animals roam around the fence 10.
(6) People dig around fence 10
[0020]
Therefore, the backscattered light received from the optical fiber by the optical fiber detection unit 30 includes a pattern according to the state of the fence, that is, a pattern according to the event occurring in the fence 10. Therefore, in the present embodiment, the state of the fence is detected by the method shown below by utilizing the fact that the pattern corresponding to the state of the fence is included in the backscattered light. Specifically, it detects whether a predetermined event has occurred on the fence 10.
[0021]
The optical fiber detection unit 30 determines the position of the fence 10 where the backscattered light is generated based on the time difference between the time when the pulsed light is incident on the optical fiber and the time when the backscattered light is received from the same optical fiber. Can be identified. Further, in the first embodiment, as described above, the fence 10 is configured by connecting a plurality of fences 10. Therefore, as shown in FIG. 3, the optical fiber detection unit 30 holds position information indicating the installation position (here, the distance from the optical fiber detection unit 30) and the installation area of ​​each of the plurality of fences 10. By setting the fence 10, the fence 10 in which the rearward scattered light is generated can be identified from the plurality of fences 10. Further, the optical fiber detection unit 30 can detect the vibration intensity of the specified fence 10 by detecting the received backscattered light with the distributed vibration sensor (Distributed Vibration Sensor).
[0022]
Therefore, the optical fiber detection unit 30 can generate vibration data as shown in FIG. 4, for example, as sensing data. In FIG. 4, the horizontal axis indicates the position (distance from the optical fiber detection unit 30), and the vertical axis indicates the passage of time.
In the example shown in FIG. 4, vibration is generated at a position about 400 m away from the optical fiber detection unit 30. The vibration pattern of this vibration is a dynamically fluctuating fluctuation pattern, and will differ depending on the type of event occurring on the fence 10 at that position.
[0023]
Therefore, in this embodiment, the control unit 41 performs machine learning (for example, deep learning) of a vibration pattern when a predetermined event occurs on the fence 10, and uses the learning result of machine learning (initial learning model) to perform a predetermined event on the fence 10. Is occurring.
[0024]
First, the machine learning method will be described with reference to FIG.
As shown in FIG. 5, a plurality of vibration patterns are prepared when a predetermined event occurs on the fence 10. The control unit 41 inputs a plurality of vibration patterns and teacher data which is fence event information indicating a predetermined event occurring in the fence 10 when the vibration patterns are used (steps S1 and S2). FIG. 6 shows an example of fence event information as teacher data. The fence event information is held by the control unit 41.
[0025]
Subsequently, the control unit 41 performs matching and classification of the two (step S3), and performs supervised learning (step S4). As a result, an initial learning model is obtained (step S5). In this initial learning model, when a vibration pattern corresponding to an event occurring in the fence 10 is input, if the event may correspond to any of the predetermined events, a predetermined event that may correspond to the event. Is the output model. Alternatively, this initial learning model may be a model that outputs the confidence (confidence) in which the predetermined event occurs together with the predetermined event that may be applicable. In addition, the importance of the event (event) may be displayed based on the reliability or the priority of the event. For example, "a person climbs the fence 10" is set to a higher priority than "a person or an animal roams around the fence 10" and is output as a high-importance event.
[0026]
Next, a method of detecting whether or not a predetermined event has occurred on the fence 10 will be described.
In this case, the control unit 41 first acquires the vibration pattern corresponding to the event occurring in the fence 10 from the optical fiber detection unit 30. Subsequently, the control unit 41 inputs the vibration pattern to the initial learning model. As a result, the control unit 41 obtains a predetermined event that may correspond as an output result of the initial learning model, and therefore detects that a predetermined event has occurred. Further, when the control unit 41 obtains the reliability together with the predetermined event that may correspond as the output result of the initial learning model, if the reliability is equal to or higher than the threshold value, the predetermined event occurs. You just have to detect that.
[0027]
As described above, in the present embodiment, the vibration pattern when a predetermined event occurs in the fence 10 is machine-learned, and the learning result of the machine learning is used to machine-learn the predetermined event occurring in the fence 10. Is detected.
It may be difficult for human analysis to extract features for detecting a predetermined event occurring in the fence 10 from the data. In the present embodiment, by constructing a learning model from a large number of patterns, it is possible to detect a predetermined event occurring in the fence 10 with high accuracy even if it is difficult for human analysis.
[0028]
In the machine learning in the present embodiment, in the initial state, a learning model may be generated based on two or more teacher data. Further, the newly detected pattern may be newly learned in this learning model. At that time, detailed conditions for detecting a predetermined event occurring in the fence 10 may be adjusted from the new learning model.
[0029]
As shown in FIG. 7, the control unit 41 holds camera information indicating the installation position (distance from the optical fiber detection unit 30) of each of the plurality of cameras 50, the imageable area, and the like. Further, the control unit 41 can acquire the position information of each of the plurality of fences 10 as shown in FIG. 3 from the optical fiber detection unit 30. Therefore, when the control unit 41 detects a predetermined event occurring in the fence 10 as described above, the control unit 41 is determined among the plurality of cameras 50 based on the above-mentioned camera information and the position information of the fence 10. The camera 50 that captures the area including the fence 10 in which the event is detected is specified, and the specified camera 50 is controlled. For example, the control unit 41 controls the angle (azimuth angle, elevation angle) of the camera 50, the zoom magnification, and the like.
[0030]
Further, the control unit 41 may control two or more cameras 50 that capture an area including the fence 10 in which a predetermined event is detected among the plurality of cameras 50. In this case, the functions may be divided for each camera 50. For example, at least one of the two or more cameras 50 captures the face of a person existing in the above area, and the captured face image is utilized for face authentication, and the two or more cameras 50. At least one of the cameras 50 may photograph the entire area described above, and the captured image may be utilized for monitoring the behavior of people or animals existing in the area described above. Further, the two or more cameras 50 may shoot an area at different angles. Further, at least one camera 50 out of the two or more cameras 50 may perform shooting to complement the shooting of another camera 50. For example, when the camera 50 has a blind spot in the above area that cannot be captured by another camera 50, the camera 50 may capture the blind spot.
[0031]
The display unit 42 is installed in a monitoring room or the like that monitors the entire area where the fence 10 is installed, and performs various displays under the control of the control unit 41.
Specifically, the control unit 41 captures an area including a sensing data image showing the sensing data generated by the optical fiber detection unit 30 and a fence 10 in which a predetermined event is detected based on the sensing data. The camera image of the camera 50 or the like is displayed on the display unit 42.
[0032]
Hereinafter, a specific display example displayed by the display unit 42 according to the first embodiment will be described.
(1) Display example 1
First, display example 1 will be described with reference to FIG.
As shown in FIG. 8, in the display example 1, the sensing data image P11 and the camera image P12 are displayed. The arrangement relationship between the sensing data image P11 and the camera image P12 is not limited to this example.
[0033]
The sensing data image P11 is an image showing the sensing data generated by the optical fiber detection unit 30. This sensing data is obtained by arranging vibration data similar to the vibration data shown in FIG. 4 in the vertical direction in chronological order. Here, this sensing data indicates that a person is moving while hitting the fence 10, and finally digging around the fence 10. Therefore, the control unit 41 detects the event that a person is digging around the fence 10 and controls the camera 50 that captures the area including the fence 10.
[0034]
The camera image P12 is a camera image of the camera 50 that is controlled by the control unit 41 by photographing the area including the fence 10 in which the event that a person is digging the surrounding area is detected.
[0035]
Note that the sensing data image P11 is not limited to that shown in FIG. For example, the sensing data image P11 is an image showing only one vibration data when an event that a person is digging around is detected (for example, an image showing vibration data as shown in FIG. 4). Is also good.
[0036]
Alternatively, as shown in FIG. 9, the sensing data image P11 may be an image showing sensing data in which data showing the laying / burying status of the optical fiber cable 20 is superimposed on the vibration data as shown in FIG. Alternatively, data indicating the laying / burying status of the optical fiber cable 20 may be displayed outside the upper or lower frame of the sensing data. Further, any other display method may be used as long as the display method corresponds to the length information of the horizontal axis of the sensing data and the laying / burying status of the optical fiber cable 20.
[0037]
In the sensing data shown in FIG. 9, the laying / burying status of the optical fiber cable 20 is shown at the upper part of the image.
The horizontally long quadrangle in the range of about 90 m to about 370 m from the optical fiber detection unit 30 indicates the ground, and the upper side of this horizontally long quadrangle indicates the boundary with the ground. That is, in this range, it is shown that the optical fiber cable 20 is buried in the ground, and also the depth of the optical fiber cable 20 from the ground.
[0038]
Further, the horizontally long quadrangle in the range of about 390 m to about 560 m from the optical fiber detection unit 30 indicates the fence 10, and the lower side of this horizontally long quadrangle indicates the boundary with the ground. That is, in this range, it is shown that the optical fiber cable 20 is laid on the fence 10, and the height of the optical fiber cable 20 from the ground is also shown.
Further, in the sensing data shown in FIG. 9, when referring to the vibration data, it can be seen that an oblique line is observed in the vicinity of about 190 m to about 220 m from the optical fiber detection unit 30. This indicates that people were wandering around here.
[0039]
Alternatively, the sensing data image P11 may be an image showing sensing data as shown in FIG. The sensing data shown in FIG. 10 is data obtained from a bird's-eye view of the optical fiber cable 20 laid on the fence 10 or embedded along the fence 10 from above, and is vibration at each location on the optical fiber cable 20. It is the data which visually showed the intensity of the above with the density.
[0040]
Further, the sensing data image P11 and the camera image P12 are vibration data detected as the sensing data image P11 at that time by the user (for example, a watchman in a monitoring room, etc., the same applies hereinafter) by designating an arbitrary time or time. The camera image P12 of the camera 50 corresponding to the position of may be displayed. Specifically, as shown in FIG. 11, the user specifies a predetermined time on the vertical axis of the sensing data image P11. The control unit 41 identifies the vibration data detected in this predetermined time, and displays the camera image P12 of the camera 50 corresponding to the position of the vibration data. Further, as shown in FIG. 12, the user may specify not only the predetermined time on the vertical axis of the sensing data image P11 but also the location on the horizontal axis. The control unit 41 displays the camera image P12 of the camera 50 corresponding to the predetermined time and place specified by the user. Further, as shown in FIG. 13, the user may specify a predetermined date and time. The control unit 41 identifies the sensing data image P11 at a predetermined interval before and after the specified date and time (1 hour before and after in the vertical direction) and the vibration data detected at the specified date and time, and determines the vibration data of the vibration data. The camera image P12 of the camera 50 corresponding to the position is displayed.
[0041]
(2) Display example 2
Subsequently, display example 2 will be described with reference to FIG.
As shown in FIG. 14, in the display example 2, the sensing data image P21, the image in which the area identification information P23 is superimposed on the bird's-eye view image P22, and the camera image P24 are displayed. The arrangement relationship of the sensing data image P21, the image in which the area identification information P23 is superimposed on the bird's-eye view image P22, and the camera image P24 is not limited to this example.
[0042]
The bird's-eye view image P22 is an image of the entire area where the fence 10 is installed as viewed from above. In the present display example 2, as in the display example 1, the control unit 41 detects an event that a person is digging around the fence 10.
The area identification information P23 has a balloon shape, and while pointing to the area including the fence 10 where the control unit 41 has detected the occurrence of the above-mentioned event with an arrow, a warning message indicating that a person is digging around the fence 10 is issued. It is included and displayed superimposed on the bird's-eye view image P22.
The sensing data image P21 and the camera image P24 are the same as the sensing data image P11 and the camera image P12 in FIG.
[0043]
(3) Display example 3
Subsequently, a display example 3 will be described with reference to FIG.
As shown in FIG. 15, in the display example 3, the sensing data image P31, the image in which the area identification information P33 is superimposed on the bird's-eye view image P32, the camera image P34, and the event information image P35 are displayed. .. The area identification information P33 is superimposed on the sensing data image P31 and the bird's-eye view image P32.The arrangement relationship of the image, the camera image P34, and the event information image P35 is not limited to this example.
[0044]
The event information image P35 is an image showing event information showing the occurrence status of an event (event). In the example of FIG. 15, the event information includes the date and time when the event occurred, the type of the event, the fence 10 where the event occurred, the position where the event occurred (distance from the optical fiber detection unit 30), and the reliability at which the event occurred. It is the information that represents each.
Further, the event information image P35 may be configured so that the user can select an event. Specifically, when the user specifies an arbitrary event from a plurality of events displayed on the event information image P35, the control unit 41 controls the sensing data image P31, the bird's-eye view image P32, and the camera image corresponding to the date and time of the event. Display P34.
[0045]
The sensing data image P31 and the camera image P34 are the same as the sensing data image P11 and the camera image P12 in FIG. Further, the bird's-eye view image P32 and the area specific information P33 are the same as the bird's-eye view image P21 and the area specific information P22 in FIG.
[0046]
Subsequently, with reference to FIG. 16, the hardware configuration of the computer 60 that realizes the image processing device 40 will be described.
As shown in FIG. 16, the computer 60 includes a processor 601, a memory 602, a storage 603, an input / output interface (input / output I / F) 604, a communication interface (communication I / F) 605, and the like. The processor 601, the memory 602, the storage 603, the input / output interface 604, and the communication interface 605 are connected by a data transmission line for transmitting and receiving data to and from each other.
[0047]
The processor 601 is, for example, an arithmetic processing unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 602 is, for example, a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 603 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a memory card. Further, the storage 603 may be a memory such as a RAM or a ROM.
[0048]
The storage 603 stores a program that realizes the function of the control unit 41 included in the image processing device 40. The processor 601 realizes the function of the control unit 41 by executing this program. Here, when the processor 601 executes the above programs, these programs may be read onto the memory 602 and then executed, or may be executed without being read onto the memory 602. The memory 602 and the storage 603 also play a role of storing information and data held by the control unit 41.
[0049]
Further, the above-mentioned program can be stored by using various types of non-transitory computer readable medium and supplied to a computer (including a computer 60). Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), optomagnetic recording media (eg, optomagnetic disks), CD-ROMs (Compact Disc-Read Only Memory). , CD-R (CD-Recordable), CD-R / W (CD-ReWritable), semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory) )including. The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.
[0050]
The input / output interface 604 is connected to a display device 6041, an input device 6042, and the like. The display device 6041 realizes the display unit 42, and is a device that displays a screen corresponding to drawing data processed by the processor 601 such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. .. The input device 6042 is a device that receives an operator's operation input, and is, for example, a keyboard, a mouse, a touch sensor, and the like. The display device 6041 and the input device 6042 may be integrated and realized as a touch panel.
[0051]
The communication interface 605 sends and receives data to and from an external device. For example, the communication interface 605 communicates with an external device via a wired communication path or a wireless communication path.
[0052]

Hereinafter, the operation of the image processing device 40 according to the first embodiment will be described. Here, the operation flow of the image processing apparatus 40 according to the first embodiment will be described with reference to FIG. Note that FIG. 17 shows the operation after the control unit 41 detects a predetermined event occurring in the fence 10 based on the sensing data generated by the optical fiber detection unit 30.
[0053]
As shown in FIG. 17, when the control unit 41 detects a predetermined event occurring on the fence 10, it controls the camera 50 that captures the area including the fence 10, and acquires the camera image of the camera 50. At the same time, the sensing data generated by the optical fiber detection unit 30 is acquired (step S11). Since the control unit 41 has already acquired the sensing data for detecting a predetermined event, it is not necessary to acquire the sensing data again in step S11 if the sensing data is to be retained.
[0054]
After that, the control unit 41 causes the display unit 42 to display a sensing data image showing the sensing data and a camera image of the camera 50 that captures the area including the fence 10 in which a predetermined event is detected (step S12). Specifically, the control unit 41 causes the display unit 42 to display as in the display example 1. Alternatively, the control unit 41 may have the display unit 42 display the display example 2 or the display example 3.
[0055]

As described above, according to the present embodiment, the sensing data image showing the sensing data of the optical fiber and the camera image of the camera 50 that captures the area including the fence 10 in which a predetermined event is detected by the sensing data are captured. It is displayed on the display unit 42. In this way, not only the camera image of the camera 50 that captures the area including the fence 10 in which the predetermined event is detected but also the sensing data image that is the basis of the detection is displayed together, so that the observer has a predetermined value. It becomes easier to visually recognize that an event (abnormality) has occurred.
[0056]
Further, according to the first embodiment, the optical fiber sensing technology using the optical fiber as a sensor is used. Therefore, there are advantages such as being unaffected by electromagnetic noise, eliminating the need for power supply to the sensor, being excellent in environmental resistance, and facilitating maintenance.
[0057]

The above-mentioned embodiment 1 is an example in which the fence 10 is installed outdoors.
On the other hand, the second embodiment is an example in which the fence 10 is experimentally installed indoors (specifically, in a room).
[0058]
FIG. 18 is a bird's-eye view of the entire room in which the fence 10 according to the second embodiment is installed, as viewed from above.
As shown in FIG. 18, two fences 10a and 10b are arranged in an L shape in the room. As shown in FIG. 19, an optical fiber cable 20 is laid on the two fences 10a and 10b. However, the method of laying the optical fiber cable 20 shown in FIG. 19 is an example and is not limited thereto.
[0059]
Further, in the second embodiment, one of the predetermined events occurring on the fences 10a and 10b is an event in which a person touches the fences 10a and 10b. Therefore, cameras 50A and 50B are installed in the room so that when a person touches the fences 10a and 10b, the fences 10a and 10b can be photographed. Further, a door 70 is installed in the room, and a person enters and exits the room through the door 70.
Note that, in FIG. 18, the components constituting the image processing device 40 are not shown on the assumption that they are arranged outside the room. However, the present invention is not limited to this, and the optical fiber detection unit 30 may also be arranged outside the room.
[0060]
Further, the monitoring system according to the second embodiment differs from the first embodiment described above in that the fence 10 is installed indoors, and the basic configuration and operation are the same. be. Therefore, in the following, only a specific display example displayed by the display unit 42 according to the second embodiment will be described.
[0061]
(1) Display example 1
First, display example 1 will be described with reference to FIG. 20.
As shown in FIG. 20, in the display example 1, the sensing data image P41 and the camera image P42 are displayed. The arrangement relationship between the sensing data image P41 and the camera image P42 is not limited to this example.
[0062]
The sensing data image P41 is an image showing the sensing data generated by the optical fiber detection unit 30. Here, this sensing data indicates that a person has touched the fence 10b. Therefore, the control unit 41 detects the event that a person is touching the fence 10b, and controls the camera 50A that captures the area including the fence 10b.
The camera image P42 is a camera image of the camera 50A that photographs the area including the fence 10b and is controlled by the control unit 41. In the camera image of the camera 50A, a person is surrounded by a square frame (the same applies to the following display examples).
[0063]
(2) Display example 2
Subsequently, display example 2 will be described with reference to FIG. 21.
As shown in FIG. 21, in the display example 2, the sensing data image P51 and the camera image P52 are displayed. The arrangement relationship between the sensing data image P51 and the camera image P52 is not limited to this example.
[0064]
Also in this display example 2, the control unit 41 detects an event that a person is touching the fence 10b, as in the display example 1. However, in the present display example 2, unlike the display example 1, the control unit 41 controls two cameras 50A and 50B for photographing the area including the fence 10b.
[0065]
Therefore, the camera image P52 includes the camera images of the two cameras 50A and 50B controlled by the control unit 41.
Further, the camera images of the cameras 50A and 50B are not only the image at the time when the above-mentioned event is detected (the second image from the left), but also the image after that time (the leftmost image) and the image thereof. It also includes the images before the time (third and fourth images from the left). At this time, in order to make it easier to distinguish the image at the time when the above-mentioned event is detected (the second image from the left) and another image, the mark or the border is thickened on the image at the time when the above-mentioned event is detected. It may be displayed in a conspicuous manner.
[0066]
The top and bottom of the camera images of the cameras 50A and 50B may be exchanged. Further, the user may select the order in which the camera images of the cameras 50A and 50B are arranged vertically.
[0067]
The 51 and the camera image P52 may display the camera image P52 of the camera 50 corresponding to the position of the sensing data image P11 and the detected vibration data at that time by designating the camera 50 at an arbitrary time or time. good. Specifically, as shown in FIG. 22, the user designates a predetermined time on the vertical axis of the sensing data image P51. The control unit 41 identifies the vibration data detected in this predetermined time, and displays the camera image P52 of the camera 50 corresponding to the position of the vibration data. Further, as shown in FIG. 23, the user may specify not only the predetermined time on the vertical axis of the sensing data image P51 but also the location on the horizontal axis. The control unit 41 displays the camera image P52 of the camera 50 corresponding to the predetermined time and place specified by the user. Further, as shown in FIG. 24, the user may specify a camera 50 displayed along a predetermined time on the vertical axis of the sensing data image P51 and a horizontal axis (location) of the sensing data image P51. The control unit 41 displays the camera image P52 of the camera 50A or 50B for a predetermined time specified by the user. Further, as shown in FIG. 25, the user may specify a predetermined date and time. The control unit 41 identifies the sensing data image P51 at a predetermined interval before and after (for example, 1 hour before and after) and the vibration data detected at the designated date and time around the designated date and time, and positions the vibration data. The camera image P52 of the camera 50 corresponding to the above is displayed.
[0068]
Further, in the second embodiment, since only the two cameras 50A and 50B are provided, the camera image P52 can include the respective camera images of the cameras 50A and 50B. However, when the number of cameras 50 itself is large and the control unit 41 controls more cameras 50, there is a possibility that the camera images of all the cameras 50 controlled by the control unit 41 cannot be included in the camera image P52. be. In that case, the camera image to be included in the camera image P52 may be selected by the user.
The sensing data image P51 is the same as the sensing data image P41 in FIG.
[0069]
(3) Display example 3
Subsequently, a display example 3 will be described with reference to FIG. 26.
As shown in FIG. 26, in the display example 3, the sensing data image P61, the bird's-eye view image P62, and the camera image P63 are displayed. The arrangement of the sensing data image P61, the bird's-eye view image P62, and the camera image P63 is not limited to this example.
The bird's-eye view image P62 is an image of the entire room in which the fences 10a and 10b are installed as viewed from above.
[0070]
The camera image P63 is the same as the camera image P42 of FIG. 20, and is a camera image of the camera 50 (here, the camera 50A) controlled by the control unit 41. However, the present invention is not limited to this, and when the user selects the camera 50 on the bird's-eye view image P62, the camera image of the camera 50 selected by the user may be displayed as the camera image P63. Further, the camera image P63 may be the same as the camera image P52 of FIG.
The sensing data image P61 is the same as the sensing data image P41 in FIG.
[0071]
(4) Display example 4
Subsequently, a display example 4 will be described with reference to FIG. 27.
As shown in FIG. 27, in the display example 4, the sensing data image P71, the bird's-eye view image P72, the camera image P73, and the event information image P74 are displayed. The arrangement relationship of the sensing data image P71, the bird's-eye view image P72, the camera image P73, and the event information image P74 is not limited to this example.
[0072]
The event information image P74 is an image showing event information showing the occurrence status of an event (event). In the example of FIG. 27, the event information is information indicating the importance of the event, the date and time when the event occurred, the type of the event, and the reliability that the event occurred. The importance of an event is set based on the reliability and the priority of the event. For example, "a person climbs the fence 10" is prioritized rather than "a person or an animal roams around the fence 10". Is set high, and it is output as an event with high importance.
Further, the event information image P74 may be configured so that the user can select an event. Specifically, when the user specifies an arbitrary event from a plurality of events displayed on the event information image P74, the control unit 41 controls the sensing data image P71, the bird's-eye view image P72, and the camera image corresponding to the date and time of the event. Display P73.
[0073]
The bird's-eye view image P72 is the same as the bird's-eye view image P62 of FIG.
The camera image P73 is the same as the camera image P42 of FIG. 20, and is a camera image of the camera 50 (here, the camera 50A) controlled by the control unit 41. However, the present invention is not limited to this, and when the user selects the camera 50 on the bird's-eye view image P72, the camera image of the camera 50 selected by the user may be displayed as the camera image P73. Further, when the user selects an event on the event information image P74, the camera image of the date and time when the event occurred may be displayed as the camera image P73. Further, the camera image P73 may be the same as the camera image P52 of FIG.
[0074]
The sensing data image P71 is the same as the sensing data image P41 in FIG. However, the present invention is not limited to this, and when the user selects an event on the event information image P74, an image showing the sensing data of the date and time when the event occurred may be displayed as the sensing data image P61.
[0075]
(5) Display example 5
Subsequently, a display example 5 will be described with reference to FIG. 28.
As shown in FIG. 28, in the display example 5, an image in which the area identification information P82 is superimposed on the bird's-eye view image P81 is displayed.
The bird's-eye view image P81 is the same as the bird's-eye view image P62 of FIG. In the present display example 5, as in the display example 1, the control unit 41 detects an event that a person is touching the fence 10b.
The area identification information P82 has a balloon shape, and a warning message indicating that a person is touching the fence 10b while pointing an arrow at an area including the position of the fence 10b where the control unit 41 has detected the occurrence of the above-mentioned event. Is included, and is superimposed and displayed on the bird's-eye view image P81.
[0076]
As described above, the second embodiment has the same basic configuration and operation as the first embodiment described above. Therefore, the effect is the same as that of the first embodiment described above.
[0077]

Although the example in which the monitoring target is the fence 10 has been described in the above-described embodiment, the monitoring target is not limited to the fence 10. First, the installation destination of the monitoring target may be an airport, a port, a plant, a nursing facility, a company building, a border, a nursery center, a home, or the like. In addition to the fence, the monitoring target may be a wall, a pipeline, a utility pole, a civil engineering structure, or the like. Further, the laying destination or burying destination of the optical fiber cable 20 when monitoring the monitoring target may be a wall, a pipeline, a utility pole, a civil engineering structure, a floor, or the like, in addition to the fence and the ground. For example, when monitoring a fence 10 installed in a nursing facility, predetermined events that occur in the fence 10 include a person hitting the fence 10, being caught by the fence 10 due to injury, etc., and a person escaping. Therefore, it is conceivable to climb the fence 10.
[0078]
Further, in the above-described embodiment, it has been described that the fence 10 vibrates when a predetermined event occurs, but when these events occur, the fence 10 also undergoes changes in sound, temperature, strain, stress, and the like. , These changes are also transmitted to the optical fiber. Further, the patterns of sound, temperature, strain, stress, and the like are also dynamically fluctuating fluctuation patterns, and differ depending on the type of the event occurring on the fence 10. Therefore, the optical fiber detection unit 30 uses a distributed acoustic sensor, a distributed temperature sensor, and the like in addition to the distributed vibration sensor, and uses vibration, sound, temperature, and strain / stress. Etc. are detected and sensing data is generated, and the control unit 41 detects an event occurring in the fence 10 based on the sensing data reflecting changes in vibration, sound, temperature, strain, stress, and the like. May be detected. This makes it possible to further improve the detection accuracy.
[0079]
Further, in the above-described embodiment, when a predetermined event occurs in the fence 10, the control unit 41 controls the angle, zoom magnification, etc. of the camera 50 that captures the area including the fence 10. However, the control may be continued even after a predetermined event occurs. For example, the control unit 41 may control the camera 50 so as to track a person, an animal, a car, or the like existing in the above-mentioned area. Further, the control unit 41 controls that when a person who is wandering around the fence 10 leaves an object such as a suspicious object, the camera 50 is controlled to take a picture of the object, and the other cameras 50 are controlled. You may control to track the person.
[0080] [0080]
Further, the control unit 41 and the display unit 42 of the image processing device 40 may be provided separately from each other. For example, the display unit 42 may be provided in the monitoring room, and the image processing device 40 including the control unit 41 may be provided outside the monitoring room.
[0081]
Further, in the above-described embodiment, only one optical fiber detection unit 30 is provided and occupies the optical fiber cable 20, but the present invention is not limited to this.
For example, even if the optical fiber detection unit 30 is provided in the communication carrier station building and the optical fiber cable 20 is shared between the existing communication equipment provided inside the communication carrier station building and the optical fiber detection unit 30. good.
[0082]
Further, one optical fiber detection unit 30 is provided in each of the plurality of communication carrier stations, and the optical fiber cable 20 is provided between the plurality of optical fiber detection units 30 provided in each of the plurality of communication carrier stations. May be shared.
Further, a plurality of optical fiber detection units 30 may be provided in one communication carrier station building, and the optical fiber cable 20 may be shared among the plurality of optical fiber detection units 30.
[0083]
Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present disclosure within the scope of the present disclosure.
Description of the sign
[0084]
10,10a,10b fence
20 Optical fiber cable
30 Optical fiber detector
40 Image processing device
41 Control unit
42 display unit
50, 50A-50C camera
60 computers
601 processor
602 memory
603 storage
604 I / O interface
6041 Display device
6042 input device
605 communication interface
70 door
P11, P31, P41, P51, P61, P71 Sensing data image
P12, P34, P42, P52, P63, P73 Camera image
P21, P32, P62, P72, P81 bird's-eye view image
P22, P33, P82 Area specific information
P35, P74 event information image
The scope of the claims
[Claim 1]
Display part and
A control unit that displays a sensing data image showing the sensing data of the optical fiber and a camera image of a camera that captures an area in which a predetermined event is detected by the sensing data on the display unit.
An image processing device equipped with.
[Claim 2]
The control unit
Information for identifying the area where the predetermined event was detected is superimposed on the bird's-eye view image showing the entire area where the optical fiber is laid or buried, and displayed on the display unit.
The image processing device according to claim 1.
[Claim 3]
The control unit
As the camera image to be displayed together with the sensing data image, the camera image of each of the two or more cameras that capture the area where the predetermined event is detected is displayed on the display unit.
The image processing device according to claim 1 or 2.
[Claim 4]
The control unit
Displayed with the sensing data image As the camera image, the camera image of the camera that captures the area where the predetermined event is detected at the time when the predetermined event is detected and the time before and after the predetermined event is displayed on the display unit.
The image processing apparatus according to any one of claims 1 to 3.
[Claim 5]
The control unit
Along with the sensing data image and the camera image, a bird's-eye view image showing the entire area where the optical fiber is laid or buried is displayed on the display unit.
The image processing apparatus according to any one of claims 1 to 3.
[Claim 6]
The control unit
Along with the sensing data image, the camera image, and the bird's-eye view image, an image showing the occurrence status of the predetermined event is displayed on the display unit.
The image processing device according to claim 5.
[Claim 7]
It is an image processing method using an image processing device.
Acquire a sensing data image showing the sensing data of the optical fiber and a camera image of a camera that captures an area in which a predetermined event is detected by the sensing data.
The sensing data image and the camera image are displayed.
Image processing method.
[Claim 8]
On the computer
A procedure for acquiring a sensing data image showing the sensing data of an optical fiber and a camera image of a camera that captures an area in which a predetermined event is detected by the sensing data.
The procedure for displaying the sensing data image and the camera image,
A non-temporary computer-readable medium that contains a program for executing.