FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION (See section 10, rule 13)
“INFORMATION PROCESSING DEVICE, MOBILE APPARATUS,
METHOD, AND PROGRAM”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-0075, Japan;
SONY SEMICONDUCTOR SOLUTIONS
CORPORATION, of 4-14-1 Asahi-cho, Atsugi-shi, Kanagawa 2430014, Japan
The following specification particularly describes the invention and the manner in which it is to
be performed.

DESCRIPTION INFORMATION PROCESSING DEVICE, MOVEMENT DEVICE, AND
METHOD, AND PROGRAM
TECHNICAL FIELD [0001]
The present disclosure relates to an information processing device, a movement device, and a method, and a program. More specifically, the present disclosure relates to an information processing device, a movement device, and a method, and a program for performing control to switch automatic driving and manual driving.
BACKGROUND ART [0002]
In recent years, a technology relating to automatic driving has been actively developed.
An automatic driving technology is a technology that enables automatic traveling on roads by using a variety of sensors, such as position detection means, that are provided in a vehicle (an automobile), and it is predicted that the automatic driving technology will rapidly spread in the future. [0003]
However, in the present situation, automatic driving is in a development stage, it is considered that it will take time to enable 100% automatic driving, and it is predicted that, for a while, traveling will be performed while automatic driving and manual driving performed by a driver are appropriately switched.
For example, it is predicted that modes will need

to be switched in such a way that traveling is performed in an automatic driving mode on linear roads having a sufficient road width, such as expressways, and in a case where a vehicle leaves an expressway and is parked in a desired position in a parking lot, or on mountain paths or the like that have a narrow road width, switching to a manual driving mode is performed, and traveling is performed according to an operation performed by a driver. [0004]
While a vehicle is performing automatic driving, a driver does not need to direct a line-of-sight in a forward direction that is a vehicle traveling direction, and has freedom of action such as dozing, watching television, reading a book, or sitting to face backward and talking with a person sitting on a rear seat, for example. [0005]
In a vehicle that is traveling while automatic driving and manual driving are switched, in a case where there is a need for switching from the automatic driving mode to the manual driving mode, a driver needs to be caused to start manual driving.
However, for example, during the performance of automatic driving, if the driver dozes, a degree of awakening of the driver deteriorates. Stated another way, a level of consciousness has deteriorated. If switching is performed to the manual driving mode in a state of a reduced degree of awakening, as described above, normal manual driving fails to be performed, and in the worst case, there is a possibility of the

occurrence of an accident. [0006]
In order to secure the safety of driving, manual driving needs to be caused to be started in a state where a driver has a high degree of awakening, and in other words, in a state where the driver is very conscious.
For this purpose, a degree of awakening of the driver needs to be checked.
An example of a prior art that discloses a technology for checking a degree of awakening of a driver is Patent Document 1 (Japanese Patent No. 4980988). This document discloses a configuration in which a state of a deterioration in attention of a driver is estimated by using face information or biological information, such as a line-of-sight or a pupil diameter, of the driver. [0007]
However, prior arts including this Patent Document 1 do not disclose a configuration in which a notice timing required for safe recovery from automatic driving to manual driving is controlled.
In a case where there is a need for switching from the automatic driving mode to the manual driving mode, notice needs to be given to cause a driver to start manual driving.
If this notice timing is too late, manual driving is caused to be started in a state where the driver has a low degree of awakening, and in other words, in a state where a level of consciousness has deteriorated, and there is a problem of an increase in a probability of the occurrence of an accident.

CITATION LIST
PATENT DOCUMENT
[0008]
Patent Document 1: Japanese Patent No. 4980988
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0009]
The present disclosure has been made, for example, in view of the program described above, and it is an object of the present disclosure to provide an information processing device, a movement device, and a method, and a program that enable switching from automatic driving to manual driving to be safely performed.
SOLUTIONS TO PROBLEMS [0010]
A first aspect of the present disclosure is:
an information processing device including a safety determination unit that receives, as an input, driver information and environmental information, and calculates a safety index value indicating whether or not a driver in a vehicle during automatic driving is in a state of being able to perform safe manual driving. [0011]
Moreover, a second aspect of the present disclosure is:
a movement device including:
a driver information acquisition unit that acquires driver information of the movement device;

an environmental information acquisition unit that acquires environmental information of the movement device; and
a safety determination unit that receives, as an input, the driver information and the environmental information, and calculates a safety index value indicating whether or not a driver in the movement device during automatic driving is in a state of being able to perform safe manual driving. [0012]
Moreover, a third aspect of the present disclosure is:
an information processing method performed by an information processing device, the information processing method including:
a safety determination unit receiving, as an input, driver information and environmental information, and performing processing for calculating a safety index value indicating whether or not a driver in a vehicle during automatic driving is in a state of being able to perform safe manual driving. [0013]
Moreover, a fourth aspect of the present disclosure is:
an information processing method performed by a movement device, the information processing method including:
a driver information acquisition step of a driver information acquisition unit acquiring driver information of the movement device;
an environmental information acquisition step of an

environmental information acquisition unit acquiring environmental information of the movement device; and
a safety determination step of a safety determination unit receiving, as an input, the driver information and the environmental information, and calculating a safety index value indicating whether or not a driver in the movement device during automatic driving is in a state of being able to perform safe manual driving. [0014]
Moreover, a fifth aspect of the present disclosure is:
a program that causes an information processing device to perform information processing, the information processing including:
a safety determination unit receiving, as an input, driver information and environmental information, and calculating a safety index value indicating whether or not a driver in a vehicle during automatic driving is in a state of being able to perform safe manual driving. [0015]
Note that the program according to the present disclosure is, for example, a program that can be provided to an information processing device or a computer system that can execute a variety of program codes by a storage medium or a communication medium that provides the program in a computer-readable form. By providing such a program in the computer-readable form, processing according to the program is performed on the information processing device or the computer system. [0016]

Yet other objects, features, or advantages of the present disclosure will be clarified by a more detailed description based on the examples described later of the present disclosure or the attached drawings. Note that the system described herein is a configuration including a logical set of a plurality of devices, and does not always include devices having respective configurations in the same housing.
EFFECTS OF THE INVENTION [0017]
By employing a configuration in one example of the present disclosure, a configuration is achieved in which driver information and environmental information are input and a safety index value indicating whether or not a driver who is performing automatic driving is in a state of being able to perform safe manual driving or a manual driving recovery available time is estimated.
Specifically, for example, the configuration includes: a driver information acquisition unit that acquires driver information of a movement device such as an automobile; an environmental information acquisition unit that acquires environmental information of the movement device; and a safety determination unit that receives, as an input, the driver information and the environmental information, and learns and calculates a safety index value indicating whether or not a driver in the movement device during automatic driving is in a state of being able to perform safe manual driving. The safety determination unit further estimates a manual driving recovery available time including a time required

until the driver in the movement device during the automatic driving becomes able to start the safe manual driving.
By employing this configuration, a configuration is achieved in which driver information and environmental information are input and a safety index value indicating whether or not a driver who is performing automatic driving is in a state of being able to perform safe manual driving or a manual driving recovery available time is estimated.
Note that effects described herein are only illustrative and are not restrictive, and additional effects may be exhibited.
BRIEF DESCRIPTION OF DRAWINGS [0018]
Fig. 1 is a diagram explaining one configuration example of a movement device according to the present disclosure.
Fig. 2 is a diagram explaining one example of data displayed on a display unit of the movement device according to the present disclosure.
Fig. 3 is a diagram explaining processing performed by the movement device according to the present disclosure.
Fig. 4 is a diagram explaining a configuration example of the movement device according to the present disclosure.
Fig. 5 is a diagram explaining a configuration example of the movement device according to the present disclosure.

Fig. 6 is a diagram explaining a sensor configuration example of the movement device according to the present disclosure.
Fig. 7 is a diagram illustrating one example of a sequence of mode switching from an automatic driving mode to a manual driving mode that is performed by the movement device according to the present disclosure.
Fig. 8 is a diagram illustrating a flowchart explaining one example of an operation sequence of automatic driving.
Fig. 9 is a diagram illustrating one example of a traveling route on which an automatic driving permitting/prohibiting section that has been determined by a driver setting a destination has been set or generated in spots.
Fig. 10 is a diagram explaining information processing for a traveling section display on a traveling route.
Fig. 11 is a diagram illustrating one example of a traveling section display that is finally conducted.
Fig. 12 is a diagram illustrating an example of a change in (an example of scrolling of) a traveling section display with the lapse of time.
Fig. 13 is a diagram illustrating one example of a traveling section display on a traveling route that is displayed on a screen of a tablet terminal device (hereinafter simply referred to as a “tablet”).
Fig. 14 is a diagram illustrating one example of a state where a driver is performing a secondary task by actually using a tablet.
Fig. 15 is a diagram illustrating a state where a

cautious traveling section Sd is newly generated in a second section and a warning is issued to a driver by conducting a flashing display.
Fig. 16 is a diagram illustrating a state where a small window has been popped up on a screen of a tablet.
Fig. 17 is a diagram explaining a specific configuration example and processing of a safety determination unit.
Fig. 18 is a diagram illustrating a flowchart explaining a sequence example of learning processing.
Fig. 19 is a diagram illustrating a flowchart explaining a sequence example of learning processing.
Fig. 20 is a diagram explaining a configuration example of learning device input/output data.
Fig. 21 is a diagram illustrating a flowchart explaining one example of a sequence of processing that is performed at the time of switching from the automatic driving mode to the manual driving mode.
Fig. 22 is a diagram explaining a distribution example of a plurality of pieces of relationship information (observation plots) between an observable evaluation value that corresponds to an observation value and a recovery delay time (= a manual driving recovery available time), and a recovery success rate.
Fig. 23 is a diagram explaining a manual driving recovery available time according to the type of processing (a secondary task) that a driver is performing in the automatic driving mode.
Fig. 24 is a diagram explaining a hardware configuration example of an information processing device.

MODE FOR CARRYING OUT THE INVENTION [0019]
Details of an information processing device, a movement device, and a method, and a program according to the present disclosure are described below with reference to the drawings. Note that description is provided according to the items described below.
1. Outline of configurations and processing of movement device and information processing device
2. Specific configuration and processing example of movement device
3. Sequence of mode switching from automatic driving mode to manual driving mode
4. Operation sequence example of automatic driving
5. Safety determination processing and manual driving recovery available time estimation processing that are performed by movement device
6. Sequence of processing performed by movement device and information processing device according to the present disclosure
7. Configuration example of information processing device
8. Summary of configuration according to the present disclosure [0020]
[1. Outline of configurations and processing of movement device and information processing device]
First, the outline of configurations and processing of a movement device and an information processing device is described with reference to Fig. 1 and the drawings

that follow.
A movement device according to the present disclosure is, for example, an automobile that is capable of traveling while switching automatic driving and manual driving.
In such an automobile, in a case where there is a need for switching from an automatic driving mode to a manual driving mode, a driver needs to be caused to start manual driving. [0021]
However, various types of processing (secondary tasks) are performed by a driver during the performance of automatic driving.
For example, a simple release of hands from a steering wheel includes a case where a driver is gazing at the front of an automobile, similarly to the time of driving, a case where the driver is reading a book, or a case where the driver is dozing.
A degree of awakening (a level of consciousness) of the driver changes due to a difference among these pieces of processing.
For example, if a driver dozes, a degree of awakening of the driver deteriorates. Stated another way, a level of consciousness has deteriorated. In a state where a degree of awakening has deteriorated, as described above, normal manual driving fails to be performed. If switching to the manual driving mode is performed in this state, in the worst case, there is a possibility of the occurrence of an accident. [0022]
In order to secure the safety of driving, manual

driving needs to be caused to be started in a state where a driver has a high degree of awakening, and in other words, in a state where the driver is very conscious.
For this purpose, a notice timing at which a request to switch from automatic driving to manual driving is issued needs to be changed according to a degree of awakening of a driver during the performance of automatic driving. [0023]
For example, in a case where a driver faces forward and is viewing a road during the performance of automatic driving, the driver has a high degree of awakening, and in other words, the driver can start manual driving at any time.
In such a case, it is sufficient if notice of switching to manual driving is given at a timing just before the time when manual driving is required. This is because a driver can immediately start safe manual driving. [0024]
However, in a case where a driver is dozing during the performance of automatic driving, the driver has an extremely low degree of awakening.
In such a case, if notice of switching to manual driving is given at a timing just before the time when manual driving is required, the driver has to start manual driving in an unconscious state. This results in an increase in a probability of the occurrence of an accident. Accordingly, in the case of a low degree of awakening, as described above, notice of switching to manual driving needs to be given in an earlier stage.

[0025]
A movement device according to the present disclosure or an information processing device that can be mounted in the movement device controls a timing of notice of switching to manual driving, for example, in accordance with a degree of awakening of a driver.
Moreover, in one example of the present disclosure, an optimum timing of notice of switching to manual driving is calculated by using information such as the traveling time of a movement device, driver operation information, environmental information of roads or the like, or information relating to an accident that has occurred in the past, in addition to driver information such as a degree of awakening. [0026]
For example, in one example of the present disclosure, learning processing is performed to calculate a degree of awakening by using each of the pieces of information described above, namely, information such as the traveling time of a movement device, driver operation information, environmental information of roads or the like, or information relating to an accident that has occurred in the past, and an optimum notice timing is calculated as a result of the learning processing.
Note that the learning processing can be performed in an automobile serving as the movement device, or may be performed in a server that is communicable with the automobile serving as the movement device. [0027]
Furthermore, in one example of the present disclosure, in the notice timing calculation processing

described above, processing is performed in consideration of information relating to an individual driver, namely, a driving history, an accident history, driving operation information after the start of manual driving of the driver, and moreover, information relating to the type of an automobile, and the like.
Moreover, the number of occupants or information relating to loaded freight is also used.
Note that, in a case where many types of information are used, the many types of information can be used in a state where the many types of information have been classified into information for advance learning and information for online learning in accordance with data characteristics. [0028]
Configurations and processing of a movement device according to the present disclosure and an information processing device that can be mounted in the movement device are described with reference to Fig. 1 and the drawings that follow.
Fig. 1 is a diagram illustrating one configuration example of an automobile 10 serving as one example of the movement device according to the present disclosure.
An information processing device according to the present disclosure is attached to the automobile 10 illustrated in Fig. 1. [0029]
The automobile 10 illustrated in Fig. 1 is an automobile that can be driven in two driving modes, a manual driving mode and an automatic driving mode.
In the manual driving mode, traveling is performed

on the basis of an operation performed by a driver 20, namely, an operation performed on a steering wheel or an operation performed on an accelerator, a brake, or the like.
In contrast, in the automatic driving mode, the driver 20 does not need to perform an operation, and driving is performed on the basis of sensor information from, for example, a position sensor, another peripheral information detection sensor, or the like.
The position sensor is, for example, a GPS receiver or the like, and the peripheral information detection sensor is, for example, a camera, an ultrasonic sensor, a radar, a light detection and ranging or laser imaging detection and ranging (LiDAR), a sonar, or the like. [0030]
Note that Fig. 1 is a diagram explaining the outline of the present disclosure, and schematically illustrates principal components. A detailed configuration is described later. [0031]
As illustrated in Fig. 1, the automobile 10 includes a data processing unit 11, a driver information acquisition unit 12, an environmental information acquisition unit 13, a communication unit 14, and a notice unit 15.
The driver information acquisition unit 12 acquires, for example, information for determining a degree of awakening of a driver, driver operation information, or the like. Specifically, the driver information acquisition unit 12 includes, for example, a camera that captures a face image of a driver, an

operation information acquisition unit of each operation unit (such as a steering wheel, an accelerator, or a brake), or the like. [0032]
The environmental information acquisition unit 13 acquires traveling environmental information of the automobile 10. Examples include image information of front, rear, left-hand, and right-hand sides of the automobile, position information from the GPS, information relating to surrounding obstacles from a light detection and ranging or laser imaging detection and ranging (LiDAR), a sonar, or the like, and the like. [0033]
The data processing unit 11 receives, as an input, driver information acquired by the driver information acquisition unit 12 or environmental information acquired by the environmental information acquisition unit 13, and calculates a safety index value indicating whether or not a driver in a vehicle during automatic driving can perform safe manual driving, whether or not a driver during manual driving is performing safe driving, and the like.
Moreover, for example, in a case where there is a need for switching from the automatic driving mode to the manual driving mode, processing for giving notice via the notice unit 15 is performed in such a way that switching to the manual driving mode is performed. [0034]
It is assumed that a timing of this notice processing is an optimum timing calculated, for example, by inputting the driver information acquisition unit 12

and the environmental information acquisition unit 13.
Stated another way, it is assumed that a timing is used at which the driver 20 can start safe manual driving.
Specifically, in a case where a driver has a high degree of awakening, notice is given just before a manual driving start time, for example, five seconds before. In a case where the driver has a low degree of awakening, processing is performed with time to spare, for example, in such a way that notice is given 20 seconds before the manual driving start time. Specific calculation of an optimum notice timing is described later. [0035]
The notice unit 15 includes a display unit, a sound output unit, or a vibrator of a steering wheel or a seat that gives this notice,
An example of a warning display conducted on the display unit that configures the notice unit 15 is illustrated in Fig. 2. [0036]
As illustrated in Fig. 2, respective displays described below are conducted on a display unit 30.
Driving mode information = “AUTOMATIC DRIVING”,
Warning display = “PLEASE SWITCH TO MANUAL DRIVING”. [0037]
In an area where the driving mode information is displayed, “AUTOMATIC DRIVING” is displayed during the performance of the automatic driving mode, and “MANUAL DRIVING” is displayed during the performance of the manual driving mode.

[0038]
In an area where warning display information is displayed, is a display area where the display described below is conducted while automatic driving is performed in the automatic driving mode.
“PLEASE SWITCH TO MANUAL DRIVING” [0039]
Note that, as illustrated in Fig. 1, the automobile 10 has a configuration that is communicable with a server 30 via the communication unit 14.
For example, part of processing performed by the data processing unit 11 for calculating a proper time for the output of notice and specifically, learning processing can be performed in the server 30.
A specific example of this is described later. [0040]
Fig. 3 is a diagram illustrating a specific example of processing performed by the movement device or the information processing device according to the present disclosure.
Fig. 3 is a diagram illustrating an example of setting a proper timing of notice requesting switching to manual driving during the performance of automatic driving in the automatic driving mode, and illustrates two examples described below of notice processing.
(a) Notice processing in a case where a driver has a high degree of awakening during the performance of automatic driving
(b) Notice processing in a case where a driver has a low degree of awakening during the performance of automatic driving

[0041]
Example (a) is an example where, during the performance of automatic driving, a driver faces forward and is viewing a road. In this case, the driver has a high degree of awakening, and in other words, the driver can start manual driving at any time.
In such a case, even if notice of switching to manual driving is given at a timing just before the time when manual driving is required, a driver can immediately start safe manual driving. [0042]
In Example (b), for example, in a case where a driver is dozing during the performance of automatic driving, the drive has an extremely low degree of awakening.
In such a case, if notice of switching to manual driving is given at a timing just before the time when manual driving is required, a driver starts manual driving in an unconscious state, and there is a high probability of the occurrence of an accident. Accordingly, in a case where a degree of awakening is low, as described above, notice of switching to manual driving needs to be given in an earlier stage. [0043]
[2. Specific configuration and processing example of movement device]
Next, a specific configuration and processing example of the movement device 10 according to the present disclosure are described with reference to Fig. 4 and the drawings that follow.
Fig. 4 illustrates a configuration example of a

movement device 100. Note that, hereinafter, in a case where a vehicle that is provided with the movement device 100 is distinguished from another vehicle, the vehicle that is provided with the movement device 100 is referred to as the local car or the local vehicle. [0044]
The movement device 100 includes an input unit 101, a data acquisition unit 102, a communication unit 103, an in-vehicle device 104, an output controller 105, an output unit 106, a drive system controller 107, a drive-system system 108, a body system controller 109, a body-system system 110, a storage 111, and an automatic driving controller 112. [0045]
The input unit 101, the data acquisition unit 102, the communication unit 103, the output controller 105, the drive system controller 107, the body system controller 109, the storage 111, and the automatic driving controller 112 are mutually connected via a communication network 121. The communication network 121 includes, for example, an on-vehicle communication network conforming to an arbitrary standard, such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), or FlexRay (registered trademark), a bus, or the like. Note that, in some cases, respective units of the movement device 100 are directly connected without the communication network 121. [0046]
Note that, hereinafter, in a case where the respective units of the movement device 100 perform

communication via the communication network 121, the description of the communication network 121 is omitted. For example, in a case where the input unit 101 and the automatic driving controller 112 perform communication via the communication network 121, it is simply described that the input unit 101 and the automatic driving controller 112 perform communication. [0047]
The input unit 101 includes a device that a passenger uses to input various types of data, instructions, or the like. For example, the input unit 101 includes an operation device such as a touch panel, a button, a microphone, a switch, or a lever, an operation device on which an input operation can be performed by using a method other than a manual operation, such as sound or a gesture, and the like. Furthermore, for example, the input unit 101 may be a remote control device that uses infrared rays or other radio waves, or an external connection device, such as a mobile device or a wearable device, that corresponds to an operation of the movement device 100. The input unit 101 generates an input signal on the basis of data, an instruction, or the like that has been input by a passenger, and supplies the input signal to the respective units of the movement device 100. [0048]
The data acquisition unit 102 includes a variety of sensors or the like that acquire data to be used in processing performed by the movement device 100, and supplies the acquired data to the respective units of the movement device 100.

[0049]
For example, the data acquisition unit 102 includes a variety of sensors that detect a state of the local car, or the like. Specifically, for example, the data acquisition unit 102 includes a gyro sensor, an acceleration sensor, an inertial measurement unit (IMU), a sensor that detects an amount of an operation performed on an accelerator pedal, an amount of an operation performed on a brake pedal, a steering angle of a steering wheel, engine speed, the rotational speed of a motor, the rotational speed of a wheel, or the like, and the like. [0050]
Furthermore, for example, the data acquisition unit 102 includes a variety of sensors that detect information relating to the outside of the local car. Specifically, for example, the data acquisition unit 102 includes an imaging device such as a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, or another camera. Furthermore, for example, the data acquisition unit 102 includes an environment sensor that detects weather, meteorological phenomena, or the like, and a peripheral information detection sensor that detects an object around the local car. The environment sensor includes, for example, a raindrop sensor, a fog sensor, a sunshine sensor, a snow sensor, or the like. The peripheral information detection sensor includes, for example, an ultrasonic sensor, a radar, a light detection and ranging or laser imaging detection and ranging (LiDAR), a sonar, or the like. [0051]

For example, Fig. 5 illustrates an installation example of a variety of sensors that detect information relating to the outside of the local car. Imaging devices 7910, 7912, 7914, 7916, and 7918 are provided, for example, in at least one position of a front nose, a side-view mirror, a rear bumper, a back door, or an upper portion of a windshield in a vehicle cabin of a vehicle 7900. [0052]
The imaging device 7910 included in the front nose and the imaging device 7918 included in the upper portion of the windshield in the vehicle cabin principally acquire an image of a front side of the vehicle 7900. The imaging devices 7912 and 7914 included in the side-view mirrors principally acquire images of sides of the vehicle 7900. The imaging device 7916 included in the rear bumper or the back door principally acquires an image of a rear side of the vehicle 7900. The imaging device 7918 included in the upper portion of the windshield in the vehicle cabin is principally used to detect a preceding vehicle, or a pedestrian, an obstacle, a traffic light, a traffic sign, a traffic lane, or the like. Furthermore, in the future, in automatic driving, when a vehicle turns right or left, usage may be extended to a pedestrian crossing a road after a right or left turn within a wider range, and further a range of an object approaching a crosswalk. [0053]
Note that Fig. 5 illustrates one example of an image capturing range of each of the imaging devices 7910, 7912, 7914, and 7916. An imaging range a indicates

an imaging range of the imaging device 7910 provided in the front nose, imaging ranges b and c respectively indicate imaging ranges of the imaging devices 7912 and 7914 provided in the side-view mirrors, and an imaging range d indicates an imaging range of the imaging device 7916 provided in the rear bumper or the back door. For example, an overhead image in which the vehicle 7900 is viewed from the above, an entire-circumference stereoscopic display image in which a vehicle periphery is surrounded by a curved plane, and the like are acquired by superimposing pieces of image data that have been captured by the imaging devices 7910, 7912, 7914, and 7916 onto each other. [0054]
Sensors 7920, 7922, 7924, 7926, 7928, and 7930 that are provided in a front portion, a rear portion, sides, corners, and the upper portion of the windshield in the vehicle cabin of the vehicle 7900 may be, for example, ultrasonic sensors or radars. The sensors 7920, 7926, and 7930 that are provided in the front nose, the rear bumper, the back door, and the upper portion of the windshield in the vehicle cabin of the vehicle 7900 may be, for example, LiDARs. These sensors 7920 to 7930 are principally used to detect a preceding vehicle, a pedestrian, an obstacle, or the like. These detection results may be further employed to improve a display of a stereoscopic object in the overhead display or the entire-circumference stereoscopic display that are described above. [0055]
Return to Fig. 4. The description of respective

components is continued. The data acquisition unit 102 includes a variety of sensors that detect a current position of the local car. Specifically, for example, the data acquisition unit 102 includes a global navigation satellite system (GNSS) receiver that receives a GNSS signal from a GNSS satellite, or the like. [0056]
Furthermore, for example, the data acquisition unit 102 includes a variety of sensors that detect in-vehicle information. Specifically, for example, the data acquisition unit 102 includes an imaging device that images a driver, a biosensor that detects biological information of the driver, a microphone that collects sound in a vehicle cabin, and the like. The biosensor is provided, for example, on a seat surface, a steering wheel, or the like, and detects a seating state of a passenger who is seated on a seat or biological information of a driver who is holding the steering wheel. As a biological signal, diversification observable data, such as the heart rate, the pulse rate, the blood flow, respiration, psychosomatic correlation, visual stimuli, brain waves, a perspiration condition, the posture and behavior of the head, eyes, gazing, blinking, a saccade, a microsaccade, fixation, drift, staring, or the pupillary reaction of the iris, can be used. These pieces of organism activity observable information reflecting an observable driving state are used by a safety determination unit 155 described later to calculate a recovery notice timing as intrinsic characteristics of a recovery delay case of a corresponding driver on the basis of a recovery delay

time characteristic that has been summarized as an observable evaluation value estimated due to observation and has been linked to a log of the evaluation value. [0057]
Fig. 6 illustrates examples of a variety of sensors that are included in the data acquisition unit 102 and acquire information relating to a driver in a vehicle. For example, the data acquisition unit 102 includes a ToF camera, a stereo camera, a seat strain gauge, or the like as a detector that detects the position or posture of a driver. Furthermore, the data acquisition unit 102 includes a face (head) recognition device, a driver eye tracker, a driver head tracker, or the like as a detector that acquires organism activity observable information of a driver. [0058]
Furthermore, the data acquisition unit 102 includes a vital signal detector as a detector that acquires the organism activity observable information of the driver. Furthermore, the data acquisition unit 102 includes a driver identification unit. Note that knowledge authentication using a password, a personal identification number, or the like and biometric authentication using a face, a fingerprint, the iris of the pupil, a voiceprint, or the like are also conceivable as an authentication method. [0059]
The communication unit 103 performs communication with the in-vehicle device 104, and a variety of outside-vehicle devices, a server, a base station, and the like, and transmits data supplied from respective units of the

movement device 100 or supplies received data to the respective units of the movement device 100. Note that a communication protocol supported by the communication unit 103 is not particularly limited, and the communication unit 103 can also support plural types of communication protocols. [0060]
For example, the communication unit 103 performs wireless communication with the in-vehicle device 104 by using a wireless LAN, Bluetooth (registered trademark), near field communication (NFC), a wireless USB (WUSB), or the like. Furthermore, for example, the communication unit 103 performs wired communication with the in-vehicle device 104 via a not-illustrated connection terminal (and a cable if necessary), by using a universal serial bus (USB), a high-definition multimedia interface (HDMI) (registered trademark), a mobile high-definition link (MHL), or the like. [0061]
Moreover, for example, the communication unit 103 performs communication with equipment (for example, an application server or a control server) that is present on an external network (for example, the Internet, a cloud network, or a company-specific network) via the base station or an access point. Furthermore, for example, the communication unit 103 performs communication with a terminal that is present near the local car (for example, a terminal of a pedestrian or a store, or a machine type communication (MTC) terminal) by using a peer to peer (P2P) technology. [0062]

Moreover, for example, the communication unit 103 performs V2X communication such as vehicle to vehicle communication, vehicle to infrastructure communication, vehicle to home communication, or vehicle to pedestrian communication. Furthermore, for example, the communication unit 103 includes a beacon receiver, receives radio waves or electromagnetic waves that have been transmitted from a wireless station that is provided on a road, or the like, and acquires information relating to a current position, a traffic jam, traffic regulations, a required time, or the like. Note that pairing may be performed via the communication unit with a forward traveling vehicle that can be a leading vehicle and is traveling in a section, information acquired by a data acquisition unit mounted in a preceding vehicle may be acquired as information during advance traveling and may be used to complement data of the data acquisition unit 102 of the local car. In particular, in columnar traveling using the leading vehicle, or the like, means for further securing the safety of a succeeding line is achieved. [0063]
The in-vehicle device 104 includes, for example, a mobile device (a tablet, a smartphone, or the like) or a wearable device that is possessed by a passenger, an information device that is carried in or attacked to the local car, a navigation device that searches for a route to an arbitrary destination, and the like. Note that, if it is considered that an occupant is not always fixed to a seating fixed position due to the spreading of automatic driving, usage may be extended to equipment

that can be used in a manner detachable from a video reproduction device, game equipment, or another vehicle installation in the future. In the present example, description is provided by using an example where information relating to a point at which driver’s intervention is required is only presented to a corresponding driver. However, the information may be further provided to a succeeding vehicle in columnar traveling or the like, and moreover, usage in appropriate combination with remote traveling support may be achieved by constantly providing the information to an operation management center of passenger transport share-riding busses or long-distance distribution commercial vehicles. [0064]
The output controller 105 controls an output of various types of information to a passenger of the local car or the outside of the vehicle. For example, the output controller 105 controls an output of visual information (for example, image data) and auditory information (for example, sound data) from the output unit 106 by generating an output signal including at least one of the visual information or the auditory information and supplying the output signal to the output unit 106. Specifically, for example, the output controller 105 combines pieces of image data that have been captured by imaging devices that are different from each other in the data acquisition unit 102, generates an overhead image, a panoramic image, or the like, and supplies an output signal including the generated image to the output unit 106. Furthermore, for example, the output controller 105 generates sound data including

warning sound, a warning message, or the like against danger such as collision, contact, or entry into a danger zone, and supplies, to the output unit 106, an output signal including the generated sound data. [0065]
The output unit 106 includes a device that can output the visual information or the auditory information to a passenger of the local car or the outside of the vehicle. For example, the output unit 106 includes a display device, an instrument panel, an audio speaker, a headphone, a wearable device, such as an eyeglasses type display, that a passenger wears, a projector, a lamp, or the like. The display device included in the output unit 106 may be a device including a normal display, or may be a device that displays auditory information in a field of view of a driver, such as a head-up display, a transmission type display, or a device having an augmented reality (AR) display function, for example. [0066]
The drive system controller 107 controls the drive-system system 108 by generating various control signals and supplying the various control signals to the drive-system system 108. Furthermore, the drive system controller 107 supplies a control signal to respective units other than the drive-system system 108, as needed, and gives notice or the like of a control state of the drive-system system 108. [0067]
The drive-system system 108 includes a variety of devices that relate to a drive system of the local car. For example, the drive-system system 108 includes a drive

force generation device that generates a drive force, such as an internal combustion engine or a drive motor, a drive force transmission mechanism that transmits a drive force to wheels, a steering mechanism that adjusts a steering angle, a braking device that generates a braking force, an antilock brake system (ABS), an electronic stability control (ESC), an electric power steering device, and the like. [0068]
The body system controller 109 controls the body-system system 110 by generating various control signals and supplying the various control signals to the body-system system 110. Furthermore, the body system controller 109 supplies a control signal to respective units other than the body-system system 110, as needed, and gives notice or the like of a control state of the body-system system 110. [0069]
The body-system system 110 includes a variety of devices of a body system equipped in a vehicle body. For example, the body-system system 110 includes a keyless entry system, a smart key system, a power window device, a power seat, a steering wheel, an air conditioner, a variety of lamps (for example, a headlamp, a back lamp, a brake lamp, a turn signal, a fog lamp, and the like), or the like. [0070]
The storage 111 includes, for example, a read only memory (ROM), a random access memory (RAM), a magnetic storage device such as a hard disc drive (HDD), a semiconductor storage device, an optical storage device,

a magneto-optical storage device, and the like. The storage 111 stores various programs, data, or the like that are used by the respective units of the movement device 100. For example, the storage 111 stores map data such as a three-dimensional high-precision map, e.g., a dynamic map, a global map that has a lower precision and covers a wider area than the high-precision map, or a local map including information relating to the surroundings of the local car. [0071]
The automatic driving controller 112 performs control relating to automatic driving such as autonomous traveling or driving support. Specifically, for example, the automatic driving controller 112 performs cooperative control aiming at implementing a function of an advanced driver assistance system (ADAS) including collision avoidance or impact mitigation of the local car, following traveling based on a distance between vehicles, vehicle speed maintaining traveling, collision warning of the local car, lane departure warning of the local car, or the like. Furthermore, for example, the automatic driving controller 112 performs cooperative control aiming at automatic driving or the like for autonomous traveling that is independent of an operation performed by a driver. The automatic driving controller 112 includes, a detection unit 131, a self-position estimation unit 132, a situation analyzer 133, a planning unit 134, and a motion controller 135. [0072]
The detection unit 131 detects various types of information required to control automatic driving. The

detection unit 131 includes an outside-vehicle information detection unit 141, an in-vehicle information detection unit 142, and a vehicle state detection unit 143. [0073]
The outside-vehicle information detection unit 141 performs processing for detecting information relating to the outside of the local car on the basis of data or a signal from each of the units of the movement device 100. For example, the outside-vehicle information detection unit 141 performs processing for detecting an object around the local car, recognition processing, and tracking processing, and processing for detecting a distance to the object and a relative speed. Examples of an object to be detected include a vehicle, a person, an obstacle, a structure, a road, a traffic light, a traffic sign, a road sign, and the like. [0074]
Furthermore, for example, the outside-vehicle information detection unit 141 performs processing for detecting a surrounding environment of the local car. Examples of a surrounding environment to be detected include weather, air temperature, humidity, brightness, a road surface state, and the like. The outside-vehicle information detection unit 141 supplies data indicating a result of detection processing to the self-position estimation unit 132, a map analyzer 151, the traffic rule recognizer 152, and the situation recognizer 153 of the situation analyzer 133, an emergency avoidance unit 171 of the motion controller 135, and the like. [0075]

With respect to information that the outside-vehicle information detection unit 141 acquires, if a traveling section is a section for which a constantly updated local dynamic map has been intensively supplied from infrastructure as a section in which automatic driving traveling can be performed, information can be principally supplied from infrastructure. Alternatively, traveling may be performed in a state where information is constantly updated in advance from a vehicle or a vehicle group that is preceding and traveling in a corresponding section, prior to invasion into the section. Furthermore, for example, in a case where a local dynamic map is not constantly updated to the latest version by infrastructure or in other cases, in particular, in columnar traveling or the like, road environmental information acquired from a leading vehicle that has invaded a section may be further used complementarily in order to acquire road information just before a safer invasion section. In many cases, whether or not automatic driving can be performed in a section is determined according to the presence or absence of these pieces of advance information provided from infrastructure. Automatic driving availability information on a route that is provided from infrastructure roughly amounts to the provision of an invisible track as what is called “information”. Note that, for convenience sake, the outside-vehicle information detection unit 141 is illustrated under the assumption that the outside-vehicle information detection unit 141 is mounted in the local vehicle. However, by using information acquired as “information” by the

preceding vehicle, advance predictability during
traveling may be further improved.
[0076]
The in-vehicle information detection unit 142 performs processing for detecting in-vehicle information on the basis of data or a signal from each of the units of the movement device 100. For example, the in-vehicle information detection unit 142 performs processing for authenticating and recognizing a driver, processing for detecting the driver’s state, processing for detecting a passenger, processing for detecting an in-vehicle environment, and the like. Examples of a driver’s state to be detected include a physical condition, a degree of awakening, a degree of concentration, a degree of fatigue, a direction of a line-of-sight, a detailed eyeball behavior, and the like. [0077]
Moreover, it is expected that automatic driving will be used in the future in a state where a driver is completely separated from a driving steering task. The driver will temporarily doze or will start another task, and a system will need to grasp to what extent the awakening and recovery of consciousness required for recovery to driving has progressed. Stated another way, a conventionally proposed driver monitoring system principally includes detection means for monitoring a deterioration in consciousness such as drowsiness. However, in the future, a driver will not intervene at all in driving steering. Therefore, a system will not include means for directly observing a degree of intervention in driving of a driver on the basis of the

steering stability of a steering device and the like, and will need to observe the transition of the recovery of consciousness required for driving from a state in which the driver’s accurate consciousness state is unknown, grasp an accurate internal awakening state of the driver, and advance the intervention transfer of steering from automatic driving to manual driving. [0078]
Accordingly, the in-vehicle information detection unit 142 principally has a significant two-stage role. A first role is passive monitoring of a driver’s state during automatic driving, and a second role is the acknowledgement, perception, and determination of the surroundings of the driver, and further the detection and determination of the operability of a steering device to a level at which manual driving can be performed after the issuance of a request for recovery from the system and before arrival at a cautious driving section. As control, failure self-diagnosis may be further performed on the entirety of a vehicle. In a case where an automatic driving function of the vehicle has deteriorated due to a failure in part of the automatic driving function, a driver may similarly be prompted to recover to manual driving early. Passive monitoring here refers to detection means that does not request a response reaction of consciousness from a driver, and detection means for emitting physical radio waves, light, or the like from a device and detecting a response signal is not excluded. Stated another way, monitoring of a state of a driver having no consciousness during a nap or the like is referred to, and a class that is not an

acknowledgement response reaction of a driver is assumed to be a passive method. An active response device is not excluded that analyzes and evaluates reflection or a diffused signal that is caused by applying radio waves, infrared rays, or the like. In contrast, means for requesting a conscious response that requests a response reaction from a driver is active. [0079]
Examples of an in-vehicle environment to be detected include air temperature, humidity, brightness, an odor, and the like. The in-vehicle information detection unit 142 supplies data indicating a result of detection processing to the situation recognizer 153 of the situation analyzer 133 and the motion controller 135. Note that, in a case where it has been discovered that a driver will fail to achieve manual driving by an accurate deadline time after a system issues a driving recovery instruction to the driver and it has been determined that takeover will not been performed in time even if deceleration control is performed in a state where self-driving is maintained and time postponement is performed, an instruction is issued to the emergency avoidance unit 171 or the like of the system, and a deceleration and retreat/stop procedure is started in order to cause a vehicle to retreat. Stated another way, even in a situation of not making it in time as an initial state, an arrival time for arrival at a takeover limit can be gained by causing a vehicle to start deceleration early. [0080]
The vehicle state detection unit 143 performs processing for detecting a state of the local car on the

basis of data or a signal from each of the units of the movement device 100. Examples of a state of the local car that serves as a target to be detected include speed, acceleration, a steering angle, the presence or absence and content of abnormality, a state of a driving operation, a position and an inclination of a power seat, a state of door lock, a state of another on-vehicle device, and the like. The vehicle state detection unit 143 supplies data indicating a result of detection processing to the situation recognizer 153 of the situation analyzer 133, the emergency avoidance unit 171 of the motion controller 135, and the like. [0081]
The self-position estimation unit 132 performs processing for estimating a position, a posture, and the like of the local car on the basis of data or a signal from each of the units of the movement device 100, e.g., the outside-vehicle information detection unit 141, the situation recognizer 153 of the situation analyzer 133, and the like. Furthermore, the self-position estimation unit 132 generates a local map used to estimate a self-position (hereinafter referred to as a self-position estimation map), as needed. [0082]
It is assumed, for example, that the self-position estimation map is a high-precision map using a technology such as simultaneous localization and mapping (SLAM). The self-position estimation unit 132 supplies data indicating a result of estimation processing to the map analyzer 151, the traffic rule recognizer 152, and the situation recognizer 153 of the situation analyzer 133,

and the like. Furthermore, the self-position estimation unit 132 stores the self-position estimation map in the storage 111. [0083]
The situation analyzer 133 performs processing for analyzing situations of the local car and the surroundings. The situation analyzer 133 includes the map analyzer 151, the traffic rule recognizer 152, the situation recognizer 153, the situation prediction unit 154, and the safety determination unit 155. [0084]
The map analyzer 151 performs processing for analyzing various maps stored in the storage 111 by using data or a signal from each of the units of the movement device 100, e.g., the self-position estimation unit 132, the outside-vehicle information detection unit 141, and the like, as needed, and constructs a map including information required for automatic driving processing. The map analyzer 151 supplies the constructed map to the traffic rule recognizer 152, the situation recognizer 153, the situation prediction unit 154, a route planning unit 161, an action planning unit 162, and a motion planning unit 163 of the planning unit 134, and the like. [0085]
The traffic rule recognizer 152 performs processing for recognizing a traffic rule in the surroundings of the local car on the basis of data or a signal from each of the units of the movement device 100, e.g., the self-position estimation unit 132, the outside-vehicle information detection unit 141, the map analyzer 151, and the like. In this recognition processing, for example, a

position and a state of a traffic light around the local car, the content of traffic regulations around the local car, a travelable traffic lane, and the like are recognized. The traffic rule recognizer 152 supplies data indicating a result of recognition processing to the situation prediction unit 154 or the like. [0086]
The situation recognizer 153 performs processing for recognizing a situation relating to the local car on the basis of data or a signal from each of the units of the movement device 100, e.g., the self-position estimation unit 132, the outside-vehicle information detection unit 141, the in-vehicle information detection unit 142, the vehicle state detection unit 143, the map analyzer 151, and the like. For example, the situation recognizer 153 performs processing for recognizing a situation of the local car, a situation of the surroundings of the local car, a situation of a driver of the local car, and the like. Furthermore, the situation recognizer 153 generates a local map used to recognize the situation of the surroundings of the local car (hereinafter referred to as a situation recognition map), as needed. It is assumed, for example, that the situation recognition map is an occupancy grid map. [0087]
Examples of a situation of the local car serving as a target to be recognized a position, a posture, and a movement (for example, speed, acceleration, a movement direction, and the like) of the local car, and a loading amount of freight that determines a motion characteristic of the local car or the movement of the center of gravity

of a vehicle body caused by the loading of freight, tire pressure, a recovery start timing required in control changes depending on conditions that are peculiar to a vehicle and further are peculiar to loaded freight, such as a movement of a braking distance according to a brake braking pad wear situation, allowable maximum deceleration braking for the prevention of freight movement that is caused in load braking, or a centrifugal relaxation limit speed in curve traveling according to a liquid mounted object, moreover, characteristics of the vehicle itself even in exactly the same road environment such as a road surface wear coefficient, a rod curve, or a grade, and moreover, a load or the like. Therefore, this variety of conditions need to be collected, be learnt, and be reflected in an optimum timing for control. This is not the content obtained by simply observing and monitoring the presence or absence, content, and the like of an abnormality of the local vehicle in determining a control timing on the basis of the type of a vehicle or a load. In transportation business or the like, a parameter for determining the addition of a grace time of recovery that is desirable to secure a prescribed level of safety according to characteristics peculiar to a load may be set as a fixed value in advance, and a method for uniformly determining all of the notice timing determination conditions in self-cumulative learning is not always employed. [0088]
Examples of a situation of the surroundings of the local car that serves as a target to be recognized include a type and a position of a surrounding static

object, a type, a position, and a movement (for example, speed, acceleration, a moving direction, and the like) of a surrounding moving object, a configuration and a road surface state of a surrounding road, weather, air temperature, humidity, and brightness of the surroundings, and the like. Examples of a driver’s state serving as a target to be recognized include a physical condition, a degree of awakening, a degree of concentration, a degree of fatigue, a movement of a line-of-sight, a driving operation, and the like. In order to cause a vehicle to safely travel, a control start point to be handled significantly changes according to a loading amount mounted in a state peculiar to the vehicle, a state where a mounting part is fixed to a chassis, a partiality state of the center of gravity, a maximum deceleration available acceleration value, a maximum load available centrifugal force, a recovery response delay amount according to a driver’s state, or the like. [0089]
The situation recognizer 153 supplies data indicating a result of recognition processing (including a situation recognition map, as needed) to the self-position estimation unit 132, the situation prediction unit 154, and the like. Furthermore, the situation recognizer 153 stores the situation recognition map in the storage 111. [0090]
The situation prediction unit 154 performs processing for predicting a situation relating to the local car on the basis of data or a signal from each of

the units of the movement device 100, e.g., the map analyzer 151, the traffic rule recognizer 152, the situation recognizer 153, and the like. For example, the situation prediction unit 154 performs processing for predicting a situation of the local car, a situation of the surroundings of the local car, a situation of a driver, and the like. [0091]
Examples of a situation of the local car that serves as a target to be predicted include a behavior of the local car, the occurrence of an abnormality, a travelable distance, and the like. Examples of a situation of the surroundings of the local car that serves as a target to be predicted include a behavior of a moving object around the local car, a change in a state of a traffic light, a change in environment such as weather, and the like. Examples of a situation of a driver that serves as a target to be predicted include a behavior, a physical condition, and the like of the driver. [0092]
The situation prediction unit 154 supplies data indicating a result of prediction processing together with data from the traffic rule recognizer 152 and the situation recognizer 153, to the route planning unit 161, the action planning unit 162, and the motion planning unit 163 of the planning unit 134, and the like. [0093]
The safety determination unit 155 learns an optimum recovery timing according to a driver’s recovery action pattern, vehicle characteristics, or the like, and

provides the learned information to the situation recognizer 153 or the like. By doing this, for example, a statistically obtained optimum timing required for a driver to recover from automatic driving to manual driving in a normal manner at a predetermined fixed ratio or more can be presented to the driver. [0094]
The route planning unit 161 plans a route to a destination on the basis of data or a signal from each of the units of the movement device 100, e.g., the map analyzer 151, the situation prediction unit 154, and the like. For example, the route planning unit 161 sets a route from a current position to a specified destination on the basis of a global map. Furthermore, for example, the route planning unit 161 appropriately changes a route on the basis of a traffic jam, an accident, traffic regulations, a situation of works or the like, a physical condition of a driver, and the like. The route planning unit 161 supplies data indicating the planned route to the action planning unit 162 or the like. [0095]
The action planning unit 162 plans an action of the local car for safely traveling on the route planned by the route planning unit 161 within a planned time, on the basis of data or a signal from each of the units of the movement device 100, e.g., the map analyzer 151, the situation prediction unit 154, and the like. For example, the action planning unit 162 plans a start, a stop, a traveling direction (for example, moving forward, moving backward, turning left, turning right, changing directions, and the like), a traveling lane, traveling

speed, passing, and the like. The action planning unit 162 supplies data indicating the planned action of the local car to the motion planning unit 163 or the like. [0096]
The motion planning unit 163 plans a motion of the local car for achieving the action planned by the action planning unit 162, on the basis of data or a signal from each of the units of the movement device 100, e.g., the map analyzer 151, the situation prediction unit 154, and the like. For example, the motion planning unit 163 plans acceleration, deceleration, a traveling track, and the like. The motion planning unit 163 supplies data indicating the planned motion of the local car to an acceleration or deceleration controller 172 and a direction controller 173 of the motion controller 135, and the like. [0097]
The motion controller 135 controls a motion of the local car. The motion controller 135 includes the emergency avoidance unit 171, the acceleration or deceleration controller 172, and the direction controller 173. [0098]
The emergency avoidance unit 171 performs processing for detecting emergency, such as collision, contact, entry to a danger zone, an abnormality in a driver, or an abnormality in a vehicle, on the basis of detection results of the outside-vehicle information detection unit 141, the in-vehicle information detection unit 142, and the vehicle state detection unit 143. In a case where the occurrence of emergency has been detected,

the emergency avoidance unit 171 plans a motion of the local car for the avoidance of emergency, such as a sudden stop or a sudden turn. The emergency avoidance unit 171 supplies data indicating the planned motion of the local car to the acceleration or deceleration controller 172, the direction controller 173, and the like. [0099]
The acceleration or deceleration controller 172 controls acceleration or deceleration to achieve the motion of the local car that has been planned by the motion planning unit 163 or the emergency avoidance unit 171. For example, the acceleration or deceleration controller 172 calculates a control target value of a drive force generator or a braking device to achieve acceleration, deceleration, or a sudden stop that has been planned, and supplies a control command indicating the calculated control target value to the drive system controller 107. Note that there are principally two cases where emergency can occur. Specifically, there are a case where, during automatic driving on a road that has originally been determined to be safe on a traveling route during automatic driving in a local dynamic map or the like that has been acquired by infrastructure, an unexpected accident occurs for an unexpected reason and emergency recovery fails to be performed in time, and a case where it is difficult for a driver to accurately recover from automatic driving to manual driving. [0100]
The direction controller 173 controls a direction to achieve the motion of the local car that has been

planned by the motion planning unit 163 or the emergency avoidance unit 171. For example, the direction controller 173 calculates a control target value of a steering mechanism to achieve a traveling track or a sudden turn that has been planned by the motion planning unit 163 or the emergency avoidance unit 171, and supplies a control command indicating the calculated control target value to the drive system controller 107. [0101]
[3. Sequence of mode switching from automatic driving mode to manual driving mode]
Next, a takeover sequence from the automatic driving mode to the manual driving mode is described.
Fig. 7 schematically illustrates one example of a sequence of mode switching from the automatic driving mode to the manual driving mode that is performed by the automatic driving controller 112. [0102]
In step S1, a driver is in a state of complete separation from driving steering. In this state, the driver can perform a secondary task such as nap, watching a video, concentration on a game, or a task using a visual tool such as a tablet or a smartphone, for example. It is conceivable that the task using the visual tool such as a tablet or a smartphone is performed, for example, in a state where a driver’s seat has been displaced, or in a seat that is different from the driver’s seat. [0103]
Depending on these states of the driver, it is expected that the time required for the driver to recover

will significantly change according to the content of a current task, when the driver approaches a section that requests recovery to manual driving on a route. If notice is given just before the approach of an event, the time required for recovery is not sufficient, or in a case where notice is given excessively early so as to allow a margin before the approach of the event, the time to a timing actually required for recovery is excessively long. If a situation in which notice is not given at an accurate timing repeatedly occurs as a result of this, the timing reliability of a notice timing of a system deteriorates, the driver reduces a consciousness of notice, consequently, the driver neglects to take accurate measures, and this results in a high risk of a failure in takeover and the hindrance of reliably performing the secondary task. Accordingly, in order for a driver to start accurate measures for driving recovery in response to notice, the system needs to optimize a notice timing. [0104]
Step S2 indicates a timing of manual driving recovery request notice, as described above with reference to Fig. 2. Notice of driving recovery is given to a driver by using dynamic paptics such as vibration or in a visual or auditory manner. For example, the automatic driving controller 112 monitors a steady state of a driver, grasps a timing of notice, and gives notice at an appropriate timing. Stated another way, it is desirable that, during a pre-stage passive monitoring period, a state where a driver performs a secondary task be constantly and passively monitored, the system can

calculate an optimum timing of an optimum timing of notice, passive monitoring during the period of step S1 be constantly and continuously performed, and a recovery timing and recovery notice be caused to match recovery characteristics peculiar to the driver. [0105]
Stated another way, it is desirable that an optimum recovery timing according to the recovery action pattern of the driver, characteristics of a vehicle, or the like be learnt, and a statistically obtained optimum timing required for the driver to recover from automatic driving to manual driving in a normal manner at a predetermined fixed ratio or more be presented to the driver. In this case, in a case where the driver has not responded to notice during a fixed time period, a warning using the emission of alarm sound, or the like is given. [0106]
In step S3, it is checked whether or not the driver has made a seating recovery. In step S4, a state of a degree of internal awakening of the driver is checked by analyzing a face or the behavior, such as saccade, of eyeballs. In step S5, the stability of an actual steering situation of the driver is monitored. Then, in step S6, takeover from automatic driving to manual driving has been completed. [0107] [4. Operation sequence example of automatic driving]
Next, one example of an operation sequence of automatic driving is described with reference to the flowchart illustrated in Fig. 8.
The flowchart illustrated in Fig. 8 is a flowchart

explaining an operation sequence of automatic driving
that implements the movement device 100.
[0108]
First, in step S11, driver authentication is performed. This driver authentication is performed by using knowledge authentication using a password, a personal identification number, or the like, or biological authentication using a face, a fingerprint, the iris of the pupil, a voiceprint, or the like, or by using both knowledge authentication and biological authentication. By performing driver authentication, as described above, even in a case where a plurality of drivers drives the same vehicle, information for determining a notice timing in association with each of the drivers can be accumulated. [0109]
Next, in step S12, the driver operates the input unit 101 to set a destination. In this case, the driver performs an input operation on the basis of a display on an instrument panel [0110]
Note that the present example describes an example in a case where it is assumed that an itinerary is set on a vehicle. However, remote advance reservation setting or the like may be performed in advance by using a smartphone before getting in a vehicle or by using a personal computer before leaving home. Moreover, a system of a car may perform preplanning setting according to a schedule that has been assumed by a driver in accordance with a schedule table, LDM information of a road environment, namely, what is called local dynamic

map (LDM) information in which traveling map information of a road on which a vehicle travels is constantly updated with high precision, or the like may be updated and acquired, and an actual traveling advice display or the like may be further conducted in a concierge manner at the time of or prior to getting in the vehicle. [0111]
Next, in step S13, a traveling section display on a traveling route is started. This traveling section display is conducted on the instrument panel, or is conducted, for example, in a tablet or the like in which a driver performs a secondary task in such a way that the traveling section and a working window are arranged side by side. By doing this, a driver who is performing a task in the working window can easily recognize a driver intervention required section and an automatic driving available section on a traveling route on an arrival prediction time axis from a current point. [0112]
In this traveling section display, a forward schedule and information relating to the approach to each point are presented. In this traveling section display, a driver intervention required section and an automatic driving available section on a traveling route are displayed on an arrival prediction time axis from a current point. Then, the driver intervention required section includes a manual driving section, a takeover section from automatic driving to manual driving, and a cautious traveling section from automatic driving. Details of this traveling section display are described later.

[0113]
Next, in step S14, the acquisition of LDM update information is started. According to the acquisition of this LDM update information, the content of the traveling section display can be changed to the latest state. Next, in step S15, traveling is started. Next, in step S16, a display of the traveling section display is updated on the basis of positional information of the local car and the acquired LDM update information. By doing this, the traveling section display is scroll-displayed according to traveling in such a way that each section is approaching the local car. [0114]
Means for presenting information relating to a forward traveling environment that is accompanied with traveling and is relatively approaching, a timing required for takeover of a corresponding local vehicle, or the like does not need to be limited to scroll means, and it is preferable that means in which an imminent time interval is intuitive and explicit and that enables when recovery to driving is to be started to be known with misrecognition reduced be employed as another method. For example, a time presenting method imitating a sandglass or means for directly presenting a takeover remaining time to equipment that a user wears in the form of a chronograph type watch may be intervened. [0115]
Next, in step S17, a driver’s state is monitored. Next, in step S18, event change handling processing is performed. This event change handing processing includes mode switching processing for handling a case where a

switching point between the automatic driving mode and the manual driving mode that is already present on a traveling route or a cautious traveling section is approaching, event occurrence processing for handling a case where a mode switching point or a driver intervention required section of the cautious traveling section has newly occurred on a traveling route, and the like. After this, the processes of step S16 to step S18 are appropriately repeated. [0116]
“Details of traveling section display” Fig. 9 illustrates one example of a traveling route that has been determined by a driver setting a destination. In this traveling route, an automatic driving available section Sa, a manual driving section Sb, a takeover section Sc from automatic driving to manual driving, and a cautious traveling section Sd from automatic driving are present. Here, the takeover section Sc is always present just before the manual driving section Sb, and it is requested that a driver be in a position of recovery to manual driving. Furthermore, the cautious traveling section Sd is a section in which traveling can be performed at a reduced speed while automatic driving is maintained in a state where a driver in the position of recovery to manual driving is performing cautious monitoring. [0117]
In the illustrated example, the automatic driving available section Sa is illustrated in green, the manual driving section Sb is illustrated in red, and the takeover section Sc and the cautious traveling section Sd

are illustrated in yellow. Note that respective colors are illustrated by using different patterns, for convenience sake. [0118]
In a traveling section display on a display device such as a center information display or a tablet, each of the sections described above of the traveling route is displayed on an arrival prediction time axis from a current point. The automatic driving controller 112 performs information processing for a traveling section display of a traveling route on the basis of traveling route information and traffic information. [0119]
Fig. 10(a) illustrates each of the sections on the traveling route on a movement distance axis from a current point on a fixed scale. Fig. 10(b) illustrates a speed v(t) of a flow of average road traffic at each point. Fig. 10(c) illustrates the respective sections indicated on the movement distance axis that have been converted into a time axis by using the speed v(t). By doing this, the respective sections on the traveling route are indicated on an arrival prediction time axis from a current point. Stated another way, a physical distance on the traveling route can be indicated on a time axis obtained by performing division using an average speed in each corresponding section. [0120]
In this embodiment, all of the sections for which a traveling section display is conducted are divided into three sections, as illustrated in Fig. 10(d), and a time axis of each of the sections is changed. Stated another

way, a first section from a current point to a first point (time t0, for example, about 10 minutes) is displayed as a time linear display closest section on a first time axis. For example, time t0 is set to be a necessary and sufficient time period after a general driver terminates a secondary task and before the general driver makes a recovery to driving. A closest section that is approached according to traveling has a visually intuitive effect that is equal to an effect of indication on a map that seems to proceed at a fixed speed, and therefore a driver can start preparation for accurate driving recovery that follows the approach to an event. There is an advantage in which a point at which recovery will be started can be instinctively grasped in a somewhat accurate manner. Stated another way, a purpose for displaying this section is to provide a user with start determination information of a driver’s accurate recovery point. [0121]
Furthermore, a second section from the first point (time t0) to a second point (time t1, for example, about one hour) is displayed as a time reciprocal display section on a time axis that has sequentially changed from the first time axis to a second time axis that has been reduced at a predetermined ratio with respect to this first time axis. A purpose for displaying this second section is principally measures to accurately provide a driver with a road situation during a long period in a small display because it is difficult to display a long period in a small display space if a display is conducted at a scale magnification equal to a scale magnification

of the first section described above. This enables a driver to easily grasp up to which point in a certain section ahead driving intervention will not be requested according to traveling, and there is an advantage in which engagement in a secondary task can be performed according to a plan. Intervention and non-intervention of driving are clearly separated from each other. Furthermore, in a secondary task that follows communication with a third party, or the like, a role is played in presenting important information in planning of releasing a driver from the secondary task, or the like. [0122]
Here, a method for setting this second display section is described with reference to Fig. 10(d). When it is assumed that the height of a triangle is h0, time t at a point that exists previous to its end by h is obtained according to Formula (1) described below.
t = t0 * h0/h (1) [0123]
Furthermore, the second time axis at the second point (time t1) has been reduced at a ratio of hs/h0 with respect to the first time axis. For example, in a case where hs = h0/8, a reduction rate is 1/8. [0124]
In a case where traveling is performed at a constant vehicle speed, a display of the second display section described above corresponds to a display at the time of viewing a map in which a traveling straight line expansion display section on the map is obliquely inclined in a traveling direction, or a state where a front side on a road plane is obliquely viewed. In other

words, it can also be said that a visual effect of this display section is a display that enables a sensory distance to be easily grasped without displaying a scale or the like of an accurate positional display on a screen, because perspective can be intuitively grasped due to a display image height position. Then, a distant section is reduced in size, but is not a point that is immediately arrived at in traveling. Therefore, a driver does not need to sensorily grasp strict arrival time information similarly to a close point, although rough prediction is important. This is preferable in planning to perform a secondary task. [0125]
Furthermore, a third section from the second point (time t1) to a third point (time t2) is displayed as a time linear display distant section on the second time axis (a reduction rate of hs/h0). As described above, by displaying three divided sections, a driver can know temporally closest section information in detail in a limited display space, and can know temporally distant section information. Note that a display of a distant part in a display form of the second section is below a person’s visual resolution and further a display resolution limit of a system, information required to plan and determine a secondary task fails to be discriminated, and a display function becomes useless. Accordingly, a most effective display is a display in which a reduction in a display scale is finished in a stage in which the sense of a time section can be sufficiently grasped in an intuitive manner and the division of a required intervention section and an

unrequired section is appropriately displayed, and the display scale is returned again to a fixed scale in sections that follow. [0126]
Note that the vehicle control system 100 has default values of times t0, t1, and t3. It is conceivable that values of time t0, t1, and t3 are changed between long-distance driving and short-distance driving. Therefore, the number of default values is not limited to 1. Plural types of default values may be used, and a driver (a user) or a system may selectively use the plural types of default values depending on a traveling route. Furthermore, it is also conceivable that values of times t0, t1, and t3 can be arbitrarily set by the driver (the user). [0127]
Figs. 11(a) and 11(b) illustrate examples of a traveling section display that is finally displayed. Note that whether or not a time axis is linear and further a change in a reduction rate of the time axis are indicated by using the length of an arrow. In the case of Fig. 11(a), all of the sections, the first section, the second section, and the third section, are displayed to have a first width. [0128]
In contrast, in the case of Fig. 11(b), the first section from a current point to the first point (time t0) is displayed to have the first width, the second section from the first point (time t0) to the second point (time t1) is displayed to have a width that has sequentially changed from the first width to a second width that is

narrower than the first width, and the third section from the second point (time T1) to the third point (time T2) is displayed to have the second width. This enables a driver to visually recognize a degree of reduction of time axes of the second section and the third section with respect to the first section. Stated another way, the display form of Fig. 10 is a display in consideration of only a reduction rate in a traveling direction. By further artificially changing a width that crosses the traveling direction of display information in accordance with perspective, the same perspective effect as a perspective effect in viewing along the progress of a road or a map to face an infinite direction can be exhibited, and a distribution of a driving intervention required section can be easily grasped in an intuitive manner in comparison with the case of glancing at a screen. In particular, in a case where only the second section is rotated instead of counterclock and is viewed, it seems to match a road width of a front side of a road and an arrival time at each corresponding point in the case of traveling at a constant speed. Therefore, even if an accurate position scale is not viewed and judged, an actual arrival feeling to each point can be intuitively grasped, and a display form that enables time allocation can be achieved. [0129]
Note that, for example, in a part having a small reduction rate hs/h0 like the third section, if a section having a short time length is displayed to have the time length with no change, the section is displayed to be very thin, and it is expected that it will be difficult

for a driver to recognize the section. Therefore, even in a case where, in practice, a driver intervention section (a manual driving section, a takeover section, and a cautious traveling section) has a length that is less than or equal to a fixed time length, the driver intervention section is displayed to have the fixed time length. In this case, for example, in a case where the takeover section and the manual driving section are continuous, a display of the takeover section may be omitted. In Figs. 11(a) and 11(b), a display of a first manual driving section Sb in the third section indicates such a state. By doing this, in the third section in which a time axis has significantly been reduced in size, a driver intervention required section having a short time length can be displayed to be recognizable to a driver. [0130]
Furthermore, in a part having a small reduction rate hs/h0 like the third section, in a case where a manual driving section Sb is intermittently continued at a short cycle, all of the manual driving sections Sb are displayed continuously. In Figs. 11(a) and 11(b), a display of a second manual driving section Sb in the third section indicates such a continuously displayed state. In practice, the manual driving section Sb displayed as described above includes takeover sections Sd and automatic driving available sections Sa that have a short period in addition to the manual driving sections Sb, as illustrated in Fig. 11(c). Note that, as described later, a detailed display can be conducted, for example, by double-touching a corresponding point in a

state where a traveling section display is conducted on a
tablet or the like.
[0131]
The traveling section display described above on the traveling route is updated on the basis of positional information of the local car and acquired LDM update information. By doing this, the traveling section display is scroll-displayed with the lapse of time in such a way that each section is approaching the local car. Figs. 12(a) to 12(d) illustrate an example of a change in a traveling section display with the lapse of time. This example indicates an example in which the second section is displayed in a tapered manner, but the similar is applied to a case where all of the sections are displayed to have the same width. [0132]
In this case, in the first section, each section moves fast. Furthermore, in the second section, a reduction in size of a time axis becomes smaller at a point closer to a side of the first section from a side of the third section, and therefore each of the sections moves faster. Furthermore, in the third section, a reduction in size of the time axis becomes larger, and therefore each of the sections moves slowly. [0133]
Figs. 13(a) and 13(b) illustrate examples of a traveling section display 181 on a traveling route displayed on a screen of a tablet 182. Fig. 13(a) illustrates an example in a case where the tablet 182 is used in a vertically long shape. In this case, the traveling section display 181 is conducted along a left-

hand side and an upper side in a bent state, and is conducted in parallel to a working window serving as an execution screen of a secondary task performed on the tablet 182. Fig. 13(b) illustrates an example in a case where the tablet 182 is used in a horizontally long shape. Similarly in this case, the traveling section display 181 is conducted along a left-hand side and an upper side in a bent state, and is conducted in parallel to a working window serving as an execution screen of a secondary task performed on the tablet 182. Note that in the illustrated example, the traveling section display 181 is disposed in a bent state on the screen of the tablet 182. However, in a case where a disposition space can be sufficiently acquired, it is also conceivable that the traveling section display 181 is disposed linearly. [0134]
Fig. 14 illustrates one example of a state where a driver is performing a secondary task by actually using the tablet 182. In this example, the tablet 182 is used in a horizontally long shape. On the screen of the tablet 182, the traveling section display 181 is conducted along a left-hand side and an upper side in a bent state. Note that whether or not this traveling section display 181 is conducted on the screen may be selected according to an operation performed by a driver (a user). In this case, for example, in a case where the traveling section display 181 is not conducted on the screen and in a case where the driver intervention required section enters within a fixed time period and notice is given to the driver, the traveling section display 181 may be automatically conducted on the screen.

[0135]
In a state where the traveling section display 181 is conducted on the screen of the tablet 182, in a case where the driver intervention required section is newly generated in the displayed section, the newly generated driver intervention required section is newly displayed. In this case, for example, this newly generated driver intervention required section is displayed to flash on and off during a fixed time period such that the newly generated driver intervention required section can be distinguished from another section. This flashing display may be accompanied with caution alarm sound. Here, a case where the driver intervention required section has been newly generated includes a case where the cautious traveling section or the manual driving section has been newly generated and a case where the cautious traveling section has been changed to the manual driving section. [0136]
Fig. 15 illustrates a state where a cautious traveling section Sd is newly generated in a second section and a warning is issued to a driver by using a flashing display. Note that, in this case, flashing, namely, a warning state, may be able to be stopped by a driver touching a display portion of the cautious traveling section Sd that is displayed to flash on and off. Alternatively, a small window may be popped up by a driver touching the display portion of this cautious traveling section Sd that is displayed to flash on and off, and flashing, namely, the warning state, may be able to be stopped by touching the screen for consent.

[0137]
Furthermore, in a state where the traveling section display 181 is conducted on the screen of the tablet 182, in a case where a driver (a user) double-touches and specifies an arbitrary point, a small window is popped up, for example, as illustrated in Fig. 16, and a display relating to the point is conducted. [0138]
[5. Safety determination processing and manual driving recovery available time estimation processing that are performed by movement device]
Next, safety determination processing and manual driving recovery available time estimation processing that are performed by a movement device are described.
These pieces of processing are principally performed by the safety determination unit 155 that is configured in the automatic driving controller 112 of the movement device 100 described with reference to Fig. 4. [0139]
As described above, the safety determination unit 155 learns an optimum recovery timing according to a driver’s recovery action pattern, vehicle characteristics, or the like, and provides the learned information to the situation recognizer 153 or the like. By doing this, for example, a statistically obtained optimum timing required for a driver to recover from automatic driving to manual driving in a normal manner at a predetermined fixed ratio or more can be presented to the driver. [0140]
Fig. 17 is a diagram explaining a specific

configuration example and processing of the safety determination unit 155.
A left-hand side illustrates a movement device 200 such as an automobile, and a right-hand side illustrates a server 300 that performs communication with the movement device 100 via a network. [0141]
The movement device 200 includes a driver information acquisition unit 201, an environmental information acquisition unit 202, a driver personal identification information acquisition unit 203, a safety determination unit 210, a notice controller 221, a notice unit 222, a communication unit 230, and a storage 240.
Note that the safety determination unit 210 corresponds to the safety determination unit 155 that is configured in the situation analyzer 133 illustrated in Fig. 4.
The safety determination unit 210 includes a learning processing unit 211, a safety determination processing performing unit (a manual driving recovery available time estimation unit) 212, and a personal authentication processing unit 213.
The server 300 includes a learning data acquisition or generation unit 301, a learning processing unit 302, a communication unit 303, and a storage 304. [0142]
First, processing performed by each component of the movement device 200 is described.
The driver information acquisition unit 201 includes a camera or a variety of sensors, and acquires information relating to a driver.

Examples of driver information acquired by the driver information acquisition unit 201 include a line-of-sight direction, an eyeball behavior, and the pupil diameter that are acquired from an image including an eyeball area, a facial expression acquired from an image including a face area, and the like.
These pieces of information acquired by the driver information acquisition unit 201 are used, for example, as information for determining a degree of awakening of a driver. [0143]
Note that the examples described above are pieces of the content that has been simplified and described, the detailed state of a driver fails to be determined by acquiring simple information relating to an eye, a face, or the like, and the determination of a degree of awakening of the driver includes advance history information or performing detailed behavior analysis on the eye, the face, or the like, and comprehensively performing performing the hierarchical determination of an internal awakening state required for manual driving recovery of the driver from the observable value. Examples of some of these pieces of biological information used in determination are described below. [0144]
An example of means for observing an awakening state of a driver an activity situation of a target to be acknowledged or a portion to be determined by observing an activity area within the brain is a measurement instrument on a large scale such as the observation of brain waves obtained by electrically detecting a

potential, the observation using infrared rays of the blood flow in the surface layer of the head, or the evaluation using fMRI of a function within the brain. However, in the today’s technology, the instrument is not suitable at all to be normally and generally used by being mounted in a vehicle, and the obtainment of a correlation with another organism such as a heartbeat waveform or EOG in an experiment environment and the use of offline learning based on these pieces of biological information are limited. [0145]
In addition, there is also information that can be used to predict awakening recovery in advance by continuously acquiring medium-term and long-term advance log information in advance, although this is difficult to be directly used to determine a short-term awakening state in a stage in which takeover is suddenly needed. The time required after the reception of recovery notice from a system and before recovery changes according to various factors such as the depth of a sleep, a degree of accumulation of fatigue, or an elapsed time from the start of a nap at a point in time at which notice is received, and according to individual characteristics. These pieces of advance information are difficult to give a direct recovery time, but serve as an influential factor of awakening recovery. Therefore, available observable information is used as an affector for the estimation of awakening recovery time. [0146]
Logs of these pieces of driver information acquired by the driver information acquisition unit 201 are

acquired according to these states of a driver, and further an awakening state is observed on the basis of a trigger such as the reception of recovery notice or the approach of a target takeover event. In a case where a driver wears and uses an observable device such as a watch, for example, the observation of a long-term log may include a life log before getting in a vehicle. In this case, information relating to an advance rest state or a state of an excessive sleep or a lack of sleep can also be used as help for determination. In normal use in a case where advance log acquisition is not available, a user can get in a vehicle and can observe a degree of loss of attention, such as fatigue or drowsiness, from a variety of sensors mounted on the vehicle, for example, by evaluating the Lorentz plot of an electrocardiogram, can also estimate a tension state or a fatigue state from information relating to a heartbeat, pulse waves, or the blood flow, blood pressure, and further a medium-term and long-term change log of respiration or exhalation/body odor, and can also estimate a state of the driver by analyzing an utterance tone or a response delay of the user in a verbal (inquiry) response with the system. Moreover, means for analyzing an action, such as a facial expression, facing downward, or the occurrence of an inattentive driving state, can also be employed.
Furthermore, a state may be determined by evaluating the speediness of gesture to determine an instruction response characteristic or performing active evaluation of an amount of activity. [0147]
A log of the driver information acquired by the

driver information acquisition unit 201 is observed in advance, as described above, and an awakening recovery affector is monitored in advance. In order to perform final takeover determination, an internal awakening state needs to be determined with a high degree of accurateness. When awakening and recovery from automatic driving to manual driving of a driver is viewed from another viewpoint, the driver’s state changes in time series according to the sequence described below.
1. The occurrence of an event requiring takeover is acknowledged,
2. A required situation that follows the necessity of takeover is grasped and acknowledged, and
3. In the case of sound notice, the grasp and acknowledgement of notice include a response to the notice, and the response is rising, acknowledgement and confirmation using a switch, a response and confirmation using gesture, detection using action acknowledgement of placing the hand on the steering wheel, or the detection of driver’s seat seating recovery.
4. Normally, it is assumed that a driver that has received recovery notice and has finally returned to a recovery posture performs a procedure of grasping and examining a situation required for recovery to manual driving and starts the steering of a device before the driver starts to operate a steering device such as a steering wheel, a brake, or an accelerator, and therefore one of principal information acquisition means is the acquisition of visual information.
5. When the driver can visually acknowledge a situation, the driver starts to operate an appropriate

CLAIMS
1. An information processing device comprising:
a safety determination unit that receives, as an input, driver information and environmental information, and calculates a safety index value indicating whether or not a driver in a vehicle during automatic driving is in a state of being able to perform safe manual driving.
2. The information processing device according to
claim 1,
wherein the safety determination unit performs: receiving, as the input, the driver information and the environmental information, and estimating a manual driving recovery available time including a time required until the driver in the vehicle during the automatic driving becomes able to start the safe manual driving.
3. The information processing device according to
claim 1,
wherein the safety determination unit performs: performing learning processing, and generating a
learning device in which the driver information and the
environmental information have been associated with the
safety index value; and
employing the learning device that has been
generated, and calculating the safety index value.
4. The information processing device according to
claim 2,
wherein the safety determination unit performs:

performing learning processing, and generating a learning device in which the driver information and the environmental information have been associated with the manual driving recovery available time; and
employing the learning device that has been generated, and estimating the manual driving recovery available time.
5. The information processing device according to
claim 1, further comprising:
a communication unit that performs communication with a server,
wherein the information processing device acquires, from the server, a learning device in which pieces of data (a) and (b) described below have been associated with each other:
(a) the driver information and the environmental information; and
(b) at least either the safety index value or a manual driving recovery available time including a time required until the driver in the vehicle during the automatic driving becomes able to start the safe manual driving.
6. The information processing device according to
claim 1,
wherein the driver information includes information that enables a degree of awakening of the driver to be calculated.
7. The information processing device according to

claim 1,
wherein the driver information includes at least one of a line-of-sight direction, an eyeball behavior, a pupil diameter, or a facial expression, the line-of-sight, the eyeball behavior and the pupil diameter being acquired from an image including an eyeball area, the facial expression being acquired from an image including a face area.
8. The information processing device according to
claim 1,
wherein the driver information includes information including operation information of the driver after a mode is switched from an automatic driving mode to a manual driving mode.
9. The information processing device according to
claim 1,
wherein the environmental information includes at least one of a captured image, depth information, three-dimensional structure information, LiDAR acquisition information, topographical information, or positional information.
10. The information processing device according to
claim 1,
wherein the safety determination unit performs: performing learning processing, and generating a learning device in which the driver information and the environmental information have been associated with the manual driving recovery available time; and

employing the learning device that has been generated, and calculating the manual driving recovery available time, and
the information processing device further includes: a notice controller that performs notice processing for giving manual driving recovery request notice to the driver via a notice unit at a timing determined on a basis of the manual driving recovery available time that has been calculated by the safety determination unit.
11. The information processing device according to
claim 10,
wherein the notice unit includes at least one of a display unit, a sound output unit, or a vibrator.
12. A movement device comprising:
a driver information acquisition unit that acquires driver information of the movement device;
an environmental information acquisition unit that acquires environmental information of the movement device; and
a safety determination unit that receives, as an input, the driver information and the environmental information, and calculates a safety index value indicating whether or not a driver in the movement device during automatic driving is in a state of being able to perform safe manual driving.
13. The movement device according to claim 12,
wherein the safety determination unit performs:
receiving, as the input, the driver information and

the environmental information, and estimating a manual driving recovery available time including a time required until the driver in the movement device during the automatic driving becomes able to start the safe manual driving.
14. The movement device according to claim 12,
wherein the safety determination unit performs:
performing learning processing, and generating a
learning device in which the driver information and the environmental information have been associated with the safety index value; and
employing the learning device that has been generated, and calculating the safety index value.
15. The movement device according to claim 13,
wherein the safety determination unit performs:
performing learning processing, and generating a
learning device in which the driver information and the environmental information have been associated with the manual driving recovery available time; and
employing the learning device that has been generated, and estimating the manual driving recovery available time.
16. The movement device according to claim 15, further
comprising:
a notice controller that performs notice processing for giving manual driving recovery request notice to the driver via a notice unit at a timing determined on a basis of the manual driving recovery available time that

has been calculated by the safety determination unit.
17. The movement device according to claim 16,
wherein the notice unit includes at least one of a
display unit, a sound output unit, or a vibrator.
18. An information processing method performed by an
information processing device, the information processing
method comprising:
a safety determination unit receiving, as an input, driver information and environmental information, and performing processing for calculating a safety index value indicating whether or not a driver in a vehicle during automatic driving is in a state of being able to perform safe manual driving.
19. An information processing method performed by a
movement device, the information processing method
comprising:
a driver information acquisition step of a driver information acquisition unit acquiring driver information of the movement device;
an environmental information acquisition step of an environmental information acquisition unit acquiring environmental information of the movement device; and
a safety determination step of a safety determination unit receiving, as an input, the driver information and the environmental information, and calculating a safety index value indicating whether or not a driver in the movement device during automatic driving is in a state of being able to perform safe

manual driving.
20. A program that causes an information processing device to perform information processing, the information processing comprising:
a safety determination unit receiving, as an input, driver information and environmental information, and calculating a safety index value indicating whether or not a driver in a vehicle during automatic driving is in a state of being able to perform safe manual driving.