JP7267221B2 - Processing device, processing method, estimation system and program - Google Patents

Description

The present disclosure relates to a processing device, a processing method, an estimation system and a program.

Industrial vehicles such as forklifts and wheel loaders have characteristics that differ from general passenger vehicles, such as the steering angle of the wheels of the vehicle body and the structure of the vehicle. Therefore, especially for drivers who have little experience driving industrial vehicles, there is a possibility that the area where the vehicle actually moves may differ from what they expected when driving industrial vehicles or performing work such as loading and unloading. There is a possibility that the blind spot area is not recognized.
Patent Literature 1 describes, as a related technique, a technique for calling the driver's attention when the forklift moves backward.

JP 2016-222428 A

By the way, while the driver is operating the vehicle, if the vehicle passes near people in the surroundings or crosses in front of the vehicle, a dangerous situation may occur, even if it does not result in an accident. In some cases, a so-called “hiyari-hatto” state, which is temporary, may occur. In some cases, the driver does not even notice that the near-miss situation has occurred. In addition, even if a safety device such as a camera or sensor is provided, there are cases in which a so-called "hiyari-hatto" state, which does not directly lead to an accident, is not notified. However, once the driver becomes aware of the near-miss situation, he/she can drive cautiously in the event of similar near-miss situations thereafter.
Therefore, there is a demand for a technology that allows a driver to appropriately estimate the degree of danger of a vehicle that is being driven.

At least one embodiment of the present disclosure has been made in view of the circumstances described above, and provides a processing device, processing method, estimation system, and program capable of estimating the degree of risk of a vehicle being driven by a driver. It is intended to

In order to solve the above problems, a processing device according to the present disclosure includes a calculation unit that calculates a position of a target object relative to the vehicle based on an image captured from the vehicle; a specifying unit for specifying a plurality of first states indicating a state of the vehicle when drawing a travel locus in contact with an object; a second state indicating the state of the vehicle when the image is captured; and a plurality of the first states. an estimating unit for estimating the degree of risk associated with the object based on the difference from each state.
Further, the processing device according to the present disclosure includes a calculation unit that calculates a position of an object with respect to the vehicle based on an image captured from the vehicle, and a contact between the vehicle and the object based on the position. an identifying unit for identifying a first state; and an estimating unit for estimating the degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and the first state. and, when the object is a person, the estimating unit estimates the degree of risk when the moving direction of the person intersects with a first line-of-sight direction indicating the line-of-sight direction of the driver. Weighting is performed so that the degree of risk is lower than that in the case where the moving direction and the first line-of-sight direction do not intersect.

A processing method according to the present disclosure comprises: a computer detecting a position of an object relative to the vehicle based on an image taken from the vehicle; specifying a plurality of first states indicating the state of the vehicle when the contact traveling locus is drawn; and specifying a second state and the plurality of the second states indicating the state of the vehicle when the image is captured by the computer. estimating a risk associated with the object based on the difference from each of the one states.
Further, the processing method according to the present disclosure includes: a computer calculating a position of an object with respect to the vehicle based on an image taken from the vehicle; identifying a first state in which an object can come into contact; and when the object is a person, the degree of risk when the movement direction of the person and the first line-of-sight direction indicating the line-of-sight direction of the driver intersect is calculated by estimating the degree of risk of the person and weighting the movement direction less than the risk if the first line-of-sight direction does not intersect.

A program according to the present disclosure causes a computer to detect a position of an object with respect to the vehicle based on an image taken from the vehicle, and based on the position, determine a travel trajectory in which the vehicle contacts the object. Based on identifying a plurality of first states indicating the state of the vehicle when the image is drawn, and a difference between a second state indicating the state of the vehicle when the image was captured and each of the plurality of first states. , and estimating the degree of risk associated with the object.
Further, the program according to the present disclosure causes a computer to calculate a position of an object with respect to the vehicle based on an image taken from the vehicle, and based on the position, the vehicle and the object contact each other. identifying a first state to be obtained; and estimating the degree of risk associated with the object based on the difference between a second state indicating the state of the vehicle when the image was captured and the first state. , when the object is a person, the degree of risk when the direction of movement of the person and the first line-of-sight direction indicating the line-of-sight direction of the driver intersect is determined by the direction of movement of the person and the first line-of-sight direction; weighting the risk to be lower than the risk when they do not intersect with each other.

According to at least one embodiment of the present disclosure, it is possible to estimate the degree of danger of the vehicle being driven by the driver.

1 is a diagram illustrating an example configuration of a risk estimation system according to at least one embodiment of the present disclosure; FIG. 1 is a schematic diagram of an industrial vehicle viewed from directly above according to at least one embodiment of the present disclosure; FIG. 1 is a diagram illustrating an example of a configuration of an information processing device according to at least one embodiment of the present disclosure; FIG. 1 is a diagram illustrating an example of a configuration of a host device according to at least one embodiment of the present disclosure; FIG. FIG. 4 is a diagram showing an example of a processing flow of a risk estimation system according to at least one embodiment of the present disclosure; FIG. FIG. 5 is a diagram for explaining gap calculation in at least one embodiment of the present disclosure; FIG. FIG. 4 is a diagram for explaining risk estimation in at least one embodiment of the present disclosure; FIG. 1 is a diagram illustrating an example of a configuration of an information processing apparatus according to at least one embodiment of the present disclosure; FIG. 2 is a diagram illustrating an example of a configuration of a higher-level device according to at least one embodiment of the present disclosure; FIG. 1 is a diagram illustrating an example of a configuration of an information processing apparatus according to at least one embodiment of the present disclosure; FIG. 1 is a schematic block diagram showing the configuration of a computer according to at least one embodiment of the present disclosure; FIG.

<First embodiment>
The risk estimation system 100 according to the first embodiment of the present disclosure will be described below.
(Configuration of risk estimation system)
A risk estimation system 100 includes an industrial vehicle 1 (an example of a vehicle) and a host device 2 .
(Construction of industrial vehicle)
The industrial vehicle 1 can detect the steering angle of the industrial vehicle 1, the traveling direction of the industrial vehicle 1, the vehicle speed of the industrial vehicle 1, the acceleration of the industrial vehicle 1, and other detectable information of the industrial vehicle 1 (hereinafter referred to as the vehicle state) and the industrial vehicle 1. This vehicle estimates the degree of danger of the industrial vehicle 1 based on the position of the object detected with respect to the vehicle 1 .

The industrial vehicle 1 is, for example, a forklift as shown in FIG. However, construction machines such as wheel loaders, vehicles with similar mechanisms such as cargo handling equipment and rear wheel steering, or vehicles with similar problems may be used.
As shown in FIG. 1, the industrial vehicle 1 includes an operating device 10, a first camera 20a, a second camera 20b, a third camera 20c, a fourth camera 20d, a first speaker 30a, a second speaker 30b and a third speaker 30c. , a fourth speaker 30 d and a processing device 40 . The processing device 40 acquires the vehicle state of the industrial vehicle 1 from sensors (not shown) and the state of the operation device 10 .
The first camera 20a, the second camera 20b, the third camera 20c, and the fourth camera 20d are collectively referred to as an imaging device 20. As shown in FIG. Also, the first speaker 30a, the second speaker 30b, the third speaker 30c, and the fourth speaker 30d are collectively referred to as the notification device 30. FIG.

The operation device 10 is a device that receives an operation by a driver to cause the industrial vehicle 1 to move or perform cargo handling work. For example, the operation device 10 includes a shift lever that determines forward/backward movement of the industrial vehicle 1, a steering wheel that determines the steering angle of the industrial vehicle 1, an accelerator and a brake that adjust the vehicle speed and acceleration of the industrial vehicle 1, and the like.

FIG. 2 is a schematic diagram of the industrial vehicle 1 viewed from directly above. The photographing device 20 is provided so that a plurality of cameras respectively photograph the outer peripheral direction of the industrial vehicle 1 , and photograph the entire periphery of the industrial vehicle 1 .
For example, the first camera 20a, the second camera 20b, the third camera 20c, and the fourth camera 20d are provided on the upper front, upper right, upper rear, and upper left sides of the industrial vehicle 1, respectively, as shown in FIG. Then, for example, the first camera 20a photographs the first photographing area A1 shown in FIG. Similarly, the second camera 20b, the third camera 20c, and the fourth camera 20d photograph the second imaging area A2, the third imaging area A3, and the fourth imaging area A4 shown in FIG. 2, respectively.
In addition, in the first embodiment of the present disclosure, the cameras are provided in the upper front, the upper right, the upper rear, and the upper left of the industrial vehicle 1, but they may be provided in different directions. In addition, the number of cameras can be further reduced by using a lens with a wide viewing angle such as a fisheye lens. Also, if there is a blind spot, five or more cameras can be installed. Objects recognized by these cameras include people. Here, the blind spot is an area that is not photographed by any camera among the areas in which objects in the vicinity of the industrial vehicle 1 should be recognized.

Notification device 30 outputs sound under the control of processing device 40 .
For example, the first speaker 30a, the second speaker 30b, the third speaker 30c, and the fourth speaker 30d are provided at the front right, rear right, rear left, and front left of the driver's seat, respectively, as shown in FIG. When the processing device 40 detects an object in the image captured by the imaging device 20, the speaker provided in the direction corresponding to the position of the object outputs sound.
Note that, when a plurality of objects are found, the notification device 30 outputs sounds from speakers corresponding to each of the recognized objects.

The processing device 40 is a device for estimating the degree of danger of the industrial vehicle 1 based on the vehicle state of the industrial vehicle 1 and the state of the object (object state) recognized from the photographed image. The vehicle state of the industrial vehicle 1 includes the steering angle of the industrial vehicle 1 , the traveling direction of the industrial vehicle 1 , the vehicle speed of the industrial vehicle 1 , and the acceleration of the industrial vehicle 1 . Also, the processing device 40 is a device that controls the notification device 30 based on the vehicle state and the object state. Note that when a plurality of objects are recognized, the processing device 40 estimates the degree of risk for each object.
As shown in FIG. 3, the processing device 40 includes a recognition unit 401, a position calculation unit 402, (an example of a calculation unit), an acquisition unit 403, an identification unit 404, an estimation unit 405, a tagging unit 406, a recording unit 407, a control A unit 408 and a storage unit 409 are provided.

The recognition unit 401 recognizes an object in the image captured by the imaging device 20 .
For example, the recognition unit 401 stores in advance features of objects including people. Then, the recognition unit 401 repeatedly acquires images shot by the first camera 20a, the second camera 20b, the third camera 20c, and the fourth camera 20d in this order. Each time an image is acquired, the recognition unit 401 uses pattern recognition technology to determine that the acquired image includes a target object including a person stored in advance. judge.

The position calculation unit 402 calculates the position of the object with respect to the industrial vehicle 1 from the image when the object is recognized in the image acquired by the recognition unit 401 . That is, the position calculation unit 402 calculates the relative position of the object with the industrial vehicle 1 as a reference.
For example, the position calculation unit 402 calculates the position of the object with respect to the industrial vehicle 1 from the image in the following procedure. The position of the object relative to the industrial vehicle 1 is represented by the direction and distance of the object relative to the industrial vehicle 1 .

The position calculation unit 402 calculates the direction of the object with respect to the industrial vehicle 1, for example, in the following procedure. First, the position calculation unit 402 identifies the direction of the object with respect to the photographing device 20 according to the position in the image where the part where the object is captured is included (hereinafter referred to as recognition direction information). Next, the position calculation unit 402 identifies from which of the first camera 20a, the second camera 20b, the third camera 20c, and the fourth camera 20d the image in which the object is recognized was acquired ( Recognition camera information below). The position calculation unit 402 identifies the position and orientation of the camera with respect to the industrial vehicle 1 based on the recognition camera information. Then, the position calculation unit 402 calculates the direction of the object with respect to the industrial vehicle 1 based on the specified position and orientation of the camera and the recognition direction information.
Further, the position calculation unit 402 calculates the distance between the recognized object and the industrial vehicle 1, for example, in the following procedure. For example, the position calculation unit 402 stores the image size (number of pixels) corresponding to the dimensions of the object, and calculates the industrial size from the ratio of the stored image size to the size (number of pixels) of the object recognized in the image. A distance from the vehicle 1 to the object is calculated.
Accordingly, the position calculation unit 402 can calculate the position of the object with respect to the industrial vehicle 1 by specifying the direction and distance information of the object with respect to the industrial vehicle 1 .
The distance from the industrial vehicle 1 to the object included in the distance information is the distance from the camera that recognized the object. Therefore, the position calculation unit 402 can specify the position of the target as viewed from the industrial vehicle 1 by considering the position and orientation of the camera that recognizes the target, which is provided on the industrial vehicle 1 .

Note that, for example, when the imaging device 20 is a stereo camera, the position calculation unit 402 uses a triangulation technique for a pair of captured images at each imaging timing captured by the stereo camera to determine whether the industrial vehicle 1 is the target object. You can calculate the distance to Alternatively, the position calculation unit 402 may calculate the distance between the industrial vehicle 1 and the object based on the detection signal output by the distance sensor provided on the industrial vehicle 1 .

The acquisition unit 403 acquires the vehicle state of the industrial vehicle 1 from a sensor (not shown) provided on the industrial vehicle 1 and the state of the operation device 10 .
For example, the acquisition unit 403 acquires the current vehicle speed of the industrial vehicle 1 from a sensor (not shown) provided on the industrial vehicle 1 . Further, the acquisition unit 403 acquires a signal output according to changes in the operating device 10 or periodically acquires a signal indicating the state of the operating device 10 . Here, the vehicle state of the industrial vehicle 1 acquired by the acquisition unit 403 from a sensor (not shown) provided in the industrial vehicle 1 and the operation device 10 includes not only the vehicle speed but also the industrial vehicle state when the driver turns the steering wheel. The steering angle of the vehicle 1, the traveling direction of the industrial vehicle 1 when the driver switches the shift lever to forward or backward, the positive acceleration of the industrial vehicle 1 when the driver depresses the accelerator, and the driver depresses the brake. A state such as deceleration (negative acceleration) of the industrial vehicle 1 is included.

Based on the position of the object relative to the industrial vehicle 1 calculated by the position calculating unit 402 and the vehicle state of the industrial vehicle 1 acquired by the identifying unit 404, the identifying unit 404 can contact the industrial vehicle 1 with the object. A vehicle state (first state) is specified. Specifically, the identification unit 404 identifies a combination of steering angle and acceleration that can cause contact between the industrial vehicle 1 and the object based on the position of the object with respect to the industrial vehicle 1 and the current vehicle speed of the industrial vehicle 1. do. Acceleration includes both positive acceleration due to acceleration and negative acceleration due to braking. It should be noted that there may be a plurality of combinations of steering angle and acceleration that cause contact. This is because the speed of the vehicle changes depending on the acceleration, and there are a plurality of travel trajectories that collide with the detected object depending on the combination with the steering angle. That is, the larger the steering angle, the smaller the turning radius of the industrial vehicle 1, and the higher the speed (the higher the acceleration), the larger the turning radius.

The estimating unit 405 estimates the degree of danger of the industrial vehicle 1 based on the vehicle state of the industrial vehicle 1 acquired by the acquiring unit 403 and the vehicle state that the industrial vehicle 1 identified by the identifying unit 404 may come into contact with an object. do.
The estimation unit 405 estimates the degree of risk in the following procedure.

The estimation unit 405 calculates a difference (gap) between the current combination of the steering angle and the acceleration (an example of the second acceleration) specified by the specifying unit 404 for each combination of the steering angle and the acceleration specified by the specifying unit 404. . It should be noted that a smaller value of the gap means that the vehicle can come into contact with the object with a slight change in the vehicle state. Note that the gap is 0 when the vehicle can come into contact with the object while maintaining the current vehicle state.
Based on the vehicle speed of the industrial vehicle 1 and the trajectory obtained from the specified acceleration and steering angle, the estimation unit 405 calculates the time until contact with the object for each combination of the specified steering angle and acceleration.
Then, the estimation unit 405 calculates the degree of risk by substituting the gap and the time until contact with the object into a predetermined function representing the degree of risk for each combination of steering angle and acceleration.
A specific method of calculating the degree of risk by the estimation unit 405 will be described later.

Each time the estimating unit 405 calculates the degree of risk, the tagging unit 406 associates the degree of risk with predetermined tag information and writes it to the storage unit 409 . The tagging unit 406 is an example of a recording unit that associates the degree of risk with tag information and records it in the storage unit 409 . The tag information includes the ID of the driver of the industrial vehicle 1, the time when the degree of danger was calculated, the vehicle state of the industrial vehicle 1 acquired by the acquisition unit 403, the image used to estimate the degree of danger, and the recognition unit 401 This information includes the positional relationship of the object with respect to the specified industrial vehicle 1 . Hereinafter, information that associates the degree of risk with tag information is also referred to as degree of risk information.

The recording unit 407 records the risk information in the database 22 of the host device 2 by transmitting the risk information written in the storage unit 409 to the host device 2 .
It should be noted that the recording unit 407 does not write all the risk information to the host device 2, but rather writes the risk in the risk information that is greater than or equal to the first threshold value and less than or equal to the second threshold value. It may be a device that transmits an object to the higher-level device 2 . Note that the first threshold is a threshold indicating that the industrial vehicle 1 can be driven safely if it is less than the first threshold. Moreover, the second threshold is a threshold indicating that the possibility of contact between the industrial vehicle 1 and the object is high and dangerous beyond the threshold. The degree of risk that is greater than or equal to the first threshold value and less than or equal to the second threshold value is a degree of risk that does not directly lead to an accident but indicates a dangerous state (that is, a near-miss state). Also, by setting the first threshold value and the second threshold value for each driver, the higher-level device 2 can notify each driver of near misses. Also, the recording unit 407 may write the degree of risk equal to or higher than the first threshold value in the host device 2 .

The control unit 408 controls the notification device 30 based on the degree of risk calculated by the estimation unit 405 .

For example, when the position of the object relative to the industrial vehicle 1 obtained from the acquired image is the right front of the industrial vehicle 1, the degree of risk associated with the image exceeds a predetermined third threshold. Furthermore, the control unit 408 controls the notification device 30 so that sound is output from the first speaker 30a. The third threshold is a risk level threshold for determining whether or not to output sound. The third threshold may be the same value as the first threshold or the second threshold. If the third threshold is different from the first threshold or the second threshold, control different from the first threshold or the second threshold may be performed.
Similarly, if the position of the object relative to the industrial vehicle 1 is the right rear, left rear, and left front of the industrial vehicle 1, when the risk associated with the image exceeds the third threshold, the control The unit 408 controls the notification device 30 so that sounds are output from the second speaker 30b, the third speaker 30c, and the fourth speaker 30d, respectively.

The storage unit 409 stores various information necessary for processing performed by the processing device 40 as described above.

(How to calculate the degree of risk)
Here, an example of a method of estimating the degree of risk by the estimating unit 405 will be described.
First, as shown in FIG. 4, the specifying unit 404 sets a plurality of steering angles in the steering angle range that the industrial vehicle 1 can take, using the position of the object with respect to the industrial vehicle 1 and the current vehicle speed of the industrial vehicle 1 as initial conditions. Acceleration that causes the industrial vehicle 1 to come into contact with the object is calculated (for example, in increments of 1 degree). For example, the identification unit 404 uses a function (i.e., a curve function that draws the trajectory of the industrial vehicle 1) in which the initial velocity, the acceleration, and the steering angle are constants, the time is the explanatory variable, and the position of the industrial vehicle 1 is the objective variable. Acceleration is obtained so that the trajectory of the vehicle 1 passes the position of the object. Note that, depending on the steering angle, there may be no acceleration that causes the industrial vehicle 1 to contact the object. The identifying unit 404 extracts steering angles that have such an acceleration that the industrial vehicle 1 contacts an object from among the plurality of steering angles. Hereinafter, the combination of the extracted steering angle and acceleration will also be referred to as a contact case. As shown in FIG. 4, the specifying unit 404 assigns a case number to each contact case. In FIG. 4, regardless of whether or not contact with the industrial vehicle 1 occurs, the current state of the industrial vehicle 1 is shown in case no. 0 is assigned.

The estimation unit 405 calculates the gap (difference) between the current state of the industrial vehicle 1 (state related to case No. 0) and the state related to the contact case for each of the extracted contact cases. For example, the estimation unit 405 multiplies a value obtained by multiplying the difference between the current steering angle and the steering angle related to the contact case by a predetermined weight, and multiplies the difference between the current acceleration and the acceleration related to the contact case by a predetermined weight. The sum with the value (weighted Manhattan distance) is calculated as the gap. Alternatively, the estimation unit 405 may calculate the gap based on other criteria such as Euclidean distance.
Specifically, in the example shown in FIG. The gap of 2 is 8. In contrast, case no. 3 is case no. Compared to 2, the difference from the current situation is large in both the steering angle and the acceleration. Therefore, case no. 3 gap is case no. 10, which is greater than the gap of 2.

For each of the extracted contact cases, the estimation unit 405 calculates the time until the industrial vehicle 1 contacts the object based on the current vehicle speed of the industrial vehicle 1 and the acceleration related to the contact case. For example, the estimation unit 405 obtains the time when the position of the industrial vehicle 1 matches the position of the object based on the curve function.

Then, the estimating unit 405 estimates the degree of risk for each contact case by substituting the gap and the time until contact with the object into a predetermined function representing the degree of risk. As shown in FIG. 5, the estimating unit 405 estimates the degree of danger using a predetermined function in which the degree of danger increases as the gap becomes smaller, and the degree of danger increases as the time until contact with the object becomes shorter. do. For example, the estimation unit 405 may estimate the degree of risk by obtaining the reciprocal of the sum of the value obtained by multiplying the gap by a predetermined weight and the value obtained by multiplying the time until contact by a predetermined weight.
Note that if the gap is larger than a certain value, or if the time until contact with the object is longer than the certain time, the possibility that the industrial vehicle 1 will be in such a situation is low and the risk is low. Alternatively, based on the idea that there is sufficient time to avoid danger and the degree of danger is low, the estimation unit 405 may exclude those cases from the estimation of the degree of danger, as shown in FIG. .
Also, the estimation unit 405 may change the function used for estimating the degree of risk according to the situation. For example, the estimating unit 405 may perform weighting such that the smaller the gap, the higher the degree of risk compared to the proportionate degree of risk. In addition, the estimation unit 405 may perform weighting such that the degree of risk becomes higher as the time until contact with the object becomes shorter, compared to the degree of risk proportional to the time.

(Configuration of host device)
The host device 2 includes a receiving unit 21, a database 22 (an example of a storage unit), and a notification unit 23, as shown in FIG.
The receiving unit 21 receives risk information from the industrial vehicle 1 . The receiving unit 21 records the received risk information in the database 22 .
The database 22 stores various information necessary for processing performed by the host device 2 .

The notification unit 23 notifies each driver of the risk associated with the tag information stored in the database 22 . That is, the notification unit 23 notifies the driver of the notification target of the risk information stored in the database 22 that includes the tag information that holds the ID of the driver of the notification target. As a result, the driver can grasp a near-miss that occurred while driving the industrial vehicle 1 and the operation in the state in which the near-miss occurred. In other words, even when a near-miss occurs without the driver's awareness, the host device 2 can make the driver aware of the existence of the near-miss. As a result, the driver can be expected to drive cautiously when similar near-miss situations occur thereafter.
In addition, the notification allows the driver to improve the driving skill of the industrial vehicle 1 by practicing the operation in the near-miss state. As a result, dangerous driving can be reduced and accidents can be reduced.

(Processing performed by information processing device)
Next, the processing performed by the risk estimation system 100 for calculating the risk will be described.
Here, the processing flow of the risk estimation system 100 shown in FIG. 7 will be described.
It is assumed that the imaging device 20 is imaging the entire circumference of the industrial vehicle 1 . The processing device 40 executes the processing shown in FIG. 7 at each predetermined control cycle.

The recognition unit 401 acquires an image captured by the imaging device 20 (step S1). The recognition unit 401 uses a pattern recognition technique to determine whether or not the image acquired in step S1 includes an object (step S2).

When the recognizing unit 401 determines that the acquired image does not include the target object (NO in step S2), the process ends.

When the recognition unit 401 determines that the acquired image includes the object (YES in step S2), the position calculation unit 402 calculates the position of the object with respect to the industrial vehicle 1 based on the image. Calculate (step S3).

The acquisition unit 403 acquires the current vehicle state of the industrial vehicle 1 from a sensor or the like (not shown) provided on the industrial vehicle (step S4).

The identification unit 404 determines whether the industrial vehicle 1 contacts the object based on the position of the object relative to the industrial vehicle 1 identified by the recognition unit 401 in step S3 and the current vehicle speed of the industrial vehicle 1 acquired in step S4. Such a combination of steering angle and acceleration (contact case) is identified (step S5).

The estimation unit 405 calculates the gap between the current vehicle state acquired in step S4 and the vehicle state related to the contact case for each identified contact case (step S6).

The estimating unit 405 calculates the time until contact with the target object for each specified contact case based on the vehicle speed of the industrial vehicle 1 acquired in step S4 and the trajectory obtained from the acceleration and steering angle related to the specified contact case. Calculate (step S7).

The estimation unit 405 estimates the degree of risk for each contact case based on the gap calculated in step S6 and the time until contact with the object calculated in step S7 (step S8).

The control unit 408 determines whether or not the risk estimated by the estimation unit 405 exceeds a predetermined third threshold (step S9). If the degree of danger exceeds the third threshold (YES in step S9), control unit 408 controls notification device 30 so that sound is output from the speaker corresponding to the object (step S10). . On the other hand, if the degree of danger does not exceed the third threshold (NO in step S9), control unit 408 does not output sound from the speaker.

Each time the estimating unit 405 calculates the degree of risk, the tagging unit 406 associates the degree of risk with predetermined tag information to generate the degree of risk information and writes it to the storage unit 409 (step S11). Then, the recording unit 407 transmits the risk level information associated with the storage unit 409 to the host device 2 immediately or at a prescribed timing (step S12).

The receiving unit 21 of the host device 2 receives the risk information and records the risk information in the database 22 . The notification unit 23 generates a near-miss report for each driver based on the degree-of-risk information stored in the database 22 . Then, the notification unit 23 notifies each driver of the generated near-miss report. As a result, the driver can grasp the operation in a state in which a near-miss occurs while driving the industrial vehicle 1 .

(action, effect)
The risk estimation system 100 according to the first embodiment of the present disclosure has been described above.
When determining that the object is recognized in the acquired image, the position calculation unit 402 calculates the position of the recognized object with respect to the industrial vehicle 1 from the image and the state of the object. Based on the position of the object with respect to the industrial vehicle 1 and the current vehicle speed of the industrial vehicle 1, the estimation unit 405 identifies a combination of steering angle and acceleration that causes the industrial vehicle 1 to contact the object. The estimating unit 405 calculates a difference (gap) between the current combination of the steering angle and the acceleration for each specified combination of the steering angle and the acceleration. The estimation unit 405 calculates the time until contact with the object based on the vehicle speed of the industrial vehicle 1 and the trajectory obtained from the specified acceleration and steering angle. Then, the estimation unit 405 calculates the degree of risk by substituting the gap and the time until contact with the object into a predetermined function representing the degree of risk.
By doing so, the processing device 40 can objectively evaluate the difficulty of transition from the current state of the industrial vehicle 1 to the state of the industrial vehicle 1 with the possibility of contact as a gap. Then, the processing device 40 can objectively identify the degree of danger of the industrial vehicle 1 by using the time until contact and the gap.

<Second embodiment>
Next, the risk estimation system 100 according to the second embodiment of the present disclosure will be described.
A target object in the second embodiment of the present disclosure is a pedestrian (an example of a person).
The estimation system 100 according to the second embodiment differs from the configuration of the first embodiment in the configuration of the industrial vehicle 1 and the processing device 40 .
In addition to the configuration of the first embodiment, the industrial vehicle 1 according to the second embodiment further includes a camera (not shown) for capturing an image of the driver's face in the driver's cab.

(Configuration of information processing device)
A processing device 40 according to the second embodiment of the present disclosure further includes a driver line-of-sight detection unit 410 and a pedestrian line-of-sight detection unit 411 in addition to the configuration of the first embodiment, as shown in FIG.

Driver line-of-sight detection unit 410 detects the line-of-sight direction of the driver. For example, the driver line-of-sight detection unit 410 acquires an image of the driver's face from a camera (not shown), performs image analysis, identifies the direction of the face and the direction of the eyes, and based on this, identifies the direction of the driver's face and the direction of the eyes. Detect eye direction. The line-of-sight direction of the driver is represented, for example, as a vector extending from the origin in a coordinate system based on the industrial vehicle 1 .

The pedestrian line-of-sight detection unit 411 detects the line-of-sight direction of the pedestrian. For example, the pedestrian line-of-sight detection unit 411 acquires an image of the pedestrian's face and performs image analysis to detect the line-of-sight direction of the pedestrian based on the direction of the face and the direction of the eyes. The line-of-sight direction of the pedestrian is expressed as a vector extending from the position of the pedestrian identified by the recognition unit 401 in a coordinate system based on the industrial vehicle 1 .
Also, the pedestrian line of sight detection unit 411 detects the moving direction of the pedestrian. For example, the pedestrian line-of-sight detection unit 411 acquires an image of a pedestrian's face and body, and performs image analysis to detect the moving direction of the face and body in the image as the pedestrian's moving direction. The moving direction of the pedestrian is expressed as a vector extending from the position of the pedestrian specified by the recognition unit 401 in the coordinate system with the industrial vehicle 1 as a reference.

The estimation unit 405 weights the degree of risk based on at least one of the driver's line-of-sight direction (an example of the first line-of-sight direction) and the pedestrian's line-of-sight direction (an example of the second line-of-sight direction).

For example, if the estimating unit 405 determines that a vector indicating the moving direction of the pedestrian (an example of the first vector) and a vector indicating the line-of-sight direction of the driver (an example of the second vector) intersect, the estimating unit 405 determines the degree of risk. Weighting to lower. If the estimating unit 405 determines that the vector indicating the moving direction of the pedestrian and the vector indicating the first line-of-sight direction indicating the line-of-sight direction of the driver do not intersect, the estimating unit 405 performs weighting to increase the degree of risk.
This is based on the idea that the pedestrian comes into the driver's field of vision and the driver recognizes the pedestrian in order to indicate that the pedestrian is moving in the line-of-sight direction of the driver.

In addition, the estimating unit 405 performs weighting to lower the degree of risk when the line-of-sight direction of the driver is within a range (an example of the first range) in which the driver can recognize the pedestrian. In addition, the estimation unit 405 performs weighting to increase the degree of danger when the line-of-sight direction of the driver is outside the range in which the driver can recognize the pedestrian.

In addition, the estimating unit 405 performs weighting to lower the degree of danger when the line-of-sight direction of the pedestrian is within the range (an example of the second range) indicating the range in which the industrial vehicle 1 can be recognized. Moreover, the estimation unit 405 performs weighting to increase the degree of danger when the line-of-sight direction of the pedestrian is outside the range indicating the range in which the industrial vehicle 1 can be recognized.

(action, effect)
The risk estimation system 100 according to the second embodiment of the present disclosure has been described above.
In the processing device 40 of the risk estimation system 100, the estimation unit 405 weights the risk based on at least one of the line-of-sight direction of the driver and the line-of-sight direction of the pedestrian.
By doing so, the processing device 40 can reflect at least one of whether or not the driver is aware of the pedestrian and whether or not the pedestrian is aware of the industrial vehicle 1 in the degree of risk. The degree of danger of the industrial vehicle 1 can be specified as a more appropriate value that matches the actual situation.

<Third Embodiment>
Next, the risk estimation system 100 according to the third embodiment of the present disclosure will be described.
The estimation system 100 according to the third embodiment differs from the configuration of the first embodiment in the configuration of the host device 2 .

(Configuration of host device)
As shown in FIG. 9, the host device according to the third embodiment further includes a learning section 24 and an analysis section 25 in addition to the configuration of the first embodiment.

Based on the risk information stored in the database 22, the learning unit 24 analyzes the driving tendency of each driver when the risk increases. Specifically, the learning unit 24 uses information corresponding to a near-miss incident for each driver, that is, information with a degree of danger exceeding a first threshold, out of the risk information stored in the database 22, to determine whether a near-miss incident occurs. The state of the industrial vehicle 1 when it occurs is machine-learned. For example, the learning unit 24 selects the time, the steering angle of the industrial vehicle 1, the traveling direction of the industrial vehicle 1, the vehicle speed of the industrial vehicle 1, the industrial Using the acceleration of the vehicle 1, the direction of the object, and the distance to the object as samples, the parameters of the Gaussian mixture model are learned by unsupervised learning. Thereby, the Gaussian mixture model can represent the distribution of the state of the industrial vehicle 1 when a near miss occurs. A Gaussian mixture model is an example of a machine learning model. The learning unit 24 learns parameters for each driver. In other words, a trained model is generated for each driver.

The analysis unit 25 analyzes the parameters of the Gaussian mixture model learned by the learning unit 24, and identifies the state of the industrial vehicle 1 in which near misses are likely to occur for each driver. Specifically, the analysis unit 25 identifies a state in which near misses are likely to occur according to the following procedure. First, the analysis unit 25 extracts all of the probability density functions that make up the Gaussian mixture model and whose variance parameters are equal to or less than a predetermined threshold. Then, for each probability density function, the analysis unit 25 extracts a combination in which the extracted attributes take values close to the average value as a state in which near misses are likely to occur. For example, a certain probability density function has a time variance less than or equal to a predetermined threshold value, the mean value of which is 17 o'clock, and an object direction variance of which is less than or equal to a predetermined threshold value, and the mean value of which is In the case of indicating the right front, it can be seen that the driver is likely to have a near miss when there is an object in the right front around 17:00.
Then, the notification unit 23 generates, for each driver, a report indicating the situation in which near misses are likely to occur, which is indicated by the analysis result of the analysis unit 25, and notifies the driver.

(action, effect)
The risk estimation system 100 according to the third embodiment of the present disclosure has been described above.
In the host device 2 of the risk estimation system 100, the learning unit 24 learns a machine learning model such as a Gaussian mixture model using the risk information for each driver. Then, the analysis unit 25 analyzes the machine learning model that has been learned, and estimates situations in which near misses are likely to occur for each driver.
By doing so, the host device 2 can notify each driver of a situation in which near misses are likely to occur. Situations in which near misses are likely to occur differ from driver to driver, so the driver can use the information to recognize the situations and habits that the driver is not good at.

In the estimation system 100 according to the third embodiment, the host device 2 performs machine learning, but the present invention is not limited to this. For example, the estimation system 100 according to another embodiment may perform machine learning in the processing device 40 .

<Fourth Embodiment>
Next, a risk estimation system 100 according to the fourth embodiment of the present disclosure will be described.
The industrial vehicle 1 according to the fourth embodiment restricts the driving operation regardless of the intention of the driver according to the degree of danger calculated by the processing device 40 .
The estimation system 100 according to the fourth embodiment differs from the first embodiment in the configuration of the processing device 40 .

As shown in FIG. 10, the processing device 40 according to the fourth embodiment further includes a learning unit 412, an analysis unit 413 and a generation unit 414 in addition to the configuration of the first embodiment.
Similar to the learning unit 24 of the host device 2 according to the third embodiment, the learning unit 412, based on the risk information for each driver stored in the storage unit 409, detects industrial vehicles that are likely to cause near misses for each driver. Learn the distribution of 1 states.
The analysis unit 413 analyzes the state of the industrial vehicle 1 in which near misses are likely to occur from the learning result of the learning unit 412, like the analysis unit 25 of the host device 2 according to the third embodiment.
Based on the analysis result of the analysis unit 413, the generation unit 414 generates a control signal for restricting the operation of the industrial vehicle 1 that is likely to cause near misses. For example, if the result of the analysis by the analysis unit 413 is that near-miss incidents are likely to occur near the maximum speed, the generation unit 414 sets the speed so that the industrial vehicle 1 does not reach the maximum speed when the degree of danger is higher than the first threshold. generates a control signal that limits the Examples of speed limiting methods include a method of limiting the movable range of the accelerator so that the accelerator cannot be physically depressed to the maximum, and a method of reducing the magnitude of acceleration relative to the amount of depression of the accelerator. Further, for example, if the result of the analysis by the analysis unit 413 indicates that near-miss incidents are likely to occur when the steering angle is close to the left-right maximum value, the generation unit 414 generates a steering angle when the degree of risk is higher than the first threshold. A control signal is generated to limit the angle so that it does not reach the maximum steering angle. Methods of limiting the steering angle include, for example, a method of limiting the movable range of the steering so that the steering cannot be physically turned to the maximum, and a method of reducing the magnitude of the steering angle relative to the amount of operation of the steering.
The generating unit 414 then outputs the generated control signal to the operating device 10 .

(action, effect)
The risk estimation system 100 according to the fourth embodiment of the present disclosure has been described above.
In the processing device 40 of the risk estimation system 100, the generator 414 generates a control signal for limiting the operation range according to the calculated risk. The generator 414 outputs the generated control signal to the operating device 10 .
By doing so, the industrial vehicle 1 is less likely to be in a high-risk state. As a result, accidents caused by the industrial vehicle 1 can be reduced.

<Other embodiments>
Note that, in the first to fourth embodiments of the present disclosure, the notification device 30 has been described as including the first speaker 30a, the second speaker 30b, the third speaker 30c, and the fourth speaker 30d. However, in another embodiment of the present disclosure, the notification device 30 outputs a sound from a direction or area corresponding to the position where the object exists relative to the industrial vehicle 1 when the object is recognized. The number of speakers and the position where the speakers are installed are not limited as long as they are speakers that can

It should be noted that the order of the processes in the embodiment of the present disclosure may be changed as long as appropriate processes are performed.

Each of the database 22, storage unit 409, and other storage devices in the embodiment of the present disclosure may be provided anywhere as long as appropriate information is transmitted and received. Further, each of the database 22, the storage unit 409, and other storage devices may exist in a plurality and store data in a distributed manner within a range in which appropriate information transmission/reception is performed.

Although embodiments of the present disclosure have been described, the processing device 40 and other control devices described above may have a computer system therein. The process of the above-described processing is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by reading and executing this program by a computer. Specific examples of computers are shown below.
FIG. 11 is a schematic block diagram showing the configuration of a computer according to at least one embodiment;
The computer 5 includes a CPU 6, a main memory 7, a storage 8, and an interface 9, as shown in FIG.
For example, each of the processing device 40 and other control devices described above is implemented in the computer 5 . The operation of each processing unit described above is stored in the storage 8 in the form of a program. The CPU 6 reads out the program from the storage 8, develops it in the main memory 7, and executes the above process according to the program. In addition, the CPU 6 secures storage areas corresponding to the storage units described above in the main memory 7 according to the program.

Examples of the storage 8 include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory). , semiconductor memory, and the like. The storage 8 may be an internal medium directly connected to the bus of the computer 5, or an external medium connected to the computer 5 via the interface 9 or communication line. Further, when this program is distributed to the computer 5 through a communication line, the computer 5 that receives the distribution may develop the program in the main memory 7 and execute the above process. In at least one embodiment, storage 8 is a non-transitory, tangible storage medium.

Further, the program may implement part of the functions described above. Furthermore, the program may be a file capable of realizing the above functions in combination with a program already recorded in the computer system, that is, a so-called difference file (difference program).
In other embodiments, each of the processing device 40 and other control devices includes a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration, ASICs (Application Specific Integrated Circuits), GPUs (Graphics Processing Units), and similar processing devices may be provided. Examples of PLD include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, part or all of the functions implemented by the processor may be implemented by the integrated circuit.

While several embodiments of the disclosure have been described, these embodiments are examples and do not limit the scope of the disclosure. Various additions, omissions, replacements, and modifications may be made to these embodiments without departing from the gist of the disclosure.

<Appendix>
The processing device 40, the information processing method, the estimation system, and the program described in each embodiment of the present disclosure are understood, for example, as follows.

(1) A processing device (40) according to a first aspect includes a calculation unit (402) that calculates the position of an object relative to the vehicle based on an image taken from the vehicle, and the vehicle based on the position. A specifying unit (404) that specifies a first state indicating a state in which (1) can come into contact with the object, and a second state and the first state that indicate the state of the vehicle when the image was captured. and an estimating unit (405) for estimating the degree of risk associated with the object based on the difference between.

This processing device (40) can appropriately identify the degree of risk of the vehicle (1) being driven by the driver.

(2) A processing device (40) according to a second aspect is the processing device (40) of (1), wherein the specifying unit (404) determines based on the position that the vehicle (1) is the The time until contact with the object may be calculated, and the estimation unit (405) may estimate the degree of risk based on the difference and the time.

With this processing device (40), it is possible to calculate a degree of risk that is closer to the actual state than the time it takes for the vehicle (1) to come into contact with the object.

(3) A processing device (40) according to a third aspect is the processing device (40) of (1) or (2), wherein the estimation unit (405) estimates the degree of risk as the difference becomes smaller. It may be calculated to be high.

It can be seen that when the first state in which the object can be contacted is close to the current second state, the possibility of contact with the object is high even if the amount of operation by the driver is small. Therefore, the processing device (40) can appropriately estimate the degree of risk of the vehicle (1) being driven.

(4) A processing device (40) according to a fourth aspect is the processing device (40) according to any one of (1) to (3), wherein the first state and the second state are the vehicle It may include the steering angle of (1) and the acceleration of the vehicle.

Since the direction of travel of the vehicle (1) is determined by the steering angle and acceleration of the vehicle (1), the processing device (40) can appropriately identify the degree of risk of the vehicle (1) during driving. .

(5) A processing device (40) according to a fifth aspect is the processing device (40) according to any one of (1) to (4), wherein the estimation unit (405) The degree of risk may be weighted based on a first line-of-sight direction indicating a direction, or a second line-of-sight direction indicating a line-of-sight direction of a person when the object is a person.

With this processing device (40), the state of the driver or the state of the target person can be reflected in the degree of risk, and the degree of risk of the vehicle (1) being driven by the driver can be specified more appropriately. be able to.

(6) The processing device (40) according to the sixth aspect is the processing device (40) of (5), wherein the estimation unit (405), when the object is a person, Weighting is performed so that a risk level when the direction intersects with the first line-of-sight direction is lower than a risk level when the moving direction of the person and the first line-of-sight direction do not intersect. good too.

With this processing device (40), the driver's condition can be reflected in the degree of risk, so the degree of risk of the vehicle (1) being driven by the driver can be specified more appropriately.

(7) A processing device (40) according to a seventh aspect is the processing device (40) according to (5) or (6), wherein the estimating unit (405) determines that the first line-of-sight direction is the driving direction. When the target object is within a first range indicating the range in which a person can recognize the target object, the risk level is weighted to be low, and when the first line-of-sight direction is outside the first range, the risk level is determined. may be weighted to increase

With this processing device (40), the driver's condition can be reflected in the degree of risk, so the degree of risk of the vehicle (1) being driven by the driver can be specified more appropriately.

(8) The processing device (40) according to the eighth aspect is the processing device (40) according to any one of (5) to (7), wherein the estimation unit (405) is configured such that the second line-of-sight direction is , when the person is within a second range indicating the range in which the vehicle (1) can be recognized, weighting is performed to lower the risk, and when the second line-of-sight direction is outside the second range , weighting may be performed to increase the degree of risk.

With this processing device (40), the state of a person, who is an object, can be reflected in the degree of risk, so the degree of risk of the vehicle (1) being driven can be more appropriately specified.

(9) A processing device (40) according to a ninth aspect is the processing device (40) according to any one of (1) to (8), wherein the estimation unit (405) determines that the object is a person. , the degree of risk may be weighted based on the moving direction of the person.

With this processing device (40), it is possible to reflect the state of a person, who is an object, on the degree of risk, so that the degree of risk of the vehicle (1) being driven by the driver can be specified more appropriately.

(10) The processing device (40) according to the tenth aspect is the processing device (40) according to any one of (1) to (9), wherein the estimation unit (405) is in the first state If the distance is greater than a certain value or the time required for the vehicle to come into contact with the object is longer than the certain time, the degree of risk is not estimated.

Regarding a certain first state, if the distance until contact with the object is long, the possibility that the vehicle (1) will be in such a state is low and the degree of danger is low. Also, when the time until the vehicle (1) contacts the object is long, there is enough time to avoid danger and the degree of danger is low. Therefore, the processing device (40) can reduce unnecessary calculations by excluding such a first state from risk estimation.

(11) The processing device (40) according to the eleventh aspect is the processing device (40) according to any one of (1) to (10), wherein the degree of risk is equal to or higher than the first threshold A recording unit (406, 407) that records the degree of risk in a database (22, 409) in association with tag information may be provided.

With this processing device (40), it is possible to record high-risk incidents such as near misses.

(12) A processing device (40) according to a twelfth aspect is the processing device (40) of (11), wherein the recording units (406, 407) have a first threshold value or more of the degrees of risk and Risk levels equal to or lower than a second threshold greater than the first threshold may be recorded in a database (22, 409) in association with tag information.

With this processing device (40), it is possible to selectively record near-miss incidents that are neither too low nor too high in risk.

(13) A processing device (40) according to a thirteenth aspect is the processing device (40) of (11) or (12), wherein the recording units (406, 407) record the The degree of risk and the tag information may be associated with each other and recorded in the database (22, 409).

This processing device (40) makes it possible to easily manage the degree of risk for each driver.

(14) A processing device (40) according to a fourteenth aspect is the processing device (40) of (13), wherein the learning unit learns parameters of a machine learning model using the tag information for each driver. (412) and an analysis unit (413) that analyzes the state of the vehicle when the degree of danger is high from the learned machine learning model for each driver.

With this information processing device (40), it is possible to analyze situations in which near misses are likely to occur for each driver, and to feed back the situation for each driver.

(15) A processing device (40) according to a fifteenth aspect is the processing device (40) according to any one of (1) to (14), wherein said A generator (414) that generates a control signal that limits the operating range of the vehicle.

With this processing device (40), it becomes difficult for the driver to be in a high-risk state even if the driver is not aware of it, and accidents can be reduced.

(16) The processing device (40) according to the sixteenth aspect is the processing device (40) of (15), wherein the generation unit (414) generates Then, the control signal is generated for each driver to limit the operation of the vehicle to approach the state of the vehicle when the degree of danger is high.

With this processing device (40), it is possible to reduce accidents by restricting operations that tend to lead to a high-risk state for each driver.

(17) An estimation system (100) according to a seventeenth aspect comprises a host device (2) comprising the processing device (40) according to any one of (11) to (16) and the database (22) The database (22) associates and stores the risk level and the tag information for each driver.

This estimation system (100) allows the host device (2) to collect high-risk states for each driver.

(18) The estimation system (100) according to the eighteenth aspect is the estimation system of (17), wherein the higher-level device (2) receives information about the risk recorded in the database (22), A notification unit (23) that notifies each driver is further provided.

With this estimation system (100), it is possible to individually inform each driver of information regarding the degree of danger of the driver.

(19) A processing method according to a nineteenth aspect includes detecting a position of an object relative to the vehicle based on an image taken from the vehicle, and detecting contact between the vehicle and the object based on the position. identifying a possible first state; and estimating the degree of risk associated with the object based on the difference between a second state indicating the state of the vehicle when the image was captured and the first state. and including.

This processing method allows the driver to appropriately estimate the degree of risk of the vehicle (1) being driven.

(20) A program according to a twentieth aspect causes a computer to identify a position of an object relative to the vehicle based on an image taken from the vehicle being driven, and based on the position, identify the vehicle and the object. Identifying a first state in which an object may come into contact with the object, and detecting a danger associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and the first state. estimating the degree;

This program enables the driver to appropriately estimate the degree of risk of the vehicle (1) being driven.

DESCRIPTION OF SYMBOLS 1... Industrial vehicle 2... Host device 5... Computer 6... CPU
7 Main memory 8 Storage 9 Interface 10 Operating device 20 Photographing device 20a First camera 20b Second camera 20c Third camera 20d Fourth camera 21 Reception unit 22 Database 23 Notification unit 24 Learning unit 25 Analysis unit 30 Notification device 30a First speaker 30b Second speaker 30c Third speaker 30d Fourth speaker 40 Processing device 401 Recognition unit 402 Calculation unit 403 Acquisition unit 404 Identification unit 405 Estimate unit 406 Tagging unit 407 Recording unit 408 Control unit 409 Storage unit 410 Driver sight line detection unit 411 Pedestrian sight line detection unit 412 ..Learning unit 413..Analyzing unit 414..Generating unit

Claims (22)

a calculation unit that calculates the position of an object relative to the vehicle based on an image captured from the vehicle;
a specifying unit that specifies, based on the position, a plurality of first states indicating states of the vehicle when the vehicle draws a travel trajectory in which the vehicle contacts the object;
an estimating unit for estimating the degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and each of the plurality of first states;
A processing device comprising: The identification unit
calculating the time until the vehicle contacts the object based on the position;
The estimation unit
estimating the risk based on the difference and the time;
2. The processing apparatus of claim 1. The estimation unit
estimating the risk higher as the difference is smaller,
3. The processing apparatus according to claim 1 or 2. 4. The processing device according to any one of claims 1 to 3, wherein said first state and said second state include a steering angle of said vehicle and an acceleration of said vehicle. The estimation unit
weighting the degree of risk based on at least one of a first line-of-sight direction indicating a line-of-sight direction of a driver and a second line-of-sight direction indicating a line-of-sight direction of the person when the object is a person;
The processing apparatus according to any one of claims 1 to 4. a calculation unit that calculates the position of an object relative to the vehicle based on an image captured from the vehicle;
a specifying unit that specifies a first state in which the vehicle and the object can come into contact based on the position;
an estimating unit for estimating a degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and the first state;
with
The estimation unit
When the object is a person, the degree of risk when the direction of movement of the person and the first line-of-sight direction indicating the line-of-sight direction of the driver intersect is determined by the direction of movement of the person and the first line-of-sight direction. give a low weight relative to the risk of not crossing,
processing equipment. The estimation unit
When the first line-of-sight direction is within a first range indicating a range in which the driver can recognize the object, weighting is performed to lower the risk, and the first line-of-sight direction If out of range, weighting the risk to increase;
7. A processing apparatus according to claim 5 or claim 6. The estimation unit
Weighting to lower the degree of risk when a second line-of-sight direction indicating a line-of-sight direction of the person when the object is a person is within a second range indicating a range in which the person can recognize the vehicle. and if the second line-of-sight direction is outside the second range, weighting is performed to increase the risk;
The processing apparatus according to any one of claims 5 to 7. The estimation unit
When the object is a person, weighting the degree of risk based on the moving direction of the person;
The processing apparatus according to any one of claims 1 to 8. The estimation unit
from claim 1, wherein, among the first states, a state in which the distance is greater than a predetermined value or a state in which the time until the vehicle contacts the object is longer than a predetermined time is excluded from the estimation of the degree of risk. 10. A processing apparatus according to any one of claims 9 to 10. a recording unit that records in a database the degree of risk that is equal to or greater than a first threshold among the degrees of risk in association with tag information;
The processing apparatus according to any one of claims 1 to 10, comprising: The recording unit records, in the database, a degree of risk that is greater than or equal to the first threshold value and less than or equal to a second threshold value that is greater than the first threshold value among the degrees of risk in association with the tag information. 12. The processing apparatus according to 11. The recording unit
For each driver, the risk level and the tag information are associated and recorded in the database;
13. A processing apparatus according to claim 11 or 12. a learning unit for learning parameters of a machine learning model using the tag information for each driver;
14. The processing device according to claim 13, further comprising: an analysis unit that analyzes, for each driver, the state of the vehicle when the degree of danger is high from the learned machine learning model. a generation unit that generates a control signal that limits an operation range of the vehicle according to the degree of risk;
15. A processing apparatus according to any one of claims 1 to 14, further comprising: wherein, when the degree of risk is higher than a first threshold value, the generation unit generates the control signal for limiting the operation of the vehicle that approaches the state of the vehicle when the degree of risk is high for each driver. 16. The processing apparatus according to 15. a processing apparatus according to any one of claims 11 to 14;
a host device comprising the database; and
The database associates and stores the risk level and the tag information for each driver.
estimation system. The host device is
a notification unit that notifies each driver of information about the degree of risk recorded in the database;
18. The estimation system of claim 17, further comprising: a computer detecting a position of an object relative to the vehicle based on an image taken from the vehicle;
the computer identifying, based on the positions, a plurality of first states indicating states of the vehicle when the vehicle draws a travel trajectory in which the vehicle contacts the object;
the computer estimating the degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and each of the plurality of first states;
processing methods, including; to the computer,
Detecting a position of an object relative to the vehicle based on an image taken from the vehicle; and Based on the position, a plurality of states of the vehicle when drawing a travel trajectory in which the vehicle contacts the object. identifying a first state of
estimating the degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and each of the plurality of first states;
program to run. a computer calculating a position of an object relative to the vehicle based on images taken from the vehicle;
the computer identifying a first state in which the vehicle and the object may contact based on the position;
the computer estimating the degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and the first state;
When the object is a person, the degree of risk when the direction of movement of the person and the first line-of-sight direction indicating the line-of-sight direction of the driver intersect is determined by the direction of movement of the person and the first line-of-sight direction. weighting it lower than the risk if it does not cross. to the computer,
calculating a position of an object with respect to the vehicle based on an image captured from the vehicle; identifying a first state in which the vehicle and the object can contact based on the position;
estimating the degree of risk associated with the object based on a difference between a second state indicating the state of the vehicle when the image was captured and the first state;
When the object is a person, the degree of risk when the direction of movement of the person and the first line-of-sight direction indicating the line-of-sight direction of the driver intersect is determined by the direction of movement of the person and the first line-of-sight direction. A program that causes the weighting to be low relative to the risk of not crossing and

Images