JP2020052869A - Identification device, identification method, and identification program - Google Patents

Abstract

To provide an identification device, an identification method, and an identification program capable of immediately identifying that an occupant of a moving object comes into an abnormal state.SOLUTION: An identification device for identifying a state of an occupant of a moving object, includes: an input part for inputting an image including the occupant obtained by an imaging device; and an identification part for identifying whether or not the state of the occupant is in an abnormal state on the basis of a relationship between a change in a position of the head and a change in a position of a predetermined portion other than the head obtained from the image.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to an identification device, an identification method, and an identification program.

  2. Description of the Related Art Conventionally, a technique for identifying a state of a driver of a vehicle is known. Patent Document 1 discloses that a state of a driver is identified based on at least one of physiological data of the driver, driving of the vehicle, and a model of the driver driving the vehicle.

Japanese Patent No. 6219952

  It is required to identify the driver in an abnormal state earlier. Therefore, the technique described in Patent Document 1 has room for improvement. An object of the present disclosure is to identify an occupant of a moving object in an abnormal state earlier.

  One embodiment of the present disclosure is an identification device that identifies a state of an occupant of a moving object, an input unit that inputs an image including the occupant obtained by an imaging device, and the occupant obtained from the image. An identification unit that identifies whether or not the occupant is in an abnormal state based on a relationship between a change in the position of the head and a change in the position of the predetermined part other than the head. is there.

  Note that one embodiment of the present disclosure may be any of a method and a program.

  According to the present disclosure, it is possible to identify earlier that the occupant of the moving object has become abnormal.

FIG. 1 is a block diagram illustrating a configuration of a driver monitoring system equipped with an identification device according to an embodiment of the present disclosure. FIG. 2 is a flowchart illustrating an example of the identification processing. FIG. 3 is a schematic diagram illustrating an image of the inside of the vehicle taken by the imaging device. FIG. 4 is a time chart showing a change in the position of the driver's head, a change in the position of the left elbow, and a correlation coefficient. FIG. 5 is a flowchart illustrating a first modification of the identification processing. FIG. 6 is a flowchart illustrating a second modification of the identification processing. FIG. 7 is a flowchart illustrating a third modification of the identification processing. FIG. 8 is a diagram illustrating an example of a hardware configuration of a computer that implements the function of each unit by a program.

  Hereinafter, a driver monitoring system (hereinafter, “DMS”) 1 equipped with an identification device 100 according to an embodiment of the present disclosure will be described in detail with reference to the drawings. The embodiment described below is an example, and the present disclosure is not limited to the embodiment.

  The DMS 1 is mounted on, for example, a vehicle (automobile). A vehicle is an example of a moving object according to the present disclosure. Hereinafter, the DMS 1 will be described as monitoring the driver of the vehicle, but is not limited to such an example. For example, the DMS 1 may monitor an occupant other than the driver (for example, an occupant sitting in a front passenger seat, a rear seat, or the like). In this case, an imaging device 200 described later uses a front surface of the upper body of the occupant other than the driver. Can be attached to a position where the photograph can be taken.

  The DMS 1 includes an imaging device 200 and an identification device 100, as shown in FIG.

  The imaging device 200 may be a single small camera that acquires an image of a vehicle interior. However, the present invention is not limited to this example, and the imaging device 200 may be a TOF (Time Of Flight) sensor or an infrared sensor. The imaging device 200 is attached to a console portion of the vehicle compartment so as to be able to photograph the interior of the vehicle compartment, and particularly to be able to photograph the front surface of the upper body of the driver 2 (see FIG. 3) in the vehicle compartment.

  The installation position of the imaging device 200 is not limited to the console portion, and may be any position that can photograph the upper body of the driver 2, such as on a steering column, near a center display, or a room mirror position.

  The identification device 100 is, for example, an ECU (Electronic Control Unit), and includes an input terminal, an output terminal, a processor, a program memory, and a main memory mounted on a control board in order to identify the state of the driver 2 in the vehicle interior. Including. In the present embodiment, two types of states, an abnormal state and a normal state, can be defined as the state of the driver 2. However, the state of the driver 2 is not limited to this example, and three or more types of states including at least an abnormal state may be determined. Here, the abnormal state may be, for example, a state in which the physical condition has suddenly changed such as loss of consciousness.

  The processor executes the program stored in the program memory using the main memory, processes various signals received through the input terminal, and outputs various signals through the output terminal.

  The identification device 100 functions as an input unit 120, an identification unit 130, a storage unit 140, an output unit 150, and the like as illustrated in FIG. 1 when a processor executes a program.

  The input unit 120 receives an input of an image signal output from the imaging device 200.

  The identification unit 130 identifies the state of the driver 2 (for example, whether or not the driver 2 is abnormal) based on the image input to the input unit 120. The identification unit 130 includes a skeleton information acquisition unit 131, a skeleton determination unit 132, and a sudden physical condition determination unit 133 as functional elements.

  The skeleton information acquisition unit 131 acquires skeleton information of the driver 2 from the image input to the input unit 120. Specifically, the skeleton information acquiring unit 131 detects the position of the head of the driver 2 and the position of a predetermined part other than the head based on the image. Hereinafter, the description will be given by taking the left elbow as an example of the predetermined part, but may be a part other than the left elbow (for example, a joint of an arm such as a right elbow, a wrist, or a shoulder).

  Such skeleton information can be obtained by using a known skeleton detection technology. For example, the skeleton information can be acquired by using local feature amounts of an image around a part to be acquired. Further, for example, the skeleton information can be acquired by machine learning such as deep learning represented by a convolutional neural network (Convolutional Neural Network).

  Based on the position of the head of the driver 2 obtained by the skeleton information obtaining unit 131 and the position of the predetermined part other than the head, the skeleton determining unit 132 changes the position of the head of the driver 2 and determines the position of the predetermined part other than the head. A value (specifically, a correlation coefficient) indicating a correlation with a change in position is calculated (specified).

  The sudden change in physical condition determination unit 133 determines the state of the driver 2 based on the relationship (correlation) between the change in the position of the head of the driver 2 and the change in the position of a predetermined part other than the head, calculated by the skeleton determination unit 132. Identify. For example, the physical condition sudden change determination unit 133 firstly compares the value indicating the correlation between the change in the position of the head and the change in the position of the predetermined part calculated by the skeleton determination unit 132 with a predetermined threshold value. It is determined whether or not the change in the position of the head and the change in the position of the predetermined part are similar. As an example, when the value indicating the correlation is larger than the predetermined threshold, the sudden change in physical condition determination unit 133 determines that the change in the position of the head and the change in the position of the predetermined part are similar. When the value indicating the correlation is equal to or smaller than the predetermined threshold, the sudden change in physical condition determination unit 133 determines that the change in the position of the head and the change in the position of the predetermined part are not similar.

  Then, the sudden change in physical condition determining unit 133 identifies whether or not the state of the driver 2 is abnormal based on the result of the determination. For example, when it is determined that the change in the position of the head of the driver 2 and the change in the position of the predetermined portion are similar, the sudden change in physical condition determination unit 133 determines that the state of the driver 2 is abnormal. When it is determined that the change in the position of the head and the change in the position of the predetermined part are not similar, the sudden change in physical condition determination unit 133 determines that the state of the driver 2 is the normal state.

  The storage unit 140 stores various types of information used in the processing performed in the identification unit 130.

  The output unit 150 identifies an alarm device 300 that outputs an alarm to the vehicle interior or exterior, and / or an ADAS (Advanced Driver Assistance System) ECU 400 (an example of a “driving assistance device”) that performs automatic driving of the vehicle. The identification result in the unit 130 is output. In addition, as shown in FIG. 1, the alarm device 300 and the ADAS ECU 400 can be installed in the DMS 1 (for example, in a vehicle). However, the present invention is not limited to this example, and the alarm device 300 may be installed in a predetermined facility. In this case, the alarm device 300 and the vehicle can be configured to be communicable by wireless communication and / or wired communication.

  An example of processing performed in the identification unit 130 of the identification device 100 will be described in detail with reference to the flowchart in FIG. The processing described below is repeatedly performed, for example, at a cycle of receiving an image from the imaging device 200 (for example, 10 fps). Hereinafter, an example in which the above-described predetermined portion is the left elbow will be described.

  First, in step S <b> 1, the identification unit 130 (specifically, the skeleton information obtaining unit 131) obtains skeleton information of the driver 2 using the image received from the imaging device 200. The skeleton information of the driver 2 includes the coordinates of the position of the head (hereinafter, “head position information”) and the coordinates of the position of the left elbow (hereinafter, “left elbow position information”).

  FIG. 3 is a schematic diagram illustrating an image of the inside of the vehicle taken by the imaging device 200. FIG. 3 shows a state in which the driver 2 normally wears the seatbelt and holds the steering wheel with both hands in a state where the driver 2 is seated in the driver's seat facing the front.

  In this example, as shown in FIG. 3, the coordinates of the position of each part of the driver 2 are acquired with the upper left end of the captured image as the origin, the right direction as the + X direction, and the lower direction as the + Y direction. The unit of coordinates is pixels.

FIG. 3 shows head position information H (X _H , Y _H ), right shoulder position coordinates RS (X _RS , Y _RS ), right wrist position coordinates RW (X _RW , Y _RW ), left shoulder position LS (X _LS , Y _LS ), left elbow position information LE (X _LE , Y _LE ), and the coordinates LW (X _LW , Y _LW ) of the position of the left wrist are shown.

  Note that the coordinate system in the captured image can be appropriately determined according to the type, installation position, FOV (Field of View), and the like of the imaging device 200.

  In the following step S2, the identification unit 130 (specifically, the skeleton information acquisition unit 131) causes the storage unit 140 to store the skeleton information acquired in step S1. The storage unit 140 accumulates skeleton information in a plurality of latest cycles including newly obtained skeleton information.

  In subsequent step S3, the identification unit 130 (specifically, the skeleton determining unit 132) uses the skeleton information in the latest plurality of cycles including the newly acquired skeleton information stored in the storage unit 140, A correlation coefficient r between the head position information H and the left elbow position information LE is calculated.

The correlation coefficient r is represented, for example, by the following equation (1).

は頭の位置のＹ座標の平均値であり、Ｂ_ｉ（ｉは１からｎまでの自然数）は、ｉ番目のフレームから検出された所定部位（左肘）の位置のＹ座標であり、

は所定部位（左肘）の位置のＹ座標の平均値である。 Here, n is the number of data, A _i (i is a natural number from 1 to n) is the Y coordinate of the head position detected from the i-th frame,

Is the average value of the Y coordinate of the position of the head, B _i (i is a natural number from 1 to n) is the Y coordinate of the position of the predetermined part (left elbow) detected from the i-th frame,

Is the average value of the Y coordinate of the position of the predetermined part (left elbow).

  In the following step S4, the identification unit 130 (specifically, the sudden change in physical condition determination unit 133) determines whether the correlation coefficient r calculated in step S3 is higher than a predetermined threshold. Such a threshold is, for example, 0.75, but is not limited thereto.

  In step S4, in order to prevent erroneous determination, it may be determined whether or not the state in which the correlation coefficient r is higher than a predetermined threshold has continued for a predetermined time.

  If it is determined in step S4 that the correlation coefficient r is higher than the predetermined threshold (step S4: YES), the process proceeds to step S5.

  In step S5, the identification unit 130 (specifically, the sudden change in physical condition determination unit 133) outputs to the output unit 150 a signal indicating that the state of the driver 2 is abnormal (hereinafter, referred to as an “abnormal state signal”). Output and end the process.

  On the other hand, if it is not determined in step S4 that the correlation coefficient r is higher than the predetermined threshold (step S4: NO), the process proceeds to step S6.

  In step S6, the identification unit 130 (specifically, the sudden change in physical condition determination unit 133) outputs to the output unit 150 a signal indicating that the driver 2 is in the normal state (hereinafter, referred to as a “normal state signal”). Output and end the process.

  When receiving the abnormal state signal from the identification unit 130, the output unit 150 outputs a signal indicating that the state of the driver 2 is abnormal to the alarm device 300, the ADAS ECU 400, and the like. When receiving the signal from the output unit 150, the alarm device 300 outputs an alarm (for example, an alarm of a type corresponding to the signal) to the inside or outside of the vehicle. When receiving the signal from the output unit 150, the ADAS ECU 400 performs driving assistance of the vehicle in accordance with the signal. For example, the ADAS ECU 400 switches the driving state of the vehicle from manual driving to automatic driving based on the signal, moves the vehicle to a predetermined stop area (road shoulder, parking lot, etc.), and stops the vehicle safely.

  Next, the operation and effect of the present embodiment will be described with reference to FIG. FIG. 4 is a time chart showing a change in the position of the driver's head, a change in the position of the left elbow, and a correlation coefficient. 4, the horizontal axis indicates time, and the vertical axis indicates the Y coordinate and the correlation coefficient.

  In FIG. 4, the solid line indicates the Y coordinate of the position of the head, the broken line indicates the Y coordinate of the position of the left elbow, and the dashed line indicates the correlation coefficient.

  From time t0 to time t1, the head position coordinates H and the left elbow position coordinates LE of the driver 2 hardly change, and no correlation is observed. Therefore, the correlation coefficient r is almost zero.

  From time t1 to time t2, the correlation coefficient r increases. This is because the position of the head of the driver 2 and the position of the left elbow change slightly with a correlation. However, since the correlation coefficient r does not reach the predetermined threshold value r1, the identification unit 130 determines that the state of the driver 2 is the normal state.

  At time t3, when the driver 2 loses consciousness and both the head position and the left elbow position of the driver 2 greatly change downward, the Y coordinate of the head position coordinate H and the Y coordinate of the left elbow position coordinate LE both change. The value greatly changes in the plus direction, and the correlation coefficient r greatly increases.

  Therefore, at time t4 when the correlation coefficient r exceeds the predetermined threshold value r1, the identification unit 130 determines that the state of the driver 2 is abnormal.

  As a comparative example, a method of determining that the state of the driver 2 is abnormal based on a state in which the position of the head has deviated from the normal position for a predetermined time has been considered. However, in this comparative example, there is a problem that even if the position of the head starts changing at time t3, the state of the driver 2 is not determined to be abnormal until time t5.

  In contrast, in the present embodiment, the state of the driver 2 is determined to be abnormal based on the correlation coefficient indicating the correlation between the change in the position of the head and the change in the position of the left elbow. Immediately after time t4, it can be determined that the state of the driver 2 is abnormal.

  As described above, in this embodiment, when the change in the position of the head of the driver 2 is similar to the change in the position of a predetermined part other than the head, the state of the driver 2 is determined to be abnormal. . In particular, when the correlation coefficient indicating the correlation between the change in the position of the head of the driver 2 and the change in the position of the predetermined part other than the head is higher than a predetermined threshold, it is determined that the state of the driver 2 is abnormal. . Therefore, it is possible to quickly identify that the driver has become abnormal.

  Furthermore, according to the present embodiment, it is possible to determine whether the state of the driver is an abnormal state with higher accuracy than in the related art. For example, even when the driver looks into the front, looks back, or takes luggage in the front passenger seat, it is possible to prevent the driver from being erroneously identified as being in an abnormal state (that is, these conditions). In the case of (1), the driver state can be accurately identified as the normal state.)

  Hereinafter, a modified example of the present embodiment will be described.

(First modification)
A first modification of the processing performed in the identification unit 130 of the identification device 100 will be described in detail with reference to the flowchart in FIG. In the above-described embodiment, the state of the driver 2 is identified based on the correlation coefficient between the head position information H and the left elbow position information LE. On the other hand, the first modification example described below identifies the state of the driver 2 based on the change direction of the head position information H and the change direction of the left elbow position information LE.

  The sudden change in physical condition determination unit 133 according to the first modification example is based on whether the position of the head of the driver 2 and the position of the predetermined portion have changed in a downward direction (more specifically, in a vertical downward direction). It can be determined whether or not the state is abnormal. For example, the position of the head and the position of the head within a predetermined time (for example, simultaneously) from the timing when it is detected that one of the position of the head of the driver 2 and the position of the predetermined part has started to move downward are detected. When it is detected that both of the positions of the predetermined part continue to move downward, the physical condition sudden change determination unit 133 identifies that the state of the driver 2 is an abnormal state. Here, the length of the predetermined time may be set to, for example, an optimum length based on a result of an experiment previously performed on each of the plurality of occupants.

  In the following, as an example, if it is detected that both the position of the head of the driver 2 and the position of the predetermined portion have moved downward simultaneously, the physical condition sudden change determination unit 133 determines that the state of the driver 2 is abnormal. An example of identifying a state will be described.

  Hereinafter, the contents of the first modified example will be described in more detail with reference to FIG. The processing contents of step S11 and step S12 are the same as the processing contents of step S1 and step S2 described above, and a description thereof will be omitted.

  In step S13 following step S12, the identification unit 130 (specifically, the skeleton determining unit 132) compares the newly obtained skeleton information and the previously obtained skeleton information stored in the storage unit 140. Then, the change direction of the head position information H and the change direction of the left elbow position information LE are calculated.

  Note that, in calculating the change direction of the head position information H and the change direction of the left elbow position information LE, skeleton information acquired earlier may be used instead of the skeleton information acquired last time. Further, the calculation of the change direction of the head position information H and the change direction of the left elbow position information LE may use skeleton information in a plurality of latest cycles including newly obtained skeleton information.

  In the following step S14, the identification unit 130 (specifically, the sudden physical condition determination unit 133) determines whether the change direction of the head position information H and the change direction of the left elbow position information LE are both downward. judge.

  If it is determined in step S14 that both the change direction of the head position information H and the change direction of the left elbow position information LE are downward (step S14: YES), the process proceeds to step S15.

  On the other hand, when it is determined in step S14 that at least one of the change direction of the head position information H and the change direction of the left elbow position information LE is not downward (step S14: NO), the process proceeds to step S16.

  The processing contents of steps S15 and S16 are the same as the processing contents of step S5 and step S6 described above, and thus description thereof will be omitted.

  According to the first modification, when the position of the head of the driver 2 and the position of the predetermined part other than the head simultaneously change in the downward direction, the state of the driver 2 is determined to be an abnormal state. There is no need to calculate numbers. Therefore, in addition to the same effects as in the above-described embodiment, the processing load on the ECU can be reduced. Further, since the change direction can be calculated using only the two latest skeleton information, the load on the storage unit 140 can be reduced.

  In the first modification, when both the change direction of the head position information H and the change direction of the left elbow position information LE are downward, it is determined that the state of the driver 2 is abnormal, but the present invention is not limited to this. Not done.

  Specifically, for example, both the change direction of the head position information H and the change direction of the left elbow position information LE are downward, and the change direction of the head position information H and the change direction of the left elbow position information LE are formed. If the angle is less than the predetermined angle, the state of the driver 2 may be determined to be abnormal.

(Second modification)
A second modification of the process performed by the identification unit 130 of the identification device 100 will be described in detail with reference to the flowchart in FIG. In the above-described embodiment, the state of the driver 2 is determined based on the correlation coefficient r between the head position information H and the left elbow position information LE, assuming that the driver 2 is gripping (operating) the steering wheel with both hands. Identified. On the other hand, a second modified example described below allows the state of the driver 2 to be identified even when the driver 2 is gripping (operating) the steering wheel with one hand.

  The processing contents of step S21 and step S22 are the same as the processing contents of step S1 and step S2 described above, and a description thereof will be omitted.

  In step S22-1 following step S22, the identification unit 130 acquires (detects) the steering grip state of the driver 2.

  The steering grip state of the driver 2 can be acquired by, for example, acquiring a signal output from a known steering grip sensor and indicating the steering grip state of the driver 2.

  When the driver 2 is not gripping the steering wheel with both hands, the signal indicating the steering grip state of the driver 2 includes a signal indicating which of the left and right hands is gripping the steering wheel.

  In the following step S22-2, the identification unit 130 determines the position of the hand holding the steering wheel among the coordinates of the position of the right elbow (hereinafter referred to as “right elbow position information”) RE and the left elbow position information LE. Select information (ie, skeleton information).

  In the following step S23, the identification unit 130 calculates a correlation coefficient between the head position information and the right elbow position information RE or the left elbow position information LE selected in step S22-2.

  The processing contents of steps S24 to S26 are the same as the processing contents of steps S4 to S6 described above, and a description thereof will be omitted.

  According to the second modification, the information of the hand holding the steering wheel can be used, and the state of the driver 2 can be accurately identified. In the second modification, the state of the driver 2 is identified using the right elbow position information RE or the left elbow position information LE. However, the present invention is not limited to this. Other parts such as shoulders and wrists may be used.

(Third modification)
A third modification of the processing performed in the identification unit 130 of the identification device 100 will be described in detail with reference to the flowchart in FIG. In the above-described embodiment, the state of the driver 2 is identified based on the correlation coefficient between the head position information H and the left elbow position information LE. On the other hand, in a third modified example described below, the subsequent movement of the driver 2 is further detected, thereby further improving the identification accuracy of the state of the driver 2. Specifically, the identification unit 130 according to the third modification example determines that the change in the position of the head of the driver 2 and the change in the position of the predetermined part other than the head are similar to each other. Based on the change amount and the change amount of the position of the predetermined part, it is determined whether or not the state of the driver 2 is abnormal. Hereinafter, the contents of the third modification will be described in more detail with reference to FIG.

  The processing contents of steps S31 to S33 are the same as the processing contents of steps S1 to S3 described above, and a description thereof will be omitted.

  In step S34 following step S33, the identification unit 130 determines whether the correlation coefficient is higher than a predetermined threshold.

  If it is determined in step S34 that the correlation coefficient is higher than the predetermined threshold (step S34: YES), the process proceeds to step S34-1.

  In step S34-1, the identification unit 130 calculates the amount of change in the head position information H and the left elbow position information LE. At this time, as an index indicating the amount of change in the head position information H and the left elbow position information LE, the standard of the skeleton information in the latest plurality of cycles including the newly acquired skeleton information stored in the storage unit 140 is stored. The deviation may be calculated.

  In subsequent step S34-2, identification unit 130 determines whether or not the amount of change in head position information H and left elbow position information LE is smaller than a second predetermined threshold.

  When it is determined in step S34-2 that the amount of change in the head position information H and the left elbow position information LE is smaller than the second predetermined threshold (step S34-2: YES), the process proceeds to step S35.

  On the other hand, when it is determined in step S34-2 that the amounts of change in the head position information H and the left elbow position information LE are not smaller than the second predetermined threshold (step S34-2: NO), the process proceeds to step S36. move on.

  It returns to description of step S34. If it is determined in step S34 that the correlation coefficient is not higher than the predetermined threshold (step S34: NO), the process proceeds to step S36.

  The processing contents of step S35 and step S36 are the same as the processing contents of step S5 and step S6 described above, and thus description thereof will be omitted.

  According to the third modification, after it is determined that the correlation coefficient between the head position information H and the left elbow position information LE is higher than a predetermined threshold, the amount of change in the head position information H and the left elbow position information LE If the state of the driver 2 is determined to be an abnormal state when it is smaller than the second predetermined threshold value, the accuracy of identifying the state of the driver 2 can be further improved.

  Note that the above-described embodiment and each of the modifications can be used in appropriate combination.

  Further, in the above-described embodiment and each modified example, the case where the left elbow or the right elbow is used as the predetermined part other than the head has been described as an example. However, the present invention is not limited thereto, and the part other than the left elbow or the right elbow may be used. Can also be used. Furthermore, the state of the driver 2 may be identified using information on the head and a plurality of predetermined parts other than the head.

  Further, in the above-described embodiment and each of the modified examples, the example in which the alarm device 300, the ADAS ECU 400, and the like are provided outside the identification device 100 has been described. However, the present invention is not limited to this. At least two of the identification device 100, the alarm device 300, and the ADAS ECU 400 may be integrally provided. When the identification device 100 and the ADAS ECU 400 are integrated, the identification device 100 may further include a driving support control unit. In this case, the driving support control unit may support driving of the vehicle based on the identification result of the identification unit 130.

  Further, in the above-described embodiment and each of the modifications, an example is described in which the state of the driver 2 is determined to be abnormal when both the position of the head and the position of a predetermined part other than the head change downward. , But is not limited to this. When both the position of the head and the position of the predetermined part other than the head change in the upward direction, the front-back direction, the left-right direction, and the like, the state of the driver 2 may be determined to be abnormal.

  Further, in the above-described embodiment and each of the modified examples, the vehicle has been described as an example of the moving body, but the moving body is not limited thereto. Other examples of the moving object include an airplane, a train, a ship, and the like.

  Further, in the above-described embodiment and each of the modified examples, the state in which the consciousness is lost has been described as an example of the abnormal state, but the present invention is not limited to this. As another example of the abnormal state, there is a state of falling asleep.

  FIG. 8 is a diagram illustrating a hardware configuration of a computer that implements the functions of each unit according to the above-described embodiment and modified examples by a program.

  As shown in FIG. 8, a computer 2100 includes an input device 2101 such as an input button and a touch pad, an output device 2102 such as a display and a speaker, a CPU (Central Processing Unit) 2103, a ROM (Read Only Memory) 2104, and a RAM (Random). Access Memory) 2105. Further, the computer 2100 is a reading device that reads information from a recording medium such as a hard disk device, a storage device 2106 such as an SSD (Solid State Drive), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. 2107, a transmission / reception device 2108 that performs communication via a network. The above-described units are connected by a bus 2109.

  Then, the reading device 2107 reads the program for realizing the function of each unit from a recording medium on which the program is recorded, and causes the storage device 2106 to store the program. Alternatively, the transmission / reception device 2108 communicates with a server device connected to the network, and causes the storage device 2106 to store a program downloaded from the server device for realizing the function of each unit described above.

  Then, the CPU 2103 copies the program stored in the storage device 2106 to the RAM 2105, and sequentially reads and executes the instructions included in the program from the RAM 2105, thereby realizing the functions of the above-described units. When the program is executed, information obtained by various processes described in each embodiment is stored in the RAM 2105 or the storage device 2106, and is appropriately used.

(Summary of the present disclosure)
An identification device according to the present disclosure is an identification device that identifies a state of an occupant of a moving object, and an input unit that inputs an image including the occupant obtained by an imaging device, and the occupant obtained from the image. An identification unit that identifies whether or not the occupant is in an abnormal state based on a relationship between a change in the position of the head and a change in the position of a predetermined part other than the head.

  In the identification device according to the present disclosure, when the change in the position of the head and the change in the position of the predetermined part are similar, the identification unit identifies the occupant as being in an abnormal state.

  In the identification device according to the present disclosure, the identification unit calculates a value indicating a correlation between the change in the position of the head and the change in the position of the predetermined part, and based on a comparison between the value indicating the correlation and a predetermined threshold. Then, it is determined whether the change in the position of the head and the change in the position of the predetermined part are similar, and when the change in the position of the head and the change in the position of the predetermined part are similar, The state of the occupant is identified as an abnormal state.

  In the identification device according to the present disclosure, the identification unit identifies whether or not the occupant is in an abnormal state based on whether or not the position of the head and the position of the predetermined part have simultaneously changed downward. I do.

  In the identification device according to an embodiment of the present disclosure, the identification unit may be configured such that, within a predetermined time from a timing when it is detected that one of the position of the head and the position of the predetermined part has begun to move downward, When it is detected that both the position of the head and the position of the predetermined part continue to move downward, the state of the occupant is determined to be abnormal.

  In the identification device according to the present disclosure, the identification unit is configured to determine, when it is determined that the change in the position of the head and the change in the position of the predetermined part are similar, the amount of change in the position of the head and the change in the predetermined part. Based on the amount of change in position, it is determined whether or not the occupant is in an abnormal state.

  In the identification device according to the present disclosure, the predetermined part is an arm joint.

  In the identification device according to the present disclosure, the arm is an arm operating a steering device of the moving body.

  The identification device according to the present disclosure further includes an output unit that outputs an identification result of the identification unit to an alarm device.

  The identification device according to the present disclosure further includes an output unit that outputs a result of the identification performed by the identification unit to a driving support device.

  The identification device according to the present disclosure further includes a driving assistance control unit that performs driving assistance of the moving object based on an identification result of the identification unit.

  An identification method according to an embodiment of the present disclosure is an identification method for identifying a state of an occupant of a moving object, including the steps of: inputting an image including the occupant obtained by an imaging device; Identifying whether or not the occupant is in an abnormal state based on a relationship between a change in the position of the head and a change in the position of the predetermined part other than the head.

  An identification program according to an embodiment of the present disclosure includes a process of inputting an image including the occupant obtained by an imaging device to a computer, a change in a position of the occupant's head obtained from the image, and a predetermined part other than the head. And a process of identifying whether or not the occupant is in an abnormal state based on the relationship with the change in the position.

  According to the identification device, the identification method, and the identification program according to the present disclosure, it is possible to quickly identify that the occupant of the moving object has become abnormal, which is suitable for in-vehicle use.

1 Driver Monitoring System (DMS)
2 driver 100 identification device 120 input unit 130 identification unit 131 skeleton information acquisition unit 132 skeleton determination unit 133 physical condition sudden change determination unit 140 storage unit 150 output unit 200 imaging device 2100 computer 2101 input device 2102 output device 2103 CPU
2104 ROM
2105 RAM
2106 storage device 2107 reading device 2108 transmitting / receiving device 2109 bus

Claims (13)

An identification device for identifying a state of an occupant of a moving body,
An input unit for inputting an image including the occupant obtained by an imaging device;
Identification for identifying whether or not the occupant is in an abnormal state based on a relationship between a change in the position of the occupant's head and a change in the position of a predetermined part other than the head, obtained from the image; And a unit,
Identification device. When the change in the position of the head and the change in the position of the predetermined part are similar, the identification unit identifies that the state of the occupant is an abnormal state.
The identification device according to claim 1. The identification unit calculates a value indicating a correlation between a change in the position of the head and a change in the position of the predetermined part, and, based on a comparison between the value indicating the correlation and a predetermined threshold value, Determine whether the change and the change in the position of the predetermined part are similar,
When the change in the position of the head and the change in the position of the predetermined portion are similar, the state of the occupant is identified as an abnormal state,
The identification device according to claim 2. The identification unit is configured to identify whether or not the occupant is in an abnormal state, based on whether or not the position of the head and the position of the predetermined part have simultaneously changed downward.
The identification device according to claim 1. The identification unit, within a predetermined time from the timing when it is detected that one of the position of the head and the position of the predetermined part has started to move downward, the position of the head and the predetermined part When it is detected that both of the positions continue to move downward, the state of the occupant is identified as an abnormal state,
The identification device according to claim 1. The identification unit, when it is determined that the change in the position of the head and the change in the position of the predetermined part are similar, based on the amount of change in the position of the head and the amount of change in the position of the predetermined part Identifying whether the occupant is in an abnormal state,
The identification device according to claim 1. The predetermined site is an arm joint,
The identification device according to claim 1. The arm is an arm operating a steering device of the moving body,
The identification device according to claim 7. An output unit that outputs an identification result of the identification unit to an alarm device,
The identification device according to claim 1. An output unit that outputs an identification result of the identification unit to the driving support device,
The identification device according to claim 1. Further comprising a driving support control unit that performs driving support of the moving object based on the identification result in the identification unit,
The identification device according to claim 1. An identification method for identifying a state of an occupant of a moving object,
Inputting an image including the occupant obtained by an imaging device;
A step of identifying whether or not the state of the occupant is abnormal based on the relationship between the change in the position of the occupant's head and the change in the position of a predetermined part other than the head, obtained from the image; And comprising
Identification method. On the computer,
A process of inputting an image including the occupant obtained by an imaging device;
A process of identifying whether or not the occupant is in an abnormal state based on the relationship between the change in the position of the occupant's head and the change in the position of a predetermined part other than the head, obtained from the image. And let
Identification program.

Images