JP2017068424A - Attitude measuring apparatus and attitude measurement method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an attitude measuring apparatus which can measure positions of each of sections of a human body in a real space.SOLUTION: An attitude measuring apparatus includes: a storage unit 4 which stores a body structure model representing positions of sections adjacent to a plurality of sections of a human body, and lengths from the sections to the adjacent sections, and reliability information that indicates whether the reliability representing the likelihood of measurement accuracy of a position in a real space corresponds to fist reliability or second reliability lower than the first reliability, for each section; a tracking unit 22 which determines a position, in the real space, of a first section having the first reliability of a person to be measured, on the basis of three-dimensional measurement data generated by a three-dimensional sensor 2 and a result of measuring a past position, in the real space, of the section; and a position estimating unit 23 which determines a position, in the real space, of a second section having the second reliability of the person to be measured, on the basis of positions of the first section in the real space and a length between the first and second sections to be determined on the basis of the body structure model.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an attitude measurement device and an attitude measurement method for measuring the positions of a plurality of parts of a human body in real space, for example.

  In recent years, research on automatic driving of vehicles has been made. In the automatic driving of the vehicle, there is a case where the driving authority is transferred from the driver to the automatic driving device that performs automatic driving of the vehicle or from the automatic driving device to the driver. When the driving authority is delegated from the automatic driving device to the driver, it is required that the driver has a posture capable of driving the vehicle in order to prevent an accident. Therefore, in order to determine whether or not the driver is in a drivable posture, the position in the real space of each part of the driver obtained by the three-dimensional sensor that can obtain the position information of the real space of the subject is obtained. Based on this, it is preferable that the attitude of the driver can be measured.

  As one technique for tracking the posture of a measurement target object, an Iterative Closest Point (ICP) algorithm is known. The ICP algorithm uses the 3D measurement data in a situation where the correspondence between the 3D measurement data of the measurement object obtained at a certain time and the 3D measurement data obtained at another time is unknown. Perform matching between data. At that time, the ICP algorithm minimizes the error between the point group obtained by affine transformation of the measurement point group included in one of the three-dimensional measurement data and the measurement point group of the other three-dimensional measurement data. An affine transformation parameter is calculated by repeatedly calculating.

  By using the ICP algorithm, it is possible to track the change in the posture of the measurement target object even when a part of the measurement target object becomes invisible due to a change in the posture of the measurement target object. However, the ICP algorithm assumes that the object to be measured is a rigid body. On the other hand, since the driver is not a rigid body, there is a possibility that the driver's posture cannot be accurately measured even if the ICP algorithm is directly applied to the driver's posture measurement. Here, as a technique for correctly tracking with the ICP algorithm even when a non-rigid body exists in the three-dimensional measurement data, a dynamic obstacle located in the first surface information and the second surface information is detected, and the dynamic obstacle is detected. A technique for generating third surface information from which an object has been removed and matching any one of the third surface information, the first surface information, and the second surface information has been proposed (see, for example, Patent Document 1).

JP 2009-271513 A

  However, since the driver is a moving object, the technique described in Patent Document 1 removes the measurement point group corresponding to the driver to be measured from the three-dimensional measurement data, making it impossible to measure the driver's posture. Sometimes.

  In one aspect, an object of the present invention is to provide an attitude measurement device that can measure the position of a person in real space.

  According to one embodiment, an attitude measurement device is provided. This posture measuring device includes a three-dimensional sensor that generates three-dimensional measurement data representing the position of each point in the measurement range in real space, a part adjacent to the part, and a part adjacent to the part. Whether the human body model representing the length from the part to the adjacent part and the reliability representing the accuracy of the measurement accuracy of the position in the real space for each of the plurality of parts is the first reliability, or the first A storage unit that stores reliability information indicating whether or not the second reliability is lower than the reliability of the first and the first reliability having a first reliability among a plurality of parts of the measurement target person within the measurement range A tracking unit for obtaining a position of one part in the real space by a tracking process based on the three-dimensional measurement data and a past measurement result of the position of the part in the real space; In the real space of the second part with a confidence of 2 Position, and a position estimation unit for determining based on the length between the first and second sites determined based on the position and anatomical model in the real space of the plurality of first parts.

  The position in real space can be measured for each part of a person.

It is a schematic block diagram of the automatic driving device by one embodiment by which the posture measuring device was mounted. It is a figure which shows an example of arrangement | positioning of the automatic driving device in a vehicle. It is a functional block diagram of a control part. It is explanatory drawing showing the outline | summary of a human body structure model. It is a figure which shows an example of a skeleton model. (A) is a figure which shows an example of the attitude | position of the driver at the time of 3D measurement data acquisition of a front frame, (b) is a figure which shows an example of the attitude | position of a driver at the time of 3D measurement data acquisition of the present frame. is there. (A)-(e) is a figure which shows an example of the determination conditions for each transferring a driving authority from an automatic driving device to a driver. It is an operation | movement flowchart of an attitude | position measurement process.

Hereinafter, the posture measuring apparatus will be described with reference to the drawings.
For example, this posture measurement apparatus uses a three-dimensional sensor capable of measuring the position of a subject in real space to obtain a predetermined range of three-dimensional measurement data including a person to be measured at regular intervals. This posture measurement device has relatively little variation in position, shape, etc. of each part of the person, and the reliability of the estimated position obtained by tracking based on the past position and the latest three-dimensional measurement data is relatively For highly reliable parts, an estimated position in real space is obtained by tracking processing. On the other hand, this posture measurement device has a relatively low variation in position, shape, etc., and the reliability of the estimated position obtained by tracking based on the past position and the latest three-dimensional measurement data is relatively low. No tracking process is applied to. Instead, this posture measurement device uses the position of the surrounding high-reliability part and the length between the high-reliability part and the low-reliability part defined by the human body structure model for the low-reliability part. Find the estimated position.

  In the present embodiment, it is assumed that the posture measurement device is mounted on an automatic driving device, and the person to be measured by the posture measurement device is a driver. However, the posture measurement device may be used for posture measurement of a person other than the driver.

  FIG. 1 is a schematic configuration diagram of an automatic driving device according to an embodiment in which an attitude measurement device is mounted. FIG. 2 is a diagram illustrating an example of the arrangement of the automatic driving devices in the vehicle. The automatic driving device 1 includes a three-dimensional sensor 2, a vehicle exterior sensor 3, a storage unit 4, and a control unit 5. The automatic driving device 1 may further include a display (not shown) for displaying various types of information. The memory | storage part 4 and the control part 5 are provided on the board | substrate 6, for example. The three-dimensional sensor 2 and the outside sensor 3 are connected to the control unit 5 via an in-vehicle network 7 that complies with a standard such as a control area network. Further, the control unit 5 is connected to an electronic control unit (ECU) (not shown) of the vehicle 10 via the in-vehicle network 7.

  The three-dimensional sensor 2 is attached toward the driver 11 near, for example, the ceiling in the front of the vehicle interior, for example, near the room mirror. The three-dimensional sensor 2 generates the three-dimensional measurement data within the measurement range including the driver 11 at regular intervals (for example, 50 msec to 100 msec), and the generated three-dimensional measurement data is transmitted via the in-vehicle network 7. Output to the controller 5. Therefore, the three-dimensional sensor 2 can be, for example, a depth camera or a stereo camera that employs the Time of Flight method. The three-dimensional measurement data is, for example, the real space of each point within the measurement range of the three-dimensional sensor 2 in a three-dimensional orthogonal coordinate system with a predetermined point (for example, the center of the sensor surface of the three-dimensional sensor 2) as the origin. Contains coordinates.

  The vehicle exterior sensor 3 has a camera attached toward the front of the vehicle 10, for example, in the vicinity of the ceiling of the vehicle interior so that the front of the vehicle 10 can be photographed. The outside sensor 3 generates an image in which the front of the vehicle 10 is captured at a predetermined shooting period (for example, 30 msec to 100 msec), and outputs the generated image to the control unit 5 via the in-vehicle network 7. Note that the camera included in the vehicle exterior sensor 3 may be a stereo camera. Further, the vehicle outside sensor 3 may include a radar sensor for measuring the distance from the vehicle 10 to an object located around the vehicle 10. The outside sensor 3 may also output distance measurement data from the radar sensor to the control unit 5 via the in-vehicle network 7.

  The storage unit 4 includes, for example, a readable / writable nonvolatile or volatile semiconductor memory and a read-only nonvolatile semiconductor memory. And the memory | storage part 4 memorize | stores the program of the automatic driving | running | working process containing the attitude | position measurement process performed on the control part 5. FIG. In addition, the storage unit 4 stores various data such as a human body structure model and a skeleton model generated within the latest fixed period, which are used or generated in the posture measurement process. Details of the human body structure model and the skeleton model will be described later.

  The control unit 5 includes one or a plurality of processors and their peripheral circuits, and is connected to each unit and the ECU of the automatic driving device 1 through a signal line or an in-vehicle network 7. For example, the control unit 5 starts the automatic driving process when the driver operates an automatic driving start switch (not shown) provided in the steering or the like. During the execution of the automatic driving process, the control unit 5 recognizes the traveling lane of the vehicle 10 based on, for example, an image ahead of the vehicle 10 obtained from the vehicle outside sensor 3 at regular intervals. Then, the control unit 5 determines the steering angle based on the recognition result and information such as the planned travel route and the current position of the vehicle 10 acquired from the navigation system (not shown). In addition, the control unit 5 determines the accelerator opening and the brake operation amount based on information representing the behavior of the vehicle 10 such as the speed of the vehicle 10 obtained from the ECU 10 and the image or distance measurement data in front of the vehicle 10. To do. And the control part 5 outputs the operation signal showing a steering angle, an accelerator opening degree, and the amount of brake operation to ECU. The control unit 5 may execute these processes according to any of various methods related to determination of travel lane recognition, steering angle, accelerator opening, and brake operation amount.

  Further, the control unit 5 measures the posture of the driver at regular intervals during execution of the automatic driving process. For example, when the vehicle 10 approaches a predetermined distance from the destination, or when the driver performs an operation for canceling the automatic driving via a switch provided on the steering wheel or the like. Set to authority transfer preparation mode. When the authority transfer preparation mode is set, the control unit 5 determines whether or not to delegate the driving authority to the driver based on the measurement result of the driver attitude.

FIG. 3 is a functional block diagram of the control unit 5 regarding the posture measurement processing. The control unit 5 includes a skeleton model generation unit 21, a high reliability part tracking unit 22, a low reliability part estimation unit 23, and a driving authority transfer determination unit 24.
Each of these units included in the control unit 5 is a functional module realized by a computer program executed on a processor included in the control unit 5.
Each of these units included in the control unit 5 may be mounted on the automatic driving device 1 as one or a plurality of integrated circuits in which circuits corresponding to the respective units are integrated, separately from the processor included in the control unit 5. Good.

  The skeleton model generation unit 21 is an example of a detection unit, and every time three-dimensional measurement data is obtained after the posture measurement process is started, a driver represented by a human body structure model based on the three-dimensional measurement data The position of each predetermined part in the real space is detected. The skeleton model generation unit 21 generates a skeleton model representing the skeleton of the driver based on the position of each part in the real space. In the following description, the position in the real space may be simply referred to as a position.

First, the human body structure model will be described.
FIG. 4 is an explanatory diagram showing an outline of the human body structure model. The human body structure model 400 is a model that represents an invariant arrangement relationship between each part in the human body. For example, the fact that the head and neck (middle shoulder) are adjacent and the right shoulder and right elbow are adjacent are unchanged in any human body. Therefore, the human body structure model 400 defines that two parts connected by a solid line in the model 400 are adjacent to each other and the length between the two parts. Actually, as a human body structure model, for example, for each part, the name or identification number of the part, the name or identification number of the part adjacent to the part, and the length to the adjacent part are listed. To be recorded. The human body structure model is stored in the storage unit 4 in advance.

  The skeletal model generation unit 21 is a three-dimensional measurement obtained from the three-dimensional sensor 2 for each part used for estimating the posture of the driver, for example, the position of the head, both shoulders, both elbows, and both wrists in real space. Detect based on data. Therefore, the skeletal model generation unit 21 uses any one of various techniques for detecting the position of each part of the human body based on one frame of three-dimensional measurement data, and for each part in the three-dimensional measurement data. A measurement point corresponding to the part is detected. For example, the skeletal model generation unit 21 detects a measurement point corresponding to a part for each part using a classifier previously learned based on machine learning such as Random Forest (for example, J. Shotton et al., “ Real-Time Human Pose Recognition in Parts from Single Depth Images ", CVPR, 2011). Alternatively, the skeleton model generation unit 21 may detect a measurement point corresponding to a part for each part by fitting a human body model combining geometric shapes such as a cylinder and a sphere with three-dimensional measurement data ( For example, see Murakami et al., “Estimation of human posture based on 3D voxel data”, Information Processing Society of Japan Research Report, 2003). Alternatively, the skeletal model generation unit 21 may detect a measurement point corresponding to the part for each part using the Reeb Graph (for example, Yuino et al., “High-speed and robust tertiary based on geodesic distance”). Skeleton extraction from original video ”, IEICE Transactions, D Vol.J90-D No.7 pp.1729-1732, 2007).

  Note that some parts may be hidden behind other parts and cannot be seen from the three-dimensional sensor 2 or may be indistinguishable from other parts because they are in close contact with other parts. The skeletal model generation unit 21 does not have to detect the corresponding measurement point for such a part.

  The skeleton model generation unit 21 generates a skeleton model based on the part where the corresponding measurement point is detected.

  FIG. 5 is a diagram illustrating an example of a skeleton model. In this example, the skeleton model 500 is generated for each part as a table in which three-dimensional coordinates representing the position of the measurement point corresponding to the part in the real space are represented. In the skeletal model 500, for a portion that has not been detected, the column representing the three-dimensional coordinates is blank, or the value of the three-dimensional coordinates is the default value.

  The skeleton model generation unit 21 stores the generated skeleton model in the storage unit 4 together with the acquisition time of the three-dimensional measurement data used for generating the skeleton model.

  The high-reliability site tracking unit 22 is an example of a tracking unit, and is generated based on the 3D measurement data of the immediately preceding frame stored in the storage unit 4 every time 3D measurement data is newly obtained. A highly reliable part is tracked based on the skeleton model and the latest three-dimensional measurement data of the frame. Thereby, the highly reliable site | part tracking part 22 updates the position in the real space of a highly reliable site | part.

  The highly reliable site is a site where the variation in position, shape, etc. is relatively small, and the reliability of the estimated position obtained by tracking based on the position obtained in the past and the latest three-dimensional measurement data is relatively high. For example, it is assumed that the driver is sitting on the seat and facing forward and holding the steering wheel with both hands. Therefore, among the parts, the face, neck, both shoulders, trunk, and both wrists have relatively little fluctuation, and the reliability of the estimated position obtained by tracking is high. Therefore, in the present embodiment, the face, neck, both shoulders, torso, and both wrists are set as highly reliable parts. On the other hand, both elbows may move greatly or be deformed depending on the steering operation or the like. As a result, there is a relatively high possibility that both elbows will not be visible from the three-dimensional sensor 2. So, in this embodiment, both elbows are set as a low reliability part. As described above, the setting of the high-reliability part or the low-reliability part for each part is performed in advance based on the structure of the human body, the posture assumed for the measurement target person, the relative position with respect to the three-dimensional sensor 2, or the like. .

  For each part defined in the human body structure model, a flag indicating whether the part is a high-reliability part or a low-reliability part is stored in advance in the storage unit 4 together with the identification number of the part. This flag is an example of reliability information indicating whether the part is a high-reliability part or a low-reliability part. In the present embodiment, the high-reliability part tracking unit 22 selects a high-reliability part by referring to a flag from parts whose positions are not obtained in the skeleton model generation unit 21. Then, the high-reliability part tracking unit 22 executes a tracking process based on the position of the high-reliability part represented in the skeleton model in the previous frame and the three-dimensional measurement data of the current frame for each of the selected high-reliability parts. Thus, the estimated position of the real space in the current frame is obtained. At that time, for example, the high-reliability part tracking unit 22 applies the ICP algorithm as the tracking process, and represents the movement from the position of the high-reliability part of interest represented in the skeleton model in the previous frame to the current frame. A position conversion parameter (for example, an affine conversion parameter) is calculated. The high-reliability part tracking unit 22 is obtained by applying the position conversion parameter to the position of the high-reliability part of interest in the previous frame among the measurement points included in the three-dimensional measurement data of the current frame. The measurement point that minimizes the error from the estimated position is specified. When the difference between the specified measurement point and the estimated position is within a predetermined allowable range (for example, 5 to 10 cm), the high reliability region tracking unit 22 determines the position of the specified measurement point in the current frame. The position of the reliability part. In addition, when the difference between the specified measurement point and the estimated position is out of a predetermined allowable range, the high-reliability site tracking unit 22 uses the estimated position itself as the position of the high-reliability site of interest in the current frame. It is good. This is because the highly reliable part of interest may be hidden behind other parts.

  The high-reliability part tracking unit 22 writes the position of the high-reliability part in the skeleton model for each high-reliability part for which the position in the current frame has been obtained, and updates the skeletal model.

  For a low-reliability part whose position is not obtained in the skeleton model generation unit 21, the position estimation precision obtained by the tracking process is low and the error may be large. This is because the low-reliability portion has a relatively large fluctuation or is likely to be invisible from the three-dimensional sensor 2.

  Therefore, the low reliability part estimation unit 23 uses the current frame based on the positions of the human body structure model and a plurality of surrounding high reliability parts for each of the low reliability parts whose positions are not obtained by the skeleton model generation unit 21. Find the estimated position. The low reliability part estimation unit 23 is an example of a position estimation unit.

  The low reliability part estimation unit 23 selects, for example, a plurality of (for example, two) high reliability parts adjacent to the low reliability part of interest. When there is only one high reliability part adjacent to the low reliability part of interest, the low reliability part estimation unit 23 sequentially selects a plurality of high reliability parts from the side closest to the low reliability part of interest. May be selected. Then, the low reliability part estimation unit 23 refers to the human body structure model for each of the selected plurality of high reliability parts, and specifies the length from the high reliability part to the focused low reliability part. For each of the two selected high reliability parts, the low reliability part estimation unit 23 sets the radius from the high reliability part to the focused low reliability part around the position of the high reliability part. And the intersection of each sphere is set as the position in the real space of the low-reliability part of interest. Note that, when there are a plurality of intersections, the low reliability part estimation unit 23 may set the position of the intersection closest to the position of the low reliability part of interest in the previous frame as the position of the low reliability part. Furthermore, the low reliability part estimation part 23 has the distance to the measurement point nearest to the intersection among the measurement points included in the three-dimensional measurement data of the current frame within a predetermined allowable range (for example, 5 to 10 cm). In this case, the position of the measurement point may be the position of the low reliability part.

  The low reliability region estimation unit 23 writes the position of the low reliability region in the skeleton model for each low reliability region for which the position in the current frame has been obtained, and updates the skeleton model.

  FIG. 6A is a diagram illustrating an example of the posture of the driver when acquiring the three-dimensional measurement data of the previous frame, and FIG. 6B is an example of the posture of the driver when acquiring the three-dimensional measurement data of the current frame. FIG. In the example shown in FIG. 6A, since the position of each part of the driver 600 is visible from the three-dimensional sensor 2 at the time of the previous frame, the skeleton model generation unit 21 includes the right wrist 601 and the right elbow 602. The position of each part is determined. On the other hand, in the example shown in FIG. 6B, the right wrist 601 and the right elbow 602 of the driver 600 are hidden by the left hand at the time of the current frame. Among these, since the right wrist 601 is a highly reliable part, the estimated position in the current frame is obtained by the highly reliable part tracking unit 22. On the other hand, the right elbow 602 is a low reliability part. Therefore, the low-reliability region estimation unit 23 estimates the estimated position of the right wrist 601 and the position of the right shoulder 603 in the current frame, the distance from the right wrist 601 to the right elbow 602, and the distance from the right shoulder 603 to the right elbow 602. Based on the above, the position of the right elbow 602 is estimated.

  The driving authority transfer determination unit 24 is an example of a posture determination unit, and when the authority transfer preparation mode is set, the driver determines the driving authority according to at least one of the positions represented in the skeleton model. It is determined whether or not the attitude is delegable. Then, the driving authority transfer determination unit 24 determines whether to transfer the driving authority from the automatic driving device 1 to the driver based on the determination result.

FIG. 7A to FIG. 7E are diagrams showing examples of determination conditions for delegating driving authority from the automatic driving apparatus 1 to the driver. As shown in FIGS. 7A to 7E, when all of the following conditions are satisfied, the driving authority transfer determination unit 24 determines the driver's posture from the automatic driving apparatus 1 to the driver. It is determined that the posture is delegable. On the other hand, when any of the conditions is not satisfied, the driving authority transfer determination unit 24 determines that the attitude of the driver is not an attitude capable of transferring the driving authority to the driver.
(a) The driver 700 holds the steering wheel 701 with both hands (FIG. 7A).
(b) The angle θ of both elbows of the driver 700 is included in a predetermined angle range (for example, 90 to 150 °) (FIG. 7B).
(c) The driver's head 702 is in a position protected by the headrest 703 (FIG. 7C).
(d) The head 702 is above a predetermined distance (for example, 20 to 40 cm) from the upper end of the dashboard 704. That is, the front view of the driver is ensured (FIG. 7D).
(e) The driver 700 is facing forward (FIG. 7E)

  Of the above conditions (a) to (e), for condition (a), when the distance from the steering position to each hand is a predetermined allowable distance (for example, 3 to 5 cm) or less, the driving authority is transferred. The determination unit 24 determines that the driver is holding the steering wheel with both hands. Similarly, for condition (c), when the distance from the headrest position to the head is equal to or less than a predetermined allowable distance, the driving authority transfer determination unit 24 determines that the driver's head is in a position protected by the headrest. What is necessary is just to judge. In addition, regarding the condition (d), when the difference obtained by subtracting the vertical coordinate of the upper end of the dashboard from the vertical coordinate of the head of the driver is greater than a predetermined distance, the driving authority transfer determination unit 24 determines the forward view of the driver May be determined to be secured. Note that the respective positions of the steering wheel, the headrest, and the upper end of the dashboard are stored in the storage unit 4 in advance. Furthermore, for the condition (b), the driving authority transfer determination unit 24 calculates, for each of the right hand and the left hand, the intersection angle between the straight line connecting the wrist and the elbow and the straight line connecting the shoulder and the elbow as the elbow angle, What is necessary is just to determine whether an angle is contained in a predetermined angle range. Further, regarding the condition (e), the driving authority transfer determination unit 24 indicates that the driver faces forward when the difference between the distance from the left shoulder to the head and the distance from the right shoulder to the head is within a predetermined range. What is necessary is just to determine that it exists.

  The driving authority transfer determination unit 24 can delegate the driving authority when the conditions (a), (d), and (e) are satisfied among the above conditions (a) to (e). It may be determined that the posture is correct.

  The driving authority transfer determination unit 24 determines that the driving authority is transferred from the automatic driving device 1 to the driver when the driver's attitude is such that the driving authority can be transferred. On the other hand, when the posture of the driver is not a posture capable of transferring the driving authority, the driving authority transfer determining unit 24 determines that the driving authority is not transferred to the driver. In addition, the driving authority transfer determination unit 24 drives the driving authority from the automatic driving device 1 to the driver when the driver's attitude is such that the driving authority can be transferred continuously for a predetermined period (for example, 1 to 3 seconds). May be determined to be delegated.

  When the driving authority transfer determination unit 24 determines that the driving authority is to be transferred from the automatic driving device 1 to the driver, a message indicating that is displayed on a display (not shown) provided in the vehicle compartment, or to that effect Is output from a speaker (not shown). And the control part 5 complete | finishes an automatic driving | operation process after the fixed period passes after the determination, and switches to the driving | operation by a driver.

  FIG. 8 is an operation flowchart of posture measurement processing executed by the control unit 5. Each time the control unit 5 receives the three-dimensional measurement data from the three-dimensional sensor 2, the control unit 5 executes the posture measurement process according to the operation flowchart.

  The skeletal model generation unit 21 obtains the position of each part used for estimating the posture of the driver, for example, the head, both shoulders, both elbows, and both wrists in the real space of the current frame obtained from the three-dimensional sensor 2. A skeleton model is generated by detection based on the three-dimensional measurement data (step S101).

  The high-reliability site tracking unit 22 selects a high-reliability site among the sites not detected by the skeleton model generation unit 21 (step S102). The high-reliability part tracking unit 22 tracks the selected high-reliability part based on the skeleton model of the previous frame stored in the storage unit 4 and the three-dimensional measurement data of the current frame. The position in the real space is updated (step S103).

  The low reliability part estimation unit 23 selects a low reliability part among the parts not detected by the skeleton model generation unit 21 (step S104). The low reliability region estimation unit 23 specifies a plurality of high reliability regions adjacent to the low reliability region for each low reliability region based on the human body structure model. Then, the low reliability part estimation unit 23 determines, for each low reliability part, the positions of the adjacent high reliability parts in the real space and the length between the adjacent high reliability parts and the low reliability parts. Based on this, the estimated position of the low reliability part is updated (step S105).

  The driving authority transfer determination unit 24 determines whether the authority transfer preparation mode is set (step S106). If the authority transfer preparation mode is not set (step S106-No), the control unit 5 ends the posture measurement process. On the other hand, when the authority transfer preparation mode is set (step S106-Yes), the driving authority transfer determination unit 24 determines that the driver's posture is the driving authority from the automatic driving device 1 to the driver based on the position of each part. It is determined whether or not the posture is delegable (step S107). When the posture of the driver is a posture capable of delegating authority (step S107—Yes), the driving authority transfer determination unit 24 determines to delegate the driving authority from the automatic driving device 1 to the driver. And the control part 5 transfers driving authority to a driver (step S108). And the control part 5 complete | finishes an attitude | position measurement process. On the other hand, when the posture of the driver is not a posture capable of delegating authority (No in step S107), the driving authority transfer determination unit 24 determines that the driving authority is not transferred from the automatic driving device 1 to the driver. And the control part 5 complete | finishes an attitude | position measurement process.

  As described above, this posture measurement device is capable of tracking a highly reliable part in the real space by the tracking process among the parts in which the position in the real space cannot be specified based on the three-dimensional measurement data of the current frame. To find the estimated position. As described above, since this posture measurement apparatus performs tracking processing on a highly reliable part that is assumed to be able to accurately estimate the position in the real space using the tracking process, the position of the part is estimated with high accuracy. it can. On the other hand, this posture measuring device, for the low-reliability part among the parts for which the position in the real space could not be specified, based on the human body structure model and the positions of a plurality of adjacent high-reliability parts in the real space Find the estimated position. Therefore, this posture measuring device can estimate the position of a low-reliability part.

  According to the modification, the skeleton model generation unit 21 may detect each part only for the three-dimensional measurement data of the first frame after the posture measurement process is started. That is, in the operation flowchart of FIG. 8, the process of step S101 may be performed only for the first frame. For the subsequent frames, the high-reliability part tracking unit 22 performs the tracking process for all high-reliability parts based on the three-dimensional measurement data of the latest frame and the skeleton model calculated for the immediately preceding frame. Then, the estimated position of the highly reliable part may be updated. In addition, the low-reliability-part estimation unit 23 updates the estimated positions of the low-reliability parts based on the estimated positions of the plurality of adjacent high-reliability parts and the human body model for all the low-reliability parts. That's fine.

  Note that any part may not be detected from the three-dimensional measurement data of the first frame by the processing of the skeleton model generation unit 21 according to the above embodiment. In such a case, the skeleton model generation unit 21 may perform the same process as the process of the low reliability part estimation unit 23 for each part that could not be detected, and obtain the estimated position of the part. . At that time, the skeletal model generation unit 21 does not use the position of the low-reliability part for the high-reliability part that could not be detected, even if the position of the adjacent low-reliability part is detected. You may utilize the position of several high reliability site | parts already detected in order from the one close | similar to a reliability site | part. Further, until the skeleton model generation unit 21 can detect all the parts from the three-dimensional measurement data of one frame, the control unit 5 executes the posture measurement process for each frame according to the operation flowchart shown in FIG. Also good. And if the skeleton model production | generation part 21 can detect all the site | parts from the three-dimensional measurement data of one frame, the control part 5 may perform attitude | position measurement processing according to this modification about the subsequent frames.

  According to this modification, the posture measurement device does not have to execute the processing of the skeleton model generation unit 21 for all the frames, so that the amount of calculation can be reduced.

  In addition, the posture measurement device according to the above-described embodiment or modification may be used for other purposes. For example, the posture measurement device according to the above-described embodiment or modification continuously measures the position of each part of the driver while the driver is driving, and checks the change in the posture of the driver based on the measurement result. Thus, it may be used to determine the state of the driver.

  In addition, the computer program that realizes the function of each unit of the control unit according to the above-described embodiment or modification is recorded in a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. May be provided. The recording medium does not include a carrier wave.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
A three-dimensional sensor that generates three-dimensional measurement data representing the position of each point in the measurement range in real space;
For each of a plurality of parts of a person, a part adjacent to the part and a human body structure model representing a length from the part to the part adjacent to the part, and a position measurement in real space for each of the plurality of parts A storage unit that stores reliability information indicating whether the reliability representing the accuracy of accuracy is the first reliability or the second reliability lower than the first reliability;
Of the plurality of parts of the person to be measured within the measurement range, the position of the first part having the first reliability in the real space is represented by the three-dimensional measurement data and the past real space of the part. A tracking unit obtained by a tracking process based on the position measurement result at
Among the plurality of parts of the person to be measured, the position of the second part having the second reliability in the real space is the position of the plurality of first parts in the real space and the human body structure model. A position estimation unit to be obtained based on the length between the first part and the second part determined based on
A posture measuring device having
(Appendix 2)
The position estimation unit selects a plurality of the first parts adjacent to the second part as the plurality of first parts based on the human body structure model, and the selected first parts For each of the above, the length between the first part and the second part is obtained based on the human body structure model, and based on the point that becomes the length from each of the selected first parts The posture measurement apparatus according to attachment 1, wherein the position of the second part in real space is obtained.
(Appendix 3)
A detector that detects positions of the measurement target person in real space based on positions of the measurement points included in the three-dimensional measurement data in real space;
The posture measurement apparatus according to appendix 1 or 2, wherein the tracking unit obtains a position in the real space of the first part of the first part that has not been detected in the real space by the detection unit. .
(Appendix 4)
The posture measurement apparatus according to appendix 3, wherein the position estimation unit obtains a position in the real space of a second part of the second part that has not been detected in real space by the detection unit.
(Appendix 5)
The three-dimensional sensor has a vehicle interior as the measurement range, and the person to be measured is a driver of the vehicle,
The posture measurement device according to any one of appendices 1 to 4, wherein the first part includes a shoulder and a wrist of the driver of the vehicle, and the second part includes an elbow of the driver of the vehicle.
(Appendix 6)
Appendix 5, further comprising an attitude determination unit that determines whether or not the driver is taking a predetermined attitude based on at least one of the first part and the second part of the driver The attitude measurement device described.
(Appendix 7)
Among the plurality of parts of the person to be measured within the measurement range of the three-dimensional sensor, in the real space of the first part having the first reliability as the reliability representing the reliability of the measurement accuracy of the position in the real space. Tracking processing based on the three-dimensional measurement data representing the position in the real space of each measurement point within the measurement range generated by the three-dimensional sensor and the measurement result of the position of the relevant part in the past in the real space Sought by,
Among the plurality of parts of the person to be measured, the position in the real space of the second part having the second reliability lower than the first reliability is defined as the real space of the plurality of first parts. And each of the plurality of parts of the person, the part adjacent to the part, and the first part determined based on the human body structure model representing the length from the part to the adjacent part and the part Obtained based on the length between the second parts,
Posture measurement method.
(Appendix 8)
Among the plurality of parts of the person to be measured within the measurement range of the three-dimensional sensor, in the real space of the first part having the first reliability as the reliability representing the reliability of the measurement accuracy of the position in the real space. Tracking processing based on the three-dimensional measurement data representing the position in the real space of each measurement point within the measurement range generated by the three-dimensional sensor and the measurement result of the position of the relevant part in the past in the real space Sought by,
Among the plurality of parts of the person to be measured, the position in the real space of the second part having the second reliability lower than the first reliability is defined as the real space of the plurality of first parts. And each of the plurality of parts of the person, the part adjacent to the part, and the first part determined based on the human body structure model representing the length from the part to the adjacent part and the part Obtained based on the length between the second parts,
A computer program for posture measurement that causes a computer to execute this.

1 Automatic driving device (Attitude measurement device)
2 3D sensor 3 Outside vehicle sensor 4 Storage unit 5 Control unit 6 Board 7 In-car network 10 Vehicle 11 Driver 21 Skeletal model generation unit 22 High reliability part tracking part 23 Low reliability part estimation part 24 Driving authority transfer judgment part

Claims (5)

A three-dimensional sensor that generates three-dimensional measurement data representing the position of each point in the measurement range in real space;
For each of a plurality of parts of a person, a part adjacent to the part and a human body structure model representing a length from the part to the part adjacent to the part, and a position measurement in real space for each of the plurality of parts A storage unit that stores reliability information indicating whether the reliability representing the accuracy of accuracy is the first reliability or the second reliability lower than the first reliability;
Of the plurality of parts of the person to be measured within the measurement range, the position of the first part having the first reliability in the real space is represented by the three-dimensional measurement data and the past real space of the part. A tracking unit obtained by a tracking process based on the position measurement result at
Among the plurality of parts of the person to be measured, the position of the second part having the second reliability in the real space is the position of the plurality of first parts in the real space and the human body structure model. A position estimation unit to be obtained based on the length between the first part and the second part determined based on
A posture measuring device having   The position estimation unit selects a plurality of the first parts adjacent to the second part as the plurality of first parts based on the human body structure model, and the selected first parts For each of the above, the length between the first part and the second part is obtained based on the human body structure model, and based on the point that becomes the length from each of the selected first parts The posture measuring apparatus according to claim 1, wherein the position of the second part in real space is obtained. A detector that detects positions of the measurement target person in real space based on positions of the measurement points included in the three-dimensional measurement data in real space;
The posture measurement according to claim 1, wherein the tracking unit obtains a position in the real space of the first part of the first part that has not been detected in the real space by the detection unit. apparatus.   The posture measurement device according to claim 3, wherein the position estimation unit obtains a position in the real space of a second part of the second part, the position of which is not detected in the real space by the detection unit. . Among the plurality of parts of the person to be measured within the measurement range of the three-dimensional sensor, in the real space of the first part having the first reliability as the reliability representing the reliability of the measurement accuracy of the position in the real space. Tracking processing based on the three-dimensional measurement data representing the position in the real space of each measurement point within the measurement range generated by the three-dimensional sensor and the measurement result of the position of the relevant part in the past in the real space Sought by,
Among the plurality of parts of the person to be measured, the position in the real space of the second part having the second reliability lower than the first reliability is defined as the real space of the plurality of first parts. And each of the plurality of parts of the person, the part adjacent to the part, and the first part determined based on the human body structure model representing the length from the part to the adjacent part and the part Obtained based on the length between the second parts,
Posture measurement method.

Images