WO2017188017A1 - Dispositif de détection, procédé de détection et programme - Google Patents

Dispositif de détection, procédé de détection et programme Download PDF

Info

Publication number
WO2017188017A1
WO2017188017A1 PCT/JP2017/015212 JP2017015212W WO2017188017A1 WO 2017188017 A1 WO2017188017 A1 WO 2017188017A1 JP 2017015212 W JP2017015212 W JP 2017015212W WO 2017188017 A1 WO2017188017 A1 WO 2017188017A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
subject
detection
distance
distance information
Prior art date
Application number
PCT/JP2017/015212
Other languages
English (en)
Japanese (ja)
Inventor
小野　博明
英史 山田
光永　知生
Original Assignee
ソニーセミコンダクタソリューションズ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーセミコンダクタソリューションズ株式会社 filed Critical ソニーセミコンダクタソリューションズ株式会社
Publication of WO2017188017A1 publication Critical patent/WO2017188017A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume

Definitions

  • the present technology relates to a detection device, a detection method, and a program, and more particularly, to a detection device, a detection method, and a program that detect a predetermined subject from an image, for example.
  • the present technology has been made in view of such a situation, and makes it possible to reduce processing when a predetermined object is recognized (detected).
  • the object is imaged from an acquisition unit that acquires distance information regarding a distance to a subject, the distance information, and a feature amount of an object to be detected.
  • a setting unit configured to set a possible region; and a determination unit configured to determine whether an image in the region is the object.
  • a second detection device sets an area where a predetermined object may be captured using an acquisition unit that acquires distance information regarding a distance to a subject and the distance information A setting unit; and an estimation unit that estimates a category to which the object belongs from the size of the region and the distance information.
  • the first detection method acquires distance information regarding a distance to a subject, and the object may be captured from the distance information and a feature amount of the object to be detected.
  • a step of setting a certain region and determining whether or not an image in the region is the object.
  • distance information related to a distance to a subject is acquired, an area where a predetermined object may be captured is set using the distance information, and the area And estimating the category to which the object belongs from the distance information and the distance information.
  • the first program may acquire distance information related to a distance to a subject in a computer, and the object may be captured from the distance information and a feature amount of the object to be detected.
  • a region having a characteristic is set, and a process including a step of determining whether or not an image in the region is the object is executed.
  • the second program of one aspect of the present technology acquires distance information related to a distance to a subject in a computer, sets an area where a predetermined object may be imaged using the distance information, A process including a step of estimating a category to which the object belongs is executed based on the size of the area and the distance information.
  • distance information regarding the distance to the subject is acquired, and an object is imaged from the distance information and the feature amount of the object to be detected.
  • An area that may be present is set, and it is determined whether or not the image in the area is an object.
  • the distance information related to the distance to the subject is acquired, and the distance information may be used to capture a predetermined object Is set, and the category to which the object belongs is estimated from the size of the region and the distance information.
  • the detection device may be an independent device or an internal block constituting one device.
  • the program can be provided by being transmitted through a transmission medium or by being recorded on a recording medium.
  • processing when recognizing (detecting) a predetermined object can be reduced.
  • the present technology is applicable to recognizing (detecting) a predetermined object, for example, an object such as a person (face, upper body, whole body), automobile, bicycle, foodstuff, or the like.
  • a predetermined object for example, an object such as a person (face, upper body, whole body), automobile, bicycle, foodstuff, or the like.
  • the present technology detects such a predetermined object using distance information.
  • distance information In the following description, a case where a human face is detected using distance information will be described as an example.
  • FIG. 1 is a diagram illustrating a configuration of an embodiment of a detection device to which the present technology is applied.
  • the detection apparatus 100 illustrated in FIG. 1 includes a distance information acquisition unit 111, a subject feature extraction unit 112, a subject candidate area detection unit 113, and an actual size database 114.
  • the distance information acquisition unit 111 measures the distance to the subject, generates a measurement result (distance information), and outputs the result to the subject feature extraction unit 112.
  • the distance information acquisition unit 111 acquires distance information by a distance measuring sensor using active light (infrared rays or the like), for example.
  • a distance measuring sensor using active light a TOF (Time-of-Flight) method, a Structured Light method, or the like can be applied.
  • the distance information acquisition unit 111 may be configured to acquire distance information by a distance measuring sensor (ranging method) using reflected light of active light, for example, a TOF light source or a camera flash light.
  • the distance information acquisition unit 111 may be configured to acquire distance information with a stereo camera.
  • the distance information acquisition unit 111 may be configured to acquire distance information using an ultrasonic sensor.
  • the distance information acquisition unit 111 may be configured to acquire distance information by a method using a millimeter wave radar.
  • the subject feature extraction unit 112 sets a detection target, for example, a frame that may have a human face, from the distance information.
  • the frame is set by referring to a table stored in the actual size database 114.
  • the actual size database 114 manages a table in which the size on the image in consideration of the distance and the actual size of the detected subject is associated. For example, when there is a person's face at a position separated by a predetermined distance, the table describes how much the face of the person is on the image.
  • the subject candidate area detection unit 113 determines whether or not there is a detection target within the set frame. If there is, the subject candidate area detection unit 113 cuts out the frame and outputs it to a subsequent processing unit (not shown).
  • the subject feature extraction unit 112 sets a pixel (target pixel) to be processed.
  • the pixel of interest is sequentially set from the upper left pixel of the image.
  • the pixel of interest is sequentially set from the upper left pixel to the lower right pixel of the distance image 131.
  • the order of the target pixels that are sequentially set is represented by an arrow, but the target pixels may be set in an order other than such an order.
  • the distance image 131 is an image generated from distance information.
  • the same distance is an image represented by the same color and colored according to the distance.
  • the distance image 131 according to the present technology does not need to be an image colored according to the distance, but simply knows how far a predetermined pixel (subject) in the image 131 is from the detection device 100. Any image can be used.
  • the description will be continued assuming that the distance image 131 is generated as shown in FIG. Further, as illustrated in the upper diagram of FIG. 3, the description will be continued with an example in which a pixel at a predetermined position in the distance image 131 is set as the pixel of interest 132.
  • step S102 distance information on the target pixel is acquired.
  • step S103 the size of the detection frame is determined from the distance and the actual size of the subject to be detected. As shown in the middle diagram of FIG. 3, the subject feature extraction unit 112 sets a detection frame 133 from the distance of the target pixel 132 and the actual size of the subject to be detected.
  • the subject feature extraction unit 112 sets a detection frame 133 with reference to a table managed by the actual size database 114.
  • a table 151 as shown in FIG. 4 is stored.
  • the table 151 shown in FIG. 4 is a table in which the size on the image based on the distance and the actual size of the face is associated. For example, when the distance is 0 (cm), the size on the face image is 30 pixels ⁇ 30 pixels, and when the distance is 50 (cm), the size on the face image is 25 pixels ⁇ 25 pixels. Yes, when the distance is 100 (cm), the relationship that the size on the face image is 20 pixels ⁇ 20 pixels is described.
  • the distance is a distance between the detection device 100 and a subject (in this case, a human face).
  • the actual face size is the average human face size at a predetermined distance, for example, 50 cm away. Since human faces vary depending on gender and age, and there are individual differences, the actual face size will be described here as an average human face size.
  • a table 151 in which the size on the image based on the actual size of a plurality of faces is associated with one distance may be created, and processing may be performed using such a table 151. For example, for one distance, the size on the image based on the actual size of the male face, the size on the image based on the actual size of the female face, and the size on the image based on the actual size of the child May be associated.
  • a detection frame 133 corresponding to the size on the image based on each actual size may be set, and the process of step S104 described later may be executed for each detection frame 133.
  • FIG. 4 shows an example in which the distances are related to 0, 50, 100 and the size on the image based on the actual size in units of 50 centimeters.
  • the distance is not limited to 50 centimeters. The value can be changed depending on the accuracy of information and the accuracy required for detection.
  • step S103 the subject feature extraction unit 112 sets the detection frame 133 from the size on the image based on the distance of the target pixel 132 and the actual size of the subject to be detected, as shown in the middle diagram of FIG. To do.
  • the pixel of interest 132 is 25 pixels ⁇ 25 pixels centering on the target pixel 132.
  • a detection frame 133 is set.
  • the detection frame 133 is described as an example of a quadrangle as illustrated in FIG. 3, but the detection frame 133 is not limited to a rectangle such as a rectangle, but may be other shapes such as a circle.
  • the actual size of the subject is the feature (feature amount) of the subject and the detection frame 133 is set using the size on the image based on the actual size will be described. It is also possible to set the detection frame 133 using the feature (feature amount).
  • the subject feature extraction unit 112 functions as a setting unit that sets the size of the subject as the feature amount and sets the detection frame 133 set to the feature amount in this case.
  • the detection frame 133 is a frame that is set by calculating the size on the distance image 131 if there is a subject to be detected, for example, a human face, at the distance of the position of the target pixel 132. It is. Note that such calculation itself is omitted, and other forms such as those described in the table 151 can be applied.
  • the subject candidate area detection unit 113 determines whether the image in the detection frame 133 is a candidate for a subject to be detected. For example, a filter having the same size as the detection frame 133 is applied to the distance image 131, and the response value is used as the probability value of the subject candidate.
  • a filter having the same size as the detection frame 133 is applied to the distance image 131, and the response value is used as the probability value of the subject candidate.
  • a DOG (Difference-of-Gaussian) filter, a Laplacian filter, or the like can be applied.
  • Whether the image in the detection frame 133 is a candidate for a subject to be detected can be determined using distance information in the detection frame 133 and the detection frame 133.
  • the distance information in the detection frame 133 is also information that varies in distance. Also, for example, when a human face is captured in the detection frame 133, but in the case of a human face shown in a photograph (poster), the distance information in the detection frame 133 is a constant value, The information is not distant.
  • the determination result is output to a processing unit (not shown) at the subsequent stage of the detection apparatus 100. Only when it is determined that there is a subject to be detected in the detection frame 133, the image in the detection device 100 can be cut out from the image 131, and the cut out image can be output.
  • the image in the detection frame 133-1 is cut out from the image 131, Is output. If it is determined that there is no face in the detection frame 133-2 and there is no subject to be detected, the image in the detection frame 133-2 is not cut out.
  • the timing at which the cutout is performed can be performed after the processing in step S105 is completed. Further, the determination result in the determination process in step S104, that is, in this case, the value when the filter is applied is set as the probability value of the subject candidate, and after the process in step S105, clipping is performed based on the probability value. good. It is also possible to have a configuration in which only the probability value is output to the subsequent stage.
  • the subject candidate area detection unit 112 functions as a determination unit that determines whether the image in the detection frame 133 is a subject to be detected.
  • an image that is determined by the subject candidate area detection unit 112 to be a subject to be detected can be cut out and output to a subsequent processing unit or the like.
  • step S105 it is determined whether or not such processing has been completed for all pixels in the image 131. If it is determined that the processing has not been completed for all pixels, the processing returns to step S101. Then, a new target pixel is set, and the processing after step S102 is performed on the set target pixel.
  • step S105 if it is determined in step S105 that such processing has been completed for all the pixels in the image 131, the recognition processing is terminated.
  • the probability value of the subject candidate is obtained for all the pixels in the distance image 131. Then, the maximum value of the probability value is set as the center position of the detection subject, the pixel at the center position is set as the target pixel 132, and the image in the detection frame 133 is cut out.
  • the target pixel 132 since the target pixel 132 is set and the detection frame 133 is set as described above, the target pixel 132 may not be set for all the pixels in the image 131.
  • the detection frame 133 cannot be set. Therefore, even if the detection frame 133 is set, a part of the detection frame 133 (in this case, 3/4) is missing. Therefore, a pixel in such a region may not be set as the target pixel 132.
  • the region near the side of the image 131 is also a region where the detection frame 133 cannot be set. Therefore, a pixel in such a region may not be set as the target pixel 132.
  • the target pixel 132 may be sequentially set pixel by pixel, but may be set at a predetermined interval, for example, every five pixels.
  • an area where the distance is determined to be far away in other words, a pixel in an area where the background can be determined may not be set as the pixel of interest 132.
  • the target pixel 132 to be processed can be reduced, and the processing can be reduced.
  • FIG. 5 shows an example of the image 131, and shows a case where the detection frames 133-1 to 133-4 are set as regions where there is a possibility that there is a detection target (human face).
  • the subject candidate area detection unit 113 determines that there is a face in the detection frame 133-1 set according to the distance of the subject by the subject feature extraction unit 112 (FIG. 1). The image within the detection frame 133-1 is cut out and output.
  • the subject candidate area detection unit 113 determines that there is no face in the detection frame 133-2 set according to the distance of the subject by the subject feature extraction unit 112 (FIG. 1). The image within the detection frame 133-2 is not cut out.
  • the subject candidate area detection unit 113 determines that there is a face within the detection frame 133-3 set according to the distance of the subject by the subject feature extraction unit 112 (FIG. 1). The image within the detection frame 133-3 is cut out and output.
  • the face is a photograph or the like.
  • the subject candidate area detection unit 113 determines that there is no face, the image in the detection frame 133-4 is not cut out.
  • an object is detected using the distance and the size of the detected object at the distance.
  • the detection target is a human face
  • the detection can be reduced by performing detection using the present technology rather than performing detection by pattern matching or the like.
  • FIG. 6 is a diagram illustrating a configuration example of the detection device 200 according to the second embodiment.
  • the same portions are denoted by the same reference numerals, and description thereof is omitted.
  • the detection device 200 in the second embodiment is configured by adding an imaging unit 211 and a subject detail recognition unit 212 to the detection device 100 in the first embodiment.
  • the imaging unit 211 includes an imaging element such as a CCD or a CMOS image sensor, captures an image of ambient light (described as a normal image), and supplies the image to the subject detail recognition unit 212.
  • the detection result from the subject candidate area detection unit 113 is also supplied to the subject detail recognition unit 212.
  • the subject candidate region detection unit 113 cuts out and outputs a region determined to have a detection target, for example, a human face, using the distance image 131.
  • the subject detail recognition unit 212 performs more detailed recognition on the subject in the region supplied from the subject candidate region detection unit 113 using the normal image. For example, a recognition process for specifying an individual such as gender and age is performed.
  • Steps S201 to S205 are processes performed by the distance information acquisition unit 111 to the subject candidate area detection unit 113, and are performed in the same manner as steps S101 to S105 of the flowchart shown in FIG.
  • the subject detail recognition unit 212 performs detail recognition using the subject candidate detection frame. For example, the subject detail recognition unit 212 sets the detection frame 133 supplied from the subject candidate region detection unit 113 as a corresponding region of the normal image from the imaging unit 211, and cuts out the image within the set detection frame 133. Then, using the extracted normal image, a preset recognition process such as a recognition process for specifying an individual such as the sex or age of the subject is executed.
  • a preset recognition process such as a recognition process for specifying an individual such as the sex or age of the subject is executed.
  • the information supplied from the subject candidate area detecting unit 113 to the subject detail recognizing unit 212 includes the size on the image (detection frame 133) based on the actual size of the subject, the representative point (for example, the target pixel 132), and the distribution map. It can be information such as (eg, heat map, filter response value).
  • the subject detail recognition unit 212 performs detail recognition using information supplied from the subject candidate region detection unit 113.
  • FIG. 8 By performing such detection (recognition) processing, for example, a detection result as shown in FIG. 8 is obtained.
  • the upper and middle views of FIG. 8 are the same as FIG. That is, by the detection process using the distance image 131, the detection frame 133-1 and the detection frame 133-3 are supplied to the subject detail recognition unit 212 as information on the area where the subject is detected.
  • the subject detail recognizing unit 212 uses, for example, a method such as DNN (DeepningLearning) to detect an image cut out from the normal image when the detection frame 133-1 and the detection frame 133-3 are set for the normal image. To perform recognition processing.
  • DNN DeepningLearning
  • the distance and the size of the detected object at the distance are used to detect the detected object, so that the detection accuracy is improved and the processing load related to the detection is reduced. It becomes possible to make it. Furthermore, in the second embodiment, since a detailed recognition process is executed using a normal image (an image other than a distance image), the subject can be detected in more detail and the subject can be recognized.
  • FIG. 9 is a diagram illustrating a configuration example of the detection apparatus 300 according to the third embodiment.
  • the same portions are denoted by the same reference numerals, and description thereof is omitted.
  • the detection apparatus 300 according to the third embodiment is different from the detection apparatus 100 according to the first embodiment in that a subject direction detection unit 311 is added to the detection apparatus 100 according to the first embodiment. .
  • the subject direction detection unit 311 detects the direction in which the detected subject is facing.
  • the detection device 300 according to the third embodiment detects the position, size, and direction of the subject.
  • Steps S301 to S306 are processes performed by the distance information acquisition unit 111 to the subject candidate area detection unit 113, and are performed in the same manner as steps S101 to S105 in the flowchart shown in FIG. Description is omitted.
  • step S305 the subject candidate region detection unit 113 determines that the region (the region set by the detection frame 133) determined that there is a subject to be detected and the image cut out from the region are displayed in the subject direction detection unit 311. Supplied.
  • the subject direction detection unit 311 detects the direction of the detected subject.
  • a case where a screen as shown in FIG. 11 is acquired will be described as an example, and the direction detection will be described.
  • the detection target is described as being a hand.
  • the subject feature extraction unit 112 and the subject candidate region detection unit 113 execute the processes of steps S302 to S304, so that a detection frame 133 is set in the distance image 131, and the detection frame 133 includes a detection target. A hand is detected.
  • the subject direction detection unit 311 divides the inside of the detection frame 133 into a predetermined size, and uses the divided area as the subject surface, and obtains the normal direction of the surface.
  • the palm faces rightward in the figure. When the palm is directed to the right, as the distance information of the palm part, distance information gradually getting farther from the front toward the back is obtained.
  • a normal in the right direction in the figure is set as shown in FIG. From this set normal, it is determined that the palm is facing the right direction in the figure.
  • the detection object is detected using the distance and the size of the detection object at the distance, so the detection accuracy is improved. It becomes possible to make it. It is also possible to determine the direction of the detected subject.
  • FIG. 12 is a diagram illustrating a configuration example of the detection apparatus 400 according to the fourth embodiment.
  • the same portions are denoted by the same reference numerals, and description thereof is omitted.
  • the detection device 400 according to the fourth embodiment is configured by adding an imaging unit 411 and a subject detail recognition unit 412 to the detection device 300 according to the third embodiment.
  • the added imaging unit 411 and subject detail recognition unit 412 perform basically the same processing as the imaging unit 211 and subject detail recognition unit 212 (both in FIG. 6) of the detection apparatus 200 in the second embodiment.
  • the imaging unit 411 captures a normal image and supplies it to the subject detail recognition unit 412.
  • the subject detail recognition unit 412 is also supplied with the detection result from the subject direction detection unit 311.
  • the subject direction detection unit 311 outputs a detection target, for example, the position of a human face (position where the detection frame 133 is set), its size (size of the detection frame 133), and its direction.
  • the subject detail recognition unit 412 performs more detailed recognition on the subject in the area supplied from the subject direction detection unit 311 using a normal image. For example, a recognition process for specifying an individual such as gender and age is performed.
  • Steps S401 to S406 are processes performed by the distance information acquisition unit 111, the subject feature extraction unit 112, the subject candidate region detection unit 113, and the subject direction detection unit 311.
  • the subject detail recognition unit 412 performs detail recognition using the subject candidate detection frame and the direction of the subject.
  • the subject detail recognition unit 412 sets the detection frame 133 supplied from the subject direction detection unit 311 as a corresponding area of the normal image from the imaging unit 411, and cuts out the image in the set detection frame 133.
  • a preset recognition process such as a recognition process for specifying an individual such as the sex or age of the subject is executed. This recognition processing is performed in consideration of the direction of the subject.
  • FIG. 14 shows a diagram comparing the recognition method performed by the detection apparatus 400 according to the fourth embodiment with other recognition methods.
  • the left diagram in FIG. 14 is a diagram illustrating an example of another recognition method. For example, when a face is detected as a detection target from a normal image, first, it is assumed that the detected object is a face, and in order to determine whether the face is facing forward or backward or left and right The determination with reference to the front / rear / left / right determination dictionary 431 is performed.
  • the front / rear determination dictionary 432 is referred to and it is determined whether it is forward-facing or backward-facing. If it is determined to be forward, the forward dictionary 434 is referred to, and it is determined whether or not it is a human face, and if it is a human face, it is determined whether or not it is a forward-facing face. With this process, when data for identifying an individual is described in the forward dictionary 434, a person is identified by matching with the data.
  • the forward / backward determination dictionary 432 is referred to and determined to be backward
  • the backward dictionary 435 is referred to to determine whether or not the face is a person's face. It is determined whether or not there is.
  • the backward dictionary 435 is referred to to determine whether or not the face is a person's face. It is determined whether or not there is.
  • the left / right determination dictionary 433 is referred to and leftward. Or whether it is facing right.
  • the leftward dictionary 436 is referred to, and it is determined whether or not the face is a human face. If the face is a human face, it is determined whether or not the face is a leftward face.
  • the right-facing dictionary 437 is referred to to determine whether or not it is a human face. It is determined whether or not there is.
  • the recognition process is performed by referring to a plurality of dictionaries and making a determination.
  • recognition processing can be performed by preparing an X-direction dictionary 451 and referring to the X-direction dictionary 451.
  • the X-direction dictionary 451 is a dictionary including a forward-facing dictionary 434, a backward-facing dictionary 435, a left-facing dictionary 436, and a right-facing dictionary 437. Since the subject direction recognition unit 412 (FIG. 12) is also supplied with the direction of the subject, only the dictionary relating to the supplied direction can be referred to and the recognition process can be performed.
  • the number of dictionaries (data amount) can be reduced, and the determination process performed a plurality of times with reference to the dictionary can be omitted. Therefore, according to the detection apparatus 400 in the fourth embodiment, it is possible to reduce processing related to recognition processing.
  • the detection target for example, an image that has a high possibility of being a face (a clipped image) is a target for detailed recognition, a region to be processed in the image is narrowed. Therefore, it is possible to reduce processing related to recognition processing.
  • the detection object is detected by using the distance and the size of the detection object at the distance, the detection accuracy can be improved.
  • a detailed recognition process is executed using a normal image (an image other than a distance image)
  • the subject can be detected in more detail and the subject can be recognized.
  • the recognition process can be a process in which the direction of the subject is acquired in advance, and the process can be reduced.
  • the detection target is detected by estimating the size of the subject and estimating the category to which the subject belongs.
  • FIG. 15 is a diagram illustrating a configuration example of the detection apparatus 500 according to the fifth embodiment.
  • the detection apparatus 500 shown in FIG. 15 includes a distance information acquisition unit 111, a subject size estimation unit 511, and a subject category estimation unit 512.
  • the distance information acquisition unit 111 has a configuration similar to that of the distance information acquisition unit 111 included in the detection device 100, for example, and has a function of acquiring distance information for generating the distance image 131.
  • the subject size estimation unit 511 estimates the size of the subject and supplies the estimated size information to the subject category estimation unit 512.
  • the subject category estimation unit 512 estimates the category to which the subject belongs from the estimated size of the subject and the distance at which the subject is located.
  • the detection apparatus 500 estimates the size of the subject, and the category to which the subject belongs, for example, a category such as a human face category or a car category is determined from the size and distance.
  • step S501 the subject size estimation unit 511 sets a target pixel. This process can be performed, for example, in the same manner as step S101 in the flowchart shown in FIG.
  • the subject size estimation unit 511 acquires a distance around the set target pixel position.
  • the subject size estimation unit 511 estimates the subject size based on the peripheral distance distribution. For example, since the distance between the object and the background is greatly different, a region where the distance greatly changes (that is, an edge) is detected with reference to the surrounding distance distribution, so that the region where the object exists (the size up to the edge portion) Can be estimated.
  • step S504 the subject category estimation unit 512 estimates the subject category based on the distance and the subject size. As described above, since the category of the object at the position can be estimated from the distance and the size, such estimation is performed in step S504.
  • a distance image 131 as shown in FIG. 17 is acquired.
  • a distance image 131 illustrated in FIG. 17 is an image in which a hand is captured.
  • the distance distribution around the target pixel 132 is referred to.
  • the distance is greatly different between the hand part and the background. That is, in this case, the part with the hand is a short distance, but the background is a long distance.
  • the distance distribution around the pixel of interest 132 when the distance distribution is referenced in a direction gradually moving away from the pixel of interest 132, there is a portion where the distance changes greatly.
  • the target pixel 132 since the target pixel 132 is set at a position approximately in the center of the palm, when searching from the palm toward the fingertip, the distance information changes rapidly from the tip of the finger to the background. To do. From the pixel of interest 132 to the position where the distance information changes abruptly, the arrows are used in FIG. Note that the position where the distance information changes suddenly decreases when the difference between the distance of the target pixel 132 and the distance of the pixel being searched for changes to a predetermined threshold value or more. It is good also as a position where distance information changed suddenly.
  • a range where an object may exist is estimated from the target pixel 132.
  • a circle or a rectangle (not shown) having a radius from the target pixel 132 to the tip of the longest arrow is set, and the size of the circle or the rectangle is the subject size.
  • This subject size corresponds to the detection frame 133 in the first to fourth embodiments. In other words, the detection frame 133 is set by such processing.
  • the category of the detected subject is estimated from the distance of the target pixel 132 and the subject size (detection frame 133). In the example illustrated in FIG. 17, it is estimated that the detected subject size at the distance of the target pixel 132 belongs to the category “hand”.
  • step S505 It is determined in step S505 whether or not such processing has been executed for all the pixels in the distance image 131. Similar to step S105 in the flowchart shown in FIG. 2, the pixels set as the target pixel 132 may not be all the pixels in the distance image 131 but may be partially excluded.
  • the size of the subject can be estimated from the distance image, and the category can be estimated.
  • a plurality of subjects (categories) can be estimated, and for example, different objects such as a person and a car can be detected.
  • FIG. 18 is a diagram illustrating a configuration example of a detection device 600 according to the sixth embodiment.
  • the same portions are denoted by the same reference numerals, and description thereof is omitted.
  • the detection device 600 in the sixth embodiment is configured by adding an imaging unit 611 and a subject detail recognition unit 612 to the detection device 500 in the fifth embodiment.
  • the added imaging unit 611 performs basically the same processing as the imaging unit 211 (FIG. 6) of the detection apparatus 200 in the second embodiment.
  • the imaging unit 611 captures a normal image and supplies it to the subject detail recognition unit 612.
  • the subject detail recognition unit 612 is also supplied with the distance, subject size, and subject category from the subject category estimation unit 712.
  • the subject detail recognition unit 612 performs more detailed recognition using the normal image using the distance, subject size, and subject category supplied from the subject category estimation unit 712.
  • Steps S601 to S605 are processes performed by the distance information acquisition unit 111, the subject size estimation unit 511, and the subject category estimation unit 512, and are performed in the same manner as steps S501 to S505 in the flowchart shown in FIG. Is omitted.
  • the subject detail recognition unit 412 performs detail recognition using the subject candidate region and the subject category. For example, the subject detail recognition unit 412 sets a frame corresponding to the subject size supplied from the subject category estimation unit 512 as a corresponding region of the normal image from the imaging unit 411, and cuts out an image within the set frame.
  • recognition processing such as recognition processing for specifying an object belonging to the category is set in advance. Execute the process. For example, when the category is determined to be a person, matching with an image belonging to a person is performed. When an individual is specified, or when the category is determined to be a car, matching with an image belonging to a car is performed. Detailed recognition to identify the vehicle type is performed.
  • the size of the subject can be estimated from the distance image, and the category to which the subject belongs can be estimated. Further, according to the detection apparatus 600, a plurality of subjects (categories) can be estimated, and for example, different objects such as a person and a car can be detected. Furthermore, the detected object can be recognized in detail.
  • FIG. 20 is a diagram illustrating a configuration example of the detection apparatus 700 according to the seventh embodiment.
  • the same portions are denoted by the same reference numerals, and description thereof is omitted.
  • the detection device 700 in the seventh embodiment adds a subject shape estimation unit 711 to the detection device 500 in the fifth embodiment, and the subject category estimation unit 712 receives an output from the subject shape estimation unit 711. This configuration differs from the detection device 500 according to the fifth embodiment.
  • the subject shape estimation unit 711 estimates the shape of the subject. Reference is again made to FIG. 17, when the distance image 131 in which the hand is imaged is acquired, the distance information greatly changes from the target pixel 132, that is, by searching up to the edge portion, the shape of the hand is obtained. It is done.
  • the subject category estimation unit 712 performs basically the same processing as the subject category estimation unit 512 of the detection apparatus 500 illustrated in FIG. 15, but the subject category estimation unit 712 illustrated in FIG.
  • the category is estimated using the estimated shape of the subject. Therefore, the category can be estimated with higher accuracy.
  • Steps S701 to S703 are processes performed by the distance information acquisition unit 111 and the subject size estimation unit 511, and are performed in the same manner as steps S501 to S503 in the flowchart shown in FIG.
  • the subject shape estimation unit 711 estimates the shape of the subject based on the distance distribution around the target pixel 132. As described with reference to FIG. 17, the shape is estimated by searching for a portion (edge) where the distance greatly changes using the distance information. In other words, the region where the distance is gradually changing is assumed to be a part of the detected object, and the shape of the object is obtained while determining whether such a distance is changing gently. It is done.
  • step S705 the subject category estimation unit 712 estimates the category to which the subject belongs based on the distance, the subject size, and the shape. In this case, since the category is estimated using not only the distance and the subject size but also the shape information, the category can be estimated with higher accuracy.
  • the detection apparatus 700 in the seventh embodiment it is possible to estimate the size of the subject, estimate the category, and estimate the shape of the subject from the distance image. Further, according to the detection device 700, a plurality of subjects (categories) can be estimated, and for example, different objects such as a person and a car can be detected.
  • category estimation by the subject category estimation unit 712 may be omitted, and the subject shape estimation result by the subject shape estimation unit 711 may be output to a subsequent processing unit (not shown).
  • FIG. 22 is a diagram illustrating a configuration example of the detection apparatus 800 according to the eighth embodiment.
  • the same portions are denoted by the same reference numerals, and description thereof is omitted.
  • the detection apparatus 800 according to the eighth embodiment is configured by adding an imaging unit 811 and a subject detail recognition unit 812 to the detection apparatus 700 according to the seventh embodiment.
  • the added imaging unit 811 performs basically the same processing as the imaging unit 211 (FIG. 6) of the detection apparatus 200 in the second embodiment.
  • the imaging unit 811 captures a normal image and supplies it to the subject detail recognition unit 812.
  • the subject detail recognition unit 812 is also supplied with the distance, subject size, subject category, and subject shape from the subject category estimation unit 712.
  • the subject detail recognition unit 812 performs more detailed recognition using the normal image using the distance, subject size, subject category, and subject shape supplied from the subject category estimation unit 712.
  • Steps S801 to S806 are processing performed by the distance information acquisition unit 111, the subject size estimation unit 511, the subject shape estimation unit 711, and the subject category estimation unit 712, and are similar to steps S701 to S706 of the flowchart shown in FIG. Since it is performed, the description thereof is omitted.
  • the subject detail recognition unit 812 performs detail recognition using the subject candidate area, the subject category, and the subject shape. For example, the subject detail recognition unit 812 sets a frame corresponding to the subject size supplied from the subject category estimation unit 712 as a corresponding region of the normal image from the imaging unit 411, and cuts out an image within the set frame.
  • a recognition process for identifying an object that matches the subject shape from among the objects belonging to the category For example, a preset recognition process is executed. For example, when the category is determined to be a person, matching with an image belonging to the person is performed, and when performing the matching, the object shape is referred to, and an image close to the shape, for example, the shape is a human face. In some cases, processing is performed such as narrowing down recognition to a person's face, and then identifying an individual after narrowing down.
  • the detection apparatus 800 in the eighth embodiment it is possible to estimate the size of the subject, estimate the category, and estimate the shape of the subject from the distance image. Further, the detection apparatus 800 can estimate a plurality of subjects (categories), and can detect different objects such as a person and a car, for example.
  • the detected object can be recognized in detail. Since the detailed recognition can be performed using information such as the estimated size, category, and shape of the subject, the processing related to the detailed recognition can be reduced.
  • an object can be detected from a distance image.
  • the present technology can be applied to a surveillance camera or the like.
  • the present technology can be applied to a game machine, and can be applied to a device that detects a person who plays a game and detects a gesture of the person (detects a hand, a direction of the hand, and the like).
  • the detection device is mounted on a car, and a person, a bicycle, a car other than the own car is detected, and information on the detected object is notified to the user or a collision is prevented.
  • the present invention can also be applied to a part of a device that performs control for avoiding safety.
  • a stacked image sensor in which a plurality of substrates (dies) are stacked can be employed.
  • the detection device 200 is configured by a laminated image sensor will be described by taking the detection device 200 (FIG. 6) in the second embodiment as an example.
  • FIG. 24 is a diagram illustrating a first configuration example of a stacked image sensor in which the entire detection device 200 of FIG. 6 is incorporated.
  • the stacked image sensor of FIG. 24 has a two-layer structure in which a pixel substrate 901 and a signal processing substrate 902 are stacked.
  • the pixel substrate 901 is formed with a distance information acquisition unit 111 (part) and an imaging unit 211 (part).
  • the distance information acquisition unit 111 obtains distance information by the TOF method
  • the distance information acquisition unit 111 includes an irradiation unit that irradiates a subject with predetermined light and an imaging element that receives the irradiated light.
  • the distance information acquisition unit 111 can be formed on the pixel substrate 901 including a part of an imaging element or a part such as an irradiation unit.
  • the imaging unit 211 also includes an image sensor for capturing a normal image.
  • the part of the image sensor that constitutes the distance information acquisition unit 211 can be formed on the pixel substrate 901.
  • a subject feature extraction unit 112 On the eyelid signal processing board 82, a subject feature extraction unit 112, a subject candidate region detection unit 113, an actual size database 114, and a subject detail recognition unit 212 are formed.
  • the distance information acquisition unit 111 of the pixel substrate 901 performs imaging by receiving light incident thereon, and an image (distance image) obtained by the imaging. ) To detect an object to be detected.
  • the imaging unit 211 of the pixel substrate 901 performs imaging by receiving light incident thereon, and is set as a detection target from an image (normal image) obtained by the imaging. A subject image or the like is cut out and output.
  • FIG. 25 is a diagram illustrating a second configuration example of the stacked image sensor in which the entire detection device 200 of FIG. 6 is incorporated.
  • the stacked image sensor in FIG. 25 has a three-layer structure in which a pixel substrate 901, a signal processing substrate 902, and a memory substrate 903 are stacked.
  • a distance information acquisition unit 111 and an imaging unit 211 are formed on the pixel substrate 901, and a subject feature extraction unit 112, a subject candidate region detection unit 113, and a subject detail recognition unit 212 are formed on the signal processing substrate 902. .
  • a real size database 114 and an image storage unit 911 are formed on the memory substrate 903.
  • an image storage unit 911 is stored in the memory substrate 903 as a storage region for storing a detection result by the subject candidate region detection unit 113, for example, an image cut out from a distance image in which a subject to be detected is captured. Is formed.
  • An actual size database 114 storing the table 151 (FIG. 4) is also formed on the memory substrate 903.
  • the pixel substrate 901, the signal processing substrate 902, and the memory substrate 903 are stacked in that order from the top. However, for example, the order of the signal processing substrate 902 and the memory substrate 903 is changed.
  • the pixel substrate 901, the memory substrate 903, and the signal processing substrate 902 can be stacked in this order.
  • the laminated image sensor can be constituted by laminating four or more layers in addition to two or three layers of substrates.
  • a series of processes performed by each of the detection devices 100 to 800 can be performed by hardware or software.
  • a program constituting the software is installed in a general-purpose computer or the like.
  • FIG. 26 is a block diagram illustrating a configuration example of an embodiment of a computer in which a program for executing the above-described series of processes is installed.
  • the program can be recorded in advance in a hard disk 1005 or ROM 1003 as a recording medium built in the computer.
  • the program can be stored (recorded) in a removable recording medium 1011.
  • a removable recording medium 1011 can be provided as so-called package software.
  • examples of the removable recording medium 1011 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a magnetic disc, and a semiconductor memory.
  • the program can be installed on the computer from the removable recording medium 1011 as described above, or can be downloaded to the computer via the communication network or the broadcast network and installed on the built-in hard disk 1005. That is, the program is transferred from a download site to a computer wirelessly via a digital satellite broadcasting artificial satellite, or wired to a computer via a network such as a LAN (Local Area Network) or the Internet. be able to.
  • a network such as a LAN (Local Area Network) or the Internet.
  • the computer includes a CPU (Central Processing Unit) 1002, and an input / output interface 1010 is connected to the CPU 1002 via a bus 1001.
  • CPU Central Processing Unit
  • input / output interface 1010 is connected to the CPU 1002 via a bus 1001.
  • the CPU 1002 executes a program stored in a ROM (Read Only Memory) 1003 accordingly. .
  • the CPU 1002 loads a program stored in the hard disk 1005 to a RAM (Random Access Memory) 1004 and executes it.
  • the CPU 1002 performs the process according to the flowchart described above or the process performed by the configuration of the block diagram described above. Then, the CPU 1002 outputs the processing result as necessary, for example, via the input / output interface 1010, from the output unit 1006, or from the communication unit 1008, and further recorded on the hard disk 1005.
  • the input unit 1007 includes a keyboard, a mouse, a microphone, and the like.
  • the output unit 1006 includes an LCD (Liquid Crystal Display), a speaker, and the like.
  • the processing performed by the computer according to the program does not necessarily have to be performed in chronological order in the order described as the flowchart. That is, the processing performed by the computer according to the program includes processing executed in parallel or individually (for example, parallel processing or object processing).
  • the program may be processed by one computer (processor), or may be distributedly processed by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.
  • the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device housing a plurality of modules in one housing are all systems. .
  • the configuration examples of the detection devices 100 to 800 described above can be combined within a possible range.
  • the present technology can take a configuration of cloud computing in which one function is shared by a plurality of devices via a network and is jointly processed.
  • each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
  • the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
  • the technology according to the present disclosure can be applied to various products.
  • the technology according to the present disclosure may be any type of movement such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, a construction machine, and an agricultural machine (tractor). You may implement
  • FIG. 27 is a block diagram illustrating a schematic configuration example of a vehicle control system 7000 that is an example of a mobile control system to which the technology according to the present disclosure can be applied.
  • the vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010.
  • the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, a vehicle exterior information detection unit 7400, a vehicle interior information detection unit 7500, and an integrated control unit 7600. .
  • the communication network 7010 for connecting the plurality of control units conforms to an arbitrary standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark). It may be an in-vehicle communication network.
  • Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores programs executed by the microcomputer or parameters used for various calculations, and a drive circuit that drives various devices to be controlled. Is provided.
  • Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and is connected to devices or sensors inside and outside the vehicle by wired communication or wireless communication. A communication I / F for performing communication is provided. In FIG.
  • a microcomputer 7610 As a functional configuration of the integrated control unit 7600, a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio image output unit 7670, An in-vehicle network I / F 7680 and a storage unit 7690 are illustrated.
  • other control units include a microcomputer, a communication I / F, a storage unit, and the like.
  • the drive system control unit 7100 controls the operation of the device related to the drive system of the vehicle according to various programs.
  • the drive system control unit 7100 includes a driving force generator for generating a driving force of a vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism that adjusts and a braking device that generates a braking force of the vehicle.
  • the drive system control unit 7100 may have a function as a control device such as ABS (Antilock Brake System) or ESC (Electronic Stability Control).
  • a vehicle state detection unit 7110 is connected to the drive system control unit 7100.
  • the vehicle state detection unit 7110 includes, for example, a gyro sensor that detects the angular velocity of the rotational movement of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, an operation amount of an accelerator pedal, an operation amount of a brake pedal, and steering of a steering wheel. At least one of sensors for detecting an angle, an engine speed, a rotational speed of a wheel, or the like is included.
  • the drive system control unit 7100 performs arithmetic processing using a signal input from the vehicle state detection unit 7110, and controls an internal combustion engine, a drive motor, an electric power steering device, a brake device, or the like.
  • the body system control unit 7200 controls the operation of various devices mounted on the vehicle body according to various programs.
  • the body system control unit 7200 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a headlamp, a back lamp, a brake lamp, a blinker, or a fog lamp.
  • the body control unit 7200 can be input with radio waves or various switch signals transmitted from a portable device that substitutes for a key.
  • the body system control unit 7200 receives input of these radio waves or signals, and controls a door lock device, a power window device, a lamp, and the like of the vehicle.
  • the battery control unit 7300 controls the secondary battery 7310 that is a power supply source of the drive motor according to various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from a battery device including the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature adjustment of the secondary battery 7310 or the cooling device provided in the battery device.
  • the outside information detection unit 7400 detects information outside the vehicle on which the vehicle control system 7000 is mounted.
  • the outside information detection unit 7400 is connected to at least one of the imaging unit 7410 and the outside information detection unit 7420.
  • the imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras.
  • the outside information detection unit 7420 detects, for example, current weather or an environmental sensor for detecting weather, or other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000. At least one of the surrounding information detection sensors.
  • the environmental sensor may be, for example, at least one of a raindrop sensor that detects rainy weather, a fog sensor that detects fog, a sunshine sensor that detects sunlight intensity, and a snow sensor that detects snowfall.
  • the ambient information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device.
  • the imaging unit 7410 and the outside information detection unit 7420 may be provided as independent sensors or devices, or may be provided as a device in which a plurality of sensors or devices are integrated.
  • FIG. 28 shows an example of installation positions of the imaging unit 7410 and the vehicle outside information detection unit 7420.
  • the imaging units 7910, 7912, 7914, 7916, and 7918 are provided at, for example, at least one of the front nose, the side mirror, the rear bumper, the back door, and the upper part of the windshield in the vehicle interior of the vehicle 7900.
  • An imaging unit 7910 provided in the front nose and an imaging unit 7918 provided in the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 7900.
  • Imaging units 7912 and 7914 provided in the side mirror mainly acquire an image of the side of the vehicle 7900.
  • An imaging unit 7916 provided in the rear bumper or the back door mainly acquires an image behind the vehicle 7900.
  • the imaging unit 7918 provided on the upper part of the windshield in the passenger compartment is mainly used for detecting a preceding vehicle or a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or
  • FIG. 28 shows an example of the shooting range of each of the imaging units 7910, 7912, 7914, and 7916.
  • the imaging range a indicates the imaging range of the imaging unit 7910 provided in the front nose
  • the imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided in the side mirrors, respectively
  • the imaging range d The imaging range of the imaging part 7916 provided in the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 7910, 7912, 7914, and 7916, an overhead image when the vehicle 7900 is viewed from above is obtained.
  • the vehicle outside information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners of the vehicle 7900 and the upper part of the windshield in the vehicle interior may be, for example, an ultrasonic sensor or a radar device.
  • the vehicle outside information detection units 7920, 7926, and 7930 provided on the front nose, the rear bumper, the back door, and the windshield in the vehicle interior of the vehicle 7900 may be, for example, LIDAR devices.
  • These outside information detection units 7920 to 7930 are mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, and the like.
  • the vehicle exterior information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. Further, the vehicle exterior information detection unit 7400 receives detection information from the vehicle exterior information detection unit 7420 connected thereto. When the vehicle exterior information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the vehicle exterior information detection unit 7400 transmits ultrasonic waves, electromagnetic waves, or the like, and receives received reflected wave information.
  • the outside information detection unit 7400 may perform an object detection process or a distance detection process such as a person, a car, an obstacle, a sign, or a character on a road surface based on the received information.
  • the vehicle exterior information detection unit 7400 may perform environment recognition processing for recognizing rainfall, fog, road surface conditions, or the like based on the received information.
  • the vehicle outside information detection unit 7400 may calculate a distance to an object outside the vehicle based on the received information.
  • the outside information detection unit 7400 may perform image recognition processing or distance detection processing for recognizing a person, a car, an obstacle, a sign, a character on a road surface, or the like based on the received image data.
  • the vehicle exterior information detection unit 7400 performs processing such as distortion correction or alignment on the received image data, and combines the image data captured by the different imaging units 7410 to generate an overhead image or a panoramic image. Also good.
  • the vehicle exterior information detection unit 7400 may perform viewpoint conversion processing using image data captured by different imaging units 7410.
  • the vehicle interior information detection unit 7500 detects vehicle interior information.
  • a driver state detection unit 7510 that detects the driver's state is connected to the in-vehicle information detection unit 7500.
  • Driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects biometric information of the driver, a microphone that collects sound in the passenger compartment, and the like.
  • the biometric sensor is provided, for example, on a seat surface or a steering wheel, and detects biometric information of an occupant sitting on the seat or a driver holding the steering wheel.
  • the vehicle interior information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, and determines whether the driver is asleep. May be.
  • the vehicle interior information detection unit 7500 may perform a process such as a noise canceling process on the collected audio signal.
  • the integrated control unit 7600 controls the overall operation in the vehicle control system 7000 according to various programs.
  • An input unit 7800 is connected to the integrated control unit 7600.
  • the input unit 7800 is realized by a device that can be input by a passenger, such as a touch panel, a button, a microphone, a switch, or a lever.
  • the integrated control unit 7600 may be input with data obtained by recognizing voice input through a microphone.
  • the input unit 7800 may be, for example, a remote control device using infrared rays or other radio waves, or may be an external connection device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000. May be.
  • the input unit 7800 may be, for example, a camera.
  • the passenger can input information using a gesture.
  • data obtained by detecting the movement of the wearable device worn by the passenger may be input.
  • the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by a passenger or the like using the input unit 7800 and outputs the input signal to the integrated control unit 7600.
  • a passenger or the like operates the input unit 7800 to input various data or instruct a processing operation to the vehicle control system 7000.
  • the storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, and the like.
  • the storage unit 7690 may be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.
  • General-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices existing in the external environment 7750.
  • General-purpose communication I / F7620 is a cellular communication protocol such as GSM (Global System of Mobile communications), WiMAX, LTE (Long Term Evolution) or LTE-A (LTE-Advanced), or wireless LAN (Wi-Fi (registered trademark)). Other wireless communication protocols such as Bluetooth (registered trademark) may also be implemented.
  • the general-purpose communication I / F 7620 is connected to a device (for example, an application server or a control server) existing on an external network (for example, the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point.
  • the general-purpose communication I / F 7620 is a terminal (for example, a driver, a pedestrian or a store terminal, or an MTC (Machine Type Communication) terminal) that exists in the vicinity of the vehicle using, for example, P2P (Peer To Peer) technology. You may connect with.
  • a terminal for example, a driver, a pedestrian or a store terminal, or an MTC (Machine Type Communication) terminal
  • P2P Peer To Peer
  • the dedicated communication I / F 7630 is a communication I / F that supports a communication protocol formulated for use in vehicles.
  • the dedicated communication I / F 7630 is a standard protocol such as WAVE (Wireless Access in Vehicle Environment), DSRC (Dedicated Short Range Communications), or cellular communication protocol, which is a combination of the lower layer IEEE 802.11p and the upper layer IEEE 1609. May be implemented.
  • the dedicated communication I / F 7630 typically includes vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication. ) Perform V2X communication, which is a concept that includes one or more of the communications.
  • the positioning unit 7640 receives, for example, a GNSS signal from a GNSS (Global Navigation Satellite System) satellite (for example, a GPS signal from a GPS (Global Positioning System) satellite), performs positioning, and performs latitude, longitude, and altitude of the vehicle.
  • the position information including is generated.
  • the positioning unit 7640 may specify the current position by exchanging signals with the wireless access point, or may acquire position information from a terminal such as a mobile phone, PHS, or smartphone having a positioning function.
  • the beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from a radio station installed on the road, and acquires information such as the current position, traffic jam, closed road, or required time. Note that the function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.
  • the in-vehicle device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle.
  • the in-vehicle device I / F 7660 may establish a wireless connection using a wireless communication protocol such as a wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB).
  • the in-vehicle device I / F 7660 is connected to a USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-definition Link) via a connection terminal (and a cable if necessary). ) Etc. may be established.
  • the in-vehicle device 7760 may include, for example, at least one of a mobile device or a wearable device that a passenger has, or an information device that is carried into or attached to the vehicle.
  • In-vehicle device 7760 may include a navigation device that searches for a route to an arbitrary destination.
  • In-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.
  • the in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010.
  • the in-vehicle network I / F 7680 transmits and receives signals and the like in accordance with a predetermined protocol supported by the communication network 7010.
  • the microcomputer 7610 of the integrated control unit 7600 is connected via at least one of a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, and an in-vehicle network I / F 7680.
  • the vehicle control system 7000 is controlled according to various programs based on the acquired information. For example, the microcomputer 7610 calculates a control target value of the driving force generation device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and outputs a control command to the drive system control unit 7100. Also good.
  • the microcomputer 7610 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, following traveling based on inter-vehicle distance, vehicle speed maintaining traveling, vehicle collision warning, or vehicle lane departure warning. You may perform the cooperative control for the purpose. Further, the microcomputer 7610 controls the driving force generator, the steering mechanism, the braking device, or the like based on the acquired information on the surroundings of the vehicle, so that the microcomputer 7610 automatically travels independently of the driver's operation. You may perform the cooperative control for the purpose of driving.
  • ADAS Advanced Driver Assistance System
  • the microcomputer 7610 is information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680.
  • the three-dimensional distance information between the vehicle and the surrounding structure or an object such as a person may be generated based on the above and local map information including the peripheral information of the current position of the vehicle may be created.
  • the microcomputer 7610 may generate a warning signal by predicting a danger such as a collision of a vehicle, approach of a pedestrian or the like or an approach to a closed road based on the acquired information.
  • the warning signal may be, for example, a signal for generating a warning sound or lighting a warning lamp.
  • the audio image output unit 7670 transmits an output signal of at least one of audio and image to an output device capable of visually or audibly notifying information to a vehicle occupant or the outside of the vehicle.
  • an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices.
  • Display unit 7720 may include at least one of an on-board display and a head-up display, for example.
  • the display portion 7720 may have an AR (Augmented Reality) display function.
  • the output device may be other devices such as headphones, wearable devices such as glasses-type displays worn by passengers, projectors, and lamps.
  • the display device can display the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, and graphs. Display visually. Further, when the output device is an audio output device, the audio output device converts an audio signal made up of reproduced audio data or acoustic data into an analog signal and outputs it aurally.
  • At least two control units connected via the communication network 7010 may be integrated as one control unit.
  • each control unit may be configured by a plurality of control units.
  • the vehicle control system 7000 may include another control unit not shown.
  • some or all of the functions of any of the control units may be given to other control units. That is, as long as information is transmitted and received via the communication network 7010, the predetermined arithmetic processing may be performed by any one of the control units.
  • a sensor or device connected to one of the control units may be connected to another control unit, and a plurality of control units may transmit / receive detection information to / from each other via the communication network 7010. .
  • a computer program for realizing each function of the detection devices 100 to 800 according to the present embodiment described with reference to FIGS. 1, 6, 9, 12, 15, 18, 20, and 22 is stored in any control unit. Etc. can be implemented. It is also possible to provide a computer-readable recording medium in which such a computer program is stored.
  • the recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.
  • the detection apparatuses 100 to 800 according to the present embodiment described with reference to FIGS. 1, 6, 9, 12, 15, 18, 20, and 22 are applied to the application example illustrated in FIG.
  • the present invention can be applied to the integrated control unit 7600.
  • the components of the detection devices 100 to 800 according to the present embodiment described with reference to FIGS. 1, 6, 9, 12, 15, 18, 20, and 22 are the integrated control unit illustrated in FIG. It may be realized in a module for 7600 (for example, an integrated circuit module composed of one die).
  • the detection devices 100 to 800 according to the present embodiment described with reference to FIGS. 1, 6, 9, 12, 15, 18, 20, and 22 are performed by a plurality of control units of the vehicle control system 7000 illustrated in FIG. It may be realized.
  • this technique can take the following structures.
  • a detection device comprising: a determination unit that determines whether an image in the region is the object.
  • the setting unit The feature amount of the object is the size of the object at a predetermined distance, Set a frame corresponding to the size of the object according to the distance in the pixel set as the processing target, The determination unit The detection device according to (1), wherein it is determined whether an image in the frame is the object.
  • An imaging unit that captures an image of ambient light; The image picked up by the image pickup unit, the size of the object set by the setting unit, an image in an area determined to be the object by the determination unit, and detected by the direction detection unit
  • the detection apparatus according to (3), further comprising: a recognition unit that performs detailed recognition on the object using at least one of the directions of the object.
  • a detection apparatus comprising: an estimation unit configured to estimate a category to which the object belongs from the size of the region and the distance information.
  • the setting unit Set up to the part where the distance information changes as an area where the object may be imaged The estimation unit includes The detection apparatus according to (5), wherein a category to which an object having a size of the area belongs at a distance represented by the distance information in the area is a category to which the object belongs. (7) The detection apparatus according to (5) or (6), further including a shape estimation unit that estimates the shape of the object in the region set by the setting unit using the distance information. (8) The detection device according to (7), wherein the estimation unit estimates the category using at least one of the distance information, the size of the region, and the shape.
  • An imaging unit that captures an image of ambient light; At least one of the image captured by the imaging unit, the size of the region set by the setting unit, the category estimated by the estimation unit, and the shape estimated by the shape estimation unit.
  • the detection device according to (7), further comprising: a recognition unit that performs detailed recognition on the object.
  • the detection device according to any one of (1) to (9), wherein the acquisition unit acquires the distance information using a TOF type sensor, a stereo camera, an ultrasonic sensor, or a millimeter wave radar.
  • (11) Get distance information about the distance to the subject, From the distance information and the feature amount of the object to be detected, set an area where the object may be imaged, A detection method including a step of determining whether an image in the region is the object.

Abstract

La présente technologie concerne un dispositif de détection, un procédé de détection et un programme permettant de détecter un objet prescrit. Le dispositif de détection est équipé : d'une unité d'acquisition qui acquiert des informations de distance relatives à la distance à un sujet photographique ; d'une unité de réglage qui, sur la base des informations de distance et d'une valeur caractéristique d'un objet à détecter, règle une région dans laquelle il existe une possibilité que l'objet soit photographié ; et d'une unité de détermination qui détermine si une image dans la région représente l'objet. Dans un autre mode de réalisation, le dispositif de détection est équipé : d'une unité d'acquisition qui acquiert des informations de distance relatives à la distance à un sujet photographique ; d'une unité de réglage, qui utilise les informations de distance pour régler une région dans laquelle il existe une possibilité qu'un objet prescrit soit photographié ; et d'une unité d'estimation qui, sur la base de la taille de la région et des informations de distance, estime une catégorie à laquelle l'objet appartient. La présente technologie peut être appliquée à des dispositifs de détection qui détectent des objets prescrits.
PCT/JP2017/015212 2016-04-28 2017-04-14 Dispositif de détection, procédé de détection et programme WO2017188017A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016-091357 2016-04-28
JP2016091357A JP2017199278A (ja) 2016-04-28 2016-04-28 検出装置、検出方法、およびプログラム

Publications (1)

Publication Number Publication Date
WO2017188017A1 true WO2017188017A1 (fr) 2017-11-02

Family

ID=60161592

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/015212 WO2017188017A1 (fr) 2016-04-28 2017-04-14 Dispositif de détection, procédé de détection et programme

Country Status (2)

Country Link
JP (1) JP2017199278A (fr)
WO (1) WO2017188017A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112470231A (zh) * 2018-07-26 2021-03-09 索尼公司 信息处理装置、信息处理方法和程序

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019202670A1 (fr) * 2018-04-17 2019-10-24 株式会社ソシオネクスト Procédé de reconnaissance de gestes et dispositif de reconnaissance de gestes
JP7046786B2 (ja) * 2018-12-11 2022-04-04 株式会社日立製作所 機械学習システム、ドメイン変換装置、及び機械学習方法
CN112633218B (zh) * 2020-12-30 2023-10-13 深圳市优必选科技股份有限公司 人脸检测方法、装置、终端设备及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006145352A (ja) * 2004-11-18 2006-06-08 Matsushita Electric Works Ltd 画像処理装置
JP2010020404A (ja) * 2008-07-08 2010-01-28 Toshiba Corp 画像処理装置及びその方法
JP2010165183A (ja) * 2009-01-15 2010-07-29 Panasonic Electric Works Co Ltd 人体検出装置
JP2012243050A (ja) * 2011-05-19 2012-12-10 Fuji Heavy Ind Ltd 環境認識装置および環境認識方法
JP2014106732A (ja) * 2012-11-27 2014-06-09 Sony Computer Entertainment Inc 情報処理装置および情報処理方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006145352A (ja) * 2004-11-18 2006-06-08 Matsushita Electric Works Ltd 画像処理装置
JP2010020404A (ja) * 2008-07-08 2010-01-28 Toshiba Corp 画像処理装置及びその方法
JP2010165183A (ja) * 2009-01-15 2010-07-29 Panasonic Electric Works Co Ltd 人体検出装置
JP2012243050A (ja) * 2011-05-19 2012-12-10 Fuji Heavy Ind Ltd 環境認識装置および環境認識方法
JP2014106732A (ja) * 2012-11-27 2014-06-09 Sony Computer Entertainment Inc 情報処理装置および情報処理方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112470231A (zh) * 2018-07-26 2021-03-09 索尼公司 信息处理装置、信息处理方法和程序

Also Published As

Publication number Publication date
JP2017199278A (ja) 2017-11-02

Similar Documents

Publication Publication Date Title
US10753757B2 (en) Information processing apparatus and information processing method
WO2017159382A1 (fr) Dispositif de traitement de signaux et procédé de traitement de signaux
JP6834964B2 (ja) 画像処理装置、画像処理方法、およびプログラム
US10946868B2 (en) Methods and devices for autonomous vehicle operation
US10880498B2 (en) Image processing apparatus and image processing method to improve quality of a low-quality image
WO2017212928A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image et véhicule
JP6764573B2 (ja) 画像処理装置、画像処理方法、およびプログラム
JP2017068589A (ja) 情報処理装置、情報端末、及び、情報処理方法
CN109791706B (zh) 图像处理装置和图像处理方法
US20190266786A1 (en) Image processing apparatus and image processing method
WO2017188017A1 (fr) Dispositif de détection, procédé de détection et programme
WO2019225349A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations, dispositif d'imagerie, dispositif d'éclairage et objet mobile
WO2018150685A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image, et programme
JP6922169B2 (ja) 情報処理装置および方法、車両、並びに情報処理システム
US11585898B2 (en) Signal processing device, signal processing method, and program
JP7363890B2 (ja) 情報処理装置、情報処理方法及びプログラム
US20220012552A1 (en) Information processing device and information processing method
CN111868778A (zh) 图像处理装置、图像处理方法、以及程序
US20240127042A1 (en) Information processing device, information processing system, information processing method, and recording medium
US20240004075A1 (en) Time-of-flight object detection circuitry and time-of-flight object detection method
WO2020195969A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations, et programme
WO2024070673A1 (fr) Dispositif d'imagerie à semi-conducteurs, dispositif électronique et programme
JP2024065130A (ja) 情報処理装置、情報処理方法及びプログラム
JPWO2020116204A1 (ja) 情報処理装置、情報処理方法、プログラム、移動体制御装置、及び、移動体

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17789303

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17789303

Country of ref document: EP

Kind code of ref document: A1