JP2022117201A - Object recognition device, object recognition method, and program - Google Patents

Abstract

To improve object recognition accuracy.SOLUTION: An object recognition device is provided, comprising a distance measurement unit configured to measure distance based on the time it takes for irradiated light to return after being reflected by an object, a light intensity measurement unit for measuring intensity of reflected light reflected by the object, a correction unit for correcting the light intensity according to the distance, and a recognition processing unit configured to generate object information regarding the object on the basis of the distance and the light intensity corrected by the correction unit.SELECTED DRAWING: Figure 4

Description

TECHNICAL FIELD Embodiments of the present invention relate to an object recognition device, an object recognition method, and a program.

As a distance measurement method, a method of measuring the distance to an object based on TOF (Time of Flight) is known, which is the time it takes for irradiated light to be reflected by the object and returned. As a device using such a ranging method, information for monitoring the inside of the vehicle is generated based on the distance from the sensor installed in the vehicle to an object (passenger, etc.) and the amount of light reflected from the object. There is a device that does (Patent Document 1).

JP 2020-106444 A

In the distance measurement method as described above, the reflected light is attenuated (the amount of light is decreased) as the distance to the object increases, so the accuracy of recognizing a distant object may decrease.

Accordingly, an object of an embodiment of the present invention is to improve object recognition accuracy.

An object recognition device as an embodiment of the present invention includes a distance measuring unit that measures a distance based on the time it takes for irradiated light to be reflected by an object and then to return, and the amount of light reflected by the object. , a correction unit that corrects the light quantity according to the distance, and a recognition processing unit that generates object information about the object based on the distance and the light quantity corrected by the correction unit.

As a result, it is possible to suppress the influence of attenuation of reflected light due to an increase in distance, and to improve the accuracy of object recognition.

Further, the correction unit may increase the amount of light corresponding to a distance longer than a predetermined reference distance so that the difference from the reference amount of light corresponding to the reference distance is equal to or less than a threshold.

As a result, it is possible to adjust the amount of reflected light from an object present at a distance farther than the reference distance so as to be equal to the amount of light when the object exists at the reference distance.

Also, the correcting unit may reduce the amount of light corresponding to a distance shorter than the reference distance so that the difference from the reference amount of light is equal to or less than a threshold.

As a result, the amount of reflected light from an object present at a distance shorter than the reference distance can be adjusted to be equal to the amount of light when the object is present at the reference distance.

Also, the reference distance may be a distance corresponding to the front seat of the vehicle from the installation position of the object recognition device.

As a result, it is possible to improve the accuracy of recognizing an object (occupant, etc.) present in the interior of the vehicle.

Further, the correction unit may correct the amount of light corresponding to the distance corresponding to the rear seats of the vehicle from the installation position of the object recognition device so that the difference from the reference amount of light corresponding to the front seats is equal to or less than a threshold.

As a result, the amount of reflected light from objects in the rear seats can be adjusted to be equal to the amount of reflected light from objects in the front seats, thereby improving the accuracy of recognizing objects in the rear seats. can be done.

Further, the recognition processing unit performs grouping processing of grouping a plurality of pixels having a distance difference equal to or less than a threshold value and a light amount equal to or greater than a threshold value for an image having pixel values corresponding to the distance and the light amount corrected by the correction unit. and object information may be generated based on the grouping results.

As a result, based on the distance to the object and the amount of light corrected according to the distance, the existence, distance, shape, movement, etc. of the object can be recognized with high accuracy.

Further, an object recognition method as another embodiment of the present invention comprises steps of measuring a distance based on the time it takes for irradiated light to be reflected by an object and then returned; The method includes measuring the amount of light, correcting the amount of light according to the distance, and generating object information about the object based on the distance and the corrected amount of light.

Further, a program as another embodiment of the present invention provides an information processing apparatus with a process of measuring a distance based on the time it takes for irradiated light to be reflected by an object and then to return, and A process of measuring the amount of reflected light, a process of correcting the amount of light according to the distance, and a process of generating object information about the object based on the distance and the corrected amount of light are executed.

1 is a diagram illustrating an example of a hardware configuration of an object recognition device according to an embodiment; FIG. FIG. 2 is a timing chart for explaining an example of the phase data acquisition method in the TOF method. FIG. 3 is a graph for explaining an example of a distance calculation method in the TOF method. 4 is a block diagram illustrating an example of a functional configuration of the object recognition device according to the embodiment; FIG. FIG. 5 is a graph illustrating an example of light amount correction processing by a correction unit according to the embodiment; FIG. 6 is a diagram illustrating an example of an image for recognition processing according to the embodiment; 7 is a flowchart illustrating an example of processing in the object recognition device according to the embodiment; FIG.

Illustrative embodiments of the invention are disclosed below. The configurations of the embodiments shown below and the actions, results, and effects brought about by the configurations are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and at least one of various effects and derivative effects based on the basic configuration can be obtained.

FIG. 1 is a diagram illustrating an example of a hardware configuration of an object recognition device 1 according to an embodiment. The object recognition device 1 generates object information about objects existing around the object recognition device 1 . The object information can include, for example, information indicating the presence or absence of an object, the distance to the object, the shape of the object, the movement of the object, and the like. The object recognition device 1 can be installed in any system or device that can use object information. For example, the object recognition device 1 can be installed in a monitoring system that monitors passengers in a vehicle, a driving support system that recognizes objects existing around the vehicle, automatically runs, avoids danger, and the like.

The object recognition device 1 exemplified here includes an irradiation device 11 , a diffusion optical system 12 , a light receiving device 13 , a condensing optical system 14 and a control device 15 .

The irradiation device 11 irradiates irradiation light 21 which is pulsed light having a predetermined pulse width. Although the specific configuration of the irradiation device 11 should not be particularly limited, it can be configured using, for example, an LED (Light Emitting Diode), a PWM (Pulse Width Modulation) circuit, or the like. The irradiation device 11 operates according to a light emission control signal or the like from the control device 15 . The irradiation light 21 is diffused by the diffusing optical system 12 and irradiated toward an object to be recognized (for example, an occupant in the vehicle, an object around the vehicle, etc.).

The light receiving device 13 is a device that photoelectrically converts light from the outside, and generates phase data (analog data) of reflected light 22 that is reflected from an object. Although the specific configuration of the light receiving device 13 should not be particularly limited, it can be configured using, for example, photodiodes, switching elements, capacitors, and the like. The reflected light 22 is received by the light receiving device 13 while being condensed by the condensing optical system 14 . The light receiving device 13 operates according to a shutter control signal or the like from the control device 15 .

The control device 15 is an information processing device that comprehensively controls the object recognition device 1 and includes a control circuit 16 and an arithmetic circuit 17 .

The control circuit 16 is a circuit that controls the irradiation device 11 and the light receiving device 13 and acquires and processes phase data generated by the light receiving device 13 . The control circuit 16 generates RAW data by digitally converting the phase data acquired from the light receiving device 13 . Although the specific configuration of the control circuit 16 should not be particularly limited, it is configured using an I/O port, an AD converter, an ASIC (Application Specific Integrated Circuit), a memory, various logic circuits, a light amount measuring device, and the like. obtain.

The arithmetic circuit 17 performs various kinds of arithmetic processing for generating object information based on the RAW data generated by the control circuit 16 and the like. The arithmetic circuit 17 determines whether the irradiation device 11 or the light receiving device is based on the TOF until the irradiation light 21 emitted from the irradiation device 11 is reflected by the object and returned as the reflected light 22 (received by the light receiving device 13). The distance from the installation position of 13 to the object is calculated. Although the specific configuration of the arithmetic circuit 17 should not be particularly limited, for example, a CPU (Central Processing Unit) that performs arithmetic processing according to a program, an FPGA (Field Programmable Gate Array), a memory, various logic circuits, etc. can be configured.

2 and 3 show an example of the distance calculation method by the TOF method. FIG. 2 is a timing chart for explaining an example of the phase data acquisition method in the TOF method. FIG. 3 is a graph for explaining an example of a distance calculation method in the TOF method.

FIG. 2 shows a state in which a phase difference φ (a value corresponding to TOF) is generated between the irradiated light 21 and the reflected light 22 . Also, the operation of four types of gates Quad1 to Quad4 that open and close in synchronization with the cycle of the irradiation light 21 is shown. Four types of accumulated charges q1 to q4 corresponding to the amount of reflected light 22 are obtained by the action of each gate Quad1 to Quad4. Then, as shown in FIG. 3, vector information including the distance and direction to the object can be obtained by plotting the obtained accumulated charges q1 to q4 on coordinates consisting of the real axis and the imaginary axis.

FIG. 4 is a block diagram showing an example of the functional configuration of the object recognition device 1 according to the embodiment. The object recognition device 1 includes a distance measurement section 101 , a light intensity measurement section 102 , a correction section 103 and a recognition processing section 104 . These functional components 101 to 104 are configured by cooperation between the hardware configuration and software configuration (programs for controlling the control device 15, etc.) shown in FIG.

The distance measurement unit 101 measures the distance based on the time (TOF) until the irradiated light is reflected by the object and returned. In this embodiment, the distance is measured based on the time from when the irradiation light 21 is emitted from the irradiation device 11 until the reflected light 22 corresponding to the irradiation light 21 is received by the light receiving device 13 .

A light quantity measuring unit 102 measures the light quantity of the reflected light 22 reflected by the object. The amount of light can be a physical quantity represented by, for example, luminous flux (lumens), illuminance (lux), luminous intensity (candela), and the like. The method for measuring the amount of light should not be particularly limited, and for example, it can be performed using a known light amount measuring device or the like.

A correction unit 103 corrects the light amount measured by the light amount measurement unit 102 based on the distance measured by the distance measurement unit 101 . The correction unit 103 corrects the light amount of the reflected light 22 so as to increase as the distance increases. Further, the correction unit 103 may correct the light amount of the reflected light 22 so as to decrease as the distance decreases.

The recognition processing unit 104 generates object information regarding the object based on the distance measured by the distance measurement unit 101 and the corrected light amount, which is the light amount corrected by the correction unit 103 . The recognition processing unit 104 according to this embodiment includes an image generation unit 111 and a grouping unit 112 .

The image generation unit 111 generates an image for recognition processing, in which pixel values are the distance measured by the distance measurement unit 101 and the corrected light amount (the light amount measured by the light amount measurement unit 102 when correction is not necessary). do. The position of each pixel in the image for recognition processing corresponds to the position within the monitoring area.

The grouping unit 112 performs grouping processing of grouping (clustering) a plurality of pixels each having a difference in distance equal to or less than a threshold value and a light amount (corrected light amount) equal to or more than a threshold value for the image for recognition processing. Based on the result of the grouping process, the presence area of each object can be specified, and object information indicating the presence or absence of the object, the distance to the object, the shape of the object, the movement of the object, and the like can be generated.

FIG. 5 is a graph showing an example of light amount correction processing by the correction unit 103 according to the embodiment. The horizontal axis of the graph indicates the distance from the installation position of the object recognition device 1 (irradiating device 11 and light receiving device 13 ), and the vertical axis indicates the amount of reflected light 22 . Here, the object recognition device 1 (the irradiation device 11 and the light receiving device 13) is installed in the front part of the interior of the vehicle (for example, the upper part of the windshield), and the irradiation light 21 is irradiated from the front part to the rear part of the room. Illustrate the case.

FIG. 5 shows that the amount of light before correction decreases exponentially as the distance from the installation position increases. The correction unit 103 corrects the amount of light so that the influence of the decrease in the amount of light that accompanies such an increase in distance is reduced.

The correction unit 103 according to the present embodiment corrects the light amount of the reflected light 22 measured by the light amount measurement unit 102 so that the difference from the reference light amount Q1 corresponding to the reference distance D1 is equal to or less than a threshold. As shown in FIG. 5, the post-correction light intensity is maintained substantially constant so as to be close to the reference light intensity Q1 over the entire distance. The reference distance D1 according to this embodiment is the distance from the installation position of the object recognition device 1 (the irradiation device 11 and/or the light receiving device 13) to the front seats of the vehicle, and the reference light quantity Q1 is the distance from the object existing in the front seats. It is the amount of light corresponding to the amount of reflected light 22 from (passenger, etc.). As a result, the light quantity corresponding to the distance D2 from the installation position of the object recognition device 1 (irradiating device 11 and/or light receiving device 13) to the rear seats is corrected to be equal to the reference light quantity Q1 corresponding to the front seats. .

FIG. 6 is a diagram showing an example of recognition processing images 201A and 201B according to the embodiment. FIG. 6 illustrates an image for recognition processing 201A before correction and an image for recognition processing 201B after correction. In the recognition processing image 201A before correction, the amount of reflected light 22 from the rear seat occupant is insufficient, making it difficult to recognize the rear seat occupant. On the other hand, in the recognition processing image 201B after correction, the amount of reflected light 22 from the rear seat occupant is corrected by the above-described correction processing to increase. clearly recognizable.

FIG. 7 is a flowchart showing an example of processing in the object recognition device 1 according to the embodiment. When the distance measuring unit 101 measures the distance to an object and the light amount measuring unit 102 measures the light amount of the reflected light 22 from the object (S101), the correcting unit 103 calculates the difference between the measured distance D and the reference distance D1. A relationship is determined (S102).

If the measured distance D is greater than the reference distance D1 (S102: D>D1), the correction unit 103 corrects so as to increase the amount of light corresponding to the distance D (S103). At this time, the correction unit 103 performs correction so that the difference between the light amount after correction and the reference light amount Q1 is equal to or less than the threshold. On the other hand, when the measured distance D is smaller than the reference distance D1 (S102: D<D1), the correction unit 103 corrects so as to reduce the amount of light corresponding to the distance D (S104). At this time, the correction unit 103 performs correction so that the difference between the light amount after correction and the reference light amount Q1 is equal to or less than the threshold. In addition, when the difference between the measured distance D and the reference distance D1 is equal to or less than the threshold value, the light quantity corresponding to the distance D may not be corrected.

After that, the image generation unit 111 generates an image for recognition processing 201B in which the measured distance D and the corrected light amount (the original light amount measured by the light amount measurement unit 102 if not corrected) are used as pixel values ( S105). Since the distance D is vector information including information about the direction, each pixel value of the image for recognition processing 201B can specify the position in the recognition area (for example, inside the vehicle).

After that, the grouping unit 112 performs grouping processing on the image for recognition processing 201B (S106), and the recognition processing unit 104 generates object information based on the result of the grouping processing (S107). The grouping process is performed so as to group a plurality of pixels whose difference in distance is equal to or less than a threshold and whose amount of light is equal to or greater than a threshold. Based on the result of the grouping process, the presence area of each object (occupant, etc.) can be identified, and object information indicating the presence or absence of the object, the distance to the object, the shape of the object, the movement of the object, and the like can be generated. The object information can be used, for example, in a monitoring system that monitors the occupants of a vehicle, a driving support system that recognizes objects existing around the vehicle, automatically runs, avoids danger, and the like.

As described above, according to the object recognition device 1 according to the present embodiment, the influence of the attenuation (decrease in light intensity) of the reflected light 22 accompanying an increase in the distance to the object is suppressed, and the recognition accuracy for distant objects is improved. can be improved.

The program for realizing the functions of the above embodiments is a file in a format that can be installed in the control device 15 or in a format that can be executed on a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), or the like. may be recorded and provided on a computer-readable recording medium. Alternatively, the program may be stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Also, the program may be configured to be provided or distributed via a network such as the Internet.

Although the embodiments of the present invention have been exemplified above, the above embodiments and modifications are merely examples, and are not intended to limit the scope of the invention. The above embodiments and modifications can be implemented in various other forms, and various omissions, replacements, combinations, and modifications can be made without departing from the scope of the invention. Also, the configuration and shape of each embodiment and each modification can be partially exchanged.

DESCRIPTION OF SYMBOLS 1... Object recognition apparatus, 11... Irradiation apparatus, 12... Diffusion optical system, 13... Light receiving apparatus, 14... Condensing optical system, 15... Control apparatus, 16... Control circuit, 17... Arithmetic circuit, 21... Irradiation light, 22 Reflected light 101 Distance measurement unit 102 Light amount measurement unit 103 Correction unit 104 Recognition processing unit 111 Image generation unit 112 Grouping unit 201A, 201B Image for recognition processing D1 Reference Distance, Q1...Reference light quantity

Claims (8)

a distance measuring unit that measures the distance based on the time it takes for the irradiated light to return after being reflected by an object;
a light amount measuring unit that measures the amount of reflected light reflected by the object;
a correction unit that corrects the amount of light according to the distance;
a recognition processing unit that generates object information about the object based on the distance and the amount of light corrected by the correction unit;
An object recognition device comprising: The correction unit increases the amount of light corresponding to a distance longer than a predetermined reference distance so that the difference from the reference amount of light corresponding to the reference distance is equal to or less than a threshold.
The object recognition device according to claim 1. The correction unit reduces the amount of light corresponding to a distance closer than the reference distance so that the difference from the reference amount of light is equal to or less than a threshold.
The object recognition device according to claim 2. The reference distance is a distance corresponding to the front seat of the vehicle from the installation position of the object recognition device.
The object recognition device according to claim 2 or 3. The correction unit corrects the amount of light corresponding to the distance corresponding to the rear seats of the vehicle from the installation position of the object recognition device so that the difference from the reference amount of light corresponding to the front seats is equal to or less than a threshold. ,
The object recognition device according to claim 4. The recognition processing unit groups a plurality of pixels in which the distance difference and the light amount corrected by the correction unit are equal to or less than a threshold value and the light amount is equal to or more than the threshold value in an image having pixel values corresponding to the distance and the light amount corrected by the correction unit. performing a grouping process to generate the object information based on the grouping result;
The object recognition device according to any one of claims 1 to 5. measuring the distance based on the time it takes for the emitted light to be reflected back to the object;
measuring the amount of reflected light reflected by the object;
correcting the amount of light according to the distance;
generating object information about the object based on the distance and the corrected amount of light;
An object recognition method comprising: information processing equipment,
a process of measuring the distance based on the time it takes for the irradiated light to be reflected by the object and return;
a process of measuring the amount of reflected light reflected by the object;
a process of correcting the amount of light according to the distance;
a process of generating object information about the object based on the distance and the corrected amount of light;
program to run.

Images