JP2021096656A - Image recognition device and image recognition program - Google Patents

Abstract

To reduce a burden of image recognition processing.SOLUTION: An image recognition device as an example of the present disclosure includes: an image acquisition unit that acquires a captured image indicating an imaging result of a target region classified into a plurality of partial regions; and a recognition processing unit that, on the basis of classification information preset based on a distance image having distance information to a target included in the target region, classifies the captured image acquired by the image acquisition unit into a plurality of partial images corresponding to the plurality of partial regions, and executes image recognition processing for each of the plurality of classified partial images.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to an image recognition device and an image recognition program.

Conventionally, the situation of the imaging target is sensed by image recognition processing using a normal image as two-dimensional information representing the imaging target and a distance image as three-dimensional information representing the distance to the imaging target. The technology to perform is known. In such a conventional technique, image recognition processing for a normal image is executed only for a range extracted based on a distance image.

JP-A-2018-156408

However, in general, the objects to be recognized by the image recognition process are diverse, such as people and objects. Therefore, in the case of comprehensively recognizing various objects in the above-mentioned conventional technology, it is necessary to execute the image recognition process for the range extracted based on the distance image by various methods according to the objects. Because there is, the burden tends to increase.

Therefore, one of the problems of the present disclosure is to provide an image recognition device and an image recognition program capable of further reducing the burden of the image recognition process.

An image recognition device as an example of the present disclosure includes an image acquisition unit that acquires an image that shows an image capture result of a target area divided into a plurality of subregions, and a distance image that has distance information to a target included in the target area. Based on the division information preset based on the above, the captured image acquired by the image acquisition unit is divided into a plurality of partial images corresponding to a plurality of partial regions, and each of the divided plurality of partial images is imaged. It includes a recognition processing unit that executes recognition processing.

According to the image recognition device described above, it is possible to execute an appropriate image recognition process for each partial image, unlike the case where a common image recognition process is executed for the entire captured image so as to cover various objects, for example. .. Therefore, the load of the image recognition process can be further reduced.

In the image recognition device described above, the target area is divided into at least a first partial area corresponding to the seat and a second partial area other than the first partial area as a plurality of partial areas. The image acquisition unit acquires an image showing the imaging result of the space in the vehicle interior as a target area, including the indoor space, and the recognition processing unit is preset based on a distance image having a common viewpoint with the captured image. Based on the division information, the captured image acquired by the image acquisition unit is divided into at least a first partial image corresponding to the first partial region and a second partial image corresponding to the second partial region. It is divided and different image recognition processes are executed for the first partial image and the second partial image. According to such a configuration, it is possible to execute appropriate image recognition processing according to at least the presence or absence of seats or the classification based on the arrangement of seats in the space inside the vehicle interior.

In this case, the first partial image corresponds to at least a third partial image corresponding to the driver's seat and a fourth partial region other than the third partial region. The recognition processing unit executes image recognition processing that is partially different from the third partial image and the fourth partial image, including the fourth partial image. According to such a configuration, it is possible to execute appropriate image recognition processing at least according to the classification according to whether or not the seat corresponds to the driver's seat.

Further, in the image recognition device described above, the target area is divided into at least a fifth partial area close to the door and a sixth partial area other than the fifth partial area as a plurality of partial areas. The image acquisition unit acquires an image showing the imaging result of the space outside the vehicle, and the recognition processing unit uses the division information preset based on the distance image having the same viewpoint as the image. Based on this, the captured image acquired by the image acquisition unit is divided into at least a fifth partial image corresponding to the fifth partial region and a sixth partial image corresponding to the sixth partial region. Different image recognition processes are executed for the fifth partial image and the sixth partial image. According to such a configuration, it is possible to execute an appropriate image recognition process at least according to the degree of proximity to the door in the space outside the vehicle.

Further, in the image recognition device described above, the recognition processing unit excludes a part of the plurality of partial images from the target of the image recognition processing, and executes the image recognition processing only for the remaining partial images. .. According to such a configuration, the load of the image recognition process can be further reduced.

Further, the image recognition device described above further includes a service processing unit that provides a service according to the result of the image recognition processing by the recognition processing unit. According to such a configuration, the function of the image recognition device can be improved by appropriately utilizing the result of the image recognition processing.

Further, in the image recognition device described above, the image acquisition unit includes a two-dimensional image as two-dimensional information representing the target area, a three-dimensional image including a distance image as three-dimensional information representing the target area, and the like. At least one of them is acquired as a captured image. According to such a configuration, the accuracy of sensing of the target region can be improved by the image recognition process using at least one of the two-dimensional image and the three-dimensional image.

An image recognition program as another example of the present disclosure has an image acquisition step of acquiring an image showing an imaging result of a target area divided into a plurality of subregions, and distance information to an object included in the target area. Based on the division information preset based on the distance image, the captured image acquired by the image acquisition step is divided into a plurality of partial images corresponding to a plurality of partial regions, and each of the divided plurality of partial images is divided. It is a program for causing a computer to execute a recognition processing step for executing image recognition processing.

According to the image recognition program described above, it is possible to execute an appropriate image recognition process for each partial image, unlike the case where a common image recognition process is executed for the entire captured image so as to cover various objects, for example. .. Therefore, the load of the image recognition process can be further reduced.

FIG. 1 is an exemplary and schematic block diagram showing a configuration of an image recognition device according to an embodiment. FIG. 2 is an exemplary and schematic diagram showing an example of the arrangement of the distance image cameras according to the embodiment. FIG. 3 is an exemplary and schematic diagram for explaining an example of the division of the target area according to the embodiment. FIG. 4 is an exemplary and schematic diagram for explaining another example different from FIG. 3 in the division of the target area according to the embodiment. FIG. 5 is an exemplary and schematic diagram for explaining another example different from FIGS. 3 and 4 in the division of the target area according to the embodiment. FIG. 6 is an exemplary and schematic diagram for explaining another example different from FIGS. 3 to 5 in the division of the target area according to the embodiment. FIG. 7 is an exemplary and schematic diagram showing an example of a service (function) provided according to the recognition result by the recognition processing unit according to the embodiment. FIG. 8 is an exemplary and schematic flowchart showing the processing executed by the image recognition device according to the embodiment. FIG. 9 is an exemplary and schematic block diagram showing a hardware configuration of an ECU (Electronic Control Unit) for realizing the image recognition device according to the embodiment. FIG. 10 is an exemplary and schematic diagram showing an example of the arrangement of the distance image camera according to the modified example of the embodiment. FIG. 11 is an exemplary and schematic diagram showing an example of the division of the target area according to the modified example of the embodiment.

Hereinafter, embodiments and modifications of the present disclosure will be described with reference to the drawings. The configurations of the embodiments and modifications described below, and the actions and effects brought about by the configurations are merely examples, and are not limited to the contents described below.

<Embodiment>
FIG. 1 is an exemplary and schematic block diagram showing the configuration of the image recognition device 100 according to the embodiment.

As shown in FIG. 1, the image recognition device 100 according to the embodiment is configured to control the actuator 50 in response to an input from the distance image camera 30. The distance image camera 30, the actuator 50, and the image recognition device 100 are mounted on, for example, a vehicle 10 (see FIG. 2).

The range image camera 30 has a function as a two-dimensional camera that acquires a two-dimensional image as two-dimensional information representing an area (space) to be imaged, and three-dimensional information representing an area to be imaged. It is a camera having a function as a three-dimensional camera for acquiring a three-dimensional image of the above. The area to be imaged can also be expressed as a target area.

In the embodiment, the two-dimensional image is, for example, an infrared image, and the three-dimensional image is a distance image having distance information indicating the distance to the target included in the target area. The distance to the target can be calculated by, for example, the TOF (Time Of Flight) method based on the difference in the timing of transmitting and receiving infrared rays.

Therefore, in the embodiment, it can be said that the distance image camera 30 is a camera having a function as an infrared camera and a function as a TOF camera. In the following, the infrared image (two-dimensional image) and the distance image (three-dimensional image) captured by the range image camera 30 may be expressed as captured images when it is not necessary to distinguish them.

Here, in the embodiment, the target area as the area to be imaged by the distance image camera 30 is the space inside the vehicle 20. Therefore, in the embodiment, the distance image camera 30 is arranged in the vehicle 20 as shown in FIG. 2 below, for example.

FIG. 2 is an exemplary and schematic diagram showing an example of the arrangement of the distance image camera 30 according to the embodiment.

As shown in FIG. 2, the distance image camera 30 is arranged in the vehicle interior 10a as an internal space of the vehicle 10. For example, the distance image camera 30 is arranged on the ceiling of the vehicle interior 10a, which is called an overhead console or the like. The arrangement of the distance image camera 30 shown in FIG. 2 is merely an example. The distance image camera 30 may be arranged at a place other than the ceiling portion of the vehicle interior 10a.

Here, in the example shown in FIG. 2, the shaded area corresponds to the field of view of the distance image camera 30. Therefore, the installation position and the installation posture of the distance image camera 30 are adjusted so that the plurality of seats 14 and the center console 15 provided in the vehicle interior 10a can be seen in the field of view.

The seat 14 is provided on the driver's seat 14a at a position facing the steering wheel 12, the passenger seat 14b provided adjacent to the driver's seat 14a across the center console 15, and behind the driver's seat 14a and the passenger seat 14b. The rear seats 14c to 14e, which are provided side by side in the vehicle width direction, are included. The driver's seat 14a and the passenger seat 14b can also be expressed as front row seats, and the rear seats 14c to 14e can also be expressed as rear row seats.

The example shown in FIG. 2 is just an example. Therefore, the technique according to the embodiment can be applied to vehicles other than the five-seater vehicle 10 as shown in FIG. Examples of other vehicles include, for example, a four-seater vehicle with two front row seats and two back row seats, and two first and second row seats, respectively. Examples include a 7-seater vehicle with three rows of seats.

In this way, the distance image camera 30 according to the embodiment can acquire an captured image showing the state of the space in the vehicle interior 10a. Then, according to the image recognition process using the captured image, it is possible to perform sensing of the situation in the vehicle interior 10a (for example, the presence or absence of a person or an object, and the face, physique, or physique of the person).

For example, conventionally, a normal image (two-dimensional image) as two-dimensional information representing an imaging target and a distance image (three-dimensional image) as three-dimensional information representing a distance to an imaging target are used. There is known a technique for performing sensing of the situation of an image target by the image recognition process. In such a conventional technique, image recognition processing for a normal image is executed only for a range extracted based on a distance image.

However, in general, the objects to be recognized by the image recognition process are diverse, such as people and objects. Therefore, in the case of comprehensively recognizing various objects in the above-mentioned conventional technology, it is necessary to execute the image recognition process for the range extracted based on the distance image by various methods according to the objects. Because there is, the burden tends to increase.

Therefore, in the embodiment, the burden of the image recognition process can be further reduced by configuring the image recognition device 100 as follows.

More specifically, returning to FIG. 1, the image recognition device 100 includes an image acquisition unit 110, a recognition processing unit 120, and a service processing unit 130.

The image acquisition unit 110 acquires an captured image showing the imaging result of the target region from the distance image camera 30. As described above, in the embodiment, the target area is the space inside the vehicle 10.

The recognition processing unit 120 executes image recognition processing on the captured image acquired by the image acquisition unit 110.

By the way, as described above, in the embodiment, the installation position and the installation posture of the distance image camera 30 are adjusted so that the plurality of seats 14 and the center console 15 provided in the vehicle interior 10a can be seen in the field of view. Has been done.

Therefore, in the embodiment, the target area as the image capture target area of the distance image camera 30 is divided into a plurality of partial areas such as a partial area corresponding to the sheet 14 and a partial area corresponding to the center console 15. It is possible to handle them separately (virtually).

For example, FIG. 3 is an exemplary and schematic diagram for explaining an example of the division of the target area according to the embodiment.

In the example shown in FIG. 3, the space 300 as the target area is divided into six subspaces 310 to 360. The subspace 310 is a partial area corresponding to the driver's seat 14a, and the subspace 320 is a partial area corresponding to the passenger seat 14b. Further, the subspaces 330 to 350 are subspaces corresponding to the rear seats 14c to 14e, respectively, and the subspace 360 is a subspace corresponding to the center console 15.

As shown in FIG. 3, the subspaces 310 to 350 as the subspaces corresponding to the seat 14 are likely to have the faces of the occupants P1 to P5, respectively, but as the subspaces corresponding to the center console 15. It is unlikely that the faces of occupants P1 to P5 are present in the subspace 360 of. Therefore, it can be said that the subspaces 310 to 350 as the subspaces corresponding to the sheet 14 and the subspaces 360 as the subspaces corresponding to the center console 15 often do not have a common target to be recognized by the image recognition process. ..

Further, when the space 300 is imaged by the distance image camera 30 in front of the front row seats, the occupants P1 and P2 that may exist in the subspaces 310 and 320 as the partial regions corresponding to the front row seats and the back row seats are supported. The occupants P3 to P5, which may exist in the subspaces 330 to 350 as the partial area, are different from the occupants P3 to P5 in that the posture and physique can be easily recognized by the image recognition process according to the difference in distance from the distance image camera 30. I can say. Therefore, although the subspaces 310 to 350 are common in that they are partial regions corresponding to the sheet 14, it can be said that the objects to be recognized by the image recognition process are not necessarily common.

Further, since the occupant P1 that can exist in the subspace 310 as the partial area corresponding to the driver's seat 14a is a driver, the occupant P2 that can exist in the subspaces 320 to 350 as the subspace corresponding to the other seat 14a. It can be said that it is more important than ~ P5. Therefore, the object to be recognized by the image recognition process may be different between the subspace 310 and the subspace 320 to 350 so that the amount of information obtained from the subspace 310 is larger than the amount of information obtained from the subspace 320 to 350. May be desired.

As described above, in the embodiment, it is appropriate to divide the target area as the image capture target area of the distance image camera 30 into a plurality of partial areas and handle them, and to execute the image recognition process by an appropriate method for each partial area. Is.

The classification shown in FIG. 3 is merely an example according to the configuration of the 5-seater vehicle 10 shown in FIG. Therefore, the technique according to the embodiment also assumes the classification as shown in FIGS. 4 to 6 below.

FIG. 4 is an exemplary and schematic diagram for explaining another example different from FIG. 3 in the division of the target area according to the embodiment.

The example shown in FIG. 4 differs from the example shown in FIG. 3 in that it is an example adapted to a four-seater vehicle.

More specifically, in the example shown in FIG. 4, the space 400 as the target area is divided into six subspaces 410 to 460. The subspaces 410 and 420 are subspaces corresponding to the front row seats on which occupants P11 and P12 may be seated, respectively, and the subspaces 430 and 440 are seated on occupants P13 and P14, respectively. This is a partial area corresponding to the back row seats. Further, the subspace 450 is a partial area corresponding to the console provided between the front row seats, and the subspace 460 is a partial area corresponding to the console provided between the rear row seats.

Also in the example shown in FIG. 4, for the same reason as in the example shown in FIG. 3, the target area can be divided into a plurality of subregions and handled, and the image recognition process can be executed by an appropriate method for each subregion. It is reasonable.

FIG. 5 is an exemplary and schematic diagram for explaining another example different from FIGS. 3 and 4 in the division of the target area according to the embodiment.

The example shown in FIG. 5 differs from the example shown in FIGS. 3 and 4 in that it is adapted to a 7-seater vehicle.

More specifically, in the example shown in FIG. 5, the space 500 as the target area is divided into eight subspaces 510 to 580. The subspaces 510 and 520 are subspaces corresponding to the first row of seats on which the occupants P21 and P22 may be seated, respectively, and the subspaces 530 and 540 are seated on the occupants P23 and P24, respectively. It is a partial area corresponding to the second row of seats having sex. Further, the subspaces 550 to 570 are partial areas corresponding to the seats in the third row in which the occupants P25 to P27 may be seated, respectively, and the subspace 580 is provided between the seats in the first row. A subspace corresponding to the console.

However, also in the example shown in FIG. 5, for the same reason as in the examples shown in FIGS. 3 and 4, the target area is divided into a plurality of partial areas and handled, and the image recognition process is performed by an appropriate method for each partial area. Is reasonable to do.

FIG. 6 is an exemplary and schematic diagram for explaining another example different from FIGS. 3 to 5 in the division of the target area according to the embodiment.

The example shown in FIG. 6 is common to the example shown in FIG. 5 in that it is an example adapted to a 7-seater vehicle, but the method of classification and the handling of the divided partial areas are shown in FIG. Different from the example shown in.

More specifically, in the example shown in FIG. 6, the space 600 as the target area is divided into six subspaces 610 to 650. The subspaces 610 and 620 are subspaces corresponding to the first row of seats on which the occupants P21 and P22 may be seated, respectively, and the subspaces 530 and 540 are seated on the occupants P23 and P24, respectively. It is a partial area corresponding to the second row of seats having sex. Further, the subspace 550 is a partial area corresponding to the seats in the third row in which the occupants P25 to P27 may be seated, and the subspace 560 corresponds to the console provided between the seats in the first row. It is a subspace.

Also in the example shown in FIG. 6, for the same reason as in the examples shown in FIGS. 3 to 5, the target area is divided into a plurality of partial areas and handled, and the image recognition process is executed by an appropriate method for each partial area. It is reasonable to do. However, in the example shown in FIG. 6, when the space 600 is imaged from the front of the first row of seats, the subspace 650 with the hatched diagonal lines exists before the subspace 650. It is more difficult to acquire information by image recognition processing than the subspaces 610 to 640 and 660. Therefore, in the example shown in FIG. 6, the shaded subspace 650 may be excluded from the image recognition process.

Based on the above, returning to FIG. 1, in the embodiment, the recognition processing unit 120 divides the captured image acquired by the image acquisition unit 110 into a plurality of partial images corresponding to the above-mentioned plurality of partial regions. The captured image to be classified may be only one of the two-dimensional image and the three-dimensional image, or both. When the division of the captured image into partial images is completed, the recognition processing unit 120 executes image recognition processing on at least a part of the plurality of partial images so as to recognize an appropriate target for each divided partial image. ..

In the embodiment, a method of dividing an captured image corresponding to a target area into a partial image corresponding to a partial area, a method of image recognition processing to be executed in order to recognize an appropriate target in the divided partial image, and the like. Information about the above is stored as classification information 121. As for this information, the relationship between the partial region in the real world and the partial image in the captured image can be specified by the distance image captured under the same conditions as the distance image camera 30, so that the division information 121 is the distance. It is set in advance based on a distance image captured under the same conditions as the image camera 30.

Here, the recognition result by the recognition processing unit 120 can be used to provide a service (function) as shown in FIG. 7 below.

FIG. 7 is an exemplary and schematic block diagram showing an example of a service (function) provided according to the recognition result by the recognition processing unit 120 according to the embodiment.

As shown in FIG. 7, in the embodiment, the area (partial image) corresponding to the seat 14 is subjected to (a1) an image recognition process for detecting the presence / absence of a non-vacant seat / presence / person (for example, luggage). , (A2) Image recognition processing for detecting the physique of the person present, (a3) Image recognition processing for executing personal identification (for example, face recognition), and (a4) Attendance. An image recognition process for detecting a gesture performed by a person and (a5) an image recognition process for detecting the position of a part of the body of a person who is present are executed.

The results of the above processes (a1) to (a5) are used to provide a service for the area (partial area) corresponding to the sheet 14. More specifically, the result of the above process (a1) is a belt reminder for encouraging the wearing of a seatbelt, an emergency call for automatically reporting in an emergency, and an auto door lock for automatically locking the door. , And to provide an auto-seat arrangement for automatically setting the position of the seat 14. Further, the result of the above process (a2) is used to provide an auto sheet adjust for automatically adjusting the sheet 14, and the result of the above process (a3) is an individual who calls a setting according to an individual. Used for playback of settings. Further, the result of the above process (a4) is used for body unit control for controlling the vehicle body such as opening and closing of a window, and the result of the above process (a5) is sandwiched by a door or a roof. It is used to prevent.

Further, in the embodiment, (b1) an image recognition process for detecting the physique of the driver and (b2) an image recognition process for detecting the posture of the driver, particularly for the area (partial image) corresponding to the driver's seat 14a. And (b3) image recognition processing for detecting the body part of the driver, (b4) image recognition processing for detecting the position of the facial parts of the driver, and (b5) the presence or absence of the driver. Image recognition processing for detection, (b6) image recognition processing for detecting the driving posture of the driver, and (b7) detecting whether or not the driver is in the deadman posture as a posture collapsed due to loss of consciousness. Image recognition processing for this is executed.

The results of the above processes (b1) to (b7) are used to provide a service for an area (partial area) corresponding to the driver's seat 14a. More specifically, the results of the above processes (b1) to (b3) are used for airbag control for controlling how to inflate the airbag, and the results of the above processes (b4) are obtained by the driver. It is used for driving support for assisting the driving operation, and the results of the above processes (b5) to (b7) are used for automatic driving for automatically moving the vehicle 10.

Further, in the embodiment, (c1) an image for detecting the physique of a person in an area corresponding to a seat 14 other than the driver's seat 14a, that is, an area (partial image) corresponding to the passenger seat 14b and the rear seats 14c to 14e. Recognition processing, (c2) image recognition processing for detecting the posture of a person, (c3) image recognition processing for detecting a part of the human body, and (c4) for detecting the presence / absence and orientation of a child seat. The image recognition process of (c5) and the image recognition process for detecting whether or not an infant is present in the passenger seat 14b and the rear seats 14c to 14e are executed.

The results of the above processes (c1) to (c5) are used to provide a service for an area (partial area) corresponding to the passenger seat 14b and the rear seats 14c to 14e. More specifically, the results of the above treatments (c1) to (c4) are used for airbag control for controlling how to inflate the airbag, and the results of the above treatment (c5) are for infants. It is used as an infant abandonment alarm to output an alarm in response to the utterance of the abandonment.

Further, in the embodiment, (d1) an image recognition process for detecting a hand gesture by a human hand and (d2) a hand for an area not corresponding to the sheet 14, that is, an area (partial image) corresponding to the center console 15. Image recognition processing for detecting whether the person who executed the gesture is a driver or another person (passenger) is executed.

The results of the above processes (d1) and (d2) are used to provide a service for an area (partial area) corresponding to the center console 15. More specifically, the results of the above processes (d1) and (d2) are used for infotainment control for providing information and entertainment in the passenger compartment 10a.

As described above, in the embodiment, the service processing unit 130 (see FIG. 1) provides the service in the vehicle interior 10a according to the rule as shown in FIG. 7 according to the result of the image recognition processing by the recognition processing unit 120. Can be done.

The example shown in FIG. 7 is just an example. Therefore, in the embodiment, the image recognition process and the provision of the service can be executed according to a rule different from the example shown in FIG.

Based on the above configuration, the image recognition device 100 according to the embodiment executes the processing as shown in FIG. 8 below when providing the service in the vehicle interior 10a.

FIG. 8 is an exemplary and schematic flowchart showing a process executed by the image recognition device 100 according to the embodiment.

As shown in FIG. 8, in the embodiment, first, in S801, the image acquisition unit 110 of the image recognition device 100 acquires an captured image from the distance image camera 30.

Then, in S802, the recognition processing unit 120 of the image recognition device 100 classifies the captured image acquired in S801 into a plurality of partial images based on the preset classification information 121. Then, in S803, the recognition processing unit 120 of the image recognition device 100 executes the image recognition process for each partial image classified in S802. Then, in S804, the service processing unit 130 of the image recognition device 100 provides a service according to the result of the image recognition processing in S803. Then, the process ends.

It should be noted that the example of the service according to the classification of the captured image into the partial image, the image recognition processing for each divided partial image, and the result of the image recognition processing has already been described with reference to FIGS. 3 to 7, and so on. The description is omitted here.

By the way, the image recognition device 100 according to the embodiment is realized by an ECU (Electronic Control Unit) 900 as an in-vehicle control device having a hardware configuration similar to that of a general computer, as shown in FIG. 9, for example. To.

FIG. 9 is an exemplary and schematic block diagram showing a hardware configuration of the ECU 900 for realizing the image recognition device 100 according to the embodiment.

As shown in FIG. 9, the ECU 900 according to the embodiment includes a processor 910, a memory 920, a storage 930, an input / output interface (I / F) 940, and a communication interface (I / F) 950. ing. These hardware are connected to bus 960.

The processor 910 is configured as, for example, a CPU (Central Processing Unit), and controls the operation of each part of the ECU 900 in an integrated manner.

The memory 920 includes, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory), and volatile or non-volatile storage of various data such as a program executed by the processor 910, and the processor 910 executes the program. To provide a work area for the purpose.

The storage 930 includes, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various data in a non-volatile manner.

The input / output interface 940 controls the input of data to the ECU 900 and the output of data from the ECU 900.

The communication interface 950 allows the ECU 900 to perform communication with other devices.

In the embodiment, the functional module group included in the image recognition device 100 shown in FIG. 1 is a collaboration between hardware and software as a result of the processor 910 executing an image recognition program stored in a memory 920, a storage 930, or the like. Is realized by. However, in the embodiment, at least a part of the functional module group included in the image recognition device 100 shown in FIG. 1 may be realized by dedicated hardware (circuit: cycle).

The image recognition program according to the embodiment may be provided in a state of being preliminarily incorporated in a storage device such as a memory 820 or a storage 830, various magnetic disks such as a flexible disk (FD), or a DVD ( It may be provided as a computer program product recorded in a format that can be installed or executed on a computer-readable recording medium, such as various optical discs such as the Digital Versaille Disk.

Further, the image recognition program according to the embodiment may be provided or distributed via a network such as the Internet. That is, the image recognition program according to the embodiment may be provided in the form of being stored on a computer connected to a network such as the Internet and being downloaded from the computer via the network.

As described above, the image recognition device 100 according to the embodiment includes an image acquisition unit 110 and a recognition processing unit 120. The image acquisition unit 110 acquires an image captured from the distance image camera 30 indicating the image capture result of the target region divided into a plurality of partial regions. The recognition processing unit 120 obtains a plurality of captured images acquired by the image acquisition unit 110 based on the division information 121 preset based on the distance image having the distance information indicating the distance to the target included in the target area. It is divided into a plurality of partial images corresponding to the partial areas, and image recognition processing is executed for each of the plurality of divided partial images.

According to the above configuration, it is possible to execute an appropriate image recognition process for each partial image, unlike the case where a common image recognition process is executed for the entire captured image so as to cover various objects, for example. it can. Therefore, the load of the image recognition process can be further reduced.

More specifically, in the embodiment, the target area is at least the first subspace corresponding to the sheet 14 (for example, the subspaces 310 to 350 shown in FIG. 3) and the second subspace other than the first subspace. (For example, the subspace 360 shown in FIG. 3) and the space in the passenger compartment 10a. The image acquisition unit 110 acquires an captured image showing the imaging result of the space in the vehicle interior 10a. Then, the recognition processing unit 120 obtains the captured image acquired by the image acquisition unit 110 based on the division information 121 preset based on the distance image having the same viewpoint as the captured image, at least in the first partial region. It is divided into a first partial image corresponding to the above and a second partial image corresponding to the second partial region, and different image recognition processes are performed for the first partial image and the second partial image. Execute (see, for example, FIG. 7). According to such a configuration, it is possible to execute an appropriate image recognition process according to at least the presence / absence of the seat 14 or the classification based on the arrangement of the seat 14 in the space in the vehicle interior 10a.

Here, in the embodiment, the first partial image is at least a third partial image corresponding to a third partial region corresponding to the driver's seat 14a (for example, the partial space 310 shown in FIG. 3). It includes a fourth sub-image corresponding to a fourth sub-region (for example, subspaces 320-350 shown in FIG. 3) other than the third sub-region. The recognition processing unit 120 executes partially different image recognition processing for the third partial image and the fourth partial image (see, for example, FIG. 7). According to such a configuration, it is possible to execute an appropriate image recognition process according to at least the classification according to whether or not the seat 14 corresponds to the driver's seat 14a.

In the embodiment, as already described with reference to FIG. 6, the recognition processing unit 120 excludes a part of the plurality of partial images from the target of the image recognition processing, and sets the remaining partial images as the remaining partial images. It is also possible to execute the image recognition process only for that. According to such a configuration, the load of the image recognition process can be further reduced.

Further, in the embodiment, the image recognition device 100 further includes a service processing unit 130 that provides a service according to the result of the image recognition processing by the recognition processing unit 120. According to such a configuration, the function of the image recognition device 100, that is, the function of the vehicle 10 can be improved by appropriately utilizing the result of the image recognition process.

Further, in the embodiment, the image acquisition unit 110 includes a two-dimensional image as two-dimensional information representing the target area and a three-dimensional image including a distance image as three-dimensional information representing the target area. At least one is acquired as a captured image from the range image camera 30. According to such a configuration, the accuracy of sensing of the target region can be improved by the image recognition process using at least one of the two-dimensional image and the three-dimensional image.

<Modification example>
In the above-described embodiment, a configuration in which only one distance image camera 30 is provided in the vehicle interior 10a is exemplified. However, the technique of the present disclosure can also be applied to a configuration in which a plurality of distance image cameras 30 are provided. In this case, it is necessary to set a plurality of division information corresponding to each of the plurality of distance image cameras 30 in advance.

Further, in the above-described embodiment, the configuration in which the distance image camera 30 captures the space in the vehicle interior 10a from the front is exemplified. However, the technique of the present disclosure is applied to both the configuration in which the distance image camera 30 images the space in the vehicle interior 10a from the rear and the configuration in which the distance image camera 30 images the space in the vehicle interior 10a from the side. It is possible to do. That is, in the technique of the present disclosure, the installation position and the installation posture of the distance image camera 30 may be adjusted as long as the object to be sensed is imaged.

Further, in the above-described embodiment, the configuration in which the target area as the image pickup target of the distance image camera 30 is the internal area (space) of the vehicle 10 is exemplified. However, as shown in FIGS. 10 and 11 below, as a modification, a configuration in which the target region is an external region (space) of the vehicle 20 is also conceivable.

FIG. 10 is an exemplary and schematic diagram showing an example of the arrangement of the distance image camera 30 according to the modified example of the embodiment.

As shown in FIG. 10, the technique according to the modified example is to realize at least avoiding contact between the doors 21 and 22 of the vehicle 20 and the user X and controlling the entrance and exit of the slope 90 for the wheelchair 70. Technology. Therefore, in the modified example, the distance image camera 30 is installed so as to take an image of the space outside the vehicle 20, more specifically, the space 1000 including the road surface in the vicinity of the doors 21 and 22 of the vehicle 20 as the target area. And the installation posture is adjusted.

In the modified example, the space 1000 as the target area is divided into, for example, the shape shown in FIG. 11 below.

FIG. 11 is an exemplary and schematic diagram showing an example of the division of the target area according to the modified example of the embodiment.

As shown in FIG. 11, in the modified example, the space 1000 as the target region is divided into three regions 111 to 1130 in a plan view.

Region 1110 is the partial region closest to the doors 21 and 22. The partial image corresponding to the region 1110 is the target of image recognition processing for detecting a person and a part of the human body. According to the image recognition process for the partial image corresponding to the area 1110, the doors 21 and 22 can be controlled so as to avoid contact between the doors 21 and 22 and the user X.

Further, the area 1120 is a partial area having an end portion farther from the doors 21 and 22 than the area 1110. The partial image corresponding to the area 1120 is subject to image recognition processing for detecting people and objects that hinder the insertion and removal of the slope 90. According to the image recognition process for the partial image corresponding to the region 1120, it is possible to execute the control to smoothly move the slope 90 in and out.

Further, the area 1130 is a partial area having an end portion further away from the doors 21 and 22 than the area 1120. The partial image corresponding to the region 1130 is subject to image recognition processing for detecting the wheelchair 70. According to the image recognition process for the partial image corresponding to the region 1130, it is possible to start the control of the slope 90 triggered by the detection of the wheelchair 70, for example.

In the modified example, the basic configuration is the same as that of the above-described embodiment except that the target area is the space outside the vehicle 20. Therefore, the configuration according to the modified example is also appropriate for each partial image corresponding to the partial region by the image acquisition unit 110, the recognition processing unit 120, the division information 121 appropriately set in advance, and the like, as in the above-described embodiment. Image recognition processing can be executed to further reduce the load of image recognition processing.

Thus, in the modified example, the target area is at least a fifth subregion (see, for example, region 1110) close to the doors 21 and 22 and a sixth subregion (eg, region) other than the fifth subregion. 1120 and 1130) and the space outside the vehicle 20. The image acquisition unit 110 acquires an captured image showing the imaging result of the space outside the vehicle. The recognition processing unit 120 corresponds to at least the fifth partial region of the captured image acquired by the image acquisition unit 110 based on the division information 121 preset based on the distance image having the same viewpoint as the captured image. It is divided into a fifth partial image and a sixth partial image corresponding to the sixth partial region, and different image recognition processes are executed for the fifth partial image and the sixth partial image. .. According to such a configuration, appropriate image recognition processing can be executed according to the degree of proximity to the doors 21 and 22 at least in the space outside the vehicle 20.

Needless to say, in the above-described modification, a plurality of distance image cameras 30 may be provided. Further, it goes without saying that in the above-described modification, the installation position and the installation posture of the distance image camera 30 may be arbitrarily adjusted according to the object of sensing outside the vehicle 20.

Although the embodiments and modifications of the present disclosure have been described above, the above-described embodiments and modifications are merely examples, and the scope of the invention is not intended to be limited. The novel embodiments and modifications described above can be implemented in various forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The above-described embodiments and modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and the equivalent scope thereof.

10, 20 Vehicles 10a Car room 14 Seats 14a Driver's seat (seat)
14b Passenger seat (seat)
14c-14e Rear seat (seat)
21, 22 Door 100 Image recognition device 110 Image acquisition unit 120 Recognition processing unit 121 Classification information 130 Service processing unit

Claims (8)

An image acquisition unit that acquires an captured image showing the imaging results of a target region divided into a plurality of subregions, and an image acquisition unit.
A plurality of captured images acquired by the image acquisition unit corresponding to the plurality of partial regions based on the division information preset based on the distance image having the distance information to the target included in the target region. A recognition processing unit that divides into partial images and executes image recognition processing for each of the plurality of divided partial images.
An image recognition device. The target area includes, as the plurality of partial areas, at least a space in the vehicle interior which is divided into a first partial area corresponding to the seat and a second partial area other than the first partial area. ,
The image acquisition unit acquires the captured image showing the imaging result of the space in the vehicle interior as the target area.
The recognition processing unit obtains at least the first image obtained by the image acquisition unit based on the classification information preset based on the distance image having the same viewpoint as the captured image. The first partial image corresponding to the partial region and the second partial image corresponding to the second partial region are divided into the first partial image and the second partial image, respectively. Perform different image recognition processes,
The image recognition device according to claim 1. The first partial image is at least a third partial image corresponding to a third partial region corresponding to the driver's seat and a fourth portion corresponding to a fourth partial region other than the third partial region. Including images,
The recognition processing unit executes the image recognition process that is partially different from the third partial image and the fourth partial image.
The image recognition device according to claim 2. The target area includes a space outside the vehicle, which is divided into at least a fifth partial area close to the door and a sixth partial area other than the fifth partial area as the plurality of partial areas.
The image acquisition unit acquires the captured image showing the imaging result of the space outside the vehicle, and obtains the captured image.
The recognition processing unit uses at least the fifth image acquired by the image acquisition unit based on the classification information preset based on the distance image having the same viewpoint as the captured image. The fifth partial image corresponding to the partial region and the sixth partial image corresponding to the sixth partial region are divided into the fifth partial image and the sixth partial image, respectively. Perform different image recognition processes,
The image recognition device according to claim 1. The recognition processing unit excludes a part of the plurality of partial images from the target of the image recognition processing, and executes the image recognition processing only on the remaining partial images.
The image recognition device according to any one of claims 1 to 4. A service processing unit that provides a service according to the result of the image recognition processing by the recognition processing unit is further provided.
The image recognition device according to any one of claims 1 to 5. The image acquisition unit captures at least one of a two-dimensional image as two-dimensional information representing the target area and a three-dimensional image including the distance image as three-dimensional information representing the target area. , Acquired as the captured image,
The image recognition device according to any one of claims 1 to 6. An image acquisition step of acquiring an image that shows an image capture result of a target area divided into a plurality of subregions, and an image acquisition step.
A plurality of captured images acquired by the image acquisition step corresponding to the plurality of partial regions based on the division information preset based on the distance image having the distance information to the target included in the target region. A recognition processing step of dividing into partial images and executing an image recognition process for each of the plurality of divided partial images.
An image recognition program that lets your computer run.

Images