JP2015210584A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can suppress a processing load related to image processing irrespective of the situation of a vehicle.SOLUTION: An image processing apparatus performs image processing to a photographed image obtained by photographing the peripheral environment of a vehicle 10. The image processing apparatus includes an area selection part 320 that selects one or a plurality of object areas to be the object of the image processing from a part of the photographed image according to the situation of the vehicle, and when the area selection part 320 selects a plurality of object areas, switches the object area to be the object of the image processing from among the plurality of object areas and sequentially performs the image processing to the object area switched.

Description

  The present invention relates to an image processing apparatus that performs image processing on a captured image captured by a camera mounted on a moving body such as a vehicle.

  2. Description of the Related Art Conventionally, there is known a driving support device that includes the above-described image processing device and provides driving support based on an image of a surrounding environment of a vehicle processed by the image processing device. Such a driving support device specifies other vehicles, obstacles, pedestrians, and the like ahead of the vehicle by performing image processing on the captured image of the traveling environment in front of the vehicle imaged through the in-vehicle camera. Information on other vehicles, obstacles, pedestrians, etc. is provided as driving support information by voice or image. For example, Patent Document 1 describes an example of an apparatus that can perform such image processing.

  The device described in Patent Literature 1 is a stereo camera device, and includes a plurality of imaging units, an image correction unit that corrects a captured image, a parallax calculation unit that calculates a parallax that is a shift amount between left and right pixels, An image recognition unit that performs image recognition processing using both or one of the captured image and the calculated parallax. In addition, this apparatus includes a processing area determining unit that determines an image area to be processed and a reduction ratio of the image so as to differ depending on a traveling environment of the vehicle on which the stereo camera device is mounted, and is determined by the processing area determining unit. The image correction unit corrects the image using the processing region including the image region and the reduction ratio, the parallax calculation unit calculates the parallax, and the image recognition unit performs image recognition processing.

JP 2009-146217 A

  By the way, image processing related to vehicle driving assistance needs to reduce the processing load so that there is no delay in the processing. For example, in the apparatus described in Patent Document 1, the processing load is suppressed by performing the image recognition process based on the image area and the reduction rate determined according to the traveling environment. On the other hand, when the driving environment or the like is not specified, the image area and the reduction ratio cannot be determined, and thus the processing load may not be appropriately suppressed.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing apparatus capable of suppressing a processing load related to image processing regardless of a vehicle situation.

Hereinafter, means for solving the above-described problems and the effects thereof will be described.
An image processing apparatus that solves the above-described problem is an image processing apparatus that performs image processing on a captured image obtained by capturing an image of a surrounding environment of a vehicle. When the selection unit selects a plurality of target areas, the target area to be subjected to image processing is switched among the plurality of target areas, and an image corresponding to the switched target area is selected. The gist is to sequentially execute the processing.

  According to such a configuration, a part of the captured image is selected as a target area according to the vehicle situation including the driving environment and the driving state of the vehicle, and the captured image is processed by performing image processing on the selected target area. The processing load can be reduced as compared with image processing of the whole. In addition, by switching one of the selected target areas as a target area and sequentially performing image processing, the processing load of image processing is reduced regardless of the vehicle situation while maintaining the amount of information obtained by image processing. Can be suppressed.

As a preferred configuration, the selection unit selects a plurality of target areas when the vehicle situation is an approach to an intersection.
According to such a configuration, it is difficult to narrow down the recognition target because there are various situations at the intersection, so by widening the recognition target by selecting a plurality of target areas, Image recognition accuracy can be maintained while performing image processing with a reduced processing load. Thereby, for example, preventive safety and sign guidance can be performed.

The block diagram which shows the schematic structure about one Embodiment which actualized the image processing apparatus. 6 is a flowchart showing a procedure for specifying a target area for image processing in the image processing apparatus. The schematic diagram which shows typically an example of the captured image imaged with the camera in the image processing apparatus. The schematic diagram which shows typically the candidate area | region narrowed down based on the vehicle speed from the captured image in the image processing apparatus. The schematic diagram which shows typically the preceding vehicle area narrowed down based on the recognition object being a preceding vehicle from the captured image in the image processing apparatus. The schematic diagram which shows typically the preceding vehicle area narrowed down based on the recognition object being a pedestrian from the captured image in the image processing apparatus. The schematic diagram which shows typically the label | marker area | region narrowed down based on the recognition target being a label | marker from the captured image in the image processing apparatus.

An embodiment embodying an image processing apparatus will be described with reference to FIGS.
First, an outline of the present embodiment will be described.
FIG. 1 shows a vehicle 10 including a specific image processing apparatus. The vehicle 10 performs an image recognition process on the front captured image captured by the in-vehicle camera 20 by the image processing unit 30 to detect a preceding vehicle, a pedestrian, a sign, and the like existing in front of the vehicle 10. The vehicle 10 outputs a preceding vehicle, a pedestrian, a sign, and the like detected by image recognition, and can be used as driving support information in the navigation system 70 or the like.

  As shown in FIG. 2, in the present embodiment, the recognition accuracy for a recognition target suitable for the vehicle situation is set by setting the region targeted for image recognition as a region defined as a part of the captured image according to the vehicle situation. The processing load related to image processing is suppressed while maintaining the above. Whether the vehicle status is an expressway (step S13), whether it is a road that passes frequently (step S15), whether it is near the destination (step S17), whether it is in front of the intersection This is determined by (Step S18).

  First, as shown in FIG. 3, the in-vehicle camera 20 captures a front image and obtains a captured image 100. For example, the captured image 100 includes an intersection 3 where the traveling road 1 and the intersection road 2 intersect. In addition, there are pedestrians 120 and 121 on the sidewalk, preceding vehicles 11a and 11b, and an oncoming vehicle 12, and traffic lights 110 and 111 and a pedestrian crossing 125 are installed at the intersection 3. A sign 115 is installed on the road side of the traveling road 1. The captured image 100 from the traveling vehicle 10 has the traveling lane at the center of the image. In FIGS. 3 to 7, for convenience of explanation, the center of the traveling road 1 will be described with reference to the center of the image. ing.

Next, as shown in FIG. 4, an exclusion area 201 to be excluded from image recognition is determined based on the vehicle speed, and a candidate area 200 narrowed down based on the vehicle speed is obtained.
Then, further narrowing down according to the vehicle situation.

  For example, as shown in FIG. 5, in the case of an expressway, the upper and left areas of the candidate area 200 are further excluded areas 231, and the area that is the target of image recognition is excluded from the candidate area 200 by the excluded area 231. The vehicle area 230 is narrowed down.

  For example, as shown in FIG. 6, in the case of a road that frequently passes or near the destination, the upper area of the candidate area 200 is set as a further upper excluded area 211, and the area to be subjected to image recognition is changed from the candidate area 200 to the upper excluded area. The pedestrian area 210 on the lower front side excluding 211 is narrowed down.

  For example, as shown in FIG. 7, when it is before an intersection, the lower area of the candidate area 200 is set as a further lower excluded area 221, and the area to be subjected to image recognition is excluded from the candidate area 200 from the lower excluded area 221. The sign area 220 on the upper front side is narrowed down.

  Then, the vehicle 10 executes image recognition processing on at least one of the candidate area 200, the preceding vehicle area 230, the pedestrian area 210, and the sign area 220 thus narrowed down, and the recognition result obtained by this execution is obtained. Provide driving assistance based on.

Next, this embodiment will be described in detail.
A vehicle 10 shown in FIG. 1 is a car such as a passenger car, a taxi, a bus, and a truck. The vehicle 10 is provided with driving assistance according to vehicle conditions such as a driving environment and a driving state. For example, in the vehicle 10, a preceding vehicle, a pedestrian, a sign, an obstacle, and the like existing in front of the vehicle are detected, and driving assistance based on these detection results is output through voice and images. Such detection results may be provided to a collision prevention device for preventive safety. For example, the collision prevention device can perform driving assistance such as brake assistance and automatic braking based on the detection result provided.

  The vehicle 10 includes a vehicle-mounted camera 20 that captures a surrounding environment such as a traveling environment in front of the vehicle, an image processing unit 30 that receives a captured image captured by the vehicle-mounted camera 20, and a speed sensor 50 that measures the speed of the vehicle 10. And a vehicle speed acquisition unit 40 that acquires the speed of the vehicle 10 as a vehicle speed based on a signal input from the speed sensor 50. In addition, the vehicle 10 includes a GPS (global positioning system) device 60 that identifies the position of the current location, and a navigation system 70 that identifies the travel route from the current location to the destination and guides the identified travel route. Yes.

  Each of the image processing unit 30, the vehicle speed acquisition unit 40, the GPS device 60, and the navigation system 70 is a device capable of information processing, such as an ECU (electronic control device). Therefore, each of these devices includes a computer that includes a CPU (arithmetic unit), a ROM, a RAM, and other storage devices and is configured to be capable of information processing. Note that each of the in-vehicle camera 20 and the speed sensor 50 may be configured in a manner that enables information processing. Further, the image processing unit 30, the vehicle speed acquisition unit 40, and the navigation system 70 are communicably connected via the in-vehicle network N1, and are configured to be capable of information communication.

  The GPS device 60 receives a signal from a GPS satellite and detects the position of the current location of the vehicle 10 based on the received signal from the GPS satellite. More specifically, the GPS device 60 calculates current position information through calculation processing of signals from received GPS satellites, and outputs the calculated position information to the navigation system 70 as the detected current position.

  The navigation system 70 specifies a travel route from the current position to the destination, and outputs information related to driving support based on the specified travel route. The navigation system 70 includes road information and the like, and includes map information used for route search and route guidance display, a route search function, and the like. In the navigation system 70, the destination is set by the driver, the current location is input from the GPS device 60, and the travel route is specified by searching map information based on the destination and the current location. Information output from the navigation system 70 is provided to a driver or the like.

  In addition, the navigation system 70 includes a route guidance unit 710 that provides route guidance to a driver and the like, and a history management unit 720 that manages a travel history of the vehicle 10. The route guidance unit 710 guides the position of the current location, information on the current location, the route to the destination, guidance on the branch road, and the like based on the identified travel route by image display or voice. The route guidance unit 710 acquires information necessary for route guidance from the map information and the travel history. The history management unit 720 acquires the travel history of the vehicle 10 and accumulates and manages the acquired travel history. The travel history includes information such as the travel position, time, and vehicle speed according to the travel of the vehicle 10. Further, the history management unit 720 can select history information corresponding to the request from the route guidance unit 710 and provide it to the request source.

  Further, the navigation system 70 receives inquiries about the driving environment and the driving frequency from the image processing unit 30 and answers these inquiries. The traveling environment includes a highway or a general road, a distance to an intersection, and the like, and these pieces of information can be acquired from the road information. The travel frequency can be acquired from the history information of the road corresponding to the position of the current location, and the history management unit 720 acquires it based on the location of the current location.

  The navigation system 70 includes an input unit such as an operation button and an output unit such as a display screen and a speaker. The input unit can be used for setting a destination. Depending on the execution of the route guidance, a map of a range including the current location is displayed on the display screen, or all or part of the roads constituting the route from the current location to the destination are displayed. Further, the navigation system 70 highlights a part of the image to be displayed, displays an image in front of the vehicle on the display screen, highlights a pedestrian or a sign that requires attention included in the display, You can also guide.

The speed sensor 50 detects the rotational speed of the wheels of the vehicle 10 and outputs a signal corresponding to the detected rotational speed to the vehicle speed acquisition unit 40.
The vehicle speed acquisition unit 40 receives a signal corresponding to the rotational speed from the speed sensor 50, calculates the current speed of the vehicle 10, that is, the vehicle speed based on the input signal, and outputs the calculated vehicle speed. . Accordingly, the image processing unit 30 and the navigation system 70 can acquire the vehicle speed from the vehicle speed acquisition unit 40.

  The in-vehicle camera 20 is a camera such as a CCD camera, for example, and captures at least an image in the visible light region. In the present embodiment, the in-vehicle camera 20 is installed, for example, on the back side of the rearview mirror in the vehicle, and images the traveling environment in front of the vehicle at a predetermined cycle or a predetermined interval, and the captured image 100 is captured. Output to the processing unit 30.

  The image processing unit 30 receives the captured image 100 from the in-vehicle camera 20 and performs image processing such as image recognition processing on the input captured image 100. For example, the preceding vehicles 11a and 11b, the pedestrians 120 and 121, the traffic lights 110 and 111, the signs 115, and the like are detected from the captured image 100 by the image recognition process.

  In the present embodiment, the image processing unit 30 selects a part of the input captured image as a target area to be processed in the image recognition process according to the vehicle situation, and performs image processing on the selected target area. I do. More specifically, the image processing unit 30 includes a region setting unit 310 that sets a target region corresponding to a vehicle situation with respect to a captured image, and a region selection that selects a region for executing image recognition processing from the set target region. Unit 320 and an image recognition unit 330 that performs image recognition processing on the captured image or the selected target region.

  The area setting unit 310 sets a narrowing condition for narrowing the captured image 100 to a partial area according to the vehicle situation, and sets the range of the area corresponding to the condition in the captured image 100. That is, the area setting unit 310 is set with a vehicle speed condition, a preceding vehicle condition, a pedestrian condition, and a sign condition as narrowing conditions. Then, the region setting unit 310 sets the range of the candidate region 200 based on the vehicle speed condition as the partial region of the narrowed captured image 100, sets the range of the preceding vehicle region 230 based on the preceding vehicle condition, The range of the pedestrian area 210 is set based on the pedestrian condition, and the range of the sign area 220 is set based on the sign condition. The area setting unit 310 outputs a captured image in which the range of each area is set to the image recognition unit 330.

Here, each region will be described in detail.
As shown in FIG. 4, the candidate area 200 is an area that excludes an excluded area 201 that is meaningless even when image recognition processing is performed in consideration of the vehicle speed. The exclusion area 201 is an area that has passed at the end of the process due to the vehicle traveling during the time required for the image recognition process, or an area that has just passed. Therefore, the exclusion area 201 becomes narrower when the vehicle speed is slow, and conversely becomes wider when the vehicle speed is faster. Therefore, the candidate area 200 becomes wider when the vehicle speed is slow, and conversely becomes narrow when the vehicle speed is fast.

  As shown in FIG. 5, the preceding vehicle region 230 is a region suitable for recognition of the preceding vehicles 11a and 11b, and is a region set to be narrowed down to the front of the vehicle where there is a high possibility that the preceding vehicle exists. That is, the exclusion areas 231 of the upper and left and right portions, which are unlikely to have a preceding vehicle, are excluded.

  As shown in FIG. 6, the pedestrian area 210 is an area suitable for recognition of the pedestrians 120 and 121 and the pedestrian crossing 125, and there is a high possibility that the pedestrians 120 and 121 and the pedestrian crossing 125 requiring attention are present. This is an area set to be narrowed down to the vicinity of the nearby ground. That is, the upper exclusion area 211 that is far away or above where the pedestrians 120 and 121 requiring attention are low is excluded.

  As shown in FIG. 7, the sign area 220 is an area suitable for recognition of the sign 115 and the traffic lights 110 and 111, and is narrowed down to a position higher than the ground where the sign 115 and the traffic lights 110 and 111 are likely to exist. It is an area set as follows. That is, the lower exclusion area 221 that is near the ground where the signs 115 and the traffic lights 110 and 111 are unlikely to exist is excluded. In the present embodiment, the pedestrian area 210 and the sign area 220 are in an exclusive relationship. However, a part of the pedestrian area 210 and the sign area 220 may overlap or a part that is not included in both may be generated.

  The area selection unit 320 determines a recognition target according to the vehicle situation, determines a target area suitable for the vehicle situation as an area to be selected, and outputs information on the determined target area to be selected to the image recognition unit 330.

  The area selection unit 320 acquires information for determining the vehicle situation from the navigation system 70 or the vehicle speed acquisition unit 40. For example, the area setting unit 310 may store information such as whether the road being traveled is an expressway or a general road, a road with a high frequency of travel or a road with a low frequency, whether it is near or far from the destination, whether it is in front of an intersection, and the like. Discriminable information is acquired from the navigation system 70. In the present embodiment, the position before the intersection is a position where the destination guidance sign of the intersection and the regulation guidance sign for the intersection are recognized, and the pedestrian, other vehicle, and the attention level to the sign are still required. It is a position in front. Further, the area selection unit 320 acquires information such as whether the vehicle is congested or whether the vehicle is traveling at high speed based on the vehicle speed acquired from the vehicle speed acquisition unit 40.

The area selection unit 320 selects a target area corresponding to the vehicle situation.
First, the region selection unit 320 selects a region to be narrowed down by the vehicle speed according to the vehicle speed. And a part of candidate area | region 200 narrowed down according to this vehicle speed is selected as an object area | region corresponding to a vehicle condition.

  Subsequently, the area selection unit 320 selects an area to be narrowed down according to the vehicle situation based on information from the navigation system 70. For example, the region selection unit 320 selects the preceding vehicle region 230 as the target region when the vehicle state is traveling on an expressway. In addition, the region selection unit 320 selects the pedestrian region 210 as the target region when the vehicle situation is a road with a high frequency of travel or when it is close to the destination. Furthermore, the area selection unit 320 selects the sign area 220 as the target area when the vehicle situation is before the intersection. In addition, the region selection unit 320 selects the preceding vehicle region 230, the pedestrian region 210, and the sign region 220 as target regions when the vehicle situation does not correspond to any of the above-described conditions. In addition, when the vehicle situation does not correspond to any of the above-described conditions, it is obvious that the vehicle condition does not correspond to the above-described condition, and of course, the case where it cannot be specified that the above-described condition is satisfied.

  As described above, when the area selection unit 320 narrows down the target area to the preceding vehicle area 230, the pedestrian area 210, and the sign area 220, the processing load required for the image recognition processing for these areas can be suppressed to a low level. In other words, the processing load required for image recognition can be reduced by narrowing the region, and the processing load required for image recognition can also be reduced by narrowing down the recognition target. The recognition target recognized with high accuracy is a preceding vehicle when the preceding vehicle area 230 is a pedestrian when the pedestrian area 210, and a sign when the sign area 220. Therefore, for example, when the recognition processing for these three regions is combined and the recognition processing is performed sequentially, the processing load is reduced compared to image recognition when the regions are not narrowed and the recognition target is not narrowed. It becomes like this.

  The image recognition unit 330 receives the captured image 100 from the in-vehicle camera 20 and performs image recognition on the input captured image 100. The image recognition unit 330 adopts a well-known image recognition algorithm, and applies the algorithm to the input captured image 100 to extract a preceding vehicle, a pedestrian, a sign, and the like included in the captured image 100. Recognize so-called. Note that the image recognition unit 330 can also recognize various obstacles in addition to the preceding vehicle, pedestrians, and signs.

  More specifically, the image recognizing unit 330 receives the captured image 100 in which the range of various regions is set from the region setting unit 310 and the information on the region to be selected as a recognition process target from the region selecting unit 320. The Then, the image recognition unit 330 extracts the target region to be selected determined by the region selection unit 320 from the captured image 100 input from the region setting unit 310, and the recognition accuracy corresponding to the extracted target region is maintained high. Determine the recognition target. For example, the image recognition unit 330 maintains high vehicle recognition accuracy for the preceding vehicle region 230, maintains high pedestrian recognition accuracy for the pedestrian region 210, and Maintain high recognition accuracy of signs. In the image recognition unit 330, recognition processing for recognizing a recognition target that is appropriate to be recognized in the target region with high accuracy is performed on the target region whose range is appropriately narrowed down. That is, the image recognition unit 330 suppresses an increase in the processing load required for the image recognition process by executing the image recognition process so as to recognize the recognition target appropriately narrowed down to a properly narrowed range, or To be able to mitigate.

  For example, the image recognition unit 330 has a low possibility that pedestrians and signs are included in the preceding vehicle area 230 in a state that is useful for driving support. ing. Thereby, while maintaining the recognition accuracy of the preceding vehicle at a high level, the processing load required for the recognition processing of the preceding vehicle region 230 can be suppressed, the processing time can be suppressed, and the required processing performance can be suppressed.

  In addition, since the image recognition unit 330 is unlikely to include a preceding vehicle or a sign in a state useful for driving support in the pedestrian area 210, the recognition accuracy of the preceding vehicle or the sign is suppressed to a minimum necessary accuracy. ing. Thus, while maintaining high pedestrian recognition accuracy, the processing load required for the recognition processing of the pedestrian area 210 can be suppressed, the processing time can be suppressed, and the required processing performance can be suppressed.

  Furthermore, since the image recognition unit 330 is unlikely to be included in the sign area 220 in a state where the preceding vehicle or pedestrian is useful for driving support, the recognition accuracy of the preceding vehicle or pedestrian is reduced to the minimum necessary accuracy. It is suppressed. As a result, the processing load required for the recognition processing of the sign area 220 can be suppressed while maintaining high recognition accuracy of the signs and traffic lights, and the processing time and required processing performance can be suppressed.

Therefore, the image recognition unit 330 can perform image processing in a short time, so-called real time, required for driving support of the vehicle 10.
Note that the image recognition unit 330 according to the present embodiment performs image recognition processing by switching the target region and the recognition target when receiving an instruction to switch the target region from the region selection unit 320, and thus images of a plurality of target regions. Recognition processing can be performed. Thereby, even if the image recognition unit 330 recognizes the target region, the image recognition unit 330 can secure a wide region as a target of image recognition by switching the target region, and recognizes a plurality of recognition targets with high recognition accuracy. Will be able to.

  For example, when the vehicle situation is a situation where the vehicle travels through the intersection 3, the image recognition unit 330 may sequentially recognize the image by switching the preceding vehicle region 230, the pedestrian region 210, and the sign region 220 from the region selection unit 320. Instructed. At the intersection 3, since it is necessary to recognize all of the preceding vehicle, the pedestrian, and the sign with high accuracy, it is preferable to perform image recognition by switching each region. In the case of recognition by switching, there is a possibility that the interval at which the recognition object becomes the recognition object may become long, but because the processing time required for each recognition process is suppressed, the vehicle speed at the intersection 3 is low, etc. It is also possible to execute recognition of each recognition target within the time required for driving support. In addition, it is possible to relatively increase the number of times of recognition of highly important areas.

  Usually, in the image recognition processing, the recognition accuracy increases as the resolution of the captured image increases, and the recognition accuracy tends to increase as the recognition processing algorithm becomes complicated. On the other hand, in the image recognition process, the processing load increases as the resolution increases, and the processing load also increases according to the complexity of the algorithm. That is, it is known that the processing load increases when the recognition accuracy by the image recognition process is improved, and the processing time increases as the processing load increases. At this time, if an attempt is made to suppress an increase in processing time, an information processing apparatus having a high processing capability is required. Although it is possible to suppress an increase in processing time by narrowing the recognition target range while increasing the recognition accuracy, it becomes difficult to acquire information from outside the recognition range by narrowing the recognition target range. End up.

  Next, the operation of this embodiment will be described with reference to FIG. The image recognition processing by the image processing unit 30 is executed in response to the in-vehicle camera 20 capturing a new image at a constant period. Note that this process may be executed at a longer interval than when the in-vehicle camera 20 captures a new image.

  When the captured image is input from the in-vehicle camera 20, the image processing unit 30 sets the candidate area 200 narrowed down based on the vehicle speed with respect to the captured image (step S10). That is, the recognition range is narrowed based on the vehicle speed. As a result, image processing for the excluded area 201 out of the candidate area 200 in the captured image is omitted. The image processing unit 30 determines whether the candidate area 200 can be narrowed down to a predetermined size (step S11). The candidate area 200 does not need to be further narrowed down when the processing load due to the recognition process is equal to or lower than a predetermined load. On the contrary, the candidate area 200 needs to be further narrowed down when the processing load by the recognition process is larger than a predetermined load. That is, the predetermined size is a size determined according to the processing load. The predetermined size of the area is reduced by the reverse, for example, in response to an increase in resolution or a complicated processing algorithm.

  When it is determined that the image has been narrowed down to a predetermined size (YES in step S11), the image processing unit 30 performs normal recognition processing and notifies the recognized recognition target to the navigation system 70 or the like (step S12). Then, the notified navigation system 70 displays the notification content as an image or text, or outputs it through sound. Further, the image may be displayed after being processed so as to emphasize the recognition target in the captured image. Then, the image recognition process is terminated.

  When it is determined that the predetermined size has not been narrowed down (NO in step S11), the image processing unit 30 narrows down the area to be recognized according to the vehicle situation. First, the image processing unit 30 determines whether the road is an expressway (step S13). Whether the road is an expressway can be determined based on the map information of the navigation system 70. If it is determined that the vehicle is a highway (YES in step S13), the image processing unit 30 selects the preceding vehicle region 230 from the captured image and performs image recognition processing while maintaining high recognition accuracy for the vehicle (step). S14). As a result, the preceding vehicle is recognized with high recognition accuracy while suppressing the processing load. And the preceding vehicle recognized in this way is notified as a recognition target (step S12), and an image recognition process is complete | finished after that.

  On the other hand, if it is determined that the road is not a highway (NO in step S13), the image processing unit 30 determines whether the road is a frequent path (step S15). When it is determined that the path is a path that passes frequently (YES in step S15), the image processing unit 30 selects the pedestrian area 210 from the captured image, and performs image recognition processing while maintaining high recognition accuracy of the pedestrian. (Step S16). One reason for selecting the pedestrian area 210 when it is a path that passes frequently is a reduction in attention due to habituation.

  If it is determined that the road is not a frequent path (NO in step S15), the image processing unit 30 determines whether or not it is near the destination (step S17). When it is determined that the vehicle is near the destination (YES in step S17), the image processing unit 30 selects the pedestrian area 210 from the captured image and performs image recognition processing while maintaining high pedestrian recognition accuracy. (Step S16). One reason for selecting the pedestrian area 210 when it is in the vicinity of the destination is a reduction in attention due to concentration on destination discovery.

  By executing step S16 in this way, the pedestrian is recognized with high recognition accuracy while suppressing the processing load. And the pedestrian recognized in this way is notified as a recognition target (step S12), and an image recognition process is complete | finished after that.

  Furthermore, when it is determined that the vehicle is not near the destination (NO in step S17), the image processing unit 30 determines whether the vehicle is in front of the intersection (step S18). When it is determined that the vehicle is in front of the intersection (YES in step S18), the image processing unit 30 selects the sign area 220 from the captured image and performs image recognition processing while maintaining high sign recognition accuracy (step). S19). As a result, the sign is recognized with high recognition accuracy while suppressing the processing load. Then, the sign recognized in this way is notified as a recognition target (step S12), and then the image recognition process is terminated.

  If it is determined that the vehicle is not in front of the intersection (NO in step S18), the image processing unit 30 selects the preceding vehicle area 230, the pedestrian area 210, and the sign area 220 from the captured image, and about these areas. The images are repeatedly selected in order (for example, cyclically), and image recognition processing is sequentially performed on the selected region (step S19). Therefore, when the preceding vehicle region 230 is selected, image recognition processing is performed while maintaining high vehicle recognition accuracy. When the pedestrian region 210 is selected, image recognition is performed while maintaining high pedestrian recognition accuracy. When the recognition process is performed and the sign area 220 is selected, the image recognition process is performed while maintaining high recognition accuracy of the sign. If it is determined that it is not in front of this intersection, intersection 3 is included. One of the reasons for selecting the three areas when the vehicle is at intersection 3 is a reduction in attention to pedestrians, obstacles, signs, and the like. In addition, you may provide the step which judges whether it is the intersection 3 or not.

  As a result, the preceding vehicle, the pedestrian, and the sign are recognized in a state where the recognition accuracy is maintained high while suppressing the processing load. That is, the processing load related to each image processing can be suppressed. In addition, since the preceding vehicle, the pedestrian, and the sign are each recognized and processed with high recognition accuracy, necessary information can be recognized while suppressing the processing load. For example, a recognition process equivalent to a recognition process for a captured image can be performed.

Then, the preceding vehicle, the pedestrian, and the sign recognized in this way are notified as recognition targets (step S12), and then the image recognition process is terminated.
According to the present embodiment, it is possible to provide an image processing apparatus capable of suppressing the processing load related to image processing regardless of the vehicle situation.

As described above, the image processing apparatus according to the present embodiment has the effects listed below.
(1) A part of the captured image is selected as a target area according to the vehicle environment including the driving environment and the driving state of the vehicle 10, and image processing is performed on the selected target area, whereby the entire captured image is image-processed. The processing load can be reduced as compared with the above. In addition, by switching one of the selected target areas as a target area and sequentially performing image processing, the processing load of image processing is reduced regardless of the vehicle situation while maintaining the amount of information obtained by image processing. Can be suppressed. Thereby, for example, preventive safety and sign guidance can be performed.

  (2) Since it is difficult to narrow down the recognition targets at the intersection because of various situations, the processing load is suppressed even at the entrance of the intersection by widening the recognition target by selecting a plurality of target areas. The accuracy of image recognition can be maintained while performing image processing.

(Other embodiments)
In addition, the said embodiment can also be implemented with the following aspects.
In the above embodiment, the case where the vehicle 10 is provided with the in-vehicle camera 20, the image processing unit 30, the navigation system 70, and the GPS device 60 is illustrated. However, the present invention is not limited to this, and if the recognition result can be output for driving support by performing image recognition processing of the captured image in front of the vehicle, the in-vehicle camera, the image processing unit, the navigation system, and the GPS device are some of them. Alternatively, all of them may be integrated, or conversely, part or all of them may be divided more finely. As a result, the degree of freedom in designing the image processing apparatus can be improved.

  In the above embodiment, the case where the in-vehicle network N1 is used has been illustrated, but examples of the in-vehicle network N1 include CAN (controller area network), LIN, FlexRay (registered trademark), and Ethernet (registered trademark). . As a result, the application range of the image processing apparatus can be expanded.

  In the above embodiment, the case where the image processing unit 30, the vehicle speed acquisition unit 40, and the navigation system 70 are connected to the in-vehicle network N1 is illustrated. However, the present invention is not limited thereto, and other devices may be connected to the in-vehicle network as long as each device is configured to be able to exchange necessary information. On the other hand, as long as each device is connected so that necessary information can be exchanged, the connection is not limited to the network. In addition, wireless communication may be included in the connection path. As a result, the application range of the image processing apparatus can be expanded.

  In the above-described embodiment, the region setting unit 310 is exemplified for the case where the captured image in which the range of each region is set is output to the image recognition unit 330. However, the present invention is not limited to this, and the region setting unit may extract image information corresponding to the set range and output the extracted image information to the image recognition unit. As a result, the degree of freedom in designing the image processing apparatus can be improved.

  In the above embodiment, the case where the region selection unit 320 outputs information on the target region to be selected to the image recognition unit 330 has been illustrated. However, the present invention is not limited to this, and the region selection unit may output the target region to be selected to the region setting unit, and may use the image information corresponding to the target region for selecting the image transmitted from the region setting unit to the image recognition unit. As a result, the degree of freedom in designing the image processing apparatus can be improved.

  -In above-mentioned embodiment, when the vehicle condition did not correspond to any conditions, the case where the preceding vehicle area | region 230, the pedestrian area | region 210, and the marker area | region 220 were selected as an object area | region was illustrated. However, the present invention is not limited to this, and when the vehicle situation does not correspond to any of the conditions, at least two of the preceding vehicle area, the pedestrian area, and the sign area may be selected as the target area. By selecting at least two, it is possible to maintain a large amount of information recognized by the recognition process as compared to a case where only one is selected, and because it is a combination of processes with low processing addition. The processing load as a whole can be kept low. Thereby, the application range as an image processing apparatus can be expanded.

  In the above embodiment, the case where the processing for the three areas is repeatedly performed in order is illustrated. However, the present invention is not limited to this, and the processing for the three regions may be switched in an arbitrary order, and the recognition processing for the switched regions may be sequentially performed. For example, it is possible to increase the number of times of recognition of a region having high importance. Therefore, the order of the pedestrian area, the preceding vehicle area, the pedestrian area, and the sign area may be repeated, or the order of the pedestrian area, the preceding vehicle area, the sign area, the pedestrian area, and the preceding vehicle area. May be repeated. In addition, a period may be set for each of the recognition process for the pedestrian area, the recognition process for the preceding vehicle area, and the recognition process for the sign area, and the recognition process for each area may be performed according to each set period. In any case, it is possible to keep the processing load of the recognition process low while maintaining the information obtained by the recognition process.

  In the above-described embodiment, the preceding vehicle area 230 is selected when the road is a highway, the pedestrian area 210 is selected when the road is close to the destination or near the destination, and the sign area 220 is selected before the intersection. The case was illustrated. However, the present invention is not limited to this, and when it is an expressway, a sign area or a pedestrian area may be appropriately selected while selecting many preceding vehicle areas. In addition, when the vehicle is in the vicinity of a road or a destination that frequently passes, the preceding vehicle region or the sign region may be appropriately selected while selecting many pedestrian regions. Furthermore, when the vehicle is in front of the intersection, the preceding vehicle region or the pedestrian region may be appropriately selected while selecting many sign regions. As a result, the application range of the image processing apparatus can be expanded.

  In the above embodiment, the case where the image recognition process is executed by the image processing unit 30 is illustrated. However, the present invention is not limited to this, and as long as image recognition processing is possible, a part of the functions and processing may be provided outside the vehicle such as a center. As a result, the degree of freedom in designing the image processing apparatus can be improved.

  In the above embodiment, the case where the image processing unit 30 is an ECU is illustrated. However, the present invention is not limited to this, and the image recognition process may be executed by a mobile information processing device such as a mobile phone, a smartphone, or a tablet. By bringing such a portable information processing apparatus into the host vehicle, it can be used for driving support of the vehicle. As a result, the application range of the image processing apparatus can be expanded.

  -In above-mentioned embodiment, the case where the vehicle 10 was a motor vehicle was illustrated. However, the present invention is not limited to this, and the image processing apparatus can be provided in a robot, a bicycle, a pedestrian guidance apparatus, or the like if it is necessary to monitor the surrounding situation as it moves. Thereby, the application range as an image processing apparatus can be expanded.

  N1 ... vehicle-mounted network, 10 ... vehicle, 11a, 11b ... preceding vehicle, 12 ... oncoming vehicle, 20 ... vehicle-mounted camera, 30 ... image processing unit, 40 ... vehicle speed acquisition unit, 50 ... speed sensor, 60 ... GPS device, 70 ... Navigation system, 100 ... captured image, 110, 111 ... traffic light, 115 ... sign, 120, 121 ... pedestrian, 125 ... crosswalk, 200 ... candidate area, 201 ... exclusion area, 210 ... pedestrian area, 211 ... upper exclusion Area 220 ... sign area 221 ... lower exclusion area 230 ... preceding vehicle area 231 ... exclusion area 310 ... area setting section 320 ... area selection section 330 ... image recognition section 710 ... route guidance section 720 ... history management department.

Claims (2)

An image processing apparatus that performs image processing on a captured image obtained by imaging a surrounding environment of a vehicle,
A selection unit that selects one or a plurality of target areas to be subjected to image processing from a part of the captured image according to a vehicle situation,
When the selection unit selects a plurality of target areas, the image processing for the switched target areas is sequentially executed while switching the target areas to be subjected to image processing among the plurality of target areas. Processing equipment. The image processing apparatus according to claim 1, wherein the selection unit selects a plurality of target areas when the vehicle situation is an approach to an intersection.

Images