WO2021176820A1

WO2021176820A1 - Processing device

Info

Publication number: WO2021176820A1
Application number: PCT/JP2020/048701
Authority: WO
Inventors: ユイビンツーン; 永崎　健; 雄飛椎名
Original assignee: 日立Astemo株式会社
Priority date: 2020-03-03
Filing date: 2020-12-25
Publication date: 2021-09-10
Also published as: DE112020005250T5; JP7277666B2; DE112020005250T8; JPWO2021176820A1

Abstract

Provided is a processing device capable of improving the ability to recognize an object such as an illuminated road sign in an image. The processing device acquires a plurality of histograms 1008, 1009 obtained by aggregating the number of bright pixels brighter than a predetermined brightness from images acquired by cameras 101, 102, and performs center estimation processing (S1001) for estimating a position corresponding to the center of a circle of an object in the images from the plurality of histograms 1008, 1009.

Description

Processing equipment

The present invention relates to a processing device, for example, a processing device for a sensing system such as a stereo camera system.

In recent years, with the spread of in-vehicle camera devices, there is an increasing demand for various recognition functions for safe driving and automatic driving. Among them, the stereo camera device simultaneously measures the visual information based on the image and the distance information to the object in the image, so that various objects (people, cars, three-dimensional objects, road surfaces, road surfaces) around the automobile are measured at the same time. It is said that it can grasp signs, signboards, etc. in detail and contribute to improving safety during driving assistance.

There is a sign as one of the objects of object recognition in the in-vehicle camera device. Generally, sign recognition is used for accelerating and decelerating autonomous vehicles in cooperation with map information. In addition, EuroNCAP (2016-2020 update), which is an evaluation index for advanced driver assistance systems, also has evaluation items related to SAS (Speed Assistance Systems), and its importance is increasing. ..

Conventionally, various technologies and devices related to in-vehicle camera devices that are mounted on vehicles and recognize the situation in front of the vehicle have been proposed. In such an in-vehicle camera device, for example, with regard to improving the performance of sign recognition, Patent Document 1 is mentioned as a technique focusing on improving the accuracy of recognition.

Japanese Unexamined Patent Publication No. 2019-125022

However, for example, in the prior art described in Patent Document 1 or the like, an object including a bright region and a region darker than the region, or an object in which a part of the light is emitted and the other region does not emit light is detected. There is not enough consideration for. More specifically, for example, some lightning signs are located above the road surface and emit light near the center (for example, a speed limit display portion), but the circumferential portion does not emit light. In places where the headlights are hard to hit (at night and in tunnels), it is difficult to detect with the conventional circle detection method using the circumferential edge.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a processing device capable of improving the recognition performance of an object in an image such as an electric sign.

In order to solve the above problems, the processing device of the present invention is a processing device that recognizes an object in an image, and acquires a plurality of histograms obtained by totaling the number of pixels brighter than a predetermined brightness from the image. It is characterized in that the center estimation process for estimating the position corresponding to the center of the circle of the object in the image from the plurality of histograms is performed.

According to the present invention, it is possible to detect an object including a bright region and a region darker than the region, or an object in which a part of the light is emitted and the other region does not emit light. More specifically, for example, by detecting a self-luminous object, the detection performance of the electric sign can be improved. Therefore, it is possible to improve the recognition performance of an object in an image such as an electric sign.

Issues, configurations and effects other than those described above will be clarified by the explanation of the following embodiments.

The block diagram which shows the schematic structure of the vehicle-mounted stereo camera device of this embodiment. The flow diagram explaining the processing content of the stereo camera processing which is the basis of this embodiment. The timing chart of various processing of the stereo camera processing which is the basis of this embodiment. The flow diagram explaining the processing content of the sign recognition process. The figure explaining the center estimation process and the radius estimation process in a sign detection process (circle detection process). Histogram used in radius estimation processing in sign detection processing (circle detection processing). The flow diagram explaining the processing content of the sign detection process. The figure explaining the image captured by the 1st shutter. The figure explaining the image captured by the 2nd shutter. The flow diagram explaining the processing content of the self-luminous object detection process. The figure explaining the center estimation process in the self-luminous object detection process. The figure explaining the area estimation processing in self-luminous object detection processing. The figure explaining the case where there is a bright region around the center of the estimated region. The figure explaining the method of determining a threshold value.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each figure, parts having the same function may be designated by the same reference numerals and repeated description may be omitted.

(Outline configuration of in-vehicle stereo camera device)
FIG. 1 is a block diagram showing a schematic configuration of an in-vehicle stereo camera device of the present embodiment. The in-vehicle stereo camera device 100 of the present embodiment is a device mounted on a vehicle and recognizes the outside environment of the vehicle based on image information of a shooting target area in front of the vehicle. The in-vehicle stereo camera device 100 recognizes, for example, white lines of roads, pedestrians, vehicles, other three-dimensional objects, signals, signs, lighting lamps, etc. from captured images (information), and mounts the stereo camera device 100. Make adjustments such as braking and steering adjustment of the vehicle (own vehicle).

The in-vehicle stereo camera device 100 captures image information in an image based on two cameras 101 and 102 (left camera 101 and right camera 102) arranged side by side and image information acquired by the

cameras

101 and 102. It has a processing device 110 that performs recognition processing of an object (object) of the above. The processing device 110 is configured as a computer including a processor such as a CPU (Central Processing Unit), a memory such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an HDD (Hard Disk Drive). Each function of the processing device 110 is realized by the processor executing the program stored in the ROM. The RAM stores data including intermediate data of operations performed by a program executed by the processor.

Specifically, the processing device 110 has an image input interface 103 for controlling the imaging of the

cameras

101 and 102 and capturing the captured images. The image captured through the image input interface 103 is sent data through the internal bus 109, processed by the image processing unit 104 and the arithmetic processing unit 105, and the result in the process of processing or the image data as the final result is stored as a memory. Is stored in the storage unit 106 of.

The image processing unit 104 compares the first image (left image) obtained from the image sensor of the camera 101 with the second image (right image) obtained from the image sensor of the camera 102, and obtains each image. On the other hand, correction of the device-specific deviation caused by the image sensor and image correction such as noise interpolation are performed, and this is stored in the storage unit 106. Further, the points corresponding to each other are calculated between the first and second images, the parallax information is calculated, and this is stored in the storage unit 106 in the same manner as before.

The arithmetic processing unit 105 recognizes various objects necessary for perceiving the environment around the vehicle by using the image and the parallax information (distance information for each point on the image) stored in the storage unit 106. Various objects include people, cars, other obstacles, traffic lights, signs, car tail lamps and head rides, and the like. A part of these recognition results and intermediate calculation results is recorded in the storage unit 106 as before. After performing various object recognition on the captured image, the arithmetic processing unit 105 calculates the vehicle control policy using these recognition results.

The vehicle control policy obtained as a result of the calculation and a part of the object recognition result are transmitted to the in-vehicle network CAN111 through the CAN interface 107, whereby the vehicle (own vehicle) is braked. Further, regarding these operations, the control processing unit 108 monitors whether or not each processing unit has caused an abnormal operation and whether or not an error has occurred during data transfer, which is a mechanism for preventing the abnormal operation. ing.

The image processing unit 104 is an image input interface 103 that is an input / output unit between the control processing unit 108, the storage unit 106, the arithmetic processing unit 105, and the image pickup elements of the

cameras

101 and 102 via the internal bus 109. It is connected to the CAN interface 107, which is an input / output unit with the external in-vehicle network CAN111. The control processing unit 108, the image processing unit 104, the storage unit 106, the arithmetic processing unit 105, and the input /

output units

103 and 107 are composed of a single computer unit or a plurality of computer units. The storage unit 106 is composed of, for example, a memory that stores image information obtained by the image processing unit 104, image information created as a result of scanning by the arithmetic processing unit 105, and the like. The CAN interface 107, which is an input / output unit with the external vehicle-mounted network CAN111, outputs the information output from the vehicle-mounted stereo camera device 100 to the control system of the own vehicle via the vehicle-mounted network CAN111.

(Processing content of in-vehicle stereo camera device)
FIG. 2 shows a processing flow (in other words, processing content of stereo camera processing) in the vehicle-mounted stereo camera device 100 which is the basis of the present embodiment.

First, images are imaged by the left and right cameras 101 and 102 (S201 and S202), and for each of the

image data

121 and 122 captured by each, image processing such as correction for absorbing the unique characteristics of the image sensor is performed. Do (S203). The processing result is stored in the image buffer 161. The image buffer 161 is provided in the storage unit 106 of FIG. Further, the two corrected images are collated with each other, and the parallax information of the images obtained by the left and

right cameras

101 and 102 is obtained. The parallax of the left and right images makes it clear where and where a certain point of interest on the object corresponds to the images of the left and

right cameras

101 and 102, and the distance to the object can be obtained by the principle of triangulation. It will be. This is done by parallax processing (S204). The image processing (S203) and the parallax processing (S204) are performed by the image processing unit 104 of FIG. 1, and the finally obtained image and the parallax information are stored in the storage unit 106.

Further, various object recognition processes are performed using the above-mentioned stored image and parallax information (S205). The object to be recognized includes a person, a car, other three-dimensional objects, a sign, a traffic light, a tail lamp, and the like, and the recognition dictionary 162 is used as necessary for recognition. The recognition dictionary 162 stores and records, for example, the features of the object to be recognized as machine learning data. Furthermore, in consideration of the result of object recognition and the state of the own vehicle (speed, steering angle, etc.), the vehicle control process issues a warning to the occupant, for example, braking of the own vehicle, braking such as steering angle adjustment, etc. (S206), and the result is output to the outside through the CAN interface 107 (S207). Various object recognition processes (S205) and vehicle control processes (S206) are performed by the arithmetic processing unit 105 of FIG. 1, and output (S207) to the external in-vehicle network CAN 111 is performed by the CAN interface 107. Each of these processes and means is composed of, for example, a single computer unit or a plurality of computer units, and is configured so that data can be exchanged with each other.

FIG. 3 shows the timing of various processes in the in-vehicle stereo camera device 100 of the present embodiment.

In the timing chart of FIG. 3, the flow of two systems is roughly shown as 301 and 302. The flow of 301 indicates the processing timing of the image processing unit 104 of FIG. 1, and the flow of 302 indicates the processing timing of the arithmetic processing unit 105 of FIG.

First, in the flow of 301, the right image input process (S303) is performed. This corresponds to the process of capturing an image with the right camera 102 in FIG. 2 (S202), then performing image processing (S203), and storing the right image in the image buffer 161. Next, the left image input process (S304) is performed. This corresponds to the process of capturing an image with the left camera 101 in FIG. 2 (S201), performing image processing (S203), and storing the left image in the image buffer 161. Next, parallax processing (S204) is performed. This is a process of reading the two left and right images from the image buffer 161 in FIG. 2, calculating the parallax by collating between the two images, and storing the calculated parallax information in the storage unit 106. Corresponds to. At this point, the image and the parallax information are gathered in the storage unit 106.

Next, in the flow of 302, various object recognition processes (S205) and vehicle control processes (S206) are performed using the image and parallax information stored in the storage unit 106, and the results are obtained by the CAN interface 107. It will be output to the in-vehicle network CAN111.

In the various object recognition processes (S205), people, cars, other three-dimensional objects, signs, traffic lights, tail lamps, etc. are recognized from the images stored in the storage unit 106 and the parallax information. A sign recognition process (S205a) for recognizing a sign installed on a road or the like, which is a characteristic part of the form, will be described.

(Sign recognition processing)
FIG. 4 shows a processing flow of the sign recognition processing (S205a).

The sign recognition process (S205a) of the present embodiment is basically a process of recognizing a sign that is a circular object by detecting a circle. As a premise for that purpose, the

cameras

101 and 102 are provided with a first shutter and a second shutter. The exposure time of the first shutter is longer than the exposure time of the second shutter in order to recognize a dark object. In other words, the shutter speed of the first shutter is slower than the shutter speed of the second shutter. Further, the imaging by the second shutter (relatively short exposure time) is performed in a frame after the imaging by the first shutter (relatively long exposure time).

On the premise of the above conditions, the sign recognition process (S205a) of the present embodiment basically passes through the image process (S203) and stores the image (left image or the left image) in the image buffer 161 as shown in FIG. , Right image), a sign detection process (S401) for extracting a circular object, a sign identification process (S402) for identifying the type of a circular object, and a sign tracking process (S403) for associating between frames of a circular object. ), The sign determination process (S404) for making a comprehensive determination in a plurality of frames (for example, 10 frames) of a circular object is included.

The sign detection process (S401) is executed using the circle detection process, and the circle detection process is divided into a center estimation process and a radius estimation process. In the center estimation process, as shown in FIG. 5, a line segment 405 is drawn from each edge in the normal direction, and the point at which the intersections of the line segments 405 overlap by a certain number or more is estimated as the center. In FIG. 5, 406 is estimated to be the center (candidate). In the radius estimation process, the radius of the circle is estimated based on the histogram shown in FIG. In the histogram of FIG. 6, the horizontal axis represents the radius from the center 406, and the vertical axis represents the number of edges within a predetermined width. How to obtain the histogram of FIG. 6 will be described. Assume a circle whose radius gradually increases from the center 406 in FIG. When this circle and an edge (for example, the first edge 407 or the second edge 408) overlap, the number of the overlapped edges is counted and used as the vertical axis of the histogram of FIG. When there is a circle as in 407, the number of edges 410 of the histogram in FIG. 6 is large, and when there is no circle as in 409, the number of edges 411 in the histogram in FIG. 6 is small. The histogram of FIG. 6 that exceeds a predetermined threshold value of 412 is used as a radius candidate. By such processing, a circular object including the features (center, radius) of the circle is extracted from the image (left image or right image) stored in the image buffer 161.

In the sign identification process (S402), an identification process using a classifier is performed on an image including the circular object. As a result of the identification process, the type of the circular object such as whether or not the circle is a sign, the type of the sign, and the content (for example, the speed limit) is recognized.

In the sign tracking process (S403), a circular object tracking process is performed, and in the sign determination process (S404), a circular object determination process is performed based on the result of the tracking process. As a result, the recognition process of the sign which is a circular object in the image is executed and used for the vehicle control process (S206).

(Sign detection processing)
Next, the above-mentioned sign detection process (S401) will be described in more detail.

FIG. 7 shows the processing flow of the sign detection process (S401). As described above, the

cameras

101 and 102 have a first shutter and a second shutter, and the sign detection process (S401) of the present embodiment is obtained by the first shutter (relatively long exposure time). It includes processing on the image (S710) and processing on the image obtained by the second shutter (relatively short exposure time) (S720).

FIG. 8 is a diagram illustrating an image captured by using the first shutter with one camera (for example, the right camera 102) of the in-vehicle stereo camera device 100. Since the exposure time of the first shutter is longer than the exposure time of the second shutter, it is possible to image the red-colored circumferential portion 805 of the speed indicator 801 in FIG. The circumferential portion 805 is a region darker than the self-luminous portion 804, which is a light emitting portion (lightning portion) near the center. The first shutter can detect the circumferential portion 805, but since the first exposure time is set to be relatively long, the self-luminous portion 804 brighter than the circumferential portion 805 may be overexposed and cannot be detected. ..

FIG. 9 is a diagram illustrating an image captured by using the second shutter with one camera (for example, the right camera 102) of the vehicle-mounted stereo camera device 100. The second exposure time of the second shutter is shorter than the first exposure time of the first shutter so that the self-luminous portion 804, which is a light emitting portion (lightning portion) near the center of the speed indicator 801 is not overexposed. It is set to be the time. As a result, as shown in FIG. 9, the circumferential portion 805 may not be detected, but the self-luminous portion 804, which is a light emitting portion, can be detected. Therefore, even if the self-luminous unit 804 cannot be detected in the image of the first shutter described above, the self-luminous unit 804 is present in the image of the second shutter, so that the self-luminous unit 804 can be detected. The area 803 around the self-luminous unit 804 in FIG. 9 represents a prediction area described later.

In the following, the imaging frame by the first shutter will be referred to as the first frame, and the imaging frame by the second shutter will be referred to as the second frame.

As shown in FIG. 7, in step S710 of the sign detection process (S401), in step S711, a process of searching a circle for the entire image (entire surface) obtained by, for example, the first shutter of FIG. 8 is performed by the processor. Will be done. In the image processing (S203) prior to step S711, for example, differential processing is performed on the image of FIG. 8 to emphasize the edges. In step S711, a process of searching for a circle is performed by performing the above-mentioned circle detection process on the entire image in which the edge is emphasized.

In step S712, information on the detected circle (that is, information for identifying the position of the circle), for example, the coordinates of the center (candidate) of the searched circle, the radius (candidate), and the number of detected circles are stored in the memory. Is stored in the storage unit 106.

Using the circle information stored in step S712, in the above-mentioned sign identification process (S402), for example, an identification process using a classifier is performed on the image of FIG.

In step S720 of the sign detection process (S401), in step S721, an image obtained by, for example, the second shutter of FIG. 9 from the information of the previous frame, that is, the circle obtained as a result of the process (S710) by the first shutter. Calculate the center (candidate) of the prediction area (explained later) inside. More specifically, by considering the elapsed time and the movement information of the own vehicle (for example, vehicle speed, yaw rate) in the center coordinates of the circle stored in step S712, the center of the circular region detected in the first frame is set. In the second frame, the coordinates are calculated.

In step S722, a prediction area 803 (see FIG. 9) including the coordinates of the center calculated in step S721 is defined. The prediction area 803 is such that at least the radius d ₁ stored in step S712 includes the circle represented by the _{radius d 2} determined in consideration of the elapsed time and the movement information of the own vehicle (for example, vehicle speed, yaw rate). Defined.

In step S723, the processor performs the above-mentioned circle detection process on the inside of the prediction area 803 defined in step S722. In this way, by performing the circle detection process for detecting the characteristics of the circle in the limited area (in other words, the processing area smaller than the processing area for the circle detection process in step S711), the entire surface is covered. On the other hand, it is possible to reduce the processing load of the processor as compared with the case of performing the circle detection process (S711).

In step S724, the processor confirms whether or not a circle is detected in the prediction area 803. If no circle is detected in the prediction area 803 (S724: Yes), the process proceeds to step S725. Step S725 will be described in detail later in FIG. 10 and the like. If a circle is detected in the prediction area 803 (S724: No), the process proceeds to step S726. In step S726, the processor stores and stores the detected circle information in the storage unit 106 as a memory, and the process returns to step S721. If a circle is detected, the process does not transition to step S725. The reason is that step S725 is a process on the premise that the circle is not detected. Step S725 is performed only on the image acquired by the second exposure time (in other words, the shutter speed) by the second shutter, which is shorter than the first exposure time by the first shutter.

Then, the information of another circle stored in step S712 is read out, and steps S721 to S726 described above are repeated. This series of processing is performed for the number of circles detected by the exposure time (in other words, the shutter speed) by the first shutter.

After that, using the circle information stored in step S726, the identification process using the classifier is performed in the above-mentioned sign identification process (S402).

(Self-luminous object detection processing)
FIG. 10 shows the processing flow of the self-luminous object detection process in step S725 described above. In this self-luminous object detection process, as described above, when a circle is not detected in the prediction area 803 (see FIG. 9) calculated in step S722 (in other words, the image of the prediction area 803 includes the feature of the circle). This is the process to be executed when it is determined that there is no such process.

FIG. 11 is a diagram illustrating the following step S1001.

In step S1001, the processor counts the number of pixels (in other words, pixels brighter than the predetermined brightness) having a brightness higher than a predetermined threshold value (described later in FIG. 14) in the prediction area 803, and a histogram. Is created (acquired). Here,

histograms

1008 and 1009 are created (acquired) for the crossed (orthogonal) x-axis and y-axis, respectively. Then, the processor obtains the position of the center of gravity of the histogram 1008 and the position of the center of gravity of the histogram 1009. As a result, the coordinates of the center of gravity 1010 of the prediction area 803 are specified from the positions of the centers of gravity of the plurality of

histograms

1008 and 1009. The center of gravity 1010 is estimated to be the center (corresponding position) of the circle of the object in the prediction area 803.

FIG. 12 is a diagram illustrating the following step S1002.

In step S1002, the rectangular region 1013 is determined using the coordinates of the center of the circle estimated in step S1001 and the radius d _{2 used in step S722.} The rectangular region 1013 is determined by drawing lines parallel to the x-axis and the y-axis from the first point 1011 and the second point 1012, respectively. The first point 1011 is determined by subtracting the radius d ₂ _{from the coordinates (X C} , Y _C ) of the center of the circle estimated in step S1001. The second point 1012 is determined by adding the radius d ₂ _{from the coordinates (X C} , Y _C ) of the center of the circle estimated in step S1001.

In step S1003, the brightness (valid / invalid) of the region 1013 is determined, and in step S1004, a predetermined brightness is determined around the center of the region 1013 (in other words, in a region of a predetermined size including the center). (Predetermined threshold value: described later in FIG. 14) It is determined whether or not there is a brighter region. As shown in FIG. 13, if there is a bright region 1014, in other words, if it is not invalid (S1004: Yes), it is determined that there is a self-luminous object that is a light emitting portion (lightning portion), and the process proceeds to step S1005. do.

In step S1005, information about the area 1013 (that is, information about the bright area 1014 near the center of the area 1013) is stored and stored in the storage unit 106 as a memory. The information (region information) regarding the region 1013 is, for example, the coordinates of the center calculated in step S1001 and the radius d ₂ .

If there is no bright area, in other words, if it is invalid (S1004: No), it is determined that there is no self-luminous object that is the light emitting part (lightning part), and the information is not stored / stored in the storage unit 106. , The process ends.

FIG. 14 is a diagram illustrating a threshold value determination method for separating the foreground (character portion, light emitting portion) and the background (black background portion, non-light emitting portion) in the prediction area 803.

There are several methods to separate the foreground part and the background part. One example is a method of observing the ratio of the number of pixels in the foreground part to the number of pixels in the boundary part of the background / foreground part.

For example, the image 1401 in FIG. 14 is a case where only high-luminance pixels are used as the foreground. Since the number of pixels in the foreground portion is small (three in the image 1401) and the number of pixels in the black portion adjacent to the foreground portion is large compared to this number, it can be seen that there are many isolated areas. Therefore, it can be estimated by looking at this ratio that it is not suitable for extracting the character part of the foreground.

Further, the image 1403 of FIG. 14 is an example in which the number of pixels occupied by the foreground portion is extremely large compared to the number of pixels of the surplus boundary portion. In such a case, it can be estimated by looking at this ratio that the numbers in the character part (especially the numbers of zero) are crushed and may not be suitable for recognition.

Image 1402 of FIG. 14 is an example of a threshold value suitable for subsequent recognition processing and the like. This is an example in which there is a certain number of pixels in the foreground part and the ratio with the number of pixels in the background part also has a certain ratio expected from the character pattern. By such means, a threshold value can be set according to the quality of the image (contrast, average gradation).

Needless to say, the self-luminous object detection process in step S725 described above may be performed when the feature of the circle is detected.

As described above, in the present embodiment, a plurality of

histograms

1008 and 1009 obtained by totaling the number of pixels brighter than a predetermined brightness are acquired from the images acquired by the

cameras

101 and 102, and from the plurality of

histograms

1008 and 1009. The center estimation process (S1001) for estimating the position corresponding to the center of the circle of the object in the image is performed.

Further, based on the determination (S724) of whether or not the image includes the features of the circle, it is determined whether or not the center estimation process (S1001) is performed, and the image does not include the features of the circle. When it is determined (S724: Yes), the center estimation process (S1001) is performed.

Further, the image is an image captured at an exposure time or a shutter speed at which a light emitting portion in the object can be detected in order to perform identification processing of the object. A circle detection process (S711) is performed to acquire information for identifying the position of the circle from among other images captured at another exposure time longer than the exposure time or at another shutter speed slower than the shutter speed. The processing area (limited area) of the image for the center estimation process (S1001) is smaller than the processing area (entire surface) of the other image for the circle detection process (S711). The image is an image captured in a frame after the other image.

Further, it is determined whether or not there is a region brighter than a predetermined brightness around the position corresponding to the center of the circle (S1004), and if the bright region exists (S1004: Yes), information on the bright region is provided. It is saved in the storage unit (S1005).

In other words, for example, the present embodiment acquires a first image (an image obtained by a first shutter (relatively long exposure time)) suitable for detecting a region having a first brightness. To acquire a second image (an image obtained by a second shutter (relatively short exposure time)) suitable for detecting a region having a second brightness that is brighter than the first brightness. One side. Further, the first process (S710) is performed on the first image, the second process (S720) is performed on the second image, and in particular, the second process is the result of the first process. Use to detect bright areas. Then, when the circle detection process using the circumferential edge cannot be performed in the limited area (prediction area 803) for imaging of the second shutter, for example, an LED type electric sign is detected.

With such a configuration, according to the present embodiment, it is possible to detect an object including a bright region and a region darker than the region, or an object that partially emits light and the other region does not emit light. become. More specifically, for example, by detecting a self-luminous object, the detection performance of the electric sign can be improved. Therefore, it is possible to improve the recognition performance of an object in an image such as an electric sign.

Although the above-described embodiment has been described using the in-vehicle stereo camera device 100 composed of two cameras, one camera may be used, or three or more cameras may be used.

The present invention is not limited to the above-described embodiment, and includes various modified forms. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. The present invention is widely applicable, for example, to detecting an object having a plurality of optical properties, for example, an object that partially emits light.

Further, each of the above configurations, functions, processing units, processing means, etc. may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be stored in a memory, a hard disk, a storage device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Also, the control lines and information lines indicate what is considered necessary for explanation, and not all control lines and information lines are necessarily shown on the product. In practice, it can be considered that almost all configurations are interconnected.

100 In-vehicle

stereo camera devices

101, 102 Camera 103 Image input interface 104 Image processing unit 105 Arithmetic processing unit 106 Storage unit 107 CAN interface 108 Control processing unit 109 Internal bus 110 Processing device 111 In-vehicle network CAN

Claims

A processing device that recognizes an object in an image.
A center estimation process is performed in which a plurality of histograms obtained by totaling the number of pixels brighter than a predetermined brightness are acquired from the image, and a position corresponding to the center of a circle of an object in the image is estimated from the plurality of histograms. A featured processing device.
In the processing apparatus according to claim 1,
A processing device, characterized in that it determines whether or not to perform the center estimation process based on the determination as to whether or not the image includes the characteristics of a circle.
In the processing apparatus according to claim 2,
A processing device characterized in that the center estimation process is performed when it is determined that the image does not include the features of the circle.
In the processing apparatus according to claim 1,
The processing device is characterized in that the image is an image captured at an exposure time or a shutter speed at which a light emitting portion in the object can be detected in order to perform identification processing of the object.
In the processing apparatus according to claim 4,
A circle detection process is performed to acquire information for identifying the position of the circle from other images captured at another exposure time longer than the exposure time or at another shutter speed slower than the shutter speed.
The processing apparatus, characterized in that the processing area of the image for the center estimation processing is smaller than the processing area of the other image for the circle detection processing.
In the processing apparatus according to claim 5,
The processing apparatus, characterized in that the image is an image captured in a frame after the other image.
In the processing apparatus according to claim 1,
It is determined whether or not there is a region brighter than a predetermined brightness around the position corresponding to the center of the circle.
A processing device, characterized in that information about the bright area is stored in a storage unit when the bright area exists.