JP2015185135A - Parking recognition device, parking recognition method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parking recognition device, a parking recognition program and a program capable of suppressing installation cost and processing cost and enhancing recognition accuracy.SOLUTION: A parking recognition device 100 is a device for performing parking recognition by using a photographed image around a vehicle, and includes an image acquisition part 110 for generating a plurality of image frames which temporally continue, and image processing part 120 for calculating difference between frames of a pixel value from two image frames, and weighting a calculation result of the difference between frames of the pixel value in a first partial area in the two image frames, and a parking recognition part 130 for recognizing parking of a vehicle on the basis of the difference between frames of the pixel value.

Description

  The present invention relates to a stop recognition device, a stop recognition method, and a program, for example, a stop recognition technology capable of suppressing installation cost and processing cost and improving recognition accuracy.

  Conventionally, if it is determined that there is a possibility of collision or abnormal approach with the movement of people or vehicles around the vehicle, information will be provided to the driver or the vehicle Various techniques for performing control and the like have been proposed.

  For example, Patent Document 1 describes a device that notifies a driver via a display or the like if there is a danger of a collision with a pedestrian or the like. This device detects the environment outside the vehicle by a vehicle speed sensor, a radar, or the like. As a result, for example, when it is determined that the vehicle is unlikely to collide with a pedestrian or the like, such as being stopped by a signal, notification to the driver is suppressed. Thereby, the effective driving assistance which suppressed unnecessary information is realized.

JP 2008-143387 A

  However, the device described in Patent Document 1 needs to be configured so that information can be acquired from various sensors, radars, and the like in order to recognize the stop of the vehicle. Therefore, there is a problem that the cost for installing the device in the vehicle is increased.

  In addition, it is desirable that the processing related to vehicle stop recognition is highly accurate so that erroneous recognition can be suppressed as much as possible from the viewpoint of suppressing unnecessary notifications. On the other hand, since the device is originally intended to notify the driver of the danger, it is desirable that the processing for suppressing erroneous recognition is as low in processing load as possible.

  The present invention has been made to solve such problems, and provides a stop recognition device, a stop recognition method, and a program capable of suppressing installation cost and processing cost and improving recognition accuracy. For the purpose.

Therefore, the present invention is an apparatus that performs stop recognition using captured images around a vehicle, and includes an image acquisition unit that generates a plurality of temporally continuous image frames, and pixel values of the two image frames. An inter-frame difference is calculated, and in a first partial region in the two image frames, an image processing unit that weights the calculation result of the inter-frame difference of the pixel value, and an inter-frame difference of the pixel value And a stop recognition unit that recognizes the stop of the vehicle.
In addition, the present invention is a device that performs stop recognition using captured images around a vehicle, an image acquisition unit that generates a plurality of temporally continuous image frames, and a second of the two image frames. There is provided a stop recognition device including an image processing unit that performs motion detection by an optical flow in a partial region, and a stop recognition unit that recognizes the stop of the vehicle based on a motion detection result by the optical flow.
In addition, the present invention is a method for performing stop recognition using captured images around a vehicle, an image acquisition step for generating a plurality of temporally continuous image frames, and pixel values of two image frames. An image processing step that calculates an inter-frame difference and weights the calculation result of the inter-frame difference of the pixel value in the first partial region in the two image frames, and based on the inter-frame difference of the pixel value And a stop recognition step for recognizing the stop of the vehicle.
Further, the present invention is a method for performing stop recognition using captured images around a vehicle, an image acquisition step of generating a plurality of temporally continuous image frames, and a second of the two image frames. There is provided a stop recognition method including an image processing step of performing motion detection by an optical flow in a partial region, and a stop recognition step of recognizing the stop of the vehicle based on a motion detection result by the optical flow.
The present invention also provides a program for causing a computer to execute the stop recognition method.

  According to the present invention, it is possible to provide a stop recognition device, a stop recognition method, and a program capable of suppressing installation cost and processing cost and improving recognition accuracy.

It is a block diagram which shows the structure of the stop recognition apparatus 100 of embodiment of this invention. It is a flowchart which shows the process of the stop recognition apparatus 100 of embodiment of this invention. It is a figure which shows the process of the stop recognition apparatus 100 of embodiment of this invention. It is a figure which shows the process of the stop recognition apparatus 100 of embodiment of this invention. It is a figure which shows the process of the stop recognition apparatus 100 of embodiment of this invention.

  First, an outline of the stop recognition method realized by the stop recognition device 100 according to the embodiment of the present invention will be described.

  The stop recognition device 100 realizes stop recognition of a vehicle by image processing. In this embodiment, as an example of image processing, (1) a stop recognition method using a difference between frames, (2) a stop recognition method using an optical flow, and (3) a method using both methods together are presented. To do.

(1) Stop recognition method using inter-frame difference The stop recognition apparatus 100 compares pixel values of two image frames having different shooting times, and performs stop recognition of the vehicle based on the difference. The pixel value is a multidimensional vector (in the case of a color image) expressed in a color space such as the luminance (in the case of a grayscale image) or RGB or the like of each pixel constituting the image frame. The pixel value difference is typically a luminance difference. Alternatively, it may be a difference between any of RGB values. Or the difference of the arbitrary index values which can be calculated using a pixel value may be sufficient. In the present embodiment, since the difference is relatively easy to make, luminance is used.

  FIG. 3 is a diagram showing how a luminance difference is observed between two temporally continuous image frames. Here, a pixel in which a difference in luminance exceeds a predetermined threshold is shown in white, and the other pixels are shown in black. FIG. 3A shows a change in luminance in an image acquired when the situation outside the vehicle hardly changes visually. In this case, since there is no luminance change exceeding the threshold value, the whole is expressed in black. FIG. 3B shows a change in luminance between frames when a bicycle crosses a pedestrian crossing or an intersection road ahead of the vehicle. In this case, the movement of the bicycle or the like is expressed as white pixels. FIG. 3C shows a change in luminance when the vehicle in the front vehicle or the adjacent lane is moving. In this case, the movements of surrounding vehicles and the like are expressed as white pixels.

  Thus, when there is a movement in a situation outside the vehicle, a difference in luminance occurs between frames. This difference can be observed not only with luminance but also with other pixel values. Typically, when the vehicle is moving, the inter-frame difference in pixel values tends to be a relatively large value, and when the vehicle is stopped, it tends to be a relatively small value. Therefore, in the present embodiment, the stop of the vehicle is recognized using this difference.

  Specifically, when the luminance difference between frames exceeds a predetermined threshold, it is determined that the vehicle is moving, and when it is equal to or less than the threshold, it is determined that the vehicle is stopped. The luminance inter-frame difference can be calculated as follows, for example. First, for each of the two image frames, the luminances of all the pixels included in the frame are added. Next, the difference between the added luminances is obtained. It can also be said that a difference is taken for each corresponding pixel between frames and the difference is added. Finally, the difference is compared with a predetermined threshold value. Note that the method of calculating the luminance inter-frame difference is not limited to this, and an arbitrary method can be adopted. For example, an average of differences per pixel may be used.

  By the way, the inter-frame difference in pixel values does not necessarily occur only when the vehicle itself is moving. For example, FIG. 4 (A) shows a pixel whose luminance has changed from two image frames obtained by photographing the front of the vehicle with an in-vehicle camera while the vehicle is stopped in front of the pedestrian crossing. Here, since the vehicle is stopped, the surrounding landscape does not move and no luminance change occurs, but a person or vehicle crossing the center of the screen is detected as the luminance change. In this way, a certain amount of luminance change is detected even when the vehicle is stopped, but this acts as noise that may reduce the accuracy of stop recognition.

  Therefore, as shown in FIG. 4 and FIG. 5, if a difference in pixel values is easily detected even when the vehicle is stopped, in other words, an area in the screen where the movement of the object is easily observed can be specified in advance. Is preferably masked. That is, the mask area can be excluded from the calculation target of the difference value. Thereby, the influence which the luminance change which generate | occur | produces in the masked area | region has on the determination process of stop recognition can be suppressed.

  In the example of FIG. 4B, a band-like region of about one third near the center in the vertical direction of the image frame is masked. According to the inventor's experiment, by setting the mask area in this way, the person or vehicle crossing the pedestrian crossing or crossing road is effectively excluded from the detection target of the difference value while the vehicle is stopped in front of the pedestrian crossing. it can. FIG. 5B shows an example in which a lane is detected from an image frame and the outside of the detected lane is masked. Since the lane detection technique is known, detailed description thereof is omitted here. Thereby, even when the vehicle in the adjacent lane starts moving while the vehicle is stopped, the vehicle can be excluded from the detection target of the difference value.

  It goes without saying that the mask region can be defined by various methods, not limited to these examples. For example, the mask region for a person or vehicle crossing the front of the vehicle exemplified in FIG. 4B can be appropriately changed according to the mounting position of the in-vehicle camera. Also, the mask area can be automatically set by learning. That is, when it is recognized that the vehicle is stopped, an area where a luminance change or the like is still detected may be identified, and a mask area including the identified area may be set.

  Further, in the stop recognition process, the mask area may be always set, or may be set only in a specific situation. For example, the mask area may be validated when the vehicle speed becomes a predetermined value or less.

  In addition, although an example in which the mask area is excluded from the difference value detection target has been described here, the present invention is not limited to this, and the pixel area difference is also calculated in the area and given a different weight from the other areas. It is good. In other words, a plurality of partial areas may be set in the image frame, and the weighting for the difference in pixel value may be changed for each partial area.

  Note that the plurality of frames that are the basis for the calculation of the interframe difference are typically two frames that are temporally adjacent. However, the present invention is not limited to this, and two frames having a predetermined number of frames (for example, 5 frames) may be used. Or you may comprise so that the space | interval of two frames may be changed dynamically according to a condition. For example, when the inter-frame difference of pixel values cannot be obtained, the calculation of the difference value may be retried by increasing the frame interval. Furthermore, the number of frames used for calculating the difference value is not limited to two, and may be three or more.

(2) Stop Recognition Method Using Optical Flow The stop recognition apparatus 100 performs stop detection of a vehicle by performing motion detection using an optical flow based on two image frames having different shooting times. According to the optical flow, based on the correspondence between the pixels and the like between frames, the movement of the pixels and the like can be expressed as a flow vector. For example, when a feature point (for example, SIFT, SURF, etc.) is detected at the position a in the first image frame and a corresponding feature point is detected at the position b in the next image frame, the motion of this feature point is a flow vector. It is expressed as (a, b). When such a flow vector is detected from a continuous image obtained by photographing the situation outside the vehicle, it can be assumed that there is a relative movement between the vehicle and the external environment. Specifically, when the magnitude of the flow vector exceeds a predetermined threshold, it can be determined that the vehicle is moving, and when it is equal to or less than the threshold, it can be determined that the vehicle is stopped.

  By the way, like the above-described inter-frame difference of pixel values, the flow vector does not necessarily occur only when the vehicle itself is moving. Even when the vehicle is stopped, the flow vector can be detected in accordance with the movement of an object around the vehicle, but this acts as noise that may reduce the accuracy of stop recognition.

  Therefore, as shown in FIG. 4B, it is preferable that only a predetermined partial region of the image frame is an optical flow detection target region. For example, in the example of FIG. 4B, the lower end of the screen frame, the band-like area near the center in the left-right direction is set as the optical flow detection area. According to the inventor's experiment, by setting the optical flow detection region in this way, it is possible to suppress the influence of the movement of surrounding people and vehicles and to effectively detect the movement of the road surface. Further, the processing cost can be suppressed by setting the optical flow detection region as small as possible. In general, since the calculation processing related to the optical flow tends to increase the processing load, suppressing the calculation processing amount greatly contributes to the suppression of the processing load of the entire stop recognition device 100.

  Note that the plurality of frames that are the basis of calculation of the flow vector are typically two frames that are temporally adjacent. However, the present invention is not limited to this, and two frames having a predetermined number of frames (for example, 5 frames) may be used. Or you may comprise so that the space | interval of two frames may be changed dynamically according to a condition. For example, when the flow vector cannot be detected, the calculation of the flow vector may be retried with a wider frame interval. Furthermore, the number of frames used for calculation of the flow vector is not limited to two but may be three or more.

(3) Method using both methods in combination The stop recognition device 100 combines each of the above (1) stop recognition method using inter-frame difference and (2) stop recognition method using optical flow. Compared to the case of using it, it is possible to realize stop recognition with higher accuracy. In other words, only a stop recognition based on the difference between the pixel values of the frame or only a stop recognition based on the optical flow, for example, due to the influence of the surrounding environment, it is not possible to recognize the stop properly even when the vehicle is stopped (false recognition). Yes. In particular, when the setting of the mask area and the optical flow detection area is inappropriate or in the process of adjustment, erroneous recognition is more likely to occur.

  On the other hand, it is possible to suppress the occurrence rate of misrecognition by using a plurality of stop recognition methods together. Typically, it is determined that the vehicle is stopped when both the stop recognition result based on the inter-frame difference of the pixel value and the stop recognition result based on the optical flow indicate “stop”. Can do. Accordingly, it is possible to suppress the occurrence of a situation where the vehicle is recognized as being stopped, although the vehicle is moving. Alternatively, when either one indicates “stop”, it may be determined that the vehicle is stopped. Thereby, it is possible to suppress the occurrence of a situation where the vehicle is not recognized as being stopped even though the vehicle is stopped.

  Or the recognition result by the above-mentioned stop recognition method can also be output as a score instead of outputting with the binary of the presence or absence of a stop. For example, in the above-described example, it is determined whether or not the vehicle is stopped after comparing the inter-frame difference of the pixel value with a threshold value. However, even if the difference value or any index value based on the difference value is output as the stop recognition score good. In the above example, the magnitude of the flow vector is compared with a threshold value to determine whether or not the vehicle is stopped. However, the magnitude of the flow vector or any index value based on the magnitude may be output as the stop recognition score. good. In this case, the stop recognition device 100 can determine whether or not the vehicle is stopped based on the correlation between the recognition score obtained by one stop recognition method and the recognition score obtained by the other stop recognition method. For example, a rule in which a determination result indicating whether or not the vehicle is stopped is associated with a combination of one score and the other score is stored in advance in the form of a table or the like and is determined by referring to the rule. It can be performed. Or it is good also as determining a stop according to the calculation result of the determination type | formula which can input one score and the other score as a parameter.

  Further, the intervals between a plurality of frames that are the basis of calculation of the recognition result may be dynamically changed according to the recognition result obtained by both recognition methods. For example, if the inter-frame difference of pixel values can be acquired, the frame interval in the flow vector may be suppressed (for example, 0, that is, an adjacent frame, etc.) or, of course, may be reversed. As a result, the storage area and time required for processing can be suppressed.

  Next, the stop recognition device 100 according to the embodiment of the present invention will be described more specifically with reference to the drawings. First, the structure of the stop recognition apparatus 100 concerning embodiment of this invention is demonstrated using FIG.

  The stop recognition device 100 includes an image acquisition unit 110, an image processing unit 120, and a stop recognition unit 130.

  The image acquisition unit 110 is a device for acquiring an image outside the vehicle, and is typically an in-vehicle camera. The image acquisition unit 110 is preferably attached at a position where the situation in the traveling direction of the vehicle can be photographed, for example, an arbitrary position facing the windshield inside the vehicle. The image acquisition unit 110 captures the situation outside the vehicle at a predetermined frame rate (the number of captured images per unit time), and sequentially transmits the generated images (frames) to the image processing unit 120.

  The image processing unit 120 performs predetermined image processing using a plurality of image frames received from the image acquisition unit 110. For example, the image processing unit 120 includes an inter-frame difference detection unit 121 that detects inter-frame differences of pixels. Alternatively, it includes a motion vector detection unit 122 that detects movement of feature points or the like between frames as a motion vector. Alternatively, the image processing unit 120 may include both the inter-frame difference detection unit 121 and the motion vector detection unit 122. Alternatively, the image processing unit 120 may include a processing unit (not shown) that performs other image processing for recognizing a situation outside the vehicle.

  The stop recognition unit 130 determines whether the vehicle is stopped according to the processing result of the image processing unit 120.

  The image acquisition unit 110 and the image processing unit 120 typically include an input / output unit that inputs and outputs data, a storage unit that stores data and a control program, and a control unit that executes predetermined processing according to the control program. An information processing entity that is logically realized by an information processing apparatus including Specifically, it can be mounted on a microcontroller or the like.

  It continues and demonstrates operation | movement of the stop recognition apparatus 100 concerning embodiment of this invention using FIG.

S101:
The image acquisition unit 110 acquires an image obtained by photographing the situation outside the vehicle. The image acquisition unit 110 is typically an in-vehicle camera capable of shooting a moving image, and generates an image (frame) at a predetermined frame rate. The image acquisition unit 110 sequentially delivers the generated image to the image processing unit 120.

S102:
When the image processing unit 120 includes the inter-frame difference detection unit 121, the inter-frame difference detection unit 121 can perform the processes shown in S102 to S104.

  The inter-frame difference detection unit 121 inputs two temporally continuous image frames from among the image frames generated by the image acquisition unit 110. Typically, the inter-frame difference detection unit 121 cuts out and inputs two adjacent image frames from the moving image captured by the image acquisition unit 110. Alternatively, the inter-frame difference detection unit 121 may input two images separated from the moving image by an arbitrary number of frames (for example, 5 frames).

  Next, if a mask area is set in advance, the inter-frame difference detection unit 121 extracts pixels included in the area excluding the mask area from each input image frame. If no mask area is set, all pixels included in the image frame are extracted.

  It is assumed that the stop recognition device 100 holds in advance a mask area setting information (position, size, shape, etc. on the image frame) in a storage area (not shown). The inter-frame difference detection unit 121 can recognize the presence / absence and position, size, shape, etc. of the mask region by referring to the setting information.

S103:
The inter-frame difference detection unit 121 calculates the inter-frame difference of pixel values using the pixels extracted from the two image frames. As an example, a method for calculating a luminance difference as an inter-frame difference in pixel values will be described.

  Now, assume that two image frames input by the inter-frame difference detection unit 121 are an image frame A and an image frame B, and n pixels are extracted from each image frame. At this time, the inter-frame difference detection unit 121 compares the luminance of the pixel extracted from the image frame A with the luminance of the pixel extracted from the image frame B one by one, and calculates and accumulates the difference. That is, first, the luminance value of the first pixel a1 of the image frame A and the luminance value of the first pixel b1 of the image frame B are compared to obtain the difference d1. Next, the luminance value of the second pixel a2 of the image frame A is compared with the luminance value of the second pixel b2 of the image frame B to obtain a difference d2. A similar process is performed for the nth pixels an and bn, and a sum of difference values d = (d1 + d2 +... + Dn) is calculated. The inter-frame difference detection unit 121 sets d as the calculation result of the inter-frame difference of the pixel value.

S104:
The inter-frame difference detection unit 121 estimates whether the vehicle is stopped based on the inter-frame difference d of the pixel values calculated in S103. For example, the inter-frame difference detection unit 121 compares the inter-frame difference d with a predefined threshold value θ, and determines that the vehicle is stopped if d ≦ θ. On the other hand, if d>, it is determined that the vehicle is moving. The inter-frame difference detection unit 121 outputs the determination result to the subsequent stop recognition unit 130.

  Alternatively, the interframe difference detection unit 121 may calculate a stop recognition score based on the interframe difference d. For example, d may be used as a score as it is. Alternatively, for example, an index added with a certain process such as normalizing d may be used as the score. In this case, the interframe difference detection unit 121 outputs the stop recognition score to the stop recognition unit 130 at the subsequent stage.

S105:
When the image processing unit 120 includes the motion vector detection unit 122, the motion vector detection unit 122 can perform the processes shown in S105 to S107. When the image processing unit 120 includes both the inter-frame difference detection unit 121 and the motion vector detection unit 122, each of the inter-frame difference detection unit 121 and the motion vector detection unit 122 is connected to the image frame from the image acquisition unit 110. Are operated independently, and each outputs the processing result to the stop recognition unit 130.

  The motion vector detection unit 122 inputs two temporally continuous image frames from among the image frames generated by the image acquisition unit 110. Typically, two adjacent image frames are cut out and input from the moving image, but two images separated by an arbitrary number of frames (for example, 5 frames) may be input.

  Next, if an optical flow detection area is set in advance, the motion vector detection unit 122 cuts out an image included in the optical flow detection area from each input image frame. When the optical flow detection area is not set, the entire image frame is set as the optical flow detection area.

  It is assumed that the stop recognition device 100 previously holds setting information (position, size, shape, etc. on the image frame) of the optical flow detection area in a storage area (not shown). The motion vector detection unit 122 can recognize the presence / absence and position, size, shape, etc. of the optical flow detection region by referring to the setting information.

S106:
The motion vector detection unit 122 tries motion detection by optical flow using images of two optical flow detection regions cut out from two image frames. Since there are various prior arts regarding a specific method of motion detection by optical flow, detailed description thereof is omitted.

S107:
The motion vector detection unit 122 estimates whether the vehicle is stopped according to the motion detection result by the optical flow. For example, when the feature point P exists in the image frame A, the feature point P ′ exists in the image frame B, and these feature points P and P ′ correspond to each other, in other words, the feature of the image frame A. When the point P moves to P ′ in the image frame B, the flow vector is defined as (P, P ′). The motion vector detection unit 122 detects one or more flow vectors from the image frames A and B. When the total value or average value of the lengths of these flow vectors is equal to or less than a predetermined threshold, it is estimated that the vehicle is stopped. On the other hand, when the total value or average value of the length of the flow vector exceeds the threshold value, it is estimated that the vehicle is moving. The motion vector detection unit 122 outputs the determination result to the subsequent stop recognition unit 130.

  Or the motion vector detection part 122 may calculate a stop recognition score based on the motion detection result by an optical flow. For example, the total value or average value of the lengths of the flow vectors may be used as a score as they are. Alternatively, for example, an index obtained by performing a certain process such as normalizing the total value or average value of the lengths of the flow vectors may be used as the score. In this case, the motion vector detection unit 122 outputs the stop recognition score to the stop recognition unit 130 at the subsequent stage.

S108:
The stop recognition unit 130 performs final stop recognition according to the analysis result of the image frame by the image processing unit 120. For example, when the image processing unit 120 includes only one of the inter-frame difference detection unit 121 and the motion vector detection unit 122, the stop recognition unit 130 stops by the inter-frame difference detection unit 121 or the motion vector detection unit 122. Follow the recognition result as it is, and use it as the final stop recognition result. In this case, the inter-frame difference detection unit 121 or the motion vector detection unit 122 outputs a binary value indicating whether or not the vehicle is stopped as a stop recognition result, and does not output a result based on the multivalue such as the above-described score. To do.

  On the other hand, when the image processing unit 120 includes both the inter-frame difference detection unit 121 and the motion vector detection unit 122, the stop recognition unit 130 determines the stop recognition results of the inter-frame difference detection unit 121 and the motion vector detection unit 122, respectively. Based on this, final stop recognition is performed.

  Typically, when both the inter-frame difference detection unit 121 and the motion vector detection unit 122 recognize both as “stop”, it can be finally determined that the vehicle is stopped. Alternatively, when either one of the inter-frame difference detection unit 121 or the motion vector detection unit 122 recognizes “stop”, it may be finally determined that the vehicle is stopped. Also in this case, the inter-frame difference detection unit 121 and the motion vector detection unit 122 output a binary value indicating whether or not the vehicle is stopped as a vehicle stop recognition result.

  On the other hand, when the inter-frame difference detection unit 121 and the motion vector detection unit 122 can output the stop recognition result with multiple values such as a score, the stop recognition unit 130 includes the inter-frame difference detection unit 121 and the motion vector detection unit 122. Based on the correlation between the scores output by both parties, a final determination can be made as to whether or not the vehicle is stopped. For example, the stop recognition device 100 does not illustrate a rule in which a determination result as to whether or not the vehicle is stopped is associated with a combination of the output score of the interframe difference detection unit 121 and the output score of the motion vector detection unit 122. Prestored in the storage area in the form of a table or the like. Then, the stop recognition unit 130 makes a final determination as to whether or not the vehicle is stopped by referring to the rule. Alternatively, a determination expression that can be input using the output score of the inter-frame difference detection unit 121 and the output score of the motion vector detection unit 122 as parameters is defined in advance, and the stop determination is made according to the calculation result of the determination expression. It is also good to do.

S109 to S111:
It is assumed that the stop recognition device 100 holds information (flag) indicating whether or not the vehicle is stopped in a storage area (not shown). If the stop recognition unit 130 finally determines that the vehicle is stopped in S108, the stop recognition unit 130 updates the flag with information indicating that the vehicle is stopped. On the other hand, when it is finally determined that the vehicle is not stopped, the stop recognition unit 130 updates the flag with information indicating that the vehicle is not stopped.

  Thereby, for example, a functional module that notifies the driver of the danger can know whether or not the vehicle is stopped by referring to the flag. When the vehicle is stopped, it is possible to perform processing such as suppressing danger notification to the driver.

  In the present embodiment, the image processing unit 120 and the stop recognition unit 130 can recognize the stop of the vehicle based only on the image acquired by the image acquisition unit 110. Therefore, it is not necessary to acquire information from various sensors, radars, etc., and the cost for installing the stop recognition device 100 in the vehicle can be suppressed.

  Further, in the present embodiment, the inter-frame difference detection unit 121 treats an area in which movements of surrounding people and cars are likely to appear as a mask area while the vehicle is stopped. Thereby, the influence which the surrounding person and a vehicle have on stop recognition can be suppressed, and misrecognition can be reduced.

  Moreover, in this Embodiment, the stop recognition part 130 performs final stop recognition using the stop recognition result by the inter-frame difference detection part 121 and the motion vector detection part 122 together. As a result, the accuracy of stop recognition can be improved and the stop recognition logic can be designed flexibly.

  In the present embodiment, the inter-frame difference detection unit 121 calculates the inter-frame difference for an area obtained by removing the mask area from the image frame. Further, the motion vector detection unit 122 performs motion detection on the optical flow detection region set in the image frame. Thus, by rationally narrowing down the processing target area, the image processing unit 120 can suppress the processing load while maintaining the recognition accuracy.

<Other embodiments>
Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention. For example, in the above-described embodiment, the stop recognition method using the difference between frames, the method using the mask region, the stop recognition method using the optical flow, the method of setting the optical flow detection region, and the like have been described. Of course, it can be executed in any combination.

  In the above-described embodiment, as the motion detection method, the method based on the difference between frames, the method based on the optical flow, and the method using both of them are mainly described. However, instead of these methods or in combination with these methods, a motion detection method based on other image processing may be adopted.

  For example, a technique called template matching may be adopted. Template matching means that the image processing unit 120 detects a partial area in an image frame that matches a predetermined original image (template), and tracks how the partial area moves in a continuous image frame. This is a technique for detecting motion from an image frame. For example, the image processing unit 120 can extract a partial region where feature points are concentrated from one image frame as an original image, and detect a partial region that matches the original image from the subsequent image frame. Alternatively, the image processing unit 120 stores in advance an image of an object (such as a sign or a number on a license plate) that often appears on a road as an original image, and selects a partial region that matches the original image from a series of image frames. It may be detected.

  In addition, the stop recognition device 100 may be used by being incorporated in various in-vehicle systems, typically systems that provide danger notification to the driver. In general, image processing involves a high processing load, but the stop recognition device 100 according to the present invention can perform stop recognition by image processing at a relatively low processing cost as described above. Therefore, even if the stop recognition device 100 is incorporated in a system having an important function of danger notification to the driver, it is difficult to prevent such an important function from operating.

  In the above-described embodiment, the present invention has been mainly described as a hardware configuration. However, the present invention is not limited to this, and a CPU (Central Processing Unit) executes a computer program for arbitrary processing. Can also be realized. In this case, the computer program can be stored and provided to the computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

DESCRIPTION OF SYMBOLS 100 Stop recognition apparatus 110 Image acquisition part 120 Image processing part 121 Inter-frame difference detection part 122 Motion vector detection part 130 Stop recognition part

Claims (5)

An apparatus for performing stop recognition using captured images around a vehicle,
An image acquisition unit that generates a plurality of temporally continuous image frames;
An image processing unit that calculates an inter-frame difference of pixel values from the two image frames, and weights a calculation result of the inter-frame difference of pixel values in a first partial region in the two image frames; ,
A stop recognition device, comprising: a stop recognition unit that recognizes stop of the vehicle based on the inter-frame difference of the pixel value. An apparatus for performing stop recognition using captured images around a vehicle,
An image acquisition unit that generates a plurality of temporally continuous image frames;
An image processing unit that performs motion detection by optical flow in the second partial regions in the two image frames;
A stop recognition device, comprising: a stop recognition unit that recognizes stop of the vehicle based on a motion detection result by the optical flow. A method of performing stop recognition using captured images around a vehicle,
An image acquisition step for generating a plurality of temporally continuous image frames;
An image processing step of calculating an inter-frame difference of pixel values from the two image frames, and weighting a calculation result of the inter-frame difference of pixel values in a first partial region in the two image frames; ,
And a stop recognition step for recognizing the stop of the vehicle based on the inter-frame difference of the pixel value. A method of performing stop recognition using captured images around a vehicle,
An image acquisition step for generating a plurality of temporally continuous image frames;
An image processing step of performing motion detection by optical flow in the second partial regions in the two image frames;
A stop recognition method, comprising: a stop recognition step for recognizing stop of the vehicle based on a motion detection result by the optical flow.   The program for making a computer perform the stop recognition method of Claim 3 or 4.