JP5892542B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention processes a captured image obtained by capturing an entrance / exit that restricts the passage of an object by an opening / closing member such as a stop bar or a chain, and determines whether the passage of the object at the entrance / exit is appropriate or inappropriate. The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  Conventionally, an ETC (Electronic Toll Collection System) lane, a railroad crossing, and the like have been provided with an open / close-type stop bar to limit the passage of vehicles and people.

  In addition, an apparatus that detects unauthorized entry into a crossing using an open / close signal of a stop bar (blocking bar) has been proposed (see Patent Document 1). In Patent Document 1, if there is an unauthorized approaching vehicle that has entered the railroad crossing when the lowering of the blocking bar is detected by a closing signal of the blocking bar or the like, a captured image obtained by imaging the unauthorized approaching vehicle is processed. To recognize the number written on the license plate.

JP 2005-254930 A

  However, Patent Document 1 uses an open / close signal of a stop bar (blocking bar) to determine whether the passage is proper or improper. For this reason, due to some trouble or noise, the stop bar open / close signal may indicate that the stop bar is open even if the stop bar is open, or vice versa. If it is indicated, it is not possible to accurately determine whether an object such as a vehicle is appropriate or inappropriate. That is, Patent Document 1 is based on the premise that the opening / closing signal of the stop bar is always normal, and the case where the opening / closing signal of the stopping bar becomes abnormal due to some trouble or noise. I did not expect.

  An object of the present invention is to process a captured image obtained by capturing an entrance / exit that restricts the passage of an object by the opening / closing member without using an opening / closing signal of the opening / closing member, and whether the passage of the object at the entrance / exit is appropriate or not. An object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of accurately determining whether it is appropriate.

  The image processing apparatus of the present invention is configured as follows in order to solve the above-described problems and achieve the object.

  A captured image captured by the imaging apparatus is input to the image input unit. This imaging device images an ETC (Electronic Toll Collection System) lane that restricts the passage of an object by an opening / closing member such as a stop bar or a chain, and an entrance such as a parking entrance.

  The image processing unit processes the captured image input to the image input unit, and detects an object (a person, a car, a motorcycle, a bicycle, or the like) that passes through the doorway. The image processing unit does not need to have a function of discriminating the type of the detected object, as long as it can detect the object passing through the entrance / exit.

Further, the image processing unit detects whether or not the opening / closing member is in a closed state that restricts the passage of the object when the detected object passes through the doorway. Specifically, the image processing unit performs edge detection on the captured image input to the image input unit, and detects whether the opening / closing member is in a closed state based on the detected edge. The opening / closing member is opened when the passage of the object at the doorway is permitted, and is closed when the passage of the object at the doorway is not permitted. That is, if the opening / closing member is open when the object passes through the entrance / exit, the passage of the object at the entrance / exit is proper, and conversely, the opening / closing member is closed when the object passes through the entrance / exit. For example, it can be said that the passage of this object at the doorway is inappropriate.

  The determination unit determines that the passage of the object at the doorway is inappropriate when the image processing unit detects an object that has passed through the doorway when the opening / closing member is in the closed state.

  As described above, the image processing apparatus can determine whether the passage of the object at the entrance / exit is proper or inappropriate without using the open / close signal of the open / close member. Therefore, even when the opening / closing signal of the opening / closing member becomes abnormal due to some trouble or noise, it is possible to accurately determine whether the passage of the object at the entrance / exit is appropriate or inappropriate. .

  In particular, in the ETC lane, it is possible to detect motorcycles (including motorbikes), bicycles, people, and the like that pass through the gate and enter the highway.

  Further, when the determination unit determines that the passage of the object at the entrance / exit is inappropriate, an output unit may be provided for outputting a message to that effect. By configuring in this way, it is possible to notify an attendant or the like that an object that has improperly passed through the doorway has been detected, and to quickly respond to an object that has improperly passed through the doorway. it can.

In addition, when the opening and closing member is in the closed state, the position of the opening and closing member on the captured image captured by the imaging device is stored in the storage unit, and the image processing unit is configured to capture the captured image input to the image input unit. It is configured to detect whether the opening / closing member is in a closed state by performing edge detection at the position of the opening / closing member on the captured image stored in the storage unit and detecting the presence / absence of the opening / closing member from the detected edge. preferable.

  With this configuration, it is possible to reduce the amount of image processing required to detect whether the opening / closing member is in the open state or the closed state, and whether the object passes through the entrance / exit is appropriate or inappropriate. Judgment can be made quickly.

Images processing unit, if example embodiment, the captured image input to the image input unit, and an edge amount storage unit detected by the edge detection carried out at the position of the closing member on the captured image to be stored, predetermined determination amount The presence / absence of the opening / closing member may be detected by comparison with the above. In addition, this determination amount is the edge amount detected by the edge detection performed at the position of the opening / closing member on the captured image stored in the storage unit for the captured image captured by the imaging device when the opening / closing member is in the closed state. You may decide based on.

  According to the present invention, it is possible to determine whether the passage of the object at the entrance / exit is appropriate or inappropriate without using the opening / closing signal of the opening / closing member. Therefore, even when the opening / closing signal of the opening / closing member becomes abnormal due to some trouble or noise, it is possible to accurately determine whether the passage of the object at the entrance / exit is appropriate or inappropriate. .

It is a block diagram which shows the structure of the principal part of an image processing apparatus. It is a figure which shows the imaging area of an imaging device. It is a figure explaining stop bar detection field data. It is a figure which shows the example of the object which passes a gate illegally. It is a flowchart which shows operation | movement of an image processing apparatus. It is a figure explaining an object map. It is a flowchart which shows an appropriate / inappropriate determination process.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing a configuration of a main part of the image processing apparatus. The image processing apparatus 1 includes a control unit 11, an image input unit 12, an image processing unit 13, a storage unit 14, a timer 15, and an output unit 16.

  The control unit 11 controls the operation of each part of the main body of the image processing apparatus 1.

  A frame image (captured image) obtained by imaging the imaging area by the imaging device 2 is input to the image input unit 12. The image input unit 12 has a frame memory that temporarily stores the input frame image.

  The imaging device 2 captures a frame image of several tens of frames (about 10 to 30 frames) per second and inputs it to the image input unit 12. As shown in FIG. 2, the imaging device 2 adjusts the imaging area to an ETC (Electronic Toll Collection System) lane. The ETC lane is provided with a stop bar that restricts passage of an object such as a vehicle. FIG. 2A is a frame image in a state in which the stop bar provided in the ETC lane is closed, and FIG. 2B is a frame image in a state in which the stop bar provided in the ETC lane is opened. is there. The stop bar is opened and closed by a switch (not shown).

  As is well known, ETC determines whether or not to allow the vehicle to pass through a gate (entrance / exit) by wireless communication with an in-vehicle device mounted on a vehicle traveling on an ETC lane. In ETC, when it determines with permitting passage of a gate, a stop bar is temporarily opened and the gate passage of the vehicle is permitted. On the other hand, when it is determined that the passage of the gate is not permitted, the gate of the vehicle is restricted without opening the stop bar (the closed state is maintained).

  The image processing unit 13 processes the frame image (captured image of the imaging device 2) input to the image input unit 12, and detects an imaged object (person, automobile, motorcycle, bicycle, etc.). In addition, an object moving within the imaging area (particularly the ETC lane) of the imaging device 2 is tracked by a plurality of temporally continuous frame images. The image processing unit 13 uses a spatio-temporal MRF (Markov Random Field) model to extract (detect) an object imaged in a frame image and track the extracted object. As is well known, the spatiotemporal MRF model is an extension of the MRF model as a spatiotemporal model, focusing on the correlation in the time axis direction of the spatiotemporal image. In this spatio-temporal MRF model, a frame image to be processed is divided into blocks of several pixels × several pixels (for example, 8 pixels × 8 pixels), and each block between temporally continuous frame images is divided. This is a model in which the correlation in the time axis direction with reference to the motion vector is defined. Details of the image processing in the image processing unit 13 will be described later.

  The storage unit 14 stores an operation program for operating the main body, setting data used during operation, processing data generated during operation, and the like. The setting data includes stop bar detection area data and stop bar presence / absence determination edge amounts. The stop bar detection area data indicates an area (stop bar detection area) on the image where the presence or absence of the stop bar is detected in the frame image captured by the imaging device 2. Specifically, in the frame image captured when the imaging device 2 shown in FIG. 3 is in the closed state, the area on the frame image where the stopping bar is located (the area surrounded by the broken line in FIG. 3). ).

The stop bar presence / absence determination edge amount is the edge amount detected by the edge detection performed on the stop bar detection area described above in the frame image shown in FIG. 3 (frame image when the stop bar is in the closed state). This is a value determined based on (total number of pixels detected as edges). Specifically, when the edge amount detected by the edge detection performed on the stop bar detection area is A,
Stop bar presence / absence judgment edge amount = α × A
The value set by. However, α is 0 <α <1.

  Although details will be described later, if α is too small, the probability of false detection that there is a stop bar even though the stop bar is not located in the stop bar detection area increases, and conversely, α is too large. This increases the probability of erroneous detection that there is no stop bar even though the stop bar is located in the stop bar detection area. α is appropriately about 0.6 to 0.8, and may be determined according to the performance of the imaging apparatus 2, the imaging environment, and the like.

  The timer 15 measures the current time.

  When the output unit 16 detects an object that has illegally passed through the gate, the output unit 16 outputs that fact to an external device such as an alarm device (not shown). This external device notifies an attendant or the like of the presence of an object that has illegally passed through the gate.

  An object that has illegally passed through the gate here means a closed state such as a motorcycle (including a motorbike) as shown in FIG. 4 (A) or a person as shown in FIG. 4 (B). It is an object that has passed through the gate after passing between the stop bars.

  Next, the operation of this image processing apparatus will be described. FIG. 5 is a flowchart showing the operation of the image processing apparatus.

  The imaging device 2 inputs a frame image obtained by imaging the imaging area to the image input unit 12. In the image processing apparatus 1, the frame image input to the image input unit 12 by the image processing unit 13 is fetched frame by frame, and the following processing is repeatedly performed.

  The image processing unit 13 captures a frame image for one frame (s1), and performs an illuminance change corresponding filtering process on the frame image (s2). This filtering process corresponding to illuminance change converts an input frame image into a normalized frame image in consideration of changes in illuminance of the frame image due to changes in the weather (clear, cloudy, etc.) around the imaging area. Filtering process.

  The image processing unit 13 generates a background model image (s3), and generates a difference image between the frame image subjected to the filtering process in s2 and the background model image generated in s3 (s4). The image processing unit 13 extracts an imaged object from the difference image generated in s4 (s5). In s5, an object larger than a predetermined size is extracted. That is, an object having a size smaller than a predetermined size is not extracted as an object. In s5, a person, a bicycle, or the like is extracted as an object, and the size of the object to be extracted is determined so as not to extract noise or a fallen leaf blown by the wind as an object.

  The image processing unit 13 associates the object extracted from the previously processed frame image with the object extracted in s5 (s6). The image processing unit 13 uses a spatio-temporal MRF model to extract an object as an object region having a block of n pixels × n pixels (for example, 8 pixels × 8 pixels) as a unit. By associating the object extracted from the previously processed frame image with the object extracted in s5 in s6, the moving direction and the moving amount of the object extracted this time can be obtained.

  In addition, if there is an object extracted in the current process and not extracted in the previous process (that is, an object extracted for the first time), an ID is assigned to each object extracted for the first time (s7, s8).

  The image processing unit 13 creates an object map for the frame image processed this time (s9). As shown in FIG. 6, this object map includes, for each object extracted this time, an ID assigned to the object (this time, it may be assigned in s8) and a coordinate position on the frame image processed this time. Is associated with each other. The image processing unit 13 also assigns a time stamp to the object map. This time stamp may be the time when the frame image is captured at s1, or the time (current time) measured by the timer 15 when it is given. When the object map is created in s9, the center of the position where the object extracted this time is in contact with the road surface is detected as the coordinate position of the object. The image processing apparatus 1 stores the object map created by the image processing unit 13 in s9 in the storage unit 14 (s10).

  Therefore, the image processing apparatus 1 tracks the movement in the imaging area of the imaging apparatus 2 for each object extracted in s5 by arranging the object maps stored in the storage unit 14 in time series. be able to.

  In addition, the image processing unit 13 determines whether any position of the object extracted in s5 is a gate passing position (s11). That is, it is determined whether or not an object is located at the passage position of the gate.

  The passage position of the gate referred to here is a predetermined range centered on a perpendicular drawn from the stop bar to the road surface. If the image processing unit 13 determines in s11 that the object is not located at the gate passing position, the image processing unit 13 returns to s1 and repeats the above processing. On the other hand, if the image processing unit 13 determines in s11 that the object is located at the gate passing position, the image processing unit 13 executes appropriate / inappropriate determination processing (s12), and then returns to s1 to repeat the above processing. .

  FIG. 7 is a flowchart showing appropriate / inappropriate determination processing.

  The image processing unit 13 performs an edge detection process in the stop bar detection region shown in FIG. 3 for the frame image that has been subjected to the illuminance change filtering process in s2 (s21). In s21, this edge detection process is not performed outside the stop bar detection area of the frame image. In the edge detection processing according to s21, the boundary between the stop bar and another object (including the background) is detected as an edge. That is, when the stop bar is in the closed state shown in FIG. 2A, FIG. 4A, or FIG. 4B, the number of pixels detected as an edge increases. On the other hand, when the stop bar is in the open state shown in FIG. 2B, the number of pixels detected as an edge decreases.

The image processing unit 13 compares the stop bar presence / absence determination edge amount stored in the storage unit 14 with the number of pixels detected as an edge in the edge detection process in s21, so that the stop bar is detected in the stop bar detection area. It is determined whether or not is located (s22). In s22 determination,
(1) Stop bar presence / absence determination If the edge amount> the number of pixels detected as an edge, it is determined that there is no stop bar in the stop bar detection area,
(2) Stop bar presence / absence determination If the edge amount ≦ the number of pixels detected as an edge, it is determined that there is a stop bar in the stop bar detection area.

  If the image processing apparatus 1 determines in s22 that the stop bar is not located in the stop bar detection area, that is, the stop bar is not in the closed state but in the open state, it determines that the passage is proper (s23). ), This process is terminated.

  Further, when the image processing apparatus 1 determines in s22 that the stop bar is located in the stop bar detection area, that is, the stop bar is not in the open state but in the closed state, it is determined that the passage is improper. (S24).

  When the image processing unit 13 determines that the improper passage has occurred in s24, the image processing apparatus 1 notifies the external device that the improper passage has occurred (s25), and ends this processing. . In s25, not only the illegal passage of the gate but also the frame image captured in s1 described above may be notified to the external device.

  Thus, the image processing apparatus 1 can determine whether the passage of the object through the gate is appropriate or inappropriate without using the opening / closing signal of the stop bar. Therefore, even when the stop bar opening / closing signal is not normal due to some trouble or noise, it is possible to accurately determine whether the passage of the object at the gate is proper or inappropriate. .

  In the above description, the present invention has been described by taking the image processing apparatus 1 for determining whether or not an object passes through a gate provided in an ETC lane as an example. However, an opening / closing member such as a stop bar or a chain is used. It can also be used at parking lots where traffic of vehicles and people is restricted, and at entrances and exits of construction sites.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... Imaging device 11 ... Control part 12 ... Image input part 13 ... Image processing part 14 ... Memory | storage part 15 ... Timer 16 ... Output part

Claims (7)

An image input unit for inputting a captured image obtained by capturing an image of an entrance / exit where the passage of an object is restricted by an opening / closing member;
Whether the open / close member is in a closed state in which the captured image input to the image input unit is processed to detect an object passing through the doorway and the opening / closing member restricts the passage of the object when the object passes through the doorway An image processing unit for detecting whether or not
A determination unit that determines that the passage of the object at the doorway is inappropriate when the image processing unit detects an object that has passed through the doorway when the opening / closing member is in a closed state ;
The image processing unit is configured to detect an edge of a captured image input to the image input unit and detect whether the opening / closing member is in a closed state based on the detected edge . A storage unit for storing the position of the opening and closing member on a captured image captured by the imaging device when the opening and closing member is in a closed state;
The image processing unit detects an edge of the captured image input to the image input unit at a position of the opening / closing member on a captured image stored in the storage unit , and detects the presence / absence of the opening / closing member from the detected edge by the opening and closing member to detect whether the closed state, the image processing apparatus according to claim 1. The image processing unit includes an edge amount detected by edge detection performed at a position of the opening / closing member on a captured image stored in the storage unit, and a predetermined determination amount for the captured image input to the image input unit. And the presence / absence of the opening / closing member is detected. The image processing apparatus according to claim 2 . The determination amount is an edge detected by edge detection performed at a position of the opening / closing member on a captured image stored in the storage unit with respect to a captured image captured by the imaging device when the opening / closing member is in a closed state. The image processing apparatus according to claim 3 , wherein the image processing apparatus is determined based on the amount. When the determination unit determines that passage of an object in the doorway is improper, the image processing apparatus according to any one of claims 1 to 4 including an output unit for outputting to that effect. A captured image obtained by imaging an entrance / exit where the passage of an object is restricted by an opening / closing member is processed by an imaging device to detect an object passing through the entrance / exit, and the opening / closing member passes the object when the object passes through the entrance / exit. A first step of detecting whether the closed state is limiting;
A second step of determining that the passage of the object at the doorway is inappropriate when detecting an object that has passed through the doorway when the opening and closing member is in the closed state in the first step; An image processing method executed by a computer ,
The first step is an image processing method in which edge detection is performed on a captured image captured by the imaging device, and whether or not the opening / closing member is in a closed state based on the detected edge. A captured image obtained by imaging an entrance / exit where the passage of an object is restricted by an opening / closing member is processed by an imaging device to detect an object passing through the entrance / exit, and the opening / closing member passes the object when the object passes through the entrance / exit. A first step of detecting whether the closed state is limiting;
A second step of determining that the passage of the object at the doorway is inappropriate when detecting an object that has passed through the doorway when the opening and closing member is in the closed state in the first step; an image processing program causing a computer to execute the,
The first step is an image processing program for performing edge detection on a captured image captured by the imaging apparatus and detecting whether the opening / closing member is in a closed state based on the detected edge.

Images