JP7099091B2 - License plate recognition device, license plate recognition method and program - Google Patents

Description

The present invention relates to a license plate recognition device, a license plate recognition method and a program.

In recent years, the development of vehicle detection processing and license plate recognition processing provided in vehicles has been actively carried out. For example, in Patent Document 1, a region corresponding to a different lane is cut out from each of a plurality of captured images obtained by repeated imaging, and a number plate is detected from the cut out region, which corresponds to any lane. A number plate detection device that changes the exposure amount for obtaining the captured image according to the captured image in which the region is cut out is disclosed. In the license plate detection device, one image pickup device can detect the license plate from a wide range.

Patent Document 2 discloses a vehicle monitoring device that recognizes information written on a license plate using images obtained by a plurality of imaging units having different imaging directions. This vehicle monitoring device divides the image obtained by the imaging unit into a plurality of regions, cuts out an image from a specific region among the plurality of regions, detects the vehicle and recognizes the license plate for the cut out image. I do. The image cutout is determined by the vehicle type determination result, the vehicle speed measurement result, and the like.

Japanese Unexamined Patent Publication No. 2016-95763 Japanese Unexamined Patent Publication No. 2017-84417

From the viewpoint of reducing the processing load and the license plate recognition accuracy, an image to be recognized by the license plate is selected based on the change in the vehicle area in the moving image of the road surface, and the license plate recognition process is performed for the image. There is also a way to do this. However, in a congested state, a part of the vehicle is hidden by another vehicle and the vehicle area is not detected. As a result, the image to be recognized by the license plate is not properly selected, and the license plate is not recognized. Is a concern. On the other hand, if the license plate recognition process is performed on all the images constituting the moving image in the congested state, the processing load increases.

Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a novel and improved method capable of suppressing an increase in processing load and a license plate recognition omission. To provide a license plate recognition device, a license plate recognition method and a program.

In order to solve the above problem, according to a certain viewpoint of the present invention, the recognition unit includes a recognition unit that recognizes a license plate from a moving image of a road surface on which a moving object travels, and the recognition unit is the moving object on the road surface. When the congestion state of is the first state, the license plate recognition process is performed on the first image selected from the moving images having the first frame rate, and the congestion state of the moving object on the road surface is changed. In the second state where the degree of congestion is higher than that of the first state, the license plate is used with respect to the second image constituting the moving image having the second frame rate lower than the first frame rate. A license plate recognition device that performs recognition processing is provided.

The number plate recognition device further includes an image selection unit that selects the first image from the image group in which the moving object is detected among the image groups constituting the moving image having the first frame rate. May be good.

The image selection unit is an image corresponding to an intermediate time-series position between the image in which the moving object is first detected and the image in which the moving object is detected last in the image group in which the moving object is detected. May be selected as the first image.

The road surface has one or more lanes, and the recognition unit performs a number plate recognition process on a region corresponding to one lane in the first image, and the recognition unit performs the recognition process of the number plate in the second image. The number plate recognition process may be performed on the region corresponding to the lane and the region corresponding to the side portion of the lane 1.

The moving image of the road surface on which the moving object travels may be a moving image obtained by imaging the road surface from the side opposite to the traveling direction of the moving object.

The license plate recognition device may further include a determination unit for determining a congestion state of the moving object on the road surface.

The number plate recognition device is a detection unit that performs detection processing of the moving object for an image group constituting the moving image of the road surface, and the moving object based on a change in the detection position of the moving object between the images. Further having a speed measuring unit for measuring the moving speed, the determining unit may determine the congestion state of the moving object on the road surface based on the result of the measurement by the speed measuring unit.

The license plate recognition device further includes an information acquisition unit that acquires information indicating a congestion state of the road surface, and the determination unit further congests the moving object on the road surface based on the information acquired by the information acquisition unit. The state may be determined.

The determination unit determines that the congested state of the moving object on the road surface is the first state in the first time zone, and the congested state of the moving object on the road surface in the second time zone. It may be determined that it is the second state.

Further, in order to solve the above problem, according to another aspect of the present invention, the license plate recognition process includes performing the license plate recognition process from the moving image of the road surface on which the moving object travels. When the congestion state of the moving object on the road surface is the first state, the license plate recognition process is performed on the first image selected from the moving images having the first frame rate. When the congestion state of the moving object on the road surface is a second state in which the degree of congestion is higher than that in the first state, a moving image having a second frame rate lower than the first frame rate is configured. A license plate recognition method including performing a license plate recognition process on a second image is provided.

Further, in order to solve the above problem, according to another aspect of the present invention, the computer is provided with a recognition unit that performs license plate recognition processing from a moving image of a road surface on which a moving object is traveling, and the recognition unit is provided. When the congested state of the moving object on the road surface is the first state, the license plate recognition process is performed on the first image selected from the moving images having the first frame rate, and the said on the road surface. When the congestion state of the moving object is the second state in which the degree of congestion is higher than that in the first state, the second image constituting the moving image having the second frame rate lower than the first frame rate. A program for functioning as a license plate recognition device that performs license plate recognition processing is provided.

According to the present invention described above, it is possible to suppress an increase in processing load and omission of license plate recognition.

It is explanatory drawing which shows the license plate recognition system by embodiment of this invention. It is explanatory drawing which shows an example of the imaging range of the imaging unit 18. It is explanatory drawing which shows the structure of the license plate recognition apparatus 20 by embodiment of this invention. It is explanatory drawing which shows the specific example of the selection method of the frame image in a normal state. It is explanatory drawing which shows the specific example of the selection method of the frame image in a traffic jam state. It is a flowchart which shows the operation of the license plate recognition apparatus 20 by embodiment of this invention. It is explanatory drawing which shows the traveling of the vehicle 10 in a traffic jam state. It is a flowchart which shows the operation of the license plate recognition apparatus 20 by the 1st modification. It is a block diagram which showed the hardware composition of the license plate recognition apparatus 20.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, and duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numerals. However, when it is not necessary to particularly distinguish each of the plurality of components having substantially the same functional configuration, only the same reference numerals are given to each of the plurality of components.

<1. Overview of license plate recognition system>
An embodiment of the present invention relates to a license plate recognition system for recognizing a license plate of a vehicle traveling on a road surface. First, the outline of the license plate recognition system according to the embodiment of the present invention will be described with reference to FIGS. 1 and 2.

FIG. 1 is an explanatory diagram showing a license plate recognition system according to an embodiment of the present invention. As shown in FIG. 1, it has a vehicle 10, a road surface image pickup device 16, and a license plate recognition device 20. The road surface image pickup device 16 and the license plate recognition device 20 are connected via a network 12. The network 12 is a wired or wireless transmission path for information transmitted from a device connected to the network 12. For example, the network 12 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Networks) including Ethernet (registered trademark), and a WAN (Wide Area Network). Further, the network 12 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

(vehicle)
The vehicle 10 is an example of a moving body and travels on a road surface. The vehicle 10 has a license plate, and identification information for identifying the vehicle 10 is formed on the license plate. Identification information includes numbers, letters, place names, and the like. Although a four-wheeled vehicle is shown as the vehicle 10 in FIG. 1, the embodiment of the present invention can be applied to a two-wheeled vehicle, a bus, a truck, and the like.

(Road surface imager)
The road surface image pickup device 16 is installed on the side of the road and has a support frame portion 17 and an image pickup section 18. The support frame unit 17 supports the image pickup unit 18 at a position where the image pickup unit 18 can image the road surface from above. The image pickup unit 18 takes an image of the road surface and acquires a moving image of the road surface. The image pickup unit 18 may be a CCD (Chaged-coupled devices) or a CMOS (Complementary metallic-oxide-semiconductor). The image pickup unit 18 is set to repeatedly image the road surface at a predetermined cycle, and images the road surface with an exposure time and a shutter speed according to changes in the environment.

Here, the imaging range of the imaging unit 18 will be described with reference to FIG. FIG. 2 is an explanatory diagram showing an example of the imaging range of the imaging unit 18. As shown in FIG. 2, the imaging range of the imaging unit 18 includes at least one lane region X. That is, the image pickup unit 18 is supported by the support frame unit 17 so that the image pickup range of the image pickup unit 18 includes the lane region X.

The road surface image pickup device 16 transmits the moving image of the road surface acquired by the image pickup unit 18 to the license plate recognition device 20 via the network 12. The moving image of the road surface may be transmitted from the road surface image pickup device 16 to the license plate recognition device 20 in real time or periodically, or in response to a request from the license plate recognition device 20. May be done.

(License plate recognition device)
The license plate recognition device 20 recognizes the license plate of the vehicle 10 traveling on the road surface from the moving image of the road surface received from the road surface image pickup device 16. The method of utilizing the license plate recognition result by the license plate recognition device 20 is not particularly limited. For example, the license plate recognition result may be transmitted to a server that processes the toll on the expressway, and the server may use the license plate recognition result to charge the toll.

<2. License plate recognition device configuration>
The outline of the license plate recognition system according to the embodiment of the present invention has been described above. Subsequently, the configuration and operation of the license plate recognition device 20 according to the embodiment of the present invention will be sequentially described in detail.

FIG. 3 is an explanatory diagram showing the configuration of the license plate recognition device 20 according to the embodiment of the present invention. As shown in FIG. 3, the number plate recognition device 20 according to the embodiment of the present invention includes a communication unit 220, a moving image storage unit 224, a vehicle detection unit 228, a speed measurement unit 232, a congestion determination unit 236, and an image selection unit 240. , Has a selection image storage unit 244 and a recognition unit 248.

(Communication department)
The communication unit 220 communicates with an external device. For example, the communication unit 220 receives a moving image of the road surface from the road surface image pickup device 16. Further, the communication unit 220 transmits the license plate recognition result by the recognition unit 248 to an external device such as a server.

(Video storage unit)
The moving image storage unit 224 stores the moving image of the road surface received from the road surface imaging device 16 by the communication unit 220. The frame rate of the moving image of the road surface stored in the moving image storage unit 224 is not particularly limited, and may be, for example, 30 fps or 60 fps.

(Vehicle detection unit)
The vehicle detection unit 228 performs detection processing of the vehicle 10 for each frame image constituting the moving image of the road surface stored in the moving image storage unit 224. The vehicle detection unit 228 may detect the vehicle 10 by analyzing the optical flow of each frame image. Alternatively, the vehicle detection unit 228 may detect the vehicle 10 from each frame image by using a discriminator for detecting the vehicle 10 obtained by machine learning. Specifically, the vehicle detection unit 228 may detect the vehicle 10 by using the luminance gradient feature amount of the frame image and using the classifier learned in advance with the same luminance gradient feature amount. Here, the vehicle detection unit 228 may perform detection processing of the vehicle 10 on all the frame images constituting the moving image stored in the moving image storage unit 224, or store the moving image in the moving image storage unit 224. The vehicle 10 may be detected for a frame image group having a frame rate lower than that of the moving image. In the present specification, the frame rate of the target frame image group for which the vehicle detection unit 228 performs the detection process of the vehicle 10 may be referred to as a first frame rate.

(Speed measurement unit)
The speed measuring unit 232 measures the moving speed of the vehicle 10 based on the change in the detection position of the vehicle 10 between the continuous frame images. Further, the speed measuring unit 232 calculates the average moving speed, which is the average value of the moving speeds of the vehicles 10 measured within the unit period, for each unit time such as 1 minute, 10 minutes, or 1 hour.

(Traffic jam judgment unit)
The traffic jam determination unit 236 determines the congestion state of the vehicle 10 on the road surface. The traffic jam determination unit 236 may determine the congestion state of the vehicle 10 based on the average moving speed calculated by the speed measurement unit 232. For example, the congestion determination unit 236 determines that the congestion state is the normal state (first state) when the average movement speed calculated by the speed measurement unit 232 exceeds the threshold value, and is calculated by the speed measurement unit 232. When the average moving speed is equal to or less than the threshold value, it may be determined that the congested state is a congested state (second state) in which the degree of congestion is higher than that of the normal state. The threshold value may be a value corresponding to a predetermined ratio of the legal speed set at the position where the road surface image pickup device 16 is installed.

(Image selection section)
The image selection unit 240 selects a frame image from which the license plate recognition process is to be performed by the recognition unit 248 from the moving image stored in the moving image storage unit 224. Here, the image selection unit 240 selects a frame image by a different method according to the determination result of the congestion state by the congestion determination unit 236. Hereinafter, a method for selecting a frame image used in a normal state and a method for selecting a frame image used in a traffic jam state will be described.

In the normal state, each vehicle 10 is difficult to hide from the other vehicle 10, so it is expected that the detection of the vehicle 10 will be performed with high accuracy. Therefore, under the normal state, if the license plate recognition process is performed on the frame image in which the vehicle 10 is detected, the license plate recognition omission does not occur and the license plate is recognized for all the frame images. The load of recognition processing can be significantly reduced as compared with the case of performing processing.

Further, since it is assumed that the same vehicle 10 is detected over a plurality of frame images, the frame image most suitable for license plate recognition is selected from the plurality of frame images detected by the vehicle 10. It is more desirable from the viewpoint of the load of recognition processing. Here, from a certain point of view, the frame image most suitable for recognizing the license plate is a frame image in which the vehicle 10 is present near the center. Therefore, in the normal state, the image selection unit 240 is a time series in the middle (particularly, the center) between the first frame image in which the vehicle 10 is detected and the last frame image in which the vehicle 10 is detected. The frame image corresponding to the position may be selected. Hereinafter, a specific example of a method for selecting a frame image in such a normal state will be described with reference to FIG.

FIG. 4 is an explanatory diagram showing a specific example of a method of selecting a frame image in a normal state. In the example shown in FIG. 4, the vehicle 10 is detected in the frame images f1 to f3, f4 to f6, and f7 to f9. In this example, the image selection unit 240 uses the center frame image f2 of the frame images f1 to f3, the center frame image f5 of the frame images f4 to f6, and the center frame of the frame images f7 to f9. Select image f8.

On the other hand, in a traffic jam state, the vehicle 10 may not be detected because each vehicle 10 is hidden by another vehicle 10. Therefore, in a traffic jam state, the license plate recognition omission may occur in the method of performing the license plate recognition process on the frame image in which the vehicle 10 is detected. If a method of performing license plate recognition processing on all frame images constituting a moving image of a road surface is used, license plate recognition omission can be suppressed, but in this method, the processing load of the recognition processing becomes enormous.

Therefore, in the congested state, the image selection unit 240 selects a frame image at a frame rate lower than the frame rate of the moving image stored in the moving image storage unit 224 (second frame rate). Hereinafter, a specific example of a method for selecting a frame image in such a congested state will be described with reference to FIG.

FIG. 5 is an explanatory diagram showing a specific example of a method of selecting a frame image in a traffic jam state. The frame image group shown in FIG. 5 is a frame image group constituting a moving image stored in the moving image storage unit 224. The image selection unit 240 selects frame images from these frame image groups at equal intervals at a frame rate such as 30 fps, 15 fps, 4 fps, 2 fps, or 1 fps. In FIG. 5, the frame image selected by the moving image storage unit 224 is hatched. In the example shown in FIG. 5, when the frame rate of the frame image group is 60 fps, the frame image flare rate selected by the moving image storage unit 224 is 4 fps.

The image selection unit 240 selects a frame image by the method described above according to the congestion state of the road surface. Then, the image selection unit 240 cuts out the lane region X described with reference to FIG. 2 from the selected frame image in both the normal state and the congested state. Since the license plate recognition process of the cut out lane area X is performed by the recognition unit 248, the cutout has a meaning of limiting the range in which the license plate recognition process is performed, and as a result, the license plate recognition process is performed. It is possible to reduce the processing load.

(Selected image storage unit)
The selection image storage unit 244 stores an image of the lane region cut out by the image selection unit 240 among the frame images selected by the image selection unit 240. An image whose storage time in the selected image storage unit 244 has reached a certain time may be deleted, but even if the setting can be changed between the setting for deleting the image and the setting for not deleting the image. good.

(Recognition unit)
The recognition unit 248 performs license plate recognition processing on the image stored in the selected image storage unit 244. That is, in the normal state, the recognition unit 248 is in the middle (particularly, the center) between the first frame image in which the vehicle 10 is detected from the moving image of the road surface and the last frame image in which the vehicle 10 is detected. License plate recognition processing is performed for the lane area included in the frame image corresponding to the series position. On the other hand, the recognition unit 248 performs license plate recognition processing on the lane region included in the frame image selected from the moving image of the road surface at a frame rate lower than the frame rate of the moving image in the congested state. The license plate recognition result by the recognition unit 248 is transmitted from the communication unit 220 to the external device.

<3. Operation of license plate recognition device>
The configuration of the license plate recognition device 20 according to the embodiment of the present invention has been described above. Subsequently, with reference to FIG. 6, the operation of the license plate recognition device 20 according to the embodiment of the present invention will be arranged.

FIG. 6 is a flowchart showing the operation of the license plate recognition device 20 according to the embodiment of the present invention. As shown in FIG. 6, when the communication unit 220 starts receiving the moving image of the road surface from the road surface imaging device 16, the moving image storage unit 224 receives the moving image of the road surface received from the road surface imaging device 16 by the communication unit 220. Remember (S304).

Then, the vehicle detection unit 228 performs detection processing of the vehicle 10 for each frame image constituting the moving image, and the speed measurement unit 232 calculates the average moving speed of the vehicle 10 (S308). Subsequently, the traffic jam determination unit 236 determines the congestion state of the vehicle 10 based on the average moving speed calculated by the speed measurement unit 232 (S312).

When the congestion state is the normal state (S316 / normal state), the image selection unit 240 selects a frame image corresponding to the time-series position of the center of the frame section in which the vehicle 10 is detected (S320). On the other hand, when the congested state is a congested state (S316 / congested state), the image selection unit 240 selects a frame image at a frame rate lower than the frame rate of the moving image stored in the moving image storage unit 224 (S316 / congested state). S324). After S320 and after S324, the image selection unit 240 cuts out a lane area from the selected frame image, and the selection image storage unit 244 stores the image of the cut out lane area (S328).

Then, the recognition unit 248 performs a license plate recognition process on the image stored in the selected image storage unit 244 (S332). The license plate recognition result by the recognition unit 248 is transmitted from the communication unit 220 to the external device (S336). After that, the processes of S308 to S336 are repeated until the reception of the moving image from the road surface image pickup device 16 is completed (S340).

<4. Action effect>
According to the embodiment of the present invention described above, various effects can be obtained. For example, in the normal state, for the license plate recognition process, the frame image most suitable for license plate recognition is selected from the plurality of frame images detected by the vehicle 10, so that the license plate has high accuracy. It is possible to reduce the load of recognition and recognition processing. Further, in the congested state, the frame image is selected at a frame rate lower than the frame rate of the moving image stored in the moving image storage unit 224 for the license plate recognition processing, so that the license plate recognition omission is prevented. It is possible to suppress it.

<5. Modification example>
The embodiment of the present invention has been described above. Hereinafter, some modifications of the embodiment of the present invention will be described. In addition, each modification described below may be applied to the embodiment of the present invention alone, or may be applied to the embodiment of the present invention in combination. Further, each modification may be applied in place of the configuration described in the embodiment of the present invention, or may be additionally applied to the configuration described in the embodiment of the present invention.

(First modification)
In the above, an example in which the image selection unit 240 cuts out the lane region X described with reference to FIG. 2 from the frame image has been described. The first modification is a modification relating to cutting out a region from a frame image. Hereinafter, the background leading to the recall of the first modification will be described with reference to FIG. 7.

FIG. 7 is an explanatory diagram showing the traveling of the vehicle 10 in a traffic jam state. As shown in FIG. 7, the vehicle 10v, which is a four-wheeled vehicle, travels in the lane in a congested state. On the other hand, in a congested state, the vehicle 10b, which is a two-wheeled vehicle, can travel on the side of the lane and pass through the vehicle 10v, which is a four-wheeled vehicle. The license plate of the two-wheeled vehicle that passes through the vehicle 10v may not be recognized from the image of the lane region X.

Therefore, the image selection unit 240 according to the first modification cuts out the lane region X from the frame image in the normal state, and in the congested state, the regions Y1 and Y2 corresponding to the side portions of the lane region X in addition to the lane region X. Is cut out from the frame image. The side portion of the lane region X may be another lane region adjacent to the lane region X, or may be a roadside or a shoulder. Further, depending on the configuration of the road shoulder, the regions Y1 and Y2 may partially overlap the lane region X or may be provided apart from the lane region X. With such a configuration, it is possible to suppress the omission of recognition of the license plate of the vehicle 10b, which is a two-wheeled vehicle, in a congested state. Since the vehicle 10b, which is a two-wheeled vehicle, generally has a license plate on the back surface, in the first modification, as shown in FIG. 7, the image pickup unit 18 images the road surface from the opposite side of the vehicle 10 in the traveling direction. Hereinafter, with reference to FIG. 8, the operation of the license plate recognition device 20 according to the first modification will be organized.

FIG. 8 is a flowchart showing the operation of the license plate recognition device 20 according to the first modification. Since the processes of S304 to S316 are substantially the same as the processes of S304 to S316 described with reference to FIG. 6, detailed description of the processes of S304 to S316 will be omitted here.

When the congestion state is the normal state (S316 / normal state), the image selection unit 240 selects a frame image corresponding to the time-series position of the center of the frame section in which the vehicle 10 is detected (S320). Then, the image selection unit 240 cuts out a lane region from the selected frame image, and the selection image storage unit 244 stores the image of the cut out lane region (S328).

On the other hand, when the congested state is a congested state (S316 / congested state), the image selection unit 240 selects a frame image at a frame rate lower than the frame rate of the moving image stored in the moving image storage unit 224 (S316 / congested state). S324). Then, the image selection unit 240 cuts out a lane region and a side region from the selected frame image, and the selection image storage unit 244 stores the image of the cut out lane region and the side region (S330).

Then, the recognition unit 248 performs a license plate recognition process on the image stored in the selected image storage unit 244 (S334). The license plate recognition result by the recognition unit 248 is transmitted from the communication unit 220 to the external device (S336). After that, the processes of S308 to S336 are repeated until the reception of the moving image from the road surface image pickup device 16 is completed (S340).

According to such a first modification, the license plate of the vehicle 10 is recognized even when the vehicle 10 which is a two-wheeled vehicle is traveling on the side of the lane such as a lane boundary or a road shoulder in a congested state. Is possible. On the other hand, in the normal state, it is possible to reduce the processing load of the recognition process by not subjecting the side portion of the lane to the license plate recognition process.

In the above description, an example in which the recognition process of the lane area and the side area is performed after the cutting of the lane area and the side area has been described, but the cutting and the recognition process may be performed in order for each area. For example, after the image selection unit 240 cuts out the lane area and the recognition unit 248 performs license plate recognition processing on the lane area, the image selection unit 240 cuts out the side area and the recognition unit 248 in the side area. On the other hand, the license plate recognition process may be performed. On the contrary, after the image selection unit 240 cuts out the side area and the recognition unit 248 performs the license plate recognition process on the side area, the image selection unit 240 cuts out the lane area and the recognition unit 248 cuts out the lane area. The license plate may be recognized. Further, since the two-wheeled vehicle traveling in the regions Y1 and Y2 is considered to be traveling at a relatively high speed, a recognition process different from the license plate recognition process in the lane region X in the congested state may be performed. For example, similar to the number plate recognition process in the lane region X in the normal state, the frame rate is higher than the frame rate used in the lane region X in the congested state, such as the frame rate of the moving image stored in the moving image storage unit 224. Using the frame image acquired at the frame rate, the frame image most suitable for the recognition of the number plate may be selected and recognized from the plurality of frame images in which the vehicle 10b, which is a two-wheeled vehicle, is detected.

(Second modification)
In the above, an example in which the traffic jam determination unit 236 determines the congestion state of the road surface based on the average moving speed of the vehicle 10 has been described. The second modification relates to another method for determining the congestion state of the road surface.

For example, the communication unit 220 may receive the congestion information from the server that distributes the congestion information, and the congestion determination unit 236 may determine the congestion state of the road surface based on the congestion information received by the communication unit 220. Alternatively, if there is statistical data such as the day and time zone when congestion occurs at the position where the road surface image pickup device 16 is installed, the first time zone in which the operation mode in the normal state is enabled based on the statistical data. , And a second time zone in which the congested operation mode is enabled may be set. Then, the traffic jam determination unit 236 may determine that the road surface is in a normal state in the first time zone, and may determine that the road surface is in a normal state in the first time zone. ..

According to the second modification, the vehicle detection unit 228 and the speed measurement unit 232 do not have to perform vehicle detection processing and average moving speed calculation processing in a traffic jam state. Therefore, according to the second modification, it is possible to reduce the processing load of the license plate recognition device 20 in the congested state.

(Third modification example)
In the above, an example in which the congestion state has two stages of a normal state and a congestion state has been described. However, the stage of the congested state is not limited to two stages, and for example, the stage of the congested state may be three or more stages such as a normal state, a light traffic jam state, and a heavy traffic jam state. In this case, the image selection unit 240 may select a frame image from the moving image at a lower frame rate as the degree of congestion becomes heavier. According to such a configuration, it is possible to reduce the processing load of the license plate recognition device 20 as a whole.

(Fourth modification)
In the above, an example of applying the embodiment of the present invention to one lane has been described, but the embodiment of the present invention can also be applied to a plurality of lanes. For example, by providing a plurality of image pickup units 18 on the road and analyzing the moving images obtained by each image pickup unit 18, it is possible to perform license plate recognition processing for a plurality of lanes.

<6. Hardware configuration>
Each embodiment of the present invention has been described above. Information processing such as vehicle detection and license plate recognition described above is realized by the cooperation between the software and the hardware of the license plate recognition device 20 described below.

FIG. 9 is a block diagram showing a hardware configuration of the license plate recognition device 20. The number plate recognition device 20 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and a host bus 204. Further, the number plate recognition device 20 includes a bridge 205, an external bus 206, an interface 207, an input device 208, a display device 209, an audio output device 210, a storage device (HDD) 211, a drive 212, and the like. It is provided with a network interface 215.

The CPU 201 functions as an arithmetic processing device and a control device, and controls the overall operation in the license plate recognition device 20 according to various programs. Further, the CPU 201 may be a microprocessor. The ROM 202 stores programs, calculation parameters, and the like used by the CPU 201. The RAM 203 temporarily stores a program used in the execution of the CPU 201, parameters that appropriately change in the execution, and the like. These are connected to each other by a host bus 204 composed of a CPU bus or the like. By the collaboration between the CPU 201, ROM 202 and RAM 203 and the software, the above-mentioned functions such as the vehicle detection unit 228, the speed measurement unit 232, the traffic jam determination unit 236, the image selection unit 240 and the recognition unit 248 can be realized.

The host bus 204 is connected to an external bus 206 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 205. It is not always necessary to separately configure the host bus 204, the bridge 205, and the external bus 206, and these functions may be implemented in one bus.

The input device 208 is an input means for the user to input information such as a mouse, a keyboard, a touch panel, a button, a microphone, a sensor, a switch, and a lever, and an input that generates an input signal based on the input by the user and outputs the input signal to the CPU 201. It consists of a control circuit and so on. By operating the input device 208, the user of the license plate recognition device 20 can input various data to the license plate recognition device 20 and instruct the processing operation.

The display device 209 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device, and a lamp. Further, the audio output device 210 includes an audio output device such as a speaker and headphones.

The storage device 211 is a data storage device configured as an example of the storage unit of the license plate recognition device 20 according to the present embodiment. The storage device 211 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded on the storage medium, and the like. The storage device 211 is composed of, for example, an HDD (Hard Disk Drive) or SSD (Solid Storage Drive), or a memory having an equivalent function. The storage device 211 drives the storage and stores programs and various data executed by the CPU 201. The storage device 211 corresponds to the moving image storage unit 224 and the selected image storage unit 244.

The drive 212 is a reader / writer for a storage medium, and is built in or externally attached to the license plate recognition device 20. The drive 212 reads the information recorded in the removable storage medium 24 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 203 or the storage device 211. The drive 212 can also write information to the removable storage medium 24.

The network interface 215 is, for example, a communication interface configured by a communication device or the like for connecting to the network 12. Further, the network interface 215 may be a wireless LAN (Local Area Network) compatible communication device or a wire communication device that performs wired communication.

<7. Supplement>
Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person having ordinary knowledge in the field of the art to which the present invention belongs can come up with various modifications or modifications within the scope of the technical idea described in the claims. , These are also naturally understood to belong to the technical scope of the present invention.

For example, each step in the processing of the license plate recognition device 20 of the present specification does not necessarily have to be processed in chronological order in the order described as a flowchart. For example, each step in the processing of the license plate recognition device 20 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

Further, it is possible to create a computer program for causing the hardware such as the CPU, ROM, and RAM built in the license plate recognition device 20 to exhibit the same functions as each configuration of the license plate recognition device 20 described above. A storage medium for storing the computer program is also provided.

10 Vehicle 12 Network 16 Road surface image pickup device 17 Support frame section 18 Image pickup section 20 License plate recognition device 220 Communication section 224 Dynamic image storage section 228 Vehicle detection section 232 Speed measurement section 236 Congestion determination section 240 Image selection section 244 Selection image storage section 248 Recognition unit

Claims (11)

Equipped with a recognition unit that recognizes the license plate from the moving image of the road surface on which the moving object travels.
The recognition unit
When the congested state of the moving object on the road surface is the first state, the license plate recognition process is performed on the first image selected from the moving images having the first frame rate.
When the congestion state of the moving object on the road surface is a second state in which the degree of congestion is higher than that in the first state, a moving image having a second frame rate lower than the first frame rate is configured. A license plate recognition device that performs license plate recognition processing on a second image. The license plate recognition device further includes an image selection unit that selects the first image from the image group in which the moving object is detected among the image groups constituting the moving image having the first frame rate. The license plate recognition device according to claim 1. The image selection unit is an image corresponding to an intermediate time-series position between the image in which the moving object is first detected and the image in which the moving object is detected last in the image group in which the moving object is detected. The number plate recognition device according to claim 2, wherein the image is selected as the first image. The road surface has one or more lanes and has one or more lanes.
The recognition unit
In the first state, the license plate recognition process is performed on the region corresponding to one lane in the first image.
According to claims 1 to 3, in the second state, the license plate recognition process is performed on the region corresponding to the first lane and the region corresponding to the side portion of the first lane in the second image. The license plate recognition device according to any one of the items. The license plate recognition device according to claim 4, wherein the moving image of the road surface on which the moving object travels is a moving image obtained by imaging the road surface from a side opposite to the traveling direction of the moving object. The license plate recognition device according to any one of claims 1 to 5, further comprising a determination unit for determining a congestion state of the moving object on the road surface. The license plate recognition device is
A detection unit that performs detection processing of the moving object on the image group constituting the moving image of the road surface, and
It further has a speed measuring unit that measures the moving speed of the moving object based on the change in the detection position of the moving object between images.
The license plate recognition device according to claim 6, wherein the determination unit determines a congestion state of the moving object on the road surface based on the result of measurement by the speed measurement unit. The license plate recognition device further includes an information acquisition unit that acquires information indicating a congestion state of the road surface.
The license plate recognition device according to claim 6, wherein the determination unit determines a congestion state of the moving object on the road surface based on the information acquired by the information acquisition unit. The determination unit determines that the congested state of the moving object on the road surface is the first state in the first time zone, and the congested state of the moving object on the road surface in the second time zone. The license plate recognition device according to claim 6, wherein the license plate recognition device is determined to be in the second state. Includes license plate recognition processing from moving images of the road surface on which moving objects travel.
Performing the license plate recognition process
When the congested state of the moving object on the road surface is the first state, the license plate recognition process is performed on the first image selected from the moving images having the first frame rate.
When the congestion state of the moving object on the road surface is a second state in which the degree of congestion is higher than that in the first state, a moving image having a second frame rate lower than the first frame rate is configured. Performing license plate recognition processing for the second image and
License plate recognition method, including. Computer,
Equipped with a recognition unit that recognizes the license plate from the moving image of the road surface on which the moving object travels.
The recognition unit
When the congested state of the moving object on the road surface is the first state, the license plate recognition process is performed on the first image selected from the moving images having the first frame rate.
When the congestion state of the moving object on the road surface is a second state in which the degree of congestion is higher than that in the first state, a moving image having a second frame rate lower than the first frame rate is configured. A program for functioning as a license plate recognition device that performs license plate recognition processing on a second image.

Images