JP2016021200A - Determination method, determination system and determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a determination method capable of improving detection accuracy of person.SOLUTION: The determination method includes steps of: acquiring a piece of detection information relevant to the detection of a person in a first image and a piece of recognition information relevant to the recognition of an identification code for identifying a vehicle in a second image; and, when the difference between the detection time included in the detection information and the recognition time included in the recognition information is within a predetermined time, determining whether or not the image is a person based on the detection information and the recognition information.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a determination method, a determination system, and a determination apparatus.

  2. Description of the Related Art Conventionally, there has been known an image recognition device that can discriminate between a bicycle and a two-wheeled vehicle, or can distinguish between an occupant and a pedestrian of a bicycle or two-wheeled vehicle (see, for example, Patent Document 1). This image recognition apparatus captures an object (bicycle, two-wheeled vehicle, occupant, pedestrian, etc.) with a camera, obtains a moving speed from the captured image, and determines the type of the object based on the moving speed.

JP 2007-249841 A

  In the image recognition apparatus described in Patent Document 1, there is a case where it is erroneously recognized when it is different from the assumed speed, and the detection accuracy of a person such as an occupant or a pedestrian is insufficient.

  The present invention has been made in view of the above circumstances, and provides a determination method, a determination system, and a determination apparatus that can improve human detection accuracy.

  The determination method of the present invention includes: an acquisition step of acquiring detection information related to human detection in a first image; and recognition information related to recognition of an identification code for identifying a vehicle in a second image; A determination step of determining whether or not the person is a person based on the detection information and the recognition information when the difference between the detection time included in the detection information and the recognition time included in the recognition information is within a predetermined time. And having.

  The determination system of the present invention generates a first imaging unit that captures a first image, a second imaging unit that captures a second image, and detection information related to human detection in the first image. A recognition processing unit that generates recognition information related to recognition of an identification code for identifying a vehicle in the second image, a detection time included in the detection information, and a recognition included in the recognition information And a determination unit that determines whether the person is a person based on the detection information and the recognition information when the difference from the time is within a predetermined time.

  The determination apparatus according to the present invention includes: an acquisition unit that acquires detection information related to human detection in a first image; and recognition information related to recognition of an identification code for identifying a vehicle in a second image; If the difference between the detection time of the person included in the detection information and the recognition time of the identification code included in the recognition information is within a predetermined time, is the person based on the detection information and the recognition information? And a determination unit for determining whether or not.

  According to the present invention, human detection accuracy can be improved.

The schematic diagram which shows the structural example of the determination system in embodiment (A), (B) The schematic diagram which shows an example of arrangement | positioning relationship with each camera installed on the road or the road side in embodiment The block diagram which shows the structural example of each apparatus in the determination system in embodiment. The block diagram which shows the 1st modification of a structure of each apparatus in the determination system in embodiment. The block diagram which shows the 2nd modification of a structure of each apparatus in the determination system in embodiment. (A)-(C) The schematic diagram which shows an example of the captured image imaged with each camera in embodiment. The schematic diagram which shows the example of a screen displayed on the monitor of the monitoring apparatus in embodiment The schematic diagram which shows the operation example of the determination system when the approach target in the embodiment appears on the exit side of the expressway The schematic diagram which shows the operation example of the determination system when the approach target in the embodiment appears on the entrance side of the expressway The schematic diagram which shows the operation example of the determination system based on the image processing result in the exit side of the expressway in embodiment The schematic diagram which shows the operation example of the determination system based on the image processing result in the entrance side of the expressway in embodiment The flowchart which shows the operation example of the integrated determination process part in embodiment The flowchart which shows an example of the determination process by the integrated determination process part in embodiment The flowchart which shows an example of the determination process by the integrated determination process part in embodiment (continuation of FIG. 13).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  In the determination system of the following embodiment, for example, based on at least one of person detection and license plate recognition, an object that is prohibited from entering the highway (for example, a pedestrian, a bicycle, a small motorcycle) or reverse running This is applied to a system that suppresses erroneous entry of an object to be performed.

  FIG. 1 is a schematic diagram illustrating a configuration example of a determination system 100 according to the embodiment. The determination system 100 includes, for example, person detection cameras 101 and 102, a number recognition camera 103, a person detection processing unit 110, a number recognition processing unit 115, an integrated determination processing unit 120, a recording device 130, a monitoring device 140, an alarm indicator 151, A speaker 152 and a warning display board 153 are provided. Each device of the determination system 100 is connected via a communication network, for example.

  The alarm display 151, the speaker 152, and the warning display board 153 are an example of the notification device 150 that issues predetermined warning information (alarm). The integrated determination processing unit 120 is an example of a determination device.

  The person detection camera 101 is an IR (InfraRed) camera, and is installed on the road (on the road) or on the road side, for example, and images an object (for example, a vehicle or a person) using infrared rays (for example, far infrared rays).

  Specifically, the vehicle as an object includes, for example, a four-wheeled vehicle (also simply referred to as a four-wheeled vehicle or a four-wheeled vehicle), a two-wheeled vehicle (also simply referred to as a two-wheeled vehicle), or a bicycle. Further, the person as the object includes, for example, a vehicle occupant and a pedestrian. The two-wheeled vehicle includes, for example, a large motorcycle (also simply referred to as large) and a small motorcycle (also simply referred to as small). In the present embodiment, the large motorcycle is, for example, a two-wheeled vehicle having a total displacement of 125 CC or more, and the small motorcycle is, for example, a two-wheeled vehicle having a total displacement of less than 125 CC.

  Since the human detection camera 101 captures an image using infrared rays, the image quality of the captured image changes depending on, for example, the weather, the time zone, and the surrounding lighting environment, and the human detection accuracy decreases, as compared with the case where visible light is used. This can be suppressed.

  The person detection camera 102 is installed on the road or on the road side, for example, and images an object using visible light. Since the human detection camera 102 uses visible light, for example, the human detection camera 102 can capture an image while suppressing the influence of heat around the object as compared with the case of using infrared rays. For example, it is possible to suppress a decrease in human detection accuracy even when the outside air temperature is close to the human body temperature or in the case of rainfall.

  Note that either one of the person detection cameras 101 and 102 may not be provided.

  The number recognition camera 103 is installed on the road or on the road side, for example, and images a target object. For example, the number recognition camera 103 captures an image of the vehicle including a license plate attached to the vehicle.

  The person detection processing unit 110 is installed, for example, in a predetermined area (for example, a toll gate) in a predetermined area (access limit area, for example, a highway) where entry is restricted, and is acquired from at least one of the person detection cameras 101 and 102. The presence or absence of a person is detected in the displayed image.

  For example, the number recognition processing unit 115 is installed on the roadside and performs number recognition on an image captured by the number recognition camera 103. In the number recognition, the number recognition processing unit 115 recognizes the character information described in the position of the license plate in the captured image, and specifies the character described in the license plate. The license plate is an example of an identification code for identifying the vehicle. For example, the vehicle may be identified by detecting the vehicle type by image recognition.

  The integrated determination processing unit 120 is installed in, for example, a traffic control room. For example, the integrated determination processing unit 120 determines whether the target object is a person or a vehicle based on the person processing result by the person detection processing unit 110 and the number recognition result by the number recognition processing unit 115. Based on the determination result, it is determined whether or not to issue the alarm using the alarm device 150. For example, if the integrated determination processing unit 120 determines that there is an object that is prohibited from entering the highway (for example, a person, a bicycle, or a small motorcycle), the integrated determination processing unit 120 issues an alarm via the alarm 150.

  For example, the recording device 130 is installed at a toll gate on an expressway, and records (records) images captured by the person detection cameras 101 and 102 at a predetermined timing. The recording apparatus 130 is, for example, an NDR (Network Disk Recorder) and includes an HDD (Hard Disk Drive). Note that the recording device 130 may include a recording medium other than the HDD. For example, the recording device 130 may record images of the person detection cameras 101 and 102 at all times.

  The monitoring device 140 is installed in, for example, a traffic control room and is appropriately operated by a traffic controller (operator). The monitoring device 140 is composed of, for example, a PC (Personal Computer) and a server device. In addition, a monitor 145 is connected to the monitoring device 140, and the traffic controller can check various images from the monitor 145.

  For example, a real-time image or a past image captured by the person detection cameras 101 and 102 may be displayed on the monitor 145. The monitor 145 may display a real-time image captured by the number recognition camera 103 or a past image. The monitor 145 may display a real-time image or a past image captured by a situation monitoring camera (not shown).

  The alarm indicator 151 is installed in a traffic control room, for example, and includes a patrol lamp. For example, when an entry to a restricted access area (for example, an expressway) (for example, erroneous entry) occurs, the alarm indicator 151 causes the traffic controller to generate an alarm by display. Thereby, for example, the traffic controller can quickly recognize the occurrence of the abnormality.

  The speaker 152 is installed, for example, in a traffic control room, on the road, or on the road side. For example, when an entry to the restricted access area occurs, the speaker 152 generates an alarm by voice to the traffic controller, the person who has entered, the vehicle, or a surrounding vehicle. Thereby, for example, a traffic controller, a person who has entered erroneously, a vehicle, or an occupant of a surrounding vehicle can quickly recognize the occurrence of an abnormality.

  The warning display panel 153 is installed on the road or on the road side, for example, and generates an alarm by display for the person who entered the vehicle, the vehicle, or the surrounding vehicle when entering the restricted access area. Thereby, for example, a person who has entered erroneously, a vehicle, or an occupant of the surrounding vehicle can quickly recognize the occurrence of the abnormality.

  The traffic control room monitors one or more toll gates (highway entrances). That is, the monitoring device 140 (integrated determination processing unit 120) installed in the traffic control room has one or more toll gates and each device installed in the vicinity thereof (for example, the person detection processing unit 110, the number recognition processing unit 115). ). For example, if the traffic control room organizes a small number of toll booths, the increase in misinformation on normal entry will have little impact on the traffic control room observers, but there will be a large number of traffic control rooms (for example, 100 locations). When collecting toll gates, increasing the number of false alarms for normal entry increases the impact on traffic control room supervisors (unnecessary work increases). For this reason, in the present embodiment, the aforementioned false alarms and misreports are reduced in the present embodiment so that unnecessary work is reduced even in a large number of toll gates.

Next, an arrangement example of each camera and an example of setting the angle of view of each camera will be described.
FIG. 2A is a schematic diagram illustrating an example of an arrangement relationship with each camera installed on the road or on the road side.

  In the determination system 100, the person detection cameras 101 and 102 and the number recognition camera 103 are aligned and arranged so that the center positions in the road traveling direction in the captured images captured by the cameras substantially coincide.

  The person detection cameras 101 and 102 and the number recognition camera 103 take a picture from a general road toward the main road of the expressway. This is because, when shooting from a highway toward a general road, an intersection of a general road enters the shooting range, and unnecessary detection (detection of people at an intersection, etc.) may occur.

  The height at which the person detection cameras 101 and 102 and the number recognition camera 103 are installed is set in consideration of a vehicle passing through the road. The angle of view of the person detection cameras 101 and 102 is, for example, a width L3 per unit (for example, 4 (m): one lane). Therefore, the person detection camera 101, 102 can detect a person in one lane. The person detection cameras 101 and 102 are installed with their orientation and height adjusted so that the position on the road whose distance from the person detection cameras 101 and 102 is the length L1 is included in the imaging range. The length L1 is determined by the focal length of the lenses included in the person detection cameras 101 and 102.

  The number recognition camera 103 includes, for example, two imaging units. The angle of view of the two imaging units is a total width of 4 (m) (for one lane). Therefore, it is possible to recognize the license plate of the vehicle in one lane by the two imaging units of the number recognition camera 103. The number recognition camera 103 is installed with its orientation and height adjusted so that the position on the road whose distance from the number recognition camera 103 is the length L2 is included in the imaging range. The length L2 is determined by the focal length of the lens of the number recognition camera 103.

  Note that the person detection cameras 101 and 102 may be capable of capturing a plurality of lanes with a single camera, or may be capable of capturing a single lane with a plurality of cameras. The number recognition camera 103 may be capable of imaging one lane or a plurality of lanes with one imaging unit, or may be capable of imaging one lane with three or more imaging units.

  Moreover, although it illustrated that each camera was comprised by the separate housing | casing in the above, each camera may be integrated and comprised by one housing | casing. FIG. 2B is a schematic diagram illustrating an example of an arrangement relationship with the integrated camera 104 configured by one housing that is installed on the road.

  The integrated camera 104 includes person detection cameras 101 and 102 and a number recognition camera 103. The person detection cameras 101 and 102 and the number recognition camera 103 are oriented in the same direction and aligned so that the center positions in the road traveling direction in the captured images captured by the respective cameras substantially coincide. As a result, even if the imaging position (mainly in the direction of gravity) of the camera shifts due to long-term operation, the relative positions of the person detection cameras 101 and 102 and the number recognition camera 103 do not change, and therefore it can be easily operated for a long time. .

  For example, when the integrated camera 104 is installed at the center in the width direction on the road (for example, near the center of the lane), it suppresses that a person walking on the road or an occupant of a vehicle traveling on the road is concealed by a large vehicle. it can. Note that one integrated camera 104 may be able to capture a plurality of lanes.

  Each camera may be installed at a position different from the positions illustrated in FIGS. 2A and 2B exemplify the arrangement of a set of cameras for imaging one lane, but when imaging a plurality of lanes, a plurality of sets of cameras may be prepared.

Next, functions of each device in the determination system 100 will be described.
FIG. 3 is a block diagram illustrating a configuration example of each device in the determination system 100. FIG. 3 illustrates a case where the person detection camera 101 is used as the person detection camera and the person detection camera 102 is not used. In FIG. 3, the description similar to the content described in FIG. 1 is omitted or simplified.

  The determination system 100 includes, for example, a person detection camera 101, a number recognition camera 103, a person detection processing unit 110, a number recognition processing unit 115, an integrated determination processing unit 120, a recording device 130, a monitoring device 140, and a notification device 150.

  The person detection processing unit 110 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). For example, the CPU implements each function of the person detection processing unit 110 by executing a program stored in the ROM or RAM.

  The person detection processing unit 110 includes, for example, a control unit 111, a detection unit 112, and a communication unit 113. The control unit 111 supervises the processing of the person detection processing unit 110. The detection unit 112 detects information relating to a person (for example, a pedestrian or a vehicle occupant) included in the image based on an image captured by the person detection camera 101, for example, by a predetermined image recognition process. The information relating to the person includes, for example, information on the presence / absence of the person in the image and the moving direction (orientation) of the person.

  For example, the communication unit 113 sequentially receives images captured by the person detection camera 101. For example, the communication unit 113 transmits information on the person detection result and an image captured by the person detection camera 101 to the integrated determination processing unit 120 and the recording device 130. It should be noted that a person detection result may be superimposed on an image captured by the person detection camera 101 in the information sent to the recording device 130.

  The person detection processing unit 110 may include a display unit (not shown) that displays various types of information (for example, information in which a person detection result is superimposed on a captured image).

  The information of the person detection result includes, for example, the presence / absence of a person detected in the image captured by the person detection camera 101, the moving direction (direction) of the person, the time when the person was detected, and the person in the image (frame). Information on the position (coordinates) made.

  The number recognition processing unit 115 includes, for example, a CPU, a ROM, and a RAM. For example, the CPU implements each function of the number recognition processing unit 115 by executing a program stored in the ROM or RAM.

  The number recognition processing unit 115 includes, for example, a control unit 116, a recognition unit 117, and a communication unit 118. The control unit 116 supervises the processing of the number recognition processing unit 115. Based on the image captured by the number recognition camera 103, the recognition unit 117 recognizes information related to the license plate included in the image (number recognition) by a predetermined image recognition process. The information related to the license plate includes, for example, information on the number written on the license plate, the vehicle type of the license plate, the moving direction (direction) of the license plate, and the moving speed of the license plate.

  For example, the communication unit 118 receives an image captured by the number recognition camera 103. Based on the recognition result of the number recognition processing unit 115, the communication unit 118 determines, for example, information on the number recognition result (number information) and an image (still image) captured by the number recognition camera 103 when the number is recognized. Transmit to the processing unit 120. The communication unit 118 may always transmit the information and the image to the integrated determination processing unit 120, not only when the number is recognized.

  The information of the number recognition result includes, for example, the type (vehicle type) of the vehicle license plate imaged by the number recognition camera 103, the moving direction (direction) of the license plate (vehicle), and the speed of the license plate (vehicle). Contains information. The license plate model includes, for example, a four-wheeled vehicle, a large motorcycle, and a small motorcycle. The number recognition result information includes, for example, information on the time at which the number is recognized and the position (coordinates) at which the number in the image (frame) is recognized.

  The integrated determination processing unit 120 includes, for example, a CPU, a ROM, and a RAM. For example, the CPU implements each function of the integrated determination processing unit 120 by executing a program stored in the ROM or RAM.

  The integrated determination processing unit 120 includes, for example, a communication unit 121, a determination unit 122, and a reporting unit 123. The communication unit 121 acquires (for example, receives) the information on the person detection result from the person detection processing unit 110 and the information on the number recognition result from the number recognition processing unit 115.

  The determination unit 122 determines whether or not the reporting unit 123 needs to report based on at least the acquired number recognition result and the person detection result. The determination result by the determination part 122 is illustrated by FIG. 10, FIG. 11, for example.

  For example, the determination unit 122 determines the presence or absence of a person detected by the person detection processing unit 110 based on at least one of the number recognition result and the person detection result. That is, the determination unit 122 supplements the detection result by the person detection processing unit 110 and determines the validity of the person detection result. Thereby, human detection accuracy can be improved. For example, the determination unit 122 determines whether or not an alarm should be issued based on at least one of the number recognition result and the person detection result.

  When the determination unit 122 determines that the notification is necessary, the notification unit 123 instructs the recording device 130 and the notification device 150 to notify (report instruction) so as to issue an alarm.

  The monitoring device 140 includes, for example, a CPU, a ROM, and a RAM. For example, the CPU implements each function (for example, a monitoring function, a notification function such as alarm notification) that the monitoring device 140 has by executing a program stored in the ROM or RAM.

  The alarm device 150 includes, for example, an alarm indicator 151, a speaker 152, or a warning display board 153. For example, the notification device 150 issues a predetermined alarm (for example, display, voice output, vibration) based on a notification instruction from the integrated determination processing unit 120.

  FIG. 4 is a block diagram illustrating a first modification of the configuration of each device in the determination system 100. 4, the description of the same configuration as that shown in FIG. 3 is omitted or simplified. FIG. 4 illustrates a case where the determination system 100 includes the person detection camera 102 and does not include the person detection camera 101 as compared with the configuration example of FIG.

  FIG. 5 is a block diagram illustrating a second modification of the configuration of each device in the determination system 100. In FIG. 5, the description of the same configuration as the configuration shown in FIG. 3 is omitted or simplified.

  In FIG. 5, the determination system 100 includes both person detection cameras 101 and 102 as compared with the configuration example of FIG. 3. In addition to the functions of the person detection processing unit 110 described above, the person detection processing unit 110 has a camera selection function for selecting which captured image of the person detection camera 101 or 102 is to be used.

  The person detection processing unit 110 selects a captured image to be adopted based on a predetermined condition. The predetermined conditions include, for example, the contrast of the image captured by the person detection cameras 101 and 102, the temperature, the humidity, the precipitation amount, and the moving speed of the person in the captured image. The person detection processing unit 110 may include, for example, a weather sensor (not shown) that provides information related to weather (for example, temperature, humidity, and precipitation), or receives information related to weather from a weather server via a communication network. You may get it. The image to be adopted may be selected based on the result after performing image recognition, or may be selected in advance based on information on weather.

  For example, the person detection processing unit 110 adopts an image captured by the person detection camera 102 under conditions of high temperature or rain, and adopts an image captured by the person detection camera 101 under conditions of room temperature. For example, the person detection processing unit 110 employs an image captured by the person detection camera 102 when the movement speed of the person is high, and employs an image captured by the person detection camera 101 when the movement speed of the person is low. . Thereby, the influence of the blur which generate | occur | produces when a person moves at high speed can be suppressed, and a person's detection accuracy can be improved.

  The person detection processing unit 110 may be basically set to use the person detection camera 101, and may adopt an image captured by the person detection camera 102 when the image captured by the person detection camera 101 does not satisfy the predetermined condition. . The person detection processing unit 110 may generate, for example, a new image by combining both images based on images captured by the person detection cameras 101 and 102, and transmit the combined image to the integrated determination processing unit 120 or the like. Note that the person detection camera 102 captures an image using visible light, and thus is easily affected by an artifact (for example, illumination).

  Next, association of objects included in images captured by the cameras will be described.

  FIG. 6A is a schematic diagram illustrating an example of an image captured by the person detection camera 101 or 102. FIG. 6B is a schematic diagram illustrating an example of an image captured by one imaging unit (for example, an imaging unit disposed on the left side) of the number recognition camera 103. FIG. 6C is a schematic diagram illustrating an example of an image captured by the other imaging unit (for example, an imaging unit disposed on the right side) of the number recognition camera 103.

  6 (A) and 6 (C), the same object is reflected in each captured image. That is, in FIG. 6A, the person 101a is detected, and in FIG. 6C, the license plate 103b is recognized.

  As shown in FIGS. 6A to 6C, the person detection cameras 101 and 102 and the number recognition camera 103 are configured so that the road traveling direction in the captured image as described above (the Y direction in FIGS. 6A to 6C). ) Is adjusted and installed so that the center positions substantially coincide. That is, for the same object, the center position is set so that person detection and number recognition are performed within a predetermined time. For example, if the timings of person detection and number recognition are largely shifted for the same object, the same object is recognized as different objects. For this reason, person detection and number recognition are performed for a same object during a predetermined time.

  In the integrated determination processing unit 120, the communication unit 121 receives information on coordinates in an image where a person is detected by the person detection processing unit 110 and information on coordinates in an image whose number is recognized by the number recognition processing unit 115. May be. The determination unit 122 may estimate the position of an object (such as an intruder) in the road crossing direction (the X direction in FIGS. 6A to 6C) based on both received coordinates.

  When the time between the time at which the person is detected and the time at which the number is recognized is within a predetermined time, the determination unit 122 determines that the person-detected object and the number-recognized object are the same object (for example, a vehicle The integrated determination is made in association with the vehicle occupant. The integrated determination includes, for example, determination as to whether or not to issue a report.

  When the movement direction (for example, the Y direction) of the person detected by the person and the movement direction (for example, the Y direction) of the license plate whose number has been recognized substantially coincide with each other, the determination unit 122 recognizes the number of the detected object. The objects may be associated with each other as the same object (for example, the vehicle and the vehicle occupant) and integrated determination may be performed.

  When the distance between the person-detected coordinate and the number-recognized coordinate in the road crossing direction (X direction) is equal to or less than a predetermined distance, the determination unit 122 determines the person-detected object and the number-recognized object. The integrated determination may be made in association with the same object (for example, the vehicle and the vehicle occupant). Note that the determination unit 122 does not have to use the association index for the position in the road traveling direction (Y direction) in consideration of the angle of view deviation.

  If the determination unit 122 satisfies the condition regarding the time or the condition regarding the distance or the direction (orientation) as well as the condition regarding the time, the determination unit 122 associates the person-detected object and the object whose number is recognized as the same object. Good.

  When there are a plurality of human-detected objects and number-recognized objects, the determination unit 122 determines the conditions regarding the distance, time, and direction, and the human-detected object and the number-recognized object When there is one each, determination of conditions regarding the distance, time, and direction may be omitted.

  In this way, by associating the object whose person has been detected with the object whose number has been recognized, for example, it is possible to suppress false or misreporting when erroneously entering the expressway. For example, if a person-detected object is a person and the license plate whose number is recognized is a license plate for a large motorcycle, the person detection result and the number recognition result are associated with each other, and it can be recognized as a large motorcycle. It may not be reported. On the other hand, as in the prior art, when determining whether or not to issue a report only by detecting a person, even if it is a large motorcycle, a report may be issued based on the person detection. That is, this association can suppress such misinformation. Suppression of false alarms and misreports is even more useful when, for example, a large number of person detection cameras 101 and 102 and number recognition cameras 103 are installed on an expressway.

Next, a screen displayed on the monitor 145 of the monitoring device 140 will be described.
FIG. 7 is a schematic diagram illustrating an example of a screen displayed on the monitor 145.

  For example, the monitoring device 140 may simultaneously display a plurality of images in the display areas 141 to 144 of the monitor 145 divided into four. This image includes, for example, a still image or a moving image.

  In the display area 141, for example, among images captured by the person detection cameras 101 and 102, an image held in the recording device 130 and detected by the person detection processing unit 110 is displayed. In this case, the monitoring device 140 repeatedly displays an image to be displayed (for example, a moving image) in a detection period including a time point when a person is detected in the display area 141.

  In the display area 142, for example, images (real-time images) captured by the person detection cameras 101 and 102 and acquired from the person detection cameras 101 and 102 are displayed.

  In the display area 143, for example, an image whose number is recognized by the number recognition processing unit 115 among images captured by the number recognition camera 103 is displayed. This image is, for example, a still image, and includes an image obtained from the integrated determination processing unit 120 when the number is recognized. The image whose number is recognized may be recorded in the recording device 130, and the monitoring device 140 may acquire the image whose number is recognized from the recording device 130.

  In the display area 144, for example, an image captured by a situation confirmation camera (not shown) is displayed. This image is, for example, a real-time image sent from a situation confirmation camera. For example, the monitoring device 140 may include an operation unit (not shown). For example, the monitoring device 140 may receive an operation input to the operation unit by the traffic controller, and may control the angle of view, pan, tilt, zoom, and the like of the situation confirmation camera based on the operation input. Thereby, for example, the traffic controller can freely remotely control the angle of view of the situation monitoring camera, and can easily visually confirm the situation of the road to be monitored. Further, for example, it is possible to easily follow and confirm an object that has moved out of the screen by moving at high speed.

  According to the screen configuration shown in FIG. 7, the operator can quickly confirm whether the information is correct or incorrect by comparing the warning information at the time of issue with the information displayed on the screen. In addition, when the operator determines that it is a false report, a predetermined operation unit (for example, a report release button) of the monitoring device 140 may be operated (for example, clicked). As a result, the alarm sound and the alarm display can be quickly stopped, and the false alarm can be easily dealt with.

  Next, an operation example of the determination system 100 will be described.

  Here, as an example, the person detection cameras 101 and 102 and the number recognition camera 103 are installed in a direction in which an image is taken from behind with respect to the direction of entering the expressway (the direction of the main line from the exit or the entrance).

  The person detection cameras 101 and 102 and the number recognition camera 103 are installed, for example, between a toll booth and a general road, and are installed near the general road so that an erroneous entry person notices at an early stage. On the highway, for example, people, small motorcycles, and bicycles are restricted from entering the main line.

  Further, as an example, a vehicle that travels at a speed of less than 20 km / h is a slow vehicle, and a vehicle that travels at a speed of 20 km / h or more is a fast vehicle. As an example, a person who is not in the vehicle and a bicycle are collectively referred to as a “person”. In the following description, “person” indicates a person who is not in the vehicle and a bicycle. As an example, a person detection camera 101 is used as a person detection camera.

  In a four-wheeled vehicle, license plates are installed before and after the vehicle. In a two-wheeled vehicle, a license plate is installed behind the vehicle, and no license plate is installed in front of the vehicle.

  FIG. 8 is a schematic diagram illustrating an operation example of the determination system 100 for an entry target (object) on the exit side of the expressway. FIG. 9 is a schematic diagram illustrating an operation example of the determination system 100 for an entry target on the entrance side of the expressway.

  8 and 9 show how the entry target is distinguished for each case (case), and whether or not the integrated determination processing unit 120 reports to the entry target person. ing.

  The case H1 will be described. When a person enters (enters) from the exit of the expressway, the person detection processing unit 110 detects the presence of the person, and detects that the person is moving (moving) from the exit of the expressway toward the main line. Detect. The number recognition processing unit 115 does not recognize a number because a person does not have a license plate. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case N1 will be described. When a person exits (exits) from the exit of the expressway, the person detection processing unit 110 detects the presence of the person and detects that the person is traveling from the main road to the exit. The number recognition processing unit 115 does not recognize a number because a person does not have a license plate. In this case, the integrated determination processing unit 120 determines that notification is unnecessary. That is, for example, since a person detected as an intruder may be a large motorcycle, priority can be given to reducing false alarms so as not to be issued. Note that it is not impossible to give priority to the safety of intruders.

  Case I1 will be described. When a slow small motorcycle enters from the exit of the expressway, the person detection processing unit 110 detects the presence of a person, and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of the small motorcycle from the license plate installed at the rear of the vehicle, recognizes that the vehicle (number plate) is traveling in the direction of the main line from the exit of the highway, and is 20 km / h. Recognize the speed of less than vehicle (number plate). In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case N1 will be described. When a slow small motorbike exits from the exit of the expressway, the person detection processing unit 110 detects the presence of a person and detects that the person is moving from the main road to the exit. The number recognition processing unit 115 does not recognize the number because the number plate installed behind the vehicle is not imaged by the number recognition camera 103. In this case, the integrated determination processing unit 120 determines that notification is unnecessary. That is, for example, there is a possibility that a small motorcycle detected and recognized as an intruder may be a large motorcycle.

  Case C1 will be described. When a fast small bike enters from the exit of the expressway, the person detection processing unit 110 does not detect the person because the person detection cameras 101 and 102 are insufficiently following the moving speed of the small bike. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 determines that notification is necessary. In case C1, since the person is not detected, it is determined whether or not the notification is necessary based on the number recognition.

  Case A1 will be described. When the fast small motorcycle exits from the exit of the expressway, the person detection processing unit 110 does not detect a person. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  In case A1, it is assumed that the person detection camera 101 that captures an image using infrared rays is used. Therefore, the person detection camera 102 that captures an image using visible light is illustrated. By using it, a person may be detected. The same applies to other cases where no person is detected by the person detection camera 101.

  Case K1 will be described. When a slow large motorcycle enters from the exit of the expressway, the person detection processing unit 110 detects the presence of a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of the large motorcycle from the license plate installed at the rear of the vehicle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and detects the number of vehicles less than 20 km per hour. Recognize speed. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case N1 will be described. When a slow large motorcycle exits from the exit of the expressway, the person detection processing unit 110 detects the presence of a person and detects that the person is traveling from the main road to the exit. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Case E1 will be described. When a fast large bike enters from the exit of the expressway, the person detection processing unit 110 does not detect the person because the person detection cameras 101 and 102 are insufficiently following the moving speed of the large bike. The number recognition processing unit 115 recognizes the number of a large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 determines that notification is necessary. In case E1, since the person is not detected, it is determined whether or not the notification is necessary based on the number recognition.

  Case A1 will be described. When the fast large motorcycle exits from the exit of the expressway, the person detection processing unit 110 does not detect a person. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Case F1 will be described. When the four-wheeled vehicle enters from the exit of the expressway, the person detection processing unit 110 does not detect an occupant in the vehicle in the images captured by the person detection cameras 101 and 102. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 determines that notification is necessary. In case F1, since no person is detected, it is determined whether or not a notification is necessary based on the number recognition.

  The case G1 will be described. When the four-wheeled vehicle exits from the exit of the expressway, the person detection processing unit 110 does not detect an occupant in the vehicle in the images captured by the person detection cameras 101 and 102. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling from the main road to the exit, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 measures the number of four-wheeled vehicles that normally exit the highway. In this case, for example, the integrated determination processing unit 120 adds the current unit to the number of units already measured.

  Case H2 will be described. When a person enters from the entrance of the expressway, the person detection processing unit 110 detects the presence of the person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 does not recognize a number because a person does not have a license plate. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case N2 will be described. When a person exits from the entrance of the expressway, the person detection processing unit 110 detects the presence of the person and detects that the person is traveling from the main road to the entrance. The number recognition processing unit 115 does not recognize a number because a person does not have a license plate. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case I2 will be described. When a slow small motorcycle enters from the entrance of the expressway, the person detection processing unit 110 detects the presence of a person, and detects that the person is moving in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the small motorcycle from the license plate installed at the rear of the vehicle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and detects the number of vehicles less than 20 km per hour. Recognize speed. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case N2 will be described. When a slow small motorbike exits from the entrance to the expressway, the person detection processing unit 110 detects that a person is present and detects that the person is moving from the main road to the entrance. The number recognition processing unit 115 does not recognize the number because the number plate installed behind the vehicle is not imaged by the number recognition camera 103. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case C2 will be described. When a fast small motorcycle enters from the entrance of the expressway, the person detection processing unit 110 does not detect the person because the person detection cameras 101 and 102 do not sufficiently follow the movement speed of the small bike. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 determines that notification is necessary. In case C2, since a person is not detected, it is determined whether or not a notification is necessary based on number recognition.

  Case A2 will be described. When the fast small motorcycle exits from the entrance of the expressway, the person detection processing unit 110 does not detect a person. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Note that in the case A2, it is assumed that the person detection camera 101 that captures an image using infrared rays is used, and thus the person detection camera 102 that captures an image using visible light is illustrated. By using it, a person may be detected. The same applies to other cases where no person is detected by the person detection camera 101.

  The case K2 will be described. When a slow large motorcycle enters from the entrance of the expressway, the person detection processing unit 110 detects the presence of a person and detects that the person is moving in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the large motorcycle from the license plate installed at the rear of the vehicle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and detects the number of vehicles less than 20 km per hour. Recognize speed. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Case N2 will be described. When a slow large motorcycle exits from the entrance of the expressway, the person detection processing unit 110 detects the presence of a person and detects that the person is moving from the main road to the entrance. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is necessary.

  Case E2 will be described. When a fast large bike enters from the entrance of the expressway, the person detection processing unit 110 does not detect the person because the person detection cameras 101 and 102 are insufficiently following the moving speed of the large bike. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Case A2 will be described. When a fast large motorcycle exits from the entrance of the expressway, the person detection processing unit 110 does not detect a person. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Case F2 will be described. When the four-wheeled vehicle enters from the entrance of the expressway, the person detection processing unit 110 does not detect an occupant in the vehicle in the images captured by the person detection cameras 101 and 102. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 measures the number of four-wheeled vehicles that normally enter the highway. In this case, for example, the integrated determination processing unit 120 adds the current unit to the number of units already measured.

  Case G2 will be described. When the four-wheeled vehicle exits from the entrance of the expressway, the person detection processing unit 110 does not detect an occupant in the vehicle in the images captured by the person detection cameras 101 and 102. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling from the main road to the entrance, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 determines that notification is necessary. In case G2, since no person is detected, it is determined whether or not the notification is necessary based on the number recognition.

  FIG. 10 is a schematic diagram illustrating an operation example of the determination system 100 based on the result of image recognition processing (person detection result, number recognition result) on the exit side of the expressway. FIG. 11 is a schematic diagram illustrating an operation example of the determination system 100 based on the result of the image recognition process on the entrance side of the expressway.

  10 and 11, for each case (case), information on the person detection result, information on the number recognition result, information on the result of the integrated determination process, and other information (remarks) are shown. The case numbers in FIGS. 10 and 11 correspond to the case numbers in FIGS.

  First, case B1 in FIG. 10 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 determines that notification is unnecessary. Thereby, for example, even when a character written on the body of a slow four-wheeled vehicle (for example, a bus) is mistakenly recognized as a character on a license plate of a small motorcycle, erroneous reporting can be suppressed.

  Case C1 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the approach target is a reverse run (erroneous approach) of a fast small motorcycle, and determines that a notification is necessary.

  Case D1 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 determines that notification is unnecessary. Thereby, for example, even when a character written on the body of a slow four-wheeled vehicle (for example, a bus) is mistakenly recognized as a character on a license plate of a large motorcycle, erroneous reporting can be suppressed.

  Case E1 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of a large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the entry target is a reverse run (erroneous entry) of a fast large motorcycle, and determines that a notification is necessary.

  Case F1 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 recognizes that the entry target is a four-wheeled vehicle reverse running (erroneous entry), and determines that a notification is necessary.

  The case G1 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling from the main road to the exit, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 recognizes that the entry target is a forward traveling of a four-wheeled vehicle and determines to measure the number of vehicles. Thereby, for example, the number of four-wheeled vehicles that normally run normally can be counted.

  The case H1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 recognizes that the entry target is a reverse run (erroneous entry) of a person, and determines that a notification is necessary.

  Case I1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 recognizes that the entry target is reverse running (erroneous entry) of a slow small motorcycle, and determines that a notification is necessary.

  Case J1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the approach target is a reverse run (erroneous approach) of a fast small motorcycle, and determines that a notification is necessary.

  Case K1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 recognizes that the approach target is reverse running (erroneous approach) of a slow large motorcycle, and determines that a notification is necessary.

  The case L1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of a large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the entry target is a reverse run (erroneous entry) of a fast large motorcycle, and determines that a notification is necessary.

  The case M1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the exit of the expressway. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling in the direction of the main line from the exit of the highway, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 ignores the person detection as a false detection, recognizes the approach target as a reverse run (mistake) of the four-wheeled vehicle based only on the number recognition result, and determines that the notification is necessary.

  Case N1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling from the main road to the exit. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 determines that notification is unnecessary. Thereby, for example, it is possible to suppress false alarms resulting from the forward running of a large motorcycle.

  Case O1 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling from the main road to the exit. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling from the main road to the exit, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 ignores the person detection as an erroneous detection, recognizes that the entry target is a forward traveling of a four-wheeled vehicle, and determines to measure the number of vehicles. Thereby, the number of four-wheeled vehicles which normally run forward can be counted.

  Next, case B2 in FIG. 11 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 determines that notification is unnecessary. Thereby, for example, even when a character written on the body of a slow four-wheeled vehicle (for example, a bus) is mistakenly recognized as a character on a license plate of a small motorcycle, erroneous reporting can be suppressed.

  Case C2 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the entry target is a forward running (erroneous entry) of a fast small motorcycle, and determines that a notification is necessary.

  Case D2 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 determines that notification is unnecessary.

  Case E2 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the entry target is a forward run of a fast large motorcycle, and determines that the notification is unnecessary.

  Case F2 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 recognizes that the entry target is a forward traveling of a four-wheeled vehicle and determines to measure the number of vehicles. Thereby, for example, the number of four-wheeled vehicles that normally run normally can be counted.

  Case G2 will be described. The person detection processing unit 110 does not detect a person. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling from the main road to the entrance, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 recognizes that the entry target is a four-wheeled vehicle reverse running (erroneous entry), and determines that a notification is necessary.

  Case H2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 recognizes that the entry target is a person's forward running (erroneous entry), and determines that reporting is necessary.

  Case I2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 recognizes that the entry target is a forward run (erroneous entry) of a slow small motorcycle, and determines that a notification is necessary.

  Case J2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the small motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that a fast small motorcycle is running forward (erroneous entry), and determines that a notification is necessary.

  The case K2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle below 20 km / h. In this case, the integrated determination processing unit 120 recognizes the approach target as a forward running of a slow large motorcycle, and determines that the notification is unnecessary.

  Case L2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the large motorcycle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle at a speed of 20 km or more. In this case, the integrated determination processing unit 120 recognizes that the entry target is a forward run of a fast large motorcycle, and determines that the notification is unnecessary.

  Case M2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling in the direction of the main line from the entrance of the expressway. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling in the direction of the main line from the entrance of the highway, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 ignores the person detection as an erroneous detection, determines that the approach target is recognized as a forward traveling of a four-wheeled vehicle, and measures the number of vehicles. Thereby, the number of four-wheeled vehicles which normally run forward can be counted.

  Case N2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling from the main road to the entrance. The number recognition processing unit 115 does not recognize the number. In this case, the integrated determination processing unit 120 recognizes that the entry target is a reverse run (erroneous entry) of a person, and determines that a notification is necessary.

  Case O2 will be described. The person detection processing unit 110 detects a person and detects that the person is traveling from the main road to the entrance. The number recognition processing unit 115 recognizes the number of the four-wheeled vehicle, recognizes that the vehicle is traveling from the main road to the entrance, and recognizes the speed of the vehicle. In this case, the integrated determination processing unit 120 ignores the person detection as an erroneous detection, recognizes the approach target as a four-wheeled vehicle reverse running (erroneous approach), and determines that a notification is necessary.

  Note that the number recognition processing unit 115 cannot recognize the number because the front of the vehicle is imaged when the vehicle is a motorcycle (small motorcycle or large motorcycle) and travels from the main road to the exit or entrance. Therefore, when the number recognition processing unit 115 recognizes the number, it is recognized that the vehicle proceeds from the exit or entrance of the expressway to the main line, and information on the number recognition result in this case is sent to the integrated determination processing unit 120.

  In addition, the number recognition processing unit 115, when the vehicle is a motorcycle and travels from the main road of the highway toward the exit or entrance, it is estimated that the character written on the body of the four-wheeled vehicle is erroneously recognized. The number recognition result information may not be transmitted to the integrated determination processing unit 120. Even in this case, the number recognition processing unit 115 transmits the information of the number recognition result to the integrated determination processing unit 120, and the integrated determination processing unit 120 makes the determination as in the above-described case, and an erroneous report or error due to erroneous recognition. Information may be suppressed.

  FIG. 12 is a flowchart illustrating an operation example of the integrated determination processing unit 120.

  First, the communication unit 121 receives information on a person detection result from the person detection processing unit 110 via, for example, a communication network (S11).

  The communication unit 121 registers the information of the person detection result received in S11 in the person detection list (S12). The person detection list is held in a storage unit (not shown) of the integrated determination processing unit 120, for example.

  The person detection list may be provided for each of the person detection cameras 101 and 102. The person detection list may be provided for each person detection processing unit 110 or for each toll gate. One person detection list may be provided within a range managed by the integrated determination processing unit 120. The person detection list may be provided in a range managed by the integrated determination processing unit 120 in common with a number recognition detection list described later (collectively). When provided together, the determination unit 122 can recognize the installation location of the person detection cameras 101 and 102 by holding the camera ID and the like together with the person detection result, and recognize the number existing at the same installation location. Association with the camera 103 can be performed.

  The communication unit 121 registers the person detection result information in the person detection list and then waits for a predetermined time (for example, n seconds) in order to receive the number recognition result information (S13). When the information on the person detection result is newly received before the predetermined time elapses, the communication unit 121 also registers the information on the person detection result in the person detection list.

  When the communication unit 121 receives the number recognition result information before the predetermined time elapses, the communication unit 121 registers the received number recognition result information in the number recognition list. For example, the number registration list is held in a storage unit (not shown) of the integrated determination processing unit 120.

  The number recognition list may be provided for each number recognition camera 103. The number recognition list may be provided for each number recognition processing unit 115. One number recognition list may be provided within a range managed by the integrated determination processing unit 120. The person detection list may be provided in a range managed by the integrated determination processing unit 120 in common with the above-described person detection list (collectively). When provided together, the determination unit 122 can recognize the installation location of the number recognition camera 103 by holding the camera ID and the like together with the number recognition result, and the person detection camera 101 existing at the same installation location. , 102 can be linked.

  The determination unit 122 determines whether or not information on a predetermined number recognition result exists in the number recognition list (S14). The information on the predetermined number recognition result is, for example, information on the number recognition result that satisfies the condition regarding the position and the condition regarding the moving direction. Specifically, the condition regarding the position (distance) is included in, for example, the number recognition position included in the number recognition result information registered in the number recognition list and the person detection result information registered in the person detection list. The person detection position to be detected is within a predetermined distance. The conditions regarding the moving direction are, for example, the same as the moving direction of the vehicle included in the number recognition result registered in the number recognition list and the moving direction of the person included in the information on the person detection result registered in the person detection list. It is to be.

  When there is information on a predetermined number recognition result, the determination unit 122 associates the corresponding person detection result information with the number recognition result information (S15). That is, the person whose person is detected is associated with the license plate whose number is recognized.

  The determination unit 122 deletes predetermined number recognition result information associated with the person detection result information from the number recognition list (S16).

  The determination unit 122 executes determination processing based on at least one of the information on the associated person detection result and the information on the number recognition result (S17). When the person detection result information and the number recognition result information are associated with each other, both the person detection result information and the number recognition result information are obtained. When the information of the person detection result and the information of the number recognition result are not associated with each other, for example, either one of the information is obtained. Details of the determination process will be described later.

  When the execution of the determination process is completed, the determination unit 122 deletes the information on the person detection result for which the determination process has been completed from the person detection list (S18).

  13 and 14 are flowcharts illustrating an example of determination processing by the integrated determination processing unit 120.

  In FIG. 13, when the determination process is started, the determination unit 122 determines the installation locations of the person detection cameras 101 and 102 and the number recognition camera 103 (S31). The installation location is, for example, near the entrance or exit of the highway. The determination unit 122 includes, for example, identification information of the person detection cameras 101 and 102 that capture an image in which an associated person is detected, and identification information of a number recognition camera 103 that captures a vehicle in which the associated number is recognized. The installation location is determined based on at least one of. The identification information (ID) of each camera includes, for example, an IP (Internet Protocol) address, a port number, and a MAC (Media Access Control) address.

  When the installation location of each camera is near the entrance, the determination unit 122 determines the presence / absence of information of the associated number recognition result and the vehicle type recognized by the number recognition (S32).

  When the recognized vehicle type is a small motorcycle, the determination unit 122 determines whether the moving speed of the small motorcycle included in the number recognition result information is equal to or higher than a predetermined threshold (for example, 20 km / h) (S33). .

  When the moving speed of the small motorcycle is equal to or higher than the predetermined threshold (C2, J2 in FIG. 11), the reporting unit 123 causes the alarm device 150 to issue an alarm (S34).

  When there is no information on the number recognition result associated in S32, or when the moving speed of the small motorcycle is less than a predetermined threshold value in S33, the determination unit 122 determines whether there is information on the associated person detection result. (S35).

  When proceeding from S33 to S35, it is assumed that there is a possibility of misrecognizing a bus or the like as a small motorcycle, and double check is performed by number recognition and person detection. Thereby, number recognition accuracy and person detection accuracy can be improved. In S35, when there is no person detection result (B2 in FIG. 11), the reporting unit 123 does not cause the alarm device 150 to issue an alarm (S37).

  When the associated person detection result exists (H2, I2, N2 in FIG. 11), the reporting unit 123 causes the alarm device 150 to issue an alarm (S36).

  When the vehicle type included in the information of the number recognition result in S32 is a large motorcycle (D2, E2, K2, L2 in FIG. 11), the reporting unit 123 does not cause the alarm device 150 to issue an alarm (S38). In S38, the double check as in S35 is not performed. This is because, even in the case of misrecognition, it is a bus or the like, and the situation where it is not reported does not change.

  When the vehicle type included in the number recognition result information in S32 is a four-wheeled vehicle, the determination unit 122 determines the moving direction of the vehicle included in the number recognition result information (S39).

  When the moving direction of the vehicle is from the main road to the entrance (G2, O2 in FIG. 11), the reporting unit 123 causes the alarm device 150 to issue an alarm (S40).

  When the moving direction of the vehicle is the direction of the main line from the entrance of the expressway (F2, M2 in FIG. 11), the determination unit 122 measures the number of vehicles traveling in this moving direction (S41). In this case, for example, the determination unit 122 adds the current one to the number of vehicles already measured.

  When the installation location of each camera is near the exit in S31, the determination unit 122 determines the presence / absence of information of the associated number recognition result and the vehicle type recognized by the number recognition (S51).

  When the recognized vehicle type is a small motorcycle or a large motorcycle, the determination unit 122 determines whether the moving speed of the small motorcycle or the large motorcycle included in the number recognition result information is equal to or higher than a predetermined threshold (for example, 20 km / h). Is determined (S52).

  When the moving speed of the small motorcycle or the large motorcycle is equal to or higher than the predetermined threshold (C1, E1, J1, L1 in FIG. 10), the reporting unit 123 causes the alarm device 150 to issue an alarm (S53).

  When there is no information on the number recognition result associated in S51, or when the moving speed of the small motorcycle or the large motorcycle is less than a predetermined threshold in S52, the determination unit 122 determines whether there is information on the associated person detection result. Is determined (S54).

  When proceeding from S52 to S54, it is assumed that there is a possibility that the bus is misrecognized as a small motorcycle or a large motorcycle, and double check is performed by number recognition and person detection. Thereby, number recognition accuracy and person detection accuracy can be improved. In S54, when there is no person detection result (B1 and D1 in FIG. 10), the reporting unit 123 does not cause the alarm device 150 to issue an alarm (S57).

  When the associated person detection result exists, the determination unit 122 determines the movement direction of the person included in the information on the person detection result (S55).

  When the movement direction of the person is from the exit of the expressway to the main line (H1, I1, K1 in FIG. 10), the reporting unit 123 causes the alarm device 150 to issue an alarm (S56).

  When the moving direction of the person is from the main road to the exit in S55 (N1 in FIG. 10), the reporting unit 123 does not cause the alarm 150 to issue an alarm (S57). The reason why the alarm is not issued when the process proceeds from S55 to S57 is because, for example, it is difficult to distinguish a large motorcycle from a person.

  When the vehicle type included in the number recognition result information in S51 is a four-wheeled vehicle, the determination unit 122 determines the moving direction of the vehicle included in the number recognition result information (S58).

  When the moving direction of the vehicle is the direction from the exit of the highway to the main line (F1, M1 in FIG. 10), the reporting unit 123 causes the alarm device 150 to issue an alarm (S59).

  When the moving direction of the vehicle is the direction from the main road to the exit (G1, O1 in FIG. 10), the determination unit 122 measures the number of vehicles traveling in this moving direction (S60). In this case, for example, the determination unit 122 adds the current one to the number of vehicles already measured.

  According to the operation example of the integrated determination processing unit 120, the accuracy of human detection can be assisted by combining recognition of an identification code with human detection, and the detection accuracy of the presence / absence of a human in captured images by the human detection cameras 101 and 102 can be improved. In addition, the recognition accuracy of the identification code in the captured image by the number recognition camera 103 can be improved by combining the recognition of the identification code with human detection to assist the recognition accuracy of the identification code.

  In FIG. 12, the number is recognized after the person is detected. However, the person may be detected after the number is recognized. In this case, for example, the description relating to person detection and the description relating to number recognition in FIG. 12 are reversed.

  Note that the process of measuring the number of four-wheeled vehicles in S41 and S60 may be omitted.

  The image recognition apparatus described in Patent Document 1 determines the presence or absence of a person using only a person detection camera. Therefore, when the movement speed of the person or the bicycle in the image captured by the camera is slow, the human detection accuracy may be lowered. Further, when identifying without using the moving speed, the human detection accuracy may be deteriorated if the image quality of the camera for human detection is not sufficient.

  According to the determination system 100, for example, by performing a number recognition process in combination with a person detection process, it is possible to distinguish whether the person detected by the person detection is a pedestrian or a motorcycle occupant. Can be made easier.

  For example, even when the movement speed of the person detected by the person detection process is slow or when the movement of the person is small, it can be easily determined whether or not the person is a person. For example, since the number recognition camera 103 is used in addition to the person detection cameras 101 and 102, the number recognition camera 103 plays a role of assisting the person detection cameras 101 and 102 and contributes to improvement of human detection accuracy.

  In addition, it can be determined whether or not the object in the captured image is a person, and when the object is a vehicle, the vehicle type of the vehicle can also be determined and the object can be classified. Moreover, since the classification accuracy of the object can be improved, it can be said that the detection accuracy of a person (pedestrian) who is not a vehicle occupant can also be improved.

  Further, based on various information obtained by person detection and number recognition, for example, it is possible to suppress erroneous entry of a person with restricted access (for example, a person, a bicycle, or a small motorcycle) into a restricted area (for example, a highway). In addition, since it is possible to improve the accuracy of determination as to whether or not the person is a restricted entry person, it is possible to suppress the occurrence of false alarms and missed alarms, and to improve the accuracy of the alarm. Therefore, for example, the burden on the traffic controller who monitors the highway can be reduced.

  In addition, for example, when a character written on the vehicle body is erroneously detected, the accuracy of number recognition may be reduced. However, the recognition accuracy of number recognition can be improved by combining number recognition and person detection. .

  Further, for example, in the human detection process, blurring may occur when the object moves at high speed, and the human detection accuracy may be reduced. In particular, blurring easily occurs in a camera using infrared rays. On the other hand, it is possible to suppress a decrease in human detection accuracy by determining the presence or absence of a person using number recognition.

  The present invention is not limited to the configuration of the above-described embodiment, and any configuration can be used as long as the functions shown in the claims or the functions of the configuration of the present embodiment can be achieved. Is also applicable.

  For example, in the above-described embodiment, the determination system 100 is exemplified for use on an expressway. However, other places (for example, a parking lot provided in a predetermined facility, a dedicated road for vehicles, a pedestrian road) Therefore, it may be used when counting people, two-wheeled vehicles, four-wheeled vehicles and the like separately.

(Overview of one embodiment of the present invention)
The determination method according to one aspect of the present invention obtains detection information relating to human detection in the first image and recognition information relating to recognition of an identification code for identifying the vehicle in the second image. If the difference between the detection time included in the detection information and the recognition time included in the recognition information is within a predetermined time, it is determined whether the person is a person based on the detection information and the recognition information Determining step.

  According to this method, human detection accuracy (determination accuracy) in the first image can be improved by combining recognition of an identification code with human detection to assist human detection accuracy. Further, for example, a person can be detected even when the movement speed of the person is slow or the movement of the person is small. Moreover, even if the accuracy of the detection processing unit that performs the human detection process included in the image is insufficient, a decrease in human detection accuracy can be suppressed.

  In the determination method according to an aspect of the present invention, in the determination step, when the movement direction of the person included in the detection information and the movement direction of the identification code included in the recognition information substantially match, the detection information Whether the person is a person or not is determined based on the recognition information.

  According to this method, an object related to person detection and an object related to recognition of an identification code can be identified as different objects based on the moving direction, and human detection accuracy can be improved.

  In the determination method according to an aspect of the present invention, in the determination step, when the difference between the detection time included in the detection information and the recognition time included in the recognition information is equal to or greater than a predetermined time, It is determined whether or not it is a person, and it is determined whether or not it is a person based on the recognition information.

  According to this method, an object related to person detection and an object related to recognition of an identification code can be identified as different objects based on the detection time and the recognition time, and human detection accuracy can be improved.

  In the determination method according to one aspect of the present invention, in the determination step, the detection position of the person in the first image included in the detection information and the identification code in the second image included in the recognition information. If the difference from the recognition position is within a predetermined distance, it is determined whether the person is a person based on the detection information and the recognition information.

  According to this method, the accuracy of associating objects can be improved by adding a distance condition as well as a temporal condition to associating an object whose identification code is recognized. The accuracy can be further improved.

  A determination system according to one aspect of the present invention includes a first imaging unit that captures a first image, a second imaging unit that captures a second image, and detection related to human detection in the first image. A detection processing unit that generates information, a recognition processing unit that generates recognition information related to recognition of an identification code for identifying a vehicle in the second image, a detection time included in the detection information, and the recognition information And a determination unit that determines whether or not the person is a person based on the detection information and the recognition information when a difference from the included recognition time is within a predetermined time.

  According to this configuration, the accuracy of human detection can be assisted by combining recognition of an identification code with human detection, and the human detection accuracy (determination accuracy) in the first image can be improved. Further, for example, a person can be detected even when the movement speed of the person is slow or the movement of the person is small. Moreover, even if the accuracy of the detection processing unit is insufficient, a decrease in human detection accuracy can be suppressed.

  The determination apparatus according to one aspect of the present invention acquires detection information related to human detection in a first image and recognition information related to recognition of an identification code for identifying a vehicle in a second image. And when the difference between the detection time of the person included in the detection information and the recognition time of the identification code included in the recognition information is within a predetermined time, based on the detection information and the recognition information, And a determination unit for determining whether or not.

  According to this configuration, the accuracy of human detection can be assisted by combining recognition of an identification code with human detection, and the human detection accuracy (determination accuracy) in the first image can be improved. Further, for example, a person can be detected even when the movement speed of the person is slow or the movement of the person is small. Moreover, even if the accuracy of the detection processing unit that performs the human detection process included in the image is insufficient, a decrease in human detection accuracy can be suppressed.

  The present invention is useful for a determination method, a determination system, a determination device, and the like that can improve human detection accuracy.

DESCRIPTION OF SYMBOLS 100 Determination system 101,102 Person detection camera 103 Number recognition camera 104 Integrated camera 110 Person detection process part 111 Control part 112 Detection part 113 Communication part 115 Number recognition process part 116 Control part 117 Recognition part 118 Communication part 120 Integrated determination process part 121 communication unit 122 determination unit 123 reporting unit 130 recording device 140 monitoring device 150 alarm

Claims (6)

An obtaining step for obtaining detection information relating to detection of a person in the first image and recognition information relating to recognition of an identification code for identifying the vehicle in the second image;
If the difference between the detection time included in the detection information and the recognition time included in the recognition information is within a predetermined time, a determination to determine whether or not the person is a person based on the detection information and the recognition information Steps,
Determination method. The determination method according to claim 1,
In the determination step,
When the movement direction of the person included in the detection information and the movement direction of the identification code included in the recognition information substantially match, whether or not the person is a person is determined based on the detection information and the recognition information. Determination method to determine. The determination method according to claim 1,
In the determination step,
When the difference between the detection time included in the detection information and the recognition time included in the recognition information is a predetermined time or more, it is determined whether or not the person is based on the detection information, and based on the recognition information A determination method for determining whether or not a person is. The determination method according to claim 1,
In the determination step, a difference between a detection position of the person in the first image included in the detection information and a recognition position of the identification code in the second image included in the recognition information is within a predetermined distance. If it is, the determination method of determining whether or not the person is a person based on the detection information and the recognition information. A first imaging unit that captures a first image, a second imaging unit that captures a second image,
A detection processing unit that generates detection information relating to detection of a person in the first image;
A recognition processing unit that generates recognition information related to recognition of an identification code for identifying a vehicle in the second image;
If the difference between the detection time included in the detection information and the recognition time included in the recognition information is within a predetermined time, a determination to determine whether or not the person is a person based on the detection information and the recognition information And
A determination system comprising: An acquisition unit for acquiring detection information relating to detection of a person in the first image and recognition information relating to recognition of an identification code for identifying the vehicle in the second image;
When the difference between the detection time of the person included in the detection information and the recognition time of the identification code included in the recognition information is within a predetermined time, the person is based on the detection information and the recognition information A determination unit for determining whether or not
A determination apparatus having

Images