JP6235956B2 - Image composition method, program, and apparatus - Google Patents

Description

  The disclosed technology relates to an image composition method, a program, and an apparatus.

  In recent years, cameras have been installed in traffic infrastructure such as roads and parking lots, and vehicles that use traffic infrastructure, and the images acquired in the eyes of humans can be used for various purposes such as facilitating vehicle traffic, ensuring safety, and entertainment. It is used for various purposes.

  For example, a virtual three-dimensional space is created using an image captured by a camera provided in the vehicle and distance information from the vehicle to a building around the vehicle measured using a horizontal laser provided in the vehicle. Techniques used for route guidance have been proposed.

  In addition, a technique has been proposed in which cameras are installed on the left and right in front of the vehicle, and the road width is estimated based on the motion vector of the point extracted from the camera image, the vehicle speed, and the angle of view of the camera.

JP 2004-265396 A JP 2009-246808 A

  However, as described above, in the past, proposals have only been made regarding the use of images acquired by cameras, and proposals aimed at improving the efficiency of camera installation work for acquiring images suitable for use. Is not done.

  Therefore, as an aspect of the disclosed technology, an object is to improve the efficiency of camera installation work that involves adjusting the size of a subject on the monitor screen of the camera.

  In the disclosed technique, the initial size of the substitute image replacing the first object is such that the recognition target portion has the same size as the recognition target portion included in the first object included in the image captured by the camera. Set the size. The second image included in both the first image in which the first object is not displayed and the second image in which the first object used for setting the initial size of the substitute image is displayed. When the size of the object on the screen changes, the ratio of the change in the size of the second object in both images is calculated. Then, the substitute image whose size is adjusted according to the calculated ratio is synthesized with the first image.

  The disclosed technique has an effect that the efficiency of the camera installation work accompanied by the adjustment of the size of the subject on the monitor screen of the camera can be improved.

FIG. 1 is a functional block diagram of the number recognition apparatus according to the first embodiment. FIG. 2 is a schematic diagram for explaining image capture of a vehicle. FIG. 3 is a schematic diagram for explaining vehicle replacement. FIG. 4 is a schematic diagram for explaining the adjustment of the size of the virtual vehicle. FIG. 5 is a block diagram of a computer that functions as a number recognition apparatus according to the first embodiment. FIG. 6 is a flowchart showing the overall flow of the image composition process. FIG. 7 is a flowchart of a captured image acquisition processing subroutine. FIG. 8 is a flowchart of a virtual vehicle initial size setting processing subroutine. FIG. 9 is a flowchart of a road width calculation processing subroutine. FIG. 10 is a flowchart of the virtual vehicle composition processing subroutine according to the first embodiment. FIG. 11 is a functional block diagram of the number recognition apparatus according to the second embodiment. FIG. 12 is a schematic diagram for explaining the indicator display. FIG. 13 is a block diagram of a computer that functions as a number recognition apparatus according to the second embodiment. FIG. 14 is a flowchart of a virtual vehicle composition processing subroutine according to the second embodiment.

  Hereinafter, an example of an embodiment of the disclosed technology will be described in detail with reference to the drawings.

  (First embodiment)

  An example of application of image recognition technology to transportation infrastructure is the installation of a number recognition device. The number recognition device is a device that acquires an image of a vehicle passing on a road with a camera, detects a number plate that is a recognition target from the acquired image, and reads number information described on the number plate.

  In order to bring out the recognition performance of the number recognition device, the size of the license plate on the monitor screen (hereinafter referred to as “on the screen”) on which the images taken by the camera are displayed is handled by the number recognition device. It must be within the range described in the manual. Accordingly, the operator is proceeding with the camera installation while checking whether the number information of the vehicle can be actually read by the number recognition device by adjusting the zoom and focus of the camera while viewing the image of the camera on the screen.

  However, since the position of the vehicle traveling on the road is not constant, the size of the license plate displayed on the screen changes as the distance from the camera to the license plate varies. Also, when the camera adjustment is performed to change the camera installation position or installation angle, the size of the license plate displayed on the screen changes.

  For this reason, it is difficult to complete the camera installation work with a single camera adjustment. In fact, the worker waits for the vehicle to pass and performs camera adjustment and vehicle number reading confirmation multiple times. It is necessary to perform camera adjustment to improve the number recognition rate.

  Furthermore, there is no guarantee that the vehicle will pass through the road in a timely manner when checking the operation of the number recognition device, and the worker will have to wait until the vehicle passes every time the camera is adjusted. This will hinder the progress of the installation work. Conversely, when the road is congested with vehicles and the distance between the vehicles is so narrow that the number plate of the vehicle cannot be photographed with the camera, the operation of the number recognition device does not proceed and the camera installation work proceeds. May cause trouble.

  For this reason, there have been cases where the operation of the number recognition device has been confirmed by adjusting traffic during the camera installation work to adjust the amount of traffic of the vehicle or running the test vehicle with the road closed. However, even in this case, it is necessary to run the test vehicle many times for each camera adjustment, and the necessity of reducing the labor required for the camera installation work has emerged.

  In the following, in order to improve the efficiency of camera installation work, a virtual vehicle image (virtual vehicle) prepared in advance is combined with an image captured by the camera, and whether or not the number of the virtual vehicle can be recognized A number recognition apparatus in which camera adjustment is advanced will be described. In order to simplify the explanation, in the following explanation, it is assumed that the image taken by the camera shows only one lane of road, but it may be an image showing a road of multiple lanes. Needless to say.

  FIG. 1 shows a number recognition device 100 according to the present embodiment.

  The number recognition device 100 includes an acquisition unit 10, a setting unit 20, a calculation unit 30, a synthesis unit 40, and a recognition unit 50. Among these, the setting unit 20, the calculation unit 30, and the synthesis unit 40 may be collectively referred to as an image synthesis unit 60.

  The acquisition unit 10 is connected to the camera by wire or wirelessly, acquires an image from the camera to detect a vehicle to be monitored, and displays the screen when the detected vehicle reaches a predetermined reference line on the screen. Get the image displayed in. Hereinafter, an image acquired when the vehicle reaches the reference line is referred to as a capture image.

  In this embodiment, in order to acquire a captured image when a vehicle traveling on a road in the vertical direction of the screen reaches the center position of the screen, the reference line is set at the bottom of the screen in consideration of the length of the vehicle in the vertical direction of the screen. Set the screen horizontally.

  The acquisition unit 10 detects whether or not the vehicle has entered a range (designated area) for detecting the vehicle set on the screen by a worker or the like. Then, when detecting that the vehicle has entered the designated area, the acquisition unit 10 detects the position of the vehicle for each frame included in the image, for example. That is, for example, if the image from the camera is 30 fps (frame per second), the acquisition unit 10 detects the position of the vehicle 30 times per second. And the acquisition part 10 acquires the whole image currently displayed on the screen as a capture image, when a vehicle reaches | attains a reference line. A known technique is applied to the detection of the vehicle position. For example, the position of the vehicle is detected from the difference between the color information of the road and the color information of the vehicle included in the frame.

  For example, when the operation mode of the number recognition apparatus 100 is set to the camera adjustment mode indicating that the camera is being adjusted, the acquisition unit 10 notifies the setting unit 20 that the captured image has been acquired. Furthermore, the acquisition unit 10 notifies the image composition unit 60 of the image from the camera even after the capture image is acquired. On the other hand, when the operation mode of the number recognition device 100 is set to an operation mode indicating that the number recognition device 100 is actually in operation, the acquisition unit 10 notifies the recognition unit 50 that a captured image has been acquired.

  Since the position of the vehicle in the captured image can be changed according to the position of the reference line set on the screen, for example, the captured image of the vehicle is acquired at a position where the license plate is displayed more clearly. Can be set.

  FIG. 2 is a diagram for explaining how the acquisition unit 10 acquires an image. In the third frame after the vehicle 2 is detected in the designated area 8 set on the screen, the vehicle 2 becomes the reference line 6. It shows the reached state. And the acquisition part 10 acquires the image of the 3rd frame as a capture image, after detecting the vehicle 2 in the designation | designated area 8. FIG.

  The setting unit 20 obtains the size of the license plate of the vehicle 2 from the captured image acquired by the acquisition unit 10. Here, the size of the license plate means the length of the license plate in the direction along the reference line 6 (number plate width) and the length of the license plate in the direction perpendicular to the reference line 6 (number plate height). Say.

  A known technique is applied to the extraction of the license plate size. For example, the license plate size can be obtained by extracting the outline of the license plate from the captured image. In addition, since the license plate display color is predetermined, the size of the license plate from the captured image can be determined by comparing the color component of each pixel in the captured image with the color component of the license plate display color. Can be extracted.

  Then, the setting unit 20 enlarges or reduces the size of the entire virtual vehicle 4 so that the size of the license plate of the virtual vehicle 4 matches the size of the license plate of the vehicle 2, Set the size of. Then, the setting unit 20 replaces the vehicle 2 of the capture image with the virtual vehicle 4 so that the display position of the license plate of the vehicle 2 on the screen matches the display position of the license plate of the virtual vehicle 4.

  In this embodiment, in order to confirm the size of the set virtual vehicle 4 on the capture screen, the vehicle 2 in the capture image is replaced with the virtual vehicle 4. However, the vehicle 2 in the capture image is replaced with the virtual vehicle 4. The replacement process is not essential. Here, it is sufficient if the size of the virtual vehicle 4 having the same size as the license plate of the vehicle 2 can be set.

  FIG. 3 is a diagram illustrating a state where the vehicle 2 displayed in the captured image by the setting unit 20 is replaced with the virtual vehicle 4. At this time, even if the length in the direction along the reference line 6 of the vehicle 2 is different from H1 and the length in the direction along the reference line 6 of the virtual vehicle 4 is different from H2, the number plate of each vehicle is It becomes the same size.

  The calculation unit 30 calculates the road width L0 on the screen of the road on which the virtual vehicle 4 passes from the captured image in which the vehicle 2 is replaced with the virtual vehicle 4. Therefore, the calculation unit 30 calculates the distance connecting the points where the reference line 6 and the vehicle lane boundary line (boundary line) existing on the left and right sides of the road intersect, specifically, the reference of the straight line connecting the two points. The number of pixels along the line 6 is measured to calculate the road width L0. A known image processing technique, such as binarization processing of a captured image or filter processing using a Laplacian filter, is used to extract a road boundary line on the capture screen.

  Further, the calculation unit 30 monitors the captured camera image (camera adjustment image) notified from the acquisition unit 10, and subsequently extracts the road boundary line displayed in each of the camera adjustment images. Thus, the road width on the screen along the reference line 6 is calculated. Then, the calculation unit 30 performs a camera adjustment such as a change of the zoom ratio, and when the road width of the image under camera adjustment changes from the road width L0 to the road width L1 in the captured image, L1 / L1 is a change rate of the road width. L0 is calculated and notified to the combining unit 40.

  Thus, when a worker or the like performs camera adjustment, the range of the image displayed on the screen is changed accordingly. Therefore, it can be said that the calculation unit 30 detects whether or not the camera adjustment has been performed based on a change in the width of the road that is a common object included in each of the captured image and the camera adjusting image. . In the present embodiment, it is assumed that the vehicle 2 is not displayed in the camera adjustment image.

  The synthesizing unit 40 maintains the aspect ratio of the virtual vehicle 4 in the captured image while maintaining the aspect ratio of the virtual vehicle 4 in the captured image so that the size of the license plate of the virtual vehicle 4 in the captured image is L1 / L0 times the change rate of the road width. 4 Adjust the overall size. The synthesizing unit 40 synthesizes the virtual vehicle 4 whose size is adjusted according to the change rate of the road width and the camera adjusting image whose road width is L1, and notifies the recognizing unit 50 of the synthesized image. To do.

  FIG. 4 is a diagram illustrating a state in which the virtual vehicle 4 in which the size of the license plate is adjusted by the combining unit 40 is combined with the camera adjusting image.

  For example, as shown in FIG. 4, for example, the captured image, that is, the road width along the reference line 6 of the image before camera adjustment is L0, and the width of the license plate of the virtual vehicle 4 at that time is W0. When the road width along the reference line 6 of the camera adjustment image becomes L1, the synthesizing unit 40 sets the width W1 of the license plate of the virtual vehicle 4 in the camera adjustment image as W1 = W0 * (L1 / L0). And the synthetic | combination part 40 adjusts the magnitude | size of the virtual vehicle 4 whole, maintaining the aspect ratio of the virtual vehicle 4 so that the width | variety of the license plate of the virtual vehicle 4 on a screen may become W1. By adjusting the size of the virtual vehicle 4 in this way, the size of the license plate of the virtual vehicle 4 becomes L1 / L0 times the same as the change rate of the road width. In this embodiment, the change in the range of the image displayed on the screen is detected by the change in the road width. However, the change is included in each of the captured image and the camera adjusting image, and is included between the images. Any object can be used as long as it can compare changes in size.

  Furthermore, the combining unit 40 has a base of the license plate of the virtual vehicle 4 in contact with the reference line 6, and a center of the road width along the reference line 6, a center of the virtual vehicle 4 in the width direction of the license plate, The virtual vehicle 4 is combined with the camera adjustment image at a position where the two match.

  And the recognition part 50 recognizes the number information of the virtual vehicle 4 synthesize | combined with the image under camera adjustment in the synthetic | combination part 40, when the operation mode of the number recognition apparatus 100 is set to camera adjustment mode. On the other hand, the recognition unit 50 recognizes the number information of the vehicle 2 displayed in the captured image acquired by the acquisition unit 10 when the operation mode of the number recognition apparatus 100 is set to the operation mode.

  Thus, in the number recognition apparatus 100 of 1st Embodiment, the number calculated from the magnitude | size of the number plate of the vehicle 2 which passes along a road according to the change of the range of the image under camera adjustment on the screen accompanying camera adjustment. A virtual vehicle 4 having a plate size is synthesized. Therefore, it is possible to confirm whether or not the number information of the vehicle 2 can be recognized with the current camera mounting position and zoom ratio without waiting for the vehicle 2 to pass every time the camera adjustment is performed.

  The number recognition apparatus 100 described above can be realized by, for example, a computer 70 shown in FIG. The computer 70 includes a CPU 72, a memory 74, and a nonvolatile storage unit 76, which are connected to each other via a bus 78.

  The storage unit 76 can be realized by an HDD (Hard Disk Drive) or a flash memory. An image composition program 80 for causing the computer 70 to function as the number recognition device 100 having the camera adjustment mode is stored in the storage unit 76 as a storage medium. The CPU 72 reads out the image composition program 80 from the storage unit 76 and expands it in the memory 74, and sequentially executes various processes included in the image composition program 80.

  The image composition program 80 includes an acquisition process 82, a replacement process 84, a calculation process 86, a composition process 88, and a recognition process 90.

  The CPU 72 operates as the acquisition unit 10 illustrated in FIG. 1 by executing the acquisition process 82. The CPU 72 operates as the setting unit 20 illustrated in FIG. 1 by executing the replacement process 84. The CPU 72 operates as the calculation unit 30 illustrated in FIG. 1 by executing the calculation process 86. Further, the CPU 72 operates as the synthesis unit 40 illustrated in FIG. 1 by executing the synthesis process 88. Further, the CPU 72 operates as the recognition unit 50 illustrated in FIG. 1 by executing the recognition process 90.

  As described above, the computer 70 that has executed the image composition program 80 operates as the number recognition apparatus 100.

  As an example, the CPU 72 has a built-in timer, and the CPU 72 has a function of starting and stopping the timer in response to a request from the image composition program 80 and notifying the image composition program 80 of the elapsed time from the timer activation. And The image composition program 80 is an example of an image composition program in the disclosed technology. The number recognition apparatus 100 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like.

  Next, the operation of the number recognition apparatus 100 according to this embodiment will be described. Note that the operation mode of the number recognition apparatus 100 according to the present embodiment is preset to the camera adjustment mode, and the image composition process shown in FIG. 6 is executed after the number recognition apparatus 100 is activated. In addition, it is assumed that an image captured by the camera is stored in a predetermined area of the memory 74 in units of frames in time series. Further, it is assumed that the designated area 8 and the reference line 6 are set in advance on the screen. Furthermore, the virtual vehicle 4 is stored in advance in the storage unit 76, and is expanded in a predetermined area of the memory 74 when the CPU 72 expands the image composition program 80 in the memory 74.

  FIG. 6 is a flowchart showing the entire image composition process executed by the number recognition apparatus 100. The image composition process includes the processes of steps S100 to S500.

  First, the captured image acquisition process in step S100 will be described. FIG. 7 is a flowchart of a subroutine of captured image acquisition processing executed in step S100.

  First, in step S <b> 105, the acquisition unit 10 refers to an image frame stored in the memory 74 and determines whether or not the vehicle 2 exists in the designated area 8 set on the screen.

  In step S110, the acquisition unit 10 proceeds to step S120 if it is determined in step S105 that the vehicle 2 is present in the designated area 8, and if not, step S115 is performed. Transfer processing to.

  In step S115, the acquisition unit 10 acquires a frame next to the frame referred to in step S105 from the memory 74, and proceeds to step S105. That is, the acquisition unit 10 repeats the processes in steps S105 to S115 until a frame in which the vehicle 2 exists in the designated area 8 is detected.

  In step S <b> 120, the acquisition unit 10 determines the position of the vehicle 2 in the designated area 8. In step S125, the acquisition unit 10 determines whether or not the vehicle 2 has reached the reference line 6. If the determination is negative, the process proceeds to step S130.

  In step S130, the acquisition unit 10 acquires a frame next to the frame for which the position of the vehicle 2 has been determined in the process of step S120 from the memory 74, and proceeds to step S120. That is, the acquisition unit 10 repeats the processes of steps S120 to S130 until a frame in which the vehicle 2 reaches the reference line 6 is detected.

  On the other hand, when an affirmative determination is made in the process of step S125, that is, when the vehicle 2 reaches the reference line 6, the process proceeds to step S135, and the acquisition unit 10 uses the currently referenced frame as a captured image as a captured image. Is stored in a predetermined area.

  With the above processing, the processing in step S100 in FIG. 6 is completed.

  Next, the virtual vehicle initial size setting process in step S200 will be described. FIG. 8 is a flowchart of a subroutine of the virtual vehicle initial size setting process executed in step S200.

  First, in step S205, the setting unit 20 detects the size of the license plate of the vehicle 2 displayed in the captured image from the captured image stored in the memory 74 in the process of step S135, and determines the detected size. Store in a predetermined area of the memory 74.

  In step S <b> 210, the setting unit 20 acquires the virtual vehicle 4 from the memory 74. Then, the setting unit 20 enlarges or reduces the size of the entire virtual vehicle 4 so that the size of the number plate of the virtual vehicle 4 matches the size of the number plate of the vehicle 2 detected in step S205. The initial size of the vehicle 4 is set.

  In step S215, the setting unit 20 sets the vehicle 2 on the captured image in the process of step S210 so that a worker or the like can confirm the size of the virtual vehicle 4 set in the process of step S210 on the screen. Replace with the virtual vehicle 4 whose size is set. At this time, the setting unit 20 arranges the virtual vehicle 4 on the capture image so that the display position of the license plate of the vehicle 2 in the capture image matches the display position of the license plate of the virtual vehicle 4.

  With the above processing, the processing in step S200 in FIG. 6 is completed.

  Next, the road width calculation process in step S300 will be described. FIG. 9 is a flowchart of a road width calculation process subroutine executed in step S300.

  First, in step S305, the calculation unit 30 calculates the road width L0 along the reference line 6 from the capture image (replacement capture image) obtained by replacing the vehicle 2 with the virtual vehicle 4 in the process of step S215. Then, the calculation unit 30 requests the CPU 72 to start a timer.

  In step S310, the calculation unit 30 requests timer 72 elapsed time information from the CPU 72, and determines whether or not a predetermined time has elapsed from the processing in step S305. Here, the predetermined time is a time that defines an execution interval of a process for determining a change in road width, which will be described later. The predetermined time is stored in advance in the storage unit 76, and is expanded in a predetermined area of the memory 74 when the CPU 72 expands the image composition program 80 in the memory 74.

  And when the determination process of this step is negative determination, the calculation part 30 repeats the process of step S310 until predetermined time passes. On the other hand, when the determination is affirmative, the process proceeds to step S315, and the calculation unit 30 requests the CPU 72 to stop the timer.

  In step S <b> 315, the calculation unit 30 acquires the frame of the latest camera adjustment image at the current time from the memory 74, and calculates the road width L <b> 1 along the reference line 6 from the frame.

  In step S320, the calculation unit 30 determines whether the road width L0 calculated in the process of step S305 and the road width L1 calculated in the process of step S315 are not the same value. The road width L0 and the road width L1 have the same value. In addition to the case where the values completely match, the difference between the road width L0 and the road width L1 is the same value for the road width L0 and the road width L1. It may also be included when it is within a predetermined range that can be regarded as.

  If the determination in this step is negative, that is, if the road width L0 and the road width L1 are the same value, the process proceeds to step S310, and the calculation unit 30 calculates the road width L0 and the road width L1. Steps S310 to S320 are repeated until the values of are different. On the other hand, if the determination is affirmative, the process proceeds to step S325.

  In step S325, the calculation unit 30 calculates the road width change rate (L1 / L0).

  With the above processing, the processing in step S300 in FIG. 6 is completed.

  Next, the virtual vehicle composition process in step S400 will be described. FIG. 10 is a flowchart of a subroutine of the virtual vehicle composition process executed in step S400.

  First, in step S405, the synthesizer 40 determines the entire virtual vehicle 4 so that the width of the license plate of the virtual vehicle 4 in the replacement capture image generated in the process of step S215 in FIG. 8 is (L1 / L0) times. Adjust the size. At this time, the combining unit 40 adjusts the size of the virtual vehicle 4 so that the aspect ratio of the virtual vehicle 4 does not change.

  In step S410, the synthesizer 40 follows the reference line 6 from the frame of the camera-adjusting image determined to have a change in the road width compared to the replacement capture image in the process of step S320 in FIG. Calculate the center point of the road width.

  In step S415, the composition unit 40 places the virtual vehicle 4 at a position where the center point of the road width of the frame of the camera adjustment image used in the process of step S410 matches the predetermined point of the virtual vehicle 4. Synthesize. Here, as the predetermined point of the virtual vehicle 4, for example, the center point in the width direction of the license plate bottom side of the virtual vehicle 4 is used.

  With the above processing, the processing in step S400 in FIG. 6 is completed.

  In step S500, the recognizing unit 50 determines whether or not the number information of the virtual vehicle 4 can be recognized from the frame of the camera adjusting image in which the virtual vehicle 4 is synthesized. In the case of an affirmative determination, since the number information of the vehicle actually passing on the road can be recognized with the current camera adjustment content, the image composition program 80 is terminated.

  On the other hand, in the case of negative determination, the process returns to the process of step S300, and the processes of steps S300 to S500 are repeatedly executed on the newly acquired frame of the camera adjustment image. In this case, since it is sufficient to obtain the road width L0 in the replacement capture image once, the process of step S305 in FIG. 9 is omitted.

  Thus, in this embodiment, even when the image displayed on the screen changes due to camera adjustment, the virtual vehicle 4 is used to check the size of the license plate displayed in each image. be able to. Therefore, if the image of the vehicle passing through the designated area 8 provided on the screen is acquired once during the camera installation work, the number recognition device even if the actual vehicle does not pass through the designated area 8 after the camera adjustment. 100 operation checks can be performed.

  In the present embodiment, the reference line 6 (determination line) for determining the position of the vehicle 2 in the captured image and the reference line 6 (change detection reference line) indicating the position for detecting the change in the road width are shared. The reference line 6 is described.

  However, the determination line and the change detection reference line may be set at different positions on the screen. In this case, in the description of the flowchart of the captured image acquisition process illustrated in FIG. 7, “reference line 6” is replaced with “determination line”. Then, in the description of the flowchart of the road width calculation process shown in FIG. 9, “reference line 6” is replaced with “change detection reference line”.

  Then, in step S405, the road width L2 along the determination line in the capture image is calculated, and the virtual plate 4 of the virtual vehicle 4 in the replacement capture image is multiplied by (L1 / L2) times. What is necessary is just to adjust the magnitude | size of the vehicle 4 whole. That is, whether or not the determination line and the change detection reference line are shared, the virtual vehicle according to the ratio of the change in the road width in the camera adjustment image to the road width in the captured image It can be seen that the size of the entire 4 may be adjusted.

  (Second Embodiment)

  Next, a second embodiment of the disclosed technique will be described. In addition, the same code | symbol is attached | subjected to the part same as 1st Embodiment, description is abbreviate | omitted, and it demonstrates centering on a different part from 1st Embodiment.

  FIG. 11 shows a number recognition device 100A according to the present embodiment.

  The number recognition device 100A according to the present embodiment is different from the number recognition device 100 according to the first embodiment in that the combining unit 40 is replaced with a combining unit 40A. Accordingly, the image composition unit 60 may be referred to as an image composition unit 60A.

  For example, in the number recognition device 100A, the size of the number plate on the screen suitable for authenticating the number information of the vehicle 2 is defined by experiments with a real machine of the number recognition device 100A or computer simulation based on design specifications. It is assumed that This size is defined by, for example, the number of pixels R corresponding to the width of the license plate displayed on the screen, and is expressed as R1 ≦ R ≦ R2 as an example. Note that the range of R1 ≦ R ≦ R2 is referred to as an appropriate image range. Accordingly, during camera installation work, workers etc. adjust the camera while looking at the screen, and the camera zoom rate etc. so that the width of the license plate of the vehicle 2 displayed on the screen is included in the appropriate image range. Will be adjusted.

  However, there is no way for a worker or the like to know whether or not the width of the license plate of the virtual vehicle 4 displayed on the screen is included in the appropriate image range at the current camera installation position and camera zoom rate. Depending on the situation, even when the width of the license plate of the virtual vehicle 4 is not included in the appropriate image range, the recognition unit 50 may recognize the number information correctly. Therefore, in order to further improve the recognition rate of the number information in the number recognition device 100A, keeping the width of the number plate of the virtual vehicle 4 within an appropriate image range is an important task in installing the camera of the number recognition device 100A. One can be said.

  Therefore, when synthesizing the virtual vehicle 4 with the camera-adjusting image, the synthesizing unit 40A displays, in addition to the virtual vehicle 4, whether or not the width of the license plate of the virtual vehicle 4 is within the appropriate image range. And the recognition unit 50 is notified.

  FIG. 12 is a diagram showing the display of the indicator 15 before and after zoom adjustment as an example. In this case, when the number of pixels R corresponding to the width of the license plate of the virtual vehicle 4 before zoom adjustment is Ra (Ra <R1), the recognition of the number information of the image during camera adjustment is not affected. , A red indicator 15 indicating “NG” is displayed. On the other hand, when the number of pixels R corresponding to the width of the license plate of the virtual vehicle 4 after zoom adjustment is Rb (R1 ≦ Rb ≦ R2), the red indicator 15 is changed to the green indicator 15 indicating “OK”. Replaced.

  The display color of the indicator 15 is not limited, and any display color may be used as long as it can display whether or not the width of the license plate of the virtual vehicle 4 is within the appropriate image range. Not too long. Needless to say, the display position of the indicator 15 is not limited to the position shown in FIG.

  The number recognition apparatus 100A described above can be realized by, for example, a computer 70A shown in FIG. The image composition program 80A includes a composition process 88A instead of the composition process 88 according to the first embodiment. Then, the CPU 72 operates as the synthesis unit 40A illustrated in FIG. 11 by executing the synthesis process 88A. Note that the number recognition device 100A can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC or the like, similarly to the number recognition device 100 according to the first embodiment.

  Next, the operation of the number recognition device 100A according to the present embodiment will be described. Note that the image synthesizing process of the number recognition device 100A according to the present embodiment is different from the image synthesizing process according to the first embodiment shown in FIG. 6 in the process of step S400.

  FIG. 14 is a flowchart of a subroutine of the virtual vehicle composition process executed in step S400 according to the present embodiment. In this flowchart, steps S420 to S435 are newly added to the flowchart showing the process in step S400 according to the first embodiment shown in FIG.

  In step S420, the composition unit 40A determines whether or not the number of pixels corresponding to the width of the license plate of the virtual vehicle 4 adjusted in the process of step S405 is within the appropriate image range. If the determination is affirmative, the process proceeds to step S425, and the composition unit 40A acquires the image of the green indicator 15 from the memory 74. On the other hand, in the case of a negative determination, the process proceeds to step S430, and the composition unit 40A acquires the image of the red indicator 15 from the memory 74. Note that the images of the red and green indicators 15 are stored in advance in the storage unit 76, and are expanded in a predetermined area of the memory 74 when the CPU 72 expands the image composition program 80A in the memory 74.

  In step S435, the combining unit 40A combines the image of the indicator 15 acquired in the process of step S425 or step S430 with the camera adjusting image in which the virtual vehicle 4 is combined in the process of step S415. At this time, the combining unit 40A combines the image of the indicator 15 at a position that does not affect the number information recognition processing in the recognition unit 50, specifically, at a position that does not overlap with the display position of the number plate of the virtual vehicle 4. To do.

  In the present embodiment, a signal indicating whether or not the size of the license plate of the virtual vehicle 4 is within the appropriate image range is notified due to the difference in the color of the indicator 15, but the signal notification method is not limited thereto. I can't. For example, when the size of the license plate of the virtual vehicle 4 falls within the appropriate image range, the notification may be made by voice or by the voice and indicator 15. In this case, the worker or the like does not need to look at the screen during the camera adjustment, and even if the screen is not within the field of view of the worker or the like performing the camera adjustment, the license plate of the virtual vehicle 4 It can be determined whether or not the size is within the appropriate image range.

  As described above, the number recognition device 100A according to the present embodiment displays on the screen, using the indicator 15, the guideline of the size of the number plate of the virtual vehicle 4 adjusted according to the image change caused by the camera adjustment. Therefore, the camera image display range can be adjusted to a position where the size of the license plate of the vehicle 2 is within the appropriate image range without performing camera adjustments many times. Efficiency can be improved.

  Although the disclosed technique has been described using the embodiment, the disclosed technique is not limited to the scope described in the embodiment. Various changes or improvements can be added to the above-described embodiment without departing from the gist of the disclosed technology, and forms to which the changes or improvements are added are also included in the technical scope of the disclosed technology. For example, the processing order may be changed without departing from the scope of the disclosed technology.

  In the above description, the mode in which the image synthesis programs 80 and 80A, which are examples of the image synthesis program according to the disclosed technology, are stored (installed) in the storage unit 76 in advance has been described. However, the present invention is not limited to this. The image composition program according to the disclosed technology can also be provided in a form recorded on a computer-readable recording medium. For example, the image composition program according to the disclosed technology may be provided in a form recorded in a portable recording medium such as a CD-ROM, a DVD-ROM, and a USB memory, and a semiconductor memory such as a flash memory. Is possible.

  Moreover, although an example of the image composition processing according to the disclosed technology has been described using the number recognition device that recognizes the vehicle number information, the image composition target is not limited to the vehicle. For example, in the installation work of a camera used for a person authentication device for authenticating a person's face or the like, an actual person may be replaced with a person image (virtual person) prepared in advance. In this case, the recognition target portion is a human face.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
On the computer,
The initial size of the substitute image that replaces the first object is set so that the recognition target portion has the same size as the recognition target portion included in the first object included in the image captured by the camera. ,
On the screen displaying the image, the first image that is an image in which the first object is not displayed and the first object used for setting the initial size of the substitute image are displayed. And the second image included in the second image on the screen when the size of the second object included in each of the images is changed. Calculating a ratio of the size of the second object included in the first image to the size of the object;
An image composition method for executing processing including combining the substitute image, the size of which is adjusted according to the calculated ratio, with the first image.

(Appendix 2)
Whether or not there is a change in the size of the second object on the screen, a reference line provided at a predetermined position on the screen and a boundary line of the second object intersect with each other. The image composition method according to appendix 1, wherein the image is detected by a change in distance between two points.

(Appendix 3)
The image composition method according to claim 2, wherein an initial size of the substitute image is set when the first object reaches the reference line.

(Appendix 4)
When adjusting the size of the substitute image to be combined with the first image, the length in the direction along the reference line of the recognition target portion included in the substitute image adjusted according to the ratio is the screen. The image composition method according to Supplementary Note 2 or Supplementary Note 3, wherein a signal is notified when the signal is included in a predetermined range suitable for recognizing a recognition target part included in the substitute image.

(Appendix 5)
On the computer,
The initial size of the substitute image that replaces the first object is set so that the recognition target portion has the same size as the recognition target portion included in the first object included in the image captured by the camera. ,
On the screen displaying the image, the first image that is an image in which the first object is not displayed and the first object used for setting the initial size of the substitute image are displayed. And the second image included in the second image on the screen when the size of the second object included in each of the images is changed. Calculating a ratio of the size of the second object included in the first image to the size of the object;
An image synthesis program for executing processing including synthesizing the substitute image whose size is adjusted in accordance with the calculated ratio with the first image.

(Appendix 6)
On the computer,
The initial size of the substitute image that replaces the first object is set so that the recognition target portion has the same size as the recognition target portion included in the first object included in the image captured by the camera. ,
On the screen displaying the image, the first image that is an image in which the first object is not displayed and the first object used for setting the initial size of the substitute image are displayed. And the second image included in the second image on the screen when the size of the second object included in each of the images is changed. Calculating a ratio of the size of the second object included in the first image to the size of the object;
A computer-readable recording medium storing a program for executing a process including combining the first image with the substitute image whose size is adjusted according to the calculated ratio.

(Appendix 7)
The initial size of the substitute image replacing the first object is set so as to have a recognition target portion having the same size as the recognition target portion included in the first object included in the image photographed by the camera. A setting section;
On the screen displaying the image, the first image that is an image on which the first object is not displayed and the first size used for setting the initial size of the substitute image by the setting unit Included in the second image on the screen when the size of the second object that is commonly included in each of the images and the second image that is the screen on which the object is displayed has changed A calculator that calculates a ratio of the size of the second object included in the first image to the size of the second object
A synthesizing unit that synthesizes the substitute image, the size of which is adjusted according to the ratio calculated by the calculating unit, with the first image;
An image synthesizing apparatus.

(Appendix 8)
The calculation unit determines whether or not the size of the second object changes on the screen, a reference line provided at a predetermined position on the screen, a boundary line of the second object, The image composition device according to claim 7, wherein the image composition device is detected by a change in a distance between two points among a plurality of points intersecting each other.

(Appendix 9)
The image composing apparatus according to claim 8, wherein the setting unit sets an initial size of the substitute image when the first object reaches the reference line.

(Appendix 10)
The adjustment unit adjusts the size of the substitute image to be combined with the first image, and the length of the recognition target unit included in the substitute image adjusted according to the ratio in the direction along the reference line. The image synthesizing apparatus according to Supplementary Note 8 or Supplementary Note 9, wherein the signal is notified when the signal is included in a predetermined range suitable for recognizing the recognition target part included in the substitute image on the screen.

2 Vehicle 4 Virtual vehicle 6 Reference line 10 Acquisition unit 15 Indicator 20 Setting unit 30 Calculation unit 40, 40A Composition unit 50 Recognition unit 60, 60A Image composition unit 70, 70A Computer 80, 80A Image composition program 100, 100A Number recognition device

Claims (6)

On the computer,
The initial size of the substitute image that replaces the first object is set so that the recognition target portion has the same size as the recognition target portion included in the first object included in the image captured by the camera. ,
On the screen displaying the image, the first image that is an image in which the first object is not displayed and the first object used for setting the initial size of the substitute image are displayed. And the second image included in the second image on the screen when the size of the second object included in each of the images is changed. Calculating a ratio of the size of the second object included in the first image to the size of the object;
An image composition method for executing processing including combining the substitute image, the size of which is adjusted according to the calculated ratio, with the first image. Whether or not there is a change in the size of the second object on the screen, a reference line provided at a predetermined position on the screen and a boundary line of the second object intersect with each other. The image composition method according to claim 1, wherein detection is performed by a change in distance between two points. The image composition method according to claim 2, wherein when the first object reaches the reference line, an initial size of the substitute image is set. When adjusting the size of the substitute image to be combined with the first image, the length in the direction along the reference line of the recognition target portion included in the substitute image adjusted according to the ratio is the screen. The image synthesizing method according to claim 2 or 3, wherein a signal is notified when the signal is included in a predetermined range suitable for recognizing a recognition target portion included in the substitute image. On the computer,
The initial size of the substitute image that replaces the first object is set so that the recognition target portion has the same size as the recognition target portion included in the first object included in the image captured by the camera. ,
On the screen displaying the image, the first image that is an image in which the first object is not displayed and the first object used for setting the initial size of the substitute image are displayed. And the second image included in the second image on the screen when the size of the second object included in each of the images is changed. Calculating a ratio of the size of the second object included in the first image to the size of the object;
An image synthesis program for executing processing including synthesizing the substitute image whose size is adjusted in accordance with the calculated ratio with the first image. The initial size of the substitute image replacing the first object is set so as to have a recognition target portion having the same size as the recognition target portion included in the first object included in the image photographed by the camera. A setting section;
On the screen displaying the image, the first image that is an image on which the first object is not displayed and the first size used for setting the initial size of the substitute image by the setting unit Included in the second image on the screen when the size of the second object that is commonly included in each of the images and the second image that is the screen on which the object is displayed has changed A calculator that calculates a ratio of the size of the second object included in the first image to the size of the second object
A synthesizing unit that synthesizes the substitute image, the size of which is adjusted according to the ratio calculated by the calculating unit, with the first image;
An image synthesizing apparatus.

Images