JP7266985B2 - Image processing device, image processing system, image processing method, and image processing program - Google Patents

Description

The present invention relates to an image processing device, an image processing system, an image processing method, and an image processing program.

In recent years, techniques for generating a sharpened image from a degraded image have been proposed. For example, in the device described in Patent Document 1, a representative pattern of each class generated by classifying a plurality of characteristic patterns of a plurality of characters into a plurality of classes is stored in advance. Then, the degree of similarity between the characteristic pattern of the degraded image and the representative pattern of each class is evaluated, and the characteristic pattern included in the class with the maximum degree of similarity is evaluated. Also, the degree of similarity between the feature patterns included in the group to be evaluated and the feature pattern of the degraded image is evaluated, and the feature pattern with the highest degree of similarity is specified. Then, based on the characteristic patterns identified in this way, a sharpened image is generated so as to restore even the detailed information lost in the degraded image, and character recognition is performed with high accuracy.

JP 2008-251029 A

However, in the device described in the above document, for example, even when recognizing a place name from a partial image area in a license plate image, the entire degraded image is targeted for sharpening, so the processing load increases. There was a problem that

The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image processing apparatus, image processing system, and image processing capable of reducing the processing load when generating a sharpened image from a degraded image. An object of the present invention is to provide a method and an image processing program.

An image processing apparatus that solves the above problems includes an area setting unit that sets a sharpening area to be sharpened for a degraded image, and a sharpening method that specifies a sharpening method based on attribute information corresponding to the sharpening area. and an image sharpening unit for generating a sharpened image in which the sharpened area is sharpened using the specified sharpening scheme.

An image processing method for solving the above problems includes area setting processing for setting a sharpening area to be sharpened for a degraded image, and specifying a sharpening method based on attribute information corresponding to the sharpening area. and an image sharpening process for generating a sharpened image in which the sharpened area is sharpened using the specified sharpening scheme.

An image processing program that solves the above problems provides a computer with an area setting process for setting a sharpening area to be sharpened for a degraded image, and a sharpening method based on attribute information corresponding to the sharpening area. and an image sharpening process for generating a sharpened image in which the sharpened area is sharpened using the specified sharpening method.

According to the above configuration, since the sharpening area set in the degraded image is the target of sharpening, compared with the case where the entire degraded image is the target of sharpening, when generating the sharpened image from the degraded image, processing load can be reduced. In addition, since a sharpening method corresponding to the sharpened area is used, a sharpened image can be preferably generated.

In the image processing device described above, it is preferable that the attribute information includes an attribute of the sharpening area.

According to the above configuration, it is possible to preferably sharpen the target region included in the sharpening area.

In the image processing device described above, it is preferable that the attribute information includes position information of the sharpening area in the degraded image.

According to the above configuration, it is possible to preferably sharpen the target area included in the predetermined position in the degraded image.

In the image processing apparatus described above, it is preferable that the area setting section can adjust the sharpening area in units of predetermined pixels.

According to the above configuration, it is possible to generate a sharpened image while finely adjusting the sharpened area.

Preferably, the image processing apparatus further includes a display control unit that controls display of the sharpened image in association with the degraded image and the sharpened area.

According to the above configuration, it becomes easy to grasp the sharpened image in association with the sharpened area.

In the above image processing device, the display control unit parallelizes the first sharpened image obtained by sharpening the first sharpened area and the second sharpened image obtained by sharpening the second sharpened area. It is preferable to display

According to the above configuration, it becomes easier to grasp the sharpened images corresponding to the plurality of sharpened areas.

In the above image processing apparatus, it is preferable that the second sharpening area is an area set based on the first sharpening area.

According to the above configuration, it becomes easier to grasp the sharpened image corresponding to the sharpened area before and after the adjustment.

In the above image processing apparatus, when there are a plurality of sharpening methods specified by the image sharpening unit based on the attribute information, the display control unit selects at least one sharpening method from among a plurality of sharpening methods. is preferably received and a sharpened image corresponding to the received sharpening method is displayed.

According to the above configuration, even if a plurality of sharpening methods are specified based on the attribute information corresponding to the sharpening area, the sharpened image corresponding to the sharpening method that matches the user's intention can be grasped. easier.

In the image processing device, the display control unit displays and controls a reference image for specifying an area to be sharpened, and the area setting unit specifies an area to be sharpened in the reference image. and setting the sharpening area based on the specified area.

According to the above configuration, since the sharpening area is set by designating the area to be sharpened in the reference image, it is possible to improve the operability when setting the sharpening area.

In the above image processing apparatus, when there are a plurality of sharpening methods specified based on the attribute information, the image sharpening unit accepts selection of at least one sharpening method from among the plurality of sharpening methods, and accepts the selection. It is preferable to generate the sharpened image using a sharpening scheme.

According to the above configuration, even when a plurality of sharpening methods are specified based on the attribute information corresponding to the sharpening area, the sharpening method selected from among the sharpening methods is used for sharpening. Since the sharpened image is generated, the processing load required for generating the sharpened image can be further reduced.

In the above image processing device, the area setting unit sets a non-target area not to be sharpened for the degraded image, and the image sharpening unit overlaps the sharpening area and the non-target area. Preferably, the sharpened image is generated by excluding the area to be sharpened from the sharpening target.

According to the above configuration, since the areas to be sharpened are limited by setting the non-target areas, it is possible to reduce the processing load required to generate the sharpened image.

In the image processing device described above, it is preferable that the degraded image is an image of a license plate.

According to the above configuration, it is possible to preferably generate a sharpened image in which the image of the license plate is sharpened.

An image processing system for solving the above problems includes the image processing device configured as described above, and a display section for displaying a sharpened image generated by the image processing device.

According to the above configuration, a sharpened image generated from a degraded image can be presented to the user.

1 is a block diagram showing a schematic configuration of an image processing system according to a first embodiment; FIG. FIG. 4 is a schematic diagram for explaining the flow of processing when sharpening a degraded image; FIG. 4 is a schematic diagram for explaining the flow of processing when evaluating the reliability of a sharpened image; FIG. 4 is a schematic diagram showing an example of a sharpening area setting screen; FIG. 4 is a schematic diagram showing an example of an output screen of a sharpened image; 4 is a flow chart showing the details of processing for generating a sharpened image. FIG. 4 is a schematic diagram showing an example of a sharpening area setting screen; FIG. 4 is a schematic diagram showing an example of an output screen of a sharpened image; 4 is a flow chart showing the details of processing for generating a sharpened image.

(First embodiment)
A first embodiment of an image processing system will be described below with reference to the drawings.

The image processing system according to the present embodiment acquires an image of a license plate of a vehicle traveling on a road from an image captured by a camera installed outdoors. Since the image quality of the license plate image is often degraded due to blurring or the like, a sharpened image is generated by sharpening such a degraded image. In the process of generating a sharpened image, the learning model is applied to the image sharpening unit, the image information of the degraded image is input to the image sharpening unit, and the image information output from the image sharpening unit is used as the sharpened image. get. A learning model is set for each class. A class includes letters, numbers, symbols, and the like. In this embodiment, the class includes the place name of the license plate to be sharpened, and the learning model is set for each place name of the license plate (for example, "Shinagawa", "Yokohama", etc.). Further, the image information of the degraded image is input to the image sharpening unit to which a plurality of learning models set for each place name to be sharpened are applied. Also, the image information output from the image sharpening unit to which each learning model is applied is acquired as a plurality of sharpened images individually corresponding to the sharpening target. Also, a plurality of sharpened images generated from the degraded image are displayed to the user. Then, the user identifies the place name included in the degraded image of the license plate by visually recognizing the displayed plurality of sharpened images.

As shown in FIG. 1 , the image processing system 100 includes an image processing device 110 and a display section 140 .

The image processing apparatus 110 includes a control unit 120 that controls the process of generating a sharpened image, various programs including an image processing program that the control unit 120 executes when the process of generating a sharpened image, and execution of the program. It is a computer having a storage unit 130 that saves various data read and written by the control unit 120 at that time. Storage unit 130 may be a non-volatile memory or a volatile memory. By executing the image processing program stored in the storage unit 130, the control unit 120 executes the image acquisition unit 121, the image extraction unit 122, the area setting unit 123, the image sharpening unit 124, the reliability calculation unit 125, And it functions as the display control unit 126 .

The image acquisition unit 121 acquires image frames from the moving image captured by the camera 10 . In this embodiment, the camera 10 is installed outdoors, captures a moving image including a vehicle running on a road, and acquires image frames from the moving image. The image acquisition unit 121 may acquire image frames from a moving image when shooting the moving image, or may acquire image frames from the recorded moving image after recording the moving image.

The image extraction unit 122 identifies the vehicle included in the image frame based on the pixel value data of the image frame acquired by the image acquisition unit 121 . Next, the image extraction unit 122 designates a license plate attached to the vehicle from the area of the vehicle. Note that the image extraction unit 122 may identify the license plate attached to the vehicle based on the pixel value data included in the area of the vehicle.

The area setting unit 123 receives the license plate area selected by the user through the operation unit 20 . The operation unit 20 is composed of, for example, a PC (personal computer), and selects a license plate area from the license plate image displayed on the display unit 140 . In this case, the license plate area includes place names, kana, numbers, and the like. More specifically, the display unit 140 displays the reference image of the license plate and the degraded image of the license plate identified from the image frame side by side. The reference image of the license plate is an image for designating an area to be sharpened. Then, when an area to be sharpened is selected from the reference image of the license plate, the operation unit 20 outputs the area of the degraded image of the license plate corresponding to the area selected from the reference image to the area setting unit 123. do.

The area setting unit 123 acquires the area of the degraded image of the license plate input from the operation unit 20, and sets the acquired area as an area to be sharpened (hereinafter referred to as "sharpening area"). Also, when the user adjusts the sharpening area through the operation unit 20, the area setting unit 123 sets the adjusted area as a new sharpening area.

The image sharpening unit 124 is configured by an autoencoder to which a convolutional neural network, which is a kind of machine learning model, is applied. The image sharpening unit 124 acquires the learning model 131 stored in the storage unit 130 and sharpens the sharpening area set by the area setting unit 123 using the acquired learning model 131 .

In the example shown in FIG. 2, the image extraction unit 122 extracts the license plate area from the input image G0. Then, the image extraction unit 122 extracts the pixel value data included in the license plate area as the degraded image G1. Next, the area setting unit 123 sets the license plate area received through the operation unit 20 as the sharpening area S1. A learning model 131 set for each class is applied to the image sharpening unit 124 . The place names of the image sharpening unit 124 in FIG. 2 represent classes of the learning model 131 . In the present embodiment, each class includes a set of character groups indicating a place name to be sharpened, and the learning model 131 is for each place name to be sharpened ("Shinagawa", "Yokohama", "Narita ”, etc.) are handled individually. The learning model 131 is a parameter generated when the image sharpening unit 124 learns using a high-quality image and a degraded image G1 obtained by degrading the image as teaching data. Learning is performed for each place name to be sharpened. The image sharpening unit 124 to which each learning model 131 is applied outputs a sharpened image G3 obtained by sharpening the image region G2 of the sharpened area S1 of the degraded image G1.

The reliability calculation unit 125 calculates the reliability of the sharpened image G3 converted by the image sharpening unit 124 to which the learning model 131 is applied. The reliability indicates the likelihood of the sharpened image in the process of generating the sharpened image. Reliability is limited to a certain numerical range, such as 0-10.

In the example shown in FIG. 3, the reliability calculation unit 125 selects each sharpened image G3 converted by the image sharpening unit 124 to which the learning model 131 corresponding to each name is applied, and the same class as the learning model 131 ( Place name) is compared with the reference template 132, and the degree of similarity between the sharpened image G3 and the reference template 132 is evaluated. The reference template 132 is an image generated using information such as the license plate font. Then, the reliability calculation unit 125 calculates the similarity calculated by comparing the reference template 132 and the sharpened image G3 as the reliability of the sharpened image G3.

The display control unit 126 controls display of the sharpened image by the display unit 140 . More specifically, the display control unit 126 displays a sharpening area setting screen on the display unit 140 as the first display process.

In the example shown in FIG. 4, the display control unit 126 divides the display area of the display unit 140 into left and right to set a first display area D1 and a second display area D2.

In the first display area D1, a degraded image G1 representing a license plate in the image frames of the moving image captured by the camera 10, and a sharpened area S1 and a non-object area S2 in the degraded image G1 are displayed in an overlapping manner. be.

A selection button F1 indicating the attribute of the license plate to be sharpened is displayed in the second display area D2. In the example shown in the figure, the attributes of the license plate include place name, classification, kana, and serial designation number.

A reference image GS of the license plate is displayed in the second display area D2. In the reference image GS of the license plate, the prescribed positions of the place name (“Shinagawa”), classification (“500”), kana (“A”), and serial designation number (“00-00”), which are attributes of the license plate, are defined. It is A first selection frame T1 corresponding to the attribute of the license plate selected by the selection button F1 is superimposed on the reference image GS of the license plate. The first selection frame T1 is associated with the sharpened area S1 in the degraded image G1. More specifically, the relative coordinates of the center position of the first selection frame T1 in the reference image GS of the license plate match the relative coordinates of the center position of the sharpened area S1 in the degraded image G1. In the example shown in the figure, since "place name" is selected as the attribute of the license plate, the first image area is drawn so as to surround the image area of "Shinagawa" corresponding to the "place name" in the reference image GS of the license plate. A selection frame T1 is displayed.

Further, in the reference image GS of the license plate, prescribed positions of bolts for attaching the license plate to the vehicle are defined. In addition, a second selection frame T2 is superimposed on the reference image GS of the license plate so as to surround the image area of the bolt. The second selection frame T2 is associated with the non-target area S2 in the degraded image G1. More specifically, the relative coordinates of the center position of the second selection frame T2 in the reference image GS of the license plate match the relative coordinates of the center position of the non-target area S2 in the degraded image G1.

Further, an adjustment button F2 for adjusting the position and size of the first selection frame T1 is displayed in the second display area D2. The adjustment buttons F2 include a first adjustment button F2A for selecting an adjustment portion of the first selection frame T1 and a second adjustment button F2B for adjusting the first selection frame T1 vertically and horizontally. . For example, if "upper left" is selected through the first adjustment button F2A and then "upper" is selected through the second adjustment button F2B, the position of the upper left corner of the first selection frame T1 moves upward. In the example shown in the figure, the position coordinates of the first selection frame T1 are adjusted in pixel units of the reference image GS of the license plate through the second adjustment button F2B. As a result, the position coordinates of the sharpened area S1 in the degraded image G1 are adjusted in units of pixels.

In addition, the display control unit 126 displays the output screen of the sharpened image on the display unit 140 as the second display process executed subsequent to the first display process.

In the example shown in FIG. 5, the display control unit 126 divides the display area of the display unit 140 into left and right to set a third display area D3 and a fourth display area D4.

In the third display area D3, a degraded image G1 representing a license plate among the image frames of the moving image captured by the camera 10, and the sharpened area S1 and the non-object area S2 in the degraded image G1 are displayed in an overlapping manner. be. Further, in the third display area D3, an adjustment button F2 is displayed in parallel with the deteriorated image G1 in order to adjust the sharpening area S1 in the deteriorated image G1. The adjustment button F2 has the same function as the adjustment button F2 shown in FIG. 4, and is used when adjusting the position coordinates of the sharpened area S1 in the degraded image G1 in units of pixels. A sharpening button F3 that is operated when sharpening the sharpening area S1 in the degraded image G1 is displayed in the third display area D3. In the example shown in the figure, since the sharpened area S1 and the non-target area S2 in the degraded image G1 partially overlap, the area obtained by excluding the non-target area S2 from the sharpened area S1 in the degraded image G1 is clear. subject to change.

In the fourth display area D4, a list of sharpened images G3 converted by the image sharpening unit 124 to which the learning model 131 corresponding to each name is applied is displayed in association with the reliability SC. In the example shown in the figure, the highest reliability value corresponding to the place name "Narita" is shown among the sharpened images G3 displayed as a list. Therefore, in the third display area D3, the sharpened image G3 corresponding to this place name is displayed in parallel with the degraded image G1 and the reliability SC.

Next, specific processing contents of the sharpened image generation processing executed by the image processing system 100 of the present embodiment will be described.

As shown in FIG. 6, in this sharpened image generation process, first, the image acquisition unit 121 acquires image frames of a moving image captured by the camera 10 (step S10). Then, the image extraction unit 122 extracts a predetermined area (a license plate in this embodiment) from the image frame acquired in step S10 as a target area (step S11). Subsequently, area setting unit 123 sets the attribute (place name in this embodiment) of the sharpening area to be sharpened from the target region extracted in step S11 based on a signal input from operation unit 20. to select (step S12). Then, the area setting unit 123 sets the area of the degraded image G1 associated with the attribute of the sharpened area selected in step S12 as the sharpened area S1 (step S13). Next, the area setting unit 123 identifies the image sharpening unit 124 to which the learning model 131 corresponding to the attribute of the sharpening area is applied (step S14). Subsequently, the area setting unit 123 inputs the image information of the sharpening area S1 to the image sharpening unit 124 specified in the previous step S14 (step S15). Then, a sharpened image G3 of the sharpened area S1 converted by the image sharpening unit 124 is acquired (step S16). Further, the reliability calculation unit 125 compares the plurality of sharpened images G3 of the sharpened area S1 acquired in the previous step S16 with the reference template 132 corresponding to each class (place name) to obtain the sharpened images G3 is calculated (step S17). Then, the display control unit 126 displays the sharpened image G3 of the sharpened area S1 acquired in the previous step S16 on the display unit 140 in association with the reliability value calculated in the previous step S17 (step S18).

Next, the operation of the image processing system 100 of this embodiment will be described.

Conventionally, when generating a sharpened image obtained by sharpening a deteriorated image of a license plate, image information of the entire deteriorated image has been input to an image sharpening unit. However, when a predetermined attribute of the license plate is to be viewed, it is not necessary to sharpen the entire degraded image of the license plate. It should be noted that the position of each attribute of the license plate is determined according to the standard. Therefore, even if the image of the license plate is degraded, it is possible to roughly identify the position of the attribute of the license plate to be sharpened.

Therefore, in the present embodiment, the relative position of each attribute in the reference image GS of the license plate and the relative position of each attribute in the degraded image G1 of the license plate are associated in advance. The relative position is, for example, a position based on the lower left corner of the license plate image. Further, when the attribute of the license plate to be sharpened is selected, based on the information associated as described above, the reference image GS of the license plate and the degraded image G1 of the license plate are compared to determine the license plate. Identify the relative position of the selected attribute in the degraded image G1. Then, the position of the attribute identified as the object of sharpening is set as the sharpening area. Also, the learning model corresponding to the attribute specified as the object of sharpening is applied to the image sharpening unit. After that, the sharpened image is generated by inputting the image information of the sharpened area to the image sharpening unit. This limits the area to be sharpened in the input image of the license plate, thereby reducing the processing load when generating the sharpened image.

As described above, according to the first embodiment, the following effects can be obtained.

(1) A sharpened image G3 obtained by identifying the learning model 131 based on the attribute information corresponding to the sharpened area S1 and sharpening the sharpened area S1 using the image sharpening unit 124 to which the identified learning model 131 is applied. to generate As a result, since the sharpening area S1 set in the degraded image G1 is to be sharpened, the sharpened image G3 is generated from the degraded image G1 compared to the case where the entire degraded image is to be sharpened. It is possible to reduce the processing load during processing. Further, since the image sharpening unit 124 to which the learning model 131 corresponding to the sharpened area S1 is applied is used, the sharpened image G3 can be preferably generated.

(2) The attribute information corresponding to the sharpening area S1 includes the attributes of the sharpening area S1. As a result, the target area included in the sharpening area S1 can be suitably sharpened.

(3) The area setting unit 123 adjusts the sharpening area S1 in units of predetermined pixels. Thereby, the sharpened image G3 can be generated while finely adjusting the sharpened area S1.

(4) The display control unit 126 displays the reference image GS, the area setting unit 123 specifies an area to be sharpened in the reference image GS, and sets the sharpening area S1 based on the specified area. . That is, since the sharpening area S1 is set by specifying the area to be sharpened in the reference image GS, the operability in setting the sharpening area S1 can be improved.

(5) The display control unit 126 performs display control of the sharpened image G3 in association with the degraded image G1 and the sharpened area S1. This makes it easier to grasp the sharpened image G3 in association with the sharpened area S1.

(6) The area setting unit 123 sets a non-target area S2 that is not targeted for sharpening with respect to the degraded image G1, and the image sharpening unit 124 sets a region where the sharpening area S1 and the non-target area S2 overlap. is excluded from the object of sharpening to generate a sharpened image G3. As a result, the area to be sharpened is limited by setting the non-target area S2, so the processing load required to generate the sharpened image G3 can be reduced.

(7) Degraded image G1 includes an image of a license plate. Therefore, the sharpened image G3 in which the license plate image is sharpened can be preferably generated.

(Second embodiment)
Next, a second embodiment of the image processing system will be described with reference to the drawings. The second embodiment differs from the first embodiment in that a list of sharpened images obtained by sharpening the sharpened areas before and after adjustment is displayed. Therefore, in the following description, the configuration different from that of the first embodiment will be mainly described, and redundant description of the configuration that is the same as or corresponding to that of the first embodiment will be omitted.

In the image processing system of the present embodiment, as shown in FIG. 7, when the display control unit 126 displays the setting screen of the sharpening area S1 on the display unit 140, the adjustment button F2 is displayed in the first display area D1. The sharpened area S1 before and after adjusted through the operation of . is superimposed on the degraded image G1 of the license plate. The adjusted sharpening area S1 is defined as the first sharpening area S1A by operating the adjusting button F2 after the sharpening area S1A is once determined as the first sharpening area S1A by operating the sharpening button F3. By operating the sharpening button F3 again after adjusting the position coordinates in units of pixels, the area is determined as the second sharpening area S1B.

Further, in the image processing system 100 of the present embodiment, as shown in FIG. 8, when the display control unit 126 displays the display screen of the sharpened image G3 on the display unit 140, the third display area D3 includes: The sharpening areas S1A and S1B before and after being adjusted through the operation of the adjustment button F2 are displayed superimposed on the degraded image G1 of the license plate. Also, the adjusted sharpening area S1B can be repeatedly adjusted through the operation of the adjustment button F2. In the fourth display area D4, each time the sharpening button F3 is operated while the sharpening area S1A is adjusted through the operation of the adjustment button F2, a sharpened image G3B obtained by sharpening the adjusted sharpened area S1B is displayed. Is displayed. Sharpened images G3B converted by the image sharpening unit 124 to which the learning model 131 corresponding to each name is applied are displayed in a list in the fourth display area D4 in association with the reliability SCB. In the example shown in the figure, the highest reliability value corresponding to the place name "Narita" is shown among the sharpened images G3B displayed as a list. Therefore, the sharpened image G3B corresponding to this place name is displayed in the third display area D3 in parallel with the sharpened area S1B and reliability SCA of the degraded image G1. In the fourth display area D4, every time the sharpened area S1 is adjusted, the display is updated to the sharpened image G3B obtained by sharpening the latest sharpened area S1B. Further, in the third display area D3, each time the sharpened image G3B is generated while the sharpened area S1B is adjusted, the generated sharpened image G3B is associated with the sharpened area S1B and the reliability SCB. will be displayed additionally. In the example shown in the figure, in the third display area D3, a first sharpened image G3A obtained by sharpening the first sharpened area S1A before adjustment and a second sharpened area S1B after adjustment are displayed. A second sharpened image G3B that has been sharpened is displayed in parallel vertically as history information together with the degrees of reliability SCA and SCB.

Next, specific processing contents of the sharpened image generation processing executed by the image processing system 100 of the present embodiment will be described.

As shown in FIG. 9, in the process of generating the sharpened images G3A and G3B, first, the image acquisition unit 121 acquires image frames of moving images captured by the camera 10 (step S20). Then, the image extracting unit 122 extracts a predetermined area (a license plate in this embodiment) from the image frame acquired in step S20 as a target area (step S21). Subsequently, the area setting unit 123 inputs the attribute (place name in this embodiment) of the sharpening area to be sharpened from the target region extracted in the previous step S21 from the operation unit 20 by the operation of the operator. is selected based on the received signal (step S22). Then, the area setting unit 123 sets the area of the degraded image G1 associated with the attribute of the sharpening area selected in the previous step S22 as the sharpening area S1A (step S23). Next, the area setting unit 123 identifies the image sharpening unit 124 to which the learning model 131 corresponding to the attribute of the sharpening area is applied (step S24). Subsequently, the area setting unit 123 inputs the image information of the sharpening area S1 to the image sharpening unit 124 specified in the previous step S24 (step S25). Subsequently, the area setting unit 123 inputs the image information of the sharpening area S1 to the image sharpening unit 124 specified in the previous step S24 (step S25). Then, a sharpened image G3A of the sharpened area S1 converted by the image sharpening unit 124 is obtained (step S26). Further, the reliability calculation unit 125 compares the sharpened image G3A of the sharpened area S1 acquired in the previous step S26 with the reference template 132 corresponding to each class (place name) to calculate the reliability of the sharpened image G3A. degree is calculated (step S27). Subsequently, the area setting section 123 adjusts the sharpening area S1A based on the signal input through the operation section 20 (step S28). Then, the image sharpening unit 124 inputs the image information of the adjusted sharpened area S1B (step S29). Subsequently, a sharpened image G3B of the adjusted sharpened area S1B converted by the image sharpener 124 is obtained (step S30). Then, the reliability calculation unit 125 compares the sharpened image G3B of the adjusted sharpened area S1B acquired in the previous step S30 with the reference template 132 corresponding to each class (place name) to obtain a sharpened image. The reliability of G3B is calculated (step S31). After that, the display control unit 126 displays a list of the sharpened images G3A and G3B before and after the adjustment on the display unit 140 in association with the degrees of reliability SCA and SCB (step S32).

Next, the operation of the image processing system 100 of this embodiment will be described.

In the image processing system 100 of the present embodiment, a first sharpened image G3A that sharpens the first sharpened area S1A and a second sharpened image G3B that sharpens the second sharpened area S1B. are displayed side by side. Therefore, sharpened images G3A and G3B corresponding to a plurality of sharpened areas S1A and S1B can be confirmed in parallel. Further, even when the attributes of a plurality of sharpening areas are to be sharpened, by setting the sharpening areas S1A and S1B in association with each attribute, the attributes of the plurality of sharpening areas can be viewed in parallel. can be verified by

In particular, in the present embodiment, the first sharpening area S1A and the second sharpening area S1B are targeted for common sharpening area attributes, and the first sharpening area S1A is the sharpening area before adjustment. S1, and the second sharpened area S1B corresponds to the adjusted sharpened area S1. Then, each time the sharpened area S1A is adjusted, a list of the sharpened image G3B obtained by sharpening the sharpened area S1B after the adjustment is displayed as history information. Therefore, it becomes easy to compare the image quality of the sharpened images G3A and G3B before and after the adjustment. Therefore, it is possible to adjust the sharpened areas S1A and S1B while confirming the influence of the adjustment of the sharpened areas S1A and S1B on the output results of the sharpened images G3A and G3B.

As described above, according to the second embodiment, in addition to the effects (1) to (7) of the first embodiment, the following effects can be obtained.

(8) The display control unit 126 displays in parallel a first sharpened image G3A that sharpens the first sharpened area S1A and a second sharpened image G3B that sharpens the second sharpened area S1B. to display. Therefore, it becomes easier to grasp the sharpened images G3A and G3B corresponding to the plurality of sharpened areas S1A and S1B.

(9) The second sharpening area S1B is an adjusted area of the first sharpening area S1A. Therefore, it becomes easier to grasp the sharpened images G3A and G3B corresponding to the sharpened areas S1A and S1B before and after the adjustment.

(Other embodiments)
Each of the above embodiments can also be implemented in the following forms.

- In each of the above-described embodiments, an example was described in which the target of sharpening was the place name on the license plate. However, other objects to be sharpened may be numbers (“0 to 9”) and symbols (“- (hyphen)”) on license plates. Further, when the target of sharpening is a place name, the target of sharpening does not necessarily have to be a group of characters indicating the place name, and the sharpening may be performed one by one. In this case, it is also possible to apply characters included in the place name of the license plate as objects of sharpening.

- In the above-described second embodiment, the display control unit 126 displays the sharpened images G3 corresponding to the sharpened areas S1 before and after adjustment in parallel. On the other hand, the display control unit 126 may display the sharpened image G3 corresponding to the latest adjusted sharpened area S1.

- In the above-described second embodiment, the first sharpening area S1A and the second sharpening area S1B are intended for the common license plate attribute, and the first sharpening area S1A is the sharpness before adjustment. The second sharpened area S1B corresponded to the adjusted sharpened area S1. On the other hand, the first sharpened area S1A and the second sharpened area S1B may target different license plate attributes.

- In each of the above embodiments, when there are a plurality of learning models 131 specified based on the attributes of the sharpening area to be sharpened, the display control unit 126 applies all the learning models 131 individually. All the sharpened images G3, G3A, and G3B generated by the image sharpening unit 124 are displayed. On the other hand, the display control unit 126 accepts the operator's selection of at least one learning model 131 from a plurality of learning models 131 specified based on the attributes of the sharpening area, and applies the accepted learning model 131. The sharpened images G3, G3A, and G3B generated by the image sharpening unit 124 may be displayed.

- In each of the above-described embodiments, when there are a plurality of learning models 131 specified based on the attributes of the sharpening area to be sharpened, the image sharpening unit 124 separates all the learning models 131 individually. applied to generate sharpened images G3, G3A, G3B. On the other hand, the image sharpening unit 124 accepts the operator's selection of at least one learning model 131 from a plurality of learning models 131 specified based on the attributes of the sharpening area, and selects the accepted learning model 131. may be applied to generate sharpened images G3, G3A, G3B.

- In each of the above embodiments, the attribute information corresponding to the sharpened area S1 was the attribute of the license plate included in the sharpened area S1. On the other hand, the attribute information corresponding to the sharpened area S1 may be position information of the sharpened area S1 in the degraded image G1. In this case, when an area is specified from the reference image GS of the degraded image G1, the position of the sharpened area S1 in the degraded image G1 is specified based on the positional information of the specified area. Further, when the specified position of the area in the reference image GS of the degraded image G1 is changed, the position of the sharpened area S1 in the degraded image G1 is changed so as to reflect the changed position information of the area.

- In each of the above-described embodiments, the sharpened image G3 is generated from the degraded image G1 using a function model whose parameters are optimized in advance by the user, instead of the image sharpening unit 124 that is trained using the teacher data. may

- In each of the above-described embodiments, the case where the object to be sharpened is the license plate has been described as an example. However, the object to be sharpened is not limited to the license plate. For example, a degraded image including characters, numbers, symbols, etc. such as document data read by a scanner can be applied as the object to be sharpened.

DESCRIPTION OF SYMBOLS 100... Image processing system 110... Image processing apparatus 124... Image sharpening part (an example of sharpening method) 126... Display control part 140... Display part G1... Degraded image G3... Sharpened image G3A... First sharpened image, G3B... Second sharpened image, GS... Reference image, S1... Sharpened area, S1A... First sharpened area, S1B... Second sharpened area, S2... Non-target area .

Claims (15)

A clearing area, which is a partial area of the degraded image extracted from the photographed image and which is to be sharpened, is set in the degraded image based on the attribute information corresponding to the sharpening area. a setting unit;
identifying , from among a plurality of sharpening methods, a sharpening method to which a learning model of a class corresponding to the attribute information is applied based on the attribute information corresponding to the sharpening area, and using the identified sharpening method; An image processing apparatus comprising: an image sharpening unit that generates a sharpened image in which the sharpened area is sharpened. The image processing apparatus according to claim 1, wherein the attribute information includes attributes of the sharpening area. The image processing apparatus according to claim 1, wherein the attribute information includes position information of the sharpening area in the degraded image. The image processing apparatus according to any one of claims 1 to 3, wherein the area setting section can adjust the sharpening area in units of predetermined pixels. The image processing apparatus according to any one of claims 1 to 4, further comprising a display control unit that performs display control of the sharpened image in association with the degraded image and the sharpened area. 6. The display control unit displays side by side a first sharpened image obtained by sharpening the first sharpened area and a second sharpened image obtained by sharpening the second sharpened area. The image processing device according to . The image processing apparatus according to claim 6, wherein the second sharpening area is an area set based on the first sharpening area. When there are a plurality of sharpening methods specified by the image sharpening unit based on the attribute information, the display control unit accepts selection of at least one sharpening method from among the plurality of sharpening methods, and accepts the selection. The image processing device according to any one of claims 5 to 7, wherein a sharpened image corresponding to the sharpening method is displayed. The display control unit displays and controls a reference image for specifying an area to be sharpened,
The image processing device according to any one of claims 5 to 8, wherein the area setting unit specifies an area to be sharpened in the reference image, and sets the sharpening area based on the specified area. . When there are a plurality of sharpening methods specified based on the attribute information, the image sharpening unit receives selection of at least one sharpening method from among the plurality of sharpening methods, and uses the received sharpening method. The image processing device according to any one of claims 1 to 9, wherein the sharpened image is generated by The area setting unit sets a non-target area that is not targeted for sharpening with respect to the degraded image,
The image sharpening unit according to any one of claims 1 to 10, wherein the sharpened image is generated by excluding an area where the sharpened area and the non-target area overlap from a sharpening target. image processing device. The image processing device according to any one of claims 1 to 11, wherein the degraded image is an image of a license plate. The image processing device according to any one of claims 1 to 12,
An image processing system, comprising: a display unit that displays a sharpened image generated by the image processing device. A clearing area, which is a partial area of the degraded image extracted from the photographed image and which is to be sharpened, is set in the degraded image based on the attribute information corresponding to the sharpening area. a configuration process;
identifying , from among a plurality of sharpening methods, a sharpening method to which a learning model of a class corresponding to the attribute information is applied based on the attribute information corresponding to the sharpening area, and using the identified sharpening method; and an image sharpening process for generating a sharpened image in which the sharpened area is sharpened. to the computer,
A clearing area, which is a partial area of the degraded image extracted from the photographed image and which is to be sharpened, is set in the degraded image based on the attribute information corresponding to the sharpening area. a configuration process;
identifying , from among a plurality of sharpening methods, a sharpening method to which a learning model of a class corresponding to the attribute information is applied based on the attribute information corresponding to the sharpening area, and using the identified sharpening method; and an image sharpening process for generating a sharpened image in which the sharpened area is sharpened.

Images