WO2023053830A1 - 画像処理装置、画像処理方法および記録媒体 - Google Patents

画像処理装置、画像処理方法および記録媒体 Download PDF

Info

Publication number
WO2023053830A1
WO2023053830A1 PCT/JP2022/032697 JP2022032697W WO2023053830A1 WO 2023053830 A1 WO2023053830 A1 WO 2023053830A1 JP 2022032697 W JP2022032697 W JP 2022032697W WO 2023053830 A1 WO2023053830 A1 WO 2023053830A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
annotation
data
candidate
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/032697
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
響子 室園
あずさ 澤田
健太 先崎
広宣 森
勝也 小高
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2023550483A priority Critical patent/JP7694679B2/ja
Priority to US18/579,930 priority patent/US20240331418A1/en
Publication of WO2023053830A1 publication Critical patent/WO2023053830A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations

Definitions

  • the present invention relates to an image processing device and the like.
  • the movement interpretation system of Patent Document 1 is a system that determines whether or not an object has been lost by image processing.
  • the movement interpretation system of Patent Literature 1 generates correct data indicating that the house in the image has disappeared based on the comparison result of two image data taken at different times.
  • Patent Literature 1 if the target object of the annotation is an object that is difficult to discriminate, the accuracy of assigning the correct answer may decrease.
  • An object of the present invention is to provide an image processing apparatus and the like capable of improving accuracy while performing annotations efficiently, in order to solve the above problems.
  • the image processing apparatus of the present invention includes area setting means for setting, as a candidate area, an area in which an annotation target object may exist in an annotation target image;
  • a reference image extracting means for extracting a reference image, which is an image of an object, from an image for which annotation has been completed; an annotation target image;
  • the image forming apparatus includes data generating means for generating data for associating the reference image and the standard image as data for annotation, and output means for outputting the data for annotation generated by the data generating means.
  • an area in which an object to be annotated may exist is set as a candidate area, and a reference image, which is an image in which the same object as the target object is photographed, is set.
  • a recording medium includes a process of setting an area in an annotation target image in which an annotation target object may exist as a candidate area; are extracted from images for which annotation has been completed, and the data associated with the target image of annotation, the reference image taken at a different time than the target image of annotation for the region containing the candidate region, and the reference image is used for annotation.
  • An image processing program for causing a computer to execute a process of generating data and a process of outputting the generated annotation data is recorded.
  • FIG. 1 is a diagram showing an overview of the configuration of a first embodiment of the present invention
  • FIG. It is a figure showing an example of composition of an image processing device of a 1st embodiment of the present invention.
  • FIG. 4 is a diagram showing an example of a target image according to the first embodiment of the present invention;
  • FIG. 4 is a diagram showing a setting example of candidate areas according to the first embodiment of the present invention;
  • FIG. 4 is a diagram showing a setting example of candidate areas according to the first embodiment of the present invention;
  • FIG. 4 is a diagram showing a setting example of candidate areas according to the first embodiment of the present invention;
  • FIG. 4 is a diagram showing an example of a reference image according to the first embodiment of the present invention;
  • FIG. 4 is a diagram showing an example of a reference image according to the first embodiment of the present invention
  • FIG. 4 is a diagram showing an example of output data according to the first embodiment of this invention
  • FIG. 4 is a diagram showing an example of output data according to the first embodiment of this invention
  • FIG. 4 is a diagram showing an example of output data according to the first embodiment of this invention
  • FIG. 4 is a diagram showing an example of output data according to the first embodiment of this invention
  • FIG. 4 is a diagram showing an example of output data according to the first embodiment of this invention
  • FIG. 4 is a diagram showing an example of output data according to the first embodiment of this invention
  • FIG. It is a figure which shows the example of the operation
  • FIG. 4 is a diagram showing another configuration example of the embodiment of the present invention.
  • FIG. 1 is a diagram showing an overview of the configuration of an image processing system according to this embodiment.
  • the image processing system includes an image processing device 10 and a terminal device 30 .
  • the image processing device 10 and the terminal device 30 are connected via a network.
  • the image processing system of this embodiment is a system that performs annotation-related processing on images.
  • Annotation-related processing is, for example, outputting an image to be annotated, and based on the input result of the operator's operation, extracting information specifying the object existing in the image and information on the area where the object exists. , to associate with an image.
  • the information associated with the image may be either information specifying an object present in the image or an area in which the object exists.
  • the information associated with the image is not limited to these.
  • the image processing system for example, performs annotation processing on an image acquired using a synthetic aperture radar (SAR). Data generated using an image processing system can be used, for example, as teacher data in machine learning.
  • SAR synthetic aperture radar
  • FIG. 2 is a diagram showing an example of the configuration of the image processing apparatus 10.
  • the image processing apparatus 10 includes an area setting section 11 , an area extraction section 12 , a reference image extraction section 13 , a data generation section 14 , an output section 15 , an input section 16 and a storage section 20 .
  • the storage unit 20 includes a target image storage unit 21, a reference image storage unit 22, an area information storage unit 23, and an annotation result storage unit 24.
  • the region setting unit 11 sets a region in which an annotation target object may exist as a candidate region in the annotation target image.
  • an image to be annotated that is, an image to be subjected to annotation processing is also referred to as a target image.
  • the area setting unit 11 sets areas in which the target object may exist as candidate areas in the target image.
  • the region setting unit 11 reads, for example, the target image to be processed from the target image storage unit 21 .
  • the region setting unit 11 stores the range of candidate regions on the target image in the region information storage unit 23 .
  • the region setting unit 11 expresses the range of the candidate region on the target image by, for example, coordinates in the target image and stores it in the region information storage unit 23 .
  • the target image is added with, for example, information on the shooting location and date and time.
  • FIG. 3 is a diagram showing an example of a target image.
  • FIG. 3 is an example of image data captured by a synthetic aperture radar.
  • the elliptical and rectangular areas in FIG. 3 indicate, for example, areas where the reflected waves are different from the surroundings, that is, areas where objects may exist.
  • the perimeters of the elliptical and rectangular regions correspond, for example, to the sea.
  • the gray area on the right side of FIG. 3 corresponds to land, for example.
  • the area setting unit 11 sets an area where the state of the reflected wave is different from that of the surrounding area as a candidate area where the target object may exist.
  • the region setting unit 11 identifies a region in the target image that has a luminance different from its surroundings, and sets a rectangular region including the identified region as a candidate region.
  • FIG. 4 shows an example in which candidate areas are set as candidate areas W on the target image.
  • the area setting unit 11 sets a rectangular area having a luminance different from that of its surroundings as a candidate area in which the target object may exist.
  • the candidate area W is set in the area enclosed by the dotted line from the upper right corner of the target image.
  • the area setting unit 11 identifies all locations where the target object may exist in one target image and sets them as candidate areas.
  • the area setting unit 11 sets a plurality of candidate areas by, for example, sliding the candidate areas in the target image.
  • the region setting unit 11 sets a plurality of candidate regions, for example, so as to cover all of the candidate regions existing in the target image.
  • 5 and 6 are diagrams showing examples of operations for setting a plurality of candidate areas.
  • the area setting unit 11 sets a plurality of candidate areas W by sequentially sliding the candidate area W set in the upper left corner area of the target image in the right direction. Further, as shown in FIG. 6, the area setting unit 11 slides the candidate area W downward from the initial position in FIG. You may At this time, the candidate areas may or may not overlap each other.
  • the method of sliding candidate areas when setting a plurality of candidate areas is not limited to the above example. For example, when the candidate area is slid, the area setting unit 11 saves information indicating the range of the candidate area in the area information storage unit 23 if there is an area in the candidate area whose luminance change satisfies the reference.
  • the area setting unit 11 may compare the position where the target image was obtained with the map information, and set a candidate area within the area preset for the target image. For example, when the target object is a ship, the candidate area may be set in areas where the ship may exist, such as seas, rivers and lakes. In such a case, the area setting unit 11, for example, refers to map information and sets candidate areas only within areas of seas, rivers, and lakes.
  • the area extraction unit 12 extracts the image of the area corresponding to the candidate area from the reference image as the corresponding image. Further, the area extracting unit 12 extracts an image of an area corresponding to the candidate area from the target image as a candidate image.
  • a reference image is an image used as a comparison object for determining whether a target object exists in the target image. Also, the reference image is an image acquired at a different time than the target image in an area including the area of the target image. A plurality of reference images may correspond to one target image.
  • a reference image is, for example, an image of an area including an area in which the target image was shot in the same method as the target image, but taken at a time different from that of the target image. For example, among images taken at the same point every day at the same time, one image is set as a target image, and images taken on other days are used as reference images. The cycle of image capturing and the time of capturing may not be constant.
  • the reference image is added with, for example, information on the shooting location and date and time.
  • the area extraction unit 12 reads the reference image from the reference image storage unit 22, for example.
  • the area extraction unit 12 identifies areas corresponding to the candidate areas on the reference image based on the candidate area information stored in the area information storage unit 23 .
  • the area extraction unit 12 extracts an image of an area corresponding to the candidate area from the reference image as a corresponding image.
  • the area extraction unit 12 may use the target image including the candidate area as the candidate image without extracting the candidate image from the target image.
  • the region extracting unit 12 may refer to the position information added to the image and set the reference image including the candidate region as the corresponding image corresponding to the candidate region without extracting the target image from the reference image.
  • FIG. 7 is a diagram showing an example of a reference image.
  • FIG. 7 shows an example in which the number of elliptical objects is different from the target image shown in FIG. 3 because the reference image is an image acquired at a different time than the target image.
  • FIG. 8 is a diagram showing an example of candidate areas W corresponding to candidate areas on the target image. The example of FIG. 8 shows a case where the area extracting unit 12 identifies a candidate area W set near an elliptical object on the reference image and extracts an image within the candidate area as a corresponding image.
  • the region extracting unit 12 extracts images of regions corresponding to candidate regions from two reference images.
  • the two reference images are images taken at different times than the target image.
  • the area extraction unit 12 extracts, for example, images in candidate areas of the target image as corresponding images G2 and G3 corresponding to the candidate image G1.
  • the region extraction unit 12 extracts the corresponding image G2 from the reference image A acquired one day before the target image was acquired by the synthetic aperture radar, and extracts the corresponding image G3 from the reference image B acquired two days before. to extract
  • the area extraction unit 12 associates the candidate image G1, the corresponding image G2, and the corresponding image G3.
  • the number of corresponding images associated with one candidate image does not have to be two, and is set according to the number of reference images. Also, the number of reference images can be set as appropriate.
  • the reference image extraction unit 13 extracts a reference image, which is an image in which the same object as the target object is captured, from the image for which annotation has been completed.
  • the reference image extraction unit 13 searches for image data with completed annotations stored as annotation completion data in the annotation result storage unit 24, and extracts an image in which the same object as the target object is photographed as a reference image. .
  • Identical objects include similar objects.
  • As the reference image for example, among images for which annotation has been completed, an image that has been determined to be correct in verification of the result of annotation is used. Verification of annotation results is performed using an optical image, for example, when the target image is captured by a synthetic aperture radar.
  • Images for which annotation has been completed may be associated with images for which the judgment result at the time of annotation is incorrect.
  • An image with an incorrect judgment result when annotated is, for example, an annotated image when judged using an image taken by another method, and the type of object judged in the annotation is This is the image that was identified as wrong.
  • an annotation operator determines that the object existing in the candidate area is a ship.
  • the reference image extraction unit 13 uses the image for which annotation has been completed and the incorrect image associated with the image for which annotation has been completed as reference images. Extract.
  • the reference image extracting unit 13 sets an image for which annotation has been completed as a correct image, and extracts a set of a correct image and an incorrect image as reference images.
  • the reference image extracting unit 13 compares the similarity between the candidate image and the image for which annotation has been completed, and determines that they are similar when the similarity is equal to or higher than the reference.
  • the reference image extraction unit 13 determines whether the object in the candidate image is the same as the object in the image for which annotation has been completed, based on, for example, similarity in map coordinates and similarity in image feature quantity.
  • the reference image extracting unit 13 may determine whether the object in the candidate image is the same as the object in the image for which annotation has been completed, based on items other than the above.
  • the reference image extracting unit 13 determines whether the photographed object is the same as the candidate image for the images for which annotation has been completed and whose photographing positions have been determined to be the same.
  • the reference image extracting unit 13 determines whether the shooting positions of the candidate image and the annotated image are the same based on, for example, the distance between the central coordinates of the candidate image and the annotated image.
  • the reference image extraction unit 13 determines that the photographing positions are the same when the distance between the center coordinates of the candidate image and the image for which annotation has been completed is equal to or less than the reference.
  • the reference image extraction unit 13 uses, for example, feature point matching when determining whether the object in the candidate image and the object in the image for which annotation has been completed are the same based on the similarity of the image feature amount, The similarity of image feature amounts between images is calculated.
  • the reference image extraction unit 13 extracts feature points from, for example, the candidate images and the images for which annotation has been completed, and when the similarity of the feature points satisfies the reference, the two images identify the same object. It is determined that the image is a photographed image.
  • a method for determining similarity of image features using feature point matching is described, for example, in P.F. Alcantarilla, J. Nuevo and A.
  • the reference image extraction unit 13 may calculate the similarity of the image feature amount using a method other than the feature point matching method. You may calculate the similarity of a feature-value.
  • the reference image extraction unit 13 extracts each image of the annotation completion data stored in the annotation result storage unit 24 and the annotation data. are compared to extract a reference image. For example, if there is no image with the same photographing position, the reference image extracting unit 13 extracts all the annotation-completed images stored in the annotation result storage unit 24, and extracts the similarity of the image feature amount. An image that satisfies the criteria of is extracted as a reference image.
  • the reference image extraction unit 13 may further use similarity in size of objects present in the image to determine whether the object in the candidate image and the object in the image for which annotation has been completed are the same. In such a case, for each of the two images, for example, the relationship between the number of pixels and the actual distance is set in advance.
  • the reference image extracting unit 13 determines, for example, the similarity in size of an object on two images based on the ratio or difference in area of the object present in each image. When the size similarity standard is set based on, for example, the ratio of the areas of the objects present in the two images, the reference image extracting unit 13 detects that the ratio of areas is within the reference range. Sometimes we determine that the size of an object on the two images is the same.
  • the reference image extracting unit 13 determines whether the difference in area is within the standard. At some point, it is determined that the size of the object on the two images is the same.
  • the data generation unit 14 generates, as annotation data, data that associates an annotation target image, a reference image, and a standard image.
  • a reference image is an image of an area including a candidate area that is captured at a time different from that of the image to be annotated.
  • the data generating unit 14 generates annotation data by associating, for example, an image to be annotated, that is, a candidate image obtained by extracting a candidate region from a target image, a corresponding image obtained by extracting a candidate image from a reference image, and a reference image. Generate.
  • the data generation unit 14 may generate annotation data by associating the candidate image, the corresponding image, and an image obtained by enlarging the vicinity of the candidate image in the reference image.
  • the data generation unit 14 outputs the generated annotation data to the terminal device 30 via the output unit 15, for example.
  • the data generation unit 14 may generate, as annotation data, display data for displaying the candidate image, the corresponding image, and the reference image so that they can be compared.
  • the display data displayed so as to be able to be compared is, for example, display data in a state in which the operator can compare two images by arranging them in the horizontal direction. say.
  • the data generator 14 may output the generated display data to a display device (not shown) connected to the image processing device 10 .
  • the data generating unit 14 When an incorrect image is associated with the reference image extracted by the reference image extracting unit 13, the data generating unit 14 generates annotation data as a set of the correct image and the incorrect image as the reference image.
  • FIG. 9 shows an example of a display screen that displays a candidate image output as annotation data, a corresponding image, and a reference image so that they can be compared.
  • the reference images are displayed as a set of a correct image and an incorrect image.
  • the image P1 in FIG. 9 is the correct image among the reference images.
  • the image N1 in FIG. 9 is an incorrect image among the reference images.
  • information about the size of the target object and the position where the image was taken is added to the images of P1 and N1. Items of information added to the reference image are not limited to these.
  • the image of G1 in FIG. 9 is a candidate image, that is, an image corresponding to a candidate area on the target image.
  • Images G2 and G3 in FIG. 9 are corresponding images, that is, images corresponding to candidate areas on the reference image.
  • Images G2 and G3 are images on the reference image taken at different times.
  • the data generation unit 14 generates annotation completion data based on information about annotations.
  • Information about the annotation is input to the terminal device 30 as annotation information by the operator's operation.
  • the annotation information is, for example, an image to be annotated, that is, information specifying the type of target object on the target image, and information specifying an area where the object exists on the image.
  • the data generation unit 14 acquires, for example, information specifying an area in which an object exists from the annotation information as a rectangular area surrounding the object on the candidate image.
  • the data generation unit 14 generates, as annotation completion data, data in which the candidate image is associated with the type of object on the candidate image and information on the area in which the object exists.
  • a region indicated by annotation information is also called an annotation region.
  • the data generation unit 14 stores the generated annotation completion data in the annotation result storage unit 24 .
  • the setting of the annotation area is not limited to the method of enclosing the area with rectangular lines.
  • the annotation area may be set by filling the annotation area.
  • FIG. 10 shows an example of the display screen when the annotation area is set on the display screen of FIG.
  • a rectangular line is set around the elliptical object as a line indicating the annotation area.
  • rectangular lines are also displayed at corresponding positions for G2 and G3, which are the corresponding images.
  • FIG. 11 is a diagram showing only the lower part of the display screen in FIG.
  • FIG. 11 is a diagram showing an example of a display screen that displays the candidate image G1, the corresponding image G2, and the corresponding image G3 so that they can be compared.
  • FIG. 12 shows, in the image of FIG. 11, areas where an object exists in the candidate image G1 but does not exist in the corresponding images G2 and G3 with dotted lines.
  • FIG. 13 shows an example in which an annotation area is set on the candidate image G1 by the operator's operation while the screen of FIG. 11 is being displayed.
  • an area surrounded by rectangular lines is set as an annotation area on the candidate image G1.
  • FIG. 14 shows an example of a display screen in which annotation regions are added to candidate image G1 and further displayed on corresponding images G2 and G3. In this manner, by displaying the annotation area not only on the candidate image G1 but also on the corresponding image G2 and the corresponding image G3 taken at different times, the operator can more clearly see the area in which the movable object exists. Annotations can be processed with clear recognition.
  • the output unit 15 outputs the annotation data generated by the data generation unit 14 to the terminal device 30 .
  • the output unit 15 may output display data generated based on the annotation data to a display device (not shown) connected to the image processing device 10 .
  • the input unit 16 receives an input of information related to the annotation of the target object as annotation information for the target image of the annotation.
  • the input unit 16 acquires from the terminal device 30 the annotation information input to the terminal device 30 by the operator's operation.
  • the input unit 16 acquires, as annotation information, information about the range of the annotation area and information specifying the type of object on the image.
  • the input unit 16 may acquire, as the annotation information, either information on the range of the annotation region or information specifying the type of object on the image.
  • the input unit 16 may acquire information on items other than the above as annotation information.
  • the input unit 16 may acquire annotation information from an input device (not shown) connected to the image processing device 10 .
  • the target image storage unit 21 of the storage unit 20 stores the image data of the target image of the annotation as the target image.
  • the target image storage unit 21 stores, for example, the target image in association with the shooting date and time and the shooting position information.
  • the reference image storage unit 22 stores image data of reference images.
  • the reference image storage unit 22 stores, for example, the reference image in association with information on the shooting date and time and the shooting position.
  • the reference image storage unit 22 may associate and store information of the target image corresponding to the reference image. Also, the information associated with the target image and the reference image is not limited to these examples.
  • the area information storage unit 23 stores information on the range of candidate areas set by the area setting unit 11 .
  • the annotation result storage unit 24 stores the annotation target image and the annotation information in association with each other as annotation completion data.
  • the annotation result storage unit 24 may store the image included in the annotation completion data in association with information on the imaging position of the image.
  • the annotation result storage unit 24 may store an image included in the annotation completion data in association with an incorrect image.
  • Each of the above data related to annotations stored in the storage unit 20 is input to the image processing device 10 by, for example, an operator.
  • Each piece of data related to annotations stored in the storage unit 20 may be obtained from the terminal device 30 or a server connected via a network.
  • the storage unit 20 is configured using, for example, a hard disk drive.
  • the storage unit 20 may be configured using another storage device such as a non-volatile semiconductor storage device, for example.
  • the storage unit 20 may be configured by combining a plurality of types of storage devices such as a nonvolatile semiconductor storage device and a hard disk drive. Also, part or all of the storage unit 20 may be provided by an external device connected to the image processing apparatus 10 via a network.
  • the terminal device 30 is a terminal device for operator operation, and includes an input device and a display device (not shown).
  • the terminal device 30 acquires annotation data from the image processing device 10 . Based on the annotation data, the terminal device 30 outputs a display screen for performing annotation work to a display device (not shown).
  • the terminal device 30 displays, for example, a display screen in which the candidate image, the corresponding image, and the reference image are associated with each other on the display device.
  • the terminal device 30 may display both the correct image and the incorrect image for the reference image.
  • the terminal device 30 accepts annotation information input by the operator's operation.
  • the terminal device 30 outputs the acquired annotation information to the image processing device 10 .
  • the terminal device 30 may be plural.
  • the number of terminal devices 30 can be set as appropriate.
  • FIG. 15 is a diagram showing an example of the operation flow of the image processing apparatus 10 of this embodiment.
  • the region setting unit 11 of the image processing device 10 reads out the target image, which is the target image for annotation, from the target image storage unit 21 of the storage unit 20 .
  • the area setting unit 11 sets an area in which the annotation target object may exist as a candidate area on the target image (step S11).
  • the region setting unit 11 identifies a region where an object may exist, for example, based on the luminance value of each pixel in the image. After specifying the area where the object may exist, a rectangular area including the specified area is set as a candidate area.
  • the region setting unit 11 sets, for example, a region smaller than the entire target image as a candidate region.
  • the area setting unit 11 After setting the candidate area, the area setting unit 11 saves information on the set candidate area in the area information storage unit 23 .
  • the region setting unit 11 stores, for example, the coordinates specifying the outer periphery of the candidate region on the target image as the candidate region information in the region information storage unit 23 .
  • the area setting unit 11 sets a plurality of candidate areas so as to cover all candidate areas existing in the target image.
  • the region setting unit 11 for example, slides the candidate region within the target image and sets the region where the object may exist as the candidate region.
  • the area extraction unit 12 selects candidate areas to be annotated from the candidate areas stored in the area information storage unit 23 (step S12). For example, the region extracting unit 12 selects, as an annotation target, a candidate region that has been saved earliest as a candidate region from candidate regions that have not yet been annotated. Other methods for selecting candidate regions may be used.
  • the area extraction unit 12 extracts the image of the candidate area from the target image as a candidate image. Further, the area extraction unit 12 reads out the reference image corresponding to the target image from the reference image storage unit 22 and extracts the image of the candidate area from the reference image as the corresponding image. (Step S13). For example, the area extracting unit 12 extracts images of portions within the candidate areas from the two reference images as corresponding images.
  • the reference image extraction unit 13 searches the annotation completion data stored in the annotation result storage unit 24, and extracts an image in which the same object as the candidate image exists in the image as the reference image. (step S14).
  • the reference image extracting unit 13 extracts an image whose similarity satisfies a reference as a reference image based on the similarity between the candidate image and the image stored as the annotation completion data, for example.
  • the data generation unit 14 When the reference image is extracted, the data generation unit 14 generates data in which the annotation target image, the reference image captured at a time different from the annotation target image for the region including the candidate region, and the reference image are associated with each other for annotation. It is generated as data (step S15). The data generation unit 14 generates, as annotation data, data in which, for example, the candidate image, the corresponding image, and the reference image are associated with each other.
  • the output unit 15 outputs the generated annotation data to the terminal device 30 (step S16).
  • the terminal device 30 After acquiring the annotation data, the terminal device 30 outputs display data based on the annotation data to a display device (not shown). When annotation information is input by an operator's operation while display data is being displayed based on annotation data, the terminal device 30 outputs the input annotation information to the image processing device 10 .
  • the input unit 16 of the image processing device 10 acquires annotation information from the terminal device 30 (step S17).
  • the data generator 14 associates the candidate image data with the annotation information to generate annotation completion data (step S18).
  • the data generation unit 14 stores the generated annotation completion data in the annotation result storage unit 24 .
  • step S19 When the annotation completion data is saved, if annotation processing has been completed for all candidate regions (Yes in step S19), the image processing device 10 ends annotation processing. When there is a candidate area for which annotation processing has not been completed (No in step S19), the image processing apparatus 10 executes the process from the candidate area selection operation in step S12.
  • the annotation completion data generated by the above method can be used, for example, as training data for generating a machine learning model for identifying images in an image recognition device.
  • the target image may be an image acquired by a method other than synthetic aperture radar.
  • the target image may be an image acquired by an infrared camera.
  • the above explanation shows an example of annotating with reference to an image acquired in the same way as the target image.
  • other types of images may be referenced to verify the result of annotation determination.
  • an annotated image acquired by a synthetic aperture radar and an optical image captured by an optical camera that captures the visible light region of the same point are displayed side by side to determine whether the object type determined in the annotation is correct or not. may be verified.
  • verifying the correctness by such a method it is possible to generate a correct image and an incorrect image to be used as the reference image.
  • FIG. 16 is a diagram showing an example of a display screen when verifying judgment results in annotations.
  • the example of FIG. 16 shows an example of a display screen in which an image G1 for which annotation has been completed and an image V1 obtained at the same point as the image G1 by a different imaging device from that of the image G1 are displayed side by side so that they can be compared.
  • the image G1 is an image obtained using, for example, a synthetic aperture radar
  • the image V1 is an image obtained using, for example, an optical camera.
  • selection buttons for "next image”, “correct answer”, and “incorrect answer” are set.
  • the “next image” selection button is a button for switching the image to be verified.
  • “Correct” is a button for inputting that the judgment result in the annotation was correct.
  • “Incorrect” is a button for inputting that the judgment result in the annotation was wrong.
  • the data generation unit 14 when the "Incorrect" button is selected, the data generation unit 14 removes the corresponding data from the annotation completion data and saves it as incorrect data.
  • the data generation unit 14 associates the information indicating the correct answer with the annotation completion data, and updates the annotation completion data. For example, the data generation unit 14 may associate an image of annotation completion data for which an incorrect answer is selected with other annotation completion data including an image of the same point as an incorrect image.
  • the image processing apparatus 10 of the image processing system of the present embodiment extracts a candidate image obtained by extracting an area in which an object may exist from a target image to be annotated, and extracts an area corresponding to the candidate image from a reference image.
  • the corresponding image and the reference image are associated with each other and output as annotation data.
  • the image processing apparatus 10 collects, as data for annotation, a target image that is an image to be annotated, an image captured at a time different from that of the target image, and a reference image in which annotation of the same object as the target object has been completed. are associated and output.
  • each image By displaying each image so that it can be compared using annotation data, for example, the operator who performs annotation can perform annotation work by referring to the presence or absence of changes in the target object and the past annotation results. This makes it easy to discriminate between objects and regions.
  • the reference image and the standard image when annotating, it is possible to suppress variation in judgment between the same worker and between workers. As a result, by using the image processing apparatus 10 of the present embodiment, it is possible to perform annotation efficiently and improve accuracy.
  • the worker can refer to an example of a mistake when annotating. Therefore, for example, it may be easier to determine the type of target object. Therefore, when the image processing apparatus 10 outputs an incorrect image in the past annotation result as a reference image, the accuracy of annotation can be further improved.
  • FIG. 17 is a diagram showing an overview of the configuration of the image processing apparatus 100.
  • the image processing apparatus 100 of this embodiment includes an area setting unit 101 , a reference image extraction unit 102 , a data generation unit 103 and an output unit 104 .
  • the region setting unit 101 sets a region in which an annotation target object may exist as a candidate region in an annotation target image.
  • a reference image extraction unit 102 extracts a reference image, which is an image in which the same object as the target object is captured, from the image for which annotation has been completed.
  • the data generation unit 103 generates, as annotation data, data in which an annotation target image, a reference image captured at a time different from the annotation target image for a region including a candidate region, and a reference image are associated.
  • the output unit 104 outputs the annotation data generated by the data generation unit 103 .
  • the area setting unit 11 is an example of the area setting unit 101 . Also, the area setting unit 101 is one aspect of area setting means.
  • the reference image extractor 13 is an example of the reference image extractor 102 . Further, the reference image extraction unit 102 is one aspect of reference image extraction means.
  • the data generator 14 is an example of the data generator 103 . Also, the data generation unit 103 is one aspect of data generation means.
  • the output unit 15 is an example of the output unit 104 . Also, the output unit 104 is one aspect of output means.
  • FIG. 18 is a diagram showing an example of the operation flow of the image processing apparatus 100.
  • the area setting unit 101 sets an area in which an annotation target object may exist as a candidate area in an annotation target image (step S101).
  • the reference image extracting unit 102 extracts a reference image, which is an image in which the same object as the target object is captured, from the image for which annotation has been completed (step S102).
  • the data generation unit 103 uses the annotation target image, the reference image captured at a time different from the annotation target image for the region including the candidate region, and the data associated with the reference image for annotation. It is generated as data (step S103).
  • the output unit 104 outputs the annotation data generated by the data generation unit 103 (step S104).
  • the image processing apparatus 100 of the present embodiment associates an annotation target image, a reference image captured at a time different from the annotation target image for a region including a candidate region, and a reference image that is an image for which annotation has been completed. Data is output as data for annotation. Therefore, by using the image processing apparatus 100, the operator can perform processing while comparing each image when annotating. As a result, by using the image processing apparatus 100 of the present embodiment, annotation processing can be performed efficiently and accuracy can be improved.
  • FIG. 19 shows an example of the configuration of a computer 200 that executes computer programs for performing processes in the image processing apparatus 10 of the first embodiment and the image processing apparatus 100 of the second embodiment.
  • the computer 200 includes a CPU (Central Processing Unit) 201 , a memory 202 , a storage device 203 , an input/output I/F (Interface) 204 and a communication I/F 205 .
  • CPU Central Processing Unit
  • the CPU 201 reads a computer program for each process from the storage device 203 and executes it.
  • the CPU 201 may be configured by a combination of a CPU and a GPU (Graphics Processing Unit).
  • the memory 202 is composed of a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores computer programs executed by the CPU 201 and data being processed.
  • the storage device 203 stores computer programs executed by the CPU 201 .
  • the memory device 203 is configured by, for example, a nonvolatile semiconductor memory device. Other storage devices such as a hard disk drive may be used as the storage device 203 .
  • the input/output I/F 204 is an interface that receives input from the operator and outputs display data and the like.
  • a communication I/F 205 is an interface that transmits and receives data to and from each device that constitutes the monitoring system. Further, the terminal device 30 can also have a similar configuration.
  • the computer program used to execute each process can also be stored and distributed in a non-temporarily recorded recording medium.
  • a recording medium for example, a magnetic tape for data recording or a magnetic disk such as a hard disk can be used.
  • an optical disc such as a CD-ROM (Compact Disc Read Only Memory) can be used.
  • a nonvolatile semiconductor memory device may be used as a recording medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Library & Information Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2022/032697 2021-09-29 2022-08-31 画像処理装置、画像処理方法および記録媒体 Ceased WO2023053830A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023550483A JP7694679B2 (ja) 2021-09-29 2022-08-31 画像処理装置、画像処理方法および画像処理プログラム
US18/579,930 US20240331418A1 (en) 2021-09-29 2022-08-31 Image processing device, image processing method, and recording medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-158568 2021-09-29
JP2021158568 2021-09-29

Publications (1)

Publication Number Publication Date
WO2023053830A1 true WO2023053830A1 (ja) 2023-04-06

Family

ID=85780605

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/032697 Ceased WO2023053830A1 (ja) 2021-09-29 2022-08-31 画像処理装置、画像処理方法および記録媒体

Country Status (3)

Country Link
US (1) US20240331418A1 (https=)
JP (1) JP7694679B2 (https=)
WO (1) WO2023053830A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118518040A (zh) * 2024-07-18 2024-08-20 洛阳市规划建筑设计研究院有限公司 基于InSAR技术的地表形变监测方法、系统及介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023127313A1 (ja) * 2021-12-28 2023-07-06 富士フイルム株式会社 撮像支援装置、撮像支援方法、及びプログラム
CN115762726A (zh) * 2022-11-30 2023-03-07 武汉联影医疗科技有限公司 超声图像文本注释添加方法和装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017151876A (ja) * 2016-02-26 2017-08-31 国立研究開発法人情報通信研究機構 画像クラスタリングシステム、画像クラスタリング方法、画像クラスタリングプログラム、および、コミュニティ構造検出システム
JP2019107084A (ja) * 2017-12-15 2019-07-04 キヤノン株式会社 医用画像装置及び医用画像の表示方法
WO2021186672A1 (ja) * 2020-03-19 2021-09-23 日本電気株式会社 画像処理方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5345963B2 (ja) 2010-02-16 2013-11-20 レノボ・シンガポール・プライベート・リミテッド 画像イメージを検索するタグデータの生成方法
US9501554B2 (en) 2013-03-06 2016-11-22 Rakuten, Inc. Image processing system, image processing method, and image processing program
JP6989485B2 (ja) 2018-12-21 2022-01-05 株式会社 日立産業制御ソリューションズ マルチラベルデータ学習支援装置、マルチラベルデータ学習支援方法およびマルチラベルデータ学習支援プログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017151876A (ja) * 2016-02-26 2017-08-31 国立研究開発法人情報通信研究機構 画像クラスタリングシステム、画像クラスタリング方法、画像クラスタリングプログラム、および、コミュニティ構造検出システム
JP2019107084A (ja) * 2017-12-15 2019-07-04 キヤノン株式会社 医用画像装置及び医用画像の表示方法
WO2021186672A1 (ja) * 2020-03-19 2021-09-23 日本電気株式会社 画像処理方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118518040A (zh) * 2024-07-18 2024-08-20 洛阳市规划建筑设计研究院有限公司 基于InSAR技术的地表形变监测方法、系统及介质
CN118518040B (zh) * 2024-07-18 2024-09-27 洛阳市规划建筑设计研究院有限公司 基于InSAR技术的地表形变监测方法、系统及介质

Also Published As

Publication number Publication date
JP7694679B2 (ja) 2025-06-18
US20240331418A1 (en) 2024-10-03
JPWO2023053830A1 (https=) 2023-04-06

Similar Documents

Publication Publication Date Title
JP7694679B2 (ja) 画像処理装置、画像処理方法および画像処理プログラム
KR101346539B1 (ko) 얼굴들을 상관시킴으로써 디지털 이미지들을 구조화하기
CN101681501B (zh) 图像处理装置、图像处理方法和存储介质
US10964057B2 (en) Information processing apparatus, method for controlling information processing apparatus, and storage medium
CN112559341A (zh) 一种画面测试方法、装置、设备及存储介质
US10347000B2 (en) Entity visualization method
US12154261B2 (en) Image defect detection method, electronic device and readable storage medium
CN103370732A (zh) 对象显示装置、对象显示方法和对象显示程序
CN113781414A (zh) 一种镜头解像力测试方法、装置和电子设备
JP2020030730A (ja) 家屋異動判読システム、家屋異動判読方法、家屋異動判読プログラム及び滅失判読モデル
JP7459927B2 (ja) 画像処理方法
CN111026924A (zh) 一种待搜索内容的获取方法及电子设备
JP7537518B2 (ja) 画像処理装置、画像処理方法および画像処理プログラム
US20250037491A1 (en) Method and device for scanning multiple documents for further processing
CN117253022A (zh) 一种对象识别方法、装置及查验设备
CN119228638A (zh) 图像拼接方法及电子设备
JP6399808B2 (ja) 画像処理装置、画像処理方法、及びプログラム
JP2020119368A (ja) 映像情報管理システムおよび映像情報管理方法
CN118644685B (zh) 一种图像识别方法、装置、存储介质及电子设备
JP2007199865A (ja) 画像処理アルゴリズム評価装置、画像処理アルゴリズムの生成装置および画像検査装置ならびに画像処理アルゴリズム評価方法、画像処理アルゴリズムの生成方法および画像検査方法
CN114529695B (zh) 全景图像处理与生成方法、装置、电子设备及存储介质
US20260017933A1 (en) Building inside structure recognition system and building inside structure recognition method
CN118411578A (zh) 标注数据集生成方法、装置及存储介质
JP2024136721A (ja) 遊技機検査システム
CN114937143A (zh) 旋转拍摄方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22875702

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023550483

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18579930

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22875702

Country of ref document: EP

Kind code of ref document: A1