WO2021214991A1 - モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム - Google Patents

モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム Download PDF

Info

Publication number
WO2021214991A1
WO2021214991A1 PCT/JP2020/017739 JP2020017739W WO2021214991A1 WO 2021214991 A1 WO2021214991 A1 WO 2021214991A1 JP 2020017739 W JP2020017739 W JP 2020017739W WO 2021214991 A1 WO2021214991 A1 WO 2021214991A1
Authority
WO
WIPO (PCT)
Prior art keywords
shape
region portion
model generation
image
object region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/017739
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
理史 藤塚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to US17/919,779 priority Critical patent/US20230177797A1/en
Priority to PCT/JP2020/017739 priority patent/WO2021214991A1/ja
Priority to JP2022516804A priority patent/JP7648192B2/ja
Publication of WO2021214991A1 publication Critical patent/WO2021214991A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/422—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects

Definitions

  • the present invention relates to a model generation system for recognizing the shape of an object, a shape recognition system, a model generation method, a shape recognition method, and a technical field of a computer program.
  • Patent Document 1 discloses a technique for identifying an object by using the characteristics (texture, color, shape, boundary, etc.) of the object.
  • Patent Document 2 discloses a technique for inferring the same object from the shape of the object.
  • Patent Document 3 discloses a technique for searching an image using the similarity of objects in the image.
  • the present invention has been made in view of the above problems, and provides a model generation system, a shape recognition system, a model generation method, a shape recognition method, and a computer program capable of appropriately recognizing the shape of an object. That is the issue.
  • One aspect of the model generation system of the present invention is an extraction means for extracting an object region portion which is an region occupied by an object from a target image, and machine learning is executed by inputting the object region portion as an input to classify the shape of the object. It is provided with a generation means for generating a shape classification model for the purpose.
  • One aspect of the shape recognition system of the present invention is to use an extraction means for extracting an object region portion which is an region occupied by an object from an object image and a shape classification model for classifying the shape of the object. It is provided with an estimation means for estimating the shape of the object of the portion.
  • One aspect of the model generation method of the present invention is to extract an object region portion, which is a region occupied by an object, from a target image, execute machine learning using the object region portion as an input, and classify the shape of the object. Generate a shape classification model.
  • the object region portion which is a region occupied by an object
  • the shape classification model for classifying the shape of the object is used to describe the object region portion. Estimate the shape of the object.
  • One aspect of the computer program of the present invention is a shape for extracting an object region portion which is an region occupied by an object from a target image, executing machine learning using the object region portion as an input, and classifying the shape of the object. Operate the computer to generate a classification model.
  • One aspect of the computer program of the present invention is to extract an object region portion, which is a region occupied by an object, from a target image, and use a shape classification model for classifying the shape of the object to use the object in the object region portion. Operate the computer to estimate the shape of.
  • model generation system the shape recognition system, the model generation method, the shape recognition method, and the embodiment of the computer program will be described with reference to the drawings.
  • FIG. 1 is a block diagram showing a hardware configuration of the model generation system according to the first embodiment.
  • the model generation system 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. It has.
  • the model generation system 10 may further include an input device 15 and an output device 16.
  • the CPU 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, and the output device 16 are connected via the data bus 17.
  • the CPU 11 reads a computer program.
  • the CPU 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14.
  • the CPU 11 may read a computer program stored in a computer-readable recording medium using a recording medium reading device (not shown).
  • the CPU 11 may acquire (that is, may read) a computer program from a device (not shown) located outside the model generation system 10 via a network interface.
  • the CPU 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the read computer program.
  • a functional block for generating a shape classification model for identifying the shape of an object is realized in the CPU 11.
  • the RAM 12 temporarily stores the computer program executed by the CPU 11.
  • the RAM 12 temporarily stores data temporarily used by the CPU 11 when the CPU 11 is executing a computer program.
  • the RAM 12 may be, for example, a D-RAM (Dynamic RAM).
  • the ROM 13 stores a computer program executed by the CPU 11.
  • the ROM 13 may also store fixed data.
  • the ROM 13 may be, for example, a P-ROM (Programmable ROM).
  • the storage device 14 stores data stored in the model generation system 10 for a long period of time.
  • the storage device 14 may operate as a temporary storage device of the CPU 11.
  • the storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.
  • the input device 15 is a device that receives an input instruction from the user of the model generation system 10.
  • the input device 15 may include, for example, at least one of a keyboard, a mouse and a touch panel.
  • the output device 16 is a device that outputs information about the model generation system 10 to the outside.
  • the output device 16 may be a display device (for example, a display) capable of displaying information about the model generation system 10.
  • FIG. 2 is a block diagram showing a functional block included in the model generation system according to the first embodiment.
  • the model generation system 10 includes an object region partial extraction unit 110 and a model generation unit 120. These functional blocks are realized, for example, in the CPU 11 (see FIG. 1).
  • the object area partial extraction unit 110 is configured to be able to extract an object region portion that is an area occupied by an object having a predetermined shape (in other words, a shape to be recognized) from the image data input to the system.
  • the object region partial extraction unit 110 extracts the object region portion using the insta-segmentation model 200.
  • FIG. 3 is a conceptual diagram showing extraction of an object region portion using an instance segmentation model.
  • the instance segmentation model 200 it is possible to extract only the object region portion from the image including the object. For example, from an image of a round object such as an apple or a golf ball, a mask image obtained by cutting out only the area occupied by them (that is, only the round area) can be extracted. Similarly, a mask image obtained by cutting out only the area occupied by them (that is, only the square area) can be extracted from the image of a square object such as a smartphone or a personal computer monitor.
  • a square object such as a smartphone or a personal computer monitor.
  • the instance segmentation model 200 is a model that extracts an object region portion by processing an image for each of a plurality of unit regions (for example, processing the image in pixel units), but the technology is existing. Therefore, a more detailed description here will be omitted. Further, although the method using the instance segmentation model is mentioned here, the object region portion may be extracted by another method.
  • the object region partial extraction unit 110 outputs the object region portion extracted using the instance segmentation model 200.
  • the information about the object region portion output from the object region portion extraction unit 110 is output to the model generation unit 120.
  • the object region partial extraction unit 110 is a specific example of the “extraction means”.
  • the model generation unit 120 is configured to be able to execute machine learning by using the object area portion extracted by the object region portion extraction unit 110 as input data (in other words, teacher data).
  • the model generation unit 120 generates a shape classification model for recognizing the shape of an object by this machine learning.
  • the object region portion may be manually annotated (for example, adding information indicating what the extracted shape is actually) before being input to the model generation unit 120. ..
  • An existing learning method can be appropriately applied to the machine learning of the model generation unit 120.
  • the model generation unit 120 is a specific example of the “generation means”.
  • FIG. 4 is a flowchart showing an operation flow of the model generation system according to the first embodiment.
  • an image data group composed of a plurality of image data is input to the model generation system 10 according to the first embodiment (step S101).
  • the image data group input here is image data obtained by capturing an object having a predetermined shape (for example, a round object or a square object) to be recognized by the shape classification model.
  • a predetermined shape for example, a round object or a square object
  • the object region portion extraction unit 110 extracts the object region portion occupied by an object having a predetermined shape from the input image data group (step S102). Then, the model generation unit 120 executes machine learning using the extracted object region portion as input data (step S103). The model generation unit 120 outputs a shape classification model for recognizing the shape of an object as a result of machine learning (step S104).
  • an object region portion is extracted using the instance segmentation model 200, and the shape classification model is obtained by machine learning in which the object region portion is input. Is generated.
  • the shape classification model generated in this way it is possible to appropriately recognize the shape of the object in the image. More specifically, by extracting the object region portion, it is possible to appropriately extract only the information regarding the shape of the object included in the image. For example, in a mask image as shown in FIG. 2, information other than the shape (for example, information about a color or a pattern) is scraped off, and only information about the shape of an object is surely extracted.
  • model generation system 10 it is possible to generate a shape classification model capable of appropriately recognizing the shape of an object.
  • a shape classification model by inputting an object region portion as an input, it is possible to realize recognition that allows ambiguity of the shape. Specifically, it is possible to recognize an ambiguous shape such as a round shape or a square shape (that is, a shape far from a beautiful square or a circle).
  • FIG. 5 is a block diagram showing a functional block included in the model generator according to the second embodiment.
  • the same components as those shown in FIG. 2 are designated by the same reference numerals.
  • the model generation device 10 according to the second embodiment includes an object area partial extraction unit 110, a model generation unit 120, a designated image extraction unit 130, and a box area extraction unit 140. .. That is, the model generation device 10 according to the second embodiment is configured to further include a designated image extraction unit 130 and a box area extraction unit 140 in addition to the configuration of the first embodiment (see FIG. 2). ..
  • the designated image extraction unit 130 is configured to be able to extract only an image including an object having a predetermined shape to be recognized from the image data group (that is, a plurality of image data) input to the model generation system 10. Has been done.
  • the designated image extraction unit 130 may be configured so that a predetermined shape can be specified. In this case, for example, when the user specifies a predetermined shape (s), the designated image extraction unit 130 displays an image including an object having the specified predetermined shape (hereinafter, appropriately referred to as a "designated image"). Extract only. More specifically, for example, when the user specifies a shape of "round", only an image including a round object such as an apple or a ball is extracted from a plurality of images.
  • the designated image extraction unit 130 extracts a designated image using the instance segmentation model 200. However, the designated image extraction unit 130 may extract the designated image without using the instance segmentation model 200.
  • the designated image extracted by the designated image extraction unit 130 is output to the box area extraction unit 140.
  • the designated image extraction unit 130 is a specific example of the "third extraction means".
  • the box area extraction unit 140 is a box area (specifically, a box area (specifically,) indicating the position of an object in the image from the designated image (that is, an image including an object having a predetermined shape) extracted by the designated image extraction unit 130. , A rectangular area surrounding an object) can be extracted.
  • the box area extraction unit 140 may extract a plurality of box areas from one designated image.
  • the box area extraction unit 140 extracts the box area using the instance segmentation model 200. However, the box area extraction unit 140 may extract the box area extraction unit 140 without using the instance segmentation model 200.
  • the box area extracted by the box area extraction unit 140 is configured to be output to the object area partial extraction unit 110.
  • the box area extraction unit 140 is a specific example of the “second extraction means”.
  • FIG. 6 is a flowchart showing an operation flow of the model generation system according to the second embodiment.
  • the same reference numerals are given to the same processes as those shown in FIG.
  • step S101 an image data group composed of a plurality of image data is input (step S101).
  • the designated image extraction unit 130 extracts a designated image including an object having a predetermined shape from the input image data group (step S102). Then, the box area extraction unit 140 extracts a box area indicating the position of the object from the designated image (step S103).
  • the object region partial extraction unit 110 extracts the object region portion occupied by an object having a predetermined shape from the extracted box region (step S102). Specifically, the object region partial extraction unit 110 extracts the object region portion by processing the rectangular region extracted as the box region, for example, in pixel units.
  • the model generation unit 120 executes machine learning using the extracted object region portion as input data (step S103).
  • the model generation unit 120 outputs a shape classification model for recognizing the shape of an object as a result of machine learning (step S104).
  • a designated image including an object having a predetermined shape is extracted from the image data group, and the position of the object is shown from the designated image.
  • the box area is extracted.
  • the object region portion can be extracted more easily and with high accuracy.
  • the color information for example, R, G, B information
  • the color information for example, red, green, blue, yellow, white, black, etc.
  • the pattern may be discriminated from the color distribution of the object and information regarding the pattern of the object may be added.
  • the color information described above may be added so as to be added to the information regarding the shape.
  • the model generation unit 120 may learn information about the shape of the object and information about the color to generate a model capable of recognizing the shape and color of the object.
  • the color information may be given in place of the information regarding the shape.
  • the model generation unit 120 may learn information about the color of the object and generate a model capable of recognizing the color of the object.
  • the shape recognition system 20 according to the third embodiment has some configurations and operations in common with the model generation system 10 according to the first and second embodiments described above (for example, the hardware configuration is shown in FIG. It may have the same configuration as the model generation system 10 shown in 1). Therefore, in the following, the matters already explained will be omitted, and the non-overlapping parts will be explained in detail.
  • FIG. 7 is a block diagram showing a functional block included in the shape recognition system according to the third embodiment.
  • the same components as those shown in FIGS. 2 and 5 are designated by the same reference numerals.
  • the shape recognition system 20 includes an object region partial extraction unit 110 and a shape estimation unit 150.
  • the object region partial extraction unit 110 is the same as that provided in the model generation system 10 according to the first and second embodiments (see FIGS. 2 and 5), and the image segmentation model 200 is used to obtain an image. It is configured so that the object area part can be extracted from the data.
  • the shape estimation unit 150 is configured to be able to estimate the shape of the object from the object region portion extracted by the object region portion extraction unit 110.
  • the shape estimation unit 150 estimates the shape of the object by using the shape classification model 300 (that is, the model generated by the model generation system 10 according to the first and second embodiments).
  • the shape estimation unit 150 is a specific example of the “estimation means”.
  • FIG. 8 is a flowchart showing an operation flow of the shape recognition system 20 according to the third embodiment.
  • image data is first input to the shape recognition system 20 according to the third embodiment (step S301).
  • the image input here is an image including an object whose shape is to be recognized.
  • a plurality of images may be input. In that case, the following processing may be executed for each image.
  • the object region partial extraction unit 110 extracts the object region portion occupied by an object having a predetermined shape from the input image (step S302). Then, the shape estimation unit 150 estimates the shape of the object corresponding to the extracted object region portion by using the shape classification model 300 (step S303). Finally, the shape estimation unit 150 outputs information indicating the shape of the object as the estimation result (step S304).
  • the shape estimation unit 150 may output information indicating which of the predetermined shapes the object corresponding to the object region portion has (for example, whether it is round or square with it). Specifically, a score indicating the roundness or a score indicating the squareness of the object may be output. This score may be output as, for example, a numerical value indicating the certainty indicating whether the object is a round object (or a square object). Further, when the object has a shape that is not classified into any of the predetermined shapes, information such as "unestimable" may be output.
  • FIG. 9 is a conceptual diagram showing a specific operation example of the shape recognition system according to the third embodiment.
  • FIG. 10 is a diagram (No. 1) showing a specific output example of the shape recognition system according to the third embodiment.
  • FIG. 11 is a diagram (No. 2) showing a specific output example of the shape recognition system according to the third embodiment.
  • the image shown in FIG. 9 includes a keyboard and a mouse.
  • the instance segmentation model 200 By applying the instance segmentation model 200 to such an image, it is possible to extract the object region portion of each of the keyboard and the mouse.
  • a score (0 to 1) indicating the shape of the object corresponding to the object region portion is displayed.
  • a score of "square (1.00)” is shown for the keyboard (keyboard). This result means that the keyboard in the image is very close to a square shape.
  • a score of "cycle (1.00)” is shown for the mouse. This result means that the mouse in the image is very close to a round shape.
  • the image shown in FIG. 10 includes a refrigerator and a microwave oven.
  • a score of "square (1.00)" is shown for the refrigerator (refrigerator). This result means that the refrigerator in the image is very close to a square shape.
  • the score of "square (1.00)” is also shown for the microwave oven. This result means that the microwave oven in the image is very close to a square shape.
  • the image shown in FIG. 11 includes a monitor (TV), a keyboard, a mouse, and a cup.
  • a score of "square (1.00)” is shown for the monitor (tv). This result means that the monitor in the image is very close to a square shape.
  • a score of "square (1.00)” is also shown for the keyboard (keyboard). This result means that the keyboard in the image is very close to a square shape.
  • a score of "cycle (1.00)” is shown for the mouse. This result means that the mouse in the image is very close to a round shape.
  • a score of "cycle (0.56)” is shown. This result means that the cup in the image is close to a slightly round shape.
  • the score indicating the shape of the object it is possible to intuitively grasp what kind of shape the object is.
  • the shape other than the round and square may be recognized.
  • it may be configured to recognize a triangular shape, a star shape, or a more complicated shape.
  • the object region portion is extracted using the instance segmentation model 200.
  • the shape of the object is estimated by using the shape classification model 300 for the object region portion.
  • the shape classification model 300 is generated as a model in which the shape of the object can be appropriately recognized, as already described in the first and second embodiments.
  • the shape is estimated after extracting the object region portion by the instance segmentation model 200, it is possible to estimate the shape of the object with extremely high accuracy.
  • the shape classification model generated by inputting the object region portion as an input it is possible to realize recognition that allows ambiguity of the shape. Specifically, it is possible to recognize an ambiguous shape such as a round shape or a square shape (that is, a shape far from a beautiful square or a circle).
  • the shape recognition system 20 according to the fourth embodiment will be described with reference to FIGS. 12 to 14. It should be noted that the fourth embodiment is different from the third embodiment described above only in a part of the configuration and operation, and the other parts are substantially the same. Therefore, in the following, the parts different from the third embodiment will be described in detail, and the description of other overlapping parts will be omitted as appropriate.
  • FIG. 12 is a block diagram showing a functional block included in the shape recognition system according to the fourth embodiment.
  • the same components as those shown in FIG. 7 are designated by the same reference numerals.
  • the shape recognition system 20 includes an object region partial extraction unit 110, a box region extraction unit 140, and a shape estimation unit 150. That is, the model generation device 10 according to the fourth embodiment is configured to further include a box region extraction unit 140 in addition to the configuration of the third embodiment (see FIG. 7). As described in the second embodiment, the box area extraction unit 140 extracts the box area indicating the position of the object from the image.
  • FIG. 13 is a flowchart showing an operation flow of the shape recognition system according to the fourth embodiment.
  • the same reference numerals are given to the same processes as those shown in FIG.
  • image data is first input (step S301).
  • the box area extraction unit 140 extracts a box area indicating the position of the object from the input image (step S401). Then, the object region partial extraction unit 110 extracts the object region portion occupied by the object having a predetermined shape from the extracted box region (step S302).
  • the shape estimation unit 150 estimates the shape of the object corresponding to the extracted object region portion using the shape classification model 300 (step S303). Finally, the shape estimation unit 150 outputs information indicating the shape of the object as the estimation result (step S304).
  • a box area indicating the position of an object is extracted from the input image.
  • the object region portion can be extracted more easily and with high accuracy.
  • FIG. 14 is a flowchart showing an operation flow of the shape recognition system according to the modified example.
  • the same reference numerals are given to the same processes as those shown in FIG.
  • the video data may be treated as a time series set of a plurality of image data.
  • N which is a parameter for counting repeated processing
  • step S501 is set to "1" (step S501).
  • "1" here is a predetermined initial value
  • the process of step S501 is a process of initializing N.
  • video data is input to the shape recognition system 20 (step S502).
  • the video data consists of T time-series image data.
  • the shape recognition system 20 extracts the Nth image data from the video data (step S503).
  • the box area extraction unit 140 extracts a box area indicating the position of the object from the extracted Nth image (step S401). Then, the object region partial extraction unit 110 extracts the object region portion occupied by the object having a predetermined shape from the extracted box region (step S302).
  • the shape estimation unit 150 estimates the shape of the object corresponding to the extracted object region portion using the shape classification model 300 (step S303). Then, the shape estimation unit 150 outputs information indicating the shape of the object as the estimation result (step S304).
  • step S505 NO
  • the process is executed again from step S503. Therefore, the processes of steps S503 to S504 are repeatedly executed until the processing of the last image data included in the video data is completed.
  • step S505: YES the series of processes ends.
  • the query of "When” can be handled by the information obtained from the time stamp of the video.
  • the "Where” query can be handled by the GPS information (latitude / longitude information) of the video.
  • the "What” query can be handled with information that can be obtained using existing object detection.
  • the shape recognition system 20 it is possible to respond to a "what (How)" query with information on the shape of the object recognized from the video data.
  • the user may specify the shape of the object, and the image including the object having the specified shape may be searched for and output from the plurality of image data constituting the video data.
  • the shape may be specified by the user using, for example, the input device 15 (see FIG. 1).
  • the output of the searched image may be performed using, for example, an output device 16 (see FIG. 1).
  • a search query such as "a round car seen in Kyoto last August” by extracting an object having a "round” shape.
  • the shape recognition system 20 according to the modified example has an extremely useful effect in the free text query search of the video data.
  • the model generation system according to Appendix 1 is for extracting the object region portion which is the region occupied by the object from the target image and executing machine learning by inputting the object region portion to classify the shape of the object. It is a model generation system characterized by having a generation means for generating the shape classification model of the above.
  • Appendix 2 The model generation system according to Appendix 2 is the model generation system according to Appendix 1, wherein the extraction means processes a target image for each of a plurality of unit regions to extract the object region portion.
  • the model generation system according to Appendix 3 further includes a second extraction means for extracting a rectangular region including the object from the target image, and the extraction means extracts the object region portion from the rectangular region.
  • the model generation system according to Appendix 4 uses a designation means for designating a shape to be classified by the shape classification model and an image including an object having a shape designated by the designation means as the target image from a plurality of images.
  • the model generation system according to any one of Supplementary note 1 to 3, further comprising a third extraction means for extraction.
  • the model generation system according to the appendix 5 is any one of the appendices 1 to 4, further comprising a color information imparting means for detecting the color of the object region portion and imparting the color information to the object region portion.
  • the shape recognition system uses an extraction means for extracting an object region portion, which is an region occupied by an object, from an object image, and a shape classification model for classifying the shape of the object, to use the object region portion. It is a shape recognition system characterized by including an estimation means for estimating the shape of the object.
  • the shape recognition system according to Appendix 7 is the shape recognition system according to Appendix 6, wherein the extraction means processes a target image for each of a plurality of unit regions to extract the object region portion.
  • the shape recognition system according to Appendix 8 further includes a second extraction means for extracting a rectangular region including the object from the target image, and the extraction means extracts the object region portion from the rectangular region.
  • the shape recognition system according to Appendix 9 includes a reception means that accepts the designation of the shape of the object, and an image including the object of the specified shape from a plurality of the target images based on the estimation result of the estimation means.
  • Appendix 10 The shape recognition system according to Appendix 10, wherein the estimation means estimates the color of the object in the object region portion in addition to the shape of the object in the object region portion.
  • the shape recognition system according to any one of the items.
  • the model generation method according to Appendix 11 extracts an object region portion that is an region occupied by an object from a target image, executes machine learning using the object region portion as an input, and classifies the shape of the object. It is a model generation method characterized by generating a model.
  • the shape recognition method according to Appendix 12 extracts an object region portion that is an area occupied by an object from a target image, and uses a shape classification model for classifying the shape of the object to use the shape classification model of the object region portion. It is a shape recognition method characterized by estimating a shape.
  • Appendix 13 The computer program according to Appendix 13 is a shape classification model for extracting an object region portion, which is an region occupied by an object, from an object image, executing machine learning using the object region portion as an input, and classifying the shape of the object. It is a computer program characterized by operating a computer so as to generate.
  • Appendix 14 The computer program according to Appendix 14 extracts an object region portion that is an area occupied by an object from an object image, and uses a shape classification model for classifying the shape of the object to form the shape of the object in the object region portion. It is a computer program characterized by operating a computer so as to estimate.
  • the present invention can be appropriately modified within the scope of claims and within a range not contrary to the gist or idea of the invention that can be read from the entire specification, and a model generation system, a shape recognition system, a model generation method, etc.
  • the shape recognition method and the computer program are also included in the technical idea of the present invention.
  • Model generation system 10
  • Shape recognition system 10
  • Object area partial extraction unit 120
  • Designated image extraction unit 140
  • Box area extraction unit 150
  • Shape estimation unit 200
  • Instance segmentation model 300
  • Shape classification model

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
PCT/JP2020/017739 2020-04-24 2020-04-24 モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム Ceased WO2021214991A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/919,779 US20230177797A1 (en) 2020-04-24 2020-04-24 Model generation system, shape recognition system, model generation method, shaperecognition method, and computer program
PCT/JP2020/017739 WO2021214991A1 (ja) 2020-04-24 2020-04-24 モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム
JP2022516804A JP7648192B2 (ja) 2020-04-24 2020-04-24 モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/017739 WO2021214991A1 (ja) 2020-04-24 2020-04-24 モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム

Publications (1)

Publication Number Publication Date
WO2021214991A1 true WO2021214991A1 (ja) 2021-10-28

Family

ID=78270636

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/017739 Ceased WO2021214991A1 (ja) 2020-04-24 2020-04-24 モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム

Country Status (3)

Country Link
US (1) US20230177797A1 (https=)
JP (1) JP7648192B2 (https=)
WO (1) WO2021214991A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024134450A (ja) * 2023-03-20 2024-10-03 日本碍子株式会社 分類方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019095827A (ja) * 2017-11-17 2019-06-20 日本電気株式会社 情報処理システム、情報処理方法及びプログラム
JP2019220116A (ja) * 2018-06-22 2019-12-26 日立造船株式会社 情報処理装置、判定方法、および対象物判定プログラム

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002032766A (ja) * 2000-07-18 2002-01-31 Matsushita Electric Ind Co Ltd 画像認識装置及びその方法
US10493629B2 (en) * 2016-05-27 2019-12-03 Ge Global Sourcing Llc Multisensory data fusion system and method for autonomous robotic operation
WO2012073421A1 (ja) * 2010-11-29 2012-06-07 パナソニック株式会社 画像分類装置、画像分類方法、プログラム、記録媒体、集積回路、モデル作成装置
JP5719230B2 (ja) * 2011-05-10 2015-05-13 キヤノン株式会社 物体認識装置、物体認識装置の制御方法、およびプログラム
JP7220062B2 (ja) * 2018-11-29 2023-02-09 富士通株式会社 学習データ生成プログラム、学習データ生成装置、及び学習データ生成方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019095827A (ja) * 2017-11-17 2019-06-20 日本電気株式会社 情報処理システム、情報処理方法及びプログラム
JP2019220116A (ja) * 2018-06-22 2019-12-26 日立造船株式会社 情報処理装置、判定方法、および対象物判定プログラム

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024134450A (ja) * 2023-03-20 2024-10-03 日本碍子株式会社 分類方法
JP7810670B2 (ja) 2023-03-20 2026-02-03 日本碍子株式会社 分類方法

Also Published As

Publication number Publication date
US20230177797A1 (en) 2023-06-08
JP7648192B2 (ja) 2025-03-18
JPWO2021214991A1 (https=) 2021-10-28

Similar Documents

Publication Publication Date Title
CN112434721B (zh) 一种基于小样本学习的图像分类方法、系统、存储介质及终端
Dvornik et al. On the importance of visual context for data augmentation in scene understanding
US10885365B2 (en) Method and apparatus for detecting object keypoint, and electronic device
AU2018250370B2 (en) Weakly supervised model for object detection
CN109447958B (zh) 图像处理方法、装置、存储介质及计算机设备
CN101097564A (zh) 参数学习方法及设备、模式识别方法及设备
US10937150B2 (en) Systems and methods of feature correspondence analysis
JP2018526754A (ja) 画像処理装置、画像処理方法及び記憶媒体
US9165220B2 (en) Image object recognition based on a feature vector with context information
US11314991B2 (en) Information display method, information display system, and storage medium
CN110910478B (zh) Gif图生成方法、装置、电子设备及存储介质
Nunes et al. A systematic review on open-set segmentation
Comia et al. Attention Mechanism-Based Dense Upsampling of Transfer Learning Mask RCNN for Improved Object Segmentation
Tan et al. Distinctive accuracy measurement of binary descriptors in mobile augmented reality
CN115049899B (zh) 模型训练方法、指代表达式生成方法及相关设备
JP7648192B2 (ja) モデル生成システム、形状認識システム、モデル生成方法、形状認識方法、及びコンピュータプログラム
EP4218246B1 (en) Detection of image space suitable for overlaying media content
US11127158B2 (en) Image indexing and retrieval using local image patches for object three-dimensional pose estimation
Klinsrisuk et al. ASDR: Automatic system to diagnose and recognize electrical drawings
CN116071833B (zh) 活体检验模型训练及活体检验方法、装置以及存储介质
CN118378717A (zh) 用于训练机器学习模型的系统和方法
EP4494041A1 (en) Data-efficient object detection of engineering schematic symbols
JP2025509822A (ja) 機械学習モデルと共に使用されるエンジニアリングスケマティックの訓練データのためのデータ拡張
CN115439868A (zh) 版面分析和模型训练方法、装置、电子设备和存储介质
CN111914866B (zh) 对抗图像检测模型的训练方法、装置和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20931999

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022516804

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20931999

Country of ref document: EP

Kind code of ref document: A1