WO2023062723A1 - 画像解析システム、画像解析方法およびプログラム - Google Patents

画像解析システム、画像解析方法およびプログラム Download PDF

Info

Publication number
WO2023062723A1
WO2023062723A1 PCT/JP2021/037743 JP2021037743W WO2023062723A1 WO 2023062723 A1 WO2023062723 A1 WO 2023062723A1 JP 2021037743 W JP2021037743 W JP 2021037743W WO 2023062723 A1 WO2023062723 A1 WO 2023062723A1
Authority
WO
WIPO (PCT)
Prior art keywords
product
identification result
image
degree
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/037743
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
八栄子 米澤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to US18/697,057 priority Critical patent/US20240412505A1/en
Priority to JP2023553798A priority patent/JP7647909B2/ja
Priority to PCT/JP2021/037743 priority patent/WO2023062723A1/ja
Publication of WO2023062723A1 publication Critical patent/WO2023062723A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns

Definitions

  • the present invention relates to technology for identifying products using images.
  • image-based processing requires a large amount of computation. Therefore, when image processing is performed on the entire image to increase the product identification accuracy, there is a risk that the processing time will increase beyond the permissible range.
  • the present invention has been made in view of the above problems.
  • One of the objects of the present invention is to provide a technique for improving the accuracy of product identification using images while suppressing the overall amount of processing.
  • the image analysis system in the present disclosure is product identification result acquisition means for acquiring identification results for each of a plurality of products appearing in an image;
  • the image area of the first product and the image of the second product calculating a first degree of matching, which is a degree of matching with the region, and determining whether or not to correct the identification result of the first product or the identification result of the second product based on the first degree of matching;
  • Necessity determination means Prepare.
  • the image analysis method in the present disclosure is the computer Acquire the identification results for each of the multiple products in the image, When the identification result of the first product among the plurality of products is different from the identification result of the second product adjacent to the first product, the image area of the first product and the image of the second product Calculate the first degree of matching, which is the degree of matching with the region, Determining whether the identification result of the first product or the identification result of the second product needs to be corrected based on the first degree of matching; Including.
  • FIG. 2 is a block diagram illustrating the hardware configuration of an information processing device having each functional component of the image analysis system; 4 is a flowchart illustrating the flow of processing executed by the image analysis system of the first embodiment;
  • FIG. 4 is a diagram showing an example of an image that is given to the image analysis system as a processing target; 5 is a diagram exemplifying an image area of each product appearing in the image of FIG. 4;
  • FIG. 5 is a diagram showing an example of identification results for each of a plurality of products appearing in the image of FIG. 4;
  • FIG. 7 is a diagram illustrating the functional configuration of an image analysis system according to a second embodiment;
  • FIG. 9 is a flowchart illustrating the flow of processing executed by the image analysis system of the second embodiment
  • FIG. 4 is a diagram showing an example of an image that is given to the image analysis system as a processing target
  • 10A and 10B are diagrams illustrating detection results of the placement member in the image of FIG. 9
  • FIG. It is a figure which shows an example of the processing result by a goods identification result acquisition part and a placement member detection part.
  • FIG. 11 is a diagram illustrating the functional configuration of an image analysis system according to a third embodiment
  • FIG. 10 is a flowchart illustrating the flow of processing executed by the image analysis system of the third embodiment
  • It is a figure which shows an example of the information which the correction necessity determination part of 3rd Embodiment outputs.
  • each block diagram does not represent a configuration in units of hardware, but a configuration in units of functions, unless otherwise specified.
  • the direction of the arrows in the figure is merely for the purpose of making the flow of information easier to understand.
  • the directions of arrows in the drawings do not limit the direction of communication (one-way communication/two-way communication) unless otherwise specified.
  • FIG. 1 is a diagram illustrating the functional configuration of an image analysis system according to the first embodiment.
  • the image analysis system 1 illustrated in FIG. 1 includes a product identification result acquisition unit 110 , a correction necessity determination unit 120 and a product identification result correction unit 130 .
  • the correction necessity determination unit 120 determines whether or not the identification result of each product needs to be corrected.
  • products of the same kind are collectively displayed in the store. Therefore, it can be said that the features (appearance features) of the image areas corresponding to each product are basically similar, except for the locations where the types of products actually switch.
  • the correction necessity determination unit 120 determines whether or not the identification result needs to be corrected by comparing the image areas in the portions where the product identification results are different.
  • the correction necessity determination unit 120 identifies a pair of products that have different product identification results and that are adjacent to each other from among the plurality of products shown in the image.
  • the correction necessity determining unit 120 for example, based on information acquired by the product identification result acquiring unit 110 (the position of the image area of each product and the identification result of the product in each image area), identifies such pairs of products. can be specified. In the following description, for convenience, one of a pair of products is also referred to as the "first product” and the other as the "second product”. Then, when the identification result of the first product and the identification result of the second product are different, the correction necessity determination unit 120 determines the image area corresponding to the first product and the image area corresponding to the second product. Calculate the degree of matching.
  • the correction necessity determining unit 120 can extract various feature amounts from each image area and compare the feature amounts extracted from each image area, thereby calculating the matching degree of the image areas.
  • the degree of matching calculated here is also referred to as "first degree of matching”. Then, the correction necessity determining unit 120 determines whether or not the identification result of the first product or the identification result of the second product needs to be corrected based on the calculated first degree of matching.
  • Each functional component of the image analysis system 1 may be implemented by hardware (eg, hardwired electronic circuit) that implements each functional component, or may be implemented by a combination of hardware and software (eg, combination of an electronic circuit and a program for controlling it, etc.).
  • hardware eg, hardwired electronic circuit
  • software e.g, combination of an electronic circuit and a program for controlling it, etc.
  • the bus 1010 is a data transmission path for the processor 1020, the memory 1030, the storage device 1040, the input/output interface 1050, and the network interface 1060 to exchange data with each other.
  • the method of connecting processors 1020 and the like to each other is not limited to bus connection.
  • the processor 1020 is a processor realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.
  • the memory 1030 is a main memory implemented by RAM (Random Access Memory) or the like.
  • the storage device 1040 is an auxiliary storage device realized by a HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, ROM (Read Only Memory), or the like.
  • the storage device 1040 stores program modules that implement each function of the image analysis system 1 described in this specification.
  • the processor 1020 reads the program module into the memory 1030 and executes it, so that each function of the image analysis system 1 described in this specification (product identification result acquisition unit 110, correction necessity determination unit 120, product identification result correction 130) are implemented.
  • the input/output interface 1050 is an interface for connecting the information processing apparatus 10 and peripheral devices.
  • the input/output interface 1050 can be connected to input devices such as keyboards, mice, and touch panels, and output devices such as displays and speakers.
  • the network interface 1060 is an interface for connecting the information processing device 10 to a network.
  • This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).
  • a method of connecting to the network via the network interface 1060 may be a wireless connection or a wired connection.
  • the information processing apparatus 10 can communicate with the terminal 20 owned by the store clerk or other external devices connected to the network via the network interface 1060 .
  • the hardware configuration shown in FIG. 2 is merely an example.
  • the hardware configuration of the image analysis system 1 according to the present disclosure is not limited to the example of FIG. 2 .
  • various functions of the image analysis system 1 according to the present disclosure may be implemented in a single information processing device, or may be distributed and implemented in a plurality of information processing devices.
  • the information processing device 10 having each function of the image analysis system 1 is depicted as a device different from the terminal 20 used by the store clerk. may be provided in the terminal 20 used by the store clerk.
  • the product identification result acquisition unit 110 acquires an image of a product captured by an imaging device (not shown) as an image to be processed (S102).
  • the image of the product is captured using, for example, a camera mounted on a terminal (eg, the terminal 20 shown in FIG. 2) possessed by the store clerk.
  • the store clerk photographs, for example, a place where products are displayed (such as a product shelf) using the camera function of the terminal.
  • the product identification result acquisition unit 110 can acquire product images from the terminal or from a server device (not shown) that collects and accumulates images generated by the terminal.
  • the product identification result acquisition unit 110 extracts image regions corresponding to individual objects (products) from the acquired image (S104).
  • the product identification result acquisition unit 110 uses, for example, an object recognition model (not shown) trained by a machine learning algorithm such as Deep Learning to recognize individual objects (objects that are presumed to be some product) in the image. can do.
  • "recognition” includes identifying the position of the image area corresponding to the object (eg, position coordinates in the image coordinate system).
  • this object recognition model is stored in advance in the storage device 1040 of the information processing apparatus 10 in FIG. 2, for example.
  • this object recognition model may be stored in an external device (not shown) communicatively connected to the information processing apparatus 10 of FIG.
  • the product identification result acquisition unit 110 uses each of the extracted image areas to identify the product corresponding to each image area (S106). For example, the product identification result acquisition unit 110 generates image feature information from each image region using a known method, and generates image feature information from each image region and master information of each product (when identifying each product). (feature information to be compared with) is calculated.
  • the master information of each product is stored in, for example, the storage device 1040 of the information processing apparatus 10 in FIG. 2 or an external storage device (not shown) communicably connected via the network interface 1060. .
  • the product identification result acquisition unit 110 identifies the product corresponding to each image area based on the degree of similarity calculated using the image feature information of each image area.
  • the processes of S104 and S106 described above may be executed by an external device (not shown).
  • the product identification result acquisition unit 110 obtains the image to be processed and the result of processing the image (detection position of the image area of each product in the image and product identification result in each image area). information) from an external device.
  • the correction necessity determining unit 120 identifies the first product and the second product to be processed based on the position of each image area and the identification result of the product corresponding to each image area (S108). For example, the correction necessity determination unit 120 compares the product identification results for two image areas that are adjacent to each other in either the up, down, left, or right direction. If the product identification results are different for the two image regions, the correction necessity determination unit 120 sets these two image regions as image regions corresponding to the first product and the second product to be processed, respectively. can be specified as
  • the correction necessity determination unit 120 determines whether or not the first degree of matching calculated in the process of S110 is equal to or greater than a predetermined reference value (S112).
  • the predetermined reference value is a value indicating a reference for judging that objects (products) appearing in each image area are identical in appearance, and an appropriate value is set in advance.
  • the predetermined reference value is set in advance in the memory 1030 or storage device 1040 of the information processing apparatus 10 in FIG. 2, for example.
  • the correction necessity determination unit 120 can refer to this reference value to determine whether or not the degree of matching calculated in S110 is equal to or higher than the reference value.
  • the correction necessity determining unit 120 determines that there is no need for correction, and does not execute the process of S114, which will be described later.
  • the predetermined reference value (S112: YES) there is a high probability that the first product and the second product are actually the same product. It can be said that there is an error in the identification result of one of them. Therefore, the correction necessity determining unit 120 determines that the correction is necessary. In this case, the product identification result correction unit 130 corrects the identification result regarding the first product and the identification result regarding the second product (S114).
  • FIG. 4 is a diagram showing an example of an image given to the image analysis system 1 as a processing target.
  • the product identification result acquisition unit 110 acquires an image as shown in FIG. 4, for example, using an object recognition model stored in the storage device 1040 of the information processing apparatus 10 in FIG. An image area corresponding to an object (product) is detected as shown in FIG.
  • FIG. 5 is a diagram exemplifying the image area of each product appearing in the image of FIG.
  • the product identification result acquisition unit 110 identifies a plurality of image areas within the image, as indicated by dotted-line rectangles in FIG. In the following description, when distinguishing between image areas, reference numerals 50-1 to 50-4 are used as illustrated.
  • the product identification result acquiring unit 110 for example, for each of the image regions 50-1 to 50-4, information indicating the shape of the image region (eg, position information of each vertex on the image and each vertex (information indicating the connection between them), and stores it in a predetermined storage area (eg, the storage device 1040 of the information processing apparatus 10 in FIG. 2). Based on the information stored in this way, the correction necessity determination unit 120 can identify the positional relationship of the image regions corresponding to each of the plurality of objects (products) in the image.
  • a predetermined storage area eg, the storage device 1040 of the information processing apparatus 10 in FIG. 2
  • the product identification result acquisition unit 110 generates image feature information for each of the plurality of image regions detected as shown in FIG. identify the product; For example, the product identification result acquisition unit 110 compares the image feature information with the master information of each product for each of a plurality of image areas, and identifies the most similar product from the comparison result (degree of matching with the master information). be able to.
  • the product identification result acquisition unit 110 associates the result of the product identified using each image area with the information indicating the position of each image area described above, and stores it in a predetermined storage area (eg, the information processing apparatus 10 in FIG. 2). storage device 1040).
  • the correction necessity determination unit 120 uses the result of processing by the product identification result acquisition unit 110 to compare the product identification results between two adjacent image areas, for example, starting from the left side of the image.
  • information as shown in FIG. 6, for example is obtained as a result of processing by the product identification result acquisition unit 110 .
  • FIG. 6 is a diagram showing an example of identification results for each of a plurality of commodities appearing in the image of FIG. In the example of FIG. 6, only the product in the image area 50-2 is identified as "beverage B", and the products in the other image areas are all identified as "beverage A".
  • the correction necessity determination unit 120 first determines the image area 50-1 and the image area 50-1 based on the positional relationship between the image area 50-1 and the image area 50-2 and the product identification results of these areas. A pair of regions 50-2 can be identified for processing. Then, the correction necessity determination unit 120 calculates a first degree of matching between the image regions 50-1 and 50-2, which indicates the degree of matching between the image regions (outer appearance features of the product).
  • correction necessity determination unit 120 recognizes that the product in image area 50-1 and the product in image area 50-2 are the same product, and the product identification result is Determine that correction is necessary. In this case, the correction necessity determining unit 120 requests the product identification result correcting unit 130 to execute correction processing. On the other hand, if the first degree of matching is less than the reference value, the correction necessity determining unit 120 recognizes that the product in the image area 50-1 and the product in the image area 50-2 are different products, and It is determined that no correction is necessary for the identification result.
  • the products placed at positions corresponding to the image areas 50-1 and 50-2 are actually the same product (beverage A).
  • the correction necessity determination unit 120 can find a relatively large number of points in common with respect to the image areas (appearance features of the product) between the image areas 50-1 and 50-2. As a result, a first degree of matching greater than or equal to the reference value is calculated for the image areas 50-1 and 50-2, and the correction necessity determining unit 120 determines whether the product identification result of the image area 50-1 or the image area 50 It is predicted that it will be determined that it is necessary to correct the product identification result of -2.
  • the product identification result correction unit 130 corrects the product identification result for one of the two target image areas in response to a request from the correction necessity determination unit 120 .
  • the product identification result correction unit 130 acquires the degree of matching with the master information used to identify the product in each image area as the second degree of matching, and uses the second degree of matching to identify the higher degree of matching.
  • a result can be determined as the product identification result of the two image regions.
  • the product identification result correction unit 130 determines the degree of matching (second degree of matching) with the master information of the product “drink A” for the image area 50-1 from the result of the product identification processing by the product identification result acquisition unit 110. can be obtained.
  • the product identification result correcting unit 130 obtains the matching degree (second matching degree ) can be obtained.
  • the product identification result acquisition unit 110 obtains the product identification result ( Beverage A) is corrected to the same identification result (beverage B) as the product identification result in the image area 50-2.
  • the product placed at positions corresponding to the image areas 50-1 and 50-2 is actually "beverage A”. Therefore, the second degree of matching regarding the image area 50-1 (the degree of matching with the master information of “drink A”) is the second degree of matching regarding the image region 50-2 (the degree of matching with the master information of “beverage B”). degree), and the product identification result correction unit 130 is expected to correct the product identification result of the image area 50-2 from “beverage B” to “beverage A”.
  • the product identification result correction unit 130 can determine that the same products are arranged side by side in the image areas 50-1 to 50-3. In this case, the product identification result correction unit 130 obtains many identification results of the product “beverage A” in the range where the same products are arranged side by side. The identification result of the product "B” is corrected to the identification result of the product "Beverage A”.
  • FIG. 7 is a diagram illustrating the functional configuration of an image analysis system according to the second embodiment.
  • the correction necessity determination unit 120 further includes a placement member detection unit 122 .
  • the placement member detection unit 122 acquires information indicating an image area of a placement member (eg, a shelf board of a product shelf) on which products are placed.
  • the correction necessity determination unit 120 of the present embodiment is configured to be able to recognize an area where products are stacked vertically (hereinafter also referred to as "stacked area"). Specifically, the correction necessity determination unit 120 identifies the position of the placement member within the processing target image based on the information acquired by the placement member detection unit 122 . By specifying the position of the placement member in the processed image, the correction necessity determination unit 120 can determine the stacked region from the positional relationship between the image region corresponding to each product and the image region of the placement member. It becomes possible. Then, the correction necessity determination unit 120 preferentially confirms the direction orthogonal to the placement member (the stacking direction of the products), and specifies the first product and the second product. When there are two commodities adjacent to each other without a placement member interposed therebetween, the correction necessity determining unit 120 preferentially identifies these two commodities as the first and second commodities. do.
  • FIG. 8 is a flowchart illustrating the flow of processing executed by the image analysis system 1 of the second embodiment. Differences from the flowchart of FIG. 3 will be mainly described below.
  • the correction necessity determining unit 120 identifies the position of the placement member in the image before identifying the first product and the second product to be processed (S202).
  • the placement member detection unit 122 can detect the region of the placement member in the image to be processed using a machine learning model capable of detecting the region of the placement member (shelf board) of the product. .
  • a machine learning model is constructed by performing training using learning data in which information indicating the area of the product placing member is given in advance, and is stored in the storage device 1040 of the information processing apparatus 10 in FIG. ing.
  • FIG. 9 is a diagram showing an example of an image given to the image analysis system 1 as a processing target.
  • the placement member detection unit 122 can obtain the results shown in FIG. 10 by using the machine learning model described above, for example.
  • 10A and 10B are diagrams illustrating detection results of the placement member in the image of FIG. 9.
  • the placement member detection unit 122 acquires the position information of the image area surrounded by the dotted line in FIG.
  • the correction necessity determination unit 120 can identify the position of the placement member in the image based on the stored information.
  • FIG. 11 is a diagram showing an example of processing results by the product identification result acquisition unit 110 and the placement member detection unit 122.
  • the correction necessity determining unit 120 can determine that the area above the upper placement member is an area where the products are not stacked (non-stacking area).
  • the correction necessity determining unit 120 can determine the area between the upper placement member and the lower placement member as an area where a plurality of products are stacked (stacked area).
  • the correction necessity determining unit 120 identifies the first product and the second product as described in the first embodiment (S108).
  • the correction necessity determination unit 120 confirms the product identification results along the stacking direction (vertical direction) of the products, and specifies the first product and the second product. .
  • the product identification results are different for the two image areas indicated by diagonal lines.
  • the correction necessity determination unit 120 identifies the two products corresponding to these image areas as the first product and the second product.
  • the correction necessity determining unit 120 identifies the first product and the second product based on the comparison results in the horizontal direction.
  • the correction necessity determination unit 120 requests the product identification result correction unit 130 to correct the product identification result based on the degree of matching between the specified image area of the first product and the second product (S110, S112).
  • the product identification result correction unit 130 corrects either the first product identification result or the second product identification result in response to a request from the correction necessity determination unit 120 (S114).
  • an area (stacking area) where the products are stacked and displayed is specified by detecting the placement member of the product. Then, in the stacking area, it is determined whether or not the product identification results differ along the direction in which the products are stacked. Here, basically, the same products are piled up and displayed. Therefore, by comparing the product identification results along the product stacking direction in the stacking area, it is possible to more efficiently detect the location where the product is erroneously identified.
  • FIG. 12 is a diagram illustrating the functional configuration of an image analysis system according to the third embodiment;
  • the image analysis system 1 of this embodiment has the same configuration as that of the first embodiment or the second embodiment, except that the product identification result correction unit 130 is not provided. That is, the product identification result acquisition unit 110 and the correction necessity determination unit 120 of this embodiment have the functions described in the first embodiment or the second embodiment.
  • the image analysis system 1 of this embodiment can be realized by a hardware configuration (eg, FIG. 2) similar to that of the first embodiment or the second embodiment.
  • the storage device 1040 in FIG. 2 stores program modules that implement the functions of the image analysis system 1, including the product identification result acquisition unit 110 and the correction necessity determination unit 120.
  • FIG. The processor 1020 in FIG. 2 loads each program module into the memory 1030 and executes it, thereby implementing the function corresponding to the loaded program module.
  • FIG. 13 is a flowchart illustrating the flow of processing executed by the image analysis system 1 of the third embodiment.
  • the processing of S302 to S312 in the flowchart of FIG. 13 is the same as the processing of S102 to S112 in FIG. Processing different from the first and second embodiments will be mainly described below.
  • the image analysis system 1 of this embodiment is configured to further execute the processing described in the second embodiment (eg, the processing of S202 and S204 in the flowchart of FIG. 8). good too.
  • the correction necessity determination unit 120 determines that the two image areas are the first product and the second product to be processed. 2 is identified as the image area corresponding to the product No. 2 (S308). Then, if the first degree of matching calculated by comparing the two specified image areas is equal to or greater than the predetermined reference value (S312: YES), the correction necessity determination unit 120 corrects the product identification result. determine that there is a need for In this case, the correction necessity determining unit 120 outputs information indicating that the product identification result needs to be corrected (S314).
  • the correction necessity determining unit 120 outputs information indicating that the product identification result needs to be corrected to a processing unit (not shown in the present embodiment) corresponding to the product identification result correcting unit 130 . This results in correction of potentially erroneous product identification results as described in other embodiments. Further, the correction necessity determining unit 120 may output information indicating that the product identification result needs to be corrected, for example, on the screen of the terminal 20 for the store clerk. For example, assume that an image as shown in FIG. 4 is given as an input image, and information as shown in FIG. 6 is obtained as a result of identifying a plurality of commodities appearing in the image. In this case, the correction necessity determination unit 120 of this embodiment may be configured to output information as shown in FIG.
  • FIG. 14 is a diagram showing an example of information output by the correction necessity determination unit 120 of the third embodiment.
  • the correction necessity determining unit 120 determines the identification result (beverage A) of the first product located on the far left of the input image and the second product located on the second from the left of the input image. , the two image areas (image area 50-1 and image area 50-2) corresponding to these products are specified as objects to be processed.
  • the correction necessity determination unit 120 determines the image area and the product identification result related to the image area, as shown in the figure. Change the display mode. Specifically, the correction necessity determining unit 120 highlights the frame or background indicating the image area, or highlights the identification result of the image area. Further, the correction necessity determination unit 120 may output a message prompting the user to confirm the product identification results of the two image areas to be processed.
  • the image analysis system 1 may further have a function of accepting input of information for correcting product identification results from the user. For example, when an image as shown in FIG. 14 is displayed on the screen of the store clerk's terminal 20, the store clerk using the terminal 20 uses the input device of the terminal 20 to select the target image area. perform an input operation that The image analysis system 1 displays on the screen a form for inputting information for correcting the product identification result of the selected image area in response to the input operation for selecting the image area being executed by the store clerk. Then, the store clerk inputs correction information (correct product identification result) into the form displayed on the screen, thereby correcting the product identification result of the selected image area. For example, the store clerk can determine from the image of FIG.
  • the store clerk can select the image area 50-2 as a target and perform an input to correct the product identification result to "beverage A".
  • the product identification result (beverage B) corresponding to the image area 50-2 in FIG. 14 is corrected to the product identification result (beverage A) input by the store clerk.
  • the image analysis system 1 is selected as the target.
  • Learning data may be generated by combining the image area and the information (label indicating correct answer information) input by the correction process. By feeding back such learning data to the product identification model that generated the product identification result acquired by the product identification result acquisition unit 110, the identification accuracy of the product identification model can be improved.
  • product identification result acquisition means for acquiring identification results for each of a plurality of products appearing in an image;
  • the image area of the first product and the image of the second product calculating a first degree of matching, which is a degree of matching with the region, and determining whether or not to correct the identification result of the first product or the identification result of the second product based on the first degree of matching;
  • Necessity determination means An image analysis system with 2.
  • the identification result of the first product and the identification result of the second product further comprising product identification result correction means for correcting one of the product identification results of 2; 1.
  • the product identification result correcting means includes: For each of the first product and the second product, obtaining a second degree of matching indicating the degree of matching with the master information of each product, determining, as the identification result of the first product and the second product, the identification result of the first product and the second product with the higher second degree of matching; 2.
  • the product identification result correcting means includes: When there is a third product that is adjacent to the first product at a position different from the second product and has the same identification result as the second product, the image area of the first product and the calculating a second degree of matching indicating the degree of matching with the image area of the third product; correcting the identification result of the first product if the second degree of matching is equal to or higher than the reference; 2.
  • the correction necessity determination means includes: Acquiring information indicating an image area of a placement member on which the product is placed, identifying two commodities adjacent to each other in a direction orthogonal to the placement member as the first and second commodities without interposing the placement member; 1. to 4.
  • the image analysis system according to any one of. 6.
  • the computer Acquire the identification results for each of the multiple products in the image, When the identification result of the first product among the plurality of products is different from the identification result of the second product adjacent to the first product, the image area of the first product and the image of the second product Calculate the first degree of matching, which is the degree of matching with the region, Determining whether the identification result of the first product or the identification result of the second product needs to be corrected based on the first degree of matching;
  • An image analysis method comprising: 7. the computer When it is determined that the identification result of the first product or the identification result of the second product needs to be corrected based on the first degree of matching, the identification result of the first product and the identification result of the second product Modifying one of the product identification results of 2, 6. The image analysis method described in . 8.
  • the computer For each of the first product and the second product, obtaining a second degree of matching indicating the degree of matching with the master information of each product, determining, as the identification result of the first product and the second product, the identification result of the first product and the second product with the higher second degree of matching; 7. including The image analysis method described in . 9. the computer When there is a third product that is adjacent to the first product at a position different from the second product and has the same identification result as the second product, the image area of the first product and the calculating a second degree of matching indicating the degree of matching with the image area of the third product; correcting the identification result of the first product if the second degree of matching is equal to or higher than the reference; 7. including The image analysis method described in . 10.
  • the computer Acquiring information indicating an image area of a placement member on which the product is placed, identifying two commodities adjacent to each other in a direction orthogonal to the placement member as the first and second commodities without interposing the placement member; 6. to 9.
  • image analysis system 10 information processing device 1010 bus 1020 processor 1030 memory 1040 storage device 1050 input/output interface 1060 network interface 110 product identification result acquisition unit 120 correction necessity determination unit 122 placement member detection unit 130 product identification result correction unit 20 terminal

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Image Analysis (AREA)
PCT/JP2021/037743 2021-10-12 2021-10-12 画像解析システム、画像解析方法およびプログラム Ceased WO2023062723A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/697,057 US20240412505A1 (en) 2021-10-12 2021-10-12 Image analysis system, image analysis method, and non-transitory computer-readable medium
JP2023553798A JP7647909B2 (ja) 2021-10-12 2021-10-12 画像解析システム、画像解析方法およびプログラム
PCT/JP2021/037743 WO2023062723A1 (ja) 2021-10-12 2021-10-12 画像解析システム、画像解析方法およびプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/037743 WO2023062723A1 (ja) 2021-10-12 2021-10-12 画像解析システム、画像解析方法およびプログラム

Publications (1)

Publication Number Publication Date
WO2023062723A1 true WO2023062723A1 (ja) 2023-04-20

Family

ID=85988456

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/037743 Ceased WO2023062723A1 (ja) 2021-10-12 2021-10-12 画像解析システム、画像解析方法およびプログラム

Country Status (3)

Country Link
US (1) US20240412505A1 (https=)
JP (1) JP7647909B2 (https=)
WO (1) WO2023062723A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120107344A (zh) * 2025-01-27 2025-06-06 未来机器人（深圳）有限公司 一种对齐状态确认方法、控制系统及搬运设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017102602A (ja) * 2015-11-30 2017-06-08 東芝テック株式会社 棚割り情報作成装置
JP2018097881A (ja) * 2014-09-30 2018-06-21 日本電気株式会社 情報処理装置、制御方法、及びプログラム
JP2018139062A (ja) * 2017-02-24 2018-09-06 株式会社マーケットヴィジョン 商品情報取得システム
WO2019107157A1 (ja) * 2017-11-29 2019-06-06 株式会社Ｎｔｔドコモ 棚割情報生成装置及び棚割情報生成プログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018097881A (ja) * 2014-09-30 2018-06-21 日本電気株式会社 情報処理装置、制御方法、及びプログラム
JP2017102602A (ja) * 2015-11-30 2017-06-08 東芝テック株式会社 棚割り情報作成装置
JP2018139062A (ja) * 2017-02-24 2018-09-06 株式会社マーケットヴィジョン 商品情報取得システム
WO2019107157A1 (ja) * 2017-11-29 2019-06-06 株式会社Ｎｔｔドコモ 棚割情報生成装置及び棚割情報生成プログラム

Also Published As

Publication number Publication date
JPWO2023062723A1 (https=) 2023-04-20
JP7647909B2 (ja) 2025-03-18
US20240412505A1 (en) 2024-12-12

Similar Documents

Publication Publication Date Title
US11049279B2 (en) Device for detecting positional relationship among objects
JP7279896B2 (ja) 情報処理装置、制御方法、及びプログラム
US20200372248A1 (en) Certificate recognition method and apparatus, electronic device, and computer-readable storage medium
US20200202095A1 (en) Positional relationship detection device and positional relationship detection system
US20160292628A1 (en) Method, and storage medium
JP2024091981A (ja) 情報処理装置、情報処理方法、およびプログラム
WO2019107157A1 (ja) 棚割情報生成装置及び棚割情報生成プログラム
US20230112215A1 (en) Monitoring device and monitoring method
JP6471835B2 (ja) 情報処理装置、制御方法、及びプログラム
WO2023062723A1 (ja) 画像解析システム、画像解析方法およびプログラム
US11580721B2 (en) Information processing apparatus, control method, and program
US20230386210A1 (en) Processing device, processing method, and non-transitory storage medium
WO2021256267A1 (ja) 情報処理システム、情報処理装置、及び、情報処理方法
JPWO2019064926A1 (ja) 情報処理装置、情報処理方法、およびプログラム
US20230290105A1 (en) Product detection device, product detection system, product detection method, and recording medium
JP7567166B2 (ja) 画像処理装置、画像処理方法、およびプログラム
JP6575628B1 (ja) 情報処理装置、情報処理システム、制御方法、及びプログラム
US12555259B2 (en) Product identification apparatus, product identification method, and non-transitory computer-readable medium
CN112532874B (zh) 生成平面热力图的方法、装置、存储介质和电子设备
CN113536822A (zh) 二维码校正方法、装置及计算机可读存储介质
WO2023062724A1 (ja) 画像解析システム、画像解析方法およびプログラム
US12079793B2 (en) Registration apparatus, registration method, and non-transitory storage medium
JP6703462B2 (ja) システム、情報処理装置、情報処理方法及びプログラム
US12205384B2 (en) Processing apparatus, processing method, and non-transitory storage medium
JP6939855B2 (ja) 情報処理装置、情報処理システム、制御方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21960582

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023553798

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21960582

Country of ref document: EP

Kind code of ref document: A1