WO2023195456A1

WO2023195456A1 - Electronic instrument, and method for controlling electronic instrument

Info

Publication number: WO2023195456A1
Application number: PCT/JP2023/013865
Authority: WO
Inventors: 暁艶戴; ジェチョルキム; 寛樹谷本; 依珊謝
Original assignee: 京セラ株式会社
Priority date: 2022-04-04
Filing date: 2023-04-03
Publication date: 2023-10-12

Abstract

An electronic instrument comprises a recognition unit that recognizes an article from an image in which the article is captured, and an identification unit that identifies the article along with the size of the article on the basis of the length of two sides forming a right angle of a rectangle circumscribing the article recognized by the recognition unit.

Description

Electronic devices and control methods for electronic devices

Cross-reference of related applications

This application claims priority to Japanese Patent Application No. 2022-62663 filed in Japan on April 4, 2022, and the entire disclosure of this earlier application is incorporated herein by reference.

The present disclosure relates to an electronic device and a method of controlling the electronic device.

In recent years, a technology has been proposed that automatically identifies an item, such as a product sold in a store, by capturing an image of the item using a digital camera. For example, Patent Document 1 discloses a technology that images a plurality of products arranged in a predetermined area and recognizes the products from the feature amounts of the captured images. It is expected that such technology will be increasingly introduced as a means of reducing labor costs and alleviating labor shortages by being adopted, for example, in unmanned cash registers at stores.

Japanese Patent Application Publication No. 2013-54673

An electronic device according to an embodiment includes:
a recognition unit that recognizes the article from an image of the article;
an identification section that identifies the article along with its size based on the lengths of two right-angled sides of a rectangle circumscribing the article recognized by the recognition section;
Equipped with.

Further, an electronic device according to an embodiment includes:
The article is identified along with its size based on the lengths of two perpendicular sides of a rectangle circumscribing the article, which are recognized from the image of the article.

Further, an electronic device according to an embodiment includes:
The article is identified along with its size based on a rectangle circumscribing the article recognized from an image of the article.

A method for controlling an electronic device according to an embodiment includes:
a recognition step of recognizing the article from an image of the article;
an identification step of identifying the article along with its size based on the lengths of two right-angled sides of a rectangle circumscribing the article recognized in the recognition step;
including.

FIG. 1 is a block diagram showing a functional configuration of an electronic device according to a first embodiment. FIG. 3 is a diagram illustrating processing by the electronic device according to the first embodiment. FIG. 3 is a diagram illustrating an example of information stored in a storage unit of the electronic device according to the first embodiment. 3 is a flowchart illustrating the operation of the electronic device according to the first embodiment. FIG. 7 is a diagram illustrating processing by an electronic device according to a second embodiment. It is a flow chart explaining operation of an electronic device concerning a 2nd embodiment. FIG. 3 is a block diagram showing a functional configuration of an electronic device according to a modification of the first embodiment. FIG. 7 is a diagram illustrating processing by an electronic device according to a modification of the first embodiment.

For example, it would be beneficial if electronic equipment could identify items of the same or similar form but different sizes, such as products that have a common design but are available in several sizes. An object of the present disclosure is to provide an electronic device that can identify articles of the same or similar shape but different sizes, and a method of controlling such an electronic device. According to one embodiment, it is possible to provide an electronic device that can identify articles of the same or similar shape but different sizes, and a method of controlling such an electronic device.

In the present disclosure, an "electronic device" may be a device driven by electric power. Further, the "system" may include equipment driven by electric power. Further, a "user" may be a person who uses an electronic device or a system according to an embodiment. A user may benefit from having items identified along with their size using an electronic device and/or system according to an embodiment.

The electronic device according to one embodiment may be used in a store such as a mass merchandiser, a supermarket, or a convenience store, for example. In such stores, it would be beneficial if items such as products of the same or similar shape but different sizes could be automatically identified (that is, without manual intervention). However, the scenarios in which the electronic device according to the embodiment is used are not limited to the above-mentioned stores. The electronic device according to the embodiment may be used, for example, in a factory to identify items such as devices and parts. In such factories, it would be beneficial if items such as equipment or parts of the same or similar shape but different sizes could be automatically identified (that is, without manual intervention).

(First embodiment)
Hereinafter, an electronic device according to a first embodiment will be described with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration of an electronic device according to a first embodiment, along with an example of its usage mode. As shown in FIG. 1, the electronic device 1 according to the first embodiment may include a recognition section 10 and an identification section 12. Further, the electronic device 1 according to the first embodiment may include a storage section 14, a communication section 16, an output section 18, and the like. The electronic device 1 according to one embodiment may not include some of the functional units shown in FIG. 1, or may include functional units other than those shown in FIG. 1 as appropriate.

The electronic device 1 according to the first embodiment can be configured as various devices. For example, the electronic device 1 according to the first embodiment may be a specially designed terminal, a general-purpose computer, a notebook computer, a server, or the like. Furthermore, the electronic device 1 according to the first embodiment may be a tablet computer, a smartphone, or the like. The electronic device 1 according to the first embodiment may be configured as any device that identifies articles of the same or similar form but different sizes.

The imaging unit 20 shown in FIG. 1 may include an image sensor that electronically captures images, such as a digital camera. The imaging unit 20 may include an imaging element that performs photoelectric conversion, such as a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The electronic device 1 and the imaging unit 20 may be connected by wire, wireless, or a combination of wire and wireless. In particular, the imaging unit 20 may be connected to the recognition unit 10 of the electronic device 1, for example. The image capturing unit 20 is not limited to an image capturing device such as a digital camera, and may be any device as long as it captures images of articles of the same or similar form but different sizes.

The imaging unit 20 captures an image of an article, for example, as shown in FIG. In FIG. 1, objects α1, α2, β1, and β2 placed on the board 100 are shown as objects to be imaged by the imaging unit 20. The board surface 100 may extend parallel to the XY plane shown in FIG. The Z axis shown in FIG. 1 may be directed upward in the vertical direction, for example. As shown in FIG. 1, it is assumed that the imaging unit 20 is arranged at a distance d from the board surface 100 in the positive direction of the Z-axis shown in FIG. That is, the imaging unit 20 is arranged to take images of the articles α1, α2, β1, and β2 placed on the board 100 from above.

FIG. 1 shows an example in which four articles α1, α2, β1, and β2 are placed on a board 100. However, the number of items placed on board 100 may be any number greater than one.

FIG. 1 shows an example in which the imaging unit 20 images objects α1, α2, β1, and β2 placed on the board 100 from above. However, in one embodiment, the imaging unit 20 may image the objects α1, α2, β1, and β2 placed on the board 100, for example, from the side (such as in a direction parallel to the XY plane). The imaging unit 20 may be installed at any position and angle at which the external shape of the item placed on the board 100 can be imaged. Further, FIG. 1 shows an example in which only one imaging unit 20 is installed. However, in one embodiment, two or more imaging units 20 may be installed. In one embodiment, by installing two or more imaging units 20, it is possible to improve the accuracy of identifying the article and the size of the article.

Information on the image captured by the imaging unit 20 may be supplied to the electronic device 1. Information on the image captured by the imaging unit 20 may be supplied to, for example, the recognition unit 10 of the electronic device 1. On the other hand, information on the image captured by the imaging unit 20 may be supplied to, for example, the identification unit 12, the storage unit 14, the communication unit 16, and/or the output unit 18 of the electronic device 1.

The recognition unit 10 recognizes the article included in the image captured by the imaging unit 20. The recognition unit 10 may recognize the article included in the image captured by the image capture unit 20 using any known image recognition technology. The recognition unit 10 may have a function of extracting an article included in the image captured by the image capturing unit 20 based on predetermined feature points of the article.

In this way, in the electronic device 1 according to the first embodiment, the recognition unit 10 recognizes the article from the image of the article (the image captured by the imaging unit 20). Information on the article recognized by the recognition unit 10 may be supplied to, for example, the identification unit 12 of the electronic device 1. On the other hand, the information on the article recognized by the recognition unit 10 may be supplied to, for example, the storage unit 14, the communication unit 16, and/or the output unit 18 of the electronic device 1.

The identification unit 12 identifies the article recognized by the recognition unit 10 along with the size of the article. Identification of articles by the identification unit 12 of the electronic device 1 according to the first embodiment will be further described later. Information on the article recognized by the identification unit 12 together with the size may be supplied to, for example, the storage unit 14, the communication unit 16, and/or the output unit 18 of the electronic device 1.

At least one of the recognition unit 10 and the identification unit 12 includes at least one CPU, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), in order to provide control and processing capabilities for performing various functions. may include one processor. At least one of the recognition unit 10 and the identification unit 12 may be implemented by a single processor, several processors, or individual processors. A processor may be implemented as a single integrated circuit (IC). A processor may be implemented as a plurality of communicatively connected integrated and discrete circuits. The processor may be implemented based on various other known technologies.

In one embodiment, at least one of the recognition unit 10 and the identification unit 12 may be configured as, for example, a CPU and a program executed by the CPU. A program executed in at least one of the recognition unit 10 and the identification unit 12, a result of a process executed in at least one of the recognition unit 10 and the identification unit 12, etc. may be stored in the storage unit 14, for example. At least one of the recognition unit 10 and the identification unit 12 may appropriately include memory necessary for operation. At least one of the recognition unit 10 and the identification unit 12 may be configured as dedicated hardware, may include at least a portion of software, or may be configured entirely of software.

The storage unit 14 may have a function as a memory that stores various information. The storage unit 14 may store, for example, a program executed in at least one of the recognition unit 10 and the identification unit 12, a result of a process executed in at least one of these, and the like. In the first embodiment, the storage unit 14 may store size information of each of a plurality of articles in association with each of the plurality of articles. The storage unit 14 may have a function as a database that stores various information. The size information of each of the plurality of articles stored in the storage unit 14 will be further described later.

Furthermore, the storage unit 14 may function as a work memory for at least one of the recognition unit 10 and the identification unit 12. The storage unit 14 can be configured by, for example, a semiconductor memory or the like, but is not limited thereto, and can be any storage device. For example, the storage unit 14 may be a storage medium such as a memory card inserted into the electronic device 1 according to one embodiment. Further, the storage unit 14 may be an internal memory of a CPU used as at least one of the recognition unit 10 and the identification unit 12, or may be connected to at least one of the recognition unit 10 and the identification unit 12 as a separate body. Good too.

The communication unit 16 has an interface function for wired or wireless communication. The communication method performed by the communication unit 16 in one embodiment may be a wireless communication standard. For example, wireless communication standards include cellular phone communication standards such as 2G, 3G, 4G, and 5G. For example, communication standards for cellular phones include LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (registered trademark) (Global System for Mobile communications), and PHS. (Personal Handy-phone System), etc. For example, wireless communication standards include WiMAX (Worldwide Interoperability for Microwave Access), IEEE802.11, WiFi, Bluetooth (registered trademark), IrDA (Infrared Data Association), NFC (Near Field Communication), and the like. Communication unit 16 may support one or more of the communication standards described above. The communication unit 16 may include, for example, an antenna for transmitting and receiving radio waves, a suitable RF unit, and the like. Further, the communication unit 16 may be configured as an interface such as a connector for connecting to the outside by wire. Since the communication unit 16 can be configured using a known technique for performing wireless communication, a detailed description of the hardware and the like will be omitted.

Various types of information received by the communication unit 16 may be supplied to, for example, the storage unit 14, the recognition unit 10, and/or the identification unit 12. Various types of information received by the communication unit 16 may be stored in the storage unit 14 and/or other memory, for example. Further, the communication unit 16 may transmit to the outside, for example, a processing result by at least one of the recognition unit 10 and the identification unit 12, and/or information stored in the storage unit 14.

The output unit 18 outputs the identification result by the identification unit 12. The output unit 18 may output the article and/or size information of the article as the identification result by the identification unit 12. The output unit 18 may be any functional unit that stimulates at least one of the user's auditory, visual, and tactile senses, such as sound, voice, light, text, video, and vibration, as output of the identification result by the identification unit 12. . Specifically, the output unit 18 may be at least one of a sound output unit such as a buzzer or a speaker, a light emitting unit such as an LED, a display unit such as an LCD, and a tactile sensation presentation unit such as a vibrator. .

The electronic device 1 shown in FIG. 1 includes an output section 18. However, in the electronic device 1 according to one embodiment, the output unit 18 may be provided outside the electronic device 1. In this case, the output unit 18 may function as an interface connected to external equipment. In this case, the output unit 18 and the electronic device 1 may be connected by wire, wireless, or a combination of wire and wireless.

Next, the operation of the electronic device 1 according to the first embodiment will be described.

Conventionally, attempts have been known to recognize (the type of) an article from an image of the article. For example, it is assumed that the article α1 and the article β1 shown in FIG. 1 are articles having different shapes. In this case, for example, by employing image recognition technology, the article α1 shown in FIG. 1 is recognized from the captured image, and the article β1 shown in FIG. 1 is recognized from the captured image. There are known techniques to do this. Furthermore, a technique is also known that employs image recognition technology or the like to distinguish and recognize the article α1 and the article β1 from an image captured including the article α1 and the article β1 shown in FIG. .

However, with the techniques described above, although it is possible to distinguish between articles having different shapes (outer shapes), it is difficult to distinguish between articles having the same or similar shapes but different sizes. For example, it is assumed that the article α1 and the article α2 shown in FIG. 1 are articles having the same or similar shapes but different sizes. That is, it is assumed that the article α1 and the article α2 have the same or similar designs, and the article α1 is small in size and the article α2 is large in size. In this case, it has been difficult to improve the accuracy of distinguishing and recognizing the article α1 and the article α2 from the captured image including the article α1 and the article α2 shown in FIG.

Therefore, in the first embodiment, articles of the same or similar form but different sizes are identified based on information on the size of each article and information on the distance at which the article was imaged.

FIG. 2 is a diagram illustrating identification processing by the electronic device 1 according to the first embodiment. FIG. 2 is a diagram illustrating an example of how the imaging unit 20 captures an image of an article from above with the article placed on the board 100 as shown in FIG. In FIG. 2, articles A1, A2, and B placed on the board 100 are shown as objects to be imaged by the imaging unit 20. The board surface 100 shown in FIG. 2 may have an extension parallel to the XY plane shown in FIG. 1, as in FIG. Further, the Z axis shown in FIG. 2 may also be directed upward in the vertical direction, for example, similarly to FIG. 1. In this way, it is assumed that the imaging unit 20 is arranged so as to image the articles A1, A2, and B placed on the board 100 from above.

In the first embodiment, the concept of a rectangle (right quadrilateral) circumscribing the article recognized by the recognition unit 10 is introduced. For an article included in an image captured, for example, from above by the imaging unit 20, a rectangle (right-angled quadrilateral) circumscribing the article can be defined. For example, although articles A1 and A2 shown in FIG. 2 each have a circular (or elliptical) shape (outer shape), a rectangle circumscribing each can be defined. In the example shown in FIG. 2, rectangles circumscribing each of articles A1 and A2 have sides parallel to the X-axis and Y-axis shown in the figure. However, the rectangles circumscribing each of the articles A1 and A2 do not necessarily have sides parallel to the X-axis and the Y-axis, but may have sides based on a predetermined direction. Since all corners of a rectangle are right angles, the size of an arbitrary article can be defined by the two right-angled sides of the rectangle that circumscribe the article.

Therefore, in the first embodiment, the lengths of two right-angled sides of a rectangle circumscribing the article recognized by the recognition unit 10 can be expressed as (x, y), for example. For example, as shown in FIG. 2, the lengths of two right-angled sides of a rectangle circumscribing the article A1 can be expressed as (x1, y1). Further, as shown in FIG. 2, the lengths of two right-angled sides of a rectangle circumscribing the article A2 are shown as (x2, y2).

Furthermore, for example, when the shape (outer shape) of the article itself is a rectangle, such as article B shown in FIG. 2, the rectangle may be a rectangle that circumscribes article B. In this case, for example, the lengths of two right-angled sides of a rectangle circumscribing article B shown in FIG. 2 may be (x3, y3).

In the first embodiment, size information for each of a plurality of articles may be stored in the storage unit 14, for example, in association with each of the plurality of articles, in order to identify the article together with the size of the article. Here, the size information of each of the plurality of articles may be, for example, the length of two right-angled sides of a rectangle circumscribing the article recognized by the recognition unit 10, as described above. That is, the storage unit 14 of the electronic device 1 according to the first embodiment stores the lengths of two right-angled sides of a rectangle that circumscribes each of the plurality of articles recognized by the recognition unit 10 for each of the plurality of articles. You can remember it in association.

FIG. 3 is a diagram showing an example of information stored in the storage unit 14 of the electronic device 1 according to the first embodiment.

As shown in FIG. 3, the electronic device 1 according to the first embodiment collects information such as the lengths of two right-angled sides of a rectangle circumscribing the article recognized by the recognition unit 10, for each of the plurality of articles. For example, they may be stored in the storage unit 14 in association with each other. The table shown in FIG. 3 shows how product ID, product name, size information x, size information y, price, and other items are stored in association with each product (article) from the left column. Shown conceptually.

In FIG. 3, the product ID may be, for example, the identification number of the product. For example, the product ID indicating article A1 may be set to "1", and the product ID indicating article A2 may be set to "2". Further, in FIG. 3, the product name may be the name of the article. For example, the product name indicating article A1 may be "Food A (large)" and the product name indicating article A2 may be "Food A (medium)." Further, in FIG. 3, the size information x and y may be the lengths (x, y) of two right-angled sides of a rectangle circumscribing the article. For example, the lengths of two right-angled sides of a rectangle circumscribing the article A1 may be 60 mm and 60 mm. Further, the lengths of two right-angled sides of the rectangle circumscribing the article A2 may be set to 45 mm and 45 mm, for example. Further, in FIG. 3, the price may be the suggested retail price of the article. For example, the price of article A1 may be set to "240 yen," and the price of article A2 may be set to "130 yen." Further, in FIG. 3, other items may be any information such as information indicating the direction in which the lengths (x, y) of two right-angled sides of a rectangle circumscribing the article are measured. The table shown in FIG. 3 is merely illustrative. The storage unit 14 of the electronic device 1 according to one embodiment may not store part of the information on the items shown in FIG. 3, or may store information on items other than those shown in FIG. 3 as appropriate.

The electronic device 1 according to the first embodiment may acquire information that may be included in the table shown in FIG. 3 and store it in the storage unit 14, for example, by the time the article identification operation is started. In one embodiment, the electronic device 1 may acquire the actual measurement value of the size information of each article by the start of the article identification operation, typically when registering the handled products. Here, as the actual measurement value of the size information of each article, a value actually measured by a user or the like may be input, or a specification such as a catalog value may be input. Further, in one embodiment, the electronic device 1 may acquire size information of each article from an image captured by the imaging unit 20 of the electronic device 1 before starting the article identification operation. Further, the electronic device 1 according to the first embodiment may acquire information that may be included in the table shown in FIG. 3 in advance and store it in the storage unit 14. Further, the electronic device 1 according to the first embodiment may acquire information that may be included in the table shown in FIG. 3 from, for example, an external server via the communication unit 16 as necessary.

For example, in the electronic device 1 according to one embodiment, if the actual size of the article imaged by the imaging unit 20 can be determined, the article can be identified together with the size by referring to (verifying) information as shown in FIG. can do. However, in the electronic device 1, it is difficult to accurately determine the actual size of the article from the image of the article captured by the imaging unit 20.

As shown in FIG. 1, it is assumed that the imaging unit 20 is arranged a distance d from the board surface 100 in the positive direction of the Z-axis shown in FIG. For example, as a usage mode of the electronic device 1, when an article is placed on the board 100 by a user or the like, the article may be identified along with its size based on an image captured by the imaging section 20. In this case, the positional relationship between the board 100 on which the article is placed and the imaging section 20 may be fixed. In this way, the distance d shown in FIG. 1 can be set to a fixed value. Moreover, the length of the distance d shown in FIG. 1 can be measured in advance by actual measurement. Therefore, in the electronic device 1 according to the first embodiment, the distance d shown in FIG. 1 may be a known fixed value.

Therefore, the electronic device 1 according to the first embodiment corrects the lengths of the two right-angled sides of the rectangle circumscribing the article recognized by the recognition unit 10 according to the distance d, which is a known fixed value. Good too. Here, the article recognized by the recognition unit 10 may be an article included in an image captured by the imaging unit 20. Further, the distance d may be, for example, as shown in FIG. 1, a distance from a position where an article is placed to a position where an imaging unit 20 that captures an image of the article is placed.

In general, as the distance d shown in FIG. 1 becomes longer, the lengths of two right-angled sides of a rectangle circumscribing the article recognized from the image captured by the imaging unit 20 tend to become shorter. Therefore, in accordance with the distance d (known fixed value) shown in FIG. may be determined, for example, by actual measurement. In addition, in accordance with the distance d (known fixed value) shown in FIG. may be determined, for example, from a theoretical value.

The corrected lengths of the two right-angled sides of the rectangle circumscribing the article recognized from the image captured by the imaging unit 20 as described above are the two right-angled sides of the rectangle circumscribing the actual article. can be equated with the length of Therefore, according to the electronic device 1, the size information in the storage unit 14 is stored based on the corrected lengths of the two right-angled sides of the rectangle circumscribing the object recognized from the image captured by the imaging unit 20. By reference, the article can be identified along with its size.

FIG. 4 is a flowchart explaining the operation of the electronic device 1 according to the first embodiment. Hereinafter, with reference to FIG. 4, the operation (identification operation) of the electronic device 1 according to the first embodiment will be described.

The operation shown in FIG. 4 may be started, for example, as shown in FIG. 1, with an article placed on the board 100 and the imaging unit 20 capturing an image of the article. In such a situation, the electronic device 1 according to the first embodiment identifies the item placed on the board 100 along with its size. Furthermore, at the time when the operation shown in FIG. 4 starts, the distance d shown in FIG. 1 is assumed to be a known fixed value, as described above. Furthermore, at the time when the operation shown in FIG. 4 starts, as described above, the ratio for correcting the lengths of the two right-angled sides of the rectangle circumscribing the article recognized from the image captured by the imaging unit 20 is known. (I have asked for it).

When the operation shown in FIG. 4 starts, in step S11, the electronic device 1 according to the first embodiment acquires an image of the article captured by the imaging unit 20. Next, in step S12, the recognition unit 10 recognizes, from the image acquired by the electronic device 1, an article included in (shown in) the image. Here, the number of recognized articles may be one, or two or more. Information on the article recognized by the recognition section 10 is supplied to the identification section 12 .

Next, in step S13, the identification unit 12 extracts a rectangle that circumscribes the article recognized by the recognition unit 10. Here, the number of rectangles to be extracted may be equal to the number of articles recognized by the recognition unit 10. Furthermore, in step S13, the rectangle circumscribing the article recognized by the recognition unit 10 does not necessarily have to be extracted as an image through image processing. In step S13, a rectangle circumscribing the article recognized by the recognition unit 10 may be conceptually extracted.

Once the above-mentioned rectangle is extracted, in step S14, the identification unit 12 extracts the lengths (x, y) of two right-angled sides of the extracted rectangle. Next, in step S15, the identification unit 12 corrects the extracted lengths of the two sides (x, y) according to the distance d to the article. Hereinafter, the length of the two sides (x, y) corrected in step S15 will also be referred to as the "corrected length of the two sides (x', y')."

Next, in step S16, the identification unit 12 refers to the size information in the storage unit 14 to match the corrected lengths of the two sides (x', y'). If the corrected lengths of the two sides (x', y') are matched, in step S17, the identification unit 12 identifies the product ID and/or name of the matched product as the product. It may be identified along with the size etc.

The thus identified article and the size of the article may be output from the output unit 18, for example, by sound, display, and/or tactile sensation. Further, the identified article and the size of the article may be transmitted to another device such as an external server via the communication unit 16. Further, the identified article and the size of the article may be stored in the storage unit 17. Additionally, the identified item and item size may be registered in a checkout machine, such as an unmanned cash register.

In this way, the identification unit 12 of the electronic device 1 according to the first embodiment identifies the item based on the lengths (x, y) of the two right-angled sides of the rectangle circumscribing the item recognized by the recognition unit 10. Identify the item along with the size of the item. Furthermore, the storage unit 14 of the electronic device 1 according to the first embodiment may store the lengths of two right-angled sides of a rectangle circumscribing each of the plurality of articles in association with each of the plurality of articles. In this case, the identification unit 12 of the electronic device 1 according to the first embodiment may perform the above-described identification by referring to information stored in the storage unit 14. Further, the identification unit 12 generates a rectangle circumscribing the item recognized by the recognition unit 10 according to a distance d from a position where the item is placed to a position where an imaging unit 20 that captures an image of the item is placed. The lengths of two sides forming right angles may be corrected.

As explained above, according to the electronic device 1 according to the first embodiment, it is possible to identify articles of the same or similar form but different sizes with good accuracy. Therefore, according to the electronic device 1 according to the first embodiment, it is possible to distinguish between articles having a common design and different sizes.

(Second embodiment)
Next, an electronic device according to a second embodiment will be described.

The electronic device according to the second embodiment may have the same or similar configuration to the electronic device 1 according to the first embodiment shown in FIG. 1. Therefore, the electronic device according to the second embodiment will be hereinafter referred to as the electronic device 1 according to the second embodiment. The electronic device 1 according to the second embodiment can be configured similarly to the electronic device 1 according to the first embodiment, but performs different processing from the electronic device 1 according to the first embodiment. Regarding the electronic device 1 according to the second embodiment, description of contents that are the same or similar to those of the electronic device 1 shown in FIGS. 1 to 4 may be simplified or omitted as appropriate.

In the first embodiment, the distance d shown in FIG. 1 is a known fixed value. However, depending on the actual usage, there may be situations where the distance d shown in FIG. 1 cannot be accurately known in advance. Furthermore, there may be a situation where the distance d shown in FIG. 1 is variable in the first place, for example because the imaging unit 20 is movable.

Therefore, the electronic device 1 according to the first embodiment makes it possible to identify the article together with the size of the article even when the distance d shown in FIG. 1 is an unknown value or a variable value.

FIG. 5 is a diagram illustrating identification processing by the electronic device 1 according to the second embodiment. In the second embodiment, in addition to the articles A1, A2, and B placed on the board 100, a predetermined reference mark R is also imaged by the imaging unit 20, as shown in FIG. This reference mark R may be imaged by the imaging unit 20 so that it fits in the same image as the article whose size is to be identified. Further, as long as the distance d shown in FIG. 1 remains unchanged, the reference mark R may be imaged by the imaging unit 20 as an image different from the image of the article whose size is to be identified.

In the second embodiment, instead of the distance d shown in FIG. 1 being a known fixed value, the actual length of a predetermined portion of the reference mark R is a known fixed value. Further, the actual length of the predetermined portion of the reference mark R shown in FIG. 5 can be measured in advance by actual measurement. Therefore, in the electronic device 1 according to the second embodiment, the actual length of the predetermined portion of the reference mark R shown in FIG. 5 may be set to a known fixed value. For example, it is assumed that the actual measurement value of the length of the reference mark R shown in FIG. 5 in the lateral direction (direction parallel to the X-axis) is 50 mm. The reference mark R may be arranged so as to be imaged by the imaging unit 20 when the imaging unit 20 images the articles A1, A2, B, etc. placed on the board 100.

The electronic device 1 according to the second embodiment has two sides forming right angles of a rectangle circumscribing the article recognized by the recognition unit 10, depending on the actual length of a predetermined portion of the reference mark R, which is a known fixed value. The length of may be corrected. Regarding other points, the electronic device 1 according to the second embodiment may operate in the same manner as the electronic device 1 according to the first embodiment.

In the second embodiment, the reference mark R can take various forms. For example, the reference mark R shown in FIG. 5 is intended to be used as a reference for the length in the lateral direction (direction parallel to the X-axis). However, in one embodiment, for example, the reference mark R is made into a ribbon-shaped area, so that it can be used as a reference for the length in the horizontal direction (direction parallel to the X axis) and the vertical direction (direction parallel to the Y axis). Good too. The reference mark R may have a form that is difficult to be determined as noise when recognized by the recognition unit 10 and identified by the identification unit 12.

Furthermore, in the second embodiment, the reference mark R may be a mark attached on the board 100 on which the article is placed, as shown in FIG. 5, for example. Further, in the second embodiment, the reference mark R may be a mark attached to an article (for example, the board 100) whose size is different from the article to be identified, as shown in FIG. 5, for example. Further, in the second embodiment, the reference mark R may be a mark attached to a position on the board 100 where the article is placed and imaged together with the article by the imaging unit 20.

FIG. 6 is a flowchart illustrating the operation of the electronic device 1 according to the second embodiment. Hereinafter, with reference to FIG. 6, the operation (identification operation) of the electronic device 1 according to the second embodiment will be described. However, in FIG. 6, points that differ from the operation of the electronic device 1 according to the first embodiment shown in FIG. 4 will be described.

At the time the operation shown in FIG. 6 starts, the distance d shown in FIG. 1 may be unknown or variable, as described above. On the other hand, at the time when the operation shown in FIG. 6 starts, and at the time when the operation shown in FIG. It is assumed that the ratio for correcting the length is known (calculated).

In step S15 of the operation shown in FIG. 4, the identification unit 12 corrected the lengths (x, y) of the two extracted sides according to the distance d shown in FIG. On the other hand, in step S25 of the operation shown in FIG. Correct the length (x, y). In the operation shown in FIG. 6, the reference mark R may be imaged by the imaging section 20 before the identification section 12 executes the operation of step S25. In other respects, the operation shown in FIG. 6 can be performed based on the same idea as the operation shown in FIG. 4.

In this way, the identification unit 12 of the electronic device 1 according to the second embodiment uses the recognition unit 10 according to the actual length of the predetermined portion of the reference mark R imaged by the imaging unit 20 that captures an image of the article. The lengths of two perpendicular sides of a rectangle circumscribed to the article recognized by can be corrected. Here, the reference mark R may be a mark attached to an article different from the article identified along with its size. Further, the reference mark R may be a mark attached to a position on a surface on which an article is placed (for example, the board surface 100) and is imaged together with the article by the imaging unit 20.

As described above, according to the electronic device 1 according to the second embodiment, it is possible to identify articles of the same or similar form but different sizes with good accuracy. Therefore, with the electronic device 1 according to the second embodiment as well, it is possible to distinguish between items that have a common design but differ in size. Furthermore, according to the electronic device 1 according to the second embodiment, even if the distance d shown in FIG. 1 is unknown or variable, the article can be identified along with the size of the article.

(Modified example)
In the embodiment shown in FIG. 1, the recognition unit 10 and the identification unit 12 have been described as separate functional units. However, as shown in FIG. 7, the electronic device 1 may include a processing section 11 having the functions of the recognition section 10 and the identification section 12 instead of the recognition section 10 and the identification section 12. Like the recognition unit 10 and/or the identification unit 12, the processing unit 11 may include at least one processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The processing unit 11 may be realized by a single processor or may be realized by several processors. In this way, the electronic device 1 may identify the article along with its size based on the lengths of the two right-angled sides of the rectangle circumscribing the article, which are recognized from the image of the article. good.

In the above-described embodiment, the two sides of the rectangle circumscribing the article are at right angles. However, the two sides of the rectangle circumscribing the article do not have to be at right angles. For example, in the case of an article having a rhombus or parallelogram shape, such as article C shown in FIG. 8, the two sides of the rectangle circumscribing the article may be set at angles that match the outer shape of the article. For example, in the case of article C shown in FIG. 8, the lengths of two sides of a rectangle circumscribing the article may be processed as (xm, ym), for example. In this case, the storage unit 14 may store size information xm and size information ym of the article, for example, for each article (article).

Furthermore, the rectangle circumscribing the article does not have to be a parallelogram. For example, in the case of an article having a shape like article D shown in FIG. ”), etc. In this case, the storage unit 14 may store size information xm', xm'', ym', ym'' of the article, for example, for each article (article).

In the above-described embodiments and modifications, the article is identified based on the length of at least two sides of the rectangle circumscribing the article, along with the size of the article. However, in one embodiment, the article may be identified based on a rectangle circumscribing the article, as well as the size of the article. That is, in one embodiment, the article may be identified along with its size based on criteria other than at least two sides of a rectangle circumscribing the article or sides constituting the rectangle circumscribing the article. In this way, the electronic device 1 may identify the article along with the size of the article based on the rectangle circumscribing the article recognized from the image of the article.

For example, in one embodiment, the article may be identified along with its size based on one of two sides forming one corner of a rectangle circumscribing the article. In this way, articles can be identified along with their size simply based on the length of one side. In this case, it may be determined in advance which one of the two sides forming one corner of a rectangle circumscribing the article is used to identify the article and the size of the article. For example, one side of two sides forming one corner of a rectangle circumscribing the article may be one side of the rectangle in a specific direction (or close to the specific direction). Further, for example, one side of two sides forming one corner of a rectangle circumscribing the article may be one of the two sides forming the rectangle and having the maximum or minimum length. In this way, the electronic device 1 may identify the article along with the size of the article based on one of the two sides of the rectangle circumscribing the article. Further, when identifying the size of the article based on the circumscribed rectangle, the electronic device 1 may identify the size based on at least one side of the circumscribed rectangle.

Furthermore, for example, in one embodiment, the article may be identified along with its size based on the length of a diagonal line of a rectangle circumscribing the article. In this case, the article and the size of the article may be identified based on one side of the two diagonals of the rectangle circumscribing the article, or the article and the size of the article may be identified based on both of the two diagonals. Good too. For example, one side of two diagonals of a rectangle circumscribing the article may be one side of the diagonal in a specific direction (or close to the specific direction). Further, for example, one side of the two diagonals of the rectangle circumscribing the article may be the longer one or the shorter one of the two diagonals. In this way, the electronic device 1 may identify the article along with the size of the article based on the diagonal of the rectangle circumscribing the article. Moreover, when identifying the size of an article based on the circumscribed rectangle, the electronic device 1 may identify the size based on the diagonal of the circumscribed rectangle.

Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various changes or modifications based on the present disclosure. It should therefore be noted that these variations or modifications are included within the scope of this disclosure. For example, the functions included in each component or each step can be rearranged to avoid logical contradictions, and multiple components or steps can be combined or divided into one. It is. Although the embodiments according to the present disclosure have been described with a focus on the apparatus, the embodiments according to the present disclosure can also be realized as a method including steps executed by each component of the apparatus. Embodiments according to the present disclosure can also be realized as a method, a program, or a storage medium on which a program is recorded, which is executed by a processor included in an apparatus. It is to be understood that these are also encompassed within the scope of the present disclosure.

Furthermore, the embodiments described above may be implemented, for example, as a method of controlling the electronic device described above. Furthermore, the embodiments described above may be implemented, for example, as a system including the electronic device described above. For example, the embodiment described above may be implemented as a system including the electronic device 1 and the imaging unit 20. Furthermore, the embodiments described above may be implemented, for example, as a program executed by the electronic device or system described above. Furthermore, the embodiments described above may be implemented, for example, as a recording medium recording a program executed by the electronic device or system described above, that is, a computer-readable recording medium.

Furthermore, as a modification of the embodiment described above, for example, the electronic device 1 according to one embodiment may be configured to have only the identification section 12 or only the processing section 11 described above. In this case, the electronic device 1 according to the embodiment can identify the article based on the lengths of two right-angled sides of a rectangle circumscribing the article, which is recognized from the image of the article. May be identified.

1 Electronic device 10 Recognition unit 12 Identification unit 14 Storage unit 16 Communication unit 18 Output unit 100 Panel surface

Claims

a recognition unit that recognizes the article from an image of the article;
an identification section that identifies the article along with its size based on the lengths of two right-angled sides of a rectangle circumscribing the article recognized by the recognition section;
Electronic equipment equipped with
The electronic device according to claim 1, further comprising a storage unit that stores lengths of two right-angled sides of a rectangle circumscribing each of the plurality of articles in association with each of the plurality of articles.
The identification unit identifies the item as the item based on the lengths of two right-angled sides of a rectangle circumscribing the item recognized by the recognition unit by referring to information stored in the storage unit. The electronic device according to claim 2, wherein the electronic device is identified along with the size of the electronic device.
The identification unit determines a right angle of a rectangle circumscribing the item recognized by the recognition unit, depending on a distance from a position where the item is placed to a position where an imaging unit that captures an image of the item is placed. The electronic device according to any one of claims 1 to 3, wherein the length of two sides formed is corrected.
The identification section is configured to form a right angle of a rectangle circumscribing the article recognized by the recognition section, depending on the actual length of a predetermined portion of the reference mark imaged by an imaging section that captures an image of the article. The electronic device according to any one of claims 1 to 3, wherein the length of the side is corrected.
The electronic device according to claim 5, wherein the reference mark is a mark attached to an article different from the article.
The electronic device according to claim 5, wherein the reference mark is a mark attached to a position on a surface where the article is placed and is imaged together with the article by the imaging unit.
The electronic device according to claim 6, wherein the reference mark is a mark attached to a position on a surface where the article is placed and is imaged together with the article by the imaging unit.
An electronic device that identifies the article along with its size based on the lengths of two right-angled sides of a rectangle circumscribing the article recognized from an image of the article.
An electronic device that identifies the article along with its size based on a rectangle circumscribing the article recognized from an image of the article.
The electronic device according to claim 10, wherein the article is identified along with its size based on one of the two sides of the rectangle.
The electronic device according to claim 10, wherein the article is identified along with the size of the article based on diagonals of the rectangle.
a recognition step of recognizing the article from an image of the article;
an identification step of identifying the article along with its size based on the lengths of two right-angled sides of a rectangle circumscribing the article recognized in the recognition step;
Control methods for electronic devices, including:
14. The method for controlling an electronic device according to claim 13, further comprising a storing step of storing the lengths of two right-angled sides of a rectangle circumscribing each of the plurality of articles in association with each of the plurality of articles.
The identifying step refers to the information stored in the storing step to identify the article based on the lengths of two right-angled sides of a rectangle circumscribing the article recognized in the recognizing step. 15. The method for controlling an electronic device according to claim 14, wherein the method identifies the electronic device along with the size of the device.