KR102046507B1 - Mehtod of controlling an electric product - Google Patents

Abstract

The present invention relates to a control method of an electric product. According to an aspect, a control method of an electrical product includes: acquiring an image of a recognition object in a recognition device; Recognizing article information in the acquired image and comparing the recognized article information with information stored in the memory unit; And displaying the comparison result on the display unit, and when the recognized article information does not match the information stored in the memory unit, the recognized article information itself is displayed on the display unit.

Description

Method of controlling an electric product

The present invention relates to a control method of an electric product.

In general, an electrical product can be understood as a device for managing or treating a specific object using electricity as a power source.

When the conventional electrical appliance is a refrigerator, the refrigerator includes a main body that forms a storage compartment and a door that selectively shields the main body. The door is provided with a display device for displaying the operating state of the refrigerator.

The display device includes a display unit for displaying the set temperature of the storage compartment and an input unit for adjusting the set temperature. However, in the display apparatus, article information stored in the storage compartment, management (processing) information of the article, and the like are not displayed. That is, the conventional refrigerator is used only as a means for storing food, and does not recognize management or processing information about the food.

In summary, in the case of a conventional electric product, since the electric product does not recognize management information or processing information of a specific object to be managed or processed, there is an inconvenience that a user must remember or search it.

In detail, if the electrical appliance is a refrigerator, the user must remember the management information (such as storage quantity or expiration date information) of the food stored in the refrigerator, and then purchase a plan for the consumption of foods with a short shelf life or for foods with a low storage amount. There was an inconvenience that can stand.

In the case where the electric product is a washing machine, the user had to check the material of the laundry or the washing method before operating the laundry, and there was an inconvenience of operating the washing machine.

When the electrical appliance is a cooking appliance, there is an inconvenience in that the user must check the cooking method of the food before the cooking of the food and operate the cooking appliance.

An object of the present invention is to provide a control method of an electric product that enables the electric product to recognize information about a processing object of the electric product.

According to an aspect, a control method of an electrical product includes: acquiring an image of a recognition object in a recognition device; Recognizing article information in the acquired image and comparing the recognized article information with information stored in the memory unit; And displaying the comparison result on the display unit, and when the recognized article information does not match the information stored in the memory unit, the recognized article information itself is displayed on the display unit.

According to the present embodiment, since the recognition target region is extracted without recognizing the information of the entire acquired image, and the information included in the extracted recognition target region is recognized, the recognition time is reduced.

In addition, since the article information is stored in the memory unit, when the recognized information coincides with the information stored in the memory unit, the user does not have to modify or directly input the article information, thereby reducing the inconvenience of the user.

In addition, since the article information not stored in the memory unit may be stored in the memory unit in the form of a category, the information stored in the memory unit may be utilized as comparison information with next recognition information.

1 is a diagram showing the configuration of a network system according to an embodiment of the present invention.
2 is a block diagram of a refrigerator constituting the network system.
3 is a flowchart illustrating a method of operating a recognition device according to an exemplary embodiment.
4 to 5 are views showing a screen displayed on the display unit of the refrigerator.
FIG. 6 is a diagram for explaining a state in which a recognition target region is extracted from an acquired image; FIG.
7 is a view showing a screen displayed on the display unit of the refrigerator.
8 is a view showing a screen displayed on the display unit according to another embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing the configuration of a network system according to an embodiment of the present invention, Figure 2 is a block diagram of a refrigerator constituting the network system.

1 and 2, a network system according to an embodiment of the present invention includes a refrigerator 10, in which cold air is generated and stores an object (article) as an example of an electric product, and in the refrigerator 10. The terminal 80 may be communicatively connected, and the server 70 may be communicatively connected to the refrigerator 10 and the terminal 80 and may store predetermined data.

The terminal 80 includes a display unit 82 on which predetermined information is displayed. The display unit 82 may display item information stored in the refrigerator or operation state information of the refrigerator. That is, the terminal 80 may monitor the state of the refrigerator 10. In addition, the operation of the refrigerator 10 may be controlled using the terminal 80. In addition, the information recognized by the terminal 80 may be transmitted to the refrigerator 10.

The terminal 80 may be, for example, a portable mobile phone or a smart phone, but is not limited thereto.

The network system includes a first interface 310 for communication between the terminal 80 and a server 70, a second interface 320 for communication between the server 70 and a refrigerator 10, and the terminal. And a third interface 330 for communication between the 80 and the refrigerator 10.

The first to third interfaces 310, 320, and 330 may communicate with each other by at least one of Wi-Fi, Zig-bee, Bluetooth, and the Internet. The method may be applied, and there is no limit to the communication method.

The refrigerator 10 includes a display unit 20 capable of displaying information, a recognition device 30 for recognizing information, a memory unit 40 storing information related to an article, and the server 70. Or a communication unit for communication with the terminal 80.

The recognition device 30 may include at least one of a photographing apparatus, an RFID reader, a QR code reader, and a barcode reader. Hereinafter, the recognition device 30 will be described as an example of a photographing device.

The recognition device 30 may acquire an image of the recognition object 400. The recognition object 400 may be, for example, a purchase receipt of an article. The article may be classified into an article to be managed and an article to be managed. The article to be managed is an article that can be managed in the refrigerator and may be, for example, food.

The information related to the article may include article itself information or article management information. The article itself information includes the name of the article, the quantity of the article or the number of articles, etc. The article management information may include the storage location, storage period, storage amount or storage method of the article stored in the refrigerator.

In addition, some of the information related to the article may be obtained from an image obtained by the recognition device 30.

The memory unit 40 may include a first memory 41 in which management item information is stored, a second memory 42 in which item information selected by a user is stored, and a third item in which information related to an item currently being managed is stored. Memory 43 may be included. The article included in the managed article information may be the above-described managed article.

The management article information may be stored in the first memory 41 at the time of manufacture of the refrigerator 10. The managed article information stored in the first memory 41 may be updated. For example, the update information may be received from the terminal 80 or the server 70.

The second memory 42 stores article information selected by the user as a management target as managed article information not stored in the first memory 41. Accordingly, the first memory 41 may be referred to as a basic memory, and the second memory 42 may be referred to as a user selected memory.

Some or all of the information stored in the memory unit 40 may be stored in the server 70 or the terminal 80 or loaded from the server 70 or the terminal 80.

3 is a flowchart illustrating a method of operating a recognition device according to an embodiment of the present disclosure, FIGS. 4 to 5 are views illustrating a screen displayed on a display unit of the refrigerator, and FIG. 6 is a view recognized within an acquired image. FIG. 7 is a diagram illustrating a state in which a target region is extracted, and FIG. 7 is a diagram illustrating a screen displayed on the display unit of the refrigerator.

3 to 7, the recognition device 30 prepares the operation of the recognition device 30 in order to recognize the recognition object (S11). That is, the recognition device 30 is activated. When the recognition device 30 is activated, the guide frame 210 for guiding the position of the recognition target 400 on the display unit 20 is displayed on the display unit 20 as shown in FIG. 4. Can be. In addition, the display unit 20 may display guide information 220 for positioning the position of the recognition target on the guide frame 210.

In addition, the display unit 20 may display a photographing button 222 for acquiring an image of the recognition target 400. Therefore, when the photographing button 222 is selected, an image of the recognition object 400 located in the guide frame 210 is obtained (S12).

As another example, when a predetermined time elapses in a state in which the photographing button 222 is positioned on the recognition target 400 in the guide frame 210, the recognition target is automatically located in the guide frame 210. An image of 400 may be obtained.

After the image of the recognition target 400 is obtained, the recognition target region is extracted from the acquired image (S13). In addition, the controller 50 recognizes information included in the extracted management recognition target area (S14).

At this time, in the process of recognizing the information in the recognition target region, as shown in FIG. 5, information 230 for notifying that recognition in the recognition target region is in progress may be displayed.

Referring to FIG. 6, in the present exemplary embodiment, the recognition target area 240 refers to a necessary area for recognizing an article in the entire acquired image.

The extraction method of the recognition target region 240 will be described.

When the recognition device 30 acquires an image of the recognition object 400, the controller 50 divides the obtained image into a plurality of pixels. Each pixel has its own coordinate value. Then, the controller 50 determines the gray value of each pixel and gives a unique value to each pixel. For example, if the gray value of the pixel exceeds the reference value, 1 is assigned, and if the gray value is less than or equal to the reference value, 0 is assigned.

Next, the controller 50 extracts the recognition target region based on the unique value of each pixel. For example, as shown in FIG. 6, the acquired image includes the order number 001 and 002 of the purchased item, the item name 243, the item code 244, the price, the quantity, and the like. At this time, the controller 50 extracts the recognition target area 240 based on one or more of the order of the article, the article name 243, and the article code 244.

When the determination criterion is the order of the article, when the shape of the pixels given with 1 within a specific coordinate range is 0 or 1, a predetermined range is extracted as the recognition target region based on this.

In the present embodiment, extracting the recognition target region 240 does not extract the information of the article from the image, but merely extracts the region necessary for the recognition on the image. The reason for extracting the area required for recognition is to reduce the recognition time required for recognizing the item information on the image. In other words, when unnecessary information is not recognized among the information on the image, the recognition time may be reduced.

The extracted recognition target region 240 may include one or more of an article name 243 and an article code 244. However, the information included in the recognition target area 240 is not limited thereto and may include a quantity, a price, a purchase date, and the like, as necessary.

The controller 50 determines whether the recognized information matches the information stored in the first memory 41 after recognizing the article name and the article code in the recognition target area 240 (S15).

The managed article information stored in the first memory 41 may include a product name (or article category), an article code corresponding to the article name (or article category), and an article ID corresponding to the article name (or article category). .

At this time, the controller 50 compares the recognized article code with the article code stored in the first memory 41 first. If not, the controller 50 stores the recognized article name and the stored article code in the first memory 41. It can be compared to match the article name. Of course, the reverse is also possible.

As a result of the determination in step S15, when the recognized information and the information stored in the first memory 41 match, category information may be displayed on the display unit 20 as shown in FIG. 7 (S16). The category information may include an item category 273 and item ID information 272 corresponding thereto. In addition, the category information may further include icon information (not shown) corresponding to the article category 273.

Of course, any one of the article category 273 and the article ID (273) is not displayed. The article ID 273 classifies the article category 273 by numbers or symbols.

In the present embodiment, a plurality of article names may belong to the article category. That is, when the article name is A milk, B milk, or the like, the article category 273 is milk. At this time, the article ID 272 is information not recognized by the recognition device 30. Therefore, in the present exemplary embodiment, information that is not recognized by the recognition device 30 may be displayed on the display unit 20.

In this case, the recognized article name and category information (eg, article category) displayed on the display unit may be identical or different.

Information recognized in this embodiment (article name or article code) can be corrected. For example, when the information actually included in the recognition target area is "milk" or when the recognition device 30 recognizes "milk", the recognized "milk" may be corrected to "milk". In addition, it may be determined whether the corrected article name matches the article name stored in the first memory 41.

In operation S15, it may be determined whether one or more keywords included in the recognized information coincide with information stored in the first memory 41. For example, when the recognized product name is Korean milk, the controller 50 may determine whether the keyword "milk" matches the information stored in the first memory 41.

On the other hand, if it is determined in step S15 that the recognized information does not match the information stored in the first memory 41, the control unit 50 is the same as the information stored in the second memory 42 It is determined whether they match (S17).

As a result of the determination in step S17, when the recognized information (or the corrected information) matches the information stored in the second memory 42, category information may be displayed on the display unit 20 (S18). As described above, the category information may include an item category 273 and item ID information 272 corresponding thereto.

As a result of the determination in step S17, when the recognized information does not match the information stored in the second memory 42, the display unit 20 displays the recognized information itself (S19). For example, as illustrated in FIG. 7, the recognized product name is "ABC milk" or "DEF red", and the recognized product name does not match the product name stored in the first memory 41 and the second memory 42. Recognized item name "ACB milk" or "DEF red" is displayed. That is, the display unit 20 may display category information and recognized product names together.

The article name displayed on the display unit 20 may be changed or modified by the user (S20). When the article name is modified, the controller 50 determines whether the modified article name and the article name stored in the first memory 41 and the second memory 42 match (S21).

As a result of the determination in step S21, when the modified article name matches the article name stored in the first memory 41 and the second memory 42, the display unit 20 displays category information corresponding to the modified article name. (S22). That is, the modified article name on the display unit 20 may be changed to the article category.

On the other hand, if the modified article name does not match the article name stored in the first memory 41 and the second memory 42 as a result of the determination in step S20, the user may select a storage button (not shown). Then, the recognized article code together with the modified article name may be stored in the second memory 42. At this time, an article ID corresponding to the modified article name is automatically generated, and the generated article ID is stored in the second memory 42 together with the article code and the modified article name (S23). The information additionally stored in the second memory 42 may be used as comparison information with the information recognized in step S14.

Of course, in step S19, the user may not modify the article name while the recognized information is displayed. In this state, the user can select a save button, not shown. Then, the recognized article name, the recognized article code, and the generated article ID may be stored in the second memory 42.

Referring to FIG. 7, when a plurality of item names are included in the recognition object 400, the plurality of category information may be displayed in the vertical direction or left and right directions in the display unit 20. In addition, the display unit 20 may further display a selection unit 271 for selecting displayed category information.

The selector 271 is used to select an item to be managed or an item to be managed. In contrast, the selection unit 271 is not provided, and the item to be managed or the item to be managed can be selected by touching the category information 272 and 273.

The user may determine the item to be managed and the item to be managed by checking the category information displayed on the display unit 20 or the recognized item name itself. Then, the user can select the selection unit 271 corresponding to the object to be managed. That is, the user may select a management target article to be managed in the refrigerator. At this time, the unselected article becomes an actual uncontrolled article.

In the present exemplary embodiment, the user selects the item to be managed by using the selector 271. Alternatively, the item to be managed may be selected by using the selector 271. In other words, it is determined that all articles displayed on the display unit 20 are management target articles. In addition, the user may select an unmanaged object from among the displayed items by using the selection unit 271. For example, a form such as "V" is displayed on each selector 271, and when the user selects the selector 271, a form such as "V" may be removed.

That is, the article to be finally managed is determined by the user's selection of the management target article or the non-management target article. However, the food may be a controlled item, but may be classified as an unmanaged item by a user's selection.

In addition, the display unit 20 may display a completion button 275 for inputting a command for starting the management of the displayed goods and a cancel button 276 for canceling the management of the displayed goods.

When the user selects the completion button 275, management of the object to be managed is performed, and category information selected as the item to be managed by the display unit 20 is stored in the third memory 43 (S24). . On the other hand, when the user selects the cancel button 276, the screen of FIG. 7 may return to FIG. 4.

As another example, when the recognized information itself is displayed on the display unit 20 and the separate button 275 is selected without selecting a separate storage button, the recognized article name, the recognized article code, and the generated article are selected. The ID may be stored in the second memory 42.

In addition, the display unit 20 may further display information on the total number of articles (categories) to be added for management.

Meanwhile, the information stored in the first memory 41 may be updated. For example, managed article information may be added to the first memory 41. Information stored in the second memory 42 when the management item information (eg, product name or article code) added in the updating of information and the information (item name or article code) stored in the second memory 41 match. May be deleted, and the corresponding information may be stored in the first memory 41.

On the other hand, when the controller 50 attempts to extract the recognition target region from the acquired image, but there is no recognition target region to be extracted, the display unit 20 may display information for requesting the image reacquisition. .

According to the present embodiment, the recognition region is extracted without recognizing the information of the entire image acquired by the controller, and when the recognition target region is determined, information included in the determined recognition target region is recognized, so that the recognition time of the controller is recognized. This has the advantage of shrinking.

In addition, since the product information is displayed on the display unit by recognizing the object to be recognized, there is an advantage in that the user does not need to directly input the product information.

8 is a diagram illustrating a screen displayed on the display unit according to another exemplary embodiment.

Referring to FIG. 8, in the previous embodiment, after recognition of information in the recognition target area is completed, category information 272 and 274 may be displayed on the display unit 20. In this case, the category information 272 and 274 may include the item ID information 272 and the item category 274. In addition, the article category 274 may include number information. As another example, the number information of the category may be displayed separately from the article category 274.

Although the present invention has been described with respect to the refrigerator as an example of the electric product, it is noted that the contents for recognizing the product information can be equally applied to other electric products other than the refrigerator.

In addition, it will be appreciated that the method of operating the recognition device described in the present invention may be equally applicable to a terminal, and the terminal may also be included in an electric product in the present invention. In this case, information of the article finally selected as the object to be managed by the terminal may be transmitted to the refrigerator. Alternatively, the image acquired by the terminal may be transmitted to the refrigerator to recognize article information included in the image.

In addition, the buttons displayed on the display unit in the present invention may be replaced by an input unit provided separately from the display unit. That is, the input unit may input a cancel command generated when the completion command or a cancel button occurs when the completion button is selected.

20: display unit 30: recognition device
40: memory

Claims (11)

Acquiring an image of a recognition object in a recognition device;
Recognizing article information in the acquired image and comparing the recognized article information with information stored in the memory unit; And
Displaying the comparison result on the display unit;
The memory includes a first memory in which the management item information is stored, and a second memory in which the item information selected by the user is stored.
Comparing the recognized item information with the information stored in the memory unit,
Determining whether the recognized article information matches the information stored in the first memory;
If the recognized article information does not match the information stored in the first memory, determining whether the recognized article information matches the information stored in the second memory;
If the recognized article information does not match the information stored in the second memory, displaying the recognized article information itself on the display unit;
Determining whether the item information displayed on the display unit is modified;
When the displayed article information is modified, determining whether the modified article information matches information stored in the first memory and the second memory; And
And storing the category information corresponding to the modified article information in the second memory when the modified article information does not match the information stored in the first memory and the second memory. The method of claim 1,
The recognized article information includes a product name or an article code. delete delete The method of claim 1,
And the recognized article information can be stored in the second memory. The method of claim 5,
And controlling the electronic device to store newly generated information in addition to the recognized item information. The method of claim 1,
Information stored in the first memory may be updated,
If the information to be added to the first memory during the update process matches the information stored in the second memory, the information stored in the second memory is deleted, and the control of the electrical appliance in which the deleted information is stored in the first memory. Way. The method of claim 1,
If the modified article information matches the information stored in the memory unit,
And category information corresponding to the modified article information is displayed on the display unit. The method of claim 8,
And the category information includes one or more of an article category and article ID information to which the article name belongs. The method of claim 1,
Comparing the recognized item information with the information stored in the memory unit,
Correcting the recognized item information;
And comparing the corrected information with the information stored in the memory unit. The method of claim 1,
And when some keywords among the item names included in the recognized item information coincide with the item names stored in the memory unit, determining that the recognized item information matches the information stored in the memory unit.

Images